Project Munin 2

Automated Information Search and Analysis System from the Dark Age of Technology

Project Structure

.
├── .chromium-profile/    # Isolated browser profile (persists login + Cloudflare session)
├── config.json           # Keywords, location, radius, base URL, profile path
├── config.example.json   # Example configuration (safe to commit)
├── cvs/                  # CV text files (*.txt)
├── results/              # Scraped job listings per keyword (gitignored)
├── match-results/        # CV-to-job matching results (gitignored)
├── watched-jobs.json     # Tracked job URLs (gitignored)
├── src/
│   ├── main.ts           # Entry point: orchestrates scraping
│   └── scraper.ts        # Job board scraper with pagination and Cloudflare handling
├── Makefile              # Common commands
├── package.json
└── tsconfig.json

First-time Setup

Before running the scraper, you need to set up an isolated Chromium profile:

make setup-profile

This opens a clean Chromium window with a dedicated profile (.chromium-profile/). In that window:

1. Navigate to your target job board URL (configured in config.json)
2. Sign in if needed (optional, but recommended for better results)
3. Solve any Cloudflare or CAPTCHA challenges that appear
4. Once the page loads normally, close the browser

All subsequent scraper runs will reuse this profile — your login session and Cloudflare clearance are preserved. You only need to do this once.

Configuration

Copy config.example.json to config.json and edit:

• baseUrl — base URL of the job board (e.g. "https://example-job-board.com")
• keywords — job search terms (e.g. ["Marketing Specialist", "Product Designer", "Business Analyst"])
• location — location filter (e.g. "London, UK")
• radius — search radius in miles
• maxResultsPerKeyword — max jobs to scrape per keyword
• minScore — minimum score threshold for matching (default: 7.5)
• topMatchesPerCv — top N matches to display per CV
• profilePath — path to the isolated browser profile (default: ./.chromium-profile)

CV Format

Place .txt files in cvs/. Each file is a plain text resume. The filename (without .txt) becomes the CV name (underscores replaced with spaces).

Commands

Command	Description
make setup-profile	Open Chromium with isolated profile for manual login + Cloudflare solve
make scrape	Scrape job listings for all keywords (paginates up to 10 pages per keyword)
make match	Run AI-powered CV-to-job matching on existing results
make full-run	Full pipeline: scrape + match (invoked via opencode)
make clean	Remove scraped results, watched-jobs, and match results
make clean-all	Clean everything including the browser profile
make build	Compile TypeScript

Scraping

make scrape          # or: npm start

The scraper opens a non-headless Chromium window using the dedicated profile (.chromium-profile/). It automatically paginates through up to 10 search result pages per keyword, clicking the "Next" button and handling any Cloudflare or CAPTCHA challenges along the way (waits for manual solve).

Already-scraped job URLs (tracked in watched-jobs.json) are skipped automatically across pages, so the scraper keeps going until it finds enough new jobs or runs out of pages.

Job results are saved to results/<keyword>/job-N-<keyword>.json.

Matching

make match

Run through opencode's AI: reads all CVs and scraped job descriptions, then performs full-text semantic analysis using language understanding to score each CV×job pair on a scale of 1–10. Only matches with a score of 7.5 or higher are kept.

Output: match-results/ directory with per-CV JSON files containing detailed justifications for each match.

Dependencies

• puppeteer-core + puppeteer-extra + puppeteer-extra-plugin-stealth — stealthy browser automation
• typescript — compiled from src/ to dist/

Requires Chromium installed at /usr/bin/chromium.