PUPILA: An A-Eye on every job board

Table of contents

Getting startedWhat · Key features · Onboarding · Forking & personalizing · How it works

Using itApplication tracking · AI Apply · AI per-job review · MCP server

Under the hoodStack · Architecture · Sources · Pipeline stages · Tuning filters

ReferenceRepo layout · Run locally · GitHub Actions · Customization · Security & quality gates · Known upstream issues · Conventional commits · License

What

PUPILA is a local-first job aggregator. Choose for schedule a task or trigger a manual refetch and it pulls from many public sources (adding more is an easy job), filters out the noise (junior roles, non-engineering, onsite-only, etc.), and scores each remaining posting against your CV-derived profile. You wake up to a single sorted table of roles that actually fit, every row links to the original posting, so you can apply directly. That's the core value: getting the data, filtering it, and scoring it. The AI layers below are optional.

Optional layers, powered by your local LLM CLI (claude / codex / gemini / opencode — no API keys, no cloud, no third party ever sees your data):

  • AI per-job review — a strong-match / match / weak-match / skip verdict with a one-line reason next to each title.
  • Jinder — a Tinder-style swipe deck for triaging the top matches in seconds: right-swipe to queue a role for application, left-swipe to skip it forever.
  • AI Apply — drafts a tailored cover letter + highlights + Q&A package from your CV and the posting for every job in the queue, while you do something else.
  • MCP server — exposes everything the UI can do (17 tools: query / detail / mark applied / enqueue / queue status / aggregator runs / profile regen) to any MCP client so you can drive Pupila from inside Claude Code, Claude Desktop, or Cursor. One-command install. Jump to section ↓

Key features

The stuff that makes the daily routine actually pleasant:

  • Day-over-day diff. Every run computes what's new and what disappeared since yesterday — surfaced as ✨ New since last run + 🗑 Removed since last run at the top of JOBS.md. The actionable bit is always the first thing you see.
  • Scheduled, no babysitting. scripts/install-launchd.sh (macOS) and scripts/install-cron.sh (Linux) install two agents — aggregator + AI review — that run on independent times. launchd catches up missed runs after wake.
  • Group-by-company (default on). Folds a 24-role Vercel into one expandable row instead of dominating the top of your table.
  • URL-encoded view state. Every filter, sort, and expanded row syncs to ?q=...&cat=...&src=... — bookmark a filtered view and it rehydrates next time.
  • Source-health banner. A 🚨 banner appears in JOBS.md when a fetcher returns zero items or errors, so silent upstream breakage isn't silent.
  • Application tracking. Click a status pill on any row (📝 applied / 💬 interview / 🎯 offer / ❌ rejected / ⏸ withdrawn) — saves to config/applied.json and feeds the "📋 Application status" section at the top of JOBS.md.
  • 9-panel Settings dashboard: switch LLM CLI / install or remove the scheduler / regenerate scoring profile / manage job sources (incl. LLM-powered company discovery) / inspect the last run / check disk usage / clean / view environment / monitor the apply queue, all from pnpm run ui → Settings.
  • Talk to your data from an AI client. Optional MCP server exposes every actionable UI surface (filter jobs, mark applied, enqueue AI Apply, trigger a refresh, regenerate the scoring profile) to Claude Code / Claude Desktop / Cursor via 17 typed tools. One command (bash scripts/install-mcp.sh) wires it in.
  • Reach past the curated boards. The bluedoor source taps a free aggregator over ~1.6M postings across 31 ATS providers (Workday, iCIMS, Oracle, ADP, …) you can't query directly — pulled by your accepted regions, with companies you already follow auto-skipped so it only adds long-tail listings.
  • RSS feed. data/feed.xml gets every "✨ new" job — point any RSS reader at the local file:// path.
  • Local-first by design. No API keys, no cloud, no hosted scheduler. The LLM features use your existing claude / codex / gemini / opencode subscription via the CLI. Your CV, brief, and applied list never leave the machine.

Onboarding

The first time you run pnpm run ui, a three-step wizard sets you up. Total time: about 30 seconds.

1. Pick your LLM CLI

PUPILA shells out to whichever AI CLI you already have authenticated locally — claude, codex, gemini, or opencode. The wizard auto-detects what's on your PATH and pre-selects the first available one. No API keys, no signup, no per-token billing — it uses your existing CLI subscription.

2. Drop your CV

Drag a .pdf / .docx / .md / .txt CV onto the drop zone (or click Choose file). The CV is parsed locally and sent to your LLM CLI to generate a short candidate brief — who you are, what stack you work in, what kind of role you want, and what to avoid. The original CV stays on disk at config/cv.<ext> (gitignored) so AI Apply can re-attach it later.

No recent CV? Click Import from LinkedIn instead (optional) and upload a LinkedIn profile PDF — on your LinkedIn profile, go to More → Save to PDF, then drop the downloaded file here. It runs through the same parse → LLM pipeline with a LinkedIn-tuned prompt, so the resulting brief is comparable to a CV-sourced one. (No login or scraping — the "Save to PDF" export is fully self-serve.)

3. Confirm the generated brief

Eyeball the brief and edit anything that needs tweaking — this natural-language description drives every scoring decision and AI verdict downstream. Hit Looks good and the wizard stamps config/preferences.json with onboardedAt, never re-triggers, and kicks off the first aggregator run automatically. You land on the Jobs view already populated.

How it works

A config-driven, local-first daily job aggregator. Pulls listings from ~20 public sources (job boards, RSS feeds, Hacker News, five ATSes — Greenhouse, Ashby, Lever, Recruitee, Personio — no-key remote feeds like Jobicy and Himalayas, a custom Aave scraper, an Ashby-private GraphQL fetcher, and bluedoor, a free cross-ATS aggregator (~1.6M postings) queried by your accepted regions; adding a new source is one file), normalizes them into a single shape, scores each one against your profile (defined in config/profile.json), deduplicates, and writes the result to your local checkout.

Designed to be cloned and run locally. No hosted scheduler, no public dashboard, no external services. Anyone — frontend, backend, mobile, data, infra, any seniority, any region — sets their role via:

  1. A short candidate brief generated from their CV (run pnpm run setup-brief --file ~/cv.pdf, or use the local UI's Profile tab).
  2. Tunable scoring weights and keyword lists in config/profile.json.
  3. Tier-S company slugs in config/slugs.json.

Then schedule the aggregator with scripts/install-launchd.sh (macOS) or scripts/install-cron.sh (Linux). Two agents are installed by default: one for the aggregator, one for the AI per-job review.

After onboarding, config/profile.json is auto-generated from your candidate brief by /api/profile-generate (scoring weights + keyword lists tuned to the stack you actually work in, and an exclusion list pulled from your brief's "what to avoid" section). Re-runnable from the Settings tab → Scoring profile → Regenerate.

Contributing rules and project invariants live in CONTRIBUTING.md.

Looking for today's matches?JOBS.md (auto-generated by the local aggregator). Raw data lives in data/jobs.json. Local RSS feed at data/feed.xml (point your reader at the file:// path). Prefer a UI? → pnpm run ui opens a local-only Vite dashboard at http://127.0.0.1:5173. Four tabs: Jobs (filter, search, sortable columns, click-to-expand rows over data/jobs.json), Jinder (Tinder-style swipe deck — right-swipe to queue an AI Apply, left-swipe to skip), Profile (drop a PDF/DOCX/MD CV — or import a LinkedIn profile PDF — to set up or refresh your candidate brief), and Settings (scheduler lifecycle, scoring profile regenerate, disk usage, apply queue). See AI Apply and AI per-job review below.


Forking & personalizing

The repo ships with neutral defaults. To make it yours:

1. Clone or fork

gh repo fork FranRom/pupila --clone
cd pupila
pnpm install

Or click "Fork" on GitHub, then git clone <your-fork>.

2. Generate your candidate brief (required)

The brief at config/candidate-brief.md is the natural-language description of who you are, what you want, and what to avoid. This step is mandatorypnpm run dev will refuse to start until the file exists. (Bypass with PUPILA_NO_BRIEF_CHECK=1 if you genuinely want raw aggregation with no AI review.)

The file is gitignored — it contains CV-derived personal information and should never be committed. setup-brief and the onboarding wizard write to the gitignored canonical path.

The friendliest path is the first-run onboarding wizard: run pnpm run ui on a clean repo and the UI walks you through (1) picking your LLM CLI, (2) dropping your CV, (3) confirming the auto-generated brief. After "Looks good", config/preferences.json is stamped with onboardedAt, the wizard never re-triggers, and you land on the Jobs view.

For the CLI-only path, run setup-brief directly:

pnpm run setup-brief --file ~/Documents/cv.pdf          # PDF
pnpm run setup-brief --file ~/Documents/cv.docx         # Word document
pnpm run setup-brief --file ~/Documents/cv.md           # markdown
pnpm run setup-brief --linkedin ~/Downloads/profile.pdf # LinkedIn "Save to PDF" export
cat resume.txt | pnpm run setup-brief                   # stdin

--linkedin runs the same pipeline as --file but tells the LLM the input is a LinkedIn profile export (so it ignores LinkedIn's boilerplate). Don't have a recent CV? On your LinkedIn profile, go to More → Save to PDF and pass the downloaded file. (A --file whose name contains "linkedin" is auto-detected as a LinkedIn source.)

Or open the UI and use the Profile tab:

pnpm run ui   # http://127.0.0.1:5173 → Profile tab → drop your CV (or "From LinkedIn")

The auto-detected provider order is claudecodexgeminiopencode (whichever is on PATH first). Override with PUPILA_LLM=codex pnpm run setup-brief .... No API keys; uses your existing CLI subscription.

The two personalization layers, briefly:

  • config/profile.json (committed defaults) controls what gets fetched + scored (weights, keyword lists, tier-S slugs).
  • config/candidate-brief.md (gitignored, CV-derived) controls the per-job AI verdict (pnpm run ai-review). Both matter for the full daily flow.

3. Scoring profile (auto-generated, hand-tunable)

config/profile.json drives what gets scored. After onboarding finishes, /api/profile-generate runs your local LLM CLI on the brief and fills in:

  • Weights: non-zero values for stacks/domains the brief actually mentions (e.g. stackPrimary: 10 if you work with React/Next/TypeScript daily). roleTitle / roleBody price a role match.
  • Keyword lists: stackPrimary, stackRn, stackOther, w3*, ai* — populated from your stack.
  • roles: the target job titles you're after (e.g. Senior Frontend Engineer and Product Engineer). Each role's titleMatch tags a posting, earns the roleTitle bonus, and rescues it from the title-based hard drops. Edit them as chips under Profile → Role interests, or let regeneration re-derive them from the brief.
  • titleExcludedSpecialties: pulled from the brief's "what to avoid" paragraph. A frontend brief gets (backend|data|devops|sre|...) engineers? here so those titles hard-drop (unless a declared role rescues them).

Universal hard-drop rules (junior/exec/sales/recruiter/support) and seniority weights stay at sensible defaults regardless of the brief.

Location & work preferences (Profile → Location). This decides which jobs are a geographic fit for you. You set three things:

  • Based in — the country you live in (e.g. Spain). This is the only field most people need. Setting it automatically fills in the regions you can work in (Spain → Europe, EMEA, EU, Spain).
  • Work typesRemote, Hybrid, On-site. If you don't pick On-site, jobs that are strictly on-site get dropped.
  • Accepted regions (advanced, optional) — the regions a job can be tied to and still match you. Auto-derived from your country; only open the Customize panel if your reach is different (e.g. based in Spain but also open to US-remote, or a specific timezone like CET).

How the matching works, in plain terms — a remote job is kept unless it requires a location you can't be in:

The job says… Result
Remote, Worldwide, Global, Anywhere (no specific place) Kept — where the company is doesn't matter
A region you accept (Europe, EMEA, your country) Kept
Remote - US but the description welcomes Europe Kept — the label isn't a hard requirement
Remote - US, Remote (US only), "must be based in / authorized to work in the US" Dropped — a real requirement you can't meet

This is fully neutral — no country is special. A Spain-based profile drops US-only jobs; a US-based profile drops Europe-only jobs; both run the same code. The "Only show jobs in my accepted regions" toggle controls strictness: on = drop out-of-region jobs; off = keep them but rank them lower.

Your regions also drive the bluedoor source. Besides filtering, Based in + Accepted regions are the search terms used to query the bluedoor aggregator (one search per region). Widen your accepted regions and bluedoor reaches further; a neutral/empty location simply pulls nothing from it. No geography is hardcoded — the same code works for any country.

Re-run after editing the brief by going to Settings → Scoring profile → Regenerate (or POST /api/profile-generate). To hand-tune, edit config/profile.json directly — your edits won't be overwritten unless you regenerate.

Tweak, run pnpm run dev, inspect JOBS.md, repeat.

4. Update the company slug list

Two ways to follow more (or fewer) Ashby / Greenhouse / Lever companies:

  • From the UI (recommended): pnpm run uiSettings → Job sources. Add or remove a company per ATS, Verify a slug against the live board before saving, and run Check board health to flag any boards that 404 or time out. Or click Discover more sources for my profile to let your local LLM CLI propose companies that fit your profile + candidate brief — each suggestion is live-probed against the supported ATSes and ranked by how many of its open roles match you, then you pick which to add. Changes are written to config/slugs.local.json (gitignored) as a per-ATS delta on top of the shipped list, so your personal picks stay separate and you keep getting upstream additions to config/slugs.json.
  • By hand: edit config/slugs.json directly (the committed, shared baseline).

Slugs come from the URL of the careers page (jobs.ashbyhq.com/<slug>, boards.greenhouse.io/<slug>, jobs.lever.co/<slug>). Either way, 404s are silently skipped, so trial-and-error is safe. Adding a slug also auto-excludes that company from bluedoor: the dedicated fetcher is authoritative, so you never get a duplicate from the aggregator.

5. Schedule the daily run

Two agents run on independent schedules so you can tune them separately:

  • Aggregator (pnpm run dev) — fetches sources, scores, writes data/jobs.json + JOBS.md + data/feed.xml. No LLM needed.
  • AI per-job review (pnpm run ai-review) — sends each top-scoring job through your local LLM CLI (claude/codex/gemini/opencode). Skipped via --no-review if you don't have a CLI installed.

macOS (launchd, recommended):

./scripts/install-launchd.sh                              # both, defaults: aggregate 07:00, review 07:15
./scripts/install-launchd.sh --aggregate-time 06:30
./scripts/install-launchd.sh --review-time 09:00
./scripts/install-launchd.sh --no-review                  # aggregator only (no LLM CLI)
./scripts/install-launchd.sh --uninstall                  # remove both
launchctl list | grep pupila                               # check status
launchctl start dev.${USER}.pupila.aggregate               # trigger now

launchd's StartCalendarInterval catches up missed runs after wake — if your laptop was asleep at 7am, it runs once the lid opens.

Linux (cron):

./scripts/install-cron.sh                                 # both, defaults
./scripts/install-cron.sh --no-review
./scripts/install-cron.sh --uninstall
crontab -l                                                # inspect

Manual / casual use:

pnpm run daily                                            # = pnpm run dev && pnpm run ai-review

Logs land in data/launchd-{aggregate,review}.{out,err}.log (macOS) or data/cron-{aggregate,review}.log (Linux). Nothing is pushed anywhere — your applied.json, candidate-brief.md, and data/ai-reviews.json stay on your machine. (The check.yml GitHub Action that runs build + tests + lint on PRs is unrelated to scheduling and stays in place.)

6. Personal data + privacy

These files are gitignored and never committed, so a public fork can't leak them:

File What it is First-run source
config/candidate-brief.md LLM-generated CV summary Onboarding wizard / pnpm run setup-brief
config/cv.{pdf,docx,md,txt} Original CV file Saved by the onboarding wizard / setup-brief so AI Apply can re-attach it
config/applied.json Your application history UI Profile → status pills
config/preferences.json Your chosen LLM CLI + onboarding-complete stamp Onboarding wizard
config/slugs.local.json Your personal add/remove of company slugs (overlay on config/slugs.json) UI Settings → Job sources
data/jobs.json Daily aggregator output, tuned to your profile Auto-created by pnpm run dev
data/ai-reviews.json Per-job LLM verdicts Auto-created by pnpm run ai-review
data/applications/<job-id>.md AI-generated application packages (cover letter, highlights, Q&A) Auto-created by AI Apply
data/apply-queue.json Pending / running / done / failed AI Apply jobs Written by the UI + pnpm run apply-worker
data/swipe-skips.json Persistent left-swipe (skip) decisions from Jinder Written by the UI
data/feed.xml RSS feed of new matches Auto-created by pnpm run dev
JOBS.md Daily readable matches table Auto-created by pnpm run dev
data/archive/*.json Monthly snapshots Auto-created on day 1 of each month
data/launchd-*.log, data/cron-*.log Local schedule logs Auto-created by the scheduler

The UI fetches data/jobs.json and data/ai-reviews.json at runtime from /api/jobs and /api/reviews (provided by the Vite middleware) — that's why a fresh clone with no data files still loads cleanly. If you genuinely want git history of any of these (e.g. you're using a private fork as a personal sync mechanism), remove the relevant line from .gitignore and git add the file.

Reset to a clean slate

pnpm run clean              # wipe generated outputs, archives, raw caches, schedule logs, queue state
pnpm run clean:onboarding   # reset just the onboarding state (preferences + brief + raw CV) — re-triggers wizard
pnpm run clean -- --all     # full fresh-clone reset: everything above PLUS brief, applied,
                            # swipe-skips, generated AI Apply packages, preferences, and uploaded CV

Idempotent — running on an already-clean state prints nothing to clean. After --all the UI re-routes to the onboarding wizard automatically; no hard refresh needed.

Stack

Layer Choice
Runtime Node 22 LTS, ESM modules
Language TypeScript 5.9 (NodeNext, strict)
Lint + format Biome 2.4
Package manager pnpm 11 (minimumReleaseAge: 1d + strictDepBuilds: true for supply-chain hardening)
Tests Vitest 3 — extensive backend + UI suite covering filters, dedup, utils, applied, salary, feed, aave, ashby-private, AI Apply core, apply queue, swipe-skips, profile bootstrap, plus every co-located UI hook (useJobsData, useApplied, useApplyQueue, useSwipeSkips, useUrlSyncedState, useOnboarding) and key components (AppliedBar, SignalChips, QueueBadge). Tiered keyword weighting + salary-aware sort tiebreak have dedicated cases.
Pre-commit simple-git-hooks (runs lint + typecheck on every commit)
HTTP Native fetch with AbortController (30s timeout, 1 retry on 5xx/network)
RSS parsing fast-xml-parser (only runtime dep)
HTML scraping Inline regex parsers (no cheerio/jsdom)
Schedule Local launchd (macOS) / cron (Linux), two agents: aggregator + AI review
Output Files in your local checkout (data/jobs.json, data/feed.xml RSS, JOBS.md, data/archive/<YYYY-MM>.json on month-start)
Static analysis Biome + tsc + build on every PR via check.yml

Architecture

                ┌─────────────────────────────────────────────────┐
                │ launchd / cron — local agent, daily               │
                └─────────────────────────────────────────────────┘
                                      │
                                      ▼
        ┌─────────────────── src/index.ts ──────────────────┐
        │                                                   │
        │   ┌──────── Fetchers (Promise.all) ─────────┐     │
        │   │ ashby (45 slugs)   greenhouse (8 slugs) │     │
        │   │ lever (8 slugs)    aave (custom)        │     │
        │   │ ashby-private (1)  cryptojobslist       │ ──► raw[] per source
        │   │ remoteok  remotive  weworkremotely      │     │
        │   │ web3career  aijobsnet                   │     │
        │   │ hn-hiring  hn-jobs                      │     │
        │   └─────────────────────────────────────────┘     │
        │                       │                           │
        │                       ▼                           │
        │   write data/raw/<source>-YYYY-MM-DD.json (gitignored)
        │                       │                           │
        │                       ▼                           │
        │   normalize (per source) → Job[]                  │
        │                       │                           │
        │                       ▼                           │
        │   filters.applyFilters: hard excludes + scoring   │
        │     (boilerplate stripped, body truncated to      │
        │      first 1500 chars before keyword scoring)     │
        │                       │                           │
        │                       ▼                           │
        │   dedup.dedupe: by URL, then by company+title     │
        │                       │                           │
        │                       ▼                           │
        │   sort by fitScore desc, postedAt desc, id asc    │
        │                       │                           │
        │                       ▼                           │
        │   attach config/applied.json status (by URL hash) │
        │                       │                           │
        │                       ▼                           │
        │   diff against previous data/jobs.json (✨ new)    │
        │                       │                           │
        │                       ▼                           │
        │   strip body field, write data/jobs.json,         │
        │     monthly archive on day-1, render JOBS.md      │
        │                                                   │
        └───────────────────────────────────────────────────┘
                                      │
                                      ▼
                  local files only -> commit manually if wanted

Each fetcher is isolated: it catches its own errors and returns an empty items array plus error details on failure. A 30-second AbortController timeout caps every HTTP call. One source going down can't break the rest of the run.

Where's the auto-commit step? Removed. The pipeline now runs locally and writes to your working tree. If you want git history of your daily snapshots, add a third launchd/cron task that runs git add data/ JOBS.md && git commit -m "chore: daily run" (or just commit by hand when you want).

Sources

The ATS fetchers (Ashby, Greenhouse, Lever, plus Recruitee and Personio for EU/DACH coverage) carry the bulk of the high-quality signal — each iterates a curated tier-S slug list. The other sources backfill long-tail listings. bluedoor is a different shape: a free aggregator spanning ~1.6M postings across 31 ATS providers (Workday, iCIMS, Oracle, ADP, …) we can't reach directly. It's queried region-by-region from your profile.location (no hardcoded geography), skips companies already covered by a dedicated fetcher, and sits at the lowest dedup priority so curated copies always win. Works fully on the anonymous tier — no signup required. To raise the per-run request budget, optionally export an API key in your shell (export BLUEDOOR_API_KEY=…); it's free via email OTP (no card). The fetcher reads it from the environment and falls back to anonymous when it's unset.

Source Type Endpoint
ashby JSON API api.ashbyhq.com/posting-api/job-board/<slug> × 45 tier-S slugs
greenhouse JSON API boards-api.greenhouse.io/v1/boards/<slug>/jobs × 8 tier-S slugs
aave HTML scraper (Next.js NEXT_DATA) aave.com/careers
ashby-private Ashby private GraphQL × N slugs jobs.ashbyhq.com/api/non-user-graphql (currently 1 slug: chainlink-labs)
lever JSON API api.lever.co/v0/postings/<slug> × 8 tier-S slugs
recruitee JSON API <slug>.recruitee.com/api/offers × 4 tier-S slugs (mostly EU scaleups; structured salary + work-type flags)
personio XML feed <slug>.jobs.personio.de/xml × 4 tier-S slugs (DACH/EU; no salary, URL rebuilt from slug + id)
remoteok JSON API remoteok.com/api
remotive JSON API remotive.com/api/remote-jobs?category=software-dev
weworkremotely RSS 2.0 weworkremotely.com/categories/remote-programming-jobs.rss
remoteyeah RSS 2.0 (custom tags) remoteyeah.com/rss.xml — global remote-engineering feed
jobicy JSON API jobicy.com/api/v2/remote-jobs — no-key remote feed; jobGeo region tags + structured salary
himalayas JSON API himalayas.app/jobs/api — no-key global remote feed (paged 20/req), structured salary
cryptojobslist RSS 2.0 api.cryptojobslist.com/jobs.rss
web3career HTML scraper 5 category pages on web3.career
aijobsnet HTML scraper aijobs.net (global + EU pages)
hn-hiring Algolia API latest "Ask HN: Who is hiring" thread
hn-jobs Algolia API hn.algolia.com/api/v1/search_by_date?tags=job
bluedoor JSON API (aggregator, ~1.6M postings / 31 ATS) api.bluedoor.sh/job-postings/v1/jobs/search — region fan-out from profile.location

The Ashby tier-S list covers the AI frontier (OpenAI, Cohere, Perplexity, Cursor, ElevenLabs, Modal, LangChain, LangFuse, LlamaIndex, OpenRouter, Pinecone, Supabase, Neon, Clerk, PostHog, Browserbase, Replit, Runway, Notion, Anyscale, BaseTen, Character, Weaviate) plus web3 (Linear, Ramp, Uniswap, Mysten Labs, Paradigm, Polygon Labs, Base, Blockworks, Succinct, Espresso, Phantom, Polymarket, Alchemy, Stacks, Morpho, Magic Eden, LiFi). Greenhouse adds Anthropic, Vercel, Mercury, Coinbase. Lever adds Binance, Ledger, CoinGecko, CoinMarketCap, Safe, Arbitrum Foundation, Celestia, and Mistral (which moved off Ashby to Lever). Custom first-party coverage: Aave via a Next.js __NEXT_DATA__ scraper, and Chainlink Labs via the ashby-private fetcher (Ashby's private GraphQL endpoint; the same fetcher generalized to a slug array, so any future org whose public posting-API is disabled is a one-line config add).

Adding another source is one new file in src/fetchers/, one entry in Source, one normalizer in normalize.ts, and one line in src/index.ts. See CONTRIBUTING.md for the contributor-facing recipe.

Pipeline stages

1. Fetch (src/fetchers/*.ts)

Each module exports fetch<Source>(): Promise<{ items: Raw[]; errors: string[] }> where Raw is source-specific. Errors are caught internally and aggregated, never thrown. Per-slug isolation in the multi-slug ATS fetchers so 404s on individual companies don't cascade. The 30s AbortController timeout in fetchWithTimeout retries once with a 2s backoff on >=500 and network errors; 4xx errors are not retried (they're permanent). Tier-S slug arrays for the multi-slug ATS sources (Ashby, Greenhouse, Lever, Recruitee, Personio) are loaded from config/slugs.json — adding a company is a non-code change.

2. Normalize (src/normalize.ts)

One function per source maps Raw → Job:

interface Job {
  id: string;                 // sha1 of normalized URL
  source: Source;             // one of the Source literal names
  title: string;
  company: string | null;
  url: string;                // http or https only — non-http schemes are filtered out
  location: string | null;
  remote: boolean;            // inferred from text + tags
  body: string;               // HTML stripped via regex (in-memory only)
  tags: string[];
  salary: string | null;          // surfaced when source provides it (Ashby/Lever/Remotive/web3career/aijobs)
  salaryMin: number | null;       // parsed annual integer (USD/EUR/etc., minor units stripped)
  salaryMax: number | null;       // parsed annual integer
  salaryCurrency: string | null;  // ISO code: 'USD' | 'EUR' | 'GBP' | …
  postedAt: string | null;    // ISO 8601
  fetchedAt: string;          // ISO 8601, set at run-start
  fitScore: number;           // 0-100, populated by filters
  categories: string[];       // ids of every matched config category (multi-label); [] = "Other"
  _signals?: JobSignals;      // per-job scoring breakdown (see below)
  applied?: AppliedEntry;     // attached when matched against config/applied.json
}

URLs are canonicalized (utm_* stripped, trailing slash normalized) before hashing into id. Any URL that isn't http(s): is rejected at the filter stage as a defense against javascript: / data: / file: payloads from upstream sources.

Note on data/jobs.json: the body field is stripped from the persisted file (it's regenerable from the URL and bloats the artifact ~10×). body is only present in-memory during a run.

3. Filter + score (src/filters.ts)

Hard excludes (drop entirely). The hard-drop chain is a named-rule list (HARD_RULES in src/filters.ts) — every drop is attributed to a rule name, and the per-rule count is surfaced in the JOBS.md "Dropped — hard filters" stat (e.g. (missing_senior_req=812, title_non_eng_role=64, ...)) so you can see which rule is doing the heavy lifting at a glance.

  • Title contains junior|jr|intern|entry-level|associate|graduate|trainee|apprentice.
  • Title does not contain senior|sr|staff|principal|lead|head|director|engineer(s)|developer(s)|architect(s).
  • Body matches a hard US-only / onsite pattern (e.g. must be located in the United States, onsite only, relocate to San Francisco).
  • Title or body matches a non-engineering pattern (marketing|sales|recruiter|...) and the title doesn't contain an engineering keyword.
  • Title matches a compound non-engineering pattern that contains the word "Engineer" but isn't real engineering: customer support engineer, sales engineer, solutions engineer, developer relations|advocate|experience, field engineering|operations, business operations, partner(ships) engineer, forward deployed engineer, implementation engineer, gtm, go-to-market.
  • Title is a non-engineering executive role: VP, Vice President, CMO, CRO, CFO, COO.
  • URL scheme is not http(s): (security gate against javascript: / data: / file: URLs from upstream).

Body preparation for scoring. Before regex-matching for keywords, the body is run through preparedScoringBody: it strips known company-boilerplate sections (EEO, privacy notices, accommodations, "About us") and truncates to the first 1500 characters. This prevents false positives from recruiter footers (e.g. "we use Anthropic Claude internally for support tooling" landing a +20 AI signal on a backend role). Hard-drop checks still see the full body so onsite/US-only language at the bottom of a posting is honored.

Soft signals (additive, capped at 100; weights live in config/profile.json):

Signal Weight
Category — title/body (or body-only) matches a configured category's keywords (e.g. Web3, AI) category's weight (e.g. +20)
Stack — body contains react|next.js|typescript (tiered) +10 base
Stack — body contains react native|expo (tiered) +5 base
Stack — body contains graphql|tailwind|vite (tiered) +5 base
Lead title — title contains lead|staff|principal|head +15
Senior title — title contains senior|sr +10
Frontend title — title contains frontend|front-end|fullstack|full-stack|web|mobile +10
Frontend body — body contains role-specific frontend phrases (design system, ship components, accessibility, etc.) (tiered) +10 base
Location — job matches an accepted region / is remote (driven by the location profile block) +10
Freshness — postedAt within 7 days +10
Freshness — postedAt within 14 days (and not within 7) +5
Penalty — job region-locked outside accepted regions (when not hard-excluding) -10

Tiered keyword weighting. The "stack/frontend body" signals (rows marked tiered above) count occurrences instead of doing a binary match: 1 mention earns half-weight, 2–3 the listed base weight, and 4+ a 1.5× boost. This lets a posting that mentions "react" eight times in concrete role context outscore one that drops it once in a "nice to have" footer. Category, title, location, and freshness signals stay binary because they're inherently low-cardinality and cheating them with repetition isn't a real concern.

Drop anything with fitScore < 30 after the cap.

Categories are config-driven, not hardcoded. They're defined in config/profile.json#categories (generated from your brief on Regenerate, or edited on the Profile tab). A job is tagged with every category whose keywords match (multi-label), and a category's optional weight also boosts the score. Jobs matching no category render under a synthetic "Other" bucket. So a frontend+web3+AI candidate gets web3 / ai buckets, while a fintech candidate's brief yields payments / compliance instead, with no code change. See the pupila-filters skill for the CategoryDef shape.

Unlike the regex-based keyword lists elsewhere in the profile, category keywords are plain literal terms (web3, smart contract, node.js): what you type is what's stored and shown. They're matched whole-word and case-insensitively by compileCategoryKeywords (in src/filters.ts), which anchors with alphanumeric lookarounds instead of \b, so punctuation-edged terms work (c++, c#, .net) and a dot between words is optional (node.js also matches nodejs). For other variants (agent / agents) list both. Regenerate emits literals too, so the Profile-tab chips never show regex noise.

Auditability — _signals. Every kept job in data/jobs.json carries a _signals object recording which scorin