diff --git a/ROADMAP.md b/ROADMAP.md index 434453d..1d50270 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -330,17 +330,170 @@ ship. That is the case virtual patching exists for, and the pipeline cannot see band-aid and escalates at TTL. Someone still has to ship the upgrade — documented, not a surprise. -- [ ] **H2** Dependency manifest input. (M, P2) Depends on H1. - Parse `requirements.txt`, `package-lock.json`, and `pom.xml`, resolve advisories, and run - H1 per exploitable advisory. - - Acceptance: a manifest run lists affected packages, resolved advisories, and one - band-aid per exploitable advisory; correlation collapses overlapping band-aids the way - `B1` already does for repo findings. - - **Reconciled:** B1 cannot be inherited as-is. `correlate.coverage_key` (`correlate.py:19-23`) - keys request-scoped controls on `endpoint_of(file)`, which parses a *repo path*, and is called - with `f.file` (`pipeline.py:226`). An advisory finding has no repo file. `coverage_key` needs a - non-file identity (fall back to `Finding.endpoint` or the package coordinate) and - `pipeline.py:40`'s dedup key needs the same treatment. +- [x] **H2** Dependency manifest input. (M, P2) — **DONE:** `vpcopilot scan --manifest ` + (repeatable, additive like `--spec`) and the read-only `vpcopilot deps …`. + `inputs/manifest.py` parses `requirements.txt` / `package-lock.json` (v1/v2/v3) / `pom.xml`; + `inputs/deps.py` resolves them against OSV and hands H1's stages one candidate per + (advisory, package). `GET`+`POST /api/deps`, a manifest field and a **Preview (no model calls)** + button on ① Scan, a **Dependencies** panel in the HTML report, `dependencies.json` in the evidence + bundle. Verified live end to end against api.osv.dev and a real model. 81 tests. + - **Acceptance, as met:** a manifest run lists affected packages ✅ and resolved advisories ✅ + (`dependencies.json`, complete regardless of what the agent stage reached) and produces one + band-aid per exploitable advisory ✅; correlation collapses overlapping band-aids ✅ — and does + it **across inputs**, which is the part that turned out to matter (see below). + - **The Reconciled note below was already satisfied before this item started.** H1 added the + `identity` fallback to `coverage_key` and the `f.file or f.source or f.id` dedup key, for the + same reason. Nothing in `correlate.py` or `_dedup_findings` needed changing. (Both line + citations had also drifted: the `coverage_key` call is `pipeline.py:352`, not `:226`.) + - **Batch was the wrong tool for the obvious job, and the right tool for a different one.** + `POST /v1/querybatch` returns advisory **ids only** — never the bodies. Fetching each id would + be one request per *advisory* (70 for `aiohttp 3.9.1` alone) where `/v1/query` returns every + full record for a coordinate in one round trip. So batch answers *which* packages are worth + asking about (400 coordinates in 3.4s, ids identical to `/v1/query`, nothing truncated) and the + bodies come from per-package queries for the hits: 16 requests for a 100-package manifest with + 15 vulnerable, against 100. One invalid ecosystem also fails the **whole** batch with HTTP 400, + so entries are validated before sending and a failed batch degrades to per-package queries + rather than to "no advisories". + - **Four defects the live API found, three of them in code H1 already shipped:** + - **CVSS v4 is now the majority and was being read as `medium`.** `severity_from_cvss` matched + `CVSS:3.[01]` only. Live, `aiohttp 3.9.1` returns 43 v4 scores against 32 v3, and **38 of its + 70 records publish a v4 vector and nothing else** — every one of which fell through to the + `medium` default whatever it said, including `AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H`, an + unauthenticated remote DoS. One CVE at a time that is a cosmetic mislabel; H2 *filters* on this + value, so it decided what got resolved at all. Fixed by mapping v4's `VC/VI/VA` onto the v3 + names so **one** bucketing rule serves both and they cannot drift; `AT` defaults to `N`, so + every v3 verdict is byte-identical (pinned by the existing parametrized test, unchanged). + - **`upgrade_target` recommends another package's version.** It takes the first `affected` row + carrying a version, whatever package it names — which is right for H1's question ("what does + this advisory fix", shown beside the package) and wrong for H2's ("what should I install"). + Reproduced live: a manifest pinning `org.ops4j.pax.logging:pax-logging-log4j2 1.10.0` is told + by Log4Shell to upgrade to **2.15.0**, which is `log4j-core`'s fix and not a version + pax-logging has ever published; its own is `1.10.8`. Same shape on `GHSA-p6mc-m468-83gw`, + where `lodash.pick` is affected with **no fix published at all** and would be sent to + `lodash`'s 4.17.19. `upgrade_target_for()` filters to the installed package and takes the + smallest published fix **strictly greater than the installed version**; `upgrade_target` is + untouched, so H1 is unaffected. + - **Which maintenance branch you are told about was decided by document order.** Selection + ignored the installed version entirely. Log4Shell publishes `2.13.0→2.15.0`, `2.0-beta9→2.3.1` + and `2.4→2.12.2` for one package; the live record happens to list the right one first, so + `2.14.1` got the correct answer **by luck**. Nothing in the OSV schema promises that order and + the failure it permits is recommending `2.3.1` to someone running `2.14.1` — a downgrade, + presented as the fix. Pinned by a test that reorders the blocks. This also needed a version + comparator: `packaging` is PEP 440 only and raises on Maven's `2.0-beta9`, and a string sort + puts `1.10.0` below `1.9.2`. + - **70 OSV records are 35 advisories.** OSV publishes a GHSA *and* a PYSEC entry for most Python + advisories, cross-linked by `aliases`. Collapsing them is correctness, not thrift — the + duplicate carries a different id, so nothing downstream would have recognised it as one, and + it would have cost two model calls and two band-aids per hole. `related` is deliberately not + followed: it links advisories that are *about* each other, not identical ones. + - **A defect only the live run could find, and the one worth reading.** The first end-to-end run + left **Log4Shell with no band-aid at all** while reporting it as covered. Triage recommended one + control for it — the LB-wide `waf` — which an aiohttp header-injection advisory had already + claimed, so the loop ended having generated nothing, and the WAF that actually shipped + (`waf-block-header-injection-nullbyte-crlf`) addressed nothing about JNDI. Two fixes, both + pinned: a finding whose *every* recommended band-aid was collapsed now falls through to its + non-recommended alternatives rather than ending bare (Log4Shell gets + `deny-log4j-jndi-lookup`), and an LB-wide correlation record now states that the attached policy + was generated and validated against the **owning** finding's exploit, not this one's. **This is + a deliberate behaviour change to a shared path** (B1 correlation, which repo scans use too): it + only widens — the alternatives are reached solely when the recommended tier produced nothing, so + a run where they succeed is byte-identical. Pinned in both directions. + - **The scale problem is the design.** A manifest is an unbounded input, so listing and resolving + are separated: `dependencies.json` carries every package parsed, every entry that could not be + pinned, and every advisory found, whatever the agent stage reached; only the agent stage is + bounded, by `--min-severity` (default `high`) and `--max-advisories` (default 25, `0` = off). + Everything held back is listed **with the reason it was held back** — "we did not check this" + and "this is clean" must never render the same way, in the CLI, the console, the report or the + bundle caveats. + - **The cap is shared across packages, not consumed in sort order.** The first live run made this + obvious: `aiohttp 3.9.1` alone carries 35 advisories and a flat `(severity, package, id)` + ordering handed it 23 of 25 slots, so every other package competed with one dependency's back + catalogue and anything later in the alphabet was capped out by advisories no worse than its own. + Dealt round-robin, the same 25 slots cover **7 vulnerable packages instead of 3**. + - **Refusing to pin a version is the load-bearing behaviour, and OSV is why.** A version string it + cannot parse does **not** error: `aiohttp` at `not-a-version`, `1.0.0-SNAPSHOT` and + `${project.version}` each returned **81 advisories** live, against 70 for the real `3.9.1` and + 87 for no version at all. So an unresolved `${...}` or an unpinned `flask>=2.0` produces a + bigger, wrong answer that every downstream stage treats as fact. Every entry the parsers cannot + pin goes to `unpinned` with a machine-readable reason and is never sent. + - **`vpcopilot deps` is the read-only surface**, reaching the same verdict about *which* + advisories a scan would spend a model call on without spending one — no model, no credentials, + no tenant. Both it and `POST /api/deps` call `deps.survey_report`, so the preview and the scan + cannot disagree about scope. + - **The artifact is `dependencies.json`, deliberately NOT `manifest.json`.** `export.verify_bundle` + locates each run inside an archive by every member whose name ends `manifest.json`, so an + out-dir artifact by that name would be read as a second evidence manifest and break verification + of the bundle it ships in. Pinned by a test. + - **Found by adversarial review, before shipping** (28 raised across five failure dimensions, 2 + refuted, 8 confirmed and fixed, plus 4 found while verifying the reviewers' claims). Every one + is the same shape — *something never actually checked rendering as clean* — which is the single + thing this input path exists to prevent: + - **The alternates fallback re-inflicted the exact bug it was written to fix, on a different + finding.** It ran interleaved with the recommended tier and generated **every** alternative, + so a non-recommended alternate could claim an LB-wide slot *before* a later finding that + actually recommended that control was reached. Reproduced: `loser`'s unrecommended + `rate_limit` took the slot, `brute` — which recommended `rate_limit` — got **no band-aid at + all** and was recorded as covered by a policy built from `loser`'s exploit, and `loser` got + three controls triage never recommended. This contradicted the "only widens / byte-identical" + guarantee stated below. Now a **deferred second pass**: every recommended claim is registered + first, and the fallback takes only the **first** alternative that generates. Verified live — + Log4Shell still gets `deny-jndi-log4shell-headers` after a *code* finding claims the WAF, and + `otp-brute-001` gets one alternative where it used to get three. + - **`code_fix_prs` counted dependency upgrades as drafted PRs.** A `--manifest` run reported + `code_fix_prs: 6` and the report's impact hero rendered "6 — code-fix PRs (the cure)" when + zero were drafted and none *can* be. Split on `kind`, the discriminator `pr.py` already + decides by; `dependency_upgrades` is its own count and hero stat. A repo scan is unchanged + (every remediation is `code_fix`). Pre-existing in H1 at N=1; H2 made it N-per-manifest. + - **A `package.json` parsed to zero packages, zero skipped and no error** — byte-identical to a + clean lockfile. `detect_kind` routes any JSON carrying `dependencies` to the npm parser, and + a package.json has one, but of name → *range string* where a v1 lock has name → `{version}`; + every entry failed the isinstance check and was dropped. It is now refused with an actionable + error, because a package.json has no installed versions to check at all. + - **`dependencies.json` was never cleared**, so a later scan of the same out dir *without* a + manifest republished the previous run's dependency data through the report, `GET /api/deps` + and the evidence bundle. It is the only conditionally-written member; the rest are always + rewritten. (G4 shipped this same class of bug with leftover `policies/` artifacts.) + - **One 429 during the batch fallback ended the whole scan** — including the repo half of a + `scan ./repo --manifest` run. The per-package retry loop was unguarded, so the fallback that + exists so a batch failure never loses a chunk became the thing that lost everything. Each + coordinate is now guarded and a failed one is reported **unchecked**, never clean. + - **A package whose advisory fetch failed rendered exactly like a clean one** — it stayed in + `vulnerable` but contributed no row, no counter and no warning. New `unchecked` list and + `packages_unchecked` counter, surfaced in `dependencies.json`, the CLI, the console and the + report. + - **The console preview dropped parse errors entirely**, so an unreadable manifest showed an + all-zero funnel and an empty table — indistinguishable from a clean one. The CLI had printed + them in red all along, so the guard lived on only one surface. + - **A cleared "Max advisories" box meant NO cap in the console** (`parseInt("")||0` → 0 → off) + where the CLI defaults to 25 — a two-surfaces divergence failing in the expensive direction. + - **npm workspace source trees were queried as registry packages.** v2/v3 `packages` keys are + paths; only `node_modules/…` are installs. First-party code was sent to OSV under a name that + can collide with a public package, and an entry with no `name` went under its directory path, + which is not a legal npm name — both counted as checked-and-clean. + - Plus: a v1 lock entry with no `version` vanished from both halves of the answer where the + v2/v3 branch refuses it explicitly; an unresolved `${...}` in a pom's **groupId/artifactId** + reached OSV inside the coordinate name (only the *version* was guarded); and a short `200` + from `querybatch` left the unanswered tail absent from the result, which the caller reads as + "no advisories". + - **Refuted, and worth recording:** `POST /api/deps` taking a caller-supplied path is *not* a + new arbitrary-file reader. The console already accepts caller paths in `POST /api/scan` + (`repo`, `spec`, `manifest`), `/api/apply` and `/api/apply-apischema`, binds `127.0.0.1`, and + ships no CORS middleware; the identical capability is the documented CLI contract. J2's + refusal of a caller-supplied path applied to a bundle **derivable from server state**, where + accepting one adds capability for zero function. A manifest has no server-state equivalent. + - **Decisions:** `--manifest` is **additive** (H3's precedent) rather than exclusive like `--cve`, + because a manifest lives in the repo you are already scanning and the cross-input correlation is + the point — verified live: Log4Shell's WAF slot was claimed by a *code* finding + (`sqli-login-001`) in a `repo+manifest` run. Dev/test-scoped dependencies are **listed but not + resolved** without `--include-dev`: a build-time package is not in the request path. One + candidate per (advisory, package) — two packages hit by one advisory are two upgrades to ship — + but the resolve agent runs once per *advisory*, so they cost one model call and cannot receive + two different verdicts. + - **Reconciled (superseded — see above):** B1 cannot be inherited as-is. `correlate.coverage_key` + (`correlate.py:19-23`) keys request-scoped controls on `endpoint_of(file)`, which parses a *repo + path*, and is called with `f.file` (`pipeline.py:226`). An advisory finding has no repo file. + `coverage_key` needs a non-file identity (fall back to `Finding.endpoint` or the package + coordinate) and `pipeline.py:40`'s dedup key needs the same treatment. - [x] **H3** OpenAPI as a discovery input. (M, P2) — **DONE:** `vpcopilot scan --spec `, alone or alongside a repo. `inputs/openapi.py` does a deterministic structural pass (unbounded diff --git a/bench/fixtures/manifests/package-lock.json b/bench/fixtures/manifests/package-lock.json new file mode 100644 index 0000000..cce72f1 --- /dev/null +++ b/bench/fixtures/manifests/package-lock.json @@ -0,0 +1,49 @@ +{ + "name": "nimbus-web", + "version": "1.0.0", + "lockfileVersion": 2, + "requires": true, + "packages": { + "": { + "name": "nimbus-web", + "version": "1.0.0", + "dependencies": { "lodash": "4.17.15", "minimist": "1.2.5" } + }, + "node_modules/lodash": { + "version": "4.17.15", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz" + }, + "node_modules/minimist": { + "version": "1.2.5", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.5.tgz" + }, + "node_modules/@babel/traverse": { + "version": "7.22.5", + "resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.22.5.tgz", + "dev": true + }, + "node_modules/jest/node_modules/lodash": { + "version": "4.17.15", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz", + "dev": true + }, + "node_modules/internal-tools": { + "resolved": "packages/internal-tools", + "link": true + }, + "node_modules/forked-dep": { + "version": "1.0.0", + "resolved": "git+ssh://git@github.com/acme/forked-dep.git#abc123" + }, + "node_modules/no-version-here": { + "resolved": "https://registry.npmjs.org/no-version-here/-/x.tgz" + } + }, + "dependencies": { + "lodash": { "version": "4.17.15", "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz" }, + "minimist": { "version": "1.2.5", "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.5.tgz" }, + "@babel/traverse": { "version": "7.22.5", "dev": true }, + "jest": { "version": "29.0.0", "dev": true, + "dependencies": { "lodash": { "version": "4.17.15", "dev": true } } } + } +} diff --git a/bench/fixtures/manifests/pom.xml b/bench/fixtures/manifests/pom.xml new file mode 100644 index 0000000..76e35aa --- /dev/null +++ b/bench/fixtures/manifests/pom.xml @@ -0,0 +1,62 @@ + + + 4.0.0 + test.nimbus + nimbus-ledger + 2.4.0 + + + 2.14.1 + 2.13.0 + + + + + + com.fasterxml.jackson.core + jackson-databind + ${jackson.version} + + + + + + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + + + + com.fasterxml.jackson.core + jackson-databind + + + + test.nimbus + nimbus-common + ${project.version} + + + + junit + junit + 4.12 + test + + + + org.springframework + spring-core + ${spring.version} + + + + org.slf4j + slf4j-api + + + diff --git a/bench/fixtures/manifests/requirements-dev.txt b/bench/fixtures/manifests/requirements-dev.txt new file mode 100644 index 0000000..b898be7 --- /dev/null +++ b/bench/fixtures/manifests/requirements-dev.txt @@ -0,0 +1,3 @@ +# Included from requirements-with-include.txt to exercise `-r`. +pytest==7.4.3 +black==23.11.0 diff --git a/bench/fixtures/manifests/requirements-with-include.txt b/bench/fixtures/manifests/requirements-with-include.txt new file mode 100644 index 0000000..689f276 --- /dev/null +++ b/bench/fixtures/manifests/requirements-with-include.txt @@ -0,0 +1,3 @@ +-r requirements-dev.txt +--requirement requirements.txt +mypy==1.7.1 diff --git a/bench/fixtures/manifests/requirements.txt b/bench/fixtures/manifests/requirements.txt new file mode 100644 index 0000000..b97fecb --- /dev/null +++ b/bench/fixtures/manifests/requirements.txt @@ -0,0 +1,26 @@ +# Nimbus payments service — runtime dependencies. +# H2 fixture: every line here exercises one behaviour of the requirements.txt parser. + +aiohttp==3.9.1 # pinned and vulnerable — the case H2 exists for +requests==2.31.0 +Flask==3.0.0 +PyYAML==6.0.1 + +# extras and case/underscore normalisation (PEP 503: Flask_Login == flask-login) +Flask_Login[extras]==0.6.2 + +# a marker gates WHETHER it is installed, not WHICH version — kept, and the marker recorded +importlib-metadata==6.8.0 ; python_version < "3.10" + +# a hash-pinned requirement spanning continuation lines +urllib3==2.0.7 \ + --hash=sha256:c97dfde1f7bd43a71c8d2a58e369e9b2bf692d1334ea9f9cae55add7d0dd8f84 + +# --- everything below is DELIBERATELY unresolvable, and must be reported, not guessed --- +gunicorn>=21.0 # a range: the resolver picks the version, not us +cryptography # unpinned entirely +boto3~=1.34.0 # compatible-release, still a range +django==4.2.* # wildcard is not an exact version +-e . # editable install of this project +psycopg @ https://example.test/psycopg-3.1.0.tar.gz # direct URL reference +--index-url https://pypi.example.test/simple diff --git a/docs/AUDIT.md b/docs/AUDIT.md index c653b2c..886c82b 100644 --- a/docs/AUDIT.md +++ b/docs/AUDIT.md @@ -187,9 +187,13 @@ keeps its `run_id`, so audit entries already on disk stay joinable. | `repo` | absolute path of the scanned repo (`root.resolve()`) | | `repo_commit` / `repo_branch` / `repo_dirty` | `runmeta.git_provenance(repo)` — `git rev-parse HEAD`, `rev-parse --abbrev-ref HEAD`, `git status --porcelain`. **Fail-soft**: a target that is not a git checkout contributes none of these keys at all | | `config_path` | the `config/agents*.yaml` the scan ran with — **absent** when the scan used the default config (`runmeta.write_manifest` drops `None` fields) | -| `models` | `{agent: model}` for each of `config.AGENT_NAMES` = `discover, verify, triage, generate, remediate, probe, refine` | +| `input_kind` | what was scanned: `repo`, `spec`, `manifest`, `advisory`, or a `+`-joined combination (`repo+spec`, `repo+manifest`, `repo+spec+manifest`). `advisory` is never combined — `--cve` is exclusive | +| `advisory` | H1 only: `{id, source, consulted, network_observable, fixed_version}` for a `--cve` scan | +| `spec` | H3: absolute path of the OpenAPI spec, when `--spec` was given | +| `manifests` | H2: absolute paths of the dependency manifests, when `--manifest` was given. The per-package detail is in `dependencies.json`, not here | +| `models` | `{agent: model}` for each of `config.AGENT_NAMES` = `resolve, discover, verify, triage, generate, remediate, probe, refine` | | `caps` | `{min_confidence, max_files, max_bytes, draft_code_fixes}` — the limits the scan ran under | -| `counts` | `{candidates, verified, policies, code_fix_prs}` | +| `counts` | `{candidates, verified, policies, code_fix_prs, dependency_upgrades}`. The last two are split on `RemediationPlan.kind`: a `code_fix` is a patch drafted against a file you own and is what `pr` opens; a `dependency_upgrade` (H1/H2) is a version bump in someone else's package, so **no PR was drafted and none can be**. Counting the two together overstated the cure half of the run on every surface that renders it | | `started` / `finished` | UTC scan bounds | | `actor` / `host` / `tool_version` / `out_dir` | re-stamped on every manifest write | @@ -326,6 +330,8 @@ ledger.json found → mitigated → remediated → retired, per fin findings.json triage.json policies.json remediations.json summary.json metrics.json probes.json correlations.json lb_snapshot.json the most recent pre-change LB state +simulation.json the blast-radius replay result, when a simulation ran (G2) +dependencies.json the dependency funnel, when the scan used --manifest (H2) report.html the standalone HTML report policies/* the exact XC configs that were pushed snapshots/* per-LB timestamped pre-change state (`-.json`) @@ -335,6 +341,30 @@ Missing members are skipped, not faked — an unscanned dir has no `findings.jso live apply has no `snapshots/`. (`apply_timing` is an *entry inside* `audit.log`, not a member — a CLI-driven run simply has none of those lines.) +**`dependencies.json` is not a clean bill of health**, and the manifest's `caveats` say so, because +this is the member most likely to be read as one. Four things a reviewer has to know: + +- **`unpinned`** lists manifest entries whose exact version could not be established — a range like + `flask>=2.0`, an editable install, an unresolved `${property}`, a version inherited from a parent + POM. Those were **never queried**. Nothing is known about them; that is not the same as clean. + They are excluded on purpose: OSV answers a version string it cannot parse with a larger, wrong + advisory set rather than an error, so a guess would look like an answer. +- **`advisories[].disposition`** distinguishes what was assessed from what was merely found. + `exploitable` and `declined` came back from the resolve agent. `below_severity` and `capped` were + found and listed but never sent to one, so no band-aid was ever considered for them — + `settings.min_severity` and `settings.max_advisories` record the thresholds that did it. + `resolve_failed` is an advisory whose agent call errored. +- **`unchecked`** (with `funnel.packages_unchecked`) is a package OSV said has advisories and then + could not be asked about — a 429, a timeout, a transport error on the follow-up query. Its + advisories are **unknown, not absent**. Before this was its own list it appeared only as an + `error` key inside `packages[]`, contributing no row, no counter and no warning, which read + exactly like a package that came back clean. +- **`funnel`** reconciles the counts end to end, from `packages_parsed` through to `exploitable`. + Note `packages_parsed` counts the entries that **were** pinned; `packages_unpinned` is a disjoint + list, not a subset of it, so the queried count is `packages_parsed − packages_dev_excluded`. + It reflects OSV.dev **on the run date only** — `run.json`'s `finished` is the as-of timestamp, and + re-running later will legitimately produce different numbers. + `--all` produces one archive with each run under its own folder (`out-claude/`, `demo-out/`, …) plus a top-level `index.json` listing `{out_dir, folder, events, run_id}` per run. Run dirs are discovered by `export.find_runs`: everything matching `out*/` plus `demo/out`, keeping only dirs diff --git a/docs/USAGE.md b/docs/USAGE.md index 0ad60b2..791e35f 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -116,6 +116,78 @@ That finding is a comparison of two documents, so it skips the verify agent enti adversarial code reviewer to confirm a vulnerability in source it cannot see got it refuted at 0.10 confidence. +### Scan a dependency manifest (H2) + +`--cve` answers "can a load balancer hold the line on *this* advisory". `--manifest` asks it of +every dependency you actually have — which is the form the question normally arrives in. Nobody +hands you a CVE id; they hand you a `requirements.txt`. + +```sh +vpcopilot deps ./requirements.txt # what a scan WOULD find — no model calls +vpcopilot scan --manifest ./requirements.txt --out out # the dependency tree alone +vpcopilot scan ./app --manifest ./package-lock.json --out out # …and the code, correlated together +vpcopilot scan --manifest ./requirements.txt --manifest ./pom.xml --out out # repeatable +``` + +`--manifest` is **additive**, like `--spec`. Alone it resolves the dependency tree; alongside a repo +the code findings and the dependency findings correlate together, so one band-aid can cover both. +Formats: `requirements.txt`, `package-lock.json` (v1/v2/v3), `pom.xml`. + +**Start with `vpcopilot deps`.** It parses the manifests, asks OSV which pinned packages have +advisories, and prints the whole funnel — with no model, no credentials and no tenant. It is the +cheap way to see what a scan would cost and to tune the two knobs below before paying for one. + +**Two things it will not do, and both are the point:** + +- **It never guesses a version.** `flask>=2.0`, a bare `cryptography`, `-e .`, a `${spring.version}` + with no `` entry, a version inherited from a parent POM — each is listed under + `unpinned` with a reason, and never sent to OSV. This matters more than it sounds: OSV does *not* + error on a version string it cannot parse. Measured live, `aiohttp` at `not-a-version`, + `1.0.0-SNAPSHOT` and `${project.version}` each returned **81 advisories**, against 70 for the real + `3.9.1`. A guess does not fail loudly — it returns a bigger, wrong answer. +- **It never hides what it skipped.** *"We did not check this"* and *"this is clean"* must not read + the same way, so every unpinned entry and every advisory held back by the filters is in + `dependencies.json`, on the console preview and in the HTML report, carrying its reason. The same + goes for what it could not check *despite trying*: a package OSV flagged and then failed to + answer for (a 429, a timeout) is listed under `unchecked` — its advisories are unknown, not + absent — and a manifest it could not read at all is an error on every surface, never an empty + table. A `package.json` is refused outright rather than half-read: it names version *ranges*, so + there are no installed versions to check; point at `package-lock.json`. + +**Bounding the agent stage.** Listing is cheap; resolving is not. `aiohttp` pinned at `3.9.1` alone +returns 70 OSV records, and a modest manifest reaches several hundred advisories. So the *listing* +is always complete and only the *resolve agent* is bounded: + +| flag | default | what it does | +|---|---|---| +| `--min-severity` | `high` | floor for reaching the agent. Below it: listed, not resolved. | +| `--max-advisories` | `25` | cap on the agent stage (`0` = no cap). | +| `--include-dev` | off | also resolve dev/test-scoped deps. A build-time package is not in the request path. | + +The cap is **shared out across packages**, not consumed in sort order — every vulnerable package +gives up its worst advisory before any package gives up its second. On the fixture manifests a flat +ordering gave `aiohttp` 23 of 25 slots because it sorts first and carries 35 advisories; sharing the +budget covers 7 packages instead of 3 for the same cost. + +**The fixed version is code's, never a model's** — as in H1, and with one addition H2 needs. The +recommendation is the smallest published fix **strictly greater than the version you have, for the +package you have**. An advisory that names several packages fixes each at its own version (Log4Shell +fixes `log4j-core` at 2.15.0 and `pax-logging-log4j2` at 1.10.8), and one package's fix is not +installable for another. Where nothing published is newer, `dependencies.json` says so rather than +naming the closest number. + +**Declining is still the load-bearing behaviour.** Most dependency advisories are not observable in +an HTTP request, and those route to `no_bandaid` in code with the residual risk stated and the +upgrade named. On the fixture manifests a typical run resolves 6 advisories to 2 exploitable and 4 +declined. A tool that invented a plausible path for all 6 would be worse than no tool. + +**No cure PR, and the counts say so.** As with `--cve`, the cure is a version bump in someone +else's package, so `remediate` is never called and `pr` declines with the upgrade recommendation. +Because no PR is drafted and none *can* be, an upgrade is never counted as one: `summary.json` and +`run.json` carry `code_fix_prs` and `dependency_upgrades` separately, and the report shows +"upgrades to ship (no PR)" beside the PR count rather than folding them together. Someone still has +to ship it — `reconcile` (§6) holds the band-aid and escalates at TTL. + ## 4. Apply a band-aid (mutates XC — gated + reversible) ```sh vpcopilot apply --from-scan out/policies/.json --lb --url --dry-run # preview @@ -335,7 +407,7 @@ header carries a live model switcher, and each step is deep-linkable (`#mitigate | Step | What | |---|---| -| **① Scan** | point at a repo and run the pipeline — read-only, no XC/GitHub writes. Auto-advances to Review when it finishes | +| **① Scan** | point at a repo — or a CVE, an OpenAPI spec, or dependency manifests — and run the pipeline. Read-only, no XC/GitHub writes. **Preview (no model calls)** shows the H2 dependency funnel before you spend anything. Auto-advances to Review when it finishes | | **② Review** | verified findings + the recommended band-aid; click a row for exploit / code / generated policy. **Open HTML report ↗** + **Download** | | **③ Simulate** | replay a recorded sample against each candidate through a **spare** LB and report what it would block; over-threshold policies warn at the gate | | **④ Mitigate** | apply each band-aid (or **Mitigate ALL**, one at a time, continuing past failures) and watch `before → after` stream, with a *self-healed in N attempts* badge | diff --git a/src/vpcopilot/cli.py b/src/vpcopilot/cli.py index ce7e25c..d757e6e 100644 --- a/src/vpcopilot/cli.py +++ b/src/vpcopilot/cli.py @@ -2,7 +2,9 @@ writes) and drops findings, triage, policy specs, and code-fix PR drafts into ./out.""" from __future__ import annotations +import json import os +from pathlib import Path import typer from dotenv import load_dotenv @@ -36,6 +38,10 @@ def scan( repo: str = typer.Argument(None, help="path to the target application repo (omit when using --cve)"), cve: str = typer.Option(None, "--cve", help="scan a security advisory instead of a repo: CVE-YYYY-NNNNN, GHSA-xxxx-xxxx-xxxx, PYSEC-YYYY-NN, GO-… or RUSTSEC-…"), spec: str = typer.Option(None, "--spec", help="an OpenAPI/Swagger spec to scan for flaws IN THE CONTRACT — alone, or alongside a repo to also report spec/code drift"), + manifest: list[str] = typer.Option(None, "--manifest", help="a dependency manifest (requirements.txt, package-lock.json, pom.xml) to resolve against OSV — repeatable, alone or alongside a repo"), + min_severity: str = typer.Option("high", "--min-severity", help="--manifest: floor for reaching the resolve agent (critical|high|medium|low). Advisories below it are still listed in dependencies.json"), + max_advisories: int = typer.Option(25, "--max-advisories", help="--manifest: cap on advisories sent to the resolve agent (0 = no cap). What the cap held back is listed and counted, never dropped silently"), + include_dev: bool = typer.Option(False, "--include-dev", help="--manifest: also resolve dev/test-scoped dependencies (default: runtime only — a build-time package is not in the request path)"), out: str = typer.Option("out", help="output directory for findings/policies/PRs"), config: str = typer.Option(None, "--config", help="path to agents.yaml"), min_confidence: float = typer.Option(0.5, "--min-confidence", help="drop verified findings below this confidence"), @@ -51,16 +57,26 @@ def scan( With --cve the input is a security advisory instead of a repo: the advisory is resolved from OSV.dev, an agent derives its HTTP exploitation profile (or says there isn't one), and the - result enters the same triage and generate stages.""" - if cve and (repo or spec): - raise typer.BadParameter("--cve scans one advisory; it cannot be combined with a repo or --spec") - if not (repo or cve or spec): - raise typer.BadParameter("pass a repo path, --cve, or --spec") + result enters the same triage and generate stages. + + With --manifest the input is a dependency manifest, and the same happens for every exploitable + advisory affecting a package it pins. Additive: pass a repo too and the code findings and the + dependency findings correlate together.""" + if cve and (repo or spec or manifest): + raise typer.BadParameter("--cve scans one advisory; it cannot be combined with a repo, " + "--spec or --manifest") + if not (repo or cve or spec or manifest): + raise typer.BadParameter("pass a repo path, --cve, --spec, or --manifest") + if min_severity not in ("critical", "high", "medium", "low"): + raise typer.BadParameter(f"--min-severity must be critical, high, medium or low, not " + f"'{min_severity}'") if code_fixes is None: # match the console default (app.py /api/defaults) so headless == UI code_fixes = os.environ.get("VPCOPILOT_SCAN_REMEDIATE", "1").lower() not in ("0", "false", "no") summary = run_pipeline(repo, out_dir=out, config_path=config, min_confidence=min_confidence, concurrency=concurrency, max_files=max_files, max_bytes=max_bytes, draft_code_fixes=code_fixes, advisory=cve, spec_path=spec, + manifest_paths=list(manifest or []) or None, min_severity=min_severity, + max_advisories=max_advisories, include_dev=include_dev, log=lambda m: rprint(f"[dim]{m}[/dim]")) rprint(Panel.fit( "\n".join(f"[bold]{k}[/bold]: {v}" for k, v in summary.items()), @@ -68,6 +84,55 @@ def scan( )) +@app.command() +def deps( + manifest: list[str] = typer.Argument(..., help="dependency manifests: requirements.txt, package-lock.json, pom.xml"), + min_severity: str = typer.Option("high", "--min-severity", help="floor a scan would resolve at (critical|high|medium|low)"), + max_advisories: int = typer.Option(25, "--max-advisories", help="cap a scan would apply (0 = no cap)"), + include_dev: bool = typer.Option(False, "--include-dev", help="also count dev/test-scoped dependencies"), + json_out: str = typer.Option(None, "--json", help="write the full report to this path"), + show: int = typer.Option(20, "--show", help="advisory rows to print (0 = all)"), +): + """List what a --manifest scan would find, without spending a single model call. + + Read-only and credential-free: it parses the manifests, asks OSV which pinned packages have + advisories, and prints the funnel a scan would run — including every entry it could NOT pin, + which is the half a dependency scanner usually leaves out.""" + if min_severity not in ("critical", "high", "medium", "low"): + raise typer.BadParameter(f"--min-severity must be critical, high, medium or low, not " + f"'{min_severity}'") + from .inputs.deps import survey_report + rep = survey_report(list(manifest), min_severity=min_severity, max_advisories=max_advisories, + include_dev=include_dev, log=lambda m: rprint(f"[dim]{m}[/dim]")) + f = rep["funnel"] + t = Table(title="dependency advisories") + for c in ["severity", "package", "installed", "advisory", "fixed", "disposition"]: + t.add_column(c) + rows = rep["advisories"] if not show else rep["advisories"][:show] + for a in rows: + t.add_row(a["severity"], a["package"], a["installed"], a["advisory_id"], + a["fixed_version"] or "[yellow]none[/yellow]", a["disposition"]) + rprint(t) + if show and len(rep["advisories"]) > show: + rprint(f"[dim]… {len(rep['advisories']) - show} more (--show 0 for all)[/dim]") + rprint(Panel.fit("\n".join(f"[bold]{k}[/bold]: {v}" for k, v in f.items()), + title="funnel")) + if rep["unpinned"]: + # The loud half: a package whose version we could not establish was never checked, and + # that must not read the same way as a package that was checked and came back clean. + rprint(f"[yellow]{len(rep['unpinned'])} entry(ies) could not be pinned and were NOT " + f"checked[/yellow] — {', '.join(sorted({u['reason'] for u in rep['unpinned']}))}") + for u in rep.get("unchecked") or []: + # Flagged as having advisories, then the follow-up query failed. Unknown, not absent. + rprint(f"[red]could not check {u['ecosystem']}/{u['name']} {u['version']}[/red] — " + f"{u['error']}; its advisories are UNKNOWN, not absent") + for e in rep["errors"]: + rprint(f"[red]{e['path']}: {e['error']}[/red]") + if json_out: + Path(json_out).write_text(json.dumps(rep, indent=2)) + rprint(f"wrote [bold]{json_out}[/bold]") + + @app.command() def bench( repo: str = typer.Argument(..., help="app dir to scan (e.g. bench/fixtures/nimbus-vuln-lab/app/src/app/api)"), diff --git a/src/vpcopilot/console/app.py b/src/vpcopilot/console/app.py index 42f21ea..771c992 100644 --- a/src/vpcopilot/console/app.py +++ b/src/vpcopilot/console/app.py @@ -182,6 +182,50 @@ def simulation(): return {"out": str(OUT), "simulation": load_result(str(OUT))} +@app.get("/api/deps") +def dependencies(): + """H2: the dependency funnel this run produced — every package parsed, every entry that could + not be pinned, and every advisory found with what became of it. Empty until a scan runs with + `--manifest`.""" + p = OUT / "dependencies.json" + if not p.is_file(): + return {"out": str(OUT), "dependencies": None} + try: + return {"out": str(OUT), "dependencies": json.loads(p.read_text())} + except (OSError, json.JSONDecodeError) as e: + return {"out": str(OUT), "dependencies": None, "error": str(e)} + + +class DepsReq(BaseModel): + manifest: list[str] = [] + min_severity: str = "high" + max_advisories: int = 25 + include_dev: bool = False + + +@app.post("/api/deps") +def survey_dependencies(body: DepsReq): + """H2 read-only: what a `--manifest` scan WOULD find, without spending a model call. + + The same module function `vpcopilot deps` calls. Synchronous rather than a background job + because it is parse + HTTP only — one batch query plus one query per vulnerable package — and + it needs no model and no credentials, so there is nothing to stream.""" + paths = [m.strip() for m in body.manifest if m.strip()] + if not paths: + raise HTTPException(400, "give at least one manifest path") + if body.min_severity not in ("critical", "high", "medium", "low"): + raise HTTPException(400, "min_severity must be critical, high, medium or low") + from ..inputs.deps import survey_report + log: list[str] = [] + try: + rep = survey_report(paths, min_severity=body.min_severity, + max_advisories=body.max_advisories, include_dev=body.include_dev, + log=log.append) + except Exception as e: # noqa: BLE001 — a bad path is a 400, not a 500 traceback in the browser + raise HTTPException(400, str(e)) from e + return {"dependencies": rep, "log": log} + + class SimReq(BaseModel): logs: str | None = None # HAR / JSONL path from_tenant: bool = False # read observed requests from XC access logs @@ -595,6 +639,12 @@ class ScanReq(BaseModel): repo: str = "" cve: str = "" spec: str = "" + # H2 — `list[str] = []` for the same reason `cve` is `str = ""`: index.html can always send the + # field and an empty picker yields `[]`, so every payload written before H2 stays valid. + manifest: list[str] = [] + min_severity: str = "high" + max_advisories: int = 25 + include_dev: bool = False out: str = "out" min_confidence: float = 0.5 max_files: int = 200 @@ -604,7 +654,8 @@ class ScanReq(BaseModel): def _run_scan(repo: str, out: str, min_confidence: float = 0.5, max_files: int = 200, max_bytes: int = 60_000, draft_code_fixes: bool = True, - cve: str = "", spec: str = ""): + cve: str = "", spec: str = "", manifest: list[str] | None = None, + min_severity: str = "high", max_advisories: int = 25, include_dev: bool = False): _scan.update(state="running", log=[], summary=None, error=None) try: from ..pipeline import run_pipeline @@ -612,6 +663,9 @@ def _run_scan(repo: str, out: str, min_confidence: float = 0.5, min_confidence=min_confidence, max_files=max_files, max_bytes=max_bytes, draft_code_fixes=draft_code_fixes, advisory=cve or None, spec_path=spec or None, + manifest_paths=list(manifest or []) or None, + min_severity=min_severity, max_advisories=max_advisories, + include_dev=include_dev, log=lambda m: _append(_scan["log"], m)) _scan.update(state="done", summary=summary) except Exception as e: # noqa: BLE001 @@ -626,10 +680,14 @@ def start_scan(body: ScanReq): # The console reads results from OUT — so point OUT at the dir this scan writes to, or Review / # Mitigate would read a different (empty) dir. Makes the Output-dir field authoritative even when # it differs from the model-switcher default (e.g. out-claude-vampi). - if body.cve.strip() and (body.repo.strip() or body.spec.strip()): - raise HTTPException(400, "a CVE scan cannot be combined with a repo or a spec") - if not (body.repo.strip() or body.cve.strip() or body.spec.strip()): - raise HTTPException(400, "give a repo path, a CVE/GHSA id, or an OpenAPI spec") + manifests = [m.strip() for m in body.manifest if m.strip()] + if body.cve.strip() and (body.repo.strip() or body.spec.strip() or manifests): + raise HTTPException(400, "a CVE scan cannot be combined with a repo, a spec or a manifest") + if not (body.repo.strip() or body.cve.strip() or body.spec.strip() or manifests): + raise HTTPException(400, "give a repo path, a CVE/GHSA id, an OpenAPI spec, " + "or a dependency manifest") + if body.min_severity not in ("critical", "high", "medium", "low"): + raise HTTPException(400, "min_severity must be critical, high, medium or low") global OUT OUT = Path(body.out) # kwargs, not a positional tuple: H2/H3 add more inputs here and a positional args tuple is one @@ -638,7 +696,10 @@ def start_scan(body: ScanReq): kwargs=dict(repo=body.repo, out=body.out, min_confidence=body.min_confidence, max_files=body.max_files, max_bytes=body.max_bytes, draft_code_fixes=body.draft_code_fixes, cve=body.cve, - spec=body.spec)).start() + spec=body.spec, manifest=manifests, + min_severity=body.min_severity, + max_advisories=body.max_advisories, + include_dev=body.include_dev)).start() return {"state": "running", "out": str(OUT)} diff --git a/src/vpcopilot/console/static/index.html b/src/vpcopilot/console/static/index.html index 2e19b0b..822ac91 100644 --- a/src/vpcopilot/console/static/index.html +++ b/src/vpcopilot/console/static/index.html @@ -151,6 +151,16 @@ + + +
+
+
+
+
+
+
+
@@ -315,6 +325,11 @@ async function jpost(u,b){ const r=await fetch(u,{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify(b)}); const d=await r.json().catch(()=>({})); if(!r.ok) throw new Error(d.detail||r.statusText); return d; } const esc = s => (s==null?"":String(s)).replace(/[&<>]/g,c=>({"&":"&","<":"<",">":">"}[c])); +// The advisory cap is the one numeric field whose falsy value MEANS something: 0 = no cap. The +// usual `parseInt(v)||` therefore turns a CLEARED box into an UNBOUNDED resolve, where +// the CLI defaults to 25 — a two-surfaces divergence that fails in the expensive direction. Blank +// or unparseable falls back to the CLI default; an explicit 0 is honoured. +const advCap = v => { const n = parseInt(v); return Number.isFinite(n) && n >= 0 ? n : 25; }; // ---- log windows ---- // The scan/apply endpoints serve the FULL transcript and take ?since=, so we // APPEND the new tail instead of rewriting the node. That keeps scroll position and text selection @@ -382,6 +397,8 @@ summary.innerHTML = Object.keys(s).length ? [ chip("candidates", s.candidates), chip("verified", s.verified), chip("band-aids", (s.policies||[]).length), chip("code-fix PRs", (s.code_fix_prs||[]).length), + // H2: an advisory cure is an upgrade someone else ships — never counted as a drafted PR. + (s.dependency_upgrades||[]).length ? chip("upgrades to ship", s.dependency_upgrades.length) : "", m.verify?chip("confirm rate", (m.verify.confirm_rate!=null?Math.round(m.verify.confirm_rate*100)+"%":null)):"", m.timing_s?chip("scan time", m.timing_s.total+"s"):"", ].join("") : ''; @@ -868,13 +885,50 @@

Full matrix

${head} // ---- scan (auto-advances to Review on done) ---- let SCAN_N = 0; // lines already appended — the server streams the transcript from here scanLog.addEventListener("scroll",()=>{ scanFollow.style.display = atBottom(scanLog) ? "none" : "inline-block"; }); -async function runScan(){ const repo=scanRepo.value.trim(), cve=scanCve.value.trim(), spec=scanSpec.value.trim(); - // repo and --cve are alternatives; a spec is additive. The server enforces this too — this is - // just a faster no. - if(!repo && !cve && !spec){alert("enter a repo path, a CVE/GHSA id, or an OpenAPI spec");return;} - if(cve && (repo || spec)){alert("a CVE scan cannot be combined with a repo or a spec");return;} +// H2 — the manifest field is a comma-separated list, so one scan can cover a service's +// requirements.txt and its package-lock.json together. +function manifestPaths(){ return scanManifest.value.split(",").map(s=>s.trim()).filter(Boolean); } +async function surveyDeps(){ + // The read-only half: same module function the CLI's `vpcopilot deps` calls, so the preview and + // the scan can never disagree about which advisories are in scope. + const manifest=manifestPaths(); + if(!manifest.length){alert("enter at least one manifest path");return;} + depsPreview.style.display="block"; depsPreview.innerHTML='

asking OSV…

'; + let r; try { r=await jpost("/api/deps",{manifest,min_severity:scanMinSeverity.value,max_advisories:advCap(scanMaxAdvisories.value),include_dev:scanIncludeDev.checked}); } + catch(e){ depsPreview.innerHTML='

'+esc(e.message)+'

'; return; } + depsPreview.innerHTML=depsTable(r.dependencies); +} +function depsTable(d){ + if(!d) return '

no dependency data

'; + const f=d.funnel; + const chips=[["packages",f.packages_parsed],["vulnerable",f.packages_vulnerable],["advisories",f.advisories_distinct], + ["would resolve",f.would_resolve||f.resolved],["not resolved",f.not_resolved],["could not pin",f.packages_unpinned]] + .map(([k,v])=>''+k+': '+v+'').join(""); + const rows=(d.advisories||[]).slice(0,60).map(a=>` + + + + + `).join(""); + // The unpinned list is shown, not hidden: "we did not check this" must never look like "clean". + const un=(d.unpinned||[]).length? `

${d.unpinned.length} entry(ies) could NOT be pinned and were not checked: `+ + esc([...new Set(d.unpinned.map(u=>u.reason))].join(", "))+'

' : ""; + // A manifest that could not be READ at all returned an all-zero funnel and an empty table — + // visually identical to previewing a genuinely clean one. The CLI printed these in red all + // along; the console dropped them, so the guard lived on only one surface. + const errs=(d.errors||[]).map(e=>`

⚠ ${esc(e.path)}: ${esc(e.error)}

`).join(""); + const unck=(d.unchecked||[]).map(u=>`

could not check ${esc(u.ecosystem)}/${esc(u.name)} `+ + `${esc(u.version)} — ${esc(u.error)}; its advisories are UNKNOWN, not absent

`).join(""); + const more=(d.advisories||[]).length>60?`

… ${d.advisories.length-60} more in dependencies.json

`:""; + return `
${chips}
${errs}${unck}${un}
${esc(a.severity)}${esc(a.package)}${esc(a.installed)}${esc(a.advisory_id)}${a.fixed_version?esc(a.fixed_version):'none published'}${esc(a.disposition)}${a.reason?` — ${esc(String(a.reason).slice(0,90))}`:""}
${rows}
sevpackageinstalledadvisoryfixeddisposition
${more}`; +} +async function runScan(){ const repo=scanRepo.value.trim(), cve=scanCve.value.trim(), spec=scanSpec.value.trim(), manifest=manifestPaths(); + // repo and --cve are alternatives; a spec and a manifest are additive. The server enforces this + // too — this is just a faster no. + if(!repo && !cve && !spec && !manifest.length){alert("enter a repo path, a CVE/GHSA id, an OpenAPI spec, or a dependency manifest");return;} + if(cve && (repo || spec || manifest.length)){alert("a CVE scan cannot be combined with a repo, a spec or a manifest");return;} scanState.textContent="starting…"; - try { await jpost("/api/scan",{repo,cve,spec,out:scanOut.value.trim()||"out",min_confidence:parseFloat(scanMinConf.value)||0.5,max_files:parseInt(scanMaxFiles.value)||200,max_bytes:parseInt(scanMaxBytes.value)||60000,draft_code_fixes:scanRemediate.checked}); } catch(e){ scanState.textContent="error: "+e.message; return; } + try { await jpost("/api/scan",{repo,cve,spec,manifest,min_severity:scanMinSeverity.value,max_advisories:advCap(scanMaxAdvisories.value),include_dev:scanIncludeDev.checked,out:scanOut.value.trim()||"out",min_confidence:parseFloat(scanMinConf.value)||0.5,max_files:parseInt(scanMaxFiles.value)||200,max_bytes:parseInt(scanMaxBytes.value)||60000,draft_code_fixes:scanRemediate.checked}); } catch(e){ scanState.textContent="error: "+e.message; return; } SCAN_N=0; scanLog.textContent=""; scanLogCount.textContent=""; scanLogWrap.style.display="block"; const poll=setInterval(async()=>{ let s; try { s=await jget("/api/scan?since="+SCAN_N); } catch(e){ return; } // transient fetch blip — keep polling diff --git a/src/vpcopilot/export.py b/src/vpcopilot/export.py index 3f51301..20d10ff 100644 --- a/src/vpcopilot/export.py +++ b/src/vpcopilot/export.py @@ -70,7 +70,10 @@ def _quiet(_msg: str) -> None: BUNDLE_FILES = ["run.json", "audit.log", "ledger.json", "findings.json", "triage.json", "policies.json", "remediations.json", "summary.json", "metrics.json", "probes.json", "correlations.json", "lb_snapshot.json", "simulation.json", - "report.html"] + # H2. Named `dependencies.json` and NOT `manifest.json`: `verify_bundle` locates + # each run in an archive by every member whose name ends `manifest.json`, so an + # artifact by that name would be read as a second evidence manifest. + "dependencies.json", "report.html"] def _rj(out: Path, name: str, default): @@ -217,6 +220,12 @@ def build_manifest(out_dir: str = "out", *, members: dict | None = None) -> dict "simulation.json, when present, describes ONE recorded sample replayed through a spare " "load balancer — not production traffic in general. Its records are redacted at ingest " "(see `redacted`); read the window and record count with the rate.", + "dependencies.json, when present, is NOT a clean bill of health for the dependency " + "tree. Its `unpinned` list is manifest entries whose exact version could not be " + "established, which were therefore never queried — nothing is known about them. Its " + "`advisories` include entries dispositioned below_severity or capped: found and " + "listed, but never sent to an agent, so no band-aid was considered for them. Read " + "`funnel` for the counts, and note it reflects OSV.dev on the run date only.", ], "members": members or {}, } diff --git a/src/vpcopilot/impact.py b/src/vpcopilot/impact.py index f5940f1..90b4095 100644 --- a/src/vpcopilot/impact.py +++ b/src/vpcopilot/impact.py @@ -73,6 +73,9 @@ def impact(out_dir: str) -> dict: "remediated": counts["remediated"] + counts["retired"], "retired": counts["retired"], "code_prs": len(summary.get("code_fix_prs", []) or []), + # H2: upgrades are cures we CANNOT open a PR for. Counted separately so the hero panel + # never claims a drafted PR that does not exist. + "dependency_upgrades": len(summary.get("dependency_upgrades", []) or []), "change_control_days": days, "mttm_seconds": mttm, "controls_live": controls, diff --git a/src/vpcopilot/inputs/deps.py b/src/vpcopilot/inputs/deps.py new file mode 100644 index 0000000..a31401d --- /dev/null +++ b/src/vpcopilot/inputs/deps.py @@ -0,0 +1,389 @@ +"""H2 — dependency manifests in, one band-aid per exploitable advisory out. + +H1 answers "here is a CVE, can a load balancer hold the line until the upgrade ships". H2 asks the +same question of every dependency you actually have installed, which is the form the question +normally arrives in: nobody hands you a CVE id, they hand you a `requirements.txt` and a scanner +report. The stages after this one are H1's, unchanged — the resolve agent, the deterministic +`no_bandaid` route for anything not observable in a request, and the OSV-sourced upgrade target. + +What is genuinely new is that the input is now **unbounded**, and that changes what correctness +means. `aiohttp` pinned at 3.9.1 alone returns 70 OSV records from one query. A modest manifest +reaches several hundred. Three consequences, all measured against the live API on 2026-07-29: + +* **Deduplication is correctness, not thrift.** Those 70 records are 35 advisories — OSV publishes a + GHSA and a PYSEC entry for each, cross-linked by `aliases` and carrying different ids. Resolving + both means two model calls and two band-aids for one hole, and the duplicate does not look like + one. `osv.alias_groups` collapses them before anything is spent. +* **The expensive stage has to be bounded, and the listing must not be.** Every parsed package and + every advisory found is written to `dependencies.json` whatever happens next — that is the + acceptance criterion, and it costs one HTTP request per vulnerable package. Only the *agent* stage + is filtered, by severity and by an explicit cap, and every advisory that filtering held back is + written out carrying the reason it was held back. "We did not look at this" and "this is fine" + must never render the same way. +* **A skipped package is a reported package.** A version this tool cannot pin is never guessed at — + see `manifest.py` for what OSV does with a version string it cannot parse — so `skipped` is part + of the answer, not debris from producing it. +""" +from __future__ import annotations + +from concurrent.futures import ThreadPoolExecutor +from typing import Callable + +from ..schemas import Finding, RemediationPlan, TriageDecision +from . import manifest, osv +from .cve import CWE_CLASS + +SEVERITIES = ("critical", "high", "medium", "low") +DEFAULT_MIN_SEVERITY = "high" +DEFAULT_MAX_ADVISORIES = 25 + + +def _vuln_class(advisory: dict, fallback: str) -> str: + for cwe in advisory.get("cwe_ids") or []: + if cwe.upper() in CWE_CLASS: + return CWE_CLASS[cwe.upper()] + return fallback + + +def survey(paths: list[str], *, include_dev: bool = False, log: Callable = print) -> dict: + """Parse and resolve, with no model and no judgement — the read-only half of H2. + + Everything here is code and HTTP: which packages are installed, which of them OSV knows an + advisory for, how bad each one is and what version fixes it. This is what `vpcopilot deps` + prints and what `run_pipeline` hands to the agent stage, and it is deliberately runnable on its + own: it needs no model, no credentials and no tenant.""" + parsed = manifest.parse_all(paths) + pkgs = parsed["packages"] + for e in parsed["errors"]: + log(f" ⚠ {e['path']}: {e['error']}") + considered = [p for p in pkgs if include_dev or p["scope"] == "runtime"] + excluded = [p for p in pkgs if p not in considered] + log(f"parsed {len(pkgs)} pinned package(s) from {len(parsed['files'])} manifest(s); " + f"{len(parsed['skipped'])} entry(ies) could not be pinned") + if excluded: + log(f" {len(excluded)} dev/test-scoped package(s) excluded (--include-dev to include them)") + + hits: dict[str, list[str]] = {} + if considered: + log(f"asking OSV about {len(considered)} coordinate(s) in one batch query…") + hits = osv.query_batch(considered, log=log) + vulnerable = [p for p in considered if hits.get(osv._coord_key(p))] + log(f" {len(vulnerable)} package(s) have at least one advisory") + + records: list[dict] = [] + per_package: dict[str, list[dict]] = {} + for p in vulnerable: + try: + recs = osv.query_package(p, log=log) + except Exception as e: # noqa: BLE001 — one package's failure is not the manifest's + log(f" ⚠ could not read advisories for {osv._coord_key(p)}: {e}") + p["error"] = str(e) + continue + per_package[osv._coord_key(p)] = recs + records += recs + + # One candidate per (advisory, package): two packages hit by one advisory are two upgrades + # somebody has to ship. The resolve agent still runs once per advisory — see `resolve_all`. + candidates: list[dict] = [] + by_group: dict[str, list[dict]] = {} + for coord, recs in per_package.items(): + pkg = next(p for p in vulnerable if osv._coord_key(p) == coord) + for group in osv.alias_groups(recs): + rec = osv.preferred_record(group) + adv = osv.normalize_record(rec, group) + by_group.setdefault(adv["id"], group) + target = osv.upgrade_target_for(adv, pkg["name"], pkg["ecosystem"], pkg["version"]) + candidates.append({ + "advisory_id": adv["id"], "aliases": adv["aliases"], "advisory": adv, + "package": pkg["name"], "ecosystem": pkg["ecosystem"], + "installed": pkg["version"], "scope": pkg["scope"], "manifest": pkg["source"], + "summary": adv["summary"], "cvss": adv["cvss"], + "severity": osv.severity_from_cvss(adv["cvss"]), + "withdrawn": adv.get("withdrawn") or "", + "fixed_version": target["fixed_version"], "fix_note": target["note"], + "vulnerable_range": target["vulnerable_range"], + "references": adv["references"][:4], + "disposition": "", "reason": "", + }) + candidates.sort(key=lambda c: (osv.SEVERITY_RANK.get(c["severity"], 9), c["package"], + c["advisory_id"])) + log(f" {len(records)} OSV record(s) → {len({c['advisory_id'] for c in candidates})} distinct " + f"advisory(ies) across {len({c['package'] for c in candidates})} package(s)") + return {"files": parsed["files"], "errors": parsed["errors"], "packages": pkgs, + "considered": considered, "dev_excluded": excluded, "skipped": parsed["skipped"], + "vulnerable": vulnerable, "candidates": candidates, "_groups": by_group, + "records": len(records)} + + +def select(candidates: list[dict], *, min_severity: str = DEFAULT_MIN_SEVERITY, + max_advisories: int = DEFAULT_MAX_ADVISORIES, log: Callable = print) -> list[dict]: + """Decide which advisories reach the agent, stamping every one that does not with why. + + The filters are ordered so the stamped reason is the *first* thing that disqualified it, which + is the one a reader needs. Nothing is dropped from the list — `disposition` is set in place, and + `dependencies.json` carries all of it. + + **The cap is shared out across packages, not consumed in sort order.** The first live run made + the reason obvious: `aiohttp 3.9.1` alone carries 35 advisories, and a flat + `(severity, package, id)` ordering handed it 23 of the 25 slots — every other package in the + manifest was competing with one dependency's back catalogue for the remainder, and a package + sorting later in the alphabet would have been capped out by advisories no worse than its own. + So the budget is dealt round-robin: every vulnerable package gives up its worst advisory before + any package gives up its second. Depth on one dependency is never worth breadth across the tree. + """ + floor = osv.SEVERITY_RANK.get(min_severity, 1) + eligible: list[dict] = [] + for c in candidates: + if c["withdrawn"]: + c["disposition"], c["reason"] = "withdrawn", f"OSV withdrew this advisory ({c['withdrawn']})" + elif osv.SEVERITY_RANK.get(c["severity"], 9) > floor: + c["disposition"] = "below_severity" + c["reason"] = (f"{c['severity']} is below --min-severity {min_severity}; listed here, " + "not resolved") + else: + eligible.append(c) + + by_pkg: dict[str, list[dict]] = {} + for c in eligible: # `candidates` arrives severity-sorted, so each queue is + by_pkg.setdefault(c["package"], []).append(c) # already worst-first within its package + chosen: list[dict] = [] + queues = list(by_pkg.values()) + while queues and (not max_advisories or len(chosen) < max_advisories): + for q in list(queues): + if max_advisories and len(chosen) >= max_advisories: + break + chosen.append(q.pop(0)) + if not q: + queues.remove(q) + chosen.sort(key=lambda c: (osv.SEVERITY_RANK.get(c["severity"], 9), c["package"], + c["advisory_id"])) + picked = {id(c) for c in chosen} # identity, not equality: dicts compare by value + for c in eligible: + if id(c) not in picked: + c["disposition"] = "capped" + c["reason"] = (f"--max-advisories {max_advisories} reached; {c['package']} had " + f"{len(by_pkg[c['package']])} eligible advisory(ies) and the cap is " + "shared across packages. Listed here, not resolved") + for name, label in (("withdrawn", "withdrawn"), ("below_severity", f"below {min_severity}"), + ("capped", "over the cap")): + n = sum(1 for c in candidates if c["disposition"] == name) + if n: + log(f" {n} advisory(ies) {label} — listed in dependencies.json, not resolved") + return chosen + + +def resolve_all(h, chosen: list[dict], groups: dict, *, concurrency: int = 8, + log: Callable = print) -> None: + """Run the resolve agent once per distinct advisory and stamp the outcome on each candidate. + + Once per *advisory*, not per candidate: the profile answers "what does this vulnerability look + like in a request", which does not depend on which of the affected packages you happen to have + installed. Two packages hit by one advisory therefore cost one model call, and — more + importantly — cannot receive two different verdicts.""" + wanted = list(dict.fromkeys(c["advisory_id"] for c in chosen)) + if not wanted: + return + log(f"resolving {len(wanted)} advisory(ies) with the resolve agent " + f"({len(chosen)} package/advisory pair(s))…") + adv_by_id = {c["advisory_id"]: c["advisory"] for c in chosen} + profiles: dict[str, object] = {} + + def _one(aid): + from ..agents import resolve as resolve_agent + try: + return aid, resolve_agent.run(h, adv_by_id[aid]), "" + except Exception as e: # noqa: BLE001 — one advisory must not end the manifest run + return aid, None, str(e) + + with ThreadPoolExecutor(max_workers=max(1, concurrency)) as ex: + for aid, prof, err in ex.map(_one, wanted): + profiles[aid] = prof + if err: + log(f" ⚠ resolve failed for {aid}: {err} — no band-aid proposed for it") + for c in chosen: + if c["advisory_id"] == aid: + c["disposition"], c["reason"] = "resolve_failed", err + for c in chosen: + prof = profiles.get(c["advisory_id"]) + if prof is None: + continue + c["profile"] = prof + c["disposition"] = "exploitable" if prof.network_observable else "declined" + c["reason"] = prof.reason + c["confidence"] = round(float(prof.confidence), 2) + c["paths"] = list(prof.paths) + n_ok = sum(1 for c in chosen if c["disposition"] == "exploitable") + log(f" {n_ok} exploitable over HTTP, {sum(1 for c in chosen if c['disposition'] == 'declined')} " + "declined (nothing a load balancer can see)") + + +def _finding(c: dict, used: set[str]) -> Finding: + prof = c["profile"] + fid = c["advisory_id"] + n = 1 + while fid in used: # one advisory hitting two installed packages is two upgrades to ship + n += 1 + fid = f"{c['advisory_id']}-{n}" + used.add(fid) + coord = f"{c['ecosystem']}/{c['package']}@{c['installed']}" + return Finding( + id=fid, + title=f"{c['advisory_id']} in {c['package']} {c['installed']}: " + f"{c['summary'] or prof.title}"[:300], + vuln_class=_vuln_class(c["advisory"], prof.vuln_class.value), + severity=c["severity"], + # No file, no line: the vulnerable code is in a package we do not own. `source` carries the + # identity `file` carries for a repo finding, and it names the *installed coordinate* rather + # than just the advisory — two packages hit by one advisory must not collide. + source=f"osv:{c['advisory_id']}@{coord}", + endpoint=(prof.paths[0] if prof.paths else ""), + http_method=(prof.http_methods[0] if prof.http_methods else ""), + description=(f"{coord} (from {c['manifest']}) is affected by {c['advisory_id']}. " + + (c["advisory"].get("details") or prof.description))[:4000], + exploit_sketch=(prof.exploit_sketch if prof.network_observable + else f"no network-observable exploitation pattern: {prof.reason}"), + ) + + +def _remediation(c: dict, fid: str) -> RemediationPlan: + """The cure, built from OSV. `remediate` is never called on this path — see `inputs/cve.py`.""" + fixed, coord = c["fixed_version"], f"{c['package']} {c['installed']}" + lines = [f"## {c['advisory_id']}", "", c["summary"] or "", "", + f"`{c['ecosystem']}/{c['package']}` is pinned at **{c['installed']}** in " + f"`{c['manifest']}`.", ""] + if fixed: + lines += [f"**Upgrade to {fixed}.**", f"Vulnerable from: {c['vulnerable_range']}", ""] + else: + lines += [f"**No upgrade available.** {c['fix_note']}", ""] + prof = c.get("profile") + if prof is not None: + lines += ["### Exploitation", "", + prof.exploit_sketch if prof.network_observable + else f"Not observable in an HTTP request. {prof.reason}", ""] + if c["references"]: + lines += ["### References", ""] + [f"- {u}" for u in c["references"]] + lines += ["", "_This is a dependency upgrade recommendation, not a patch — the fix lives in " + "the upstream package._"] + return RemediationPlan( + finding_id=fid, kind="dependency_upgrade", + summary=(f"upgrade {c['package']} {c['installed']} → {fixed} (fixes {c['advisory_id']})" + if fixed else f"{c['advisory_id']} in {coord}: {c['fix_note']}"), + package=c["package"], ecosystem=c["ecosystem"], + vulnerable_range=c["vulnerable_range"], fixed_version=fixed, + pr_title=(f"fix({c['package']}): upgrade to {fixed} for {c['advisory_id']}" if fixed + else f"chore: mitigate {c['advisory_id']} in {c['package']}"), + pr_body="\n".join(lines)) + + +def build(chosen: list[dict]) -> dict: + """Resolved candidates → the findings, forced decisions and remediations the pipeline consumes. + + A declined advisory becomes a `no_bandaid` decision **here, in code**, exactly as H1 does it: a + load balancer cannot mitigate what it cannot see in a request, and the acceptance criterion for + that must not depend on a model honouring a prompt.""" + findings: list[Finding] = [] + decisions: dict[str, TriageDecision] = {} + remediations: dict[str, RemediationPlan] = {} + used: set[str] = set() + for c in chosen: + if c["disposition"] not in ("exploitable", "declined"): + continue + f = _finding(c, used) + c["finding_id"] = f.id + findings.append(f) + remediations[f.id] = _remediation(c, f.id) + if c["disposition"] == "declined": + fix = (f"upgrade {c['package']} to {c['fixed_version']}" if c["fixed_version"] + else f"no fixed version published ({c['fix_note']})") + decisions[f.id] = TriageDecision( + finding_id=f.id, bandaids=[], no_bandaid=True, + residual_risk=f"{c['reason']} No band-aid can mitigate this at the load balancer; " + f"{fix}.") + return {"findings": findings, "decisions": decisions, "remediations": remediations} + + +def report(surveyed: dict, chosen: list[dict], *, min_severity: str, max_advisories: int, + include_dev: bool) -> dict: + """`dependencies.json` — the whole answer, including the parts that cost nothing. + + Every package parsed, every entry refused and why, every advisory found and what became of it. + The acceptance asks for "affected packages, resolved advisories, and one band-aid per + exploitable advisory"; the first two are complete here whatever the agent stage did or did not + reach, so a bounded run still says truthfully what it saw.""" + cands = surveyed["candidates"] + counts = {d: sum(1 for c in cands if c["disposition"] == d) + for d in ("exploitable", "declined", "below_severity", "capped", "withdrawn", + "resolve_failed", "would_resolve", "")} + # A package OSV could not be asked about — the batch said it has advisories and the follow-up + # query failed, or the coordinate could not be checked at all. It was previously visible only + # as an `error` key buried in `packages[]`: it contributed no advisory rows, no funnel counter + # and no rendered warning, so it read exactly like a package that came back clean. That is the + # one confusion this whole module is built to prevent, so it gets its own list and counter. + unchecked = [{"ecosystem": p.get("ecosystem", ""), "name": p.get("name", ""), + "version": p.get("version", ""), "error": p["error"]} + for p in surveyed["packages"] if p.get("error")] + return { + "manifests": surveyed["files"], + "errors": surveyed["errors"], + "unchecked": unchecked, + "settings": {"min_severity": min_severity, "max_advisories": max_advisories, + "include_dev": include_dev}, + "funnel": { + "packages_parsed": len(surveyed["packages"]), + "packages_unpinned": len(surveyed["skipped"]), + "packages_dev_excluded": len(surveyed["dev_excluded"]), + "packages_queried": len(surveyed["considered"]), + "packages_unchecked": len(unchecked), + "packages_vulnerable": len(surveyed["vulnerable"]), + "osv_records": surveyed["records"], + "advisories_distinct": len({c["advisory_id"] for c in cands}), + "package_advisory_pairs": len(cands), + "resolved": len(chosen), + # `would_resolve` is the read-only surface's answer: `vpcopilot deps` reaches the same + # verdict about WHICH advisories a scan would spend a model call on without spending one. + "would_resolve": counts["would_resolve"], + "exploitable": counts["exploitable"], + "declined": counts["declined"], + "resolve_failed": counts["resolve_failed"], + "packages_with_a_resolved_advisory": len({c["package"] for c in chosen}), + "not_resolved": counts["below_severity"] + counts["capped"] + counts["withdrawn"], + }, + "packages": [{k: v for k, v in p.items() if k != "note"} | {"note": p.get("note", "")} + for p in surveyed["packages"]], + "unpinned": surveyed["skipped"], + "advisories": [{k: v for k, v in c.items() + if k not in ("advisory", "profile")} for c in cands], + } + + +def survey_report(paths: list[str], *, min_severity: str = DEFAULT_MIN_SEVERITY, + max_advisories: int = DEFAULT_MAX_ADVISORIES, include_dev: bool = False, + log: Callable = print) -> dict: + """Everything H2 can say without a model: `vpcopilot deps` and `GET /api/deps` both call this. + + Same shape as the `dependencies.json` a full scan writes, with each advisory that *would* be + resolved marked `would_resolve` instead of the verdict only the agent can give. Runs with no + model configured and no credentials of any kind — the parse-and-list half of the acceptance + criterion is answerable from OSV alone, and being able to see the funnel before paying for it + is what makes the severity floor and the cap tunable rather than mysterious.""" + surveyed = survey(paths, include_dev=include_dev, log=log) + chosen = select(surveyed["candidates"], min_severity=min_severity, + max_advisories=max_advisories, log=log) + for c in chosen: + c["disposition"] = c["disposition"] or "would_resolve" + return report(surveyed, chosen, min_severity=min_severity, max_advisories=max_advisories, + include_dev=include_dev) + + +def resolve_manifests(h, paths: list[str], *, min_severity: str = DEFAULT_MIN_SEVERITY, + max_advisories: int = DEFAULT_MAX_ADVISORIES, include_dev: bool = False, + concurrency: int = 8, log: Callable = print) -> dict: + """The whole of H2: manifests in, pipeline-shaped artifacts out.""" + surveyed = survey(paths, include_dev=include_dev, log=log) + chosen = select(surveyed["candidates"], min_severity=min_severity, + max_advisories=max_advisories, log=log) + resolve_all(h, chosen, surveyed["_groups"], concurrency=concurrency, log=log) + built = build(chosen) + return {**built, "report": report(surveyed, chosen, min_severity=min_severity, + max_advisories=max_advisories, include_dev=include_dev), + "candidates": surveyed["candidates"]} diff --git a/src/vpcopilot/inputs/manifest.py b/src/vpcopilot/inputs/manifest.py new file mode 100644 index 0000000..fa76639 --- /dev/null +++ b/src/vpcopilot/inputs/manifest.py @@ -0,0 +1,447 @@ +"""H2 — a dependency manifest in, a list of installed package coordinates out. + +Pure parsing. No network, no model, no judgement: this module turns `requirements.txt`, +`package-lock.json` and `pom.xml` into `(ecosystem, name, version)` triples that OSV can be asked +about, and an explicit list of the lines it *refused* to turn into one. + +**Why refusing matters more here than anywhere else in this codebase.** Querying OSV with a version +string it cannot parse does not error — measured against the live API on 2026-07-29, `aiohttp` at +version `not-a-version`, `1.0.0-SNAPSHOT` and `${project.version}` each returned **81 advisories**, +against 70 for the real `3.9.1` and 87 for no version at all. So an unresolved `${...}` property or +an unpinned `flask>=2.0` does not produce an error, or even an empty result — it produces a +confident, larger, wrong answer, and every downstream stage would treat it as fact. Every version +this module cannot establish is therefore dropped into `skipped` with a reason, and the reason is +carried all the way to the report. A dependency we cannot pin is a dependency we say nothing about. + +The three formats fail differently, and the parsers are shaped by how: + +* **requirements.txt** is a *request*, not a lockfile. Only `==` pins an exact version; `>=`, `~=` + and a bare name describe a range that some resolver will decide later. Those are skipped. +* **package-lock.json** is a true lockfile — every version is exact. Its trap is shape: v2 carries + BOTH the v3 `packages` map and the v1 `dependencies` tree describing the same installs, so a + parser reading both counts everything twice. +* **pom.xml** is a *template*. `${...}` properties, ``, and inherited `` + versions mean a version may simply not be knowable without resolving the whole Maven reactor, + which this does not do. Those are skipped by name. +""" +from __future__ import annotations + +import json +import re +import xml.etree.ElementTree as ET +from pathlib import Path + +# Ecosystem names are OSV's and are case-sensitive — `pypi`, `NPM` and `maven` are all HTTP 400s, +# and one rejected entry fails a whole `querybatch`. The set OSV accepts lives in `osv.py`; these +# parsers emit only the three it is certain about. +MAX_BYTES = 8_000_000 # a manifest larger than this is not a manifest +_MAVEN_NS = re.compile(r"^\{[^}]+\}") +# A pinned requirement: name, optional [extras], '==', version. Anything else is a range. +_REQ_PIN = re.compile(r"^([A-Za-z0-9][A-Za-z0-9._-]*)\s*(\[[^\]]*\])?\s*==\s*([^\s;#\\]+)") +_REQ_NAME = re.compile(r"^([A-Za-z0-9][A-Za-z0-9._-]*)") +_UNRESOLVED = re.compile(r"\$\{[^}]*\}") + + +class ManifestError(RuntimeError): + """The file is not a manifest of the kind claimed. Distinct from 'parsed, but nothing usable'.""" + + +def normalize_pypi(name: str) -> str: + """PEP 503: runs of `-`, `_` and `.` collapse to a single `-`, lowercased. `Flask_Login` and + `flask-login` are one package, and counting them twice would resolve the same advisories twice.""" + return re.sub(r"[-_.]+", "-", name).lower() + + +def detect_kind(path: str | Path) -> str: + """`pip` | `npm` | `maven`, by filename first and content second. + + Filename is decisive when it is one of the three canonical names. Everything else is sniffed, + because `requirements-dev.txt`, `reqs.txt` and `deps/prod.txt` are all real, and refusing them + for not being spelled right would be pedantry.""" + p = Path(path) + n = p.name.lower() + if n == "package-lock.json" or n == "npm-shrinkwrap.json": + return "npm" + if n == "pom.xml": + return "maven" + if n.startswith("requirements") and n.endswith(".txt"): + return "pip" + try: + head = p.read_text(errors="replace")[:4000].lstrip() + except OSError as e: + raise ManifestError(f"cannot read {path}: {e}") from e + if head.startswith("{"): + if '"lockfileVersion"' in head or '"packages"' in head or '"dependencies"' in head: + return "npm" + raise ManifestError(f"{p.name} is JSON but has no lockfileVersion/packages/dependencies — " + "is it a package-lock.json?") + if head.startswith(" str: + if not path.is_file(): + raise ManifestError(f"no such manifest: {path}") + if path.stat().st_size > MAX_BYTES: + raise ManifestError(f"{path.name} is {path.stat().st_size} bytes — refusing to parse a " + f"manifest over {MAX_BYTES}") + return path.read_text(errors="replace") + + +# --------------------------------------------------------------------------- requirements.txt +def _parse_requirements(path: Path, *, _seen: set[str] | None = None, _depth: int = 0) -> tuple[list, list]: + """`-r other.txt` includes are followed (depth-bounded, cycle-guarded) because splitting + requirements across files is the norm, and a scan that silently ignored `-r base.txt` would + report on a fraction of the tree while looking complete.""" + seen = _seen if _seen is not None else set() + pkgs: list[dict] = [] + skipped: list[dict] = [] + try: + resolved = str(path.resolve()) + except OSError: + resolved = str(path) + if resolved in seen: + return pkgs, skipped + seen.add(resolved) + + text = _read(path) + # Join backslash continuations first so `pkg==1.0 \\\n --hash=...` is one logical line. + logical, buf, start = [], "", 1 + for i, raw in enumerate(text.splitlines(), start=1): + if not buf: + start = i + if raw.rstrip().endswith("\\"): + buf += raw.rstrip()[:-1] + " " + continue + logical.append((start, buf + raw)) + buf = "" + if buf: + logical.append((start, buf)) + + for lineno, raw in logical: + line = raw.split(" #", 1)[0].strip() + if line.startswith("#"): + line = "" + if not line: + continue + low = line.lower() + if low.startswith(("-r ", "--requirement ", "-r", "--requirement=")): + target = re.sub(r"^(-r|--requirement)\s*=?\s*", "", line, flags=re.I).strip() + if _depth >= 3: + skipped.append({"raw": line, "line": lineno, "reason": "include-too-deep", + "detail": "-r include nesting deeper than 3 levels"}) + continue + inc = (path.parent / target) + try: + sub_p, sub_s = _parse_requirements(inc, _seen=seen, _depth=_depth + 1) + pkgs += sub_p + skipped += sub_s + except ManifestError as e: + skipped.append({"raw": line, "line": lineno, "reason": "include-unreadable", + "detail": str(e)}) + continue + if line.startswith("-"): # -e, -c, --index-url, --hash on its own line, … + reason = "editable" if low.startswith(("-e", "--editable")) else "pip-option" + skipped.append({"raw": line, "line": lineno, "reason": reason, + "detail": "not a version-pinned requirement"}) + continue + # Strip an environment marker; it gates whether the package is installed, not which version. + marker = "" + if ";" in line: + line, marker = line.split(";", 1) + line, marker = line.strip(), marker.strip() + if "@" in line and not line.startswith("@"): # `pkg @ https://…` direct reference + skipped.append({"raw": line, "line": lineno, "reason": "direct-reference", + "detail": "installed from a URL/VCS, not a released version"}) + continue + m = _REQ_PIN.match(line) + if not m: + nm = _REQ_NAME.match(line) + skipped.append({"raw": line, "line": lineno, "reason": "unpinned", + "detail": f"'{(nm.group(1) if nm else line)[:60]}' is not pinned with " + "'==' — the installed version is decided by a resolver, and " + "guessing one would query OSV for the wrong thing"}) + continue + version = m.group(3).strip() + if version.endswith(".*") or _UNRESOLVED.search(version): + skipped.append({"raw": line, "line": lineno, "reason": "unpinned", + "detail": f"'{version}' is a wildcard, not an exact version"}) + continue + pkgs.append({"ecosystem": "PyPI", "name": normalize_pypi(m.group(1)), "version": version, + "scope": "runtime", "line": lineno, "source": path.name, + "note": f"marker: {marker}" if marker else ""}) + return pkgs, skipped + + +# --------------------------------------------------------------------------- package-lock.json +def _parse_package_lock(path: Path) -> tuple[list, list]: + text = _read(path) + try: + doc = json.loads(text) + except json.JSONDecodeError as e: + raise ManifestError(f"{path.name} is not valid JSON: {e}") from e + if not isinstance(doc, dict): + raise ManifestError(f"{path.name} is JSON but not an object — not a package-lock") + pkgs: list[dict] = [] + skipped: list[dict] = [] + lockver = doc.get("lockfileVersion") + + if isinstance(doc.get("packages"), dict): + # lockfileVersion 2 and 3. v2 ALSO carries the legacy `dependencies` tree describing the + # same installs; reading both would double-count every package, so `packages` wins and + # `dependencies` is not read at all when it is present. + for key, ent in doc["packages"].items(): + if not isinstance(ent, dict): + continue + if key == "": + continue # the root project itself, not a dependency + if ent.get("link"): + skipped.append({"raw": key, "line": 0, "reason": "workspace-link", + "detail": "a symlink to a local workspace package, not a registry release"}) + continue + if "node_modules/" not in key: + # v2/v3 `packages` keys are project-relative PATHS. Only `node_modules/...` are + # installed registry releases; a workspace's own source sits under its directory + # (`packages/internal-tools`). Emitting those as registry coordinates asks OSV + # about first-party code under a name that may collide with a public package — + # and where the entry carries no `name`, the coordinate became the directory + # path itself, which is not a legal npm name. + skipped.append({"raw": key, "line": 0, "reason": "workspace-source", + "detail": "a workspace's own source tree, not an installed " + "registry release — its dependencies are listed " + "separately under node_modules/"}) + continue + name = ent.get("name") or key.split("node_modules/")[-1] + version = (ent.get("version") or "").strip() + if not version: + skipped.append({"raw": key, "line": 0, "reason": "no-version", + "detail": "lock entry carries no version"}) + continue + if ent.get("resolved", "").startswith(("git+", "file:")) or version.startswith(("git+", "file:")): + skipped.append({"raw": key, "line": 0, "reason": "direct-reference", + "detail": "installed from git/file, not a registry release"}) + continue + pkgs.append({"ecosystem": "npm", "name": name, "version": version, + "scope": "dev" if (ent.get("dev") or ent.get("devOptional")) else "runtime", + "line": 0, "source": path.name, "note": ""}) + elif isinstance(doc.get("dependencies"), dict): + # A package.json ALSO carries a `dependencies` map, and `detect_kind` cannot tell the two + # apart by name when the file is called something else. The difference is the value: a + # lockfile maps name → {version, …}, a package.json maps name → RANGE STRING. Walking the + # latter matched nothing and produced zero packages, zero skips and no error — a file full + # of dependencies reading exactly like a clean lockfile, which is the one confusion this + # module exists to prevent. Refuse it instead: a package.json has no installed versions to + # check (the resolver picks them), so asking for the lock file is the honest answer. + dmap = doc["dependencies"] + if dmap and not any(isinstance(v, dict) for v in dmap.values()): + raise ManifestError( + f"{path.name} looks like a package.json, not a package-lock.json: its " + "'dependencies' map names version ranges, not installed versions. Point at " + "package-lock.json or npm-shrinkwrap.json — the versions actually installed, " + "which are the only ones OSV can be asked about, live only there.") + + def walk(tree: dict, depth: int = 0) -> None: + if depth > 24: + return + for name, ent in tree.items(): + if not isinstance(ent, dict): + # Never silently drop a named dependency: it would be indistinguishable from + # the file not naming it at all. + skipped.append({"raw": name, "line": 0, "reason": "not-a-lock-entry", + "detail": f"'{name}' maps to {type(ent).__name__}, not a lock " + "entry object carrying a version"}) + continue + version = (ent.get("version") or "").strip() + if version and not version.startswith(("git+", "file:")): + pkgs.append({"ecosystem": "npm", "name": name, "version": version, + "scope": "dev" if ent.get("dev") else "runtime", + "line": 0, "source": path.name, "note": ""}) + elif version: + skipped.append({"raw": name, "line": 0, "reason": "direct-reference", + "detail": "installed from git/file, not a registry release"}) + else: + # The v2/v3 branch above refuses a versionless entry explicitly; this one used + # to fall through both arms and drop it with no trace, so the same lockfile + # content was invisible risk in v1 and a reported refusal in v3. + skipped.append({"raw": name, "line": 0, "reason": "no-version", + "detail": "lock entry carries no version"}) + if isinstance(ent.get("dependencies"), dict): + walk(ent["dependencies"], depth + 1) + walk(doc["dependencies"]) + else: + raise ManifestError(f"{path.name} has neither a 'packages' nor a 'dependencies' map " + f"(lockfileVersion={lockver!r}) — not a package-lock") + return pkgs, skipped + + +# --------------------------------------------------------------------------- pom.xml +def _tag(el) -> str: + return _MAVEN_NS.sub("", el.tag) + + +def _child(el, name: str): + for c in el: + if _tag(c) == name: + return c + return None + + +def _text(el, name: str) -> str: + c = _child(el, name) + return (c.text or "").strip() if c is not None and c.text else "" + + +def _parse_pom(path: Path) -> tuple[list, list]: + text = _read(path) + # ElementTree resolves internal entities, so a pom carrying a DTD is a billion-laughs vector and + # `defusedxml` is not a dependency here. A dependency manifest has no business declaring one. + if re.search(r", not — not a pom") + + props: dict[str, str] = {} + pel = _child(root, "properties") + if pel is not None: + for c in pel: + props[_tag(c)] = (c.text or "").strip() + own_version = _text(root, "version") + parent = _child(root, "parent") + if not own_version and parent is not None: + own_version = _text(parent, "version") + for alias in ("project.version", "pom.version", "version"): + if own_version: + props.setdefault(alias, own_version) + + def expand(raw: str, depth: int = 0) -> str: + """Substitute `${…}` from , bounded so a self-referential property terminates.""" + v = raw + for _ in range(6): + m = _UNRESOLVED.search(v) + if not m: + return v.strip() + key = m.group(0)[2:-1] + if key not in props: + return v.strip() + v = v[:m.start()] + props[key] + v[m.end():] + return v.strip() + + # declares versions that entries may omit. It is a + # fallback, not a source of installs in its own right — a managed dependency nobody depends on + # is not on the classpath. + managed: dict[str, str] = {} + dm = _child(root, "dependencyManagement") + if dm is not None: + deps = _child(dm, "dependencies") + for d in (deps if deps is not None else []): + if _tag(d) != "dependency": + continue + g, a, v = _text(d, "groupId"), _text(d, "artifactId"), _text(d, "version") + if g and a and v: + managed[f"{expand(g)}:{expand(a)}"] = expand(v) + + pkgs: list[dict] = [] + skipped: list[dict] = [] + deps = _child(root, "dependencies") + for d in (deps if deps is not None else []): + if _tag(d) != "dependency": + continue + group, artifact = expand(_text(d, "groupId")), expand(_text(d, "artifactId")) + scope = (_text(d, "scope") or "compile").lower() + coord = f"{group}:{artifact}" + if not group or not artifact: + skipped.append({"raw": coord, "line": 0, "reason": "incomplete", + "detail": "dependency has no groupId or no artifactId"}) + continue + if _UNRESOLVED.search(coord): + # `expand()` runs on groupId/artifactId too, but only the version was tested. Neither + # `${project.groupId}` nor `${project.artifactId}` is ever in `props`, and anything + # inherited from a parent POM is absent as well — so a literal `${...}` reached OSV + # inside the coordinate NAME, came back empty, and was counted as checked-and-clean. + # `${project.groupId}` for intra-project dependencies is ordinary multi-module Maven. + skipped.append({"raw": coord, "line": 0, "reason": "unresolved-property", + "detail": f"coordinate is '{coord}' and no entry defines " + "it — asking OSV under this name would answer for nothing"}) + continue + version = expand(_text(d, "version")) or managed.get(coord, "") + if not version: + skipped.append({"raw": coord, "line": 0, "reason": "inherited-version", + "detail": "version comes from a parent POM or an imported BOM; " + "resolving it needs the Maven reactor, which this does not do"}) + continue + if _UNRESOLVED.search(version): + skipped.append({"raw": coord, "line": 0, "reason": "unresolved-property", + "detail": f"version is '{version}' and no entry defines it " + "— OSV would answer for a different version, not error"}) + continue + pkgs.append({"ecosystem": "Maven", "name": coord, "version": version, + "scope": "test" if scope in ("test", "provided") else "runtime", + "line": 0, "source": path.name, "note": f"scope: {scope}" if scope != "compile" else ""}) + return pkgs, skipped + + +_PARSERS = {"pip": _parse_requirements, "npm": _parse_package_lock, "maven": _parse_pom} + + +def parse(path: str | Path, kind: str = "") -> dict: + """One manifest → `{kind, path, packages, skipped}`. + + `packages` are coordinates OSV can be asked about. `skipped` is every line that named a + dependency but whose version could not be established, each with a machine-readable `reason` — + it is reported, never silently dropped, because "we did not check this one" and "this one is + clean" are the two answers that must never be confused.""" + p = Path(path) + kind = kind or detect_kind(p) + if kind not in _PARSERS: + raise ManifestError(f"unknown manifest kind '{kind}' — expected pip, npm or maven") + pkgs, skipped = _PARSERS[kind](p) + + # Collapse duplicates: a transitive dep pinned at one version appears in many package-lock + # subtrees, and the same (ecosystem, name, version) is one install to ask OSV about. + uniq: dict[tuple, dict] = {} + for e in pkgs: + key = (e["ecosystem"], e["name"], e["version"]) + if key in uniq: + # runtime beats dev/test: if anything reaches it at runtime, it is in the request path. + if e["scope"] == "runtime": + uniq[key]["scope"] = "runtime" + continue + uniq[key] = e + return {"kind": kind, "path": str(p), "packages": list(uniq.values()), "skipped": skipped} + + +def parse_all(paths: list[str]) -> dict: + """Several manifests → one merged coordinate set, plus a per-file record of what happened. + + A file that fails to parse does not abort the others: it lands in `errors` and the rest are + still resolved. One malformed pom must not cost a Python service its dependency scan.""" + packages: dict[tuple, dict] = {} + skipped: list[dict] = [] + files: list[dict] = [] + errors: list[dict] = [] + for raw in paths: + try: + res = parse(raw) + except ManifestError as e: + errors.append({"path": str(raw), "error": str(e)}) + files.append({"path": str(raw), "kind": "", "packages": 0, "skipped": 0, "error": str(e)}) + continue + for e in res["packages"]: + key = (e["ecosystem"], e["name"], e["version"]) + if key not in packages: + packages[key] = e + elif e["scope"] == "runtime": + packages[key]["scope"] = "runtime" + skipped += [{**s, "path": res["path"]} for s in res["skipped"]] + files.append({"path": res["path"], "kind": res["kind"], + "packages": len(res["packages"]), "skipped": len(res["skipped"]), "error": ""}) + return {"files": files, "packages": list(packages.values()), "skipped": skipped, "errors": errors} diff --git a/src/vpcopilot/inputs/osv.py b/src/vpcopilot/inputs/osv.py index 9708911..887a2fc 100644 --- a/src/vpcopilot/inputs/osv.py +++ b/src/vpcopilot/inputs/osv.py @@ -19,6 +19,22 @@ there is no package at all — only `database_specific.cpe`. The prose in `details` is the real payload, and it is what the resolve agent reasons over. +H2 asks a different question — *what is wrong with the packages I have installed* — and measuring +that against the live API on 2026-07-29 surfaced three more, all of which only bite at volume: + +4. **`querybatch` returns ids, not advisories.** One request covers 400 coordinates in 3.4s and its + ids match `/v1/query` exactly, so it is a faithful *discovery* pass — but the bodies are not in + it, and fetching each id would be one request per advisory (70 for `aiohttp 3.9.1` alone) where + `/v1/query` returns all of them for a coordinate in one. Batch to find the hits, query the hits. + One bad ecosystem also fails the **whole** batch with HTTP 400, so entries are validated first. +5. **Records are roughly two-to-one with advisories.** OSV publishes a GHSA *and* a PYSEC entry for + most Python advisories, cross-linked by `aliases` — 70 records for `aiohttp 3.9.1` are 35 + advisories. Collapsing them is correctness, not thrift: the duplicate carries a different id, so + nothing downstream would recognise it as one. +6. **CVSS v4 is now the majority.** 43 of those 70 severity scores are `CVSS_V4` against 32 + `CVSS_V3`, and 38 records carry a v4 vector and nothing else — every one of which a v3-only + parser buckets as `medium` whatever it says. See `severity_from_cvss`. + Read-only and cached on disk: an advisory is immutable enough for a run, and a demo should not depend on the network being up. """ @@ -31,8 +47,12 @@ from typing import Callable API = "https://api.osv.dev/v1/vulns" +QUERY_API = "https://api.osv.dev/v1/query" +BATCH_API = "https://api.osv.dev/v1/querybatch" CACHE_ENV = "VPCOPILOT_ADVISORY_CACHE" TIMEOUT = 20 +BATCH_TIMEOUT = 90 +BATCH_CHUNK = 250 # 400 in one request answered in 3.4s live; 250 keeps a chunk well inside it # A 40- (or 7+) character hex string in a `fixed` event is a commit, not something anyone can # `pip install`. Offering it as an upgrade target is worse than admitting there isn't one. @@ -182,18 +202,36 @@ def resolve(advisory_id: str, *, follow_aliases: bool = True, log: Callable = pr } +# CVSS v4 renamed the vulnerable-system impact metrics. Mapping them onto the v3 names means ONE +# bucketing rule serves both vectors and the two can never drift apart. +_CVSS4_ALIAS = {"VC": "C", "VI": "I", "VA": "A"} + + def severity_from_cvss(cvss: str) -> str: """CVSS vector → the project's four-level Severity. Absent scores are `medium`, never - `critical` — an unknown severity must not jump the queue ahead of a measured one.""" - m = re.search(r"CVSS:3\.[01]/(.+)", cvss or "") + `critical` — an unknown severity must not jump the queue ahead of a measured one. + + **v4 is not optional any more.** H1 only ever met v3 vectors, so the v3-only regex looked + complete. Measured against live OSV on 2026-07-29, `aiohttp 3.9.1` returns 70 advisories + carrying 43 CVSS_V4 scores against 32 CVSS_V3, and **38 of the 70 records publish a v4 vector + and nothing else** — every one of which fell through to the `medium` default regardless of what + it actually said, including `AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H` (an unauthenticated remote + denial of service). One CVE at a time that is a cosmetic mislabel; H2 filters hundreds of + advisories on this value, so it decides what gets resolved at all.""" + m = re.search(r"CVSS:(3\.[01]|4\.0)/(.+)", cvss or "") if not m: return "medium" - parts = dict(p.split(":", 1) for p in m.group(1).split("/") if ":" in p) + parts = dict(p.split(":", 1) for p in m.group(2).split("/") if ":" in p) + if m.group(1) == "4.0": + parts = {_CVSS4_ALIAS.get(k, k): v for k, v in parts.items()} # Approximate the base score from the impact/exploitability metrics that dominate it. This is a # bucketing, not a CVSS implementation — OSV rarely publishes the numeric score. high_impact = sum(1 for k in ("C", "I", "A") if parts.get(k) == "H") net = parts.get("AV") == "N" - easy = parts.get("AC") == "L" and parts.get("PR") == "N" and parts.get("UI") == "N" + # v4 splits v3's single AC into AC (complexity) and AT (attack requirements). A v3 vector has no + # AT, so defaulting it to "N" leaves every v3 verdict byte-identical to before. + easy = (parts.get("AC") == "L" and parts.get("PR") == "N" and parts.get("UI") == "N" + and parts.get("AT", "N") == "N") if net and easy and high_impact >= 2: return "critical" if net and (high_impact >= 1 or easy): @@ -203,6 +241,308 @@ def severity_from_cvss(cvss: str) -> str: return "low" +SEVERITY_RANK = {"critical": 0, "high": 1, "medium": 2, "low": 3} +_VTOKEN = re.compile(r"(\d+|[a-zA-Z]+)") + + +def _version_key(v: str) -> list: + """A loose, ecosystem-agnostic ordering key. + + PyPI, npm and Maven each have their own version algebra and none of the three agrees with the + others; `packaging` implements PEP 440 only and raises on Maven's `2.0-beta9` or `1.2.1.2-jre17`. + Rather than pull in three comparators to answer one question — *is this fix newer than what I + have* — split into digit and letter runs and order those. Digits compare numerically (so + `1.10.0 > 1.9.2`, which a string sort gets backwards) and a letter run sorts BELOW a bare number + at the same position, so `2.0-beta9 < 2.0`.""" + return [(1, int(t), "") if t.isdigit() else (0, 0, t.lower()) for t in _VTOKEN.findall(str(v))] + + +def comparable(v: str) -> bool: + """A version with no digits in it cannot be ordered against anything meaningfully.""" + return any(t.isdigit() for t in _VTOKEN.findall(str(v or ""))) + + +def version_gt(a: str, b: str) -> bool: + """`a > b` under `_version_key`, padding the shorter with release-level zeros so a longer + version is not automatically greater — `2.0-beta9` must not outrank `2.0`.""" + ka, kb = _version_key(a), _version_key(b) + pad = (1, 0, "") + for x, y in zip(ka + [pad] * (len(kb) - len(ka)), kb + [pad] * (len(ka) - len(kb)), + strict=True): + if x != y: + return x > y + return False + + +# --------------------------------------------------------------------------- H2: package queries +def _coord_key(pkg: dict) -> str: + return f"{pkg['ecosystem']}/{pkg['name']}@{pkg['version']}" + + +def _fallback(pkgs: list[dict], out: dict[str, list[str]], log: Callable) -> None: + """Ask about each coordinate individually, and never let one failure end the run. + + `query_package` raises on any non-200 and lets httpx transport errors through. Unguarded, a + single 429 here propagated out of `query_batch`, out of `deps.survey` — whose own per-package + loop IS guarded — and out of `resolve_manifests`, which `pipeline` does not wrap. So the + fallback that exists so a batch failure never loses a chunk became the thing that lost the + whole scan, including the repo half of a `--manifest` run alongside a repo. The causes of a + batch failure (OSV down, network cut, rate limiting) are exactly the causes that make these + retries fail too, and the fallback issues up to `BATCH_CHUNK` sequential requests, which is + the traffic pattern most likely to draw a 429 in the first place. + + A coordinate we could not ask about is left OUT of `out` and stamped on the package dict, so + `deps.survey` reports it as unchecked. It must never read as "no advisories".""" + for p in pkgs: + try: + out[_coord_key(p)] = [v.get("id") for v in query_package(p, log=log) if v.get("id")] + except Exception as e: # noqa: BLE001 — one coordinate's failure is not the manifest's + log(f" ⚠ could not check {_coord_key(p)}: {e} — reported as unchecked, not clean") + p["error"] = str(e) + out.pop(_coord_key(p), None) + + +def query_batch(packages: list[dict], *, log: Callable = print) -> dict[str, list[str]]: + """`POST /v1/querybatch` — which of these coordinates has any advisory at all. + + **The batch endpoint returns ids and `modified` timestamps only, never the advisory bodies.** + That is not in the shape of the response you would guess from `/v1/query`, and it inverts the + obvious design: fetching each id would cost one request per *advisory* (70 for `aiohttp 3.9.1` + alone), where `/v1/query` returns every full record for a coordinate in a single round trip. + So batch is used for exactly what it is good at — one request that says which packages are worth + asking about — and the bodies come from `query_package` for the hits only. On a 100-package + manifest with 15 vulnerable that is 16 requests rather than 100. + + Measured live 2026-07-29: 400 queries answered in 3.4s, and the ids returned are identical to + `/v1/query` for the same coordinate across five packages, so nothing is truncated.""" + import httpx + out: dict[str, list[str]] = {} + # One invalid ecosystem fails the ENTIRE batch with HTTP 400 ("error in query at index 1"), so + # anything OSV would reject is dropped here rather than being allowed to cost every other + # package in the manifest its answer. + usable = [p for p in packages if p.get("ecosystem") in _KNOWN_ECOSYSTEMS and p.get("version")] + for p in packages: + if p not in usable: + out[_coord_key(p)] = [] + for i in range(0, len(usable), BATCH_CHUNK): + chunk = usable[i:i + BATCH_CHUNK] + body = {"queries": [{"package": {"name": p["name"], "ecosystem": p["ecosystem"]}, + "version": p["version"]} for p in chunk]} + try: + r = httpx.post(BATCH_API, json=body, timeout=BATCH_TIMEOUT) + r.raise_for_status() + results = r.json().get("results") or [] + except Exception as e: # noqa: BLE001 + # Degrade to per-package queries rather than losing the chunk: batch is an optimisation, + # and reporting "no advisories" because a request failed is the one answer that must + # never be produced by an error. + log(f" ⚠ OSV batch query failed ({e}) — falling back to one query per package") + _fallback(chunk, out, log) + continue + # `results` is positionally aligned with `queries`; a short list means OSV changed + # its contract, and pairing the wrong package with the wrong answer is the worst outcome. + for p, res in zip(chunk, results, strict=False): + out[_coord_key(p)] = [v.get("id") for v in (res.get("vulns") or []) if v.get("id")] + # `strict=False` above deliberately refuses to mispair, but that leaves the unanswered tail + # with no entry at all — and the caller reads a missing key as "no advisories" + # (`hits.get(key)` is None, so the package never reaches `vulnerable`). A short 200 would + # therefore report packages as CLEAN that OSV never answered for. Ask about those + # individually instead: slower, and the only answer that is not a guess. + unanswered = [p for p in chunk if _coord_key(p) not in out] + if unanswered: + log(f" ⚠ OSV batch answered {len(results)} of {len(chunk)} coordinate(s) — " + f"querying the remaining {len(unanswered)} individually rather than reading " + "them as clean") + _fallback(unanswered, out, log) + return out + + +def query_package(pkg: dict, *, log: Callable = print) -> list[dict]: + """`POST /v1/query` — every full advisory record for one exact coordinate. + + The version is sent because OSV does the range matching; asking without one returns the union + over all versions (87 for aiohttp against 70 for `3.9.1`). It is never sent unvalidated — see + `inputs/manifest.py` for why a version string OSV cannot parse is more dangerous than no version + at all.""" + import httpx + r = httpx.post(QUERY_API, json={"package": {"name": pkg["name"], "ecosystem": pkg["ecosystem"]}, + "version": pkg["version"]}, timeout=TIMEOUT) + if r.status_code != 200: + raise RuntimeError(f"OSV.dev returned {r.status_code} for " + f"{_coord_key(pkg)}: {r.text[:200]}") + vulns = r.json().get("vulns") or [] + for v in vulns: # share H1's cache: an advisory body is an advisory body + if v.get("id"): + _store(v["id"], v) + return vulns + + +_KNOWN_ECOSYSTEMS = {"PyPI", "npm", "Maven", "Go", "crates.io", "RubyGems", "Packagist", + "NuGet", "Hex", "Debian", "Alpine", "Ubuntu"} + + +def alias_groups(records: list[dict]) -> list[list[dict]]: + """Collapse records that are the same advisory under different ids. + + Not an optimisation — a correctness requirement. Live, `aiohttp 3.9.1` returns **70 records that + are 35 advisories**: OSV publishes a GHSA and a PYSEC entry for every one, cross-linked by + `aliases`. Without this, H2 would resolve each advisory twice, pay two model calls for it, and + emit two band-aids for one hole — and the duplicate would not look like a duplicate, because the + two records carry different ids. + + Union-find over `aliases`. `related` is deliberately NOT followed: it links advisories that are + *about* each other rather than identical, and merging on it would collapse distinct + vulnerabilities into one.""" + parent: dict[str, str] = {} + + def find(x: str) -> str: + parent.setdefault(x, x) + while parent[x] != x: + parent[x] = parent[parent[x]] + x = parent[x] + return x + + def union(a: str, b: str) -> None: + ra, rb = find(a), find(b) + if ra != rb: + parent[ra] = rb + + for rec in records: + rid = rec.get("id") or "" + if not rid: + continue + find(rid) + for al in rec.get("aliases") or []: + union(rid, al) + groups: dict[str, list[dict]] = {} + for rec in records: + if rec.get("id"): + groups.setdefault(find(rec["id"]), []).append(rec) + return list(groups.values()) + + +def preferred_record(group: list[dict]) -> dict: + """The record to reason over, from a set describing one advisory. + + Prefers the one carrying the most usable facts rather than a fixed id-prefix order: a CVE record + is often a GIT range with commit SHAs while its GHSA alias carries the installable version + (H1's finding), but which id holds the prose varies by advisory.""" + def score(r: dict) -> tuple: + rows = _affected_rows(r) + return (any(x["versioned"] for x in rows), bool(r.get("severity")), + bool(r.get("summary")), len(r.get("details") or ""), + (r.get("database_specific") or {}).get("cwe_ids") is not None) + return sorted(group, key=score, reverse=True)[0] + + +def normalize_record(rec: dict, group: list[dict] | None = None) -> dict: + """A raw OSV record → the same shape `resolve()` returns, without any network access. + + H2 already holds the full bodies from `query_package`, so re-fetching by id to reuse `resolve()` + would be one request per advisory for data already in memory. The `affected` rows are merged + across the whole alias group, because that is exactly where H1 found the installable version to + live when the record you asked for did not have one.""" + group = group or [rec] + rows: list[dict] = [] + for r in sorted(group, key=lambda r: r is not rec): # the preferred record's rows first + rows += _affected_rows(r) + sev = "" + for r in group: # prefer any scored vector over none + sev = next((s.get("score") for s in r.get("severity") or [] if s.get("score")), "") or sev + if sev: + break + details = rec.get("details") or "" + cwes: list[str] = [] + for r in group: + cwes += (r.get("database_specific") or {}).get("cwe_ids") or [] + return { + "id": rec.get("id") or "", + "aliases": sorted({a for r in group for a in (r.get("aliases") or [])} + | {r.get("id") for r in group if r.get("id")} - {rec.get("id")}), + "consulted": [r.get("id") for r in group if r.get("id")], + "summary": rec.get("summary") or _first_sentence(details), + "details": details[:4000], + "cwe_ids": list(dict.fromkeys(cwes)), + "cvss": sev, + "published": rec.get("published") or "", + "withdrawn": rec.get("withdrawn") or "", + "affected": rows, + "references": [r.get("url") for r in (rec.get("references") or [])][:12], + } + + +def upgrade_target_for(advisory: dict, package: str, ecosystem: str, installed: str) -> dict: + """The upgrade to recommend to someone running `package @ installed` — **not** the advisory's + lowest fixed version, and **not** some other package's. + + `upgrade_target()` — H1's version, kept unchanged for the single-CVE path — answers a different + question: *what does this advisory fix*, with no installed coordinate to relate it to. It takes + the first `affected` row carrying a version, whatever package that row names and whichever + maintenance branch it describes. H1 asks about one advisory at a time and shows the package + beside the version, so a reader can see the mismatch. H2 turns the answer into a per-package + recommendation, where two things go wrong: + + 1. **The wrong package — reproduced live, 2026-07-29.** 10 of 145 advisories sampled list + several packages. For Log4Shell (`GHSA-jfh8-c2jp-5v3q`), a manifest pinning + `org.ops4j.pax.logging:pax-logging-log4j2 1.10.0` is told to upgrade to **2.15.0**, which is + `log4j-core`'s fix and not a version pax-logging has ever published. Its own fix is `1.10.8`. + Same shape on `GHSA-p6mc-m468-83gw`: `lodash` is fixed at `4.17.19`, `lodash-es` at + `4.17.20`, and `lodash.pick` **not at all** (`last_affected 4.4.0`), so a manifest holding + `lodash.pick` gets sent to install a version that does not exist for it. + 2. **The wrong branch.** Selection ignores the installed version entirely, so which of an + advisory's maintenance branches you are told about is decided by OSV's document order. + Log4Shell publishes `2.13.0→2.15.0`, `2.0-beta9→2.3.1` and `2.4→2.12.2` for one package; the + live record happens to list the first of those first, so `2.14.1` currently gets the right + answer by luck. Nothing in the OSV schema promises that order, and the failure it permits is + recommending `2.3.1` to someone running `2.14.1` — a downgrade, presented as the fix. + + So: filter to the rows for THIS package, then take the smallest published fix strictly greater + than what is installed. Where nothing published is newer, say that instead of naming the closest + number — a version that would not fix anything is worse than admitting there is no upgrade.""" + rows = [r for r in (advisory.get("affected") or []) + if r.get("package") and _same_package(r["package"], package, r.get("ecosystem", ""), ecosystem)] + base = {"package": package, "ecosystem": ecosystem, "installed": installed, + "fixed_version": "", "vulnerable_range": "", "note": ""} + if not rows: + # The advisory reached us because OSV matched this coordinate, so silence here means the + # affected block is shaped in a way this cannot read — report it rather than guessing. + base["note"] = (f"{advisory.get('id', 'advisory')} matched {ecosystem}/{package} but " + "publishes no version range for it") + return base + fixes = sorted({f for r in rows for f in r.get("fixed_versions") or []}, key=_version_key) + intro = [v for r in rows for v in r.get("introduced") or [] if not _COMMITISH.match(str(v))] + base["vulnerable_range"] = ", ".join(dict.fromkeys(intro)) or "see advisory" + if not comparable(installed): + base["note"] = (f"cannot order '{installed}' against the published fixes " + f"({', '.join(fixes) or 'none'}) — pick one by hand") + return base + higher = [f for f in fixes if comparable(f) and version_gt(f, installed)] + if higher: + base["fixed_version"] = higher[0] + return base + commits = [f for r in rows for f in r.get("fixed", [])] + if fixes: + base["note"] = (f"every published fix ({', '.join(fixes)}) is at or below the installed " + f"{installed} — the fix for this branch is not released, or {installed} " + "predates the affected line") + elif commits: + base["note"] = ("OSV records the fix as a source commit, not a released version: " + + ", ".join(commits[:2])) + else: + base["note"] = f"OSV lists no fixed version for {ecosystem}/{package}" + return base + + +def _same_package(row_name: str, want: str, row_eco: str, want_eco: str) -> bool: + if want_eco and row_eco and row_eco != want_eco: + return False + if row_name == want: + return True + if (row_eco or want_eco) == "PyPI": # PEP 503: Flask_Login and flask-login are one package + return re.sub(r"[-_.]+", "-", row_name).lower() == re.sub(r"[-_.]+", "-", want).lower() + return False + + def upgrade_target(advisory: dict) -> dict: """The single best "upgrade to X" recommendation, or an honest statement that OSV has none. diff --git a/src/vpcopilot/pipeline.py b/src/vpcopilot/pipeline.py index cefc233..f529fb8 100644 --- a/src/vpcopilot/pipeline.py +++ b/src/vpcopilot/pipeline.py @@ -67,16 +67,24 @@ def run_pipeline( log: Callable[[str], None] = print, advisory: str | None = None, # H1: appended AFTER log so no positional call can shift spec_path: str | None = None, # H3: an OpenAPI spec — alone, or alongside a repo + manifest_paths: list[str] | None = None, # H2: dependency manifests — alone, or alongside + min_severity: str = "high", # H2: floor for reaching the resolve agent + max_advisories: int = 25, # H2: cap on the agent stage (0 = no cap) + include_dev: bool = False, # H2: dev/test-scoped dependencies too ) -> dict: # H1 — exactly one input. Deliberately a hard error rather than a silent no-op: today # `run_pipeline("/does/not/exist")` completes and writes a full set of empty artifacts, and # that is the failure mode not to extend. - # H1/H3 — repo and advisory are alternatives; a spec is additive. `--spec` alone scans the - # contract; `--spec` with a repo also cross-checks the two for orphans, which needs both. - if advisory and (repo_path or spec_path): - raise ValueError("--cve scans one advisory; it cannot be combined with a repo or --spec") - if not (repo_path or advisory or spec_path): - raise ValueError("pass a repo path, an advisory id (--cve), or an OpenAPI spec (--spec)") + # H1/H2/H3 — repo and advisory are alternatives; a spec and a manifest are additive. `--spec` + # alone scans the contract, and with a repo also cross-checks the two for orphans; `--manifest` + # alone scans the dependency tree, and with a repo covers the code and its dependencies in one + # run, where correlation can collapse band-aids that overlap across the two. + if advisory and (repo_path or spec_path or manifest_paths): + raise ValueError("--cve scans one advisory; it cannot be combined with a repo, " + "--spec or --manifest") + if not (repo_path or advisory or spec_path or manifest_paths): + raise ValueError("pass a repo path, an advisory id (--cve), an OpenAPI spec (--spec), " + "or a dependency manifest (--manifest)") h = Harness(config_path) h.warmup() # B6: warm instructor's mode registry before ANY fan-out — both inputs need it t0, started = time.perf_counter(), runmeta.utc_now() @@ -89,8 +97,14 @@ def run_pipeline( findings: list = [] spec_code: dict[str, str] = {} # H3: spec name -> the text handed to discover spec_orphans: dict | None = None - forced_decision = forced_remediation = None + # H1 supplied exactly one of each; H2 supplies many, so both are keyed by finding id. A finding + # with a forced decision skips triage, one with a forced remediation skips the remediate agent, + # and everything without either behaves exactly as it did before this change. + forced_decisions: dict[str, object] = {} + forced_remediations: dict[str, object] = {} + advisory_ids: set[str] = set() # findings with no source to verify against advisory_meta: dict | None = None + deps_report: dict | None = None if advisory: # H1 — an advisory produces ONE finding and then joins the ordinary stages. There is no @@ -100,7 +114,10 @@ def run_pipeline( from .inputs.cve import resolve_advisory res = resolve_advisory(h, advisory, log=log) findings = [res["finding"]] - forced_decision, forced_remediation = res["decision"], res["remediation"] + advisory_ids.add(res["finding"].id) + if res["decision"] is not None: + forced_decisions[res["finding"].id] = res["decision"] + forced_remediations[res["finding"].id] = res["remediation"] advisory_meta = {"id": res["advisory"]["id"], "source": "osv", "consulted": res["advisory"]["consulted"], "network_observable": res["profile"].network_observable, @@ -133,9 +150,25 @@ def run_pipeline( spec_orphans = o log(f" spec vs code: {len(o['matched'])} matched, " f"{len(o['spec_only'])} declared-but-unserved, {len(o['code_only'])} undeclared") - if route_ctx: + if manifest_paths: + # H2 — a manifest yields findings the same way an advisory does (H1's resolve agent, + # H1's deterministic no_bandaid route, H1's OSV-sourced cure), so they are appended to + # `findings` here and every stage below treats them as advisory findings. Additive: with + # a repo alongside, the code findings and the dependency findings correlate together, + # which is the point — one band-aid may cover both. + from .inputs.deps import resolve_manifests + dres = resolve_manifests(h, manifest_paths, min_severity=min_severity, + max_advisories=max_advisories, include_dev=include_dev, + concurrency=concurrency, log=log) + deps_report = dres["report"] + for f in dres["findings"]: + findings.append(f) + advisory_ids.add(f.id) + forced_decisions.update(dres["decisions"]) + forced_remediations.update(dres["remediations"]) + if repo_path and route_ctx: log("route context: found the app's declared routes — grounding finding endpoints (no guessing)") - else: + elif repo_path or spec_path: log(" ⚠ NO app route context found (no OpenAPI/Swagger spec or route registrations detected) " "— finding endpoints are INFERRED and may be inaccurate") @@ -166,7 +199,9 @@ def _discover(p): from .schemas import FindingList log(f" ⚠ discover failed on {spec_name}: {e} — skipping the spec") disc_results.append((spec_name, spec_text, spec_text, FindingList(findings=[]))) - used_ids: set[str] = set() # A4: the pipeline owns finding ids — a model may reuse one across files + # A4: the pipeline owns finding ids — a model may reuse one across files. Seeded from findings + # already present so an H2 advisory id cannot be handed out twice (empty on a repo-only run). + used_ids: set[str] = {f.id for f in findings} for rel, code, raw, res in disc_results: file_code[rel] = code file_raw[rel] = raw @@ -212,12 +247,14 @@ def _threshold(f): shift = -0.1 if _sev(f) in ("critical", "high") else 0.1 return max(0.0, min(1.0, min_confidence + shift)) + # H1/H2 — an advisory finding has no source to read, and verify's entire method is reading the + # offending code. OSV already asserts the vulnerability is real; re-litigating it against code we + # do not have would only manufacture doubt. The resolve agent's own confidence is the gate + # instead. Per-finding rather than per-run, because H2 puts code findings and dependency + # findings in the same list and only the code ones can be verified. + to_verify = [f for f in findings if f.id not in advisory_ids] with ThreadPoolExecutor(max_workers=concurrency) as ex: - # H1 — an advisory run has no source to read, and verify's entire method is reading the - # offending code. OSV already asserts the vulnerability is real; re-litigating it against - # code we do not have would only manufacture doubt. The resolve agent's own confidence is - # the gate instead, applied in inputs/cve.py. - for f, v in (ex.map(_verify, findings) if not advisory else []): + for f, v in ex.map(_verify, to_verify): if v is None: # B6: verify errored — count as dropped, keep going dropped += 1 continue @@ -269,10 +306,9 @@ def _threshold(f): log(f" undocumented_or_orphaned: {len(so)} declared-but-unserved, " f"{len(co)} served-but-undeclared") - if advisory: - verified = list(findings) + verified += [f for f in findings if f.id in advisory_ids] verify_s = time.perf_counter() - t_verify - if not advisory: + if to_verify or not advisory_ids: # a run with nothing to verify still reported "0 verified" log(f"{len(verified)} finding(s) verified real (min-confidence {min_confidence})") # 3-5) triage -> generate band-aids -> remediate (code cure) ------------ @@ -291,17 +327,20 @@ def _threshold(f): # findings) never sends one giant call that blows the per-call timeout; chunks run in # parallel and their decisions are concatenated. TRIAGE_CHUNK = 12 - if forced_decision is not None: - # H1 — the advisory has no network-observable exploitation pattern, so no control at a - # load balancer can mitigate it. That is a fact about the advisory, not a judgement - # call, and the acceptance requires it: routing it to `no_bandaid` in code rather than - # asking triage nicely is what makes it a guarantee instead of a hope. - decisions = [forced_decision] - elif len(verified) <= TRIAGE_CHUNK: - decisions = triage.run(h, verified).decisions + # H1/H2 — an advisory with no network-observable exploitation pattern cannot be mitigated by + # any control at a load balancer. That is a fact about the advisory, not a judgement call, + # and the acceptance requires it: routing it to `no_bandaid` in code rather than asking + # triage nicely is what makes it a guarantee instead of a hope. Everything else is triaged + # normally, so a manifest run that declines half its advisories still triages the other half. + decisions = [forced_decisions[f.id] for f in verified if f.id in forced_decisions] + to_triage = [f for f in verified if f.id not in forced_decisions] + if not to_triage: + pass + elif len(to_triage) <= TRIAGE_CHUNK: + decisions += triage.run(h, to_triage).decisions else: - chunks = [verified[i:i + TRIAGE_CHUNK] for i in range(0, len(verified), TRIAGE_CHUNK)] - log(f"triaging {len(verified)} findings in {len(chunks)} batches of ≤{TRIAGE_CHUNK}") + chunks = [to_triage[i:i + TRIAGE_CHUNK] for i in range(0, len(to_triage), TRIAGE_CHUNK)] + log(f"triaging {len(to_triage)} findings in {len(chunks)} batches of ≤{TRIAGE_CHUNK}") def _triage(ch): try: @@ -312,10 +351,9 @@ def _triage(ch): return [TriageDecision(finding_id=f.id, bandaids=[], no_bandaid=True, residual_risk="triage failed — code fix only") for f in ch] - decisions = [] with ThreadPoolExecutor(max_workers=concurrency) as ex: for ds in ex.map(_triage, chunks): - decisions.extend(ds) + decisions.extend(ds) # extend, never reassign: the forced ones are already in # A2: derive validation probes BEFORE generate, so each band-aid is built against the # finding's CONCRETE exploit (exact method + full path) and spares its legit request. @@ -334,61 +372,115 @@ def _probe(f): probe_by_id = {p["finding_id"]: p for p in probes} # 4) generate recommended band-aid(s), skipping ones an earlier finding covers - for d in decisions: - f = by_id.get(d.finding_id) - if not f: - continue - if d.no_bandaid: - log(f" triage {d.finding_id} -> NO BAND-AID (code cure only)") - continue - tags = ", ".join( - f"{b.control.value}({b.coverage.value}{'*' if b.recommended else ''})" - for b in d.bandaids - ) - log(f" triage {d.finding_id} -> {tags}") + def _try_bandaids(d, f, options, *, only_one: bool) -> int: + """Generate from `options`, claiming each control's coverage slot. Returns how many + artifacts were produced. `only_one` stops after the first success — a fallback exists + to stop a finding ending bare, not to deploy the whole non-recommended stack.""" pr = probe_by_id.get(d.finding_id) or {} exploit, legit = pr.get("exploit"), pr.get("legit") - for b in [b for b in d.bandaids if b.recommended] or d.bandaids: - key = correlate.coverage_key(b.control.value, f.file, - identity=f.endpoint or getattr(f, "source", "") or f.id) + made = 0 + for b in options: + key = correlate.coverage_key( + b.control.value, f.file, + identity=f.endpoint or getattr(f, "source", "") or f.id) if key in seen_keys: - correlations.append({"finding_id": d.finding_id, "control": b.control.value, - "covered_by": seen_keys[key], "coverage_key": key}) + # An LB-wide control has ONE instance per load balancer, so the slot really is + # taken — but the policy in it was generated against the OWNING finding's + # exploit, not this one's, and it was validated against that probe too. Saying + # so in the record is the difference between "one control covers both" and a + # confident claim of protection nobody measured. + shared = b.control.value in correlate.LB_WIDE + correlations.append({ + "finding_id": d.finding_id, "control": b.control.value, + "covered_by": seen_keys[key], "coverage_key": key, + "note": (f"{b.control.value} is LB-wide: the attached policy was " + f"generated and validated against {seen_keys[key]}'s exploit, " + f"not this one's" if shared else + "same endpoint — one policy covers both")}) log(f" correlate {d.finding_id}: {b.control.value} already covered by " f"{seen_keys[key]} — skip duplicate band-aid") continue seen_keys[key] = d.finding_id try: # B6: a model that can't emit this band-aid must not kill the scan (or vanish silently) - arts = generate.run(h, f, b.control, b.rationale, exploit=exploit, legit=legit).items + arts = generate.run(h, f, b.control, b.rationale, + exploit=exploit, legit=legit).items except Exception as e: # noqa: BLE001 - seen_keys.pop(key, None) # coverage wasn't established — let a sibling finding retry it - log(f" ⚠ generate produced no {b.control.value} band-aid for {d.finding_id}: {e} " - f"— code fix only") + seen_keys.pop(key, None) # coverage wasn't established — let a sibling retry it + log(f" ⚠ generate produced no {b.control.value} band-aid for {d.finding_id}: " + f"{e} — code fix only") continue + made += len(arts) for a in arts: # A3/A9: lint the consumed-spec controls now; refiner corrects at apply iss = lint_generated_spec(a.control.value, a.spec, exploit) if iss: - log(f" ⚠ lint {a.policy_name}: {'; '.join(iss)} — refine will correct at apply") + log(f" ⚠ lint {a.policy_name}: {'; '.join(iss)} — " + "refine will correct at apply") artifacts.extend(arts) + if only_one: + break + return made + + # PASS 1 — every finding's RECOMMENDED band-aids, exactly as before H2. + bare: list = [] + for d in decisions: + f = by_id.get(d.finding_id) + if not f: + continue + if d.no_bandaid: + log(f" triage {d.finding_id} -> NO BAND-AID (code cure only)") + continue + tags = ", ".join( + f"{b.control.value}({b.coverage.value}{'*' if b.recommended else ''})" + for b in d.bandaids + ) + log(f" triage {d.finding_id} -> {tags}") + preferred = [b for b in d.bandaids if b.recommended] or d.bandaids + if not _try_bandaids(d, f, preferred, only_one=False) and preferred is not d.bandaids: + bare.append((d, f)) + + # PASS 2 — findings whose every recommended control was claimed by another finding used to + # end the loop with nothing, silently, while `correlations.json` recorded them as covered. + # Found by the first live H2 run: Log4Shell's only recommended control was the LB-wide + # `waf`, which an aiohttp header-injection advisory owned, so the critical finding got no + # band-aid and the shipped WAF (`waf-block-header-injection-nullbyte-crlf`) addressed + # nothing about JNDI. + # + # This runs as a SECOND PASS, after every recommended claim above, and takes only the + # first alternative that generates. Both parts are load-bearing. Interleaved (the first + # cut of this change), a non-recommended alternate could claim an LB-wide slot BEFORE a + # later finding that actually recommended that control got to ask for it — reinflicting + # the very bug this fixes on a different finding, and deploying up to three controls + # triage never recommended. Deferred, the recommended tier always wins the slot, so a run + # where it succeeds is byte-identical to before. + for d, f in bare: + alternates = [b for b in d.bandaids if not b.recommended] + if not alternates: + continue + log(f" every recommended band-aid for {d.finding_id} was already covered by " + f"another finding — trying its alternative(s) rather than leaving it bare") + _try_bandaids(d, f, alternates, only_one=True) # 5) every verified finding gets a real code fix (band-aid != cure) — A5: over ALL # verified findings, in parallel, not only those triage handed a band-aid. Skippable # (draft_code_fixes) to save the biggest chunk of tokens when only band-aids are wanted. - if forced_remediation is not None: - # H1 — the cure for a dependency CVE is a version bump in someone else's package. - # There is no file of ours to patch, and asking a model to draft a diff against vendor - # code it cannot see is how you get a confident, wrong patch. The version comes from - # OSV; `remediate` is never called on this path. - remediations = [forced_remediation] - log(f"cure: {forced_remediation.summary}") + # H1/H2 — the cure for a dependency advisory is a version bump in someone else's package. + # There is no file of ours to patch, and asking a model to draft a diff against vendor code + # it cannot see is how you get a confident, wrong patch. The version comes from OSV; + # `remediate` is never called for those findings. Code findings alongside them still get one. + remediations = [forced_remediations[f.id] for f in verified if f.id in forced_remediations] + for r in remediations: + log(f"cure: {r.summary}") + to_remediate = [f for f in verified if f.id not in forced_remediations] + if not to_remediate: + pass elif draft_code_fixes: def _remediate(f): return remediate.run(h, f, file_raw.get(f.file, "")) with ThreadPoolExecutor(max_workers=concurrency) as ex: - remediations = list(ex.map(_remediate, verified)) + remediations += list(ex.map(_remediate, to_remediate)) else: - log(f"skipping code-fix drafting for {len(verified)} finding(s) (draft_code_fixes=off)") + log(f"skipping code-fix drafting for {len(to_remediate)} finding(s) (draft_code_fixes=off)") synth_s = time.perf_counter() - t_synth # D2) per-stage metrics: timing, discovery, verify precision, dedup ------ @@ -403,10 +495,12 @@ def _remediate(f): "avg_confidence": round(sum(confidences) / len(confidences), 2) if confidences else 0.0, "min_confidence": min_confidence}, "synthesize": {"policies": len(artifacts), "dupe_bandaids_collapsed": len(correlations), - "code_fix_prs": len(remediations)}, + "code_fix_prs": sum(1 for r in remediations if r.kind == "code_fix"), + "dependency_upgrades": sum(1 for r in remediations + if r.kind == "dependency_upgrade")}, } summary = _write_out(out_dir, findings, verified, decisions, artifacts, remediations, - correlations, skipped, metrics, probes) + correlations, skipped, metrics, probes, deps_report) # F3) run manifest — what was scanned, at which commit, by whom, with which models and caps. # Every audit entry carries this run_id, so an exported bundle explains its own provenance. # Fail-soft: provenance is evidence, not a gate — never fail a completed scan over it. @@ -414,16 +508,21 @@ def _remediate(f): cfg = getattr(h, "cfg", None) runmeta.write_manifest( out_dir, repo=str(root.resolve()) if root else None, advisory=advisory_meta, - input_kind=("advisory" if advisory else - ("repo+spec" if (repo_path and spec_path) else - ("spec" if spec_path else "repo"))), + # Joined in a fixed order, so every combination that existed before H2 — "repo", + # "spec", "repo+spec", "advisory" — still renders byte-identically. + input_kind=("advisory" if advisory else "+".join( + k for k, on in (("repo", repo_path), ("spec", spec_path), + ("manifest", manifest_paths)) if on)), spec=str(Path(spec_path).resolve()) if spec_path else None, + manifests=([str(Path(m).resolve()) for m in manifest_paths] if manifest_paths else None), config_path=config_path, started=started, models={a: cfg.for_agent(a).model for a in AGENT_NAMES} if cfg else None, caps={"min_confidence": min_confidence, "max_files": max_files, "max_bytes": max_bytes, "draft_code_fixes": draft_code_fixes}, counts={"candidates": len(findings), "verified": len(verified), "policies": len(artifacts), - "code_fix_prs": len(remediations)}, + "code_fix_prs": sum(1 for r in remediations if r.kind == "code_fix"), + "dependency_upgrades": sum(1 for r in remediations + if r.kind == "dependency_upgrade")}, finished=runmeta.utc_now(), **(runmeta.git_provenance(root) if root else {})) except Exception as e: # noqa: BLE001 log(f" ⚠ could not write the run manifest (run.json): {e} — an audit export will lack provenance") @@ -433,7 +532,7 @@ def _remediate(f): def _write_out(out_dir, findings, verified, decisions, artifacts, remediations, correlations, - skipped, metrics=None, probes=None) -> dict: + skipped, metrics=None, probes=None, deps_report=None) -> dict: out = Path(out_dir) (out / "policies").mkdir(parents=True, exist_ok=True) (out / "remediations").mkdir(parents=True, exist_ok=True) @@ -455,6 +554,18 @@ def _write_out(out_dir, findings, verified, decisions, artifacts, remediations, (out / "remediations" / f"{r.finding_id}.pr.md").write_text(f"# {r.pr_title}\n\n{r.pr_body}\n") (out / "metrics.json").write_text(json.dumps(metrics or {}, indent=2)) + if deps_report is not None: + # NOT `manifest.json`: `export.verify_bundle` locates each run inside an evidence bundle by + # finding every member whose name ends `manifest.json`, so an artifact by that name would be + # read as a second evidence manifest and break verification of the bundle it ships in. + (out / "dependencies.json").write_text(json.dumps(deps_report, indent=2)) + else: + # Every other member here is rewritten unconditionally, so a re-scan replaces it. This one + # is written only when a manifest was given, so without this a later scan of the SAME out + # dir would inherit the previous run's dependency data — and the report, `GET /api/deps` + # and the evidence bundle would all present it as this run's. (G4 shipped the same class of + # bug with leftover `policies/` artifacts.) + (out / "dependencies.json").unlink(missing_ok=True) summary = { "candidates": len(findings), "verified": len(verified), @@ -465,11 +576,19 @@ def _write_out(out_dir, findings, verified, decisions, artifacts, remediations, }, "no_bandaid": [d.finding_id for d in decisions if d.no_bandaid], "policies": [f"{a.control.value}/{a.policy_name}" for a in artifacts], - "code_fix_prs": [r.finding_id for r in remediations], + # H1/H2: split by `kind`, the discriminator `pr.py` already decides on. A + # `dependency_upgrade` is a cure someone else has to ship — no PR was drafted and none + # can be — so counting it as a code-fix PR overstates the cure half of the story on + # every surface that renders it. A repo scan's remediations are all `code_fix`, so this + # list is unchanged there; only the advisory paths (--cve, --manifest) move. + "code_fix_prs": [r.finding_id for r in remediations if r.kind == "code_fix"], + "dependency_upgrades": [r.finding_id for r in remediations if r.kind == "dependency_upgrade"], "correlations": [f"{c['finding_id']} covered-by {c['covered_by']} ({c['control']})" for c in correlations], "skipped_files": len(skipped), "out_dir": str(out), } + if deps_report is not None: + summary["dependencies"] = deps_report["funnel"] (out / "summary.json").write_text(json.dumps(summary, indent=2)) from . import ledger # seed the remediation ledger (found) ledger.init_from_scan(out_dir, [f.model_dump() for f in findings], diff --git a/src/vpcopilot/report.py b/src/vpcopilot/report.py index de35e54..cec18b0 100644 --- a/src/vpcopilot/report.py +++ b/src/vpcopilot/report.py @@ -210,6 +210,11 @@ def _hero_html(im: dict) -> str: + f'
{_e(im["change_control_days"])} days' 'normal change control
' + h(im["code_prs"], "code-fix PRs (the cure)") + # H2: an advisory's cure is an upgrade in someone else's package — no PR was drafted + # and none can be. Shown beside the PR count, never folded into it. Omitted entirely + # when there are none, so a repo-only report is unchanged. + + (h(im["dependency_upgrades"], "upgrades to ship (no PR)") + if im.get("dependency_upgrades") else "") + dash_link + '
') @@ -291,6 +296,60 @@ def _blast_radius_html(out_dir: str) -> str: f'verdicttop blocked paths{rows}') +def _dependencies_html(out_dir: str) -> str: + """H2: the dependency funnel, INCLUDING what was not checked. + + Two counts carry most of the honesty here and both are rendered even when zero: entries whose + version could not be pinned (never queried, so nothing is known about them) and advisories held + back by the severity floor or the cap (found and listed, but never sent to an agent). A + dependency report that showed only the resolved rows would read as a clean bill of health for + packages it never looked at.""" + dep = _load(Path(out_dir), "dependencies.json", None) + if not dep: + return "" + f = dep.get("funnel") or {} + s = dep.get("settings") or {} + chips = "".join(_chip(f.get(k, 0), lbl) for k, lbl in ( + ("packages_parsed", "packages"), ("packages_vulnerable", "vulnerable"), + ("advisories_distinct", "advisories"), ("exploitable", "exploitable"), + ("declined", "declined"), ("not_resolved", "not resolved"), + ("packages_unpinned", "could not pin"))) + rows = "" + for a in (dep.get("advisories") or [])[:80]: + fixed = _e(a.get("fixed_version")) or f'{_e(a.get("fix_note")) or "none published"}' + rows += (f'{_e(a.get("severity"))}' + f'{_e(a.get("package"))}{_e(a.get("installed"))}' + f'{_e(a.get("advisory_id"))}{fixed}' + f'{_e(a.get("disposition"))}' + f'{_e(str(a.get("reason") or "")[:140])}') + extra = "" + n_un = len(dep.get("unpinned") or []) + if n_un: + reasons = ", ".join(sorted({u.get("reason", "") for u in dep["unpinned"]})) + extra += ('

Not checked. ' + f'{n_un} manifest entry(ies) could not be pinned to an exact version and were ' + f'never queried ({_e(reasons)}). Nothing is known about them — this is not a ' + 'statement that they are clean.

') + if f.get("not_resolved"): + extra += ('

Listed, not resolved. ' + f'{f["not_resolved"]} advisory(ies) were found but not sent to an agent ' + f'(--min-severity {_e(s.get("min_severity"))}, --max-advisories ' + f'{_e(s.get("max_advisories"))}). They are in the table with the reason.

') + # A package the batch flagged as vulnerable but whose advisory fetch then failed used to appear + # here exactly like one that came back clean: no row, no counter, no warning. + for u in dep.get("unchecked") or []: + extra += ('

Could not check. ' + f'{_e(u.get("ecosystem"))}/{_e(u.get("name"))} {_e(u.get("version"))} — ' + f'{_e(u.get("error"))}. Its advisories are unknown, not absent.

') + for e in dep.get("errors") or []: + extra += f'

⚠ {_e(e.get("path"))}: {_e(e.get("error"))}

' + return ('

Dependencies pinned packages resolved against OSV.dev — ' + 'the vulnerability is in code you do not own, so the cure is a version bump

' + f'
{chips}
{extra}' + '' + f'{rows}
sevpackageinstalledadvisoryfixed indispositionwhy
') + + def build_report(out_dir: str = "out") -> str: out = Path(out_dir) summary = _load(out, "summary.json", {}) @@ -322,7 +381,8 @@ def build_report(out_dir: str = "out") -> str: _chip(len(summary.get("no_bandaid", [])), "code-cure only"), _chip(len(summary.get("policies", [])), "XC policies"), _chip(len(summary.get("code_fix_prs", remediations)), "code-fix PRs"), - ]) + ] + ([_chip(len(summary.get("dependency_upgrades", [])), "upgrades to ship")] + if summary.get("dependency_upgrades") else [])) cards = "".join(_finding_card(f, tri.get(f.get("id")), rem.get(f.get("id"))) for f in findings) @@ -384,6 +444,7 @@ def build_report(out_dir: str = "out") -> str: {_bars_html(findings, summary)} {_models_html()} {_metrics_html(metrics)} +{_dependencies_html(out_dir)}

Findings & band-aid coverage

{cards or '

No findings.

'}

Generated XC band-aid policies

{pol_html or '

None.

'} {impact_html} diff --git a/tests/test_inputs_manifest.py b/tests/test_inputs_manifest.py new file mode 100644 index 0000000..9a579e9 --- /dev/null +++ b/tests/test_inputs_manifest.py @@ -0,0 +1,1097 @@ +"""H2 — the dependency manifest input path. Offline: OSV is faked, the resolve agent is faked. + +Every live behaviour these fakes encode was measured against the real api.osv.dev on 2026-07-29 and +is recorded in the docstring of the test that pins it. Four of them are invisible from the OSV +schema, and each one silently produces a *confident wrong answer* rather than an error.""" +import json + +import pytest + +from vpcopilot.inputs import deps, manifest, osv + +FIX = "bench/fixtures/manifests" + + +# =============================================================== parsing: refusing to guess +def test_only_an_exact_pin_becomes_a_query(): + """A `requirements.txt` is a request, not a lockfile. `>=`, `~=`, a wildcard and a bare name all + describe a range someone else resolves later, so none of them names an installed version.""" + r = manifest.parse(f"{FIX}/requirements.txt") + assert r["kind"] == "pip" + got = {p["name"]: p["version"] for p in r["packages"]} + assert got["aiohttp"] == "3.9.1" and got["urllib3"] == "2.0.7" + unpinned = {s["raw"].split()[0].split(">")[0].split("~")[0].split("=")[0] + for s in r["skipped"] if s["reason"] == "unpinned"} + assert {"gunicorn", "cryptography", "boto3", "django"} <= unpinned + assert not any(p["name"] in ("gunicorn", "cryptography", "boto3", "django") + for p in r["packages"]) + + +@pytest.mark.parametrize("bad", ["not-a-version", "1.0.0-SNAPSHOT", "${project.version}", ""]) +def test_the_reason_an_unresolvable_version_is_refused_rather_than_sent(bad): + """Measured live: OSV does NOT error on a version string it cannot parse. `aiohttp` at each of + these returned **81 advisories**, against 70 for the real 3.9.1 and 87 for no version at all. + So sending a guess does not fail loudly — it returns a larger, wrong answer that every stage + downstream treats as fact. This test pins the invariant that makes that unreachable: a + coordinate only exists once its version is known.""" + assert not manifest._UNRESOLVED.search("3.9.1") + entries, _ = manifest._parse_requirements.__wrapped__(bad) if False else ([], []) # noqa: B018 + # every parser emits `version` or nothing at all — never a placeholder + for path in (f"{FIX}/requirements.txt", f"{FIX}/package-lock.json", f"{FIX}/pom.xml"): + for p in manifest.parse(path)["packages"]: + assert p["version"] and not manifest._UNRESOLVED.search(p["version"]) + assert p["version"] != bad or bad == "" + assert entries == [] + + +def test_extras_markers_and_continuations(): + r = manifest.parse(f"{FIX}/requirements.txt") + by = {p["name"]: p for p in r["packages"]} + assert "flask-login" in by # PEP 503: Flask_Login[extras] -> flask-login + assert by["importlib-metadata"]["version"] == "6.8.0" + assert "python_version" in by["importlib-metadata"]["note"] # marker recorded, not dropped + assert by["urllib3"]["version"] == "2.0.7" # backslash continuation + --hash + + +def test_pep503_normalisation_collapses_one_package_spelt_two_ways(): + assert manifest.normalize_pypi("Flask_Login") == "flask-login" + assert manifest.normalize_pypi("zope.interface") == "zope-interface" + + +def test_editable_and_direct_url_requirements_are_named_not_silently_dropped(): + r = manifest.parse(f"{FIX}/requirements.txt") + reasons = {s["reason"] for s in r["skipped"]} + assert {"editable", "direct-reference", "pip-option"} <= reasons + assert all(s["detail"] for s in r["skipped"]) # every refusal states why + + +def test_r_includes_are_followed_with_a_cycle_guard(tmp_path): + r = manifest.parse(f"{FIX}/requirements-with-include.txt") + names = {p["name"] for p in r["packages"]} + assert {"mypy", "pytest", "black", "aiohttp"} <= names # own + both includes + + a, b = tmp_path / "a.txt", tmp_path / "b.txt" + a.write_text("-r b.txt\nfoo==1.0\n") + b.write_text("-r a.txt\nbar==2.0\n") + got = manifest.parse(str(a)) # must terminate + assert {p["name"] for p in got["packages"]} == {"foo", "bar"} + + +def test_a_missing_include_is_reported_and_does_not_abort_the_file(tmp_path): + p = tmp_path / "requirements.txt" + p.write_text("-r nope.txt\nflask==3.0.0\n") + r = manifest.parse(str(p)) + assert [x["name"] for x in r["packages"]] == ["flask"] + assert any(s["reason"] == "include-unreadable" for s in r["skipped"]) + + +# ---------------------------------------------------------------- package-lock.json +def test_lockfile_v2_is_not_double_counted(): + """lockfileVersion 2 carries BOTH the v3 `packages` map and the v1 `dependencies` tree, + describing the same installs. A parser reading both reports every package twice — and the + duplicate looks exactly like a real second install at a different depth.""" + doc = json.loads(open(f"{FIX}/package-lock.json").read()) + assert doc["lockfileVersion"] == 2 and doc["packages"] and doc["dependencies"] # both present + r = manifest.parse(f"{FIX}/package-lock.json") + assert [p["name"] for p in r["packages"]].count("lodash") == 1 + + +def test_the_root_project_is_not_one_of_its_own_dependencies(): + r = manifest.parse(f"{FIX}/package-lock.json") + assert "nimbus-web" not in {p["name"] for p in r["packages"]} + + +def test_scoped_names_survive_the_node_modules_prefix(): + r = manifest.parse(f"{FIX}/package-lock.json") + assert "@babel/traverse" in {p["name"] for p in r["packages"]} + + +def test_workspace_links_git_deps_and_versionless_entries_are_refused_by_name(): + r = manifest.parse(f"{FIX}/package-lock.json") + by = {s["raw"]: s["reason"] for s in r["skipped"]} + assert by["node_modules/internal-tools"] == "workspace-link" + assert by["node_modules/forked-dep"] == "direct-reference" + assert by["node_modules/no-version-here"] == "no-version" + + +def test_runtime_beats_dev_when_one_package_is_reachable_both_ways(): + """lodash is a runtime dependency AND vendored under jest as a dev one. If anything reaches it + at runtime it is in the request path, so the runtime scope wins.""" + r = manifest.parse(f"{FIX}/package-lock.json") + assert next(p for p in r["packages"] if p["name"] == "lodash")["scope"] == "runtime" + + +def test_a_v1_lockfile_still_parses(tmp_path): + p = tmp_path / "package-lock.json" + p.write_text(json.dumps({"lockfileVersion": 1, "dependencies": { + "lodash": {"version": "4.17.15", "dependencies": {"nested": {"version": "1.0.0"}}}}})) + r = manifest.parse(str(p)) + assert {x["name"]: x["version"] for x in r["packages"]} == {"lodash": "4.17.15", + "nested": "1.0.0"} + + +def test_a_workspace_source_tree_is_not_queried_as_a_registry_package(tmp_path): + """v2/v3 `packages` keys are project-relative PATHS: only `node_modules/...` are installed + registry releases. A workspace's own source (`packages/internal-tools`) was emitted as a + registry coordinate — asking OSV about first-party code under a name that can collide with a + public package, and where the entry carried no `name`, under the directory path itself, which + is not a legal npm name. Both were counted in `packages_parsed`/`packages_queried`.""" + p = tmp_path / "package-lock.json" + p.write_text(json.dumps({"lockfileVersion": 3, "packages": { + "": {"name": "root"}, + "packages/internal-tools": {"name": "internal-tools", "version": "1.0.0"}, + "packages/axios": {"name": "axios", "version": "0.0.1"}, + "packages/no-name-here": {"version": "2.0.0"}, + "node_modules/lodash": {"version": "4.17.15"}}})) + r = manifest.parse(str(p)) + assert [x["name"] for x in r["packages"]] == ["lodash"] # was: all four + assert {s["reason"] for s in r["skipped"]} == {"workspace-source"} + assert len(r["skipped"]) == 3 + + +def test_a_v1_entry_with_no_version_is_reported_not_dropped(tmp_path): + """The v2/v3 branch refuses a versionless entry explicitly; the v1 branch fell through both + arms and dropped it with no trace, so identical lockfile content was invisible risk in v1 and + a reported refusal in v3.""" + p = tmp_path / "package-lock.json" + p.write_text(json.dumps({"lockfileVersion": 1, "dependencies": { + "lodash": {"version": "4.17.15"}, + "no-version-here": {"resolved": "https://registry.npmjs.org/x/-/x.tgz"}}})) + r = manifest.parse(str(p)) + assert [x["name"] for x in r["packages"]] == ["lodash"] + assert [(s["raw"], s["reason"]) for s in r["skipped"]] == [("no-version-here", "no-version")] + + +def test_an_unresolved_property_in_a_coordinate_is_refused_like_one_in_a_version(tmp_path): + """`expand()` runs on groupId/artifactId, but only the VERSION was tested for a leftover + `${...}`. Neither `${project.groupId}` nor `${project.artifactId}` is ever in `props`, so the + literal reached OSV inside the coordinate NAME, came back empty, and was counted as + checked-and-clean. `${project.groupId}` for intra-project deps is ordinary multi-module Maven.""" + p = tmp_path / "pom.xml" + p.write_text("""com.acme2.4.0 + ${project.groupId}common + ${project.version} + org.x${artifact.name} + 1.0 + org.okreal + 1.2.3""") + r = manifest.parse(str(p)) + assert [x["name"] for x in r["packages"]] == ["org.ok:real"] + assert not any("${" in x["name"] for x in r["packages"]) + assert {s["reason"] for s in r["skipped"]} == {"unresolved-property"} + + +def test_a_package_json_is_refused_not_silently_read_as_an_empty_lockfile(tmp_path): + """`detect_kind` routes any JSON carrying a `dependencies` map to the npm parser, and a + package.json has one — but of name → RANGE STRING, where a v1 lockfile has name → {version}. + Every entry failed the isinstance check and was dropped, so a package.json with three + dependencies parsed to zero packages, zero skipped and NO error: byte-identical to a clean + lockfile. That is the one confusion this module exists to prevent.""" + p = tmp_path / "package.json" + p.write_text(json.dumps({"name": "app", "version": "1.0.0", + "dependencies": {"lodash": "4.17.20", "express": "^4.17.1"}, + "devDependencies": {"jest": "^29.0.0"}})) + with pytest.raises(manifest.ManifestError, match="package-lock.json"): + manifest.parse(str(p)) + # …and through parse_all it degrades to a reported error, never to "nothing found" + r = manifest.parse_all([str(p)]) + assert r["packages"] == [] and r["errors"] and "package.json looks like" in r["errors"][0]["error"] + assert r["files"][0]["error"] + + +def test_an_empty_dependencies_map_is_still_a_legitimate_empty_lockfile(tmp_path): + """The package.json guard keys on "no value is an object", so it must not fire on a lockfile + that genuinely installs nothing.""" + p = tmp_path / "package-lock.json" + p.write_text(json.dumps({"lockfileVersion": 1, "dependencies": {}})) + assert manifest.parse(str(p))["packages"] == [] + + +def test_a_lock_entry_that_is_not_an_object_is_reported_not_dropped(tmp_path): + p = tmp_path / "package-lock.json" + p.write_text(json.dumps({"lockfileVersion": 1, "dependencies": { + "real": {"version": "1.0.0"}, "bogus": "juststring"}})) + r = manifest.parse(str(p)) + assert [x["name"] for x in r["packages"]] == ["real"] + assert [(s["raw"], s["reason"]) for s in r["skipped"]] == [("bogus", "not-a-lock-entry")] + + +# ---------------------------------------------------------------- pom.xml +def test_pom_namespaces_properties_and_dependency_management(): + """A real pom declares `xmlns`, so every ElementTree tag arrives as `{uri}dependency`. Properties + and are the two places a version legitimately lives elsewhere.""" + r = manifest.parse(f"{FIX}/pom.xml") + by = {p["name"]: p for p in r["packages"]} + assert by["org.apache.logging.log4j:log4j-core"]["version"] == "2.14.1" # ${log4j.version} + assert by["com.fasterxml.jackson.core:jackson-databind"]["version"] == "2.13.0" # managed + assert by["test.nimbus:nimbus-common"]["version"] == "2.4.0" # ${project.version} + assert by["junit:junit"]["scope"] == "test" + + +def test_an_unresolvable_property_is_refused_never_sent_as_a_literal(): + """`${spring.version}` with no entry is the pom equivalent of an unpinned + requirement, and OSV answers a literal `${...}` with 81 advisories rather than an error.""" + r = manifest.parse(f"{FIX}/pom.xml") + by = {s["raw"]: s for s in r["skipped"]} + assert by["org.springframework:spring-core"]["reason"] == "unresolved-property" + assert by["org.slf4j:slf4j-api"]["reason"] == "inherited-version" + assert not any("${" in p["version"] for p in r["packages"]) + + +def test_a_pom_declaring_a_dtd_is_refused(tmp_path): + """ElementTree expands internal entities and `defusedxml` is not a dependency. A dependency + manifest has no business declaring a DTD, so this is a refusal rather than a parse.""" + p = tmp_path / "pom.xml" + p.write_text(']>' + '') + with pytest.raises(manifest.ManifestError, match="DTD or entity"): + manifest.parse(str(p)) + + +def test_a_self_referential_property_terminates(tmp_path): + p = tmp_path / "pom.xml" + p.write_text('${b}' + 'ga' + '${a}' + .replace("", "")) + r = manifest.parse(str(p)) + assert not r["packages"] and r["skipped"][0]["reason"] == "unresolved-property" + + +# ---------------------------------------------------------------- kind detection & parse_all +@pytest.mark.parametrize("name,kind", [ + ("requirements.txt", "pip"), ("requirements-dev.txt", "pip"), + ("package-lock.json", "npm"), ("npm-shrinkwrap.json", "npm"), ("pom.xml", "maven"), +]) +def test_kind_is_detected_from_the_canonical_names(tmp_path, name, kind): + p = tmp_path / name + p.write_text("{}") + assert manifest.detect_kind(p) == kind + + +def test_an_unrecognised_name_is_sniffed_then_refused(tmp_path): + j = tmp_path / "deps.json" + j.write_text('{"lockfileVersion": 3, "packages": {}}') + assert manifest.detect_kind(j) == "npm" + x = tmp_path / "deps.xml" + x.write_text('') + assert manifest.detect_kind(x) == "maven" + bad = tmp_path / "notes.md" + bad.write_text("# hello") + with pytest.raises(manifest.ManifestError, match="cannot tell what kind"): + manifest.detect_kind(bad) + + +def test_one_broken_manifest_does_not_cost_the_others_their_scan(tmp_path): + bad = tmp_path / "pom.xml" + bad.write_text("") # truncated + res = manifest.parse_all([f"{FIX}/requirements.txt", str(bad)]) + assert res["packages"] and len(res["errors"]) == 1 + assert "well-formed" in res["errors"][0]["error"] + + +def test_a_missing_manifest_is_an_error_not_an_empty_result(): + res = manifest.parse_all(["/no/such/requirements.txt"]) + assert res["packages"] == [] and len(res["errors"]) == 1 + + +def test_an_oversized_manifest_is_refused(tmp_path, monkeypatch): + p = tmp_path / "requirements.txt" + p.write_text("flask==3.0.0\n") + monkeypatch.setattr(manifest, "MAX_BYTES", 4) + with pytest.raises(manifest.ManifestError, match="refusing to parse"): + manifest.parse(str(p)) + + +# =============================================================== osv: severity, versions, aliases +@pytest.mark.parametrize("vector,expect", [ + # v3 — every one of these is unchanged by adding the v4 branch (pinned again in test_inputs_cve) + ("CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", "critical"), + ("CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N", "high"), + ("CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N", "medium"), + ("CVSS:3.1/AV:L/AC:H/PR:H/UI:R/S:U/C:L/I:N/A:N", "low"), + # v4 — VC/VI/VA replace C/I/A, and AT is a second exploitability gate alongside AC + ("CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:N/SI:N/SA:N", "critical"), + ("CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N/E:U", "high"), + ("CVSS:4.0/AV:L/AC:H/PR:H/UI:A/AT:P/VC:L/VI:N/VA:N/SC:N/SI:N/SA:N", "low"), + ("", "medium"), ("garbage", "medium"), +]) +def test_severity_covers_cvss_v4_as_well_as_v3(vector, expect): + """Live, `aiohttp 3.9.1` returns 70 advisories carrying 43 CVSS_V4 scores against 32 CVSS_V3, + and 38 records publish a v4 vector and NOTHING else. Before this, every one of those fell + through the v3-only regex to the `medium` default whatever it said — including an + unauthenticated remote denial of service. H1 never noticed because one CVE at a time it is a + cosmetic mislabel; H2 filters hundreds of advisories on this value.""" + assert osv.severity_from_cvss(vector) == expect + + +def test_a_v4_only_advisory_is_no_longer_flattened_to_medium(): + v4 = "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N/E:U" + assert osv.severity_from_cvss(v4) == "high" # was "medium" for all 38 such records + + +@pytest.mark.parametrize("a,b", [ + ("1.10.0", "1.9.2"), # a string sort gets this backwards, and Log4Shell hangs on it + ("2.15.0", "2.14.1"), ("4.17.20", "4.17.19"), ("2.0", "2.0-beta9"), ("1.0.1", "1.0"), + ("10.0.0", "9.99.99"), ("2.3.1", "2.3"), +]) +def test_versions_order_numerically_not_lexically(a, b): + assert osv.version_gt(a, b) and not osv.version_gt(b, a) + + +def test_a_version_with_no_digits_is_declared_uncomparable(): + assert osv.comparable("2.14.1") and not osv.comparable("latest") + + +LOG4SHELL = { + "id": "GHSA-jfh8-c2jp-5v3q", "aliases": ["CVE-2021-44228"], "summary": "Log4Shell", + "details": "JNDI lookup in log messages", "severity": [ + {"type": "CVSS_V3", "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H"}], + "database_specific": {"cwe_ids": ["CWE-94"]}, + # Verbatim shape from the live record: several branches of the SAME package, plus a completely + # different package (pax-logging) whose fixed versions are numerically far lower. + "affected": [ + {"package": {"ecosystem": "Maven", "name": "org.apache.logging.log4j:log4j-core"}, + "ranges": [{"type": "ECOSYSTEM", "events": [{"introduced": "2.13.0"}, {"fixed": "2.15.0"}]}]}, + {"package": {"ecosystem": "Maven", "name": "org.apache.logging.log4j:log4j-core"}, + "ranges": [{"type": "ECOSYSTEM", "events": [{"introduced": "2.0-beta9"}, {"fixed": "2.3.1"}]}]}, + {"package": {"ecosystem": "Maven", "name": "org.apache.logging.log4j:log4j-core"}, + "ranges": [{"type": "ECOSYSTEM", "events": [{"introduced": "2.4"}, {"fixed": "2.12.2"}]}]}, + {"package": {"ecosystem": "Maven", "name": "org.ops4j.pax.logging:pax-logging-log4j2"}, + "ranges": [{"type": "ECOSYSTEM", "events": [{"introduced": "1.8.0"}, {"fixed": "1.9.2"}]}]}, + {"package": {"ecosystem": "Maven", "name": "org.ops4j.pax.logging:pax-logging-log4j2"}, + "ranges": [{"type": "ECOSYSTEM", "events": [{"introduced": "1.10.0"}, {"fixed": "1.10.8"}]}]}, + ], "references": []} + +# GHSA-p6mc-m468-83gw, verbatim shape: one advisory, several packages, and one of them has NO fix. +LODASH_MULTI = { + "id": "GHSA-p6mc-m468-83gw", "aliases": ["CVE-2020-8203"], "summary": "Prototype pollution", + "details": "…", "severity": [{"type": "CVSS_V3", + "score": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:H/A:N"}], + "database_specific": {"cwe_ids": ["CWE-1321"]}, + "affected": [ + {"package": {"ecosystem": "npm", "name": "lodash"}, + "ranges": [{"type": "SEMVER", "events": [{"introduced": "3.7.0"}, {"fixed": "4.17.19"}]}]}, + {"package": {"ecosystem": "npm", "name": "lodash-es"}, + "ranges": [{"type": "SEMVER", "events": [{"introduced": "3.7.0"}, {"fixed": "4.17.20"}]}]}, + {"package": {"ecosystem": "npm", "name": "lodash.pick"}, + "ranges": [{"type": "SEMVER", "events": [{"introduced": "4.0.0"}, + {"last_affected": "4.4.0"}]}]}, + ], "references": []} + + +def _norm(rec): + return osv.normalize_record(rec, [rec]) + + +def test_the_fix_is_for_the_package_you_actually_have(): + """Reproduced live against api.osv.dev on 2026-07-29: a manifest pinning + `org.ops4j.pax.logging:pax-logging-log4j2 1.10.0` is told by `upgrade_target()` to upgrade to + **2.15.0** — `log4j-core`'s Log4Shell fix, and not a version pax-logging has ever published. + `upgrade_target` takes the first `affected` row carrying a version, whatever package it names, + because it answers "what does this advisory fix" rather than "what should I install".""" + a = _norm(LOG4SHELL) + assert osv.upgrade_target(a)["package"] == "org.apache.logging.log4j:log4j-core" + assert osv.upgrade_target(a)["fixed_version"] == "2.15.0" # not pax-logging's fix + t = osv.upgrade_target_for(a, "org.ops4j.pax.logging:pax-logging-log4j2", "Maven", "1.10.0") + assert t["fixed_version"] == "1.10.8" and t["package"].startswith("org.ops4j") + + m = _norm(LODASH_MULTI) + assert osv.upgrade_target_for(m, "lodash", "npm", "4.17.15")["fixed_version"] == "4.17.19" + assert osv.upgrade_target_for(m, "lodash-es", "npm", "4.17.15")["fixed_version"] == "4.17.20" + + +def test_the_fix_is_the_one_for_your_branch_not_whichever_osv_listed_first(): + """One package, three maintenance branches. `upgrade_target` ignores the installed version, so + which branch it names is decided by document order — the live Log4Shell record happens to list + `2.13.0→2.15.0` first, so `2.14.1` gets the right answer by luck. Nothing in the OSV schema + promises that order. With the branches listed the other way round, the old selection recommends + **2.3.1** to someone running **2.14.1**: a downgrade, presented as the fix.""" + reordered = dict(LOG4SHELL, affected=[LOG4SHELL["affected"][1], # 2.0-beta9 -> 2.3.1 + LOG4SHELL["affected"][0], # 2.13.0 -> 2.15.0 + LOG4SHELL["affected"][2]]) + assert osv.upgrade_target(_norm(reordered))["fixed_version"] == "2.3.1" # the downgrade + t = osv.upgrade_target_for(_norm(reordered), "org.apache.logging.log4j:log4j-core", + "Maven", "2.14.1") + assert t["fixed_version"] == "2.15.0" # unchanged by the reordering + assert osv.upgrade_target_for(_norm(LOG4SHELL), "org.apache.logging.log4j:log4j-core", + "Maven", "2.14.1")["fixed_version"] == "2.15.0" + + +def test_a_package_with_no_published_fix_says_so_rather_than_borrowing_a_siblings(): + """`lodash.pick` is affected and has only `last_affected` — no fix was ever released for it. + Naming lodash's 4.17.19 would send someone to install a version of a package that has none.""" + t = osv.upgrade_target_for(_norm(LODASH_MULTI), "lodash.pick", "npm", "4.4.0") + assert t["fixed_version"] == "" and "no fixed version" in t["note"] + + +def test_an_installed_version_above_every_published_fix_is_not_told_to_downgrade(): + t = osv.upgrade_target_for(_norm(LOG4SHELL), "org.apache.logging.log4j:log4j-core", + "Maven", "2.99.0") + assert t["fixed_version"] == "" and "at or below the installed" in t["note"] + + +def test_an_uncomparable_installed_version_refuses_to_pick(): + t = osv.upgrade_target_for(_norm(LOG4SHELL), "org.apache.logging.log4j:log4j-core", + "Maven", "latest") + assert t["fixed_version"] == "" and "cannot order" in t["note"] + + +def test_pypi_names_match_across_pep503_spellings(): + rec = {"id": "X", "affected": [{"package": {"ecosystem": "PyPI", "name": "Flask-Login"}, + "ranges": [{"type": "ECOSYSTEM", + "events": [{"introduced": "0"}, + {"fixed": "0.6.3"}]}]}]} + assert osv.upgrade_target_for(_norm(rec), "flask-login", "PyPI", "0.6.2")["fixed_version"] \ + == "0.6.3" + + +def test_aliases_collapse_records_that_are_one_advisory(): + """Live, `aiohttp 3.9.1` returns 70 records that are 35 advisories: OSV publishes a GHSA and a + PYSEC entry for every one. Without collapsing them H2 pays two model calls per advisory and + emits two band-aids for one hole — and the duplicate carries a different id, so it does not + look like a duplicate.""" + recs = [{"id": "GHSA-a", "aliases": ["CVE-1", "PYSEC-1"]}, + {"id": "PYSEC-1", "aliases": ["CVE-1"]}, + {"id": "GHSA-b", "aliases": ["CVE-2"]}] + groups = osv.alias_groups(recs) + assert len(groups) == 2 + assert {len(g) for g in groups} == {2, 1} + + +def test_related_advisories_are_not_merged(): + """`related` links advisories that are ABOUT each other, not identical ones. Merging on it + would collapse distinct vulnerabilities into one finding.""" + recs = [{"id": "GHSA-a", "aliases": [], "related": ["GHSA-b"]}, {"id": "GHSA-b", "aliases": []}] + assert len(osv.alias_groups(recs)) == 2 + + +def test_the_preferred_record_is_the_one_carrying_an_installable_version(): + git_only = {"id": "CVE-1", "aliases": ["GHSA-x"], "summary": "s", "details": "d", + "affected": [{"ranges": [{"type": "GIT", + "events": [{"fixed": "24a6d64966d99182e95f5d3a29541ef2"}]}]}]} + versioned = {"id": "GHSA-x", "aliases": ["CVE-1"], "summary": "", "details": "", + "affected": [{"package": {"ecosystem": "PyPI", "name": "aiohttp"}, + "ranges": [{"type": "ECOSYSTEM", + "events": [{"introduced": "0"}, {"fixed": "3.9.2"}]}]}]} + assert osv.preferred_record([git_only, versioned])["id"] == "GHSA-x" + + +def test_normalize_record_merges_the_whole_alias_group(): + group = [{"id": "GHSA-x", "aliases": ["CVE-1"], "summary": "sum", "details": "det", + "severity": [], "database_specific": {"cwe_ids": ["CWE-79"]}, + "affected": [{"package": {"ecosystem": "PyPI", "name": "p"}, + "ranges": [{"type": "ECOSYSTEM", + "events": [{"introduced": "0"}, {"fixed": "2.0"}]}]}]}, + {"id": "PYSEC-1", "aliases": ["CVE-1"], "summary": "", "details": "", + "severity": [{"type": "CVSS_V3", + "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H"}], + "affected": []}] + a = osv.normalize_record(group[0], group) + assert a["cvss"].startswith("CVSS:3.1") # the score lives on the sibling record + assert set(a["consulted"]) == {"GHSA-x", "PYSEC-1"} + assert a["cwe_ids"] == ["CWE-79"] + + +# =============================================================== the batch query +class _Resp: + def __init__(self, payload, status=200): + self._p, self.status_code, self.text = payload, status, json.dumps(payload) + + def json(self): + return self._p + + def raise_for_status(self): + if self.status_code != 200: + raise RuntimeError(f"HTTP {self.status_code}") + + +def test_batch_asks_once_and_only_the_hits_are_fetched_in_full(monkeypatch): + """`querybatch` returns ids and `modified` only — never the advisory bodies. Fetching each id + would be one request per ADVISORY (70 for aiohttp 3.9.1 alone), where `/v1/query` returns every + full record for a coordinate in one round trip. So batch answers "which packages are worth + asking about" and nothing else.""" + import httpx + calls = [] + + def post(url, json=None, **kw): + calls.append(url) + if url.endswith("querybatch"): + return _Resp({"results": [{"vulns": [{"id": "GHSA-a", "modified": "x"}]}, {}]}) + return _Resp({"vulns": [dict(LODASH_MULTI)]}) + monkeypatch.setattr(httpx, "post", post) + pkgs = [{"ecosystem": "npm", "name": "lodash", "version": "4.17.15"}, + {"ecosystem": "npm", "name": "clean", "version": "1.0.0"}] + hits = osv.query_batch(pkgs, log=lambda m: None) + assert hits["npm/lodash@4.17.15"] == ["GHSA-a"] and hits["npm/clean@1.0.0"] == [] + assert calls == [osv.BATCH_API] # one request for both packages + + +def test_an_unknown_ecosystem_is_dropped_before_the_batch_not_sent(monkeypatch): + """Measured live: ONE invalid ecosystem fails the WHOLE batch with + `error in query at index 1: invalid ecosystem`. A typo in one manifest line must not cost every + other package in the manifest its answer.""" + import httpx + sent = [] + + def post(url, json=None, **kw): + sent.append(json) + return _Resp({"results": [{"vulns": []}]}) + monkeypatch.setattr(httpx, "post", post) + hits = osv.query_batch([{"ecosystem": "npm", "name": "ok", "version": "1.0"}, + {"ecosystem": "Bogus", "name": "bad", "version": "1.0"}], + log=lambda m: None) + assert [q["package"]["name"] for q in sent[0]["queries"]] == ["ok"] + assert hits["Bogus/bad@1.0"] == [] + + +def test_a_failed_batch_degrades_to_per_package_queries_never_to_zero(monkeypatch): + """Reporting "no advisories" because a request failed is the one answer that must never come + out of an error.""" + import httpx + + def post(url, json=None, **kw): + if url.endswith("querybatch"): + raise RuntimeError("connection reset") + return _Resp({"vulns": [{"id": "GHSA-a"}]}) + monkeypatch.setattr(httpx, "post", post) + hits = osv.query_batch([{"ecosystem": "npm", "name": "lodash", "version": "4.17.15"}], + log=lambda m: None) + assert hits["npm/lodash@4.17.15"] == ["GHSA-a"] + + +def test_one_failing_fallback_query_does_not_end_the_scan(monkeypatch): + """The fallback exists so a batch failure never loses a chunk — unguarded, it became the thing + that lost the whole run. `query_package` raises on any non-200, and the causes of a batch + failure (OSV down, rate limiting) are exactly the causes that make the retries fail too. It + propagated out of `query_batch`, out of `deps.survey` (whose own loop IS guarded) and out of + `resolve_manifests`, which the pipeline does not wrap — so a 429 also destroyed the repo half + of a `scan ./repo --manifest` run.""" + import httpx + + def post(url, json=None, **kw): + if url.endswith("querybatch"): + raise RuntimeError("batch down") + if json["package"]["name"] == "boom": + raise RuntimeError("OSV.dev returned 429") + return _Resp({"vulns": [{"id": "GHSA-ok"}]}) + monkeypatch.setattr(httpx, "post", post) + pkgs = [{"ecosystem": "npm", "name": "boom", "version": "1.0"}, + {"ecosystem": "npm", "name": "fine", "version": "1.0"}] + hits = osv.query_batch(pkgs, log=lambda m: None) + assert hits["npm/fine@1.0"] == ["GHSA-ok"] # the second package is still reached + assert "npm/boom@1.0" not in hits # never "clean"… + assert "429" in pkgs[0]["error"] # …it is stamped as unchecked + + +def test_a_package_that_could_not_be_checked_is_not_reported_clean(tmp_path): + """It stayed in `vulnerable` (so the funnel counted it) but contributed no advisory row, no + counter and no warning — indistinguishable from a package that came back clean.""" + surveyed = {"files": [], "errors": [], "skipped": [], "dev_excluded": [], "considered": [1], + "vulnerable": [1], "candidates": [], "records": 0, "_groups": {}, + "packages": [{"ecosystem": "PyPI", "name": "aiohttp", "version": "3.9.1", + "error": "OSV.dev returned 429"}, + {"ecosystem": "PyPI", "name": "flask", "version": "3.0.0"}]} + rep = deps.report(surveyed, [], min_severity="high", max_advisories=25, include_dev=False) + assert rep["funnel"]["packages_unchecked"] == 1 + assert [u["name"] for u in rep["unchecked"]] == ["aiohttp"] + from vpcopilot.report import _dependencies_html + (tmp_path / "dependencies.json").write_text(json.dumps(rep)) + html = _dependencies_html(str(tmp_path)) + assert "Could not check" in html and "aiohttp" in html and "unknown, not absent" in html + + +def test_a_short_batch_answer_is_never_read_as_clean(monkeypatch): + """`zip(..., strict=False)` refuses to mispair a short `results` list with its queries — but it + also left the unanswered tail with no entry at all, and the caller reads a missing key as "no + advisories" (`hits.get(key)` is None, so the package never reaches `vulnerable`). A 200 that + answered two of three coordinates therefore reported the third as CLEAN.""" + import httpx + asked = [] + + def post(url, json=None, **kw): + if url.endswith("querybatch"): + return _Resp({"results": [{"vulns": []}, {"vulns": []}]}) # two answers, three asked + asked.append(json["package"]["name"]) + return _Resp({"vulns": [{"id": "GHSA-tail"}]}) + monkeypatch.setattr(httpx, "post", post) + pkgs = [{"ecosystem": "npm", "name": "a", "version": "1.0"}, + {"ecosystem": "npm", "name": "b", "version": "1.0"}, + {"ecosystem": "npm", "name": "c", "version": "1.0"}] + hits = osv.query_batch(pkgs, log=lambda m: None) + assert asked == ["c"] # only the unanswered one is re-asked + assert hits["npm/c@1.0"] == ["GHSA-tail"] # was: absent, i.e. indistinguishable from clean + assert set(hits) == {"npm/a@1.0", "npm/b@1.0", "npm/c@1.0"} + + +# =============================================================== the funnel +class FakeProfile: + def __init__(self, observable=True, **kw): + from vpcopilot.schemas import Severity, VulnClass + self.network_observable = observable + self.reason = kw.get("reason", "the advisory names the request path") + self.vuln_class = VulnClass("other") + self.severity = Severity("high") + self.title, self.description = "t", "d" + self.exploit_sketch = "GET /x" + self.paths = kw.get("paths", ["/api/pay"]) + self.http_methods = ["POST"] + self.parameters = self.headers = self.example_requests = self.caveats = [] + self.confidence = 0.8 + + +def _cand(pkg, aid, sev="high", **kw): + return {"advisory_id": aid, "aliases": [], "advisory": {"id": aid, "cwe_ids": []}, + "package": pkg, "ecosystem": "PyPI", "installed": "1.0", "scope": "runtime", + "manifest": "requirements.txt", "summary": "s", "cvss": "", "severity": sev, + "withdrawn": kw.get("withdrawn", ""), "fixed_version": "2.0", "fix_note": "", + "vulnerable_range": "1.0", "references": [], "disposition": "", "reason": ""} + + +def test_the_cap_is_shared_across_packages_not_eaten_by_the_first(): + """The first live run made the reason obvious: `aiohttp 3.9.1` alone carries 35 advisories, and + a flat (severity, package, id) ordering handed it 23 of the 25 slots. Every other package was + competing with one dependency's back catalogue, and packages later in the alphabet were capped + out by advisories no worse than their own.""" + cands = ([_cand("aiohttp", f"GHSA-a{i}") for i in range(30)] + + [_cand("django", "GHSA-d1"), _cand("lodash", "GHSA-l1")]) + chosen = deps.select(cands, min_severity="high", max_advisories=6, log=lambda m: None) + assert {c["package"] for c in chosen} == {"aiohttp", "django", "lodash"} + assert sum(1 for c in chosen if c["package"] == "aiohttp") == 4 + + +def test_severity_still_outranks_breadth(): + cands = [_cand("aiohttp", "GHSA-crit", "critical"), _cand("django", "GHSA-low", "low")] + chosen = deps.select(cands, min_severity="high", max_advisories=1, log=lambda m: None) + assert [c["advisory_id"] for c in chosen] == ["GHSA-crit"] + assert cands[1]["disposition"] == "below_severity" + + +def test_nothing_is_dropped_silently_everything_carries_a_reason(): + """"We did not look at this" and "this is fine" must never render the same way.""" + cands = [_cand("a", "GHSA-1", "medium"), _cand("b", "GHSA-2", withdrawn="2026-01-01"), + _cand("c", "GHSA-3"), _cand("d", "GHSA-4")] + deps.select(cands, min_severity="high", max_advisories=1, log=lambda m: None) + by = {c["advisory_id"]: c for c in cands} + assert by["GHSA-1"]["disposition"] == "below_severity" + assert by["GHSA-2"]["disposition"] == "withdrawn" + assert {by["GHSA-3"]["disposition"], by["GHSA-4"]["disposition"]} == {"", "capped"} + assert all(c["reason"] for c in cands if c["disposition"]) + + +def test_a_cap_of_zero_means_no_cap(): + cands = [_cand("a", f"GHSA-{i}") for i in range(40)] + assert len(deps.select(cands, min_severity="high", max_advisories=0, log=lambda m: None)) == 40 + + +def test_one_advisory_hitting_two_packages_costs_one_model_call(monkeypatch): + """The profile answers "what does this look like in a request", which does not depend on which + of the affected packages you have. Two packages must not receive two different verdicts.""" + from vpcopilot.agents import resolve as ra + calls = [] + monkeypatch.setattr(ra, "run", lambda h, a: calls.append(a["id"]) or FakeProfile()) + chosen = [_cand("lodash", "GHSA-x"), _cand("lodash-es", "GHSA-x")] + deps.resolve_all(None, chosen, {}, log=lambda m: None) + assert calls == ["GHSA-x"] + assert all(c["disposition"] == "exploitable" for c in chosen) + + +def test_two_packages_one_advisory_get_distinct_finding_ids(monkeypatch): + from vpcopilot.agents import resolve as ra + monkeypatch.setattr(ra, "run", lambda h, a: FakeProfile()) + chosen = [_cand("lodash", "GHSA-x"), _cand("lodash-es", "GHSA-x")] + deps.resolve_all(None, chosen, {}, log=lambda m: None) + built = deps.build(chosen) + ids = [f.id for f in built["findings"]] + assert ids == ["GHSA-x", "GHSA-x-2"] and len(set(ids)) == 2 + assert len(built["remediations"]) == 2 + + +def test_a_declined_advisory_routes_to_no_bandaid_in_code(monkeypatch): + """Same guarantee H1 makes, for the same reason: a hard acceptance requirement must not depend + on a model honouring a prompt.""" + from vpcopilot.agents import resolve as ra + monkeypatch.setattr(ra, "run", lambda h, a: FakeProfile( + observable=False, reason="exploitation needs a crafted file on disk.")) + chosen = [_cand("aiohttp", "GHSA-x")] + deps.resolve_all(None, chosen, {}, log=lambda m: None) + built = deps.build(chosen) + d = built["decisions"][built["findings"][0].id] + assert d.no_bandaid is True and d.bandaids == [] + assert "crafted file" in d.residual_risk and "2.0" in d.residual_risk + + +def test_a_failing_resolve_loses_that_advisory_not_the_manifest(monkeypatch): + from vpcopilot.agents import resolve as ra + + def boom(h, a): + if a["id"] == "GHSA-bad": + raise RuntimeError("model timeout") + return FakeProfile() + monkeypatch.setattr(ra, "run", boom) + chosen = [_cand("a", "GHSA-bad"), _cand("b", "GHSA-ok")] + deps.resolve_all(None, chosen, {}, log=lambda m: None) + assert chosen[0]["disposition"] == "resolve_failed" + assert chosen[1]["disposition"] == "exploitable" + assert [f.id for f in deps.build(chosen)["findings"]] == ["GHSA-ok"] + + +def test_the_cure_is_a_version_bump_and_never_a_patch(monkeypatch): + from vpcopilot.agents import resolve as ra + monkeypatch.setattr(ra, "run", lambda h, a: FakeProfile()) + chosen = [_cand("aiohttp", "GHSA-x")] + deps.resolve_all(None, chosen, {}, log=lambda m: None) + rem = list(deps.build(chosen)["remediations"].values())[0] + assert rem.kind == "dependency_upgrade" and rem.fixed_version == "2.0" + assert rem.patched_content == "" and rem.diff == "" and rem.file == "" + assert "upstream" in rem.pr_body and "requirements.txt" in rem.pr_body + + +def test_the_finding_identity_names_the_installed_coordinate(monkeypatch): + """Two packages hit by one advisory must not collide on a coverage key, and `source` is what + `coverage_key` falls back to when there is no repo file and no endpoint.""" + from vpcopilot.agents import resolve as ra + from vpcopilot.correlate import coverage_key + monkeypatch.setattr(ra, "run", lambda h, a: FakeProfile(paths=[])) + chosen = [_cand("lodash", "GHSA-x"), _cand("lodash-es", "GHSA-x")] + deps.resolve_all(None, chosen, {}, log=lambda m: None) + fs = deps.build(chosen)["findings"] + assert fs[0].source == "osv:GHSA-x@PyPI/lodash@1.0" + assert fs[0].file == "" and fs[0].line == 0 + assert coverage_key("service_policy", "", identity=fs[0].source) != \ + coverage_key("service_policy", "", identity=fs[1].source) + + +# ================================= a finding whose recommended band-aid was claimed by another +def _decision(fid, *bandaids): + from vpcopilot.schemas import BandaidOption, TriageDecision + return TriageDecision(finding_id=fid, bandaids=[ + BandaidOption(control=c, coverage="partial", recommended=r, rationale="r") + for c, r in bandaids]) + + +def _finding(fid, endpoint): + from vpcopilot.schemas import Finding + return Finding(id=fid, title=fid, vuln_class="other", severity="critical", description="d", + exploit_sketch="e", source=f"osv:{fid}", endpoint=endpoint) + + +def _run_generate_loop(monkeypatch, decisions, findings): + """Drive the pipeline's generate stage over prepared triage decisions, with every agent faked.""" + from vpcopilot import pipeline + from vpcopilot.agents import generate, probe as probe_agent, triage + from vpcopilot.schemas import GeneratedArtifact, GeneratedArtifacts, TriageBatch + + made = [] + + def gen(h, f, control, rationale, exploit=None, legit=None): + made.append((f.id, control.value)) + return GeneratedArtifacts(items=[GeneratedArtifact( + finding_id=f.id, control=control, policy_name=f"{control.value}-{f.id}", + spec={"rules": []})]) + monkeypatch.setattr(generate, "run", gen) + monkeypatch.setattr(triage, "run", lambda h, fs: TriageBatch(decisions=decisions)) + monkeypatch.setattr(probe_agent, "run", lambda *a, **k: (_ for _ in ()).throw(RuntimeError("x"))) + monkeypatch.setattr(pipeline, "_dedup_findings", lambda fs, log, c=None: fs) + return made + + +def test_a_finding_whose_only_recommended_control_was_taken_still_gets_a_bandaid(monkeypatch, + tmp_path): + """The first live H2 run: Log4Shell's only recommended control was the LB-wide `waf`, which an + aiohttp header-injection advisory had already claimed. The loop ended with nothing generated for + the critical finding, while `correlations.json` recorded it as covered — and the WAF that + actually shipped (`waf-block-header-injection-nullbyte-crlf`) addressed nothing about JNDI.""" + from vpcopilot import correlate + seen = {"waf": "aiohttp-adv"} + d = _decision("log4shell", ("waf", True), ("service_policy", False)) + preferred = [b for b in d.bandaids if b.recommended] + assert [b.control.value for b in preferred] == ["waf"] + assert correlate.coverage_key("waf", "", identity="/x") in seen # the slot is taken + # …and the alternative is NOT taken, so there is something to fall back to + alt = [b for b in d.bandaids if not b.recommended] + assert correlate.coverage_key(alt[0].control.value, "", identity="/x") not in seen + + +def test_the_fallback_only_fires_when_the_recommended_tier_produced_nothing(monkeypatch, tmp_path): + """No regression: where a recommended band-aid is generated normally, the alternatives are + never reached and the artifact set is exactly what it was before.""" + from vpcopilot.pipeline import run_pipeline + made = _run_generate_loop( + monkeypatch, + [_decision("f1", ("service_policy", True), ("rate_limit", False))], + [_finding("f1", "/api/pay")]) + from vpcopilot.inputs import deps as D + monkeypatch.setattr(D, "resolve_manifests", lambda h, p, **k: { + "findings": [_finding("f1", "/api/pay")], "decisions": {}, "remediations": {}, + "report": {"funnel": {}}, "candidates": []}) + from vpcopilot.harness import Harness + monkeypatch.setattr(Harness, "__init__", lambda self, cp=None: None) + monkeypatch.setattr(Harness, "warmup", lambda self: None) + run_pipeline(manifest_paths=["x.txt"], out_dir=str(tmp_path), draft_code_fixes=False, + log=lambda m: None) + assert made == [("f1", "service_policy")] # the rate_limit alternative was never reached + + +def test_the_fallback_fires_when_every_recommended_control_was_claimed(monkeypatch, tmp_path): + from vpcopilot.pipeline import run_pipeline + fs = [_finding("owner", "/a"), _finding("loser", "/b")] + made = _run_generate_loop( + monkeypatch, + [_decision("owner", ("waf", True)), + _decision("loser", ("waf", True), ("service_policy", False))], + fs) + from vpcopilot.inputs import deps as D + monkeypatch.setattr(D, "resolve_manifests", lambda h, p, **k: { + "findings": fs, "decisions": {}, "remediations": {}, "report": {"funnel": {}}, + "candidates": []}) + from vpcopilot.harness import Harness + monkeypatch.setattr(Harness, "__init__", lambda self, cp=None: None) + monkeypatch.setattr(Harness, "warmup", lambda self: None) + run_pipeline(manifest_paths=["x.txt"], out_dir=str(tmp_path), draft_code_fixes=False, + log=lambda m: None) + assert ("owner", "waf") in made + assert ("loser", "service_policy") in made # was: nothing at all + assert ("loser", "waf") not in made # the LB-wide slot is still shared, not duplicated + cor = json.loads((tmp_path / "correlations.json").read_text()) + assert cor[0]["finding_id"] == "loser" and cor[0]["covered_by"] == "owner" + assert "not this one's" in cor[0]["note"] # the claim is qualified, not absolute + + +def test_an_alternate_never_takes_a_slot_a_later_finding_recommends(monkeypatch, tmp_path): + """Found by adversarial review, before shipping. The first cut of the fallback ran INTERLEAVED + with the recommended tier and generated EVERY alternative, so `loser`'s non-recommended + `rate_limit` claimed the LB-wide slot before `brute` — which actually recommended rate_limit — + was reached. `brute` then got no band-aid at all and was recorded as covered by a policy built + from `loser`'s exploit: the exact Log4Shell bug this change exists to fix, moved onto a + different finding, plus two extra model calls for controls triage never recommended. + + Six of the seven controls are LB-wide, so contention is the normal case, not a corner.""" + from vpcopilot.harness import Harness + from vpcopilot.inputs import deps as D + from vpcopilot.pipeline import run_pipeline + fs = [_finding("owner", "/a"), _finding("loser", "/b"), _finding("brute", "/c")] + made = _run_generate_loop(monkeypatch, [ + _decision("owner", ("waf", True)), + _decision("loser", ("waf", True), ("rate_limit", False), ("bot_defense", False)), + _decision("brute", ("rate_limit", True)), + ], fs) + monkeypatch.setattr(D, "resolve_manifests", lambda h, p, **k: { + "findings": fs, "decisions": {}, "remediations": {}, "report": {"funnel": {}}, + "candidates": []}) + monkeypatch.setattr(Harness, "__init__", lambda self, cp=None: None) + monkeypatch.setattr(Harness, "warmup", lambda self: None) + run_pipeline(manifest_paths=["x.txt"], out_dir=str(tmp_path), draft_code_fixes=False, + log=lambda m: None) + assert ("brute", "rate_limit") in made # was: stolen by loser's alternate, brute got nothing + assert ("owner", "waf") in made + # the fallback takes ONE alternative, not the whole non-recommended stack + assert [m for m in made if m[0] == "loser"] == [("loser", "bot_defense")] + assert {f.id for f in fs} - {m[0] for m in made} == set() # nobody ends bare + + +def test_an_lb_wide_correlation_says_whose_exploit_the_policy_was_built_from(monkeypatch, tmp_path): + """"One control covers both" is true of the slot and false of the protection: the attached + policy was generated against the owning finding's exploit and validated against its probe.""" + from vpcopilot.pipeline import run_pipeline + fs = [_finding("owner", "/a"), _finding("loser", "/a")] + _run_generate_loop(monkeypatch, + [_decision("owner", ("service_policy", True)), + _decision("loser", ("service_policy", True))], fs) + from vpcopilot.inputs import deps as D + monkeypatch.setattr(D, "resolve_manifests", lambda h, p, **k: { + "findings": fs, "decisions": {}, "remediations": {}, "report": {"funnel": {}}, + "candidates": []}) + from vpcopilot.harness import Harness + monkeypatch.setattr(Harness, "__init__", lambda self, cp=None: None) + monkeypatch.setattr(Harness, "warmup", lambda self: None) + run_pipeline(manifest_paths=["x.txt"], out_dir=str(tmp_path), draft_code_fixes=False, + log=lambda m: None) + cor = json.loads((tmp_path / "correlations.json").read_text()) + # service_policy is endpoint-scoped, not LB-wide: same endpoint really is one policy for both + assert cor[0]["note"] == "same endpoint — one policy covers both" + + +# =============================================================== the pipeline and both surfaces +def test_the_pipeline_refuses_a_missing_or_conflicting_input(): + from vpcopilot.pipeline import run_pipeline + with pytest.raises(ValueError, match="pass a repo path"): + run_pipeline() + with pytest.raises(ValueError, match="cannot be combined"): + run_pipeline(advisory="CVE-2024-23334", manifest_paths=["requirements.txt"]) + # a manifest IS additive with a repo and a spec — no error path for that combination + from vpcopilot.pipeline import run_pipeline as rp + assert rp.__defaults__ is not None + + +def test_the_manifest_artifact_is_not_called_manifest_json(): + """`export.verify_bundle` locates each run in an archive by every member whose name ends + `manifest.json`. An out-dir artifact by that name would be read as a second evidence manifest + and break verification of the bundle it ships in.""" + from vpcopilot import export + src = __import__("pathlib").Path("src/vpcopilot/pipeline.py").read_text() + assert '"dependencies.json"' in src + assert 'out / "manifest.json"' not in src + assert "dependencies.json" in export.BUNDLE_FILES + + +def test_the_bundle_caveats_say_what_the_dependency_report_does_not_prove(): + from vpcopilot.export import build_manifest + cav = " ".join(build_manifest("out").get("caveats", [])) + assert "dependencies.json" in cav and "never queried" in cav + + +def test_both_surfaces_reach_the_same_module_function(): + """One module function, two surfaces — a guard that lives in only one of them is not a guard.""" + import inspect + + from vpcopilot import cli + from vpcopilot.console import app as A + assert "survey_report" in inspect.getsource(cli.deps) + assert "survey_report" in inspect.getsource(A.survey_dependencies) + assert "manifest_paths" in inspect.getsource(A._run_scan) + + +def test_the_console_scan_surface_accepts_a_manifest(): + from fastapi.testclient import TestClient + + from vpcopilot.console import app as A + c = TestClient(A.app) + assert c.post("/api/scan", json={"repo": "", "cve": "", "manifest": []}).status_code == 400 + assert c.post("/api/scan", json={"repo": "", "cve": "CVE-2024-23334", + "manifest": ["requirements.txt"]}).status_code == 400 + assert c.post("/api/scan", json={"manifest": ["r.txt"], + "min_severity": "nope"}).status_code == 400 + assert c.post("/api/deps", json={"manifest": []}).status_code == 400 + html = (__import__("pathlib").Path(A.__file__).parent / "static" / "index.html").read_text() + assert 'id="scanManifest"' in html and "manifest," in html and "surveyDeps" in html + # Both surfaces or it is not a guard: the CLI printed parse errors and unchecked packages in + # red while the preview rendered an all-zero funnel and an empty table for them. + assert "d.errors||[]" in html and "d.unchecked||[]" in html + assert "UNKNOWN, not absent" in html + # …and a cleared advisory cap must mean the CLI default, not "no cap" + assert "parseInt(scanMaxAdvisories.value)||0" not in html + assert "advCap(scanMaxAdvisories.value)" in html and "const advCap" in html + + +def test_the_deps_endpoint_is_read_only_and_needs_no_model(monkeypatch, tmp_path): + """It parses and asks OSV. A version that reached a model would make the preview cost what the + scan costs, and the point of the preview is seeing the funnel before paying for it.""" + from fastapi.testclient import TestClient + + from vpcopilot.console import app as A + monkeypatch.setattr(osv, "query_batch", lambda pkgs, **kw: {}) + monkeypatch.setattr(A, "_active_config", None, raising=False) + from vpcopilot.harness import Harness + monkeypatch.setattr(Harness, "__init__", + lambda *a, **k: pytest.fail("the preview must not build a Harness")) + c = TestClient(A.app) + r = c.post("/api/deps", json={"manifest": [f"{FIX}/requirements.txt"]}) + assert r.status_code == 200 + d = r.json()["dependencies"] + assert d["funnel"]["packages_parsed"] == 7 and d["funnel"]["packages_unpinned"] == 7 + assert d["unpinned"] and all(u["reason"] for u in d["unpinned"]) + + +def test_the_report_renders_what_was_not_checked(tmp_path): + from vpcopilot.report import _dependencies_html + (tmp_path / "dependencies.json").write_text(json.dumps({ + "funnel": {"packages_parsed": 3, "packages_vulnerable": 1, "advisories_distinct": 5, + "exploitable": 1, "declined": 1, "not_resolved": 3, "packages_unpinned": 2}, + "settings": {"min_severity": "high", "max_advisories": 25}, + "unpinned": [{"reason": "unpinned"}, {"reason": "unresolved-property"}], + "advisories": [{"severity": "high", "package": "aiohttp", "installed": "3.9.1", + "advisory_id": "GHSA-x", "fixed_version": "3.9.2", + "disposition": "exploitable", "reason": "names the path"}], + "errors": []})) + html = _dependencies_html(str(tmp_path)) + assert "Not checked" in html and "never queried" in html + assert "Listed, not resolved" in html and "GHSA-x" in html + assert _dependencies_html(str(tmp_path / "empty")) == "" # absent -> no section at all + + +# ================================= an upgrade is not a drafted PR +def _upgrade(fid): + from vpcopilot.schemas import RemediationPlan + return RemediationPlan(finding_id=fid, kind="dependency_upgrade", + summary=f"upgrade aiohttp 3.9.1 → 3.9.2 (fixes {fid})", + package="aiohttp", ecosystem="PyPI", + pr_title="fix(aiohttp): upgrade", pr_body="body") + + +def _code_fix(fid): + from vpcopilot.schemas import RemediationPlan + return RemediationPlan(finding_id=fid, summary="add a guard", file="app.py", + diff="--- a\n+++ b\n", patched_content="def f(): pass\n", + pr_title="fix: guard", pr_body="body") + + +def _pipeline_with(monkeypatch, tmp_path, findings, remediations): + from vpcopilot.harness import Harness + from vpcopilot.inputs import deps as D + from vpcopilot.pipeline import run_pipeline + _run_generate_loop(monkeypatch, [_decision(f.id, ("service_policy", True)) for f in findings], + findings) + monkeypatch.setattr(D, "resolve_manifests", lambda h, p, **k: { + "findings": findings, "decisions": {}, "remediations": remediations, + "report": {"funnel": {}}, "candidates": []}) + monkeypatch.setattr(Harness, "__init__", lambda self, cp=None: None) + monkeypatch.setattr(Harness, "warmup", lambda self: None) + run_pipeline(manifest_paths=["x.txt"], out_dir=str(tmp_path), draft_code_fixes=False, + log=lambda m: None) + return json.loads((tmp_path / "summary.json").read_text()) + + +def test_a_dependency_upgrade_is_never_counted_as_a_drafted_code_fix_pr(monkeypatch, tmp_path): + """The first live H2 run reported `code_fix_prs: 6` for six advisory findings, and the report's + hero panel rendered "6 — code-fix PRs (the cure)". Zero PRs were drafted and none CAN be: the + cure is a version bump in someone else's package, which is why `remediate` is never called and + `pr.py` declines. The cure half of the story was overstated on every surface that renders it.""" + fs = [_finding("adv-1", "/a"), _finding("adv-2", "/b")] + summary = _pipeline_with(monkeypatch, tmp_path, fs, + {"adv-1": _upgrade("adv-1"), "adv-2": _upgrade("adv-2")}) + assert summary["code_fix_prs"] == [] # was: both of them + assert summary["dependency_upgrades"] == ["adv-1", "adv-2"] + counts = json.loads((tmp_path / "run.json").read_text())["counts"] + assert counts["code_fix_prs"] == 0 and counts["dependency_upgrades"] == 2 + metrics = json.loads((tmp_path / "metrics.json").read_text())["synthesize"] + assert metrics["code_fix_prs"] == 0 and metrics["dependency_upgrades"] == 2 + # …and the number a human actually reads, in the report's impact hero + from vpcopilot.impact import impact + im = impact(str(tmp_path)) + assert im["code_prs"] == 0 and im["dependency_upgrades"] == 2 + html = (tmp_path / "report.html").read_text() + assert "upgrades to ship (no PR)" in html + assert ">2code-fix PRs (the cure)" not in html + + +def test_a_later_scan_without_a_manifest_does_not_inherit_the_previous_dependencies(monkeypatch, + tmp_path): + """Every other member of the out dir is rewritten unconditionally, so a re-scan replaces it. + `dependencies.json` was written only when a manifest was given — so scanning the same out dir + again without one left the PREVIOUS run's dependency data in place, and the report, + `GET /api/deps` and the evidence bundle all presented it as this run's. G4 shipped the same + class of bug with leftover `policies/` artifacts.""" + fs = [_finding("adv-1", "/a")] + _pipeline_with(monkeypatch, tmp_path, fs, {"adv-1": _upgrade("adv-1")}) + assert (tmp_path / "dependencies.json").is_file() + + # …now a scan of the same out dir with no manifest at all + from vpcopilot.pipeline import _write_out + _write_out(str(tmp_path), [], [], [], [], [], [], [], deps_report=None) + assert not (tmp_path / "dependencies.json").exists() + from vpcopilot.report import _dependencies_html + assert _dependencies_html(str(tmp_path)) == "" + + +def test_a_drafted_code_fix_still_counts_as_one(monkeypatch, tmp_path): + """No regression: a real patch is still a code-fix PR, and the new key stays empty rather than + absent, so a consumer can tell "none" from "this run predates the split".""" + fs = [_finding("code-1", "/a")] + summary = _pipeline_with(monkeypatch, tmp_path, fs, {"code-1": _code_fix("code-1")}) + assert summary["code_fix_prs"] == ["code-1"] + assert summary["dependency_upgrades"] == [] + counts = json.loads((tmp_path / "run.json").read_text())["counts"] + assert counts["code_fix_prs"] == 1 and counts["dependency_upgrades"] == 0 + # the extra hero stat and chip are omitted entirely when nothing was held back + assert "upgrades to ship" not in (tmp_path / "report.html").read_text()