Skip to content

H3 — OpenAPI spec as a discovery input - #13

Merged
henleda merged 1 commit into
mainfrom
feat-h3-openapi-input
Jul 29, 2026
Merged

H3 — OpenAPI spec as a discovery input#13
henleda merged 1 commit into
mainfrom
feat-h3-openapi-input

Conversation

@henleda

@henleda henleda commented Jul 29, 2026

Copy link
Copy Markdown
Owner

A4 goes one way: you hand XC a schema and it enforces it. This is the other direction — read the spec and find the flaws in it. A spec is a security artifact whether or not anyone treats it as one, and it is often the only thing you have for a service you do not own.

vpcopilot scan --spec ./openapi.yaml            # the contract alone
vpcopilot scan ./app --spec ./openapi.yaml      # …and the drift between them

--spec is additive, unlike --cve: alone it scans the contract, alongside a repo it also compares the two (which needs both inputs).

Split by what needs judgement

A deterministic pass finds what the document leaves unstated — a number with no minimum, a string with no maxLength, an operation with no security, additionalProperties left open, $ref followed with a cycle bound. Those are facts about the document, so recall does not depend on which model is configured.

The agent decides which of them matter. An unbounded page is nothing; an unbounded amount on a transfer is a business-logic hole. That is the only part a model should be making.

The spec goes to the existing discover agent as one more file rather than through a fourth agent — its whole job is "read this and tell me what is wrong with it", and routing the spec through the existing stage means these findings verify, triage and generate exactly like code findings, with no parallel path to keep in step.

Acceptance, verified live

criterion result
spec declaring amount with no lower bound → finding → api_schema neg-amount-001service_policy + api_schema
spec endpoints with no code match → undocumented_or_orphaned ✓, and the reverse direction too
findings flow into the same triage table

Two real bugs the first live run surfaced

A spec is one file declaring many endpoints. endpoint_of("pay-api.yaml") returned the same string for every finding, so all six collapsed onto one service_policy coverage key and five were logged "already covered"generating no band-aid at all. Spec findings now carry source and leave file empty, and the coverage identity prefers endpoint. Three endpoints → three keys, and only the two genuinely on /api/v1/transfer collapse, which is what B1 is for.

The orphan finding was going through verify. That agent's entire method is reading the offending source; with no source to read it refuted the finding at 0.10 confidence and dropped it. It is appended after verify now — a comparison of two documents is not a claim about code.

Drift is reported in both directions

They fail differently, so both are stated:

  • declared but unserved — dead documentation, or a shadow API removed from one place and not the other
  • served but undeclared — the one that bites: applying an api_schema band-aid built from this spec would start rejecting those routes

run.json records input_kind as repo / spec / repo+spec / advisory.

Tests

502 passing, ruff clean. Fixture at bench/fixtures/specs/pay-api.yaml; demo fixtures unchanged.

🤖 Generated with Claude Code

A4 goes one way: you hand XC a schema and it enforces it. This is the other
direction — read the spec and find the flaws IN it. A spec is a security
artifact whether anyone treats it as one, and it is often the only thing you
have for a service you do not own.

    vpcopilot scan --spec ./openapi.yaml            # the contract alone
    vpcopilot scan ./app --spec ./openapi.yaml      # …and the drift between them

`--spec` is additive, unlike `--cve`: alone it scans the contract, alongside a
repo it also compares the two (which needs both).

SPLIT BY WHAT NEEDS JUDGEMENT. A deterministic pass finds what the document
leaves unstated — a number with no `minimum`, a string with no `maxLength`, an
operation with no `security`, `additionalProperties` left open, `$ref` followed
with a cycle bound. Those are facts, so recall does not depend on which model is
configured. The agent decides which of them MATTER: an unbounded `page` is
nothing, an unbounded `amount` on a transfer is a business-logic hole.

The spec goes to the EXISTING discover agent as one more file — no fourth agent,
so spec findings verify, triage and generate exactly like code findings with no
parallel path to keep in step.

Acceptance verified live: a spec declaring `amount` with no lower bound produced
neg-amount-001 -> service_policy + api_schema; spec/code drift reports as
`undocumented_or_orphaned`; findings flow into the same triage table.

TWO REAL BUGS THE FIRST LIVE RUN SURFACED:
  * A spec is ONE file declaring MANY endpoints, so endpoint_of("pay-api.yaml")
    returned the same string for every finding — all six collapsed onto one
    service_policy coverage key and five were logged "already covered" and given
    no band-aid at all. Spec findings now carry `source` and leave `file` empty,
    and the coverage identity prefers `endpoint`: three endpoints, three keys,
    and only the two genuinely on /api/v1/transfer collapse.
  * The undocumented_or_orphaned finding was going through `verify`, whose whole
    method is reading the offending source. With no source to read it refuted
    the finding at 0.10 confidence. It is appended after verify instead — a
    comparison of two documents is not a claim about code.

Drift is reported in both directions and they fail differently: declared-but-
unserved is dead documentation or a shadow API; served-but-undeclared is what an
api_schema band-aid built from this spec would start rejecting.

run.json records input_kind as repo / spec / repo+spec / advisory.

502 tests, ruff clean. Fixture: bench/fixtures/specs/pay-api.yaml.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@henleda
henleda merged commit faf75d8 into main Jul 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant