Skip to content

feat(verify): read artifacts through rivet, adopt score status vocabulary - #13

Merged
avrabe merged 1 commit into
mainfrom
feat/verify-reads-via-rivet
Nov 17, 2026
Merged

avrabe merged 1 commit into
mainfrom
feat/verify-reads-via-rivet

Conversation

@avrabe

@avrabe avrabe commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Closes #5 for rivet artifact files (artifacts/*.yaml). variants/bindings.yaml is not a rivet artifact file and stays on PyYAML until pulseengine/rivet#1094.

Changes

  • tools/verify.py reads artifacts with rivet list --format json --full instead of yaml.safe_load.
  • All 34 artifacts change status: approved to status: valid. The vendored score schema (vendor/rivet-schemas/score.yaml) allows only draft, valid, invalid, in_progress, obsolete. rivet v0.13.0 never checked the value, so CI's rivet validate was blind to it. v0.41.1 rejects all 34. valid is also eclipse-score's own term.
  • CI installs rivet v0.41.1 instead of v0.13.0, because v0.13.0 has no list --full.
  • Gate fix: a malformed verified-by entry or a bazel build/discovery error on a waived comp-req is now FAILED. Previously it was KNOWN-GAP, which let an unbuilt test pass the gate. Only a test that ran and failed can be waived.
  • New tools/test_verify.py: 9 bucketing cases with rivet and bazel stubbed. CI runs it before make verify.
  • New DR-KVS-ARTIFACTS-THROUGH-RIVET records the decision.

Evidence (local)

Check Result
make validate with rivet 0.41.1 PASS, 18 warnings (same as before the new DR)
Old PyYAML reader vs rivet reader on main's vs this branch's artifacts Same 6 comp-reqs, same verified-by lists, same waivers. Only the order differs because rivet sorts by id.
tools/test_verify.py 9/9 OK
Mutation: revert the discovery-error fix Exactly 1 test fails (test_waiver_does_not_cover_discovery_error)
verify.py with a missing or failing rivet binary Exit 1
make verify with real bazel Not run to completion locally. On a loaded host, the bazel server under taskpolicy -b did not connect within 120 s. CI's rivet + bazel + verify job is the first full run. Expected: 3 PASSED, 2 KNOWN-GAP, exit 0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KEjmjapTTSvsFZti3qcD56

…lary

Closes #5 for rivet artifact files.

- tools/verify.py reads artifacts with `rivet list --format json --full`
  instead of PyYAML. variants/bindings.yaml stays on PyYAML until
  pulseengine/rivet#1094.
- All 34 artifacts move from `status: approved` to `status: valid`. The
  vendored score schema allows draft/valid/invalid/in_progress/obsolete;
  rivet v0.13.0 did not check the value, v0.41.1 rejects all 34.
- CI installs rivet v0.41.1 (v0.13.0 has no `list --full`).
- A build or discovery error on a waived comp-req is now FAILED, not
  KNOWN-GAP: only a test that ran and failed is waivable.
- tools/test_verify.py: 9 bucketing cases with rivet and bazel stubbed,
  run in CI. Reverting the discovery-error fix fails exactly one case.
- DR-KVS-ARTIFACTS-THROUGH-RIVET records the decision.

Local evidence:
  rivet 0.41.1 `make validate`: PASS (18 warnings, unchanged)
  old PyYAML reader vs rivet reader: same 6 comp-reqs, same verified-by
    lists, same waivers; only order differs (rivet sorts by id)
  missing or failing rivet binary: verify.py exits 1
  `make verify` with bazel: not run to completion locally (bazel server
    under taskpolicy -b did not start within 120 s on a loaded host);
    CI runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEjmjapTTSvsFZti3qcD56
@avrabe
avrabe merged commit 73186f4 into main Nov 17, 2026
1 check passed
@avrabe
avrabe deleted the feat/verify-reads-via-rivet branch November 17, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tools/verify.py: read artifacts through rivet instead of PyYAML

1 participant