Repository navigation
Description
Parent
Refs #8651. Phase S7 ("Phase 10: S7 pr-verify capture job") of the evidence-verification plan, durable copy at #11364 (comment); Brief at #11364 (comment). #11393 (S5c) waits on this job; before this issue existed it was blocked by #8651 itself, its own parent.
Relates to #7529 (Claude Code UI leg of the per-PR smoke).
Depends on
- feat(harness-ops): let control-claude-code run unattended on sessions it launches #11386 (S4a, control-claude-code unattended and credential)
- feat(harness-ops): mask control-claude-code captures and launch changed plugins #11390 (S4b, control-claude-code masks and changed-plugin launch)
- S2a (feat(testing): make run-e2e consume affected and write e2e-summary.json #11388) and S6 (feat(pr-pipeline): add the verify-capture key and verdict evidence metrics #11385) are closed.
- The plan's two owner-reserved gates are answered. U1, "CI credential exposure", was answered by Q10 (feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364 (comment)). U2, "PR code in verdict run", was answered by the superseding Q11 record (feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364 (comment)). The gates are named in the design's table (feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364 (comment)).
Outcome
pr-verify runs a capture job on in-repo PR heads. The job drives the affected app unattended, uploads the publishable captures as a retained artifact, and feeds them to the judge jobs as untrusted data. A missing or blocked capture never fails a verdict.
Plan (phase S7)
Satisfies: AC1, AC2, AC5, AC9, AC10
Size: Large. Files: .github/workflows/pr-verify.yml, .github/scripts/pr-verify/prompt.md, .github/scripts/pr-verify/workflow.test.mjs. Design: T8, sequence flow 2.
- Probe step first. After
setup-runner-isolation, a head-code step tries to read the runtime token and the credential from a hook. The job fails closed if either is readable. - Job
capture. It uses the sameifas the existing judge jobs, so fork heads never run. It runs the gates and uses a read-only token,.baseplugins and--trusted-root .base, then calls/testing:run-e2e unattended app=<merge.verify-capture.app> base=<base-sha>. It uploads the publishable files aspr-capture-<pr>-<sha>withretention-days. When the key is unset, it skips with a note. - Gates first. The capture job's first steps are the kill switch, resolver, trusted-trigger and agree steps (check-intent's copies stay), so it captures nothing on a run that judges nothing. It sets
timeout-minutes: 30. The outer session runs through ci-workflows'run-claude-cliwith its own role, a tool allowlist and the drivemaxEntryPointscap (S4b). A capture timeout leavescapture: blockedand never stops the judge. - Judge jobs. Only
check-intentgainsneeds: capture, with!cancelled().check-intent-subscriptionandcheck-intent-fallbackkeep their success chain and download the same run's artifact by id. All three download intopr-verify-data/capture/. The prompt frames the capture as untrusted data and says a missing or blocked capture is never a FAIL reason. A trusted-text manifest lineEvidence loop: 3/3 cap reached, last NEEDS WORKis a FAIL (AC7's park path then runs). - Last step. After the probe passes on a real PR, set
merge.verify-capture.app: claude-codeand show one in-repo PR with a judged capture (AC1). Update the header comment block inpr-verify.ymlthat describes the jobs (reviewed change: jobs now take an artifact).
Credential separation (Q10, Q11)
The superseding Q11 record (#11364 (comment)) is the Issue Triage session's reading of the owner's 2026-10-31 direction. It says:
- The capture job may hold pr-verify's existing credential (Q10).
- The PR's own code runs in a child process whose environment and readable files hold no token. The credential stays with the Claude Code driver and is never passed to, exported to, or written where the app under test can read it.
- In-repo heads only.
If the build cannot keep the token out of the PR code's reach, it stops and brings that back to the owner as a new question. It does not ship without the separation.
Done when
-
node --test .github/scripts/pr-verify/workflow.test.mjsexits 0 with these cases:- the capture job's permissions contain no
write; - the three judge jobs carry identical capture-download steps;
- the capture
ifincludes the head-repository check; pr-verify.ymlhas no job-levelalways();- only
check-intenthasneeds: capture; - a probe step exists, runs after
setup-runner-isolationand before any head code, and has nocontinue-on-error.
- the capture job's permissions contain no
- The capture job's step that starts the app under test passes it no credential variable, and a test asserts that the app process's environment holds none of the pr-verify credential names.
-
actionlintpasses onpr-verify.yml. - One in-repo PR shows a judged capture with
merge.verify-capture.app: claude-codeset (AC1).
Builder: the Automation Lanes session. This is host-run work: it edits a workflow file and needs a live PR run.
Activity