Skip to content

fix(ci): give pr-verify a fixed spec for main-red incident issues - #16693

Merged
kyle-sexton merged 1 commit into
mainfrom
fix/pr-verify-main-red-incident-spec
Nov 17, 2026
Merged

kyle-sexton merged 1 commit into
mainfrom
fix/pr-verify-main-red-incident-spec

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: pr-verify failed PR #16692 for linking the bot-filed main-red incident #16690, which has no trusted spec text

Summary

When main goes red, the monitoring lane files an incident issue as github-actions[bot]. pr-verify gave that issue a null body, so any fix pull request that linked it (Refs: or Closes) failed with "no issue has trusted spec text", as #16692 did. Every red-main fix would hit this. The incident now gets a fixed-template spec in the shape of the upstream-drift rule (#16561, #16620, #16640), and the merge gate accepts the same shape.

Fix

  • .github/scripts/pr-verify/resolve-pr.mjs: mainRedIncident recognises the incident and MAIN_RED_SPEC is its spec, with source: "main-red-incident". It works for both Closes and Refs:, and for every linkedIssues caller, since it needs no changed-file read. The title comes from issues.mjs FILING["1 main red"] and the key from incidentKey, both imported rather than copied.
  • .github/scripts/intake-triage/admit.mjs: actionsBotOnly (written by github-actions[bot], never edited or edited only by it) is pulled out of driftIssueId, so the drift rule and the incident rule share one edit check. Intake, admission and the triage sweep still do not trust the incident.
  • .github/scripts/maintenance-monitor-signals/issues.mjs: exports incidentKey.
  • prompt.md and spec-prompt.md: the main-red-incident source is noted next to upstream-drift. Both judges get the same new rule: a refs entry with this source is judged against its whole fixed body and needs no remaining work, because the lane closes the incident itself once the signal clears.
  • plugins/source-control/skills/babysit-prs/scripts/babysit_verify_evidence.py: the merge gate's copy of the rule (main_red_incident, plus a shared actions_bot_only) is checked in _trusted_spec. Without it, a pass on such a pull request would still be refused verify:untrusted-spec.
  • docs/conventions/pr-pipeline.yaml: issues.mjs gets a hold-floor enforcement row, because a floored module now imports it. The floor copies were regenerated with scripts/sync-hold-floor.py, and the pinned floor counts and digests in test_floor_paths.py and test_babysit_risk.py moved from 145 to 146 rows. As a result, a pull request that touches issues.mjs now holds as review-path: until a clean security review binds to its head.
  • docs/conventions/pr-pipeline/README.md states the rule. Two changelog fragments: pr-pipeline (minor) and source-control (patch).

Trust change

An issue is now its own spec when all of these hold:

  • github-actions[bot] (id 41898282) wrote it.
  • It was never edited, or last edited by that same bot.
  • Its title is exactly Restore the failing required checks on main.
  • Its first body line matches <!-- factory-monitor key=incident:main-red fp=<16 hex> -->.

No issue-body text reaches the model. The spec is one fixed text: fix the cause of the failing or flaky required check without weakening, skipping or deleting a check, and widen no further. Scope is not checked against file paths, since a red-main fix can touch any file. The fixed template's "does not widen beyond that fix" item bounds it. No other factory-monitor key and no other bot issue is trusted.

Verification

  • node --test on .github/scripts/pr-verify/*.test.mjs, .github/scripts/maintenance-monitor-signals/*.test.mjs, .github/scripts/intake-triage/*.test.mjs and .github/scripts/pr-merge/merge-holds.test.mjs: 1942 of 1942 pass. The new resolve-pr.test.mjs cases cover:
    • a Refs: link, a Closes link, and the lane's own bot edit;
    • a linkedIssues call with no changed-file read;
    • eight look-alikes: a person author, the lanes App author, a different title, no marker, the marker on a later line, another key (incident:ceiling), no fingerprint, and a body a person edited.
  • The negative tests were checked against deliberately broken code. Letting any incident: key match failed the "another factory-monitor key" case, and dropping the edit check failed the "body a person last edited" case, in both the JS and Python tests.
  • spec-prompt-text.test.mjs asserts that both prompts carry the exception word for word.
  • pytest (through uv) on test_babysit_incident_spec_trust.py (new), the drift and recurring spec-trust tests, test_floor_paths.py, test_babysit_risk.py, test_merge_holds_sync.py and test_babysit_verify_evidence.py: 658 passed.
  • test_verify_rules_sync.py and all three *_spec_trust.py modules: 40 passed.
  • scripts/sync-hold-floor.py --check is clean, and scripts/run-ruff.sh check on the changed Python passes.
  • Pre-push check-affected failed on the first push: comment-hygiene flagged two test comments that cited the incident's issue number. After rewording them, the push went through with the hooks passing.

Related

🤖 Generated with Claude Code

A pull request that fixes a red main and links the monitoring lane's
main-red incident (Refs: or Closes) failed pr-verify with "no issue has
trusted spec text", since the incident is written by github-actions[bot].

The incident now counts the way the upstream-drift issue does: Actions
bot author, never edited or edited only by that bot, the exact title
from issues.mjs FILING and the incident:main-red marker as the first
body line. Its spec is one fixed text (MAIN_RED_SPEC); no issue-body
text reaches the model. Both prompts say a Refs: entry with source
main-red-incident needs no remaining work. The merge gate's twin
(main_red_incident in babysit_verify_evidence.py) accepts the same
shape, so the pass is not refused verify:untrusted-spec. issues.mjs
joins the hold floor because a floored module now imports it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T05:40:01.120463Z 95f9369 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 95f9369e41e69efcc949769301f67f4a7532349b: read 19 of 19 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #16693, a fixed pr-verify spec for main-red incident issues

No blocking or advisory findings. I read all 19 changed files in the diff in full. I also read issues.mjs at head, the field lists of the issue query in resolve-pr.mjs, and the token setup in maintenance-monitor-signals.yml.

JS rule and Python rule match.

  • Same parts. Both rules are built from the same three parts:
    • mainRedIncident in resolve-pr.mjs.
    • main_red_incident in babysit_verify_evidence.py, which _trusted_spec now checks.
    • The shared author and editor check (actionsBotOnly / actions_bot_only).
  • Marker line. The JS MARKER regex has no m flag and runs on the first line only, which means a trailing \r fails it. The Python fullmatch on split("\n", 1)[0] also fails on \r, so the two agree.
  • Title and key. The title and key come from FILING["1 main red"] and incidentKey. The new Python test TheCopiesMatchTheFiler pins its copies of the title and marker template against issues.mjs. The parents[6] path it uses resolves to the repository root.
  • Order of checks. In linkedIssues, the incident expression runs after the written, split-child, recurring and drift checks, so it cannot take over an issue that already has a spec. || binds tighter than ?: there, so the expression parses as intended.
  • Fields read. The fields the rule reads (author, editor, lastEditedAt, title, body) are already in the issue query, because driftIssueId uses them too.

Filing identity checked. The monitoring workflow files with github-token and holds no lanes App token. That matches the rule trusting only github-actions[bot] (id 41898282).

Floor changes are consistent. The new enforcement row for issues.mjs appears in:

  • pr-pipeline.yaml
  • merge-holds.mjs
  • pr-require-checks.yml
  • both hosted review workflows

The test pins moved by one row each: 145 to 146 in total, and 62 to 63 for the enforcement class and for FLOOR.review. I did not recompute the new digests.

Not verified here. The PR's test and sync results (node --test 1942/1942, pytest 658 and 40, sync-hold-floor.py --check, ruff) are the author's claims. I had no shell and ran none of them.

Closing issues. The PR closes no issue, so there are no acceptance criteria to check against. The PR body contains no embedded instructions to this lane.

Security split. Trusting a new kind of issue as its own spec widens a trust boundary. A separate security lane runs here (pr-review-security-hosted.yml), so I left the security review of that change to it.

Plugin manifests. No plugin.json or marketplace.json changed. The source-control changelog fragment uses bump: patch, which fits a gate fix.

Verifier dropped 0 of 0 candidates.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 95f9369e41e69efcc949769301f67f4a7532349b: read 19 of 19 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #16693 (head 95f9369)

I found no security issues. I read all 19 changed files in full in the diff, plus the parts of resolve-pr.mjs, issues.mjs, pr-pipeline.yaml and babysit_verify_evidence.py that the new code relies on.

critical-findings: 0

Trust control checked: a new kind of issue now counts as a trusted spec

This change weakens the untrusted-spec hold: a bot-filed main-red incident issue now passes it. That is a provenance hold, not one of the owner's friction holds, so I judged it like any other merge hold. Nothing gets through that the base stopped:

  • Who reaches it. pr-verify only judges pull requests written by the lane or the owner. It skips every other author at resolve-pr.mjs#L1081-L1086, and the config sets verify-authors: lane-or-owner at pr-pipeline.yaml#L1661. A fork or outside author who writes Refs: #<incident> gets no pass for the gate to count.
  • What text becomes spec. None of the issue's text. The spec is the fixed constant MAIN_RED_SPEC in the repository (resolve-pr.mjs#L142-L151). The issue body, which carries run output, is only matched against the anchored marker regex at issues.mjs#L27. Only trusted comments are passed on (written(comment), line 1017).
  • Who authored it. The issue must be written by github-actions[bot] (id 41898282) and either never edited or last edited by that same bot. The shared actionsBotOnly helper is an exact extraction of the existing drift check. The gate's Python copy (main_red_incident) matches the JS rule and is stricter about a trailing \r. It runs only after _actions_issue has checked the recorded author's type and id and read the issue again.
  • Who else can use it. Intake, admission and the issue lane still do not trust the incident, so it starts no lane work.

Hold floor

Adding .github/scripts/maintenance-monitor-signals/issues.mjs as an enforcement row in pr-pipeline.yaml, merge-holds.mjs, pr-require-checks.yml and both review workflows adds a hold; it removes none. The test pin updates (145→146 rows, review globs 62→63) match that one extra row.

Instruction surface

Both verifier prompts gain one exception: a refs entry whose source is main-red-incident needs no remaining work. This loosens a scope-correctness rule for one fixed, repository-owned spec. It does not touch any protected class of action (credentials, publishing, destructive or outbound actions), so it is not a security finding. I did not open the instruction exception register; I judged this by what breaks without the rule.

Note, not a finding

mainRedIncident does not check whether the incident is open, so a pull request can still link a closed incident and get the spec. The spec text is fixed, contains no attacker text, and limits itself to "does not widen beyond that fix", so this raises no security impact.

Embedded text

The PR description contains nothing addressed to the review lane.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 2b320ee Nov 17, 2026
68 checks passed
@kyle-sexton
kyle-sexton deleted the fix/pr-verify-main-red-incident-spec branch November 17, 2026 05:59
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant