Skip to content

feat(finishingbot): report whether each pull request is ready to merge - #693

Merged
hyperpolymath merged 5 commits into
mainfrom
claude/finishingbot-pr-readiness
Nov 17, 2026
Merged

hyperpolymath merged 5 commits into
mainfrom
claude/finishingbot-pr-readiness

Conversation

@hyperpolymath

@hyperpolymath hyperpolymath commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

What

finishingbot now reports whether each pull request is ready for the owner to review and merge, and the verdict reaches the fleet report-back. The grid row "finishingbot (release readiness)" then states it on every PR without spending Claude credit. Advisory only: it never changes finishing-bot's exit status and gates nothing.

finishing-bot audit --pr N --pr-repo owner/name [--exclude-run-id ID] adds one analyzer, pr_readiness. Each unmet condition is one finding:

Rule Condition Severity
PRR-001 unresolved review conversations (count, a link to each thread) error if the base requires resolution, else warning
PRR-002 a reviewer whose latest review is CHANGES_REQUESTED error if approvals are required, else warning
PRR-003 head behind base error if the base requires an up-to-date head, else warning
PRR-004 merge conflict error
PRR-005 check runs or statuses failing, timed out, cancelled, action_required or startup_failure error
PRR-006 checks still pending (the evaluating run's own checks excluded) info
PRR-007 a required status context (rulesets plus classic protection) that no check run or commit status on the head provides warning "not yet" while any check is unfinished (a needs: job has no check run until it starts; seen on this PR itself); error once every check has finished
PRR-008 draft info

The verdict is one of ready, not ready with reasons, or readiness unknown, which names any source that could not be read. It also states draft and auto-merge, and it becomes the envelope reason. A PR with an unread source is never reported ready. There is also a standalone command, finishing-bot pr-readiness --repo o/r --pr N (or --snapshot FILE).

Why finishingbot's row read "no report-back was received"

The adapter was not the cause. finishingbot already produces an envelope wherever it is enrolled: gitbot-fleet main run 44109280783 shows finishingbot -> ran (802 finding(s)), and rsr-template-repo fork-governance run 44124672727 shows ran (150 finding(s)). The orange row comes from the consumer side, in rsr-template-repo Publish verdict grid. That workflow fires when any of Fork governance, Hypatia or Static Analysis completes. It looks for a completed Fork governance run on the SHA, and when there is none it renders without fleet evidence. On rsr-template-repo#436 (publisher run 44127710882), Fork governance run 44127169826 was still queued when the publisher ran. So all three fleet rows (rhodibot, glambot, finishingbot) read "not_run: enrolled, but no report-back was received". The fix belongs in that publisher, not here (see Next).

Wiring

  • scripts/fleet-report-back.sh gains --pr and --pr-repo, validated as both-or-neither. The token arrives as FLEET_PR_TOKEN. The driver unsets it before any adapter runs and exports it only into finishing-bot's subshell, as GH_TOKEN, never on a command line. PRR findings are exempt from the no-path warning because they are about the change as a whole.
  • fleet-report-back-reusable.yml passes github.event.pull_request.number, GITHUB_REPOSITORY and the job's token. Permissions are unchanged (contents: read). No uses: changed, so actions.lock is untouched; scripts/check-lock-sync.sh passes, with output identical to main.
  • No Cargo.toml or Cargo.lock change: the crate's existing reqwest (rustls) and tokio are used. This avoids the files feat(finishingbot,glambot): read the chora deed, not the retired .scm files #689 edits.

Evidence (run locally)

  • cargo test --locked (bots/finishingbot): 84 passed (lib), 84 passed (bin), 37 passed (analyzer_tests), 0 failed. Baseline before this change: 65/65/37.
  • cargo clippy --locked --all-targets -- -D warnings: clean. cargo fmt --check: clean. cargo build --release --locked: ok.
  • bash tests/fleet-report-back-test.sh: 228 passed, 0 failed (baseline 214); build helper: 69 passed, 0 failed. The new section 12 covers argument passing, token isolation (another adapter sees no token), the token never reaching the log, the verdict as the reason, and refusal of bad --pr values.
  • shellcheck -S warning: only the SC2140 warning that main already has.
  • Fixtures were recorded 2026-11-17. rsr-template-repo#435 → exactly PRR-007 for `CI/gate` (none of its 54 check runs or statuses provides it). standards#1619 → one PRR-001 with 20 conversations, plus PRR-003 (14 behind). Negative controls: a clean PR yields no finding and is ready; exactly one unresolved thread yields exactly one finding.
  • Live REST run of the real binary through the real driver against rsr-template-repo#435: finishingbot -> ran (151 finding(s)) [PR #435 not ready: required context(s) not provided: CI/gate; ...].

Not verified

  • GraphQL reviewThreads was not run live: GraphQL is blocked from the authoring session, which read it as "could not read". The standards#1619 thread fixture is reconstructed from REST review comments to reproduce the prototype's count of 20, and its _source says so.
  • That a contents: read GITHUB_TOKEN can read pulls, reviews, check runs, rules and GraphQL threads for public repos has not been observed in Actions yet. This PR's own Fleet report-backs run is the first test.
  • The fleet job runs while the PR's other checks are still running, so PRR-006 (pending) will be common. It is info.

Next (outside this repo)

  • rsr-template-repo and standards run the fleet inline in fork-governance.yml. To get the readiness they must pass --pr/--pr-repo and FLEET_PR_TOKEN and bump the gitbot-fleet pin.
  • Fix the publisher race in rsr-template-repo verdict-grid-publish.yml. It should either wait for or re-trigger on the Fork governance completion, or render "fleet evidence pending (run N queued)" instead of "no report-back".

https://claude.ai/code/session_01J4QvDj7XpKyxE4WbsgTsTh


Generated by Claude Code

finishing-bot `audit --pr N --pr-repo o/r` now also reads the pull
request from GitHub and adds one advisory analyzer, `pr_readiness`, whose
findings flatten into the fleet report-back like the others:

  PRR-001 unresolved review conversations (count, a link to each)
  PRR-002 a reviewer whose latest review is CHANGES_REQUESTED
  PRR-003 head behind base        PRR-004 merge conflict
  PRR-005 checks failing          PRR-006 checks pending
  PRR-007 a required status context (rulesets and classic protection)
          that no check run or commit status on the head provides
  PRR-008 draft (informational)

The verdict (ready / not ready with reasons / unknown, naming what could
not be read; plus draft and auto-merge) becomes the envelope reason, so
the grid row "finishingbot (release readiness)" states it. It never
changes the exit status and gates nothing.

The logic is pure (`evaluate` over a Snapshot of GitHub response shapes)
and tested on responses recorded 2026-11-17: rsr-template-repo#435
yields exactly PRR-007 for `CI/gate`; standards#1619 yields one PRR-001
with 20 conversations; a clean PR yields no finding; one unresolved
thread yields exactly one. Requests are REST plus GraphQL
reviewThreads { isResolved }, via the crate's existing reqwest and
tokio (no Cargo.toml change).

Driver: --pr/--pr-repo; the token arrives as FLEET_PR_TOKEN, is unset
before any adapter runs and is exported to finishing-bot alone (as
GH_TOKEN, never on a command line); PRR findings are exempt from the
no-path warning because they are change-scoped by design. The reusable
workflow passes the caller's PR number and its read-only token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDj7XpKyxE4WbsgTsTh
…ks run

Dogfooding on this PR (gitbot-fleet#693) showed PRR-007 naming `CI/gate`,
`scan / rust-secrets` and `scan / shell-secrets` as missing while 47
checks were still queued: a job waiting on `needs:`, or a called
workflow's job, has no check run until it starts. Claiming the absence
as settled then would be an overclaim.

PRR-007 is now a warning ("not provided yet", checks still running)
while any check run or status on the head is unfinished, and the error
("every check has finished, so nothing on the pull request produces
them") only once all have. rsr-template-repo#435 (all 54 checks
finished, no `CI/gate`) still yields the error. New unit test covers
both states.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDj7XpKyxE4WbsgTsTh
@hyperpolymath
hyperpolymath marked this pull request as ready for review November 17, 2026 17:48
Repository owner deleted a comment from coderabbitai Bot Nov 17, 2026
Repository owner deleted a comment from coderabbitai Bot Nov 17, 2026
Repository owner deleted a comment from coderabbitai Bot Nov 17, 2026
@hyperpolymath
hyperpolymath enabled auto-merge (squash) November 17, 2026 20:48
@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 6a6b748 and 20ca273.

Files selected for processing: 11

  • .github/workflows/fleet-report-back-reusable.yml
  • bots/finishingbot/src/analyzers/mod.rs
  • bots/finishingbot/src/lib.rs
  • bots/finishingbot/src/main.rs
  • bots/finishingbot/src/pr_readiness/github.rs
  • bots/finishingbot/src/pr_readiness/mod.rs
  • bots/finishingbot/tests/fixtures/pr_readiness/rsr-template-repo-435.json
  • bots/finishingbot/tests/fixtures/pr_readiness/standards-1619.json
  • docs/REPORT-BACK.adoc
  • scripts/fleet-report-back.sh
  • tests/fleet-report-back-test.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@sonarqubecloud

Copy link
Copy Markdown

@hyperpolymath
hyperpolymath enabled auto-merge (squash) November 17, 2026 20:59
@hyperpolymath
hyperpolymath merged commit dec1e7c into main Nov 17, 2026
66 of 68 checks passed
@hyperpolymath
hyperpolymath deleted the claude/finishingbot-pr-readiness branch November 17, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants