Skip to content

docs(pr-flow): make an exhaustive Copilot review loop the step after opening a PR - #2466

Merged
cliffhall merged 3 commits into
v2/mainfrom
v2/docs/2464-copilot-review-loop
Sep 24, 2026
Merged

cliffhall merged 3 commits into
v2/mainfrom
v2/docs/2464-copilot-review-loop

Conversation

@cliffhall

@cliffhall cliffhall commented Sep 24, 2026

Copy link
Copy Markdown
Member

Closes #2464

What changed

  • AGENTS.md, Issue-driven Work Style: new rule. After opening a PR, run a Copilot review loop to exhaustion without being asked, weigh each finding against the issue the PR closes, and decline scope expansion. The same scope rule is added to Responding to Code Reviews.
  • pr-flow step 7 is now the loop:
    • 7a requests a round (the mutation is unchanged).
    • 7b waits for a posted review or for Copilot's pending request to disappear without one, plus a 25-minute cap. The old wait only watched for a review, so it hung forever when Copilot's session ended without posting.
    • 7c is a stop/continue table. The loop stops on the first clean round, with no confirming round "just to be sure"; on a round holding only out-of-scope findings; or on two consecutive rounds that end without a review. A round counts as clean only when the inline comments, the body headline and the Suppressed comments block are all empty.
  • pr-flow step 6 now links the PR to its issue with the addCloseIssueReferences GraphQL mutation. Closing keywords do not link a PR whose base is v2/main, so a v2 card showed nothing in the board's Linked pull requests field. Step 6 also ends with "go straight to step 7". The AGENTS.md note on Closes #N points to the recipe.
  • The pr-flow description leads with "create a PR for an issue", and three new eval cases pin that prompt shape and the review-loop request.

Verification

Verified live on this PR

🤖 Generated with Claude Code

…opening a PR (#2464)

AGENTS.md now requires the loop, unprompted, after every PR, and requires
findings to be weighed against the issue with scope expansion declined.

pr-flow step 7 becomes the loop: request, wait for a posted review or for
Copilot's pending request to lapse without one (plus a 25-minute cap), then
a stop/continue table. The first clean round is the stopping point; no
confirming round. The description leads with "create a PR for an issue" and
eval cases pin that prompt shape.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall cliffhall added the v2 Issues and PRs for v2 label Sep 24, 2026
@cliffhall
cliffhall requested a balanced review from Copilot September 24, 2026 00:15

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

A timed-out pending request cannot be retried as a distinct round using the documented mutation.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds a mandatory post-creation Copilot review loop to the repository’s PR workflow.

Changes:

  • Documents review-loop and scope-control rules.
  • Adds request, wait, retry, and exit procedures.
  • Expands skill-trigger evaluation cases.
File Description
AGENTS.md Establishes review-loop rules.
.claude/​skills/​pr-flow/​SKILL.md Documents the operational workflow.
.claude/​skills/​pr-flow/​evals/​evals.json Adds trigger cases.

Comment thread .claude/skills/pr-flow/SKILL.md Outdated
cliffhall and others added 2 commits September 23, 2026 20:22
 review)

A timeout fires only while the request is still pending, and requestReviews
with union:true is a no-op for an already-requested bot, so a retry would
start nothing new.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
…#2464)

Closing keywords do not link a PR whose base is not the default branch, so
a v2 card showed no linked PR. The GraphQL mutation adds a manual closing
reference, which populates the card's Linked pull requests field. Verified
on #2466/#2464 with the manual link removed first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 1: 1 finding, fixed.

  • Retrying a timed-out review never starts a new request (pr-flow 7c). Fixed in 3be44dc. timed-out now stops and reports the round as still pending; only ended-without-review is retried.

Also in this push (2623167), at the maintainer's request: pr-flow step 6 now links the PR to its issue with the addCloseIssueReferences GraphQL mutation. Closing keywords do not link a PR whose base is v2/main, so a v2 card showed no linked PR. Verified on this PR: with the manual link removed first, the mutation alone put #2466 in #2464's Linked pull requests field on board #28. The matching AGENTS.md note is updated too.

local:gate: EXIT=0. Requesting round 2.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The documented workflow is coherent, complete, and consistent with issue #2464.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 2: clean. Review loop closed.

No inline comments, no finding in the headline ("coherent, complete, and consistent with issue #2464"), and no Suppressed comments block. Round 1's finding is listed as resolved. Under the new 7c a clean round is the stopping point, so no further round is requested. The loop ran as documented: two rounds, one fix, and both waits resolved as ROUND=posted.

@cliffhall
cliffhall merged commit d483427 into v2/main Sep 24, 2026
5 checks passed
@cliffhall
cliffhall deleted the v2/docs/2464-copilot-review-loop branch September 24, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v2 Issues and PRs for v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pr-flow: make an exhaustive Copilot review loop the default step after opening a PR

2 participants