Skip to content

ci(release): approve green, reviewed develop-into-next syncs as supabase-oss - #7124

Merged
avallete merged 14 commits into
developfrom
avallete/sync-next-auto-approve
Oct 14, 2026
Merged

avallete merged 14 commits into
developfrom
avallete/sync-next-auto-approve

Conversation

@avallete

Copy link
Copy Markdown
Member

Summary

Claude-resolved sync/develop-into-next PRs now get approved without a maintainer when the merge is clean in practice:

  • every check except Require fast-forward is green on the head,
  • the AI review posted a review on that exact head,
  • Claude fixed or declined every finding (declining the ones the merge did not introduce),
  • no .github/ workflow suggestions are waiting.

sync-repair plans an approve action in that state, and a new approve job submits an approval for that commit as supabase-oss through the auto-approve environment. The approval runs fast-forward.yml as usual, which re-checks the approver, the head and the checks before moving next.

A head moved past the reviewed commit, by a repair commit or a maintainer push, is never re-reviewed, so it still waits for a person. The job skips if the head moved or supabase-oss already approved that commit.

This reverses the documented rule that the bot never approves sync heads; release-process.md records the new accepted decision. Reviewers should confirm the auto-approve environment's branch rule lets sync-repair runs on develop deploy.

Linked issue

  • I'm a Supabase maintainer.

Checklist

  • The PR title follows Conventional Commits.
  • Tests added or updated for the change.

🤖 Generated with Claude Code

avallete and others added 2 commits October 13, 2026 15:21
…ase-oss

A Claude-resolved sync whose checks are green on a head the AI review saw,
with every finding fixed or declined, now gets approved by supabase-oss so
fast-forward.yml moves next without waiting for a maintainer. A head moved
by a repair commit or a maintainer push still needs a person.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Coly010 Coly010 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few gaps in what counts as "reviewed and answered" that i think need closing before the bot moves next on its own - inline below.

one that isn't in the diff: if either model job fails or times out, adjudicate swaps in empty findings and post-review still posts the marker (ai-review.yml:427-438), so a half review (or one model skimming a huge next...sync diff) counts as reviewed. could the auto-approve path require both models to have completed?

on your env question - auto-approve only allows develop, so the workflow_run/schedule runs deploy fine. rest looks good to me, and with those four sorted i'm happy to approve!

Comment thread .github/workflows/sync-repair.yml Outdated
Comment thread .github/scripts/sync-agent.ts Outdated
Comment thread .github/scripts/sync-agent.ts Outdated
@avallete

Copy link
Copy Markdown
Member Author

/ai-review

…ered AI review

The sync bot now requires both AI review models to finish, pins the review to the
commit the models read, approves only when every unrefuted finding has an answered
thread, and holds while a reviewer requests changes or after its approval on that
head was dismissed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avallete

Copy link
Copy Markdown
Member Author

/ai-review

@avallete
avallete marked this pull request as ready for review October 13, 2026 15:33
@avallete
avallete requested review from a team and supabase-oss as code owners October 13, 2026 15:33

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Verified all six findings from both independent reviews; none overlap. Five are confirmed, and the force-push posting failure remains uncertain because its GitHub API trigger is not established by the repository code. The main issue is an approval race that can bypass a maintainer's change request. Other confirmed concerns are workflow compatibility, missing automatic promotion retries, external reviewers blocking automation, and contradictory release documentation.

Findings

Severity Location Category Sources Claim
🟠 MAJOR .github/workflows/sync-repair.yml:170 race-condition codex The approval job can approve and promote a sync after a maintainer requests changes because it does not refresh the review eligibility decision.
🟡 MINOR .github/scripts/ai-review/post-review.ts:1176 compatibility claude Older pull_request workflow copies fail to post reviews after develop receives this script because they do not provide the newly required HEAD_SHA and REVIEWS_COMPLETE variables.
🟡 MINOR .github/scripts/ai-review/post-review.ts:797 error-handling claude Pinning commit_id can lose a completed review when GitHub rejects a reviewed commit removed by a force-push, because the 422 fallback retains the rejected SHA.
🟡 MINOR .github/workflows/sync-repair.yml:176 error-handling codex An existing bot approval permanently suppresses automatic promotion retries for that head even when the original promotion failed or was refused.
🟡 MINOR .github/scripts/sync-agent.ts:630 robustness claude A change-request review from an account without write access can indefinitely stop automatic approval of a sync PR.
🟡 MINOR apps/cli/docs/release-process.md:363 documentation codex The release runbook retains a maintainer-approval guarantee for edits that the new automation can land without a person reviewing them.

Stats

Claude findings: 3 · Codex findings: 3 · Confirmed: 5 · Refuted: 0 · Uncertain: 1


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread .github/workflows/sync-repair.yml
Comment thread .github/scripts/ai-review/post-review.ts Outdated
Comment thread .github/scripts/ai-review/post-review.ts Outdated
Comment thread .github/workflows/sync-repair.yml
Comment thread .github/scripts/sync-agent.ts Outdated
Comment thread apps/cli/docs/release-process.md
…-auto-approve

# Conflicts:
#	.github/scripts/sync-agent.test.ts
#	.github/scripts/sync-agent.ts
Comment thread .github/workflows/sync-repair.yml
avallete and others added 2 commits October 13, 2026 17:42
…lands

Fast-forward refuses supabase-oss's approval while a collaborator requests
changes, so a change request between collect and approve still holds the sync.
Older AI review workflow copies post without review state instead of failing,
and a rejected pinned review is retried on the current head while its state
line still names the reviewed commit. The runbook now describes when Claude's
edits land without a maintainer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avallete
avallete requested a review from Coly010 October 13, 2026 16:20
Comment thread .github/scripts/fast-forward.ts Outdated
avallete and others added 2 commits October 14, 2026 15:41
…is dismissed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jgoux jgoux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The maintainer hold on automatic approval can be bypassed (P1), and there are two paths where supabase-oss approves despite findings that were never addressed (P2). Details inline.

Comment thread .github/scripts/fast-forward.ts Outdated
Comment thread .github/scripts/sync-agent.ts
Comment thread .github/scripts/sync-agent.ts
Comment thread .github/scripts/fast-forward.ts Outdated
avallete and others added 3 commits October 14, 2026 15:53
Any reviewer's change request now holds the sync, because author_association
hides private org membership. Fast-forward reads the approver from the event
payload, so sync PRs running an older workflow copy are still re-checked. The
AI review marker leads the body so truncation cannot drop it, superseded
reviews are ignored, and workflow suggestions are posted before the repair
replies so a withheld fix never reads as answered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jgoux
jgoux self-requested a review October 14, 2026 15:44

@jgoux jgoux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes. All four earlier points hold up at this head: change requests count regardless of author_association, the marker leads the body and superseded reviews are skipped, suggestions are posted before replies, and the approver comes from the event payload. One minor test-coverage note inline.

Comment thread .github/scripts/ai-review/post-review.test.ts Outdated
…g markers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avallete
avallete enabled auto-merge October 14, 2026 15:51
@avallete
avallete added this pull request to the merge queue Oct 14, 2026
Merged via the queue into develop with commit 825f6d2 Oct 14, 2026
39 checks passed
@avallete
avallete deleted the avallete/sync-next-auto-approve branch October 14, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants