Skip to content

fix(review): stop same-head AI re-review churn and add a maintainer-gated freeze#3461

Merged
JSONbored merged 1 commit into
mainfrom
fix/gittensory-review-churn
Jul 5, 2026
Merged

fix(review): stop same-head AI re-review churn and add a maintainer-gated freeze#3461
JSONbored merged 1 commit into
mainfrom
fix/gittensory-review-churn

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Fixes the wasteful/incorrect same-head PR re-review churn observed in production (PR #3379's repeated AI calls and comment edits, PR #3383's held-verdict flip). Three compounding bugs, all confirmed against live audit-event evidence before fixing:

  1. baseSha in the AI review cache fingerprint (src/review/ai-review-cache-input.ts). It hashed the PR's live base.sha on top of the actual diff content. base.sha is the live tip of the target branch — it advances on every unrelated merge, independent of whether this PR's own files changed. On an active repo that means the fingerprint differs on nearly every scheduled regate-sweep tick, defeating both the durable cache and the existing bounded non-cacheable-cooldown reuse (added for a prior incident, perf(review): cache the AI review by (repo, pr, head SHA) to skip redundant LLM calls #1462) — since that lookup is keyed on an exact fingerprint match. The diff content (reviewFiles' patches) already captures "did the reviewed content change"; hashing baseSha on top was redundant when unchanged and actively harmful when the repo is busy.
  2. No durable "published" concept (src/db/repositories.ts). A genuinely non-cacheable (dynamic-context: grounding/RAG/reputation) review was only reusable for a bounded 30-minute cooldown — past that, every sweep tick paid for a fresh LLM call forever. Since LLM output isn't perfectly deterministic, two calls against identical input could disagree, flipping the published verdict.
  3. The reviewing placeholder posted unconditionally on every pass (src/queue/processors.ts), before checking whether anything had changed — guaranteeing two real comment edits per sweep tick (placeholder-in, final-out) forever, defeating the existing byte-identical no-op guard at the comment layer.

What changed

  • Removed baseSha from AiReviewCacheInput/aiReviewCacheInputFingerprint.
  • New ai_review_cache.published_at column (migrations/0112_ai_review_cache_published.sql) + markAiReviewPublished() / getLatestPublishedAiReview(). Once published, a review is reused indefinitely for its exact head+fingerprint regardless of the cooldown; a fresh write resets published_at to NULL.
  • The reviewing placeholder now only posts when the head is genuinely new or a maintainer forced a rerun.
  • New maintainer-gated freeze: once a PR carries the manual-review label, further contributor pushes reuse the last published review instead of spending a fresh AI call — closing a gaming surface where a held PR could be pushed to repeatedly at real LLM cost while waiting for a maintainer to actually look. Only an explicit maintainer/collaborator retrigger (the existing PR-panel checkbox, unchanged authorization) unfreezes it. CI/mergeable-status rendering and label/assignee reconciliation are unaffected — both still run every pass.

Two pre-existing tests had their assertions deliberately updated (not regressed) because they encoded the now-superseded policy: one asserted a bounded-but-still-unbounded-over-time retry cadence for non-cacheable reviews, the other asserted baseSha alone should miss the cache.

No linked issue — this is a direct fix for production incident evidence gathered from audit_events on the live deployment, not a pre-filed feature request.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — full unsharded run, 9078 passed, 0 failed
  • npm run test:workers
  • npm run build:mcp (not run — no MCP package changes)
  • npm run test:mcp-pack (not run — no MCP package changes)
  • npm run ui:openapi:check
  • npm run ui:lint / ui:typecheck / ui:build (not run — no UI files touched; confirmed via ui:openapi:settings-parity that RepositorySettings is unaffected)
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — 9 new end-to-end tests reproducing fix(selfhost): use own-property check for retired config-lint fields #3379, fix(review): skip additional binary extensions in full-file grounding #3383, pure base-branch movement, real content changes, explicit force-rerun, and the maintainer-gated freeze (including edge cases: label disabled, nothing-ever-published, and read/audit-write failures), plus updated unit coverage in ai-review-cache.test.ts / ai-review-cache-input.test.ts. Targeted branch coverage on every changed line range is 100% (one defensive ?? null fallback marked v8 ignore, consistent with 6 identical pre-existing instances in the same file).

If any required check was skipped, explain why:

  • build:mcp / test:mcp-pack: no changes under the MCP package. ui:lint / ui:typecheck / ui:build: no changes under apps/gittensory-ui/**; also separately ran npm run cf-typegen:check (clean, no wrangler binding changes) even though it's not in the template's list.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no auth/session/CORS changes.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no public API/schema changes; confirmed via ui:openapi:check + ui:openapi:settings-parity.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A — no UI changes.)
  • Visible UI changes include a UI Evidence section below with screenshots. (N/A — backend-only change, no visible UI.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (N/A — no changelog edit.)

Notes

  • New migration 0112_ai_review_cache_published.sql (contiguous, verified via db:migrations:check); src/db/schema.ts updated to match (verified via db:schema-drift:check).

…ated freeze

Removes baseSha from the AI review cache fingerprint -- it is the live tip
of the base branch, which advances on every unrelated merge regardless of
whether this PR's own diff changed, so on an active repo it defeated the
cache on nearly every scheduled re-gate sweep. Adds a durable "published"
marker to ai_review_cache so a review that has actually reached the PR is
reused indefinitely instead of only for a bounded cooldown, and stops the
reviewing placeholder from posting when nothing is about to change. Also
adds a maintainer-gated freeze: once a PR carries the manual-review label,
new pushes reuse the last published review instead of spending a fresh AI
call, until a maintainer/collaborator explicitly retriggers it.

Validated with npm run test:coverage (9078 passed), npm run test:workers,
db:migrations:check, db:schema-drift:check, cf-typegen:check,
ui:openapi:check, ui:openapi:settings-parity, actionlint, and
npm audit --audit-level=moderate.
@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 5, 2026
@JSONbored JSONbored merged commit 5a4d011 into main Jul 5, 2026
8 checks passed
@JSONbored JSONbored deleted the fix/gittensory-review-churn branch July 5, 2026 07:55
@gittensory-orb gittensory-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.5x multiplier. label Jul 5, 2026
@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.77419% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.99%. Comparing base (1dcc412) to head (70d0488).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/queue/processors.ts 95.23% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3461   +/-   ##
=======================================
  Coverage   92.99%   92.99%           
=======================================
  Files         296      296           
  Lines       30972    30995   +23     
  Branches    11294    11303    +9     
=======================================
+ Hits        28803    28825   +22     
- Misses       1514     1515    +1     
  Partials      655      655           
Files with missing lines Coverage Δ
src/db/repositories.ts 96.47% <100.00%> (+0.01%) ⬆️
src/db/schema.ts 70.00% <ø> (ø)
src/review/ai-review-cache-input.ts 100.00% <ø> (ø)
src/queue/processors.ts 93.05% <95.23%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

JSONbored added a commit that referenced this pull request Jul 5, 2026
… path (#3477)

* test(review): cover the markAiReviewPublished write-failure fail-open path

#3461 added markAiReviewPublished alongside the existing
markPullRequestSurfacePublished stamp but only tested the latter's
failure path, leaving the new call's own catch/console.error branch
without a regression test (codecov/patch flagged it after merge).
Mirrors the existing "over-publish dedup" test for the sibling stamp.

* test(review): assert the exact stamp identity in the fail-open regression

Addresses the gate review's nit on #3477: the markAiReviewPublished
spy assertion only checked that it was called, not with which
repo/PR/head, so a wrong-identity call would have passed silently.
JSONbored added a commit that referenced this pull request Jul 5, 2026
…view freeze (#3490)

The maintainer-gated freeze (#3461) reused the last published AI
review for any PR carrying the manual-review label, with no author
exemption. Confirmed live on PR #3476: pushing genuine follow-up
commits to the owner's own held PR kept replaying the ORIGINAL,
now-stale AI verdict instead of evaluating the new commits, because
github_app.ai_review_frozen_reuse fired on every push. The anti-gaming
concern the freeze exists for is specific to a contributor iterating
pushes against the bot; it never applies to the repo owner, an
ADMIN_GITHUB_LOGINS fleet operator, or a protected automation bot,
matching the exemption this codebase already grants those authors
everywhere else (auto-close, review-nag, contributor caps).

Root-caused via the live audit_events trail on the self-host VPS,
which showed ai_review_frozen_reuse firing on every one of #3476's
own follow-up commits despite the PR being owner-authored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.5x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant