Skip to content

fix(antigravity): complete per-source failure diagnostics - #3683

Merged
steipete merged 10 commits into
steipete:mainfrom
hhh2210:antigravity/auto-error-honesty
Sep 18, 2026
Merged

steipete merged 10 commits into
steipete:mainfrom
hhh2210:antigravity/auto-error-honesty

Conversation

@hhh2210

@hhh2210 hhh2210 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Antigravity failures could hide which source failed, and an account-mismatch message claimed OAuth data was being used before OAuth succeeded. Report ordered source outcomes for auto-mode CLI failures and describe the account restriction accurately. The existing error-selection policy from #3701 remains authoritative.

Add optional strategy IDs and explicit outcomes to diagnostic exports, with legacy Codable compatibility in both directions. Terminal and exhausted pipeline failures produce one debug record using safe error categories; successful and cancelled runs stay quiet. The synthetic CLI fixture pins both its home and Gemini directory inside its own root.

Validation on b4c6a91b3277e1690dab64aa89839e215f27b0e9:

  • make test: all 1,196 selections across 110 groups passed on the first attempt, with no retries or timeouts (941 seconds).
  • Focused provider, export, architecture, and CLI coverage: 146 Swift Testing tests and 43 XCTest tests passed.
  • make check: zero violations. Independent P0–P2 review: no actionable findings.
  • Synthetic production CLI subprocess tests verify explicit-OAuth diagnostic serialization and safe terminal-failure logging. Ordered auto-mode summaries have separate helper coverage. No live account, provider request, browser import, or Keychain read is used. Contributor's captured CLI output illustrates the same explicit-OAuth scenario.

Current-head CI passed all checks. The combined five-fix integration tree also passed all 1,203 selections across 111 groups without retries or timeouts, with clean lint and independent review. The changelog entry is under 0.60.6 Unreleased.

Part of #3673; related to #3146 and #3662. Thanks @hhh2210 for the diagnostic fix and regression coverage.

When every auto source fails, keep the most authoritative attempted
source's error (app > cli > ide > oauth > offline) instead of the last
one written: only .notRunning placeholders still yield to later errors.
Add per-source outcome diagnostics: an auto-failure summary line in
codexbar usage text output, one debug log line when a pipeline exhausts
its strategies, and strategyID + outcome fields in the diagnose export.

Part of steipete#3673 (workstream 1); targets the error-masking class behind
steipete#3146 and steipete#3662. Source precedence is unchanged.
Copilot AI lite review requested due to automatic review settings September 16, 2026 10:04
@clawsweeper

clawsweeper Bot commented Sep 16, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-16T10:19:15.689247Z 58f2602 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89586d5674

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +235 to +236
guard (previous as? AntigravityStatusProbeError) == .notRunning else {
return previous

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply the authoritative error on terminal failures

When Auto includes OAuth credentials and OAuth fails while no offline data exists, AntigravityOAuthFetchStrategy.shouldFallback returns false, causing ProviderFetchPipeline.fetch to return the current OAuth error immediately rather than the value folded by this resolver. Consequently, an earlier substantive app or CLI failure is still masked in this common failure path; the pipeline's terminal return needs to use the resolved lastAvailableError as well.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Terminal fallback paths can bypass the first-authoritative error fold, allowing earlier app or CLI failures to remain masked.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR improves Antigravity auto-fallback error reporting and adds per-source diagnostics.

Changes:

  • Preserves authoritative errors and records strategy outcomes.
  • Adds CLI, logging, and diagnostic-export visibility.
  • Extends tests, documentation, and changelog entries.
File summaries
File Reviewed changes
Tests/CodexBarTests/ProviderDiagnosticExportTests.swift Tests diagnostic fields, legacy decoding, and redaction.
Tests/CodexBarTests/ProviderArchitectureGatekeeperTests.swift Updates source anchors.
Tests/CodexBarTests/CLIAntigravityFallbackSummaryTests.swift Tests ordered CLI source summaries.
Tests/CodexBarTests/AntigravityCLIHTTPSFetchStrategyTests.swift Tests fallback precedence and per-source outcomes.
Sources/CodexBarCore/Providers/ProviderFetchPlan.swift Tracks outcomes and logs exhaustion; terminal failures can bypass the authoritative-error fold (moderate), and log wording is provider-specific (nit).
Sources/CodexBarCore/Providers/ProviderDiagnosticExport.swift Exports strategy IDs and outcomes.
Sources/CodexBarCore/Providers/Antigravity/AntigravityProviderDescriptor.swift Adds authoritative error resolution; terminal paths can bypass it and mask earlier failures (moderate).
Sources/CodexBarCLI/CLIUsageCommand.swift Integrates Antigravity fallback summaries.
Sources/CodexBarCLI/CLIHelpers.swift Formats per-source diagnostics.
docs/antigravity.md Documents diagnostic behavior.
CHANGELOG.md Records the fix.
Review details

Suppressed comments (2)

Sources/CodexBarCore/Providers/ProviderFetchPlan.swift:426

  • This log message is emitted for every provider, but only Antigravity's resolver selects the most-authoritative error; the pipeline's default resolver still surfaces the last available error. The wording therefore mislabels diagnostics for all other providers. Use provider-neutral wording such as Fetch strategies exhausted; surfacing fallback error, or make the message conditional on the provider's policy.
        CodexBarLog.logger(LogCategories.provider(provider)).debug(
            "Fetch strategies exhausted; surfacing most authoritative error",

Sources/CodexBarCore/Providers/ProviderFetchPlan.swift:403

  • The new resolver is bypassed whenever a strategy declines fallback: AntigravityOAuthFetchStrategy.shouldFallback returns false when no offline data exists, and AntigravityOfflineFetchStrategy is terminal. In those cases the pipeline returns the current OAuth/offline error directly, so a later, less-authoritative failure can still mask an earlier app/CLI failure despite this PR's first-authoritative guarantee. Return lastAvailableError ?? error for the terminal path (while retaining the current error when it is the first one) so all terminal failures use the same fold.
        Self.logPerSourceOutcomes(provider: provider, attempts: attempts, surfacedError: error)
        return ProviderFetchOutcome(result: .failure(error), attempts: attempts)
  • Files reviewed: 11/11 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +235 to +238
guard (previous as? AntigravityStatusProbeError) == .notRunning else {
return previous
}
return current
@clawsweeper

clawsweeper Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 18, 2026, 9:13 AM ET / 13:13 UTC (Revision 8).

ClawSweeper review

What this changes

Adds Antigravity source outcomes to CLI errors, safe debug logs, and diagnostic exports, and corrects misleading account-mismatch guidance.

Merge readiness

Ready for maintainer review

This PR remains useful: the merged terminal-error repair covers only part of the original work. The current patch resolves the prior findings, preserves error precedence, and has no actionable correctness or security findings.

Priority: P2
Reviewed head: b4c6a91b3277e1690dab64aa89839e215f27b0e9

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused diagnostic repair with relevant production CLI evidence, compatibility coverage, and no remaining actionable findings.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): Captured macOS production CLI output exercises the actual pipeline and exporter with synthetic empty OAuth credentials, showing the new safe failure log and JSON fields; auto-chain ordering is supported separately by focused tests.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): Captured macOS production CLI output exercises the actual pipeline and exporter with synthetic empty OAuth credentials, showing the new safe failure log and JSON fields; auto-chain ordering is supported separately by focused tests.
Evidence reviewed 8 items Repository policy and introduced scope: Read the full root AGENTS.md; no nested AGENTS.md or maintainer-notes directory was present. Reviewed all 14 introduced files against the pinned merge base. Applied guidance on focused CLI validation, isolated fixtures, and provider data separation; no builds or tests were executed during this read-only review.
Prior findings resolved: Both terminal and exhausted failures now emit diagnostics. Cancellation returns before logging, successful fetches return without failure logging, and the terminal return retains the resolved error. Account-mismatch copy no longer claims OAuth succeeded.
Main and release still lack the export additions: Inspected the diagnostic attempt model on fetched main and v0.60.5: both contain kind, availability, and error category, but lack strategy identity and explicit outcome. Current-main CLI helpers also lack the Antigravity summary.
Findings None None.
Security None None.

How this fits together

CodexBar’s provider pipeline tries configured usage sources and records their results. These records feed CLI error messages, debug logs, and shareable diagnostic JSON.

flowchart TD
  A[Configured usage sources] --> B[Provider fetch pipeline]
  B --> C[Ordered source outcomes]
  B --> D[Existing error selection]
  C --> E[CLI failure summary]
  C --> F[Safe debug log]
  C --> G[Diagnostic JSON]
  D --> E
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta Production +139/-9 lines; tests +341/-13 lines Production growth implements the stated diagnostic outputs, with broader coverage for precedence, privacy, compatibility, and CLI integration.

Technical review

Best possible solution:

Retain the established source-selection policy while exposing truthful, ordered diagnostics with backward-compatible exports.

Do we have a high-confidence way to reproduce the issue?

Yes, source inspection establishes the misleading account-mismatch guidance and missing diagnostic fields on main; no current-main runtime reproduction was executed.

Is this the best way to solve the issue?

Yes. The patch extends existing diagnostic paths, preserves the maintainer-confirmed error policy, and tests legacy export compatibility in both directions.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against aca09cf2ff87.

Labels

Label justifications:

  • P2: Improves Antigravity failure diagnosis and corrects misleading guidance without changing source selection.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Captured macOS production CLI output exercises the actual pipeline and exporter with synthetic empty OAuth credentials, showing the new safe failure log and JSON fields; auto-chain ordering is supported separately by focused tests.
  • proof: sufficient: Contributor real behavior proof is sufficient. Captured macOS production CLI output exercises the actual pipeline and exporter with synthetic empty OAuth credentials, showing the new safe failure log and JSON fields; auto-chain ordering is supported separately by focused tests.

Evidence

What I checked:

  • Repository policy and introduced scope: Read the full root AGENTS.md; no nested AGENTS.md or maintainer-notes directory was present. Reviewed all 14 introduced files against the pinned merge base. Applied guidance on focused CLI validation, isolated fixtures, and provider data separation; no builds or tests were executed during this read-only review. (AGENTS.md:1, b4c6a91b3277)
  • Prior findings resolved: Both terminal and exhausted failures now emit diagnostics. Cancellation returns before logging, successful fetches return without failure logging, and the terminal return retains the resolved error. Account-mismatch copy no longer claims OAuth succeeded. (Sources/CodexBarCore/Providers/ProviderFetchPlan.swift:405, b4c6a91b3277)
  • Main and release still lack the export additions: Inspected the diagnostic attempt model on fetched main and v0.60.5: both contain kind, availability, and error category, but lack strategy identity and explicit outcome. Current-main CLI helpers also lack the Antigravity summary. (Sources/CodexBarCore/Providers/ProviderDiagnosticExport.swift:264, 2ac2323629ce)
  • Merged repair is intentionally narrower: GitHub confirms fix(providers): honor resolved terminal fetch errors #3701 merged as e56bb5b. Its patch changes the terminal return to use the resolved error. The owner’s supplied comment explicitly leaves the broader diagnostic work open. (Sources/CodexBarCore/Providers/ProviderFetchPlan.swift, e56bb5b20183)
  • Export compatibility and privacy coverage: The decoder supplies defaults for legacy exports. Added coverage checks new output with a legacy reader, legacy output with the new reader, all three outcomes, and exclusion of sensitive error payloads. No settings or credential migration is introduced. (Tests/CodexBarTests/ProviderDiagnosticExportTests.swift:460, b4c6a91b3277)
  • Captured production CLI behavior: Read the complete captured macOS terminal transcript at its linked revision and in the reviewed checkout. With isolated configuration and empty synthetic OAuth credentials, production usage emits the safe per-source failure log and diagnose emits strategyID, outcome, and auth category. This proves the explicit-OAuth diagnostic path; auto-mode ordering has separate helper and pipeline coverage. (.github/pr-proof/antigravity-source-diagnostics.log:1, 0ad0f8421c37)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Yuxin-Qiao: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (7 earlier review cycles)
  • reviewed 2026-09-16T10:19:36.387Z sha 58f2602 :: needs real behavior proof before merge. :: [P2] Apply the selected error and diagnostics to terminal failures
  • reviewed 2026-09-16T11:34:11.243Z sha 40b7239 :: needs real behavior proof before merge. :: [P2] Apply the selected error and diagnostics to terminal failures
  • reviewed 2026-09-17T06:08:43.577Z sha c3754b3 :: needs real behavior proof before merge. :: [P2] Emit per-source diagnostics before terminal failure returns | [P2] Keep retained account-mismatch guidance truthful after OAuth fails
  • reviewed 2026-09-17T06:44:41.458Z sha f2fd17d :: needs real behavior proof before merge. :: [P2] Emit per-source diagnostics before terminal failure returns | [P2] Keep retained account-mismatch guidance truthful after OAuth fails
  • reviewed 2026-09-18T11:33:48.845Z sha 0ad0f84 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-18T11:51:40.398Z sha 0ad0f84 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-18T12:56:48.479Z sha b4c6a91 :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 16, 2026
steipete added a commit that referenced this pull request Sep 17, 2026
Apply the existing fallback error resolver to terminal failures as well as exhausted fallback chains. Preserve cancellation, original routing errors, and per-attempt diagnostics.

Extracted from #3683, whose broader proposal remains open.

Co-authored-by: hhh2210 <hzy2210@gmail.com>
@steipete

Copy link
Copy Markdown
Owner

The terminal-error repair is now on main through #3701, merged as e56bb5b. An unavailable final source now preserves the error selected by the existing provider policy, while cancellation and later substantive failures keep their existing precedence. Each source still records its original diagnostic.

Validation: make test-fast FILTER='ProviderFetchErrorTests|AntigravityCLIHTTPSFetchStrategyTests|ProviderArchitectureGatekeeperTests' passed 110 tests after reproducing 13 failing assertions on the prior implementation. make check, independent P2 review, and every macOS/Linux CI check passed: https://github.com/steipete/CodexBar/actions/runs/35173592826. All provider inputs were synthetic.

Thanks @hhh2210. This extracts only the terminal resolver repair; the broader first-error/per-source diagnostic proposal in #3683 remains open.

@hhh2210 hhh2210 changed the title fix(antigravity): surface first authoritative auto error with per-source diagnostics fix(antigravity): complete per-source failure diagnostics Sep 18, 2026
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Sep 18, 2026
@clawsweeper clawsweeper Bot added the status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. label Sep 18, 2026
@steipete
steipete merged commit b6d885c into steipete:main Sep 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants