Skip to content

fix: report the gateway's refusals honestly - #175

Open
viztor wants to merge 1 commit into
claude/vigilant-mendel-bctz9n-p-t3from
claude/vigilant-mendel-bctz9n-p-r3
Open

viztor wants to merge 1 commit into
claude/vigilant-mendel-bctz9n-p-t3from
claude/vigilant-mendel-bctz9n-p-r3

Conversation

@viztor

@viztor viztor commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Phase P-R3 of docs/improvement-plan.md, stacked on #173.

What changed

Item Change
L2: honest refusal codes New pure module src/utils/gateway-refusal.ts. It reads the gateway's typed refusal from pi's failure text and maps it to one of two codes. FREE_TIER covers FreeTierError and FreeUsageLimitError. MODEL_UNAVAILABLE covers RegionError, ModelDeprecated and ModelProtocolUnsupported. Neither code is AUTH, QUOTA or ACCOUNT_QUOTA, so the chat UI shows the gateway's own words instead of replacing them with "API key is invalid" or "Request quota exhausted.". Neither code is in the host's default retryable set. pi's message is kept unchanged, and status is read only from the head of the text.
Plane adapter PlaneAdapter.prepareCall and stream() wrap the prepared stream and rewrite the final finish failure. pi-ai's own routes and stepped-aside planes (D27) keep the bridge's codes.
L7: free-tier tool fallback src/tool-fallback.ts now treats a read or bash tool without non-empty properties as missing, and replaces it rather than adding a duplicate. A free chat-completions body that had no tools gets tool_choice: "none". Responses bodies are never touched.
Probe rows (recorded, never asserted) test/support/keyed-probes.ts adds four kinds of row: OC9 (a raw, non-ses_ session id, keyed and free), OC8 (session-only headers, keyed), OC6 (the harness User-Agent, keyed and free), and a free baseline. A keyless 429 from the usage cap now reads as free-usage-limited, not rate-limited, and is left out of the Retry-After record. They are formatted into the variants table in docs/redirect-planes.md §13.
OC10 Two remaining bullets in the README's paired section "With OpenCode Console installed".
L9, L10 No code. Run 43542273415 answered no keyed 429, so there was no Retry-After to act on. L10's reasoning_effort: "none" was never measured: the reasoning model was refused both with and without the field.
Docs and tests The plan, AGENTS.md, CONTRIBUTING.md, docs/testing.md and the CI comments now describe the above. New tests: test/gateway-refusal.test.ts, test/mock/plane-refusals.mock.ts, and new cases in the plane-provider, tool-fallback, keyed-probes and endpoint-record tests.

Behaviour changes

  • A plane turn the gateway refuses on its own terms (free tier, usage cap, region, retired model, unsupported protocol) used to finish as AUTH, QUOTA, INVALID_REQUEST or PI_AI_ERROR, depending on how the bridge read the text. It now finishes as FREE_TIER or MODEL_UNAVAILABLE. The conversation shows the gateway's words and, except on Google-protocol models (whose failure text carries none), its status. It is not retried by default.
  • The free-tier fallback replaces a read or bash that has no properties instead of adding a second copy. It sends tool_choice: "none" on a free chat-completions turn that had no tools.
  • The live keyed-probes record's empty Retry-After line now reads "Retry-After on a keyed 429: no keyed 429 was answered." (it was "no 429 was answered."). This keeps it from appearing to contradict keyless free rows that answer free-usage-limited (429) in the variants table.

Review

Three rounds, each with correctness, tests and rules lenses, and a verify pass after each fix round.

Round Found Upheld Fixed
1 24 22 22
2 10 10 10
3 6 5 5

Round 1 found drift in the evidence and docs: L10 read as "not accepted", a cited run that §13 did not hold, the chat UI replacing five codes rather than two, and the Google no-status qualifier. It also found assertions that could never fail. Round 2 caught the keyless usage-cap 429 being recorded as rate-limited, and an L7 guide naming a verdict the table could never print. Round 3 caught doc, comment and test-name drift left by round 2's narrowing of Retry-After to keyed rate-limit 429s. None of round 3's fixes changed behaviour beyond the record line above.

These reviews ran on the general model instead of the dedicated reviewer, because the dedicated reviewer was unavailable: spec:P-R3, review:P-R3:correctness#1, review:P-R3:rules#1, verify:P-R3#1.1, review:P-R3:correctness#2, verify:P-R3#2.1, review:P-R3:correctness#3, verify:P-R3#3.1.

Verification

Node Step Result
26.12.1 pnpm run check All 310 files formatted; 0 errors, 12 warnings (at the ceiling)
26.12.1 vp test run (alpha channel) 132 files passed; 2105 tests passed, 1 skipped
26.12.1 vp test run shuffled, seeds 1 and 42 132 files; 2105 passed, 1 skipped (each seed)
26.12.1 mock suite (vitest.mock.config.ts) 9 files; 120 passed
26.12.1 pnpm run build + scripts/check.ts Build complete (lib/index.mjs 358.69 kB); consumer install with plain npm (19 entries); lib/client.js 136211 bytes, under the 139264 tripwire; check passed
26.12.1 rc channel (dsh-llm-pi-ai 0.2.2-rc.2, pi 1.5.2, cordis 4.1.0) unit 132 files, 2105 passed, 1 skipped; mock 9 files, 120 passed; switched back to alpha with no residue
26.12.1 vp test run --coverage Statements 98.35%, Branches 95.97%, Functions 98.66%, Lines 98.35% (ratchet 98 / 95.4 / 97.9 / 98)
24.22.1 shuffle seed 7 + mock 132 files, 2105 passed, 1 skipped; mock 9 files, 120 passed
26.12.1 binary e2e, alpha and rc npm cells 15 passed, 2 skipped (17) on each; the skips are the two known host-limitation cases
n/a local-path and credential scan of the diff and new files no machine path; the one credential-shaped hit is the existing fake sk-probe-… test fixture

The check, unit (alpha), mock and build/check.ts rows were re-run on the final tree immediately before this commit, with the same counts. The other rows come from the phase's gate run after the last fix round. The freshness scope answers report for all three checks on this diff, so catalog freshness does not gate this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD

The chat UI replaces an AUTH failure with "API key is invalid" and a
QUOTA one with "Request quota exhausted.", so a plane turn the gateway
refused for its own reasons (free tier, usage cap, region, retired
model, unsupported protocol) read as a bad key or an empty quota. A new
pure module, src/utils/gateway-refusal.ts, reads the gateway's typed
refusal from pi's failure text and finishes the turn as FREE_TIER or
MODEL_UNAVAILABLE instead: neither code is replaced by the UI and
neither is retried by default, so the user sees the gateway's own
words and, except on Google-protocol models, its status. The plane
adapter applies it in prepareCall and stream; pi-ai's own routes and
stepped-aside planes keep the bridge's codes.

The free-tier tool fallback now treats a read or bash tool without
properties as missing, replacing it rather than adding a duplicate,
and sends tool_choice "none" on a free chat-completions body that
carried no tools.

The keyed probes gain recorded rows for a raw session id, session-only
headers and the harness User-Agent, plus a free baseline; a keyless
usage-cap 429 now reads as free-usage-limited and is left out of the
Retry-After record. The README gains the remaining bullets of its
"With OpenCode Console installed" section. The rate-limit and
reasoning-effort lessons land no code: the recorded run answered no
keyed 429 and never measured the field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants