fix(responses): refuse oversized input and reframe routed compaction to SSE (#1128) - #1167
Conversation
A chained-turn replay can balloon a request far past the model's context window (observed: a 4x expansion pushed a ~400k-token conversation to 1.6M input tokens). The proxy forwarded it verbatim; processing it on Windows ballooned bun RSS and native-crashed the whole service (upstream Bun memory bug, lidge-jun#314), taking every active thread down until restart. Reject the request with a clean 413 before any upstream I/O when the parsed input exceeds the model's configured modelContextWindows value. The client compacts well before the window, so the guard only fires on abnormal duplication.
The lidge-jun#875 transport policy forces a bounded JSON upstream for providers like deepseek (modelResponsesUpstreamStreaming: false), even when the client asked for SSE. The passthrough branch already reframes that JSON back to the canonical terminal SSE sequence, but routed-compaction turns skip the passthrough branch: they returned application/json, so Codex SSE parser hit EOF at the first byte and the remote compact task failed with "stream closed before response.completed", retrying 6x and leaving the thread unusable. Mirror the passthrough reframe in both routed branches (runTurn and parseStream): when the client requested stream and the upstream policy forced JSON, emit response.created, one output_item.done per output item (including the synthetic compaction item), response.completed, then [DONE].
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
⏳ DRAFT
What to do
Review readiness checklist
0/4 boxes ticked. This PR stays in draft until every box above is ticked. |
|
✅ Deterministic PR hygiene checks passed. |
|
Closing for now — will re-open after reviewing the process. Thanks! |
Summary
Two fixes on the Codex desktop context/compaction failure chain (reported upstream in #1128):
deepseek-v4-flasheven when the client asked for SSE. The passthrough branch already reframes that JSON into the canonical terminal SSE sequence, but routed-compaction turns returnedapplication/json, so Codex's SSE parser hit EOF and remote compaction failed deterministically with "stream closed before response.completed", retried 6x, and left the thread unusable. Both routed return points (runTurnandparseStream) now apply the same reframe.Verification
bun run typecheck— clean.bun test tests/responses-input-guard.test.ts— 4 pass / 0 fail (guard above/below the window; compaction reframe with and withoutstream: true).response.created -> output_item.done (compaction) -> response.completed -> [DONE]instead of bare JSON; an oversized input is rejected with 413 without reaching upstream.Checklist
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
All CI tests are green on my local testing.
I pushed my PR to the latest dev commit.
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.