Skip to content

compute: exec hardening — no auth-replay of mutations, wall-clock deadlines, structured errors#305

Open
samcm wants to merge 4 commits into
masterfrom
fix/exec-hardening
Open

compute: exec hardening — no auth-replay of mutations, wall-clock deadlines, structured errors#305
samcm wants to merge 4 commits into
masterfrom
fix/exec-hardening

Conversation

@samcm

@samcm samcm commented Jul 15, 2026

Copy link
Copy Markdown
Member

panda compute sandboxes exec had four sharp edges: a 401/403 on the proxy hop replayed the entire buffered request, re-running the guest command; the CLI had no HTTP deadline anywhere, so a hung path blocked it for up to 5 minutes; the proxy's default write timeout exactly equalled the 5-minute exec ceiling; and upstream compute errors were flattened into an opaque HTTP 500: internal server error.

Auth-refresh replay is now gated to idempotent (GET/HEAD) compute requests. Mutations invalidate the cached token and return a clear retry message instead of silently double-executing. Exec calls run under a wall-clock deadline of the guest timeout plus 60s, the proxy default write_timeout moves to 6m with a 5m30s ResponseHeaderTimeout on the reverse-proxy transport, and non-2xx JSON error bodies from the compute backend are forwarded verbatim so the CLI can render their code and request_id fields alongside a new hint for 500s.

samcm added 4 commits July 15, 2026 12:16
…ejection

The proxy layer re-sent the entire request after a 401/403
invalidate-and-refresh, which silently re-executed guest commands for
compute.exec_sandbox. Thread a replay policy down to the proxy request:
compute mutations refresh the token but return a clear retry error
instead of re-sending; read-only requests and all other operations keep
the single retry.
The CLI HTTP client has no timeout, so a hung exec path stalled the
command until the proxy write timeout expired. Cap the exec call at the
guest timeout (or the 30s server default) plus a minute of transport
headroom, and point the user at the sandbox when the deadline fires.
The default write timeout equalled the 5m guest command maximum, so a
near-max exec raced the proxy deadline; bump it to 6m. Give the reverse
proxy transport a 5m30s response-header timeout so a hung backend fails
fast instead of holding the connection until the write timeout.
Non-2xx compute backend bodies were re-wrapped as {"error": "<raw json>"},
hiding the structured code and request_id fields from the CLI. Forward
JSON error objects verbatim, render code/request_id in CLI error
messages, and add a 500 hint pointing at the sandbox.
@github-actions

Copy link
Copy Markdown
Contributor

🐼 Smoke eval — c4033b8: ✅ 8/8 pass

📊 Interactive report — tokens p50 17,234 · tokens/solve 17,118.

Reference points: master@a045b91 100% · v0.38.1 100%.

question result tokens tools
forky_node_coverage 15,414 6
tracoor_node_coverage 18,125 8
mainnet_block_arrival_p50 17,727 11
list_datasources 13,023 2
block_count_24h 16,228 8
missed_slots_24h 16,741 8
chartkit_default_arrival_distribution 20,413 13
storage_upload_session_scoped 19,275 13
🔭 Langfuse traces (8 runs; ⚠️ = failed)

The report walks this branch's commits against the master baseline and the most recent release. A self-contained copy is in the run's eval-smoke-* artifact.

@samcm
samcm marked this pull request as ready for review July 15, 2026 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants