FE-1218: Fix Petrinaut optimization streaming backpressure and cancellation latency#9057
Conversation
…lation latency NodeAPI (apps/hash-api): - Race the backpressure wait against drain, close, and the request lifecycle abort signal, removing the losing listeners via an internal AbortController so stalled writes cannot leak listeners, strand a rejected promise, or hold an optimization slot past the timeouts. - Mark terminal events as committed at write hand-off so an abort during the final backpressured write cannot append a second, contradictory terminal event. - Write the failure-path terminal event without awaiting backpressure so teardown never blocks on a stalled client. - Skip transport heartbeats while the response buffer needs draining. Petrinaut Optimizer (apps/petrinaut-opt): - PetrinautModel.close(graceful=False) signals the CLI process group immediately instead of waiting up to five seconds for stdin EOF; the graceful EOF wait is now reserved for studies that completed normally. All failure, timeout, and cancellation paths terminate promptly. - Share one idempotent per-run cleanup between the stream wrapper and a StreamingResponse background task, so a client that aborts before the body is ever pulled cannot leak the admission slot or a live CLI. - Close an abandoned CLI via an initializer done-callback when a second cancellation races the init-cancel recovery. - Emit Retry-After on the 429 study-limit response and declare it in the OpenAPI spec. Tests cover drain-first, close-first, client abort, idle timeout during backpressure, single-terminal-event, heartbeat suppression, slot release, prompt process-group termination, and 429/Retry-After propagation across NodeAPI, the optimizer client, and the Python service.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
github-actions
Bot
added
area/apps > hash*
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9057 +/- ##
=======================================
Coverage 59.22% 59.23%
=======================================
Files 1391 1391
Lines 135140 135153 +13
Branches 6223 6226 +3
=======================================
+ Hits 80032 80053 +21
+ Misses 54152 54142 -10
- Partials 956 958 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PR SummaryMedium Risk Overview hash-api NDJSON proxy now waits on backpressure with a race among petrinaut-opt adds Tests cover backpressure teardown, slot release, timeouts, single terminal events, prompt CLI shutdown, and 429 header propagation. Reviewed by Cursor Bugbot for commit 7e9f6ef. Bugbot is set up for automated code reviews on this repo. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7e9f6ef. Configure here.
3 participants
🌟 What is the purpose of this PR?
Hardens the Petrinaut optimization streaming and cancellation path from #9040 so a slow or disconnecting client cannot leak NodeAPI listeners, promises, or optimizer capacity, and so cancelling a busy study terminates its CLI promptly. Also restores and regression-tests propagation of the optimizer's
429/Retry-Afterthrough NodeAPI.🔗 Related links
🔍 What does this change?
create-petrinaut-optimization-handler.ts): races the backpressure wait againstdrain,close, and the request abort signal, removing the losing listeners via an internalAbortController— no accumulated listeners, no orphaned rejected promise, no slot held past the timeouts.petrinaut_client.py):close(graceful=False)signals the CLI process group immediately instead of waiting up to 5s for stdin EOF a mid-trial CLI never observes; the graceful EOF wait is reserved for studies that completed normally. An idempotent per-run cleanup shared between the stream wrapper and aStreamingResponsebackground task covers a client that aborts before the body is pulled.Retry-Afteron the429study-limit response (declared in the OpenAPI spec); the client preserves status +Retry-After, and NodeAPI passes them through.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
network erroron long optimization streams — the evidence points to a max-connection-duration reset at an intermediate proxy/LB, upstream of NodeAPI. This PR removes NodeAPI as a suspect and guarantees a downstream reset frees capacity and kills the CLI promptly; the confirming logs land in FE-1221: Add structured correlated logging across the Petrinaut optimization pipeline #9059.🛡 What tests cover this?
New/updated unit tests in
apps/hash-api,apps/petrinaut-opt, and@local/petrinaut-optimizer-clientcovering: drain-first, close-first, client abort, idle timeout during backpressure, single-terminal-event, heartbeat suppression under backpressure, prompt process-group termination (fake and real subprocess), slot released exactly once (including never-started stream generators and double cancellation), and429/Retry-Afterpropagation end to end.❓ How to test this?
cd apps/petrinaut-opt && uv run pytestturbo run test:unit --filter '@apps/hash-api' --filter '@local/petrinaut-optimizer-client'