FE-1224: Add detached, reconnectable optimization runs to the Petrinaut optimizer#9064
FE-1224: Add detached, reconnectable optimization runs to the Petrinaut optimizer#9064kube wants to merge 4 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
github-actions
Bot
added
area/apps > hash*
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
| if (response.destroyed || response.writableEnded) { | ||
| throw new Error("The optimization client disconnected"); | ||
| } | ||
| if (response.write(`${JSON.stringify(event)}\n`)) { |
There was a problem hiding this comment.
Detected directly writing to a Response object from user-defined input. This bypasses any HTML escaping and may expose your application to a Cross-Site-scripting (XSS) vulnerability. Instead, use 'resp.render()' to render safely escaped HTML.
⭐ Fixed in commit 760a6f0 ⭐
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9064 +/- ##
=======================================
Coverage 59.23% 59.24%
=======================================
Files 1391 1391
Lines 135181 135208 +27
Branches 6235 6242 +7
=======================================
+ Hits 80073 80100 +27
+ Misses 54145 54143 -2
- Partials 963 965 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PR SummaryMedium Risk Overview Introduces JSON structured logging ( hash-api gains request-scoped optimization logs (accept, stream open, outcomes, capacity/validation rejects), forwards Reviewed by Cursor Bugbot for commit 760a6f0. Bugbot is set up for automated code reviews on this repo. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit fec71b8. Configure here.
…ization pipeline
One optimization is now traceable end to end: NodeAPI's x-hash-request-id
is forwarded upstream, the optimizer's run_id comes back as
X-Optimization-Run-ID and is logged on both sides, and the CLI receives
the run id at spawn as PETRINAUT_CORRELATION_ID.
- petrinaut-optimizer-client: openPetrinautOptimizationStream accepts a
requestId (sent as x-hash-request-id) and returns { events,
optimizationRunId } read from the X-Optimization-Run-ID header.
- hash-api: request-scoped child logger (Apollo pattern); logs request
acceptance, validation failures (issue counts and schema paths only),
capacity rejections, stream opening, and a terminal outcome with
durationMs (completed / timeout kind / upstream-error /
client-disconnected — disconnects no longer return silently).
- petrinaut-opt: stdlib JSON-lines logging to stdout (logging_config.py,
HASH_PETRINAUT_OPT_LOG_LEVEL, idempotent setup); inbound
x-hash-request-id and traceparent headers threaded into every study
lifecycle log (admission, capacity rejection, initialization,
study start/completion, per-trial outcomes, wall-clock ceiling,
disconnect, worker join timeout, CLI termination path, slot release).
CLI stderr is no longer discarded: each line is forwarded to a pn_cli
logger, truncated with a bounded pending buffer.
- petrinaut-cli: minimal JSON stderr diagnostics after the unchanged
ready handshake (bootstrap_completed, per-request id/method/durationMs/
outcome, stdin_eof), correlation id injected via cli.ts.
Logs never include manifests, user-authored code, or raw request bodies;
docs updated (README logging section, CLI stderr contract, threat model).
- Cap forwarded CLI stderr at 1000 lines/run (drain the rest) so a chatty or hostile CLI cannot flood the log pipeline; seed the drain thread with bytes read past the bootstrap handshake so an early CLI diagnostic is forwarded rather than dropped. - Truncate the per-trial pruned-error field and stop interpolating the raw CLI error into the log message; correct the docstring. - Log an error type and coarse category instead of the raw initialization error string, which can embed a user identifier or expression error; the full message still reaches the requester in the 500 detail. - Cap validation issue-path segments before they reach the 400 response or the log, since manifest keys are user-controlled and unbounded. - Capture the optimizer run id from a failed response so a pre-stream upstream failure still correlates end to end. - Charset-validate the traceparent trace id instead of int(..., 16), which accepted 0x-prefixed, underscored, and signed values. - Document the aggregate stderr cap and header validation in the threat model.
fec71b8 to
760a6f0
Compare
3 participants
🌟 What is the purpose of this PR?
Decouples a Petrinaut optimization study's lifetime from the HTTP connection consuming it. A run is created detached and identified by a
run_id; consumers attach, detach, and re-attach to its event stream with a cursor, and killing the socket no longer kills the study. This is the optimizer half of the fix for the intermittent stagingnetwork erroron long optimization streams (a mid-run transport reset currently destroys the study); cancellation becomes an explicitDELETEinstead of being implied by disconnect.🔗 Related links
🚫 Blocked by
🔍 What does this change?
POST /optimize/runs— admits (existing 4-slot cap, 429 +Retry-After), starts the CLI + Optuna study in a background task owned by the run, returns201 {"run_id"}immediately.GET /optimize/runs/{run_id}/events?cursor=N(orLast-Event-ID) — SSE that replays buffered events with sequence > cursor (id:field carries the sequence, starting at 1), then live-tails with heartbeats. Disconnect does not affect the run; a new attachment supersedes a stale one; a terminal run replays fully then closes; 404 for unknown/expired runs.DELETE /optimize/runs/{run_id}— idempotent explicit cancel: prompt CLI close, slot release, dedicatedevent: cancelledterminal frame (also used by the reaper and shutdown).HASH_PETRINAUT_OPT_DETACH_GRACE_SECONDS(default 300; ≤0 disables) is cancelled; terminal run logs are retained for the same window for replay./optimize/alland/optimize/bestkeep their exact behavior (stream from start, cancel-on-disconnect) — their shared worker scaffolding was extracted and reused by the new engine rather than rewritten, and all 83 pre-existing tests pass unmodified.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
🐾 Next steps
@local/petrinaut-optimizer-clientproxy the create/attach/cancel routes with ownership checks.🛡 What tests cover this?
apps/petrinaut-opt: 83 → 112 pytest. New coverage: create → 201 with the slot held until terminal and released exactly once; replay + live tail with strictly increasingid:s; cursor andLast-Event-IDre-attach (query wins); post-terminal full replay then clean close; disconnect not cancelling the run; attachment supersession; prompt idempotent DELETE (graceful=False close, terminal frame); reaper reaping unattached runs while sparing attached ones (and disabled at ≤0); retention expiry; pump-crash backstop with redacted logging; lifespan shutdown cancelling pumps; 404/429+Retry-After/413/500-with-run-id; OpenAPI shape for the three routes. The OpenAPI drift gate (yarn test:unit) is clean.❓ How to test this?
cd apps/petrinaut-opt && uv run pytestPOST /optimize/runswith the checked-in supply-chain manifest, note therun_id.curl -N .../optimize/runs/<run_id>/events, kill the curl mid-run, re-run it with?cursor=<last id seen>— confirm the study kept running and the stream resumes after the cursor.DELETE .../optimize/runs/<run_id>— confirm a promptevent: cancelledterminal frame and slot release.🤖 Generated with Claude Code