Skip to content

Drain daemon responses during store uploads - #3

Draft
ryantm wants to merge 1 commit into
feat/direct-dynamic-installablesfrom
fix/duplex-store-uploads
Draft

ryantm wants to merge 1 commit into
feat/direct-dynamic-installablesfrom
fix/duplex-store-uploads

Conversation

@ryantm

@ryantm ryantm commented Nov 9, 2026 •

Copy link
Copy Markdown

Summary

Drain worker-daemon responses while sending an AddMultipleToStore upload. Previously the client finished writing every frame before reading stderr; a daemon sending enough progress before accepting more input could fill both socket buffers and deadlock.

  • Send framed data on a scoped, session-bound thread while the caller continues parsing daemon messages and invoking callbacks.
  • Terminate the framed stream on a body error or panic, restore byte accounting, and unblock the sender if the daemon rejects the upload or a callback panics.
  • Make streamed decompression readers transferable to the sender thread, retaining the existing size, ratio, and compression-layer limits.
  • Add bounded socket regressions for duplex backpressure, open-socket rejection, local errors/panics, callback panic, and connection reuse.

Stacked on the direct dynamic-output installables branch. This is a worker-protocol correction, not a timeout increase or an evaluator speedup claim.

Validation

The new backpressure test fails on the unchanged implementation: the daemon writes a 4 MiB log message before reading an 8 MiB upload, and both sides hit their socket timeout. With concurrent draining, the same test completes successfully.

Passed: six new upload regression cases, the existing upload-error regression, 14 HTTP/decompression tests, two fetched-closure tests, two layering tests, and four dynamic-derivation tests. Formatting, structural lint, and Clippy with warnings denied all pass.

The broad library run passed 1,365 tests, with two previously established root-permission failures and four ignored tests. Real daemon integration passed five tests; its three failures were reproduced on the unchanged parent (two cases requiring kernel namespaces and a garbage-collector error-message expectation).

Post-publication check: the remote head matches the locally validated commit. macOS and Windows CI are pending.

No live deployment or workload timing claim is included. The protocol regression establishes the deadlock independently; it does not isolate every cause of slower evaluation.

~ written by ⠕ Replit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant