Repository navigation
fix: precompile Python bytecode in the tools image so pypi MCP servers start cold in time - #6
Merged
Conversation
…s start cold in time
Contributor
|
I'll fix CI failures and address comments from users with write access that start with 'Devin'.
|
davidlbowman
approved these changes
Nov 9, 2026
davidlbowman
left a comment
There was a problem hiding this comment.
hmmmm, very good python precompilation lgtm (didn't read)
gfadlallah
approved these changes
Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Codex's streamable-HTTP MCP client gives each server 30 s (
startup_timeout_secdefault) to answerinitialize. On a freshtoolscontainer the six pypi servers (okta, atlassian, aws-api, aws-cloudwatch, aws-iam, pagerduty) regularly miss that window:uv pip installwrites no bytecode, so the venvs under/opt/pyship 20,706.pyfiles and zero__pycache__, and the first spawn of every server compiles its whole import graph, while Codex initializes all 32 servers at once. The runs that fail are always those pypi servers, the first error lands at exactly 30.0 s after session start, and a second session in the same container (bytecode now cached) is ~2× faster — so the warm-up is the bytecode, not anything network-side.Fix:
UV_COMPILE_BYTECODE=1in thepy-buildstage ofimages/tools/Dockerfile, so the venvs are compiled at image build time (20,683.pycfor 20,706.py; the 23 misses are Python-2-only files uv skips). No runtime change: the bridge still execs the venv binary, one child per MCP session.The image tag is a content hash, so the tools image moves
521c33382f76f96a → 7d245e8909a89dd0and everytasks/*/docker-compose.yamlis regenerated bytaskgen(151 files, 1 line each). Image size 2.96 GB → 3.42 GB (+461 MB, +15.6%).Measurements
Local build of
images/toolsandimages/worldat this commit, okta scenario compose (world + tools only), 32 concurrent streamable-HTTPinitializerequests against the tools container (what Codex does at session start), time to theinitializeresponse, cold container each row. The tools container's CPU share is varied withcpus:to stand in for the shared sandbox CPU the agent, world and tools compete for.npm servers for reference: slowest (datadog) 2.9 / 5.2 / 11.4 / 26.5 s before, unchanged after (no Python involved). At 1 CPU the stock image reproduces the 30 s failures seen in real runs; the compiled image's slowest server is 15.8 s there, a 1.9× margin.
Residual risk: the fix roughly halves cold start, it does not remove the CPU dependence. If the tools container is squeezed to ≈0.5 CPU while all 32 servers initialize, the pypi servers (and datadog) still take 31–35 s and Codex times them out. Nothing here changes the agent's MCP timeout or readiness handling.
Sequencing
Tags are published by the
tool-bench imagesworkflow on push tomain, not on the PR, sotool-bench-tools:7d245e8909a89dd0does not exist until this merges and that run finishes (it builds only tags that are missing). Runs that clonemainin the interval between merge and publish will fail to pull the tools image; wait for the publish run before launching anything frommain.Verification
cargo fmt --check,cargo clippy -D warnings,cargo test --workspace,taskgen --checkgreen locally.images/toolsfrom this branch and counted__pycache__/.pycin/opt/py(above); world + tools came up healthy and all 32 servers answeredinitializefrom the compiled image.🟢 Devin Review found nothing to fix · View findings