Skip to content

fix: precompile Python bytecode in the tools image so pypi MCP servers start cold in time - #6

Merged
LukasParke merged 1 commit into
mainfrom
devin/1794185951-tools-image-bytecode
Nov 9, 2026
Merged

LukasParke merged 1 commit into
mainfrom
devin/1794185951-tools-image-bytecode

Conversation

@LukasParke

@LukasParke LukasParke commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Codex's streamable-HTTP MCP client gives each server 30 s (startup_timeout_sec default) to answer initialize. On a fresh tools container the six pypi servers (okta, atlassian, aws-api, aws-cloudwatch, aws-iam, pagerduty) regularly miss that window: uv pip install writes no bytecode, so the venvs under /opt/py ship 20,706 .py files and zero __pycache__, and the first spawn of every server compiles its whole import graph, while Codex initializes all 32 servers at once. The runs that fail are always those pypi servers, the first error lands at exactly 30.0 s after session start, and a second session in the same container (bytecode now cached) is ~2× faster — so the warm-up is the bytecode, not anything network-side.

Fix: UV_COMPILE_BYTECODE=1 in the py-build stage of images/tools/Dockerfile, so the venvs are compiled at image build time (20,683 .pyc for 20,706 .py; the 23 misses are Python-2-only files uv skips). No runtime change: the bridge still execs the venv binary, one child per MCP session.

The image tag is a content hash, so the tools image moves 521c33382f76f96a → 7d245e8909a89dd0 and every tasks/*/docker-compose.yaml is regenerated by taskgen (151 files, 1 line each). Image size 2.96 GB → 3.42 GB (+461 MB, +15.6%).

Measurements

Local build of images/tools and images/world at this commit, okta scenario compose (world + tools only), 32 concurrent streamable-HTTP initialize requests against the tools container (what Codex does at session start), time to the initialize response, cold container each row. The tools container's CPU share is varied with cpus: to stand in for the shared sandbox CPU the agent, world and tools compete for.

tools cpus server before (s) after (s)
4.0 aws-cloudwatch / okta / atlassian / aws-api / pagerduty / aws-iam 7.6 / 6.9 / 6.5 / 5.8 / 4.1 / 3.8 3.0 / 4.1 / 3.2 / 3.1 / 2.7 / 2.5
2.0 same 12.4 / 11.6 / 10.6 / 10.2 / 7.5 / 7.1 6.4 / 7.0 / 6.2 / 5.5 / 5.1 / 4.4
1.0 same 30.3 / 30.1 / 26.3 / 24.7 / 17.5 / 16.0 15.0 / 15.8 / 13.9 / 13.7 / 11.3 / 10.8
0.5 same 59.7 / 58.0 / 54.1 / 51.4 / 39.1 / 36.9 33.8 / 35.3 / 32.9 / 31.2 / 28.0 / 25.1

npm servers for reference: slowest (datadog) 2.9 / 5.2 / 11.4 / 26.5 s before, unchanged after (no Python involved). At 1 CPU the stock image reproduces the 30 s failures seen in real runs; the compiled image's slowest server is 15.8 s there, a 1.9× margin.

Residual risk: the fix roughly halves cold start, it does not remove the CPU dependence. If the tools container is squeezed to ≈0.5 CPU while all 32 servers initialize, the pypi servers (and datadog) still take 31–35 s and Codex times them out. Nothing here changes the agent's MCP timeout or readiness handling.

Sequencing

Tags are published by the tool-bench images workflow on push to main, not on the PR, so tool-bench-tools:7d245e8909a89dd0 does not exist until this merges and that run finishes (it builds only tags that are missing). Runs that clone main in the interval between merge and publish will fail to pull the tools image; wait for the publish run before launching anything from main.

Verification

  • cargo fmt --check, cargo clippy -D warnings, cargo test --workspace, taskgen --check green locally.
  • Built images/tools from this branch and counted __pycache__ / .pyc in /opt/py (above); world + tools came up healthy and all 32 servers answered initialize from the compiled image.

🟢 Devin Review found nothing to fix · View findings

Devin Review

@devin-ai-integration

devin-ai-integration Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@davidlbowman davidlbowman left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmmm, very good python precompilation lgtm (didn't read)

@LukasParke
LukasParke merged commit 0a66c60 into main Nov 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants