feat: add tool_code_execution() built-in tool (OpenAI, Anthropic, Google)#332
Draft
cpsievert wants to merge 9 commits into
Draft
feat: add tool_code_execution() built-in tool (OpenAI, Anthropic, Google)#332cpsievert wants to merge 9 commits into
cpsievert wants to merge 9 commits into
Conversation
…ent) Widen last_code_execution_container_id's parameter to Sequence[Turn] instead of list[Turn] so callers can pass list[AssistantTurn] without tripping pyright's list invariance (matches precedent in _chat.py and _turn.py). Also documents the type: ignore in ToolCodeExecution._openai_definition per project convention.
Adds live-conversation tests for the code execution built-in tool across all three providers, plus a "Code execution" section in the tools guide. Live testing against the real Anthropic API revealed two inaccuracies in the code execution wiring merged earlier: the documented beta header (code-execution-2026-05-21) doesn't exist -- the correct value is code-execution-2025-05-22, paired with tool version code_execution_20250522 (the newer 20260521 version routes calls through a bash/text-editor bundle chatlas doesn't parse). Also, contrary to the tool's own docstring, Claude's code execution does not persist Python interpreter state across separate turns (only OpenAI's does), so no persistence test/claim is made for Claude, matching the existing treatment of Google.
Recorded against live OpenAI, Anthropic, and Google APIs; scanned clean for secrets by scripts/check_vcr_secrets.py.
The streaming response accumulator only copied stop_reason/stop_sequence/ usage from message_delta events, silently dropping the container field Anthropic sends there (never on message_start). Since chat.chat() streams by default, this meant code execution's cross-turn container reuse never actually worked outside of stream=False -- confirmed live and closed by the final whole-branch review's one Important finding.
cpsievert
force-pushed
the
worktree-code-execution-tools
branch
from
July 21, 2026 23:34
de57072 to
1bb43f8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
tool_code_execution(), a provider-agnostic built-in tool that letsChatOpenAI()/ChatAnthropic()/ChatGoogle()write and run code in eachprovider's own server-side sandbox -- useful for math, data analysis, or
letting the model check its own logic, without chatlas needing to expose a
local, unsandboxed execution path. Follows the same pattern as the existing
tool_web_search()/tool_web_fetch()built-ins.v1 scope: text output only. Only stdout/stderr are surfaced as structured
content. Files the code produces (plots, CSVs, etc.) are NOT downloaded or
decoded -- the raw provider data is still available via each content's
extrafield, but there's no first-class support for those files yet.Before merging -- open item to decide on
(
code_execution_20250522). Newer versions (20250825/20260120/20260521)route execution through a
bash_code_executionsub-tool chatlas doesn'tparse yet, so this PR deliberately stays on the oldest version rather
than adding that parsing now. Confirm we're OK shipping v1 this way, and
that we won't forget to revisit if Anthropic deprecates this version.
What changed
ContentToolRequestCodeExecution/ContentToolResponseCodeExecutioncontent types, and a
ToolCodeExecution/tool_code_execution()built-intool, following the existing web-search/fetch precedent.
code_interpreter), Anthropic (code_executionbeta tool), and Google (
code_execution) -- each translating to thatprovider's native tool shape and parsing its native response shape back
into the shared content types.
conversation automatically (so e.g. a variable defined in one turn is
available in the next, for OpenAI); Google starts fresh each turn, since
Gemini has no cross-turn persistence mechanism.
does not actually persist Python interpreter state across turns, contrary
to Anthropic's own SDK docs -- only OpenAI does. Docs/docstrings describe
this accurately rather than over-promising.
ever captured from non-streaming responses -- the streaming accumulator
silently dropped it, so container reuse never actually worked via the
default
chat.chat()path. Now fixed and regression-tested.Test plan
response-parsing across all three providers
test for the streaming container-capture fix
make check-types/make check-formatclean