Skip to content

feat(agents)!: ship Claude Code in the wheel as the default agent - #257

Open
earakely-scale wants to merge 16 commits into
mainfrom
feat/claude-code-agent
Open

earakely-scale wants to merge 16 commits into
mainfrom
feat/claude-code-agent

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Why

agent-env shipped no agent. A deploy_agent step or an agent judge that named no agent deployed a2a-default, which a fresh install doesn't have, and wrapping a harness was left to users.

What

The agents bundle, in the wheel beside hello and echo-mcp. Its one agent folder, agents/claude-code/, is @local/agentenv-framework/agents/claude-code, and its smoke task runs with agent-env run agents.

  • The image.
    • python:3.11-slim at a pinned multi-arch digest, plus Anthropic's native Claude Code binary at a pinned CLAUDE_CODE_VERSION, checked against the release manifest's sha256.
    • It runs as a non-root user, or as root where the provider ignores USER (IS_SANDBOX). CLAUDE_CONFIG_DIR is fixed, so the CLI's state doesn't follow the uid. The auto-updater and nonessential traffic are off.
    • The model's scripts get node (copied from a pinned node:22-slim, without npm or npx) and Python with a pinned boto3 (model-requirements.txt).
    • 254 MB, built natively on x86 and arm64 in 33–38 s.
  • The agent (a2a_server.py, claude_bridge.py, kit.py): an AgentEnvAgent running claude -p --output-format stream-json, ported from Scale's internal Claude Code wrapper.
    • It keeps:
      • the task env's MCP servers, through --mcp-config with the role header;
      • live and context trajectories;
      • skills, snapshots and triggers;
      • a resume after a transient API error, within the caller's budget;
      • cancellation that kills the CLI's whole process tree.
    • New: max_turns, max_thinking_tokens, and an error that explains a 404 from the endpoint.
  • The model endpoint.
    • [model] reaches the CLI as ANTHROPIC_BASE_URL and a bearer ANTHROPIC_AUTH_TOKEN. The API key is blanked. LITELLM_API_KEY, LITELLM_BASE_URL and LITELLM_EXTRA_HEADERS stay in the CLI's environment, so a script the model runs can call the endpoint.
    • Claude Code's model aliases are pinned to the run's model on that endpoint.
    • [model.headers], and the run's project and task attribution, go on every call (ANTHROPIC_CUSTOM_HEADERS).
    • It needs an endpoint serving Anthropic's Messages API, such as a LiteLLM gateway.
  • TLS checks stay on. The on-machine providers already give agent containers NODE_EXTRA_CA_CERTS.
  • install/v1. install_agent installs the agent into a container a task already runs, or onto a vm sandbox's Linux host, with install.sh from its folder.
    • It brings Claude Code's native build for the target's CPU and libc, and uv, each checked against its published sha256. The agent's requirements go into a Python of uv's own, so the target's Python is never touched.
    • Claude Code runs its commands in bash or zsh, so a target with neither (Alpine, say) gets bash from its own package manager. A target with no package manager fails, saying so.
    • The model key, base URL and [model.headers] reach the agent through a 0600 env file, not its command line.
    • The install returns once the agent answers on its port. An agent that exits first fails it, with the end of its log.
    • Each container install lives under its port, so two agents can share a container.
    • The agent works in the container's working directory, or /app on a host. The CLI's transcripts follow that directory.
    • In core, install_agent gains a trust_env param: on a sandbox on this machine it puts the local transfer CA in the container, or beside the work dir on a host, and passes the variables that find it.
    • A host install on a local sandbox gets the local store's grants only when grant_advertise_host names an address the host reaches, such as the bridge address, since host.docker.internal resolves only in containers. AGENTS.md says so.
  • A snapshot load keeps the live transcript of every other conversation the agent holds, even when the archive has an older copy of it.
  • requirements.txt is the protocol alone, and a release pins it. publish-pypi.yml now rewrites every shipped example's requirements, not only echo-mcp's.

The default agent is now this one.

  • _DEFAULT_A2A_AGENT_ID changes. A bundle run pulls the agent from the installed bundle for any agent field left unset. So a bare deploy_agent deploys Claude Code, and so does a rubrics_verifier that names no judge agent, since it's an agent judge by default.
  • An explicit direct LLM judge (use_agent_judge: false) still pulls it, but nothing is built for it.
  • The Explorer features the agent, and AGENTS.md and the example config say what the default is.

Breaking: a store agent named a2a-default is no longer the default. Set [agents] default_a2a_agent_id = "a2a-default" to keep it. A host that sets the default in its config is unaffected.

Not in this PR:

  • Peer agents and the changelog come next. A file part naming a stored object needs nothing new: core already sends it as an https grant, and this PR adds the tests.
  • The retired attribution-probe/v1 isn't served: nothing calls it, and attribution goes through agent-config's project_id.
  • install_agent takes no env_ids, so an installed agent gets no env MCP tools; deploy_agent is the step for those.
  • Caching VM-built images comes after that, so each new remote VM builds the image cold, in about 40 s on modal_vm.

Tests

  • Unit: the agent's suite, 165 tests in tst/unit/examples/claude_code/, covers the card and its install commands, turns, retries and the failure messages, trajectories, snapshots, transfers, the CLI's environment and workspace, and the image contract. install_agent_test.py covers trust_env. Unit and protocol tiers: 8766 passed.

  • Integration: run_bundle_claude_code_test.py runs the real image and the real CLI against a fake Anthropic Messages endpoint (tst/util/fake_anthropic.py) on local docker, so no model key is needed. It covers:

    • a task naming no agent;
    • the bearer key, [model.headers] and the model reaching the endpoint;
    • a second bundle reusing the image;
    • a call to the tool of echo-mcp's env, through the gateway;
    • a run's project_id posted over agent-config, as a host's runner does, reaching every model call as x-litellm-customer-id and x-litellm-tags;
    • a file part naming a stored object, which core sends as an https grant, downloaded into the task's inputs and read by the CLI.

    It passes on fresh x86 and arm64 Linux VMs.

  • Install integration: run_bundle_claude_code_install_test.py installs the default agent, from a step naming none, into a Debian and an Alpine task container with no Python of their own (Alpine has no bash either), as two agents into one container, again into the same container by a retried step, and onto the sandbox's host under a name with a space. A skill then reaches it through a read grant over the local CA's TLS. Each time, the CLI runs a command where the task's files are, against the fake endpoint, and reads the skill back. It passes on x86 and arm64 Linux VMs.

  • Live, with a real model through a LiteLLM gateway, on fresh Linux VMs: the results are in the verification comment.

  • Chaos: the results are in the verification comment.

🤖 Generated with Claude Code

earakely-scale and others added 6 commits October 19, 2026 09:56
…laude-code agent

An agent folder built from its own Dockerfile: Anthropic's native Claude Code binary at a pinned version, checked
against its release manifest, behind an A2A server on agentenv-framework-protocol. The configured endpoint reaches the
CLI as Anthropic's variables, with [model.headers] on every call and the model's aliases pinned to the run's model. A
release pins every shipped example's protocol requirement.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ies, snapshots and environment

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…point on local docker

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A step that names no agent deploys @local/agentenv-framework/agents/claude-code, which a bundle run pulls from the
installed agents bundle. A store agent named a2a-default is no longer the default: set [agents] default_a2a_agent_id
to keep it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d assert on what a run surely prints

The CLI follows a tool result with system messages, and serves a tool's output as JSON; a rerun on one machine
reuses the agent image without building it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale marked this pull request as ready for review October 19, 2026 17:17
@earakely-scale
earakely-scale requested a review from a team as a code owner October 19, 2026 17:17
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread src/agent_env/examples/agents/agents/claude-code/a2a_server.py Outdated
Comment thread src/agent_env/examples/agents/agents/claude-code/a2a_server.py
Comment thread src/agent_env/examples/agents/agents/claude-code/a2a_server.py Outdated
Comment thread src/agent_env/examples/agents/agents/claude-code/claude_bridge.py Outdated
Comment thread src/agent_env/examples/agents/agents/claude-code/a2a_server.py
Comment thread src/agent_env/examples/agents/agents/claude-code/kit.py Outdated
earakely-scale and others added 3 commits October 19, 2026 10:53
…its tools

asyncio kills a process cancelled mid-start, but not what it already started, and the bridge had no process to
kill yet. The spawn is shielded, and a cancel waits for it, then kills the whole tree.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…start resumes nothing, and links survive

A load clears the CLI's state, and kept the transcripts of no other bound context: it now keeps them. A turn marks
its context active before writing its attachments, so a load can't clear them, and resumes its session only once the
CLI has written its transcript, so a turn that failed before then doesn't break the next. A timed-out turn keeps its
tool calls. A link to a directory is archived as a link, and a link the load's data filter would refuse is left out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… onto a sandbox's host

install_agent naming no agent now installs it. install.sh, in the agent's folder, brings Claude Code's native build for
the target's CPU and libc and uv, each checked against its published checksum, and the agent's requirements in a
Python of uv's own, so the target's Python is untouched. A target with neither bash nor zsh, which the CLI runs its
commands in, gets bash from its package manager. The model endpoint reaches the agent in a 0600 env file. The install
returns once the agent answers on its port, and fails with the end of its log if the agent exits first. The agent and
the CLI work in the container's working directory, and the CLI's transcripts follow it.

install_agent gains a trust_env param: on a sandbox on this machine, the local transfer CA goes into the container, or
beside the work dir on a host, and the variables that find it reach the install.

A snapshot load restores the live transcript of every other conversation the agent holds, not the archive's older copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread src/agent_env/examples/agents/agents/claude-code/install.sh Outdated
Comment thread src/agent_env/examples/agents/agents/claude-code/install.sh Outdated
…talls, and a name with a space installs

A container install lived at /opt/agentenv-agent whatever its port, so a second agent there rewrote the first's
environment and stopped it: each now lives under its port. The env file and the commands an install runs in the target
quote every path, so a host install whose agent name has a space starts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale

earakely-scale commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Verification

All on fresh x86 and arm64 Linux VMs, with a real model through a LiteLLM gateway unless noted. No run printed the model key, and nothing was left behind: no containers, agent processes, work dirs or Modal sandboxes.

Deploying the agent (deploy_agent, judges)

Provider smoke named agent env MCP tool agent judge
local, x86 and arm64 7–9 s pass 45–48 s pass
smol, x86 52 s (33 s warm) 35 s 106 s 70 s
modal 41 s cold, 10 s warm
modal_vm 41–49 s pass 192 s 109 s

Eight repeat runs passed.

Installing it (install_agent, def7431 and f025888)

Provider Debian container Alpine container host
local, x86 20 s 18 s 14 s
local, arm64 23 s 20 s 18 s
modal_vm 64–67 s 62–73 s 60–65 s
  • The agent used both Bash and Grep in the Alpine container, where it is musl and bash came from apk.
  • verify_a2a_install reports supported (rubric 1.0) on x86 and arm64. It was run as the validator's install sub-chain: the full a2a-agent validate needs a fixture env a local store doesn't hold.
  • CI's integration-local-slow ran all six Claude Code tests. Four are install tests: Debian, Alpine, two agents in one container, and the host under a name with a space. Each installs a skill through a TLS grant.

beta_scale (Scale's sandbox platform)

Smoke Agent judge Install, Debian container Install, host
110 s 226 s 132 s 121 s
  • An env's MCP tools don't reach the agent there. Inside the cluster, the env's MCP URL is served with a self-signed certificate that doesn't name its host, and the agent keeps TLS verification on.
  • The old wrapper switched verification off. Its beta_scale-only equivalent goes into the Scale provider when the hub moves onto this agent, so nothing changes in this PR.
  • Every other provider made real MCP tool calls.

Snapshot round trip (the hub's next-turn resume)

How the hub resumes a run: snapshot_agent_state after turn 1, then a fresh agent loads the snapshot under a target context, and turn 2 continues it.

  • Turn 2 recalled a codeword only the restored conversation held.
  • Its cat printed random bytes only the restored workspace held, matched against the snapshot's own workspace archive.
beta_scale, S3 modal_vm, S3
pass, 231 s pass, 150 s

A stored image as a file part

A PNG in the store (s3://) is named in a prompt. Core sends it as an https grant, and the agent downloads it for the CLI. The model then reported which colour was on which side.

beta_scale modal_vm
pass, 110 s pass, 69 s

Chaos

The trials fault the run, then check it ends as allowed, leaves nothing behind and leaks no key. A failed trial is rerun, and the rerun must pass.

  • Deploy path, 179 trials. Faults: killing the CLI or the container mid-turn, a Docker restart, a cancel, an env's MCP server stopped mid-call, Anthropic API faults (fake and through a proxy), a killed image build, blocked binary downloads, and four concurrent cold builds.
    • The only flagged trials were one pass's blocked-download trials: the build cache meant the download never ran, so the harness's expectation was wrong.
    • With the cache pruned, the cold-build passes ran clean: 61 trials, 0 violations.
  • Install path, 246 trials over five passes. Faults: install.sh killed mid-run (Debian, Alpine, host), the task container killed mid-install, Claude Code's download blocked, and Debian, Alpine and host installs at once.
    • 1 flagged trial was the harness's: the container was killed after the agent had answered, so the run rightly passed.
    • Every rerun passed.
    • A container killed mid-turn fails in about 60 s, from prompt_agent's liveness check.
    • Before the install waited for the agent's /health, a container killed just after the install took 300 s to fail, install_agent's card wait. Since then, the slowest is 71 s.
    • It recurred once in about 40 later kill-container trials, and not in 14 targeted ones on f025888.

🤖 Generated with Claude Code

earakely-scale and others added 5 commits October 19, 2026 14:19
…left in the container

A step retried from install_agent installs again into the same container and port, where the first attempt's agent
still runs. uv refused the existing venv, so the retry failed. Now the old agent stops first, freeing its port, files
are replaced by a rename so a binary it still runs can be swapped, the venv is recreated, and the CLI's state starts
afresh, so the retry's skills install again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n call the endpoint

The key was dropped from the CLI's environment once mapped to ANTHROPIC_AUTH_TOKEN, and LITELLM_EXTRA_HEADERS with it,
so a script the model ran could no longer reach the gateway as $LITELLM_API_KEY. The CLI's own tools can read the
bearer token anyway, so dropping them protected nothing. Both are kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…oto3

The scripts the model runs had no node, so a coding task that runs JavaScript broke, and Python without boto3. node
comes alone, without npm or npx, from a pinned node:22-slim; boto3 is pinned in model-requirements.txt, which the image
installs beside the agent's own requirements. The image grows from 192 MB to 254 MB and builds 5 to 6 s slower.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…model call

A host attributes an agent's spend by posting the run's project_id over agent-config before each prompting step. With
the real CLI against the fake endpoint, every model call that follows carries it as x-litellm-customer-id, and the
project and task as x-litellm-tags.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ugh a grant

core sends an object one of its stores owns as an https grant, and the agent downloads it into the task's inputs,
where the CLI reads it. That's now proven end to end with the real CLI, and the agent's own https download, a failed
one that keeps the URL's signature out of the error, and its refusal of any other scheme have unit tests. That
refusal now says how a stored object arrives.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant