Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 36 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Inspired by Cloudflare's [Code Mode](https://blog.cloudflare.com/code-mode/) and
- **Call ADK tools from sandbox code** — imports against the `tools` package proxy back to the host and run through ADK's `before_tool` / `after_tool` / `on_error` callbacks and the plugin manager exactly as direct tool calls would.
- **Failed tool calls raise** — a REST tool whose API answers with an HTTP error raises `HTTPStatusError` (with `status_code` and `body`), and any other failure, including an MCP tool that reports one, raises `ToolError`. ADK returns these as values for a model to read, but sandbox code reads return values, so it would otherwise carry on as if the call had worked.
- **Bake any Python package into the image** — extend the published base image with anything the model's code needs to `import`, no runtime `pip install` required. The sandbox reports what it has installed, so the model is told about it automatically.
- **Cross-turn persistence via ADK Artifacts** — `save_artifact` / `load_artifact` / `list_artifacts` are auto-injected and route through your configured `ArtifactService`. Files the code creates or changes are saved as artifacts automatically too.
- **Cross-turn persistence via ADK Artifacts** — files the code creates or changes are saved as artifacts automatically, and `save_artifact` / `load_artifact` / `list_artifacts` are auto-injected and route through your configured `ArtifactService`.
- **Tool results saved as artifacts** — on by default; every tool's result is persisted as a `code_mode.tool_result` artifact (with optional model-supplied name/description) so hosts can forward outputs and large results stay out of the prompt. Opt out with `save_tool_results_as_artifacts=False`.
- **Bounded stdout/stderr** — overflow lands in a session artifact instead of poisoning the prompt.
- **Production-ready remote sandbox** — `RemoteBackend` connects to an isolated per-turn container over WebSocket, reused across the turn's `execute_code` calls. Deploy on any cloud platform (Cloud Run, Fargate, ACI, Kubernetes, Fly.io, etc.).
Expand Down Expand Up @@ -225,7 +225,7 @@ save_artifact(
mime_type="application/json",
)
print(list_artifacts())
report = load_artifact(filename="report.json") # {"status": "ready"}, already parsed
report = json.loads(load_artifact(filename="report.json")["data"])
```

Pass `include_artifact_tools=False` to opt out. To react when the model saves an artifact, pass `on_artifacts_saved`:
Expand All @@ -241,7 +241,7 @@ ExecuteCodeTool(tools=..., backend=..., on_artifacts_saved=on_saved)

### Tool results as artifacts

By default (`save_tool_results_as_artifacts=True`) every non-artifact tool is wrapped so its return value is saved as a session artifact tagged `code_mode.tool_result = "true"`. This lets a host forward tool outputs to the user (read the marker in `on_artifacts_saved`) and lets a later turn reload a result with `load_artifact`. The code always gets the full result.
By default (`save_tool_results_as_artifacts=True`) every non-artifact tool is wrapped so its return value is saved as a session artifact tagged `code_mode.tool_result = "true"`. This lets a host forward tool outputs to the user (read the marker in `on_artifacts_saved`) and lets a later turn reload a result with `load_artifact`. Results over 50,000 bytes return a dictionary with a `tool_result_artifact` filename and a `note` instead of the inline result.

Naming is transparent: the filename is derived from the tool name and call id. Each wrapped tool also gains two **optional** parameters — `artifact_name` and `artifact_description` — that the model may pass to name/describe the saved artifact; both land in the artifact's `custom_metadata` (`code_mode.artifact_name` / `code_mode.artifact_description`) alongside the marker. Set `save_tool_results_as_artifacts=False` to return tool results inline without persisting them.

Expand Down Expand Up @@ -281,7 +281,7 @@ The model can read spilled stdout back from the overflow artifact:
```python
from tools import load_artifact
spilled = load_artifact(filename="code_mode/stdout/<call-id>.txt")
print(spilled[-2000:])
print(spilled["data"][-2000:])
```

### Turn-scoped sessions
Expand Down Expand Up @@ -309,14 +309,18 @@ Two surfaces, deliberately kept apart. The **tool description** is the invariant
Execute Python code in a sandboxed container. Custom tools are Python functions to
import from the `tools` package (e.g. `from tools.slack import send_message`). See
the `<code-mode>` section of the system instruction for the Python version,
preinstalled packages, and available functions. Print anything you want to see; only
stdout/stderr are returned. Variables and the working directory persist across calls
within the same turn and reset on the next turn — use `save_artifact` /
`load_artifact` (imported the same way) to persist across turns. Files created or
changed by the code are saved as artifacts automatically
preinstalled packages, and available functions. Print function calls, variables, etc.
to see output, only stdout/stderr are returned. Variables and files persist across
calls within the same turn and reset on the next turn. Every file the code writes in
the working directory, and every tool's result, is saved as an artifact; use
`load_artifact` (imported the same way) to read one in a later turn. Large tool
results return a `tool_result_artifact` pointer. Tool calls that
call APIs raise HTTP errors as `HTTPStatusError` (with `status_code` and `body`);
other failures raise `ToolError`. Import both from `tools`. Each call can run for up
to 60s.
~~~

With `append_code_mode_metadata_to_system_instruction` disabled that middle sentence is replaced by `List /tools/ and read a file's docstring to see what's available.`, so the description never points at a section that wasn't appended.
With `append_code_mode_metadata_to_system_instruction` disabled that middle sentence is replaced by the discovery text the pointer tier uses (see [Fitting the budget](#fitting-the-budget)), so the description never points at a section that wasn't appended. With `save_tool_results_as_artifacts=False` the description drops "and every tool's result" and the sentence about large results, and with `include_artifact_tools=False` it drops the `load_artifact` clause. The last sentence follows `timeout_seconds` and is left out when it's `None`.

The **`<code-mode>` block** is the per-deployment inventory of the sandbox, appended to the system instruction on every model turn:

Expand All @@ -337,7 +341,15 @@ yaml: PyYAML 6.0.2

from tools.slack import list_channels, send_message

def list_channels() -> Any:
class ListChannelsResponse(TypedDict):
channels: list[ChannelsItem]
next_cursor: NotRequired[str]

class ChannelsItem(TypedDict):
id: str
name: str

def list_channels() -> ListChannelsResponse:
"""List Slack channels."""
...

Expand All @@ -353,6 +365,8 @@ from tools import save_artifact, load_artifact, list_artifacts
</code-mode>
~~~

Return types come from each tool's `response_json_schema`, or an MCP tool's `outputSchema`; a tool without one returns `Any`. Objects become `TypedDict` classes named after their `$defs` key or `title`, else the property holding them. Identical shapes share one class across a module, defined once at the top of its catalog section; a name clash with another module's class falls back to the tool's or parent's name; each stub file carries the classes its tool uses. An MCP tool's `outputSchema` describes the `structuredContent` field in its result. A saved tool result can also be a `ToolResultArtifact` pointer when it exceeds the size threshold. A tool with exactly one required parameter takes it positionally (`load_artifact("report.csv", version=2)`); every other parameter is keyword-only.

Tags are indented; their content is not. The `<tools-package>` body is Python source, where leading whitespace is meaningful. Each installed-package line is `<import name>: <distribution> <version>`; namespace imports supplied by multiple distributions list every provider in deterministic order. Import names and providers are sorted so the block stays byte-stable for prompt caching. `<python-version>` and `<installed-packages>` are omitted entirely — rather than rendered blank — when the sandbox hasn't reported yet or the image has no extra packages, since an empty tag reads as "none exist".

With `save_tool_results_as_artifacts` enabled (the default), each non-artifact tool above — e.g. `list_channels` and `send_message` — also carries two optional `artifact_name: str | None = ...` / `artifact_description: str | None = ...` parameters for naming its saved result.
Expand All @@ -362,7 +376,10 @@ With `save_tool_results_as_artifacts` enabled (the default), each non-artifact t
A block over `max_code_mode_metadata_chars` degrades in tiers instead of vanishing. The Python version and package list survive every tier — they're small, and the model cannot discover them any other way.

1. **Full** — as above.
2. **Names only** — signatures and docstrings dropped, import lines kept:
2. **Names only** — types, signatures and docstrings dropped, import lines kept. `<how-to-use>` gains:

> Before calling a function you haven't used, print its stub to see its signature, docstring and the fields of its return type: `/tools/<module>/<fn>.py`, or `/tools/<fn>.py` for a function imported from `tools`. Print every stub you need in one `execute_code` call.


~~~
<tools-package>
Expand All @@ -372,23 +389,24 @@ A block over `max_code_mode_metadata_chars` degrades in tiers instead of vanishi
</tools-package>
~~~

3. **Pointer** — the catalog is replaced by a discovery note, and this tier is emitted no matter how small the budget:
3. **Pointer** — the catalog is replaced by a note, and this tier is emitted no matter how small the budget. `<how-to-use>` gains the same text, prefixed with "List `/tools/` and its module directories to find the functions available.":

~~~
<tools-package>
The full catalog is too large to include here. List `/tools/` and read a file's docstring to see what's available.
The full catalog is too large to include here.
</tools-package>
~~~

At the last tier the model navigates the sandbox from Python instead:

```python
import pathlib
print(list(pathlib.Path("/tools").iterdir()))
print(open("/tools/slack/send_message.py").read()) # signature + docstring
print(sorted(pathlib.Path("/tools").rglob("*.py")))
print(open("/tools/slack/send_message.py").read()) # types, signature + docstring
print(open("/tools/slack/list_channels.py").read())
```

Text and JSON-like MIME types travel as plain strings in artifact tools; binary content is base64-encoded. `load_artifact` returns the content itself: the parsed value for JSON, a `str` for other text, and `{"mime_type": str, "bytes": str}` (base64) for binary.
Text and JSON-like MIME types travel as plain strings in artifact tools; binary content is base64-encoded. `load_artifact` returns a dictionary with `kind`, `data`, and `mime_type`. Text and JSON use `kind="text"` and keep the string in `data`. Binary artifacts use `kind="bytes"` and store base64 in `data`.

## 🛡️ Safety

Expand Down Expand Up @@ -421,7 +439,7 @@ Named "Unsafe" intentionally: it binds a TCP listener on `0.0.0.0`, communicates
## ⚠️ Limitations

- **No credential-requesting tools.** Tools that need ADK to request credentials, confirmations, UI widgets, agent transfer, escalation, or that yield without an immediate response are rejected with a structured error.
- **State is turn-scoped.** Variables and the working directory persist across `execute_code` calls **within** a turn, but reset between turns. Use `save_artifact` / `load_artifact` to persist across turns.
- **State is turn-scoped.** Variables and the working directory persist across `execute_code` calls **within** a turn, but reset between turns. Files the code writes and tool results are saved as artifacts; use `load_artifact` to read them in a later turn.
- **No runtime package installation.** The sandbox ships with the Python Standard Library and the runtime's own dependencies only. Extra packages must be baked into the image at build time.

## 🛠️ Development
Expand Down
88 changes: 38 additions & 50 deletions src/adk_code_mode/_artifact_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@

The wire format is JSON. Binary content is base64 over the wire — the model
encodes it before calling ``save_artifact`` and decodes it after
``load_artifact``. ``load_artifact`` returns the content itself: parsed JSON,
a ``str`` for other text, or ``{"mime_type", "bytes"}`` for binary.
``load_artifact``. Loaded artifacts return a flat ``{"kind", "data",
"mime_type"}`` dict so the model dispatches on ``kind`` without importing
host-side types.
"""

from __future__ import annotations

import base64
import json
from typing import Any

from google.adk.tools.function_tool import FunctionTool
Expand All @@ -31,31 +31,13 @@
def _is_text_mime(mime_type: str | None) -> bool:
if not mime_type:
return False
mime_type = _base_mime(mime_type)
if mime_type.startswith(_TEXT_MIME_PREFIXES):
return True
if mime_type in _TEXT_MIME_TYPES:
return True
return mime_type.endswith("+json") or mime_type.endswith("+xml")


def _is_json_mime(mime_type: str | None) -> bool:
base = _base_mime(mime_type or "")
return base == "application/json" or base.endswith("+json")


def _base_mime(mime_type: str) -> str:
return mime_type.split(";", 1)[0].strip().lower()


def _charset(mime_type: str | None) -> str:
for param in (mime_type or "").split(";")[1:]:
key, _, value = param.partition("=")
if key.strip().lower() == "charset":
return value.strip().strip('"')
return "utf-8"


async def save_artifact(
*,
filename: str,
Expand Down Expand Up @@ -83,13 +65,11 @@ async def save_artifact(
)

``custom_metadata`` is custom metadata to associate with the artifact.
``load_artifact`` gives back the parsed JSON, the text, or
``{"mime_type", "bytes"}`` with the same base64 string.
"""
if _is_text_mime(mime_type):
part = genai_types.Part(
inline_data=genai_types.Blob(
data=content.encode(_charset(mime_type)),
data=content.encode("utf-8"),
mime_type=mime_type or "text/plain",
)
)
Expand All @@ -108,42 +88,50 @@ async def load_artifact(
filename: str,
version: int | None = None,
tool_context: ToolContext,
) -> Any:
"""Load an artifact by filename and return its content. Returns ``None`` if not found.
) -> dict[str, Any] | None:
"""Load an artifact by filename. Returns ``None`` if not found.

The return value depends on the artifact's mime type:
Returned dict shape:

- JSON (``application/json``, ``*+json``): the parsed value — for a saved
tool result, exactly what the original call returned.
- Other text (``text/*``, XML, JavaScript): the text as a ``str``.
- Binary: ``{"mime_type": <str>, "bytes": <base64 str>}``.
{
"kind": "text" | "bytes",
"data": <str>, # text: the string; bytes: base64-encoded
"mime_type": <str | None>,
}

Example:

rows = load_artifact(filename="search_call-1.json") # list/dict, already parsed
notes = load_artifact(filename="notes.txt") # str
image = load_artifact(filename="chart.png")
png = base64.b64decode(image["bytes"])
result = load_artifact(filename="report.json")
if result is None:
return
if result["kind"] == "text" and result["mime_type"] == "application/json":
payload = json.loads(result["data"])
elif result["kind"] == "bytes":
blob = base64.b64decode(result["data"])
"""
part = await tool_context.load_artifact(filename, version=version)
if part is None:
return None
inline = part.inline_data
if inline is None:
return part.text or ""
mime_type = inline.mime_type
data = inline.data or b""
if isinstance(data, str):
data = data.encode("utf-8")
if not _is_text_mime(mime_type):
return {"mime_type": mime_type, "bytes": base64.b64encode(data).decode("ascii")}
text = data.decode(_charset(mime_type))
if _is_json_mime(mime_type):
try:
return json.loads(text)
except ValueError:
return text
return text
if inline is not None:
mime_type = inline.mime_type
data = inline.data or b""
if isinstance(data, str):
data = data.encode("utf-8")
if _is_text_mime(mime_type):
return {
"kind": "text",
"data": data.decode("utf-8"),
"mime_type": mime_type,
}
return {
"kind": "bytes",
"data": base64.b64encode(data).decode("ascii"),
"mime_type": mime_type,
}
if part.text is not None:
return {"kind": "text", "data": part.text, "mime_type": None}
return {"kind": "text", "data": "", "mime_type": None}


async def list_artifacts(*, tool_context: ToolContext) -> list[str]:
Expand Down
2 changes: 1 addition & 1 deletion src/adk_code_mode/_json.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# SPDX-FileCopyrightText: 2025-present Agenta <hello@buildagenta.com>
#
# SPDX-License-Identifier: Apache-2.0
"""JSON-safe conversion shared by the RPC reply and saved tool results."""
"""JSON-safe conversion for the sandbox RPC reply."""

from __future__ import annotations

Expand Down
38 changes: 16 additions & 22 deletions src/adk_code_mode/mcp_results.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,7 @@
# SPDX-FileCopyrightText: 2025-present Agenta <hello@buildagenta.com>
#
# SPDX-License-Identifier: Apache-2.0
"""Return an MCP tool's ``structuredContent`` to code instead of the whole result.

ADK's ``McpTool`` returns the full ``CallToolResult`` (``content``,
``structuredContent``, ``isError``). When the tool declares an ``outputSchema``,
the MCP spec requires ``structuredContent`` to match it, and ``content`` is meant to
be a serialised copy for older clients. So code gets ``structuredContent`` alone,
and the stub's return type is built from the ``outputSchema``.
"""
"""Describe the MCP result envelope returned by ADK."""

from __future__ import annotations

Expand All @@ -18,19 +11,20 @@
from google.adk.tools.mcp_tool.mcp_tool import McpTool


def output_schema(tool: BaseTool) -> dict[str, Any] | None:
"""Return the ``outputSchema`` of an MCP tool, or ``None`` for any other tool.

``tool`` is the tool itself, not a ``ToolResultArtifactTool`` wrapping it.
"""
return tool.raw_mcp_tool.outputSchema if isinstance(tool, McpTool) else None


def structured_result(tool: BaseTool, result: Any) -> Any:
"""Return ``structuredContent`` when ``tool`` is an MCP tool with an ``outputSchema``."""
if output_schema(tool) and isinstance(result, dict) and "structuredContent" in result:
return result["structuredContent"]
return result
def response_schema(tool: BaseTool) -> dict[str, Any] | None:
"""Describe an MCP result and type its structured content from the tool's output schema."""
if not isinstance(tool, McpTool) or not (output := tool.raw_mcp_tool.outputSchema):
return None
return {
**{key: output[key] for key in ("$defs", "definitions") if key in output},
"type": "object",
"properties": {
"content": {"type": "array", "items": {}},
"isError": {"type": "boolean"},
"structuredContent": {"anyOf": [output, {"type": "null"}]},
},
"required": ["content", "isError"],
}


__all__ = ["output_schema", "structured_result"]
__all__ = ["response_schema"]
Loading
Loading