Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,12 @@ print(list_artifacts())
report = json.loads(load_artifact(filename="report.json")["data"])
```

Pass `load_artifact_format="parsed"` to have `load_artifact` return the content itself instead: the parsed value for JSON, a `str` for other text, and `{"mime_type": str, "bytes": str}` (base64) for binary. Its docstring in the stubs and catalog follows the format, so the model is told which one it gets:

```python
report = load_artifact(filename="report.json") # {"status": "ready"}, already parsed
```

Pass `include_artifact_tools=False` to opt out. To react when the model saves an artifact, pass `on_artifacts_saved`:

```python
Expand All @@ -241,7 +247,7 @@ ExecuteCodeTool(tools=..., backend=..., on_artifacts_saved=on_saved)

### Tool results as artifacts

By default (`save_tool_results_as_artifacts=True`) every non-artifact tool is wrapped so its return value is saved as a session artifact tagged `code_mode.tool_result = "true"`. This lets a host forward tool outputs to the user (read the marker in `on_artifacts_saved`) and lets a later turn reload a result with `load_artifact`. Results over 50,000 bytes return a dictionary with a `tool_result_artifact` filename and a `note` instead of the inline result.
By default (`save_tool_results_as_artifacts=True`) every non-artifact tool is wrapped so its return value is saved as a session artifact tagged `code_mode.tool_result = "true"`. This lets a host forward tool outputs to the user (read the marker in `on_artifacts_saved`) and lets a later turn reload a result with `load_artifact`. Results over 50,000 bytes return a dictionary with a `tool_result_artifact` filename and a `note` instead of the inline result. Pass `large_result_threshold` to change the limit, or `None` so the code always gets the full result. It is still saved as an artifact, and the stubs, catalog and tool description stop mentioning the pointer.

Naming is transparent: the filename is derived from the tool name and call id. Each wrapped tool also gains two **optional** parameters — `artifact_name` and `artifact_description` — that the model may pass to name/describe the saved artifact; both land in the artifact's `custom_metadata` (`code_mode.artifact_name` / `code_mode.artifact_description`) alongside the marker. Set `save_tool_results_as_artifacts=False` to return tool results inline without persisting them.

Expand Down Expand Up @@ -272,6 +278,8 @@ All settings are `ExecuteCodeTool` constructor arguments:
| `max_code_mode_metadata_chars` | `50_000` | Budget for the whole rendered `<code-mode>` block, tags included. Oversized blocks degrade in tiers rather than disappearing — see [What the model sees](#what-the-model-sees). |
| `max_output_chars` | `50_000` | Caps stdout/stderr handed back to the model. Overflow is saved as a session artifact at `code_mode/stdout/<call-id>.txt` and the model sees a head-and-tail view pointing to it. |
| `max_code_chars` | `1_000_000` | Rejects oversized code payloads before starting a container. |
| `large_result_threshold` | `50_000` | With `save_tool_results_as_artifacts`, a tool result whose serialised form is over this many bytes is replaced with a `tool_result_artifact` pointer to its saved artifact. `None` always returns the full result. See [Tool results as artifacts](#tool-results-as-artifacts). |
| `load_artifact_format` | `"envelope"` | What `load_artifact` returns: `"envelope"` for `{"kind", "data", "mime_type"}`, or `"parsed"` for the content itself. See [Storage](#-storage). |
| `timeout_seconds` | `60` | Caps overall execution time of one `execute_code` call, and with it every tool the block calls. A block past the cap is stopped with its output so far, keeping the turn's variables and working directory. If it won't stop within a few seconds or leaves a worker thread or child process running, its sandbox is restarted and its output and state are lost. The model is told the cap. `None` lifts it, leaving the platform request timeout (e.g. Cloud Run `--timeout`) as the only bound. |
| `per_tool_timeout_seconds` | `None` | Caps each individual tool call made from within the sandbox. |
| `session_idle_timeout_seconds` | `600` | Idle reaper: closes a turn's container once it goes untouched this long. Backstop for turns that never call `release_invocation`. |
Expand Down Expand Up @@ -320,7 +328,7 @@ other failures raise `ToolError`. Import both from `tools`. Each call can run fo
to 60s.
~~~

With `append_code_mode_metadata_to_system_instruction` disabled that middle sentence is replaced by the discovery text the pointer tier uses (see [Fitting the budget](#fitting-the-budget)), so the description never points at a section that wasn't appended. With `save_tool_results_as_artifacts=False` the description drops "and every tool's result" and the sentence about large results, and with `include_artifact_tools=False` it drops the `load_artifact` clause. The last sentence follows `timeout_seconds` and is left out when it's `None`.
With `append_code_mode_metadata_to_system_instruction` disabled that middle sentence is replaced by the discovery text the pointer tier uses (see [Fitting the budget](#fitting-the-budget)), so the description never points at a section that wasn't appended. With `save_tool_results_as_artifacts=False` the description drops "and every tool's result" and the sentence about large results, with `large_result_threshold=None` it drops only the sentence about large results, and with `include_artifact_tools=False` it drops the `load_artifact` clause. The last sentence follows `timeout_seconds` and is left out when it's `None`.

The **`<code-mode>` block** is the per-deployment inventory of the sandbox, appended to the system instruction on every model turn:

Expand Down Expand Up @@ -365,7 +373,7 @@ from tools import save_artifact, load_artifact, list_artifacts
</code-mode>
~~~

Return types come from each tool's `response_json_schema`, or an MCP tool's `outputSchema`; a tool without one returns `Any`. Objects become `TypedDict` classes named after their `$defs` key or `title`, else the property holding them. Identical shapes share one class across a module, defined once at the top of its catalog section; a name clash with another module's class falls back to the tool's or parent's name; each stub file carries the classes its tool uses. An MCP tool's `outputSchema` describes the `structuredContent` field in its result. A saved tool result can also be a `ToolResultArtifact` pointer when it exceeds the size threshold. A tool with exactly one required parameter takes it positionally (`load_artifact("report.csv", version=2)`); every other parameter is keyword-only.
Return types come from each tool's `response_json_schema`, or an MCP tool's `outputSchema`; a tool without one returns `Any`. Objects become `TypedDict` classes named after their `$defs` key or `title`, else the property holding them. Identical shapes share one class across a module, defined once at the top of its catalog section; a name clash with another module's class falls back to the tool's or parent's name; each stub file carries the classes its tool uses. An MCP tool's `outputSchema` describes the `structuredContent` field in its result. A saved tool result can also be a `ToolResultArtifact` pointer when it exceeds the size threshold, unless `large_result_threshold=None`. A tool with exactly one required parameter takes it positionally (`load_artifact("report.csv", version=2)`); every other parameter is keyword-only.

Tags are indented; their content is not. The `<tools-package>` body is Python source, where leading whitespace is meaningful. Each installed-package line is `<import name>: <distribution> <version>`; namespace imports supplied by multiple distributions list every provider in deterministic order. Import names and providers are sorted so the block stays byte-stable for prompt caching. `<python-version>` and `<installed-packages>` are omitted entirely — rather than rendered blank — when the sandbox hasn't reported yet or the image has no extra packages, since an empty tag reads as "none exist".

Expand Down Expand Up @@ -406,7 +414,7 @@ print(open("/tools/slack/send_message.py").read()) # types, signature + docstri
print(open("/tools/slack/list_channels.py").read())
```

Text and JSON-like MIME types travel as plain strings in artifact tools; binary content is base64-encoded. `load_artifact` returns a dictionary with `kind`, `data`, and `mime_type`. Text and JSON use `kind="text"` and keep the string in `data`. Binary artifacts use `kind="bytes"` and store base64 in `data`.
Text and JSON-like MIME types travel as plain strings in artifact tools; binary content is base64-encoded. `load_artifact` returns a dictionary with `kind`, `data`, and `mime_type`. Text and JSON use `kind="text"` and keep the string in `data`. Binary artifacts use `kind="bytes"` and store base64 in `data`. With `load_artifact_format="parsed"` it returns the parsed value for JSON, a `str` for other text, and `{"mime_type": str, "bytes": str}` (base64) for binary.

## 🛡️ Safety

Expand Down
4 changes: 4 additions & 0 deletions src/adk_code_mode/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
#
# SPDX-License-Identifier: Apache-2.0
from adk_code_mode.__about__ import __version__
from adk_code_mode._artifact_tools import LoadArtifactFormat
from adk_code_mode.runtime import (
RemoteBackend,
SandboxBackend,
Expand All @@ -10,6 +11,7 @@
)
from adk_code_mode.tool import ArtifactsSavedCallback, ExecuteCodeTool
from adk_code_mode.tool_result_artifacts import (
DEFAULT_LARGE_RESULT_THRESHOLD,
TOOL_RESULT_DESCRIPTION_KEY,
TOOL_RESULT_FILENAME_KEY,
TOOL_RESULT_METADATA_KEY,
Expand All @@ -20,7 +22,9 @@

__all__ = [
"ArtifactsSavedCallback",
"DEFAULT_LARGE_RESULT_THRESHOLD",
"ExecuteCodeTool",
"LoadArtifactFormat",
"RemoteBackend",
"SandboxBackend",
"SandboxSession",
Expand Down
120 changes: 110 additions & 10 deletions src/adk_code_mode/_artifact_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,18 @@

The wire format is JSON. Binary content is base64 over the wire — the model
encodes it before calling ``save_artifact`` and decodes it after
``load_artifact``. Loaded artifacts return a flat ``{"kind", "data",
"mime_type"}`` dict so the model dispatches on ``kind`` without importing
host-side types.
``load_artifact``. By default loaded artifacts return a flat ``{"kind",
"data", "mime_type"}`` dict so the model dispatches on ``kind`` without
importing host-side types. ``ExecuteCodeTool(load_artifact_format="parsed")``
swaps in a ``load_artifact`` that returns the content itself: parsed JSON, a
``str`` for other text, or ``{"mime_type", "bytes"}`` for binary.
"""

from __future__ import annotations

import base64
from typing import Any
import json
from typing import Any, Literal

from google.adk.tools.function_tool import FunctionTool
from google.adk.tools.tool_context import ToolContext
Expand All @@ -27,17 +30,43 @@
_TEXT_MIME_PREFIXES = ("text/",)
_TEXT_MIME_TYPES = {"application/json", "application/xml", "application/javascript"}

LoadArtifactFormat = Literal["envelope", "parsed"]
"""What ``load_artifact`` returns.

``"envelope"`` (the default) is ``{"kind", "data", "mime_type"}``. ``"parsed"``
is the content itself: parsed JSON, a ``str`` for other text, or
``{"mime_type", "bytes"}`` for binary.
"""


def _is_text_mime(mime_type: str | None) -> bool:
if not mime_type:
return False
mime_type = _base_mime(mime_type)
if mime_type.startswith(_TEXT_MIME_PREFIXES):
return True
if mime_type in _TEXT_MIME_TYPES:
return True
return mime_type.endswith("+json") or mime_type.endswith("+xml")


def _is_json_mime(mime_type: str | None) -> bool:
base = _base_mime(mime_type or "")
return base == "application/json" or base.endswith("+json")


def _base_mime(mime_type: str) -> str:
return mime_type.split(";", 1)[0].strip().lower()


def _charset(mime_type: str | None) -> str:
for param in (mime_type or "").split(";")[1:]:
key, _, value = param.partition("=")
if key.strip().lower() == "charset":
return value.strip().strip('"')
return "utf-8"


async def save_artifact(
*,
filename: str,
Expand Down Expand Up @@ -69,7 +98,7 @@ async def save_artifact(
if _is_text_mime(mime_type):
part = genai_types.Part(
inline_data=genai_types.Blob(
data=content.encode("utf-8"),
data=content.encode(_charset(mime_type)),
mime_type=mime_type or "text/plain",
)
)
Expand Down Expand Up @@ -115,13 +144,11 @@ async def load_artifact(
inline = part.inline_data
if inline is not None:
mime_type = inline.mime_type
data = inline.data or b""
if isinstance(data, str):
data = data.encode("utf-8")
data = _inline_bytes(inline)
if _is_text_mime(mime_type):
return {
"kind": "text",
"data": data.decode("utf-8"),
"data": data.decode(_charset(mime_type)),
"mime_type": mime_type,
}
return {
Expand All @@ -134,6 +161,56 @@ async def load_artifact(
return {"kind": "text", "data": "", "mime_type": None}


async def load_artifact_parsed(
*,
filename: str,
version: int | None = None,
tool_context: ToolContext,
) -> Any:
"""Load an artifact by filename and return its content. Returns ``None`` if not found.

The return value depends on the artifact's mime type:

- JSON (``application/json``, ``*+json``): the parsed value — for a saved
tool result, exactly what the original call returned.
- Other text (``text/*``, XML, JavaScript): the text as a ``str``.
- Binary: ``{"mime_type": <str>, "bytes": <base64 str>}``.

Example:

rows = load_artifact(filename="search_call-1.json") # list/dict, already parsed
notes = load_artifact(filename="notes.txt") # str
image = load_artifact(filename="chart.png")
png = base64.b64decode(image["bytes"])
"""
part = await tool_context.load_artifact(filename, version=version)
if part is None:
return None
inline = part.inline_data
if inline is None:
return part.text or ""
mime_type = inline.mime_type
data = _inline_bytes(inline)
if not _is_text_mime(mime_type):
return {"mime_type": mime_type, "bytes": base64.b64encode(data).decode("ascii")}
text = data.decode(_charset(mime_type))
if _is_json_mime(mime_type):
try:
return json.loads(text)
except ValueError:
return text
return text


# Code imports it as ``load_artifact`` whichever format the host picked.
load_artifact_parsed.__name__ = load_artifact_parsed.__qualname__ = "load_artifact"


def _inline_bytes(inline: genai_types.Blob) -> bytes:
data = inline.data or b""
return data.encode("utf-8") if isinstance(data, str) else data


async def list_artifacts(*, tool_context: ToolContext) -> list[str]:
"""Return the filenames of artifacts visible to this session.

Expand All @@ -151,5 +228,28 @@ async def list_artifacts(*, tool_context: ToolContext) -> list[str]:
FunctionTool(list_artifacts),
)

_PARSED_ARTIFACT_TOOLS: tuple[FunctionTool, ...] = (
ARTIFACT_TOOLS[0],
FunctionTool(load_artifact_parsed),
ARTIFACT_TOOLS[2],
)


__all__ = ["ARTIFACT_TOOLS", "list_artifacts", "load_artifact", "save_artifact"]
def artifact_tools(load_format: LoadArtifactFormat = "envelope") -> tuple[FunctionTool, ...]:
"""Return the artifact tools, with ``load_artifact`` returning ``load_format``."""
if load_format == "envelope":
return ARTIFACT_TOOLS
if load_format == "parsed":
return _PARSED_ARTIFACT_TOOLS
raise ValueError(f"load_artifact_format must be 'envelope' or 'parsed', not {load_format!r}")


__all__ = [
"ARTIFACT_TOOLS",
"LoadArtifactFormat",
"artifact_tools",
"list_artifacts",
"load_artifact",
"load_artifact_parsed",
"save_artifact",
]
2 changes: 1 addition & 1 deletion src/adk_code_mode/_json.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# SPDX-FileCopyrightText: 2025-present Agenta <hello@buildagenta.com>
#
# SPDX-License-Identifier: Apache-2.0
"""JSON-safe conversion for the sandbox RPC reply."""
"""JSON-safe conversion shared by the RPC reply and saved tool results."""

from __future__ import annotations

Expand Down
10 changes: 10 additions & 0 deletions src/adk_code_mode/runtime/remote.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ async def _connect(
return _RemoteSandboxSession(
ws=ws,
workdir_path=workdir_path,
max_message_size=self.max_message_size,
max_upload_workspace_bytes=self.max_upload_workspace_bytes,
max_download_workspace_bytes=self.max_download_workspace_bytes,
)
Expand All @@ -148,11 +149,13 @@ def __init__(
*,
ws: Any,
workdir_path: str,
max_message_size: int,
max_upload_workspace_bytes: int,
max_download_workspace_bytes: int,
) -> None:
self._ws = ws
self._workdir_path = workdir_path
self._max_message_size = max_message_size
self._max_upload_workspace_bytes = max_upload_workspace_bytes
self._max_download_workspace_bytes = max_download_workspace_bytes

Expand All @@ -172,6 +175,13 @@ async def begin_block(self, input_paths: Sequence[str]) -> None:

async def send(self, frame: Frame) -> None:
payload = encode(frame)
# The sandbox closes the connection on an oversized message, so refuse it
# here instead; a tool reply then reaches the code as an error.
if len(payload) > self._max_message_size:
raise ValueError(
f"{type(frame).__name__} ({len(payload):,} bytes) exceeds "
f"max_message_size ({self._max_message_size:,})"
)
try:
await self._ws.send(payload.decode("utf-8"))
except websockets.ConnectionClosed as exc:
Expand Down
Loading
Loading