diff --git a/README.md b/README.md index bbc096f..087cbba 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ Inspired by Cloudflare's [Code Mode](https://blog.cloudflare.com/code-mode/) and - **Call ADK tools from sandbox code** — imports against the `tools` package proxy back to the host and run through ADK's `before_tool` / `after_tool` / `on_error` callbacks and the plugin manager exactly as direct tool calls would. - **Failed tool calls raise** — a REST tool whose API answers with an HTTP error raises `HTTPStatusError` (with `status_code` and `body`), and any other failure, including an MCP tool that reports one, raises `ToolError`. ADK returns these as values for a model to read, but sandbox code reads return values, so it would otherwise carry on as if the call had worked. - **Bake any Python package into the image** — extend the published base image with anything the model's code needs to `import`, no runtime `pip install` required. The sandbox reports what it has installed, so the model is told about it automatically. -- **Cross-turn persistence via ADK Artifacts** — `save_artifact` / `load_artifact` / `list_artifacts` are auto-injected and route through your configured `ArtifactService`. Files the code creates or changes are saved as artifacts automatically too. +- **Cross-turn persistence via ADK Artifacts** — files the code creates or changes are saved as artifacts automatically, and `save_artifact` / `load_artifact` / `list_artifacts` are auto-injected and route through your configured `ArtifactService`. - **Tool results saved as artifacts** — on by default; every tool's result is persisted as a `code_mode.tool_result` artifact (with optional model-supplied name/description) so hosts can forward outputs and large results stay out of the prompt. Opt out with `save_tool_results_as_artifacts=False`. - **Bounded stdout/stderr** — overflow lands in a session artifact instead of poisoning the prompt. - **Production-ready remote sandbox** — `RemoteBackend` connects to an isolated per-turn container over WebSocket, reused across the turn's `execute_code` calls. Deploy on any cloud platform (Cloud Run, Fargate, ACI, Kubernetes, Fly.io, etc.). @@ -225,7 +225,7 @@ save_artifact( mime_type="application/json", ) print(list_artifacts()) -report = load_artifact(filename="report.json") # {"status": "ready"}, already parsed +report = json.loads(load_artifact(filename="report.json")["data"]) ``` Pass `include_artifact_tools=False` to opt out. To react when the model saves an artifact, pass `on_artifacts_saved`: @@ -241,7 +241,7 @@ ExecuteCodeTool(tools=..., backend=..., on_artifacts_saved=on_saved) ### Tool results as artifacts -By default (`save_tool_results_as_artifacts=True`) every non-artifact tool is wrapped so its return value is saved as a session artifact tagged `code_mode.tool_result = "true"`. This lets a host forward tool outputs to the user (read the marker in `on_artifacts_saved`) and lets a later turn reload a result with `load_artifact`. The code always gets the full result. +By default (`save_tool_results_as_artifacts=True`) every non-artifact tool is wrapped so its return value is saved as a session artifact tagged `code_mode.tool_result = "true"`. This lets a host forward tool outputs to the user (read the marker in `on_artifacts_saved`) and lets a later turn reload a result with `load_artifact`. Results over 50,000 bytes return a dictionary with a `tool_result_artifact` filename and a `note` instead of the inline result. Naming is transparent: the filename is derived from the tool name and call id. Each wrapped tool also gains two **optional** parameters — `artifact_name` and `artifact_description` — that the model may pass to name/describe the saved artifact; both land in the artifact's `custom_metadata` (`code_mode.artifact_name` / `code_mode.artifact_description`) alongside the marker. Set `save_tool_results_as_artifacts=False` to return tool results inline without persisting them. @@ -281,7 +281,7 @@ The model can read spilled stdout back from the overflow artifact: ```python from tools import load_artifact spilled = load_artifact(filename="code_mode/stdout/.txt") -print(spilled[-2000:]) +print(spilled["data"][-2000:]) ``` ### Turn-scoped sessions @@ -309,14 +309,18 @@ Two surfaces, deliberately kept apart. The **tool description** is the invariant Execute Python code in a sandboxed container. Custom tools are Python functions to import from the `tools` package (e.g. `from tools.slack import send_message`). See the `` section of the system instruction for the Python version, -preinstalled packages, and available functions. Print anything you want to see; only -stdout/stderr are returned. Variables and the working directory persist across calls -within the same turn and reset on the next turn — use `save_artifact` / -`load_artifact` (imported the same way) to persist across turns. Files created or -changed by the code are saved as artifacts automatically +preinstalled packages, and available functions. Print function calls, variables, etc. +to see output, only stdout/stderr are returned. Variables and files persist across +calls within the same turn and reset on the next turn. Every file the code writes in +the working directory, and every tool's result, is saved as an artifact; use +`load_artifact` (imported the same way) to read one in a later turn. Large tool +results return a `tool_result_artifact` pointer. Tool calls that +call APIs raise HTTP errors as `HTTPStatusError` (with `status_code` and `body`); +other failures raise `ToolError`. Import both from `tools`. Each call can run for up +to 60s. ~~~ -With `append_code_mode_metadata_to_system_instruction` disabled that middle sentence is replaced by `List /tools/ and read a file's docstring to see what's available.`, so the description never points at a section that wasn't appended. +With `append_code_mode_metadata_to_system_instruction` disabled that middle sentence is replaced by the discovery text the pointer tier uses (see [Fitting the budget](#fitting-the-budget)), so the description never points at a section that wasn't appended. With `save_tool_results_as_artifacts=False` the description drops "and every tool's result" and the sentence about large results, and with `include_artifact_tools=False` it drops the `load_artifact` clause. The last sentence follows `timeout_seconds` and is left out when it's `None`. The **`` block** is the per-deployment inventory of the sandbox, appended to the system instruction on every model turn: @@ -337,7 +341,15 @@ yaml: PyYAML 6.0.2 from tools.slack import list_channels, send_message -def list_channels() -> Any: +class ListChannelsResponse(TypedDict): + channels: list[ChannelsItem] + next_cursor: NotRequired[str] + +class ChannelsItem(TypedDict): + id: str + name: str + +def list_channels() -> ListChannelsResponse: """List Slack channels.""" ... @@ -353,6 +365,8 @@ from tools import save_artifact, load_artifact, list_artifacts ~~~ +Return types come from each tool's `response_json_schema`, or an MCP tool's `outputSchema`; a tool without one returns `Any`. Objects become `TypedDict` classes named after their `$defs` key or `title`, else the property holding them. Identical shapes share one class across a module, defined once at the top of its catalog section; a name clash with another module's class falls back to the tool's or parent's name; each stub file carries the classes its tool uses. An MCP tool's `outputSchema` describes the `structuredContent` field in its result. A saved tool result can also be a `ToolResultArtifact` pointer when it exceeds the size threshold. A tool with exactly one required parameter takes it positionally (`load_artifact("report.csv", version=2)`); every other parameter is keyword-only. + Tags are indented; their content is not. The `` body is Python source, where leading whitespace is meaningful. Each installed-package line is `: `; namespace imports supplied by multiple distributions list every provider in deterministic order. Import names and providers are sorted so the block stays byte-stable for prompt caching. `` and `` are omitted entirely — rather than rendered blank — when the sandbox hasn't reported yet or the image has no extra packages, since an empty tag reads as "none exist". With `save_tool_results_as_artifacts` enabled (the default), each non-artifact tool above — e.g. `list_channels` and `send_message` — also carries two optional `artifact_name: str | None = ...` / `artifact_description: str | None = ...` parameters for naming its saved result. @@ -362,7 +376,10 @@ With `save_tool_results_as_artifacts` enabled (the default), each non-artifact t A block over `max_code_mode_metadata_chars` degrades in tiers instead of vanishing. The Python version and package list survive every tier — they're small, and the model cannot discover them any other way. 1. **Full** — as above. -2. **Names only** — signatures and docstrings dropped, import lines kept: +2. **Names only** — types, signatures and docstrings dropped, import lines kept. `` gains: + + > Before calling a function you haven't used, print its stub to see its signature, docstring and the fields of its return type: `/tools//.py`, or `/tools/.py` for a function imported from `tools`. Print every stub you need in one `execute_code` call. + ~~~ @@ -372,11 +389,11 @@ A block over `max_code_mode_metadata_chars` degrades in tiers instead of vanishi ~~~ -3. **Pointer** — the catalog is replaced by a discovery note, and this tier is emitted no matter how small the budget: +3. **Pointer** — the catalog is replaced by a note, and this tier is emitted no matter how small the budget. `` gains the same text, prefixed with "List `/tools/` and its module directories to find the functions available.": ~~~ - The full catalog is too large to include here. List `/tools/` and read a file's docstring to see what's available. + The full catalog is too large to include here. ~~~ @@ -384,11 +401,12 @@ At the last tier the model navigates the sandbox from Python instead: ```python import pathlib -print(list(pathlib.Path("/tools").iterdir())) -print(open("/tools/slack/send_message.py").read()) # signature + docstring +print(sorted(pathlib.Path("/tools").rglob("*.py"))) +print(open("/tools/slack/send_message.py").read()) # types, signature + docstring +print(open("/tools/slack/list_channels.py").read()) ``` -Text and JSON-like MIME types travel as plain strings in artifact tools; binary content is base64-encoded. `load_artifact` returns the content itself: the parsed value for JSON, a `str` for other text, and `{"mime_type": str, "bytes": str}` (base64) for binary. +Text and JSON-like MIME types travel as plain strings in artifact tools; binary content is base64-encoded. `load_artifact` returns a dictionary with `kind`, `data`, and `mime_type`. Text and JSON use `kind="text"` and keep the string in `data`. Binary artifacts use `kind="bytes"` and store base64 in `data`. ## 🛡️ Safety @@ -421,7 +439,7 @@ Named "Unsafe" intentionally: it binds a TCP listener on `0.0.0.0`, communicates ## ⚠️ Limitations - **No credential-requesting tools.** Tools that need ADK to request credentials, confirmations, UI widgets, agent transfer, escalation, or that yield without an immediate response are rejected with a structured error. -- **State is turn-scoped.** Variables and the working directory persist across `execute_code` calls **within** a turn, but reset between turns. Use `save_artifact` / `load_artifact` to persist across turns. +- **State is turn-scoped.** Variables and the working directory persist across `execute_code` calls **within** a turn, but reset between turns. Files the code writes and tool results are saved as artifacts; use `load_artifact` to read them in a later turn. - **No runtime package installation.** The sandbox ships with the Python Standard Library and the runtime's own dependencies only. Extra packages must be baked into the image at build time. ## 🛠️ Development diff --git a/src/adk_code_mode/_artifact_tools.py b/src/adk_code_mode/_artifact_tools.py index becde1b..bf3bcb2 100644 --- a/src/adk_code_mode/_artifact_tools.py +++ b/src/adk_code_mode/_artifact_tools.py @@ -10,14 +10,14 @@ The wire format is JSON. Binary content is base64 over the wire — the model encodes it before calling ``save_artifact`` and decodes it after -``load_artifact``. ``load_artifact`` returns the content itself: parsed JSON, -a ``str`` for other text, or ``{"mime_type", "bytes"}`` for binary. +``load_artifact``. Loaded artifacts return a flat ``{"kind", "data", +"mime_type"}`` dict so the model dispatches on ``kind`` without importing +host-side types. """ from __future__ import annotations import base64 -import json from typing import Any from google.adk.tools.function_tool import FunctionTool @@ -31,7 +31,6 @@ def _is_text_mime(mime_type: str | None) -> bool: if not mime_type: return False - mime_type = _base_mime(mime_type) if mime_type.startswith(_TEXT_MIME_PREFIXES): return True if mime_type in _TEXT_MIME_TYPES: @@ -39,23 +38,6 @@ def _is_text_mime(mime_type: str | None) -> bool: return mime_type.endswith("+json") or mime_type.endswith("+xml") -def _is_json_mime(mime_type: str | None) -> bool: - base = _base_mime(mime_type or "") - return base == "application/json" or base.endswith("+json") - - -def _base_mime(mime_type: str) -> str: - return mime_type.split(";", 1)[0].strip().lower() - - -def _charset(mime_type: str | None) -> str: - for param in (mime_type or "").split(";")[1:]: - key, _, value = param.partition("=") - if key.strip().lower() == "charset": - return value.strip().strip('"') - return "utf-8" - - async def save_artifact( *, filename: str, @@ -83,13 +65,11 @@ async def save_artifact( ) ``custom_metadata`` is custom metadata to associate with the artifact. - ``load_artifact`` gives back the parsed JSON, the text, or - ``{"mime_type", "bytes"}`` with the same base64 string. """ if _is_text_mime(mime_type): part = genai_types.Part( inline_data=genai_types.Blob( - data=content.encode(_charset(mime_type)), + data=content.encode("utf-8"), mime_type=mime_type or "text/plain", ) ) @@ -108,42 +88,50 @@ async def load_artifact( filename: str, version: int | None = None, tool_context: ToolContext, -) -> Any: - """Load an artifact by filename and return its content. Returns ``None`` if not found. +) -> dict[str, Any] | None: + """Load an artifact by filename. Returns ``None`` if not found. - The return value depends on the artifact's mime type: + Returned dict shape: - - JSON (``application/json``, ``*+json``): the parsed value — for a saved - tool result, exactly what the original call returned. - - Other text (``text/*``, XML, JavaScript): the text as a ``str``. - - Binary: ``{"mime_type": , "bytes": }``. + { + "kind": "text" | "bytes", + "data": , # text: the string; bytes: base64-encoded + "mime_type": , + } Example: - rows = load_artifact(filename="search_call-1.json") # list/dict, already parsed - notes = load_artifact(filename="notes.txt") # str - image = load_artifact(filename="chart.png") - png = base64.b64decode(image["bytes"]) + result = load_artifact(filename="report.json") + if result is None: + return + if result["kind"] == "text" and result["mime_type"] == "application/json": + payload = json.loads(result["data"]) + elif result["kind"] == "bytes": + blob = base64.b64decode(result["data"]) """ part = await tool_context.load_artifact(filename, version=version) if part is None: return None inline = part.inline_data - if inline is None: - return part.text or "" - mime_type = inline.mime_type - data = inline.data or b"" - if isinstance(data, str): - data = data.encode("utf-8") - if not _is_text_mime(mime_type): - return {"mime_type": mime_type, "bytes": base64.b64encode(data).decode("ascii")} - text = data.decode(_charset(mime_type)) - if _is_json_mime(mime_type): - try: - return json.loads(text) - except ValueError: - return text - return text + if inline is not None: + mime_type = inline.mime_type + data = inline.data or b"" + if isinstance(data, str): + data = data.encode("utf-8") + if _is_text_mime(mime_type): + return { + "kind": "text", + "data": data.decode("utf-8"), + "mime_type": mime_type, + } + return { + "kind": "bytes", + "data": base64.b64encode(data).decode("ascii"), + "mime_type": mime_type, + } + if part.text is not None: + return {"kind": "text", "data": part.text, "mime_type": None} + return {"kind": "text", "data": "", "mime_type": None} async def list_artifacts(*, tool_context: ToolContext) -> list[str]: diff --git a/src/adk_code_mode/_json.py b/src/adk_code_mode/_json.py index 14f7cc2..541c2cc 100644 --- a/src/adk_code_mode/_json.py +++ b/src/adk_code_mode/_json.py @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: 2025-present Agenta # # SPDX-License-Identifier: Apache-2.0 -"""JSON-safe conversion shared by the RPC reply and saved tool results.""" +"""JSON-safe conversion for the sandbox RPC reply.""" from __future__ import annotations diff --git a/src/adk_code_mode/mcp_results.py b/src/adk_code_mode/mcp_results.py index a41d847..73f077e 100644 --- a/src/adk_code_mode/mcp_results.py +++ b/src/adk_code_mode/mcp_results.py @@ -1,14 +1,7 @@ # SPDX-FileCopyrightText: 2025-present Agenta # # SPDX-License-Identifier: Apache-2.0 -"""Return an MCP tool's ``structuredContent`` to code instead of the whole result. - -ADK's ``McpTool`` returns the full ``CallToolResult`` (``content``, -``structuredContent``, ``isError``). When the tool declares an ``outputSchema``, -the MCP spec requires ``structuredContent`` to match it, and ``content`` is meant to -be a serialised copy for older clients. So code gets ``structuredContent`` alone, -and the stub's return type is built from the ``outputSchema``. -""" +"""Describe the MCP result envelope returned by ADK.""" from __future__ import annotations @@ -18,19 +11,20 @@ from google.adk.tools.mcp_tool.mcp_tool import McpTool -def output_schema(tool: BaseTool) -> dict[str, Any] | None: - """Return the ``outputSchema`` of an MCP tool, or ``None`` for any other tool. - - ``tool`` is the tool itself, not a ``ToolResultArtifactTool`` wrapping it. - """ - return tool.raw_mcp_tool.outputSchema if isinstance(tool, McpTool) else None - - -def structured_result(tool: BaseTool, result: Any) -> Any: - """Return ``structuredContent`` when ``tool`` is an MCP tool with an ``outputSchema``.""" - if output_schema(tool) and isinstance(result, dict) and "structuredContent" in result: - return result["structuredContent"] - return result +def response_schema(tool: BaseTool) -> dict[str, Any] | None: + """Describe an MCP result and type its structured content from the tool's output schema.""" + if not isinstance(tool, McpTool) or not (output := tool.raw_mcp_tool.outputSchema): + return None + return { + **{key: output[key] for key in ("$defs", "definitions") if key in output}, + "type": "object", + "properties": { + "content": {"type": "array", "items": {}}, + "isError": {"type": "boolean"}, + "structuredContent": {"anyOf": [output, {"type": "null"}]}, + }, + "required": ["content", "isError"], + } -__all__ = ["output_schema", "structured_result"] +__all__ = ["response_schema"] diff --git a/src/adk_code_mode/metadata.py b/src/adk_code_mode/metadata.py index f378a7b..9818db5 100644 --- a/src/adk_code_mode/metadata.py +++ b/src/adk_code_mode/metadata.py @@ -26,7 +26,7 @@ from typing import Any from adk_code_mode.runtime.protocol import ReadyFrame -from adk_code_mode.tools.catalog import render_catalog +from adk_code_mode.tools.catalog import Detail, render_catalog from adk_code_mode.tools.namespacing import NamespacedTool _OPEN = "" @@ -39,9 +39,20 @@ "functions you can import from the `tools` package." ) -DISCOVER_TOOLS = "List `/tools/` and read a file's docstring to see what's available." +# Every tier without signatures needs these: the full tier shows them inline. +READ_BEFORE_CALLING = ( + "Before calling a function you haven't used, print its stub to see its " + "signature, docstring and the fields of its return type: " + "`/tools//.py`, or `/tools/.py` for a function imported from " + "`tools`. Print every stub you need in one `execute_code` call." +) + +DISCOVER_TOOLS = ( + f"List `/tools/` and its module directories to find the functions available. " + f"{READ_BEFORE_CALLING}" +) -_TOOLS_TOO_LARGE = f"The full catalog is too large to include here. {DISCOVER_TOOLS}" +_TOOLS_TOO_LARGE = "The full catalog is too large to include here." # identity -> (python version, {import name: {distribution name: version}}) _ENVIRONMENTS: dict[str, tuple[str, dict[str, dict[str, str]]]] = {} @@ -96,8 +107,10 @@ def render(*, identity: str, namespaced: list[NamespacedTool], max_chars: int) - """Render the ```` block, degrading until it fits ``max_chars``. ``max_chars`` is measured against the final string, tags included. Tiers, - in order: the full catalog; import lines only (no signatures or - docstrings); a pointer telling the model to read ``/tools/`` itself. The + in order: the full catalog; import lines only (no types, signatures or + docstrings), with ``READ_BEFORE_CALLING`` added to the how-to-use text; a + pointer, with ``DISCOVER_TOOLS`` added instead, telling the model to read + ``/tools/`` itself. The last tier is unconditional — an oversized tool surface must never cost the model the Python version and package list, which are small and which it cannot discover any other way. @@ -108,9 +121,9 @@ def render(*, identity: str, namespaced: list[NamespacedTool], max_chars: int) - """ python_version, packages = _ENVIRONMENTS.get(identity, ("", {})) - head = [_tag("how-to-use", HOW_TO_USE)] + environment: list[str] = [] if python_version: - head.append(_inline_tag("python-version", python_version)) + environment.append(_inline_tag("python-version", python_version)) if packages: lines = [] for import_name, distributions in sorted( @@ -127,21 +140,29 @@ def render(*, identity: str, namespaced: list[NamespacedTool], max_chars: int) - if providers: lines.append(f"{import_name}: {providers}") if lines: - head.append(_tag("installed-packages", "\n".join(lines))) + environment.append(_tag("installed-packages", "\n".join(lines))) + how_to_use = _tag("how-to-use", HOW_TO_USE) if not namespaced: - return _wrap(head) - - for detail in ("full", "names"): - block = _wrap([*head, _tag("tools-package", render_catalog(namespaced, detail=detail))]) + return _wrap([how_to_use, *environment]) + + tiers: list[tuple[Detail, str]] = [ + ("full", how_to_use), + ("names", _tag("how-to-use", f"{HOW_TO_USE} {READ_BEFORE_CALLING}")), + ] + for detail, tier_how_to_use in tiers: + catalog = _tag("tools-package", render_catalog(namespaced, detail=detail)) + block = _wrap([tier_how_to_use, *environment, catalog]) if len(block) <= max_chars: return block - return _wrap([*head, _tag("tools-package", _TOOLS_TOO_LARGE)]) + discover = _tag("how-to-use", f"{HOW_TO_USE} {DISCOVER_TOOLS}") + return _wrap([discover, *environment, _tag("tools-package", _TOOLS_TOO_LARGE)]) __all__ = [ "DISCOVER_TOOLS", "HOW_TO_USE", + "READ_BEFORE_CALLING", "backend_identity", "record", "render", diff --git a/src/adk_code_mode/runtime/remote.py b/src/adk_code_mode/runtime/remote.py index 42a95df..5e10f1e 100644 --- a/src/adk_code_mode/runtime/remote.py +++ b/src/adk_code_mode/runtime/remote.py @@ -137,7 +137,6 @@ async def _connect( return _RemoteSandboxSession( ws=ws, workdir_path=workdir_path, - max_message_size=self.max_message_size, max_upload_workspace_bytes=self.max_upload_workspace_bytes, max_download_workspace_bytes=self.max_download_workspace_bytes, ) @@ -149,13 +148,11 @@ def __init__( *, ws: Any, workdir_path: str, - max_message_size: int, max_upload_workspace_bytes: int, max_download_workspace_bytes: int, ) -> None: self._ws = ws self._workdir_path = workdir_path - self._max_message_size = max_message_size self._max_upload_workspace_bytes = max_upload_workspace_bytes self._max_download_workspace_bytes = max_download_workspace_bytes @@ -175,13 +172,6 @@ async def begin_block(self, input_paths: Sequence[str]) -> None: async def send(self, frame: Frame) -> None: payload = encode(frame) - # The sandbox closes the connection on an oversized message, so refuse it - # here instead; a tool reply then reaches the code as an error. - if len(payload) > self._max_message_size: - raise ValueError( - f"{type(frame).__name__} ({len(payload):,} bytes) exceeds " - f"max_message_size ({self._max_message_size:,})" - ) try: await self._ws.send(payload.decode("utf-8")) except websockets.ConnectionClosed as exc: diff --git a/src/adk_code_mode/tool.py b/src/adk_code_mode/tool.py index 59554d8..51000c9 100644 --- a/src/adk_code_mode/tool.py +++ b/src/adk_code_mode/tool.py @@ -133,22 +133,40 @@ def __init__(self, state: _BlockRunState) -> None: "version, preinstalled packages, and available functions." ) -_DESCRIPTION_SUFFIX = ( +_DESCRIPTION_OUTPUT = ( " Print function calls, variables, etc. to see output, only stdout/stderr are " - "returned. Variables and the working directory persist across calls within the " - "same turn and reset on the next turn — use `save_artifact` / `load_artifact` " - "(imported the same way) to persist across turns. Files created or changed by " - "the code are saved as artifacts automatically. A tool call whose API answers " - "with an HTTP error raises `HTTPStatusError` (with `status_code` and `body`), and " - "one that fails for any other reason raises `ToolError`; import both from `tools`." + "returned. Variables and files persist across calls within the same turn and " + "reset on the next turn." +) + +_DESCRIPTION_ERRORS = ( + " Tool calls that call APIs raise HTTP errors as `HTTPStatusError` (with " + "`status_code` and `body`); other failures raise `ToolError`. Import both from " + "`tools`." ) -def _build_description(*, append_metadata: bool, timeout_seconds: int | None) -> str: +def _build_description( + *, + append_metadata: bool, + artifact_tools: bool, + tool_results: bool, + timeout_seconds: int | None, +) -> str: """Build the tool description for this tool's configured surface.""" pointer = _DESCRIPTION_WITH_METADATA if append_metadata else metadata.DISCOVER_TOOLS + saved = "Every file the code writes in the working directory" + if tool_results: + saved += ", and every tool's result," + saved = f" {saved} is saved as an artifact" + if artifact_tools: + saved += "; use `load_artifact` (imported the same way) to read one in a later turn" + if tool_results: + saved += ". Large tool results return a `tool_result_artifact` pointer" limit = f" Each call can run for up to {timeout_seconds}s." if timeout_seconds else "" - return f"{_DESCRIPTION_PREFIX}{pointer}{_DESCRIPTION_SUFFIX}{limit}" + return ( + f"{_DESCRIPTION_PREFIX}{pointer}{_DESCRIPTION_OUTPUT}{saved}.{_DESCRIPTION_ERRORS}{limit}" + ) ArtifactsSavedCallback = Callable[[InvocationContext, dict[str, int]], Awaitable[None]] @@ -226,23 +244,27 @@ def __init__( ``UnsafeLocalDockerBackend`` for local development). include_artifact_tools: If true (default), ``save_artifact`` / ``load_artifact`` / ``list_artifacts`` are injected as top-level - tools. Set ``False`` for a strict tool surface. + tools, and the tool description points the model at + ``load_artifact``. Set ``False`` for a strict tool surface. save_tool_results_as_artifacts: If true (default), every non-artifact tool is wrapped so its result is also saved as a session artifact, - reloadable in later turns via ``load_artifact``. The code always - gets the full result. Set ``False`` to skip saving. + reloadable in later turns via ``load_artifact``. Large results + return an artifact pointer. Set ``False`` to skip saving and + always get the full result inline. append_code_mode_metadata_to_system_instruction: If true (default), every model turn's system instruction gets a ```` block describing the sandbox: its Python version, its preinstalled - packages, and every available function's signature and docstring. + packages, and every available function's signature, return type and + docstring. Set ``False`` for a bare system instruction — the tool description then tells the model to discover functions by listing ``/tools/`` from within the code it runs, and the Python version and package list are not surfaced at all. max_code_mode_metadata_chars: Budget for the whole rendered ```` block, tags included. An oversized block first - drops signatures and docstrings (keeping the import lines), then - replaces the catalog with a pointer to ``/tools/``. The Python + drops types, signatures and docstrings (keeping the import lines + and telling the model to read a function's stub before calling it), + then replaces the catalog with a pointer to ``/tools/``. The Python version and package list are always kept. Only consulted when ``append_code_mode_metadata_to_system_instruction`` is true. max_output_chars: Caps stdout/stderr handed back to the model. @@ -272,6 +294,8 @@ def __init__( name="execute_code", description=_build_description( append_metadata=append_code_mode_metadata_to_system_instruction, + artifact_tools=include_artifact_tools, + tool_results=save_tool_results_as_artifacts, timeout_seconds=timeout_seconds, ), ) diff --git a/src/adk_code_mode/tool_result_artifacts.py b/src/adk_code_mode/tool_result_artifacts.py index c7b3271..211e9c3 100644 --- a/src/adk_code_mode/tool_result_artifacts.py +++ b/src/adk_code_mode/tool_result_artifacts.py @@ -16,8 +16,10 @@ markers, land in the artifact's ``custom_metadata`` so a host (e.g. an A2A layer) can decide whether and how to forward the result to the user. -The result is always returned unchanged; the artifact lets a later turn reload it -with ``load_artifact``. +Small results are returned to the model unchanged. Results whose serialised form +exceeds ``large_result_threshold`` are replaced with a short notice pointing at +the artifact, keeping large payloads out of the model's context while remaining +reloadable with ``load_artifact``. """ from __future__ import annotations @@ -30,8 +32,6 @@ from google.adk.tools.tool_context import ToolContext from google.genai import types -from adk_code_mode.mcp_results import structured_result -from adk_code_mode._json import json_safe from adk_code_mode.tool_failures import raise_for_failed_result # Transport-neutral metadata stamped on saved tool-result artifacts. Downstream @@ -41,6 +41,8 @@ TOOL_RESULT_DESCRIPTION_KEY = "code_mode.artifact_description" TOOL_RESULT_FILENAME_KEY = "code_mode.artifact_filename" +DEFAULT_LARGE_RESULT_THRESHOLD = 50_000 + _ARTIFACT_NAME_PARAM = "artifact_name" _ARTIFACT_DESCRIPTION_PARAM = "artifact_description" @@ -53,13 +55,19 @@ class ToolResultArtifactTool(BaseTool): ``artifact_description``, for naming the saved artifact. """ - def __init__(self, wrapped: BaseTool) -> None: + def __init__( + self, + wrapped: BaseTool, + *, + large_result_threshold: int = DEFAULT_LARGE_RESULT_THRESHOLD, + ) -> None: super().__init__( name=wrapped.name, description=wrapped.description, is_long_running=wrapped.is_long_running, ) self._wrapped = wrapped + self._large_result_threshold = large_result_threshold # Don't shadow a real tool parameter that happens to share the name. existing = _declared_param_names(wrapped) self._inject_name = _ARTIFACT_NAME_PARAM not in existing @@ -107,8 +115,7 @@ async def run_async(self, *, args: dict[str, Any], tool_context: ToolContext) -> stem = artifact_name or _default_stem(self.name, tool_context) filename = f"{stem}.json" - # Save what code receives, so reloading the artifact gives the same value. - data, mime_type = _serialise_result(structured_result(self._wrapped, result)) + data, mime_type = _serialise_result(result) metadata: dict[str, Any] = { TOOL_RESULT_METADATA_KEY: "true", TOOL_RESULT_FILENAME_KEY: filename, @@ -127,12 +134,23 @@ async def run_async(self, *, args: dict[str, Any], tool_context: ToolContext) -> ) raise_for_failed_result(self._wrapped, result) + if len(data) > self._large_result_threshold: + return { + "tool_result_artifact": filename, + "note": ( + f"The tool result ({len(data):,} bytes) was saved as an artifact " + f"{filename!r} instead of being returned inline. Reload it with " + f"load_artifact(filename={filename!r})." + ), + } return result -def wrap_tool_result_as_artifact(tool: BaseTool) -> ToolResultArtifactTool: +def wrap_tool_result_as_artifact( + tool: BaseTool, *, large_result_threshold: int = DEFAULT_LARGE_RESULT_THRESHOLD +) -> ToolResultArtifactTool: """Wrap ``tool`` so its results are saved as session artifacts.""" - return ToolResultArtifactTool(tool) + return ToolResultArtifactTool(tool, large_result_threshold=large_result_threshold) def _declared_param_names(tool: BaseTool) -> set[str]: @@ -158,10 +176,11 @@ def _serialise_result(result: Any) -> tuple[bytes, str]: return bytes(result), "application/octet-stream" if isinstance(result, str): return result.encode("utf-8"), "text/plain; charset=utf-8" - return json.dumps(json_safe(result), ensure_ascii=False).encode("utf-8"), "application/json" + return json.dumps(result, default=str, ensure_ascii=False).encode("utf-8"), "application/json" __all__ = [ + "DEFAULT_LARGE_RESULT_THRESHOLD", "TOOL_RESULT_DESCRIPTION_KEY", "TOOL_RESULT_FILENAME_KEY", "TOOL_RESULT_METADATA_KEY", diff --git a/src/adk_code_mode/tools/catalog.py b/src/adk_code_mode/tools/catalog.py index dfa8066..80f6a57 100644 --- a/src/adk_code_mode/tools/catalog.py +++ b/src/adk_code_mode/tools/catalog.py @@ -5,10 +5,10 @@ The catalog is grouped by module: ``# tools.`` sections (sorted), then ``# tools`` for any top-level tools. Each section opens with an import -line and is followed by ``.pyi``-style function definitions (signature + -docstring + ``...`` body). ``adk_code_mode.metadata`` wraps the result in a -```` tag inside the ```` block; this module returns -just the inner content. +line, then the ``TypedDict`` classes its functions return, then ``.pyi``-style +function definitions (signature + docstring + ``...`` body). +``adk_code_mode.metadata`` wraps the result in a ```` tag inside +the ```` block; this module returns just the inner content. """ from __future__ import annotations @@ -26,13 +26,15 @@ def _render_entry(rt: RenderedTool) -> str: return f"{rt.signature_for('catalog')}\n {rt.docstring}\n ...\n" -def _render_module_section(*, header: str, import_line: str, entries: list[str]) -> str: - # At ``names`` detail there are no entries, so the import line — which - # already lists every function in the module — is the whole section. - if not entries: - return f"# {header}\n\n{import_line}\n" - blocks = "\n".join(entries) - return f"# {header}\n\n{import_line}\n\n{blocks}" +def _render_module_section(module: str, rendered: list[RenderedTool], detail: Detail) -> str: + import_line = f"from {module} import {', '.join(rt.attribute for rt in rendered)}" + # At ``names`` detail the import line, which already lists every function + # in the module, is the whole section. + if detail == "names": + return f"# {module}\n\n{import_line}\n" + types = dict.fromkeys(source for rt in rendered for source in rt.types) + blocks = [*(f"{source}\n" for source in types), *(_render_entry(rt) for rt in rendered)] + return f"# {module}\n\n{import_line}\n\n" + "\n".join(blocks) def render_catalog(namespaced: list[NamespacedTool], *, detail: Detail = "full") -> str: @@ -40,44 +42,24 @@ def render_catalog(namespaced: list[NamespacedTool], *, detail: Detail = "full") Namespaced sections come first in namespace-sorted order, then top-level tools. Tools within a section are sorted by attribute name for - determinism. + determinism. A section defines its tools' ``TypedDict`` return types, + shared by shape, once ahead of their signatures. Class names are unique + across sections, since the catalog reads as one Python block. - ``detail="names"`` drops every signature and docstring, leaving only the - per-module import lines. It is the first fallback when the full catalog + ``detail="names"`` drops every type, signature and docstring, leaving only + the per-module import lines. It is the fallback when the full catalog doesn't fit the system-instruction budget. """ - grouped: dict[str | None, list[NamespacedTool]] = {} - for nt in namespaced: - grouped.setdefault(nt.namespace, []).append(nt) - - sections: list[str] = [] - - for ns in sorted(n for n in grouped if n is not None): - tools = sorted(grouped[ns], key=lambda t: t.attribute) - rendered = render_tools(tools) - names = ", ".join(rt.attribute for rt in rendered) - entries = [_render_entry(rt) for rt in rendered] if detail == "full" else [] - sections.append( - _render_module_section( - header=f"tools.{ns}", - import_line=f"from tools.{ns} import {names}", - entries=entries, - ) - ) - - top_level = sorted(grouped.get(None, []), key=lambda t: t.attribute) - if top_level: - rendered = render_tools(top_level) - names = ", ".join(rt.attribute for rt in rendered) - entries = [_render_entry(rt) for rt in rendered] if detail == "full" else [] - sections.append( - _render_module_section( - header="tools", - import_line=f"from tools import {names}", - entries=entries, - ) - ) - + grouped: dict[str | None, list[RenderedTool]] = {} + for rt in sorted(render_tools(namespaced), key=lambda rt: rt.attribute): + grouped.setdefault(rt.namespace, []).append(rt) + + sections = [ + _render_module_section(f"tools.{ns}", grouped[ns], detail) + for ns in sorted(n for n in grouped if n is not None) + ] + if None in grouped: + sections.append(_render_module_section("tools", grouped[None], detail)) return "\n".join(sections) diff --git a/src/adk_code_mode/tools/dispatcher.py b/src/adk_code_mode/tools/dispatcher.py index af7bc6a..f51eedf 100644 --- a/src/adk_code_mode/tools/dispatcher.py +++ b/src/adk_code_mode/tools/dispatcher.py @@ -17,9 +17,6 @@ ``agent.canonical_on_tool_error_callbacks``. If no replacement, re-raise. 5. ``plugin_manager.run_after_tool_callback`` 6. ``agent.canonical_after_tool_callbacks`` (first non-``None`` replaces result) - -Callbacks see the result in ADK's shape; only then does an MCP tool with an -``outputSchema`` hand code its ``structuredContent`` (see ``mcp_results``). """ from __future__ import annotations @@ -36,7 +33,6 @@ from google.adk.tools.tool_context import ToolContext from google.genai import types as genai_types -from adk_code_mode.mcp_results import structured_result from adk_code_mode.tool_failures import HTTPStatusError, raise_for_failed_result from adk_code_mode.tool_result_artifacts import ToolResultArtifactTool from adk_code_mode.tools.namespacing import Registry @@ -182,8 +178,7 @@ async def _run_and_merge(self, tool: BaseTool, args: dict[str, Any]) -> Any: async with self._merge_lock: _deep_merge_dicts(self._ctx.session.state, tool_context.actions.state_delta) self._artifact_delta.update(tool_context.actions.artifact_delta) - inner = tool._wrapped if isinstance(tool, ToolResultArtifactTool) else tool - return structured_result(inner, response) + return response async def _run_with_callbacks( self, diff --git a/src/adk_code_mode/tools/stubs.py b/src/adk_code_mode/tools/stubs.py index f06343a..3519675 100644 --- a/src/adk_code_mode/tools/stubs.py +++ b/src/adk_code_mode/tools/stubs.py @@ -26,10 +26,20 @@ from dataclasses import dataclass from typing import Any, Literal -from adk_code_mode.mcp_results import output_schema +from adk_code_mode.mcp_results import response_schema from adk_code_mode.tool_result_artifacts import ToolResultArtifactTool from adk_code_mode.tools.namespacing import NamespacedTool, PythonNameCollisionError +_TOOL_RESULT_ARTIFACT_SCHEMA = { + "title": "ToolResultArtifact", + "type": "object", + "properties": { + "tool_result_artifact": {"type": "string"}, + "note": {"type": "string"}, + }, + "required": ["tool_result_artifact", "note"], +} + _PRIMITIVES = { "string": "str", "integer": "int", @@ -596,15 +606,16 @@ class _ModuleTypes: Identical shapes get one class, so the catalog defines a shape that many tools return once. A class is named after its schema (a ``$defs`` key or - ``title``), else the property holding it; when a different shape already - has that name, the parent's name is prefixed, then a number appended. An + ``title``), else the property holding it; when that name is taken, in any + module, the parent's name is prefixed, then a number appended. An untitled top-level object is the tool's own ``Response``, never shared. """ - def __init__(self, reserved: set[str]) -> None: + def __init__(self, taken: set[str]) -> None: self.sources: dict[str, str] = {} self._by_shape: dict[str, str] = {} - self._reserved = _TYPING_NAMES | reserved + # Shared by every module, so a name means one class across the catalog. + self._taken = taken def uses(self, type_expr: str) -> tuple[str, ...]: """Sources of the classes ``type_expr`` refers to, transitively, parents first.""" @@ -688,10 +699,11 @@ def _claim(self, candidates: list[str]) -> str: valid = [ name for name in candidates - if name.isidentifier() and not keyword.iskeyword(name) and name not in self._reserved + if name.isidentifier() and not keyword.iskeyword(name) and name not in _TYPING_NAMES ] or ["Type"] numbered = (f"{valid[0]}{n}" for n in itertools.count(2)) - name = next(name for name in itertools.chain(valid, numbered) if name not in self.sources) + name = next(name for name in itertools.chain(valid, numbered) if name not in self._taken) + self._taken.add(name) self.sources[name] = "" # Taken; the source follows once the fields are rendered. return name @@ -853,12 +865,11 @@ def render_tools(tools: list[NamespacedTool]) -> list[RenderedTool]: Captures everything stub and catalog rendering need: signature parts, return type and its ``TypedDict``s, formatted docstring, dotted path, namespace. Tools of one module share their ``TypedDict``s; names are - claimed in attribute order, so stubs and the catalog agree on them. + unique across modules and claimed in attribute order, so stubs and the + catalog agree on them. """ - modules = { - ns: _ModuleTypes({t.attribute for t in tools if t.namespace == ns}) - for ns in {t.namespace for t in tools} - } + taken = {t.attribute for t in tools} + modules = {ns: _ModuleTypes(taken) for ns in {t.namespace for t in tools}} rendered = { id(nt): _render_tool(nt, modules[nt.namespace]) for nt in sorted(tools, key=lambda t: t.attribute) @@ -918,13 +929,19 @@ def _render_tool(nt: NamespacedTool, types: _ModuleTypes) -> RenderedTool: docstring = _format_docstring(description=description, param_docs=param_docs) tool = nt.resolved.tool + wraps_result = isinstance(tool, ToolResultArtifactTool) inner = tool._wrapped if isinstance(tool, ToolResultArtifactTool) else tool - response = output_schema(inner) or getattr(declaration, "response_json_schema", None) + response = response_schema(inner) or getattr(declaration, "response_json_schema", None) return_type = "Any" if isinstance(response, dict) and response: defs = {**(response.get("definitions") or {}), **(response.get("$defs") or {})} short = _pascal(nt.attribute) + "Response" return_type = types.type_of(response, short=short, parent="", defs=defs, own=True) + if wraps_result and return_type != "Any": + pointer = types.type_of( + _TOOL_RESULT_ARTIFACT_SCHEMA, short="ToolResultArtifact", parent="", defs={} + ) + return_type = f"{return_type} | {pointer}" return RenderedTool( attribute=nt.attribute, diff --git a/tests/test_artifact_tools.py b/tests/test_artifact_tools.py index 4b8eb9d..bb5383a 100644 --- a/tests/test_artifact_tools.py +++ b/tests/test_artifact_tools.py @@ -154,58 +154,40 @@ async def test_save_artifact_defaults_custom_metadata_to_none() -> None: assert custom_metadata is None -@pytest.mark.parametrize( - ("data", "mime_type", "expected"), - [ - (b'{"ok": true, "rows": [1, 2]}', "application/json", {"ok": True, "rows": [1, 2]}), - (b"[1, 2]", "application/json; charset=utf-8", [1, 2]), - (b'{"a": 1}', "application/ld+json", {"a": 1}), - (b"not json", "application/json", "not json"), - (b"# Report", "text/markdown", "# Report"), - (b"plain", "text/plain; charset=utf-8", "plain"), - ("é".encode("iso-8859-1"), "application/xml; charset=iso-8859-1", "é"), - (b"\x00\x01\x02", "image/png", {"mime_type": "image/png", "bytes": "AAEC"}), - ], -) @pytest.mark.asyncio -async def test_load_artifact_returns_the_content( - data: bytes, mime_type: str, expected: Any -) -> None: - part = genai_types.Part(inline_data=genai_types.Blob(data=data, mime_type=mime_type)) +async def test_load_artifact_returns_text_kind_for_text_mime() -> None: + part = genai_types.Part( + inline_data=genai_types.Blob(data=b'{"ok": true}', mime_type="application/json") + ) ctx = _StubToolContext(load_result=part) - result = await load_artifact(filename="a", tool_context=ctx) # type: ignore[arg-type] - assert result == expected + result = await load_artifact( + filename="report.json", + tool_context=ctx, # type: ignore[arg-type] + ) + + assert result == { + "kind": "text", + "data": '{"ok": true}', + "mime_type": "application/json", + } @pytest.mark.asyncio -async def test_load_artifact_returns_text_part_as_str() -> None: - ctx = _StubToolContext(load_result=genai_types.Part(text="hello")) - assert await load_artifact(filename="a", tool_context=ctx) == "hello" # type: ignore[arg-type] - - -@pytest.mark.parametrize( - ("content", "mime_type", "expected"), - [ - ('{"status": "ok", "n": [1, 2]}', "application/json", {"status": "ok", "n": [1, 2]}), - ("# Notes", "text/markdown", "# Notes"), - ("café", "text/plain; charset=iso-8859-1", "café"), - ("AAEC", "image/png", {"mime_type": "image/png", "bytes": "AAEC"}), - ("AAEC", None, {"mime_type": "application/octet-stream", "bytes": "AAEC"}), - ], -) -@pytest.mark.asyncio -async def test_save_artifact_round_trips_through_load_artifact( - content: str, mime_type: str | None, expected: Any -) -> None: - saver = _StubToolContext() - await save_artifact( - filename="a", - content=content, - mime_type=mime_type, - tool_context=saver, # type: ignore[arg-type] +async def test_load_artifact_returns_bytes_kind_for_binary_mime() -> None: + raw = b"\x00\x01\x02" + part = genai_types.Part( + inline_data=genai_types.Blob(data=raw, mime_type="application/octet-stream") + ) + ctx = _StubToolContext(load_result=part) + result = await load_artifact( + filename="blob.bin", + tool_context=ctx, # type: ignore[arg-type] ) - loader = _StubToolContext(load_result=saver.saved[0][1]) - assert await load_artifact(filename="a", tool_context=loader) == expected # type: ignore[arg-type] + + assert result is not None + assert result["kind"] == "bytes" + assert result["mime_type"] == "application/octet-stream" + assert base64.b64decode(result["data"]) == raw @pytest.mark.asyncio diff --git a/tests/test_catalog.py b/tests/test_catalog.py index e1221d3..73881c6 100644 --- a/tests/test_catalog.py +++ b/tests/test_catalog.py @@ -18,15 +18,18 @@ def __init__( *, description: str, schema: dict[str, object] | None = None, + response: dict[str, object] | None = None, ) -> None: super().__init__(name=name, description=description) self._schema = schema or {} + self._response = response def _get_declaration(self) -> genai_types.FunctionDeclaration | None: return genai_types.FunctionDeclaration( name=self.name, description=self.description, parameters_json_schema=self._schema, + response_json_schema=self._response, ) @@ -104,3 +107,47 @@ def test_render_catalog_groups_tools_by_namespace_and_sorts() -> None: gmail_idx = catalog.index("# tools.gmail") slack_idx = catalog.index("# tools.slack") assert gmail_idx < slack_idx, "namespaces sorted alphabetically" + + +def test_render_catalog_defines_shared_module_types_once_before_signatures() -> None: + bookings = _FakeToolset("bookings") + item = {"type": "object", "properties": {"id": {"type": "string"}}, "required": ["id"]} + get = _SchemaTool( + "get_booking", + description="Get a booking.", + response={"type": "object", "properties": {"booking": item}}, + ) + search = _SchemaTool("search", description="Search.", response={"type": "array", "items": item}) + ping = _SchemaTool("ping", description="Ping.") + catalog = render_catalog(_build([(get, bookings), (search, bookings), (ping, bookings)])) + assert catalog == ( + "# tools.bookings\n\n" + "from tools.bookings import get_booking, ping, search\n\n" + "class GetBookingResponse(TypedDict, total=False):\n" + " booking: Booking\n\n" + "class Booking(TypedDict):\n" + " id: str\n\n" + "def get_booking() -> GetBookingResponse:\n" + ' """Get a booking.\n """\n' + " ...\n\n" + "def ping() -> Any:\n" + ' """Ping.\n """\n' + " ...\n\n" + "def search() -> list[Booking]:\n" + ' """Search.\n """\n' + " ...\n" + ) + + +def test_render_catalog_keeps_class_names_unique_across_modules() -> None: + def record(key: str) -> dict[str, object]: + return {"title": "Record", "type": "object", "properties": {key: {"type": "string"}}} + + tasks, teams = _FakeToolset("tasks"), _FakeToolset("teams") + get_task = _SchemaTool("get_task", description="Get a task.", response=record("count")) + get_team = _SchemaTool("get_team", description="Get a team.", response=record("id")) + catalog = render_catalog(_build([(get_task, tasks), (get_team, teams)])) + assert "def get_task() -> Record:" in catalog + assert "class Record(TypedDict, total=False):\n count: str\n" in catalog + assert "def get_team() -> GetTeamResponse:" in catalog + assert "class GetTeamResponse(TypedDict, total=False):\n id: str\n" in catalog diff --git a/tests/test_dispatcher.py b/tests/test_dispatcher.py index bfc926e..14c19ff 100644 --- a/tests/test_dispatcher.py +++ b/tests/test_dispatcher.py @@ -426,7 +426,7 @@ async def run_async(self, *, args: dict[str, Any], tool_context: Any) -> Any: } -async def test_mcp_tool_with_output_schema_returns_structured_content() -> None: +async def test_mcp_tool_with_output_schema_keeps_the_result_envelope() -> None: seen: list[Any] = [] def after(tool: BaseTool, args: dict[str, Any], tool_context: Any, tool_response: Any) -> Any: @@ -438,7 +438,7 @@ def after(tool: BaseTool, args: dict[str, Any], tool_context: Any, tool_response d = Dispatcher(invocation_context=ctx, registry=_registry(tool), execution_id="e1") # type: ignore[arg-type] result = await d.dispatch("lookup", {}) assert result.ok is True - assert result.value == {"id": "b1"} + assert result.value == _MCP_RESULT assert seen == [_MCP_RESULT] # Callbacks still see ADK's result. # The artifact holds what code got, so reloading it gives the same value. [key] = await ctx.artifact_service.list_artifact_keys(app_name="a", user_id="u", session_id="s") @@ -446,7 +446,7 @@ def after(tool: BaseTool, args: dict[str, Any], tool_context: Any, tool_response app_name="a", user_id="u", session_id="s", filename=key ) assert saved is not None and saved.inline_data is not None - assert json.loads(saved.inline_data.data or b"") == {"id": "b1"} + assert json.loads(saved.inline_data.data or b"") == _MCP_RESULT async def test_mcp_tool_without_output_schema_returns_the_whole_result() -> None: diff --git a/tests/test_integration.py b/tests/test_integration.py index 7c5d99c..8c7debb 100644 --- a/tests/test_integration.py +++ b/tests/test_integration.py @@ -201,7 +201,7 @@ async def test_artifact_helpers_save_list_and_load_json() -> None: ")\n" "print('items', list_artifacts())\n" "loaded = load_artifact(filename='report.json')\n" - "print('loaded', loaded)\n" + "print('loaded', json.loads(loaded['data']))\n" ) result = await _run(tool, ctx, code, call_id="run-ws") @@ -273,11 +273,11 @@ async def test_artifact_helpers_support_binary_content() -> None: "import base64\n" "from tools import load_artifact\n" "blob = load_artifact(filename='blob.bin')\n" - "print(base64.b64decode(blob['bytes']), blob['mime_type'])\n" + "print(blob['kind'], base64.b64decode(blob['data']), blob['mime_type'])\n" ) result = await _run(tool, ctx, load, call_id="run-bin-2") - assert "b'\\x00\\x01\\x02' application/octet-stream" in result["stdout"] + assert "bytes b'\\x00\\x01\\x02' application/octet-stream" in result["stdout"] @pytest.mark.asyncio @@ -314,7 +314,7 @@ async def test_workspace_is_fresh_each_turn_while_artifact_helpers_persist() -> "import os\n" "from tools import load_artifact\n" "print('workspace_has_temp', os.path.exists('temp.txt'))\n" - "print('artifact_value', load_artifact(filename='persist.txt'))\n" + "print('artifact_value', load_artifact(filename='persist.txt')['data'])\n" ) ctx.invocation_id = "inv-fresh-2" result = await _run(tool, ctx, second, call_id="run-fresh-2") diff --git a/tests/test_metadata.py b/tests/test_metadata.py index d7e1107..28a8cb8 100644 --- a/tests/test_metadata.py +++ b/tests/test_metadata.py @@ -30,6 +30,7 @@ def _get_declaration(self) -> genai_types.FunctionDeclaration | None: "properties": {"message": {"type": "string", "description": "What to say."}}, "required": ["message"], }, + response_json_schema={"type": "object", "properties": {"echo": {"type": "string"}}}, ) @@ -144,9 +145,11 @@ def test_empty_package_list_omits_the_tag() -> None: def test_full_tier_includes_signatures_and_docstrings() -> None: block = metadata.render(identity="img", namespaced=_namespaced(), max_chars=10_000) - assert "def tool_0" in block + assert "class Tool0Response(TypedDict, total=False):\n echo: str" in block + assert "def tool_0(message: str) -> Tool0Response:" in block assert "What the model sees" not in block assert "message: str" in block + assert metadata.READ_BEFORE_CALLING not in block def test_names_tier_keeps_import_lines_only() -> None: @@ -156,15 +159,17 @@ def test_names_tier_keeps_import_lines_only() -> None: assert "from tools import tool_0, tool_1, tool_2" in block assert "def tool_0" not in block + assert "Tool0Response" not in block assert "message: str" not in block + assert f"{metadata.HOW_TO_USE} {metadata.READ_BEFORE_CALLING}" in block def test_pointer_tier_is_unconditional() -> None: block = metadata.render(identity="img", namespaced=_namespaced(20), max_chars=10) assert len(block) > 10 # the floor wins over the budget - assert metadata.DISCOVER_TOOLS in block - assert metadata.HOW_TO_USE in block + assert f"{metadata.HOW_TO_USE} {metadata.DISCOVER_TOOLS}" in block + assert metadata.READ_BEFORE_CALLING in metadata.DISCOVER_TOOLS assert "from tools import" not in block diff --git a/tests/test_remote_backend.py b/tests/test_remote_backend.py index 5cf75a9..0392c7f 100644 --- a/tests/test_remote_backend.py +++ b/tests/test_remote_backend.py @@ -12,8 +12,7 @@ from adk_code_mode import RemoteBackend from adk_code_mode.runtime.base import SandboxResult, SandboxSession -from adk_code_mode.runtime.protocol import Frame, ToolResultFrame -from adk_code_mode.runtime.remote import _RemoteSandboxSession +from adk_code_mode.runtime.protocol import Frame class _NoopSession: @@ -81,28 +80,3 @@ async def fake_connect(*, tools_files: Mapping[str, str], workdir_path: str) -> await backend.start(tools_files={}, workdir_path="/tmp", timeout_seconds=None) assert calls == 1 - - -@pytest.mark.asyncio -async def test_send_refuses_a_frame_over_max_message_size() -> None: - class _Ws: - def __init__(self) -> None: - self.sent: list[str] = [] - - async def send(self, message: str) -> None: - self.sent.append(message) - - ws = _Ws() - session = _RemoteSandboxSession( - ws=ws, - workdir_path="/tmp", - max_message_size=1_000, - max_upload_workspace_bytes=0, - max_download_workspace_bytes=0, - ) - - await session.send(ToolResultFrame(id="1", ok=True, value="small")) - with pytest.raises(ValueError, match="exceeds max_message_size"): - await session.send(ToolResultFrame(id="2", ok=True, value="x" * 2_000)) - - assert len(ws.sent) == 1 diff --git a/tests/test_stubs.py b/tests/test_stubs.py index 08a80ca..b50b59e 100644 --- a/tests/test_stubs.py +++ b/tests/test_stubs.py @@ -12,6 +12,7 @@ from google.genai import types as genai_types from mcp.types import Tool as McpToolSpec +from adk_code_mode.tool_result_artifacts import ToolResultArtifactTool from adk_code_mode.tools import namespacing, stubs from adk_code_mode.tools.normaliser import ResolvedTool @@ -531,7 +532,11 @@ def test_mcp_output_schema_becomes_typed_dict_return_type() -> None: rt = stubs.render_tool(_build([(tool, None)])[0]) assert rt.signature_for("catalog") == "def get_bookings() -> GetBookingsResponse:" assert rt.types == ( - "class GetBookingsResponse(TypedDict, total=False):\n" + "class GetBookingsResponse(TypedDict):\n" + " content: list[Any]\n" + " isError: bool\n" + " structuredContent: NotRequired[StructuredContent | None]", + "class StructuredContent(TypedDict, total=False):\n" " bookings: list[BookingsItem]\n" " totals: dict[str, int]\n" " cursor: str | None\n" @@ -548,6 +553,17 @@ def test_mcp_output_schema_becomes_typed_dict_return_type() -> None: assert _load_stub(source)["BookingsItem"].__name__ == "BookingsItem" +def test_wrapped_tool_return_type_includes_the_large_result_pointer() -> None: + tool = ToolResultArtifactTool( + _SchemaTool("count", description="Count records.", response={"type": "integer"}) + ) + rt = stubs.render_tool(_build([(tool, None)])[0]) + assert rt.return_type == "int | ToolResultArtifact" + assert rt.types == ( + "class ToolResultArtifact(TypedDict):\n tool_result_artifact: str\n note: str", + ) + + def test_identical_shapes_share_one_class_across_a_module() -> None: user = {"type": "object", "properties": {"gid": {"type": "string"}}} other = {"type": "object", "properties": {"email": {"type": "string"}}} diff --git a/tests/test_tool.py b/tests/test_tool.py index 0ac2051..99a0256 100644 --- a/tests/test_tool.py +++ b/tests/test_tool.py @@ -75,6 +75,40 @@ def test_description_tells_the_model_the_timeout() -> None: assert "Each call can run for up to" not in unbounded.description +@pytest.mark.parametrize( + ("artifact_tools", "tool_results", "expected"), + [ + ( + True, + True, + " Every file the code writes in the working directory, and every tool's result," + " is saved as an artifact; use `load_artifact` (imported the same way) to read" + " one in a later turn.", + ), + ( + True, + False, + " Every file the code writes in the working directory is saved as an artifact;" + " use `load_artifact` (imported the same way) to read one in a later turn.", + ), + ( + False, + False, + " Every file the code writes in the working directory is saved as an artifact.", + ), + ], +) +def test_description_says_what_is_saved_and_how_to_load_it( + artifact_tools: bool, tool_results: bool, expected: str +) -> None: + tool = ExecuteCodeTool( + backend=FakeRuntime(), + include_artifact_tools=artifact_tools, + save_tool_results_as_artifacts=tool_results, + ) + assert expected in tool.description + + def test_tools_and_backend_properties_expose_the_configured_surface() -> None: schema_tool = _SchemaTool("ping", "Ping.") backend = FakeRuntime() diff --git a/tests/test_tool_result_artifacts.py b/tests/test_tool_result_artifacts.py index e8077da..6b2a995 100644 --- a/tests/test_tool_result_artifacts.py +++ b/tests/test_tool_result_artifacts.py @@ -4,7 +4,6 @@ from __future__ import annotations import json -from dataclasses import dataclass from typing import Any, cast import pytest @@ -19,8 +18,6 @@ TOOL_RESULT_NAME_KEY, ToolResultArtifactTool, ) -from adk_code_mode._artifact_tools import load_artifact -from adk_code_mode._json import json_safe from adk_code_mode.tool import ExecuteCodeTool from adk_code_mode.tools import normaliser from tests._fake_runtime import FakeRuntime @@ -87,12 +84,12 @@ def _get_declaration(self) -> types.FunctionDeclaration: @pytest.mark.asyncio -async def test_result_saved_and_returned_unchanged() -> None: +async def test_small_result_saved_and_returned_unchanged() -> None: ctx = cast(Any, _FakeToolContext()) result = await ToolResultArtifactTool(_ResultTool({"ok": True})).run_async( args={"q": "hi"}, tool_context=cast(ToolContext, ctx) ) - assert result == {"ok": True} + assert result == {"ok": True} # transparent for small results saved = ctx.saved[0] assert saved["custom_metadata"][TOOL_RESULT_METADATA_KEY] == "true" assert saved["filename"] == saved["custom_metadata"][TOOL_RESULT_FILENAME_KEY] @@ -115,55 +112,20 @@ async def test_optional_name_and_description_used() -> None: @pytest.mark.asyncio -async def test_large_result_returned_in_full_and_saved() -> None: - big = {"blob": "x" * 200_000} +async def test_large_result_elided_with_reload_tip() -> None: + big = {"blob": "x" * 100} ctx = cast(Any, _FakeToolContext()) - result = await ToolResultArtifactTool(_ResultTool(big)).run_async( + result = await ToolResultArtifactTool(_ResultTool(big), large_result_threshold=10).run_async( args={"q": "hi"}, tool_context=cast(ToolContext, ctx) ) - assert result == big - assert json.loads(ctx.saved[0]["artifact"].inline_data.data) == big - - -@pytest.mark.parametrize( - "value", - [{"rows": [{"id": 1, "name": "Ada"}], "next": None}, [1, "two", 3.0], "plain text", None], -) -@pytest.mark.asyncio -async def test_saved_result_reloads_as_the_original_value(value: Any) -> None: - ctx = cast(Any, _FakeToolContext()) - await ToolResultArtifactTool(_ResultTool(value)).run_async( - args={"q": "hi"}, tool_context=cast(ToolContext, ctx) + assert set(result) == {"tool_result_artifact", "note"} + assert result["tool_result_artifact"] == ctx.saved[0]["filename"] + assert result["note"] == ( + "The tool result (112 bytes) was saved as an artifact " + "'search_call-1.json' instead of being returned inline. Reload it with " + "load_artifact(filename='search_call-1.json')." ) - part: types.Part = ctx.saved[0]["artifact"] - - async def _load(filename: str, *, version: int | None = None) -> types.Part: - return part - - ctx.load_artifact = _load - assert await load_artifact(filename=ctx.saved[0]["filename"], tool_context=ctx) == value - - -@dataclass -class _Record: - label: str - - -@pytest.mark.asyncio -async def test_saved_result_reloads_as_the_value_sent_to_the_code() -> None: - record = _Record(label="ready") - ctx = cast(Any, _FakeToolContext()) - await ToolResultArtifactTool(_ResultTool(record)).run_async( - args={"q": "hi"}, tool_context=cast(ToolContext, ctx) - ) - part: types.Part = ctx.saved[0]["artifact"] - - async def _load(filename: str, *, version: int | None = None) -> types.Part: - return part - - ctx.load_artifact = _load - loaded = await load_artifact(filename=ctx.saved[0]["filename"], tool_context=ctx) - assert loaded == json_safe(record) == {"label": "ready"} + assert json.loads(ctx.saved[0]["artifact"].inline_data.data) == big # full payload persisted @pytest.mark.asyncio