Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/plans/compose-managed-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ stickered = subagent({
attachments: [cropped],
output_schema: {
type: "object",
properties: { result: { "$ref": "kit://schemas/file/v1" } },
properties: { result: { "$ref": "kit://schemas/file/v1", "x-kit-image-index": 0 } },
required: ["result"],
additionalProperties: false
}
Expand All @@ -91,6 +91,14 @@ The pipeline remains one compose invocation with final-return-only delivery, usi

Extend the currently text-oriented ACP child prompt/output path to retain typed attachments and assistant media. Import native generated media into managed storage. Bind actual emitted media to a file-aware output contract; models must not invent file IDs. Specify single-image binding and reject ambiguous multiple output. Validate shape AND reference existence/access, with explicit contract failure rather than silent string fallback for the new file-aware contract. Parent outputs must survive child close; grants and promotion must preserve session isolation.

### Phase 3 implemented contract

The subagent tool layer accepts optional typed `attachments` on `subagent`, `prompt`, and `fork`, resolving authority from `ToolRequest.session_id`. Native ACP image inputs and generated outputs use managed storage rather than model-created identities. Attachment grants cross working-directory stores explicitly; generated-image parent publication and final access validation complete before the existing success transition. Errors use the existing create cleanup, continuation retry-handle, and fork cleanup paths without adding shared-state instrumentation.

File-aware `output_schema` uses the locally resolved `kit://schemas/file/v1` reference. The supported binding is exactly one root File or one required fixed nested object-property path. By default, Kit requires exactly one distinct native assistant image; an optional caller-fixed `x-kit-image-index` integer 0–7 beside the exact File `$ref` instead selects a distinct image in first-emission order. Kit rejects attempted model binding, validates surrounding JSON and the completed schema, and only constructs omitted surrounding objects when the complete result is valid. Arrays, unions, conditionals, indirect references and multiple bindings are unsupported. Every native image occurrence is independently validated and charged against occurrence, encoded/decoded byte, and aggregate pixel budgets before deduplication. Only equal MIME types and byte-identical image payloads from the current output collapse; visually identical images with different bytes remain distinct and require an explicit index to select one. Every occurrence, including unselected images, must validate and satisfy budgets before selection. Out-of-range indices fail without fallback, and only the selected image is imported/published. Kit does not strip signed metadata or deduplicate perceptually. No model identity, input image, or previous turn participates in that comparison. Ordinary text-only schema fallback is preserved. Native images without a file-aware schema have the explicit `output: { value, files }` surface, not base64 diagnostic updates. The [user guide](../user/compose-and-local-tools.md#attach-files-to-subagents-and-return-native-images) specifies the contract and final-return-only behavior.

The complete read/rotate/crop → built-in ACP subagent → managed output → child close → `return output.result` pipeline has been verified against the canonical OpenRouter `google/gemini-3-pro-image` route. The caller explicitly selected distinct native output index 0; this does not claim that the backend emits only one image. The selected bytes were visually verified to contain the requested Hello Kitty sticker. Eligibility comes from exact-model and concrete-endpoint capability discovery, not a model allowlist. Other harness/provider routes require their own native-output support. Shared TUI presentation remains Phase 4 work.

### Shared TUI presentation

Generalize user-image rendering into reusable media presentation for user attachments, tool results, assistant-generated media, and Markdown image nodes. Share decode/cache/terminal protocols and budgets, and align live updates with history replay. Keep media out of text-only search/previews/logs.
Expand Down
60 changes: 59 additions & 1 deletion docs/user/compose-and-local-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,12 +161,70 @@ The canonical tool result retains typed images. Supported terminal graphics rend

File descriptors reserve `"$kit": "file"` and use schema version 1. They contain an opaque ID, a bounded display name, MIME type, encoded byte count, and image dimensions. Unknown fields, unknown versions, altered metadata, missing objects, and inaccessible IDs fail rather than appearing as successful text-only image delivery. Do not edit descriptors or invent IDs.

Kit stores immutable snapshots under `~/.kit/files/<session-namespace>/` (or `<root>/.kit/files` when HOME is unavailable). A descriptor is returned only after the binary object and directory entries cross the disk durability barrier. Storage failure returns no usable descriptor. Each versioned binary envelope has a bounded metadata header and a digest-verified payload; truncated or corrupted objects are rejected.
Kit stores immutable snapshots under `~/.kit/files/<session-namespace>/` (or `<root>/.kit/files` when HOME is unavailable). A descriptor is returned only after the binary object and directory entries cross the disk durability barrier. Storage failure returns no usable descriptor. Each versioned binary envelope has a bounded metadata header and a digest-verified payload; truncated or corrupted objects are rejected. Imports first complete and sync a private staging object outside session authority, then publish it with an atomic no-replace rename and sync both directories. A failure before publication cannot leave a partial object in the inherited set. A failure after rename may leave a complete object even though no descriptor was returned; Kit does not classify it as garbage.

References survive process restart and source modification or deletion. Authorization comes from the calling session, not from possession of a marker or an arbitrary filesystem path. Copying a descriptor to a fork or another session does **not** grant access. Cross-session grants are not part of this reader.

There is no automatic managed-file garbage collection in this phase. Calls, cancellation, session close, and process exit do not delete these objects. Cancelled or failed imports can leave unreachable objects. Explicit removal of a session's managed-file directory invalidates its references; do not remove retained objects that you need after resume. Finalization can repeat against the same immutable references without importing again. A delivery error states that the compose program already completed and side effects may have occurred; it is not a rollback or an invitation to retry blindly.

## Attach files to subagents and return native images

The hidden `subagent`, `prompt`, and `fork` tools accept optional `attachments: FileReference[]` (at most eight). Pass managed File values explicitly; a file ID, path, URL, or descriptor pasted into prompt text does not attach an image. Kit resolves attachments in the invoking session, checks the existing aggregate image budgets, grants durable copies to the child, and sends native ACP image blocks after the text prompt. The harness must advertise ACP image **input** support. That advertisement does not promise image generation.

Native Kit forks inherit the source session's complete managed-file authority, including valid version-1 objects written by older Kit versions. An empty, truncated, or otherwise malformed historical object blocks inheritance: Kit cannot distinguish an interrupted old import from damage to a previously published object, and transcript absence is not evidence that removal is safe. The source remains unchanged and destination authority is not committed. Restore the object, or investigate and explicitly remove it only if losing its references is acceptable. Kit never automatically skips or classifies these objects as garbage.

Use the exact local schema reference `{"$ref":"kit://schemas/file/v1"}` for a strict native-image output contract:

```text
source = read_file({ path: "source.png" })
edited = subagent({
model: "openrouter:google/gemini-3-pro-image",
prompt: "Edit the attached image and emit exactly one distinct native assistant image.",
attachments: [source],
output_schema: {
type: "object",
properties: { result: { "$ref": "kit://schemas/file/v1" } },
required: ["result"],
additionalProperties: false
}
})
return edited.output.result
```

This example uses the built-in Kit ACP harness and a concrete OpenRouter image-output model. Select an eligible model explicitly; a vision model behind a text-only harness cannot generate native output through that harness. Standard ACP advertises image **input**, not image generation. Actual valid assistant image bytes, not the capability flag or a model name, establish output success.

For the canonical OpenRouter endpoint, selecting a concrete model advertised with image output opts into generation and its additional provider cost. Kit checks the exact model catalogue and its concrete endpoint document, derives output modalities from advertised capabilities, and requires tools support so compose remains available. Automatic routing entries without concrete endpoints retain ordinary behavior. Native requests require providers to support all requested parameters; Kit does not silently remove compose or relax routing to make an incompatible model work. Editing also requires advertised image input. Custom endpoints do not inherit official OpenRouter capability assertions. Discovery failures do not manufacture support, and a required File contract fails if no native image arrives.

This route uses bounded nonstreaming chat completions: at most 24 MiB of raw response, eight images, 8 MiB per image, 16 MiB of decoded image bytes and 32 megapixels in aggregate. Each image also passes the managed-file PNG/JPEG, animation, dimension, pixel and allocation checks. Only inline image bytes are accepted; Kit never fetches provider-generated HTTP or file URLs. Native media does not inject synthetic image-label text into the structured output. Generation has a 300-second attempt timeout and 310-second logical budget, with no automatic retries of ambiguous billable failures. Cancellation remains available. On continuation or replay, historical assistant images stay typed in canonical history and are projected into supported image-input blocks only in the outgoing provider request, after any complete parallel tool-result batch. Delivery quotas apply separately to each assistant item, not cumulatively across history. Before base64 encoding on the Completions fallback route, a separate 64 MiB budget bounds retained assistant-image payloads. This is not a whole-request, user-attachment, or model-context limit. An oversized history fails explicitly; compact history or start a fresh session with selected attachments. The existing text-oriented behavior and tool-image user-message fallback remain unchanged for other routes.

A file-aware schema supports **exactly one** File location: the root, or one fixed object-property path whose properties are required at every level. Arrays, unions, conditional binding, indirect references, multiple locations, and sibling keywords on the File `$ref` other than the optional `x-kit-image-index` annotation are rejected. Kit resolves the File schema locally. By default, the child must emit exactly one **distinct** native assistant image and return only the surrounding JSON fields, omitting the binding field. Kit independently validates every occurrence, including its declared MIME type, actual PNG/JPEG bytes, and pixels. Repeated occurrences collapse to one output only when both MIME type and actual bytes match exactly within this turn. Count, encoded/decoded byte, and aggregate pixel budgets count every occurrence before deduplication. Without an explicit selection index, different image bytes remain ambiguous even if they render identically; model IDs and File references do not determine identity. There is no cross-turn or input/output deduplication. Kit rejects model-written binding fields, including `null` placeholders and invented File IDs. A root binding requires empty text. Empty surrounding text is allowed only when Kit can construct the required object path and the resulting complete value validates; Kit does not invent other required fields or apply schema defaults.

Missing images, multiple distinct images without an explicit index, out-of-range selection indices, capture errors, malformed image bytes, invalid surrounding JSON, failed schema validation, and inaccessible files fail the call explicitly. Tool-result images, thought images, resource links, and textual base64 are not native assistant output. Kit imports real PNG/JPEG bytes under the existing managed-file limits and publishes a durable parent-authorized copy before returning success. Outputs survive child close and different child working directories; unrelated sessions do not acquire access. Failed continuation calls retain the accepted handle generation for retry, as with existing text-only failures.

To deliberately select one output from a backend that can emit multiple distinct images, fix `x-kit-image-index` beside the exact File `$ref` **before** starting the call. The annotation must be an integer from 0 through 7. It indexes distinct validated images in first-emission order, after byte-identical duplicates collapse. The default contract above remains strict: Kit never chooses among distinct outputs unless the caller supplies this annotation.

```text
source = read_file({ path: "source.png" })
edited = subagent({
model: "openrouter:google/gemini-3-pro-image",
prompt: "Add the requested sticker to the attached image and emit native image output.",
attachments: [source],
output_schema: {
type: "object",
properties: {
result: { "$ref": "kit://schemas/file/v1", "x-kit-image-index": 0 }
},
required: ["result"],
additionalProperties: false
}
})
return edited.output.result
```

The root form is also supported: `output_schema: { "$ref": "kit://schemas/file/v1", "x-kit-image-index": 1 }` selects the second distinct image. Kit removes the annotation when expanding the local File schema. The model cannot supply or override the index or binding field. All occurrences—including unselected images—must pass validation and occurrence/byte/pixel budgets before selection. A requested index with no corresponding image fails explicitly, with no fallback. Only the selected image is imported and published; unselected images produce no File descriptors or diagnostic image updates. Kit does not strip signed metadata, compare images perceptually, or treat identical pixels with different PNG/JPEG bytes as duplicates.

Without a file-aware schema, text-only results keep their existing behavior, including ordinary `output_schema` validation with string fallback. When native images accompany such a result, `output` is explicitly `{ value, files }`: `value` is the legacy text/JSON result and `files` contains imported File descriptors. Images are not hidden in diagnostic `updates`, and raw base64 is not included there. Returning `edited.output.files[0]` delivers that image; keeping it intermediate does not. All these tools remain callable only through compose, and final-return-only image delivery is unchanged.

## Make exact file changes with `edit`

`edit` operates on one file path with `op: "add"`, `"edit"`, or `"delete"`. Relative paths are resolved from Kit's working directory. Absolute paths, `..`, and paths through symlinks are accepted, so `edit` can change files outside the root when the Kit process has permission. Paths must be non-empty.
Expand Down
8 changes: 0 additions & 8 deletions fixtures/mock-acp.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,14 +114,6 @@ def prompt(request):
"sessionUpdate": "agent_thought_chunk",
"content": {"type": "text", "text": "internal"},
},
{
"sessionUpdate": "agent_message_chunk",
"content": {
"type": "image",
"data": "aGVsbG8=",
"mimeType": "image/png",
},
},
{
"sessionUpdate": "tool_call",
"toolCallId": "call-1",
Expand Down
Loading
Loading