Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/plans/compose-managed-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,12 @@ return stickered.output.result

Explicit attachment arguments, not concatenating a reference into prompt text, determine image input. A model capable of images behind a text-only harness is still unsupported.

### Phase 2 implemented contract

The chosen explicit export name is `export_file({ file, path })`; its path/status receipt has no File marker. Hidden `image_rotate({ image, degrees })`, `image_crop({ image, aspect_ratio, anchor })`, and `image_resize({ image, width, height, fit })` use the unchanged version-1 session-authorized File descriptor and immutable disk publication. Geometry, all nine anchors, contain/cover/stretch rounding, format/orientation/metadata policy, per-stage allocation limits, and disk-only no-clobber export semantics are specified in [the user guide](../user/compose-and-local-tools.md#transform-images-in-one-compose-program).

The pipeline remains one compose invocation with final-return-only delivery, using the existing foreground/background/replay finalizer and native/user-role provider transport. Export uses OS process permissions rather than inventing a filesystem sandbox; its explicit create-new/partial-output cancellation contract does not overwrite or roll back user paths. No dependency or persistent envelope schema change is intended. Phase 3 subagent contracts and phase 4 presentation remain separate milestones.

### Multimodal subagents

Extend the currently text-oriented ACP child prompt/output path to retain typed attachments and assistant media. Import native generated media into managed storage. Bind actual emitted media to a file-aware output contract; models must not invent file IDs. Specify single-image binding and reject ambiguous multiple output. Validate shape AND reference existence/access, with explicit contract failure rather than silent string fallback for the new file-aware contract. Parent outputs must survive child close; grants and promotion must preserve session isolation.
Expand Down
39 changes: 38 additions & 1 deletion docs/user/compose-and-local-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,47 @@ return { screenshot: image }

Only File references reachable from the final return deliver pixels to the parent model. A reference used only in an intermediate binding does not attach its image. Arrays and nested objects work; repeated references deliver one image, labeled with its first position as an escaped JSON Pointer. Every occurrence must have valid metadata, including duplicates. The whole selection is validated before any image is delivered.

The initial reader supports **nonanimated PNG and JPEG**. It sniffs content rather than trusting the extension, rejects corrupt images and nonregular files, and reads at most 8 MiB. An image can have at most 8,192 pixels on either axis, 16 megapixels, and a 64 MiB decoder allocation. GIF, WebP, animated PNG, SVG, PDF, URLs, and text files are unsupported. Use `shell` for ordinary text reads. Relative paths resolve from Kit's working directory; absolute paths follow the Kit process's filesystem access, not a new project sandbox.
The reader supports **nonanimated PNG and JPEG**. PNG compressed profiles (`iCCP`), compressed text (`zTXt`), and international text (`iTXt`) are rejected before decoding to prevent ancillary metadata expansion; this also rejects uncompressed international text. It sniffs content rather than trusting the extension, rejects corrupt images and nonregular files, and reads at most 8 MiB. An image can have at most 8,192 pixels on either axis, 16 megapixels, and a 64 MiB decoder allocation. GIF, WebP, animated PNG, SVG, PDF, URLs, and text files are unsupported. Use `shell` for ordinary text reads. Relative paths resolve from Kit's working directory; absolute paths follow the Kit process's filesystem access, not a new project sandbox.

Imports preserve the original bytes, metadata, and orientation. Width and height describe the encoded raster. There is no automatic transformation or export, and importing never overwrites the source.

### Transform images in one compose program

`image_rotate`, `image_crop`, and `image_resize` consume authorized File references and create new immutable references. `export_file` explicitly writes a reference's exact bytes to a new local file. All four are hidden callables; **compose remains the only exposed tool**.

```text
source = read_file({ path: "screenshot.jpg" })
rotated = image_rotate({ image: source, degrees: 90 })
cropped = image_crop({
image: rotated,
aspect_ratio: { width: 1, height: 1 },
anchor: "center"
})
thumbnail = image_resize({ image: cropped, width: 256, height: 256, fit: "contain" })
receipt = export_file({ file: thumbnail, path: "thumbnail.png" })
return { thumbnail, receipt }
```

Only `thumbnail` delivers pixels in this example. Returning just `receipt` delivers no image. The export has a dependency on `thumbnail`; source order alone does not sequence independent compose calls. Export and transforms are effectful, including when their return values are unused.

Geometry is evaluated after normalizing EXIF orientation:

- **Rotate:** `degrees` is exactly `90`, `180`, or `270`, clockwise.
- **Crop:** ratio `width` and `height` are positive integers at most 8,192. Take the largest inscribed crop with an integer-rounded aspect ratio: retain one source dimension and floor the other. Reject a dimension rounded to zero. `anchor` is required: `center`, `top_left`, `top`, `top_right`, `left`, `right`, `bottom_left`, `bottom`, or `bottom_right`. Center offsets are floored, leaving an odd extra pixel outside the crop on the right/bottom. The rounded ratio need not be mathematically exact.
- **Resize:** `width`, `height`, and `fit` are required. `contain` preserves the ratio within the requested box, floors the shortened dimension, and adds no padding; zero-rounded dimensions fail. `cover` center-crops using the target's integer-rounded ratio, then resizes exactly to the target dimensions; rounding can cause slight ratio distortion. `stretch` directly resizes to the exact dimensions. All fits permit upscaling and use Triangle filtering. Resizing premultiplies RGB by alpha in floating point before filtering and unpremultiplies afterward, so transparent edges do not darken or bleed invisible colors. Zero-alpha output has zero RGB. Filtering still uses encoded color values, not linear-light or color-managed conversion.

Transforms support nonanimated PNG/JPEG inputs and emit fresh **RGBA8 PNG**, including when the input was JPEG. All eight EXIF orientations, including mirrored ones, are applied before geometry. The decoder either rejects malformed orientation metadata or falls back to identity. Source EXIF, ICC profiles, text, and other ancillary metadata are not copied. Stripping a profile is **not** color-managed conversion. Imports retain their original encoded dimensions and bytes; transformed descriptors describe the new raster. Source paths and existing managed objects are never overwritten.

Transforms enforce the reader's encoded, dimension, pixel, and decoder limits on input and output, plus a **128 MiB resize-scratch limit** and **256 MiB estimated live-pixel-work limit**. The Triangle implementation's intermediate buffer depends on source width × target height, so even narrow images can exceed scratch limits. Floating-point alpha-correct resize buffers are included in the live-work estimate and can reject additional otherwise-valid requests. These are allocation checks, not a total-process RSS guarantee; independent compose operations can run concurrently. Encoded output is capped at 8 MiB. Cancellation is cooperative at stage boundaries and during bounded writes; a running decoder/filter cannot be forcibly interrupted. Failure or cancellation can leave unreachable managed objects but returns no usable new reference.

### Explicit file export

`export_file({ file, path })` returns `{ path, size_bytes, status: "exported" }`, not a File reference. It writes the exact stored bytes without decoding, format conversion, or extension-based rewriting. Relative paths resolve from Kit's working directory. Absolute paths and parent symlinks use ordinary process OS authority, as with `shell` and `edit`; there is no project sandbox or ancestor confinement.

The destination's parent must already exist. Atomic create-new refuses any existing destination, including files, directories, and dangling final symlinks. It never overwrites an imported source. New Unix files are created with mode `0600` (subject to umask). Export bypasses volatile storage fallback: success requires disk writing and successful file sync. It does not promise atomic visibility or crash-durable directory creation.

Cancellation before creation produces no destination. Errors or cancellation after creation retain a potentially partial **or complete** destination and report that path; Kit does not unlink it because another actor could have replaced it. A successful file sync is the commit point, with no cancellation rollback afterward. Retrying the same destination fails until you explicitly deal with the existing file. Do not rerun export blindly after interruption or delivery failure.

### Delivery limits and provider support

A final compose return can select at most 8 distinct images, 16 MiB of encoded image bytes, and 32 megapixels in total. Selection traversal is bounded to 100,000 JSON nodes, depth 64, and 64 reference occurrences, with position labels bounded to 2 KiB each and 4 KiB in total. These limits are separate from the **8 KiB text-output budget**. Large returned JSON spills to a text artifact without hiding the selected image parts or their labels; media bytes do not enter the text artifact.
Expand Down
36 changes: 29 additions & 7 deletions src/managed_files.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ use serde_json::Value;

use crate::resilient_fs as fs;

mod operations;
pub(crate) use operations::{Anchor, AspectRatio, Fit, Transform};

const MAX_FILE_BYTES: u64 = 8 * 1024 * 1024;
const MAX_HEADER_BYTES: usize = 4096;
const MAX_DIMENSION: u32 = 8192;
Expand Down Expand Up @@ -115,6 +118,19 @@ impl FileStore {
"file name must be nonempty UTF-8, at most 255 bytes, without control characters",
)?
.to_owned();
self.publish(session, bytes, name, mime_type, image, cancellation)
}

fn publish(
&self,
session: &str,
bytes: Vec<u8>,
name: String,
mime_type: String,
image: ImageDimensions,
cancellation: Option<&TurnCancellation>,
) -> Result<FileReference> {
check_cancelled(cancellation)?;
let mut random = [0_u8; 32];
getrandom::fill(&mut random).map_err(display)?;
let id = format!("file_{}", blake3::Hash::from_bytes(random).to_hex());
Expand Down Expand Up @@ -396,6 +412,17 @@ impl Selection {

fn inspect_image(bytes: &[u8]) -> Result<(String, ImageDimensions)> {
let format = image::guess_format(bytes).map_err(display)?;
let decoder = bounded_decoder(bytes, format)?;
let (width, height) = decoder.dimensions();
// Validate pixels without changing the original bytes or dimensions.
DynamicImage::from_decoder(decoder).map_err(display)?;
Ok((
format.to_mime_type().into(),
ImageDimensions { width, height },
))
}

fn bounded_decoder(bytes: &[u8], format: ImageFormat) -> Result<Box<dyn ImageDecoder + '_>> {
if !matches!(format, ImageFormat::Png | ImageFormat::Jpeg) {
return Err("read_file supports only nonanimated PNG and JPEG images".into());
}
Expand All @@ -404,6 +431,7 @@ fn inspect_image(bytes: &[u8]) -> Result<(String, ImageDimensions)> {
limits.max_image_height = Some(MAX_DIMENSION);
limits.max_alloc = Some(MAX_DECODE_BYTES);
let decoder: Box<dyn ImageDecoder> = if format == ImageFormat::Png {
operations::check_png_metadata(bytes)?;
let decoder = image::codecs::png::PngDecoder::with_limits(Cursor::new(bytes), limits)
.map_err(display)?;
if decoder.is_apng().map_err(display)? {
Expand All @@ -423,13 +451,7 @@ fn inspect_image(bytes: &[u8]) -> Result<(String, ImageDimensions)> {
{
return Err("image exceeds decoded pixel or allocation budget".into());
}
// Fully decode to reject corrupt payloads before publication, then discard
// pixels. Preserve source bytes, EXIF orientation and metadata unchanged.
DynamicImage::from_decoder(decoder).map_err(display)?;
Ok((
format.to_mime_type().into(),
ImageDimensions { width, height },
))
Ok(decoder)
}

fn valid_name(name: &str) -> bool {
Expand Down
Loading
Loading