diff --git a/docs/plans/compose-managed-files.md b/docs/plans/compose-managed-files.md index a9fe800..813fac4 100644 --- a/docs/plans/compose-managed-files.md +++ b/docs/plans/compose-managed-files.md @@ -81,6 +81,12 @@ return stickered.output.result Explicit attachment arguments, not concatenating a reference into prompt text, determine image input. A model capable of images behind a text-only harness is still unsupported. +### Phase 2 implemented contract + +The chosen explicit export name is `export_file({ file, path })`; its path/status receipt has no File marker. Hidden `image_rotate({ image, degrees })`, `image_crop({ image, aspect_ratio, anchor })`, and `image_resize({ image, width, height, fit })` use the unchanged version-1 session-authorized File descriptor and immutable disk publication. Geometry, all nine anchors, contain/cover/stretch rounding, format/orientation/metadata policy, per-stage allocation limits, and disk-only no-clobber export semantics are specified in [the user guide](../user/compose-and-local-tools.md#transform-images-in-one-compose-program). + +The pipeline remains one compose invocation with final-return-only delivery, using the existing foreground/background/replay finalizer and native/user-role provider transport. Export uses OS process permissions rather than inventing a filesystem sandbox; its explicit create-new/partial-output cancellation contract does not overwrite or roll back user paths. No dependency or persistent envelope schema change is intended. Phase 3 subagent contracts and phase 4 presentation remain separate milestones. + ### Multimodal subagents Extend the currently text-oriented ACP child prompt/output path to retain typed attachments and assistant media. Import native generated media into managed storage. Bind actual emitted media to a file-aware output contract; models must not invent file IDs. Specify single-image binding and reject ambiguous multiple output. Validate shape AND reference existence/access, with explicit contract failure rather than silent string fallback for the new file-aware contract. Parent outputs must survive child close; grants and promotion must preserve session isolation. diff --git a/docs/user/compose-and-local-tools.md b/docs/user/compose-and-local-tools.md index 8939540..5dec819 100644 --- a/docs/user/compose-and-local-tools.md +++ b/docs/user/compose-and-local-tools.md @@ -108,10 +108,47 @@ return { screenshot: image } Only File references reachable from the final return deliver pixels to the parent model. A reference used only in an intermediate binding does not attach its image. Arrays and nested objects work; repeated references deliver one image, labeled with its first position as an escaped JSON Pointer. Every occurrence must have valid metadata, including duplicates. The whole selection is validated before any image is delivered. -The initial reader supports **nonanimated PNG and JPEG**. It sniffs content rather than trusting the extension, rejects corrupt images and nonregular files, and reads at most 8 MiB. An image can have at most 8,192 pixels on either axis, 16 megapixels, and a 64 MiB decoder allocation. GIF, WebP, animated PNG, SVG, PDF, URLs, and text files are unsupported. Use `shell` for ordinary text reads. Relative paths resolve from Kit's working directory; absolute paths follow the Kit process's filesystem access, not a new project sandbox. +The reader supports **nonanimated PNG and JPEG**. PNG compressed profiles (`iCCP`), compressed text (`zTXt`), and international text (`iTXt`) are rejected before decoding to prevent ancillary metadata expansion; this also rejects uncompressed international text. It sniffs content rather than trusting the extension, rejects corrupt images and nonregular files, and reads at most 8 MiB. An image can have at most 8,192 pixels on either axis, 16 megapixels, and a 64 MiB decoder allocation. GIF, WebP, animated PNG, SVG, PDF, URLs, and text files are unsupported. Use `shell` for ordinary text reads. Relative paths resolve from Kit's working directory; absolute paths follow the Kit process's filesystem access, not a new project sandbox. Imports preserve the original bytes, metadata, and orientation. Width and height describe the encoded raster. There is no automatic transformation or export, and importing never overwrites the source. +### Transform images in one compose program + +`image_rotate`, `image_crop`, and `image_resize` consume authorized File references and create new immutable references. `export_file` explicitly writes a reference's exact bytes to a new local file. All four are hidden callables; **compose remains the only exposed tool**. + +```text +source = read_file({ path: "screenshot.jpg" }) +rotated = image_rotate({ image: source, degrees: 90 }) +cropped = image_crop({ + image: rotated, + aspect_ratio: { width: 1, height: 1 }, + anchor: "center" +}) +thumbnail = image_resize({ image: cropped, width: 256, height: 256, fit: "contain" }) +receipt = export_file({ file: thumbnail, path: "thumbnail.png" }) +return { thumbnail, receipt } +``` + +Only `thumbnail` delivers pixels in this example. Returning just `receipt` delivers no image. The export has a dependency on `thumbnail`; source order alone does not sequence independent compose calls. Export and transforms are effectful, including when their return values are unused. + +Geometry is evaluated after normalizing EXIF orientation: + +- **Rotate:** `degrees` is exactly `90`, `180`, or `270`, clockwise. +- **Crop:** ratio `width` and `height` are positive integers at most 8,192. Take the largest inscribed crop with an integer-rounded aspect ratio: retain one source dimension and floor the other. Reject a dimension rounded to zero. `anchor` is required: `center`, `top_left`, `top`, `top_right`, `left`, `right`, `bottom_left`, `bottom`, or `bottom_right`. Center offsets are floored, leaving an odd extra pixel outside the crop on the right/bottom. The rounded ratio need not be mathematically exact. +- **Resize:** `width`, `height`, and `fit` are required. `contain` preserves the ratio within the requested box, floors the shortened dimension, and adds no padding; zero-rounded dimensions fail. `cover` center-crops using the target's integer-rounded ratio, then resizes exactly to the target dimensions; rounding can cause slight ratio distortion. `stretch` directly resizes to the exact dimensions. All fits permit upscaling and use Triangle filtering. Resizing premultiplies RGB by alpha in floating point before filtering and unpremultiplies afterward, so transparent edges do not darken or bleed invisible colors. Zero-alpha output has zero RGB. Filtering still uses encoded color values, not linear-light or color-managed conversion. + +Transforms support nonanimated PNG/JPEG inputs and emit fresh **RGBA8 PNG**, including when the input was JPEG. All eight EXIF orientations, including mirrored ones, are applied before geometry. The decoder either rejects malformed orientation metadata or falls back to identity. Source EXIF, ICC profiles, text, and other ancillary metadata are not copied. Stripping a profile is **not** color-managed conversion. Imports retain their original encoded dimensions and bytes; transformed descriptors describe the new raster. Source paths and existing managed objects are never overwritten. + +Transforms enforce the reader's encoded, dimension, pixel, and decoder limits on input and output, plus a **128 MiB resize-scratch limit** and **256 MiB estimated live-pixel-work limit**. The Triangle implementation's intermediate buffer depends on source width × target height, so even narrow images can exceed scratch limits. Floating-point alpha-correct resize buffers are included in the live-work estimate and can reject additional otherwise-valid requests. These are allocation checks, not a total-process RSS guarantee; independent compose operations can run concurrently. Encoded output is capped at 8 MiB. Cancellation is cooperative at stage boundaries and during bounded writes; a running decoder/filter cannot be forcibly interrupted. Failure or cancellation can leave unreachable managed objects but returns no usable new reference. + +### Explicit file export + +`export_file({ file, path })` returns `{ path, size_bytes, status: "exported" }`, not a File reference. It writes the exact stored bytes without decoding, format conversion, or extension-based rewriting. Relative paths resolve from Kit's working directory. Absolute paths and parent symlinks use ordinary process OS authority, as with `shell` and `edit`; there is no project sandbox or ancestor confinement. + +The destination's parent must already exist. Atomic create-new refuses any existing destination, including files, directories, and dangling final symlinks. It never overwrites an imported source. New Unix files are created with mode `0600` (subject to umask). Export bypasses volatile storage fallback: success requires disk writing and successful file sync. It does not promise atomic visibility or crash-durable directory creation. + +Cancellation before creation produces no destination. Errors or cancellation after creation retain a potentially partial **or complete** destination and report that path; Kit does not unlink it because another actor could have replaced it. A successful file sync is the commit point, with no cancellation rollback afterward. Retrying the same destination fails until you explicitly deal with the existing file. Do not rerun export blindly after interruption or delivery failure. + ### Delivery limits and provider support A final compose return can select at most 8 distinct images, 16 MiB of encoded image bytes, and 32 megapixels in total. Selection traversal is bounded to 100,000 JSON nodes, depth 64, and 64 reference occurrences, with position labels bounded to 2 KiB each and 4 KiB in total. These limits are separate from the **8 KiB text-output budget**. Large returned JSON spills to a text artifact without hiding the selected image parts or their labels; media bytes do not enter the text artifact. diff --git a/src/managed_files.rs b/src/managed_files.rs index 92cb7a9..e24ce5d 100644 --- a/src/managed_files.rs +++ b/src/managed_files.rs @@ -13,6 +13,9 @@ use serde_json::Value; use crate::resilient_fs as fs; +mod operations; +pub(crate) use operations::{Anchor, AspectRatio, Fit, Transform}; + const MAX_FILE_BYTES: u64 = 8 * 1024 * 1024; const MAX_HEADER_BYTES: usize = 4096; const MAX_DIMENSION: u32 = 8192; @@ -115,6 +118,19 @@ impl FileStore { "file name must be nonempty UTF-8, at most 255 bytes, without control characters", )? .to_owned(); + self.publish(session, bytes, name, mime_type, image, cancellation) + } + + fn publish( + &self, + session: &str, + bytes: Vec, + name: String, + mime_type: String, + image: ImageDimensions, + cancellation: Option<&TurnCancellation>, + ) -> Result { + check_cancelled(cancellation)?; let mut random = [0_u8; 32]; getrandom::fill(&mut random).map_err(display)?; let id = format!("file_{}", blake3::Hash::from_bytes(random).to_hex()); @@ -396,6 +412,17 @@ impl Selection { fn inspect_image(bytes: &[u8]) -> Result<(String, ImageDimensions)> { let format = image::guess_format(bytes).map_err(display)?; + let decoder = bounded_decoder(bytes, format)?; + let (width, height) = decoder.dimensions(); + // Validate pixels without changing the original bytes or dimensions. + DynamicImage::from_decoder(decoder).map_err(display)?; + Ok(( + format.to_mime_type().into(), + ImageDimensions { width, height }, + )) +} + +fn bounded_decoder(bytes: &[u8], format: ImageFormat) -> Result> { if !matches!(format, ImageFormat::Png | ImageFormat::Jpeg) { return Err("read_file supports only nonanimated PNG and JPEG images".into()); } @@ -404,6 +431,7 @@ fn inspect_image(bytes: &[u8]) -> Result<(String, ImageDimensions)> { limits.max_image_height = Some(MAX_DIMENSION); limits.max_alloc = Some(MAX_DECODE_BYTES); let decoder: Box = if format == ImageFormat::Png { + operations::check_png_metadata(bytes)?; let decoder = image::codecs::png::PngDecoder::with_limits(Cursor::new(bytes), limits) .map_err(display)?; if decoder.is_apng().map_err(display)? { @@ -423,13 +451,7 @@ fn inspect_image(bytes: &[u8]) -> Result<(String, ImageDimensions)> { { return Err("image exceeds decoded pixel or allocation budget".into()); } - // Fully decode to reject corrupt payloads before publication, then discard - // pixels. Preserve source bytes, EXIF orientation and metadata unchanged. - DynamicImage::from_decoder(decoder).map_err(display)?; - Ok(( - format.to_mime_type().into(), - ImageDimensions { width, height }, - )) + Ok(decoder) } fn valid_name(name: &str) -> bool { diff --git a/src/managed_files/operations.rs b/src/managed_files/operations.rs new file mode 100644 index 0000000..ed45dbf --- /dev/null +++ b/src/managed_files/operations.rs @@ -0,0 +1,372 @@ +//! Geometry uses encoded channel values, not a color-managed conversion. +//! Allocation audit (image 0.25.10 / png 0.18): decode is limited to 64 MiB; +//! compressed PNG metadata is rejected before decoder construction because its +//! expansion is not covered by pixel limits. Other metadata is bounded by the +//! 8 MiB input. RGBA conversion/orientation hold at most two 64 MiB pixel buffers. +//! Resize converts to premultiplied RGBA32F, holding float source + destination +//! and the RGBA32F vertical intermediate and a small weight vector. Conversion +//! peaks at 20 bytes/pixel (RGBA8 + RGBA32F). The intermediate has a separate +//! 128 MiB ceiling; the float source is dropped before output conversion. +//! PNG's level compressor buffers the entire compressed stream before writing; +//! allow twice the raw scanline size for its Vec capacity, plus 8 MiB capped +//! output and 8 MiB codec/row overhead. These are live-work estimates, not RSS +//! guarantees. Encoded input and old geometry buffers are dropped before encode. +use super::*; +use image::{ImageEncoder as _, RgbaImage, imageops}; +use std::io; + +const MAX_SCRATCH_BYTES: u64 = 128 * 1024 * 1024; +const MAX_LIVE_BYTES: u64 = 256 * 1024 * 1024; +const CODEC_HEADROOM: u64 = 8 * 1024 * 1024; + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(deny_unknown_fields, rename_all = "snake_case")] +pub(crate) enum Transform { + Rotate { + degrees: u32, + }, + Crop { + aspect_ratio: AspectRatio, + anchor: Anchor, + }, + Resize { + width: u32, + height: u32, + fit: Fit, + }, +} + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(deny_unknown_fields)] +pub(crate) struct AspectRatio { + pub(crate) width: u32, + pub(crate) height: u32, +} + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(rename_all = "snake_case")] +pub(crate) enum Anchor { + Center, + TopLeft, + Top, + TopRight, + Left, + Right, + BottomLeft, + Bottom, + BottomRight, +} + +#[derive(Clone, Copy, Debug, Deserialize)] +#[serde(rename_all = "snake_case")] +pub(crate) enum Fit { + Contain, + Cover, + Stretch, +} + +impl FileStore { + pub(crate) fn transform( + &self, + session: &str, + selected: &FileReference, + transform: Transform, + cancellation: Option<&TurnCancellation>, + ) -> Result { + check_cancelled(cancellation)?; + let bytes = self.resolve(session, selected)?; + check_cancelled(cancellation)?; + let mut decoder = bounded_decoder(&bytes, image::guess_format(&bytes).map_err(display)?)?; + let orientation = decoder.orientation().map_err(display)?; + check_cancelled(cancellation)?; + let decoded = DynamicImage::from_decoder(decoder).map_err(display)?; + check_cancelled(cancellation)?; + drop(bytes); + // Convert before orientation to bound copies even for 16-bit input. + let mut normalized = DynamicImage::ImageRgba8(decoded.into_rgba8()); + check_cancelled(cancellation)?; + normalized.apply_orientation(orientation); + check_cancelled(cancellation)?; + let source = normalized.into_rgba8(); + let result = geometry(source, transform, cancellation)?; + check_cancelled(cancellation)?; + let (width, height) = result.dimensions(); + let scanlines = (u64::from(width) * 4 + 1) * u64::from(height); + if u64::from(width) * u64::from(height) * 4 + + 2 * scanlines + + MAX_FILE_BYTES + + CODEC_HEADROOM + > MAX_LIVE_BYTES + { + return Err("PNG encode exceeds live pixel work budget".into()); + } + let mut output = CappedWriter { + bytes: Vec::new(), + cancellation, + }; + // Explicit level compression avoids the fast codec's fallback keeping + // both compressed and uncompressed streams alive simultaneously. + image::codecs::png::PngEncoder::new_with_quality( + &mut output, + image::codecs::png::CompressionType::Level(6), + image::codecs::png::FilterType::Adaptive, + ) + .write_image( + result.as_raw(), + width, + height, + image::ExtendedColorType::Rgba8, + ) + .map_err(display)?; + check_cancelled(cancellation)?; + drop(result); + self.publish( + session, + output.bytes, + "transformed.png".into(), + "image/png".into(), + ImageDimensions { width, height }, + cancellation, + ) + } + + /// Disk-only, create-new export. Successful sync is the commit point: never + /// report cancellation after it. Failed files are retained, since unlinking + /// a pathname after a write failure could remove somebody else's replacement. + pub(crate) fn export( + &self, + session: &str, + selected: &FileReference, + path: &Path, + cancellation: Option<&TurnCancellation>, + ) -> Result { + check_cancelled(cancellation)?; + let bytes = self.resolve(session, selected)?; + check_cancelled(cancellation)?; + let mut options = std::fs::OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt as _; + options.mode(0o600); + } + let mut file = options + .open(path) + .map_err(|error| format!("export {}: {error}", path.display()))?; + let write = || -> Result<()> { + for chunk in bytes.chunks(64 * 1024) { + check_cancelled(cancellation)?; + file.write_all(chunk).map_err(display)?; + } + check_cancelled(cancellation)?; + file.sync_all().map_err(display) + }; + let mut write = write; + write().map_err(|error| { + format!( + "export {}: {error}; partial-or-complete file retained at this path", + path.display() + ) + })?; + Ok(bytes.len() as u64) + } +} + +fn dimensions(width: u32, height: u32) -> Result<()> { + if width == 0 + || height == 0 + || width > MAX_DIMENSION + || height > MAX_DIMENSION + || u64::from(width) * u64::from(height) > MAX_PIXELS + { + return Err("image dimensions exceed nonzero 8192 / 16 megapixel limits".into()); + } + Ok(()) +} + +fn inscribed(width: u32, height: u32, ratio: AspectRatio) -> Result<(u32, u32)> { + if ratio.width == 0 + || ratio.height == 0 + || ratio.width > MAX_DIMENSION + || ratio.height > MAX_DIMENSION + { + return Err("aspect ratio components must be positive integers at most 8192".into()); + } + let (w, h) = if u64::from(width) * u64::from(ratio.height) + > u64::from(height) * u64::from(ratio.width) + { + ( + (u64::from(height) * u64::from(ratio.width) / u64::from(ratio.height)) as u32, + height, + ) + } else { + ( + width, + (u64::from(width) * u64::from(ratio.height) / u64::from(ratio.width)) as u32, + ) + }; + dimensions(w, h)?; + Ok((w, h)) +} + +fn crop(source: RgbaImage, ratio: AspectRatio, anchor: Anchor) -> Result { + let (width, height) = inscribed(source.width(), source.height(), ratio)?; + let dx = source.width() - width; + let dy = source.height() - height; + let x = match anchor { + Anchor::TopLeft | Anchor::Left | Anchor::BottomLeft => 0, + Anchor::TopRight | Anchor::Right | Anchor::BottomRight => dx, + _ => dx / 2, + }; + let y = match anchor { + Anchor::TopLeft | Anchor::Top | Anchor::TopRight => 0, + Anchor::BottomLeft | Anchor::Bottom | Anchor::BottomRight => dy, + _ => dy / 2, + }; + Ok(imageops::crop_imm(&source, x, y, width, height).to_image()) +} + +fn geometry( + source: RgbaImage, + transform: Transform, + cancellation: Option<&TurnCancellation>, +) -> Result { + check_cancelled(cancellation)?; + let result = match transform { + Transform::Rotate { degrees } => match degrees { + 90 => imageops::rotate90(&source), + 180 => imageops::rotate180(&source), + 270 => imageops::rotate270(&source), + _ => return Err("rotation must be 90, 180, or 270 degrees clockwise".into()), + }, + Transform::Crop { + aspect_ratio, + anchor, + } => crop(source, aspect_ratio, anchor)?, + Transform::Resize { width, height, fit } => { + dimensions(width, height)?; + let (source, width, height) = match fit { + Fit::Stretch => (source, width, height), + Fit::Cover => { + let source = crop(source, AspectRatio { width, height }, Anchor::Center)?; + check_cancelled(cancellation)?; + (source, width, height) + } + Fit::Contain => { + let (width, height) = inscribed( + width, + height, + AspectRatio { + width: source.width(), + height: source.height(), + }, + )?; + (source, width, height) + } + }; + let scratch = u64::from(source.width()) * u64::from(height) * 16; + // Every dimension is already bounded by 8192, so these u64 + // products cannot overflow. Include both float conversion peaks + // and all simultaneously live resize buffers before allocating. + let source_pixels = u64::from(source.width()) * u64::from(source.height()); + let target_pixels = u64::from(width) * u64::from(height); + let live = (source_pixels + target_pixels) * 16 + scratch + CODEC_HEADROOM; + if scratch > MAX_SCRATCH_BYTES + || live > MAX_LIVE_BYTES + || source_pixels * 20 + CODEC_HEADROOM > MAX_LIVE_BYTES + || target_pixels * 20 + CODEC_HEADROOM > MAX_LIVE_BYTES + { + return Err("resize exceeds scratch or live pixel work budget".into()); + } + check_cancelled(cancellation)?; + let mut source = DynamicImage::ImageRgba8(source).into_rgba32f(); + for row in source.rows_mut() { + check_cancelled(cancellation)?; + for pixel in row { + let alpha = pixel[3]; + for channel in &mut pixel.0[..3] { + *channel *= alpha; + } + } + } + check_cancelled(cancellation)?; + let mut resized = + imageops::resize(&source, width, height, imageops::FilterType::Triangle); + drop(source); + check_cancelled(cancellation)?; + for row in resized.rows_mut() { + check_cancelled(cancellation)?; + for pixel in row { + let alpha = pixel[3]; + for channel in &mut pixel.0[..3] { + *channel = if alpha > 0.0 { + (*channel / alpha).clamp(0.0, 1.0) + } else { + 0.0 + }; + } + } + } + check_cancelled(cancellation)?; + let mut result = DynamicImage::ImageRgba32F(resized).into_rgba8(); + for row in result.rows_mut() { + check_cancelled(cancellation)?; + for pixel in row { + if pixel[3] == 0 { + pixel.0[..3].fill(0); + } + } + } + result + } + }; + check_cancelled(cancellation)?; + Ok(result) +} + +struct CappedWriter<'a> { + bytes: Vec, + cancellation: Option<&'a TurnCancellation>, +} +impl io::Write for CappedWriter<'_> { + fn write(&mut self, bytes: &[u8]) -> io::Result { + check_cancelled(self.cancellation).map_err(io::Error::other)?; + if bytes.len() as u64 > MAX_FILE_BYTES - self.bytes.len() as u64 { + return Err(io::Error::other( + "transformed PNG exceeds 8 MiB output limit", + )); + } + self.bytes.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> io::Result<()> { + check_cancelled(self.cancellation).map_err(io::Error::other) + } +} + +/// Scan chunk envelopes without decompressing metadata. Reject compressed text +/// and profiles rather than relying on codecs' nonuniform metadata limits. +/// Import still preserves accepted ancillary chunks byte-for-byte. +pub(super) fn check_png_metadata(bytes: &[u8]) -> Result<()> { + let mut offset = 8_usize; + while offset < bytes.len() { + let header = bytes.get(offset..offset + 8).ok_or("truncated PNG chunk")?; + let length = u32::from_be_bytes([header[0], header[1], header[2], header[3]]) as usize; + let end = offset + .checked_add(12) + .and_then(|n| n.checked_add(length)) + .ok_or("invalid PNG chunk length")?; + if end > bytes.len() { + return Err("truncated PNG chunk".into()); + } + if matches!(&header[4..8], b"iCCP" | b"zTXt" | b"iTXt") { + return Err("compressed or international PNG metadata is not supported".into()); + } + // The decoder stops at IEND; preserve opaque trailing import bytes. + if &header[4..8] == b"IEND" { + break; + } + offset = end; + } + Ok(()) +} diff --git a/src/managed_files/tests.rs b/src/managed_files/tests.rs index 8394c93..d9545a1 100644 --- a/src/managed_files/tests.rs +++ b/src/managed_files/tests.rs @@ -556,3 +556,5 @@ fn fresh_process_resolve_child() { } mod faults; + +mod operations; diff --git a/src/managed_files/tests/operations.rs b/src/managed_files/tests/operations.rs new file mode 100644 index 0000000..68eed7b --- /dev/null +++ b/src/managed_files/tests/operations.rs @@ -0,0 +1,676 @@ +use super::*; +use image::{ImageEncoder as _, Rgba, RgbaImage}; + +fn pixels(width: u32, height: u32) -> RgbaImage { + RgbaImage::from_fn(width, height, |x, y| { + Rgba([(y * width + x + 1) as u8, 0, 0, 255]) + }) +} + +fn import_pixels(f: &Fixture, image: &RgbaImage) -> FileReference { + let path = f.dir.path().join("pixels.png"); + image.save(&path).unwrap(); + f.store.import("session", &path, None).unwrap() +} + +fn transformed(f: &Fixture, file: &FileReference, op: Transform) -> (FileReference, RgbaImage) { + let result = f.store.transform("session", file, op, None).unwrap(); + let bytes = f.store.resolve("session", &result).unwrap(); + ( + result, + image::load_from_memory(&bytes).unwrap().into_rgba8(), + ) +} + +fn reds(image: &RgbaImage) -> Vec { + image.pixels().map(|p| p[0]).collect() +} + +#[test] +fn resize_preserves_transparent_edge_colors() { + let f = Fixture::new(); + for fit in [Fit::Contain, Fit::Cover, Fit::Stretch] { + for invisible in [[0, 0, 0, 0], [255, 0, 0, 0]] { + let source = RgbaImage::from_fn(2, 2, |x, _| { + Rgba(if x == 0 { + [255, 255, 255, 255] + } else { + invisible + }) + }); + let file = import_pixels(&f, &source); + let (_, result) = transformed( + &f, + &file, + Transform::Resize { + width: 1, + height: 1, + fit, + }, + ); + // Half the coverage, unchanged visible white. Transparent RGB must + // neither darken the edge nor bleed its invisible red into it. + assert_eq!(result.get_pixel(0, 0).0, [255, 255, 255, 128], "{fit:?}"); + } + for (pixel, expected) in [ + ([128, 64, 32, 1], [128, 64, 32, 1]), + ([255, 0, 255, 0], [0, 0, 0, 0]), + ] { + let source = RgbaImage::from_pixel(2, 2, Rgba(pixel)); + let file = import_pixels(&f, &source); + let (_, result) = transformed( + &f, + &file, + Transform::Resize { + width: 1, + height: 1, + fit, + }, + ); + assert_eq!(result.get_pixel(0, 0).0, expected, "{fit:?}"); + } + } +} + +#[test] +fn clockwise_rotations_publish_new_immutable_authorized_pngs() { + let f = Fixture::new(); + let file = import_pixels(&f, &pixels(3, 2)); + let original = f.store.resolve("session", &file).unwrap(); + for (degrees, expected, dims) in [ + (90, vec![4, 1, 5, 2, 6, 3], (2, 3)), + (180, vec![6, 5, 4, 3, 2, 1], (3, 2)), + (270, vec![3, 6, 2, 5, 1, 4], (2, 3)), + ] { + let (result, image) = transformed(&f, &file, Transform::Rotate { degrees }); + assert_eq!(reds(&image), expected); + assert_eq!(image.dimensions(), dims); + assert_ne!(result.id, file.id); + assert_eq!(result.mime_type, "image/png"); + assert_eq!( + image::guess_format(&f.store.resolve("session", &result).unwrap()).unwrap(), + ImageFormat::Png + ); + } + assert_eq!(f.store.resolve("session", &file).unwrap(), original); + let op = Transform::Rotate { degrees: 90 }; + assert!(f.store.transform("other", &file, op, None).is_err()); + let mut forged = file.clone(); + forged.name = "forged.png".into(); + assert!(f.store.transform("session", &forged, op, None).is_err()); + assert!( + f.store + .export("other", &file, &f.dir.path().join("unauthorized"), None) + .is_err() + ); + assert!( + f.store + .export("session", &forged, &f.dir.path().join("forged"), None) + .is_err() + ); +} + +#[test] +fn nine_anchors_and_odd_center_rounding() { + let f = Fixture::new(); + let wide = import_pixels(&f, &pixels(5, 2)); + let tall = import_pixels(&f, &pixels(2, 5)); + for (anchor, x, y) in [ + (Anchor::TopLeft, 0, 0), + (Anchor::Top, 1, 0), + (Anchor::TopRight, 3, 0), + (Anchor::Left, 0, 1), + (Anchor::Center, 1, 1), + (Anchor::Right, 3, 1), + (Anchor::BottomLeft, 0, 3), + (Anchor::Bottom, 1, 3), + (Anchor::BottomRight, 3, 3), + ] { + let op = Transform::Crop { + aspect_ratio: AspectRatio { + width: 1, + height: 1, + }, + anchor, + }; + let (_, image) = transformed(&f, &wide, op); + assert_eq!(reds(&image), vec![1 + x, 2 + x, 6 + x, 7 + x]); + let (_, image) = transformed(&f, &tall, op); + assert_eq!( + reds(&image), + vec![1 + 2 * y, 2 + 2 * y, 3 + 2 * y, 4 + 2 * y] + ); + } + let file = import_pixels(&f, &pixels(7, 5)); + let (_, image) = transformed( + &f, + &file, + Transform::Crop { + aspect_ratio: AspectRatio { + width: 2, + height: 3, + }, + anchor: Anchor::Center, + }, + ); + assert_eq!(image.dimensions(), (3, 5)); + assert_eq!(image.get_pixel(0, 0)[0], 3); +} + +fn exif(orientation: u16) -> Vec { + let mut data = b"II\x2a\0\x08\0\0\0\x01\0\x12\x01\x03\0\x01\0\0\0".to_vec(); + data.extend(orientation.to_le_bytes()); + data.extend([0; 6]); + data +} + +#[test] +fn all_eight_exif_orientations_are_normalized_before_geometry() { + let f = Fixture::new(); + let image = pixels(3, 2); + let expected = [ + vec![1, 2, 3, 4, 5, 6], + vec![3, 2, 1, 6, 5, 4], + vec![6, 5, 4, 3, 2, 1], + vec![4, 5, 6, 1, 2, 3], + vec![1, 4, 2, 5, 3, 6], + vec![4, 1, 5, 2, 6, 3], + vec![6, 3, 5, 2, 4, 1], + vec![3, 6, 2, 5, 1, 4], + ]; + for orientation in 1..=8 { + let mut bytes = Vec::new(); + let mut encoder = image::codecs::png::PngEncoder::new(&mut bytes); + encoder.set_exif_metadata(exif(orientation)).unwrap(); + encoder + .write_image(image.as_raw(), 3, 2, image::ExtendedColorType::Rgba8) + .unwrap(); + let path = f.dir.path().join("oriented.png"); + disk::write(&path, &bytes).unwrap(); + let file = f.store.import("session", &path, None).unwrap(); + assert_eq!((file.image.width, file.image.height), (3, 2)); + assert_eq!(f.store.resolve("session", &file).unwrap(), bytes); + let (result, actual) = transformed(&f, &file, Transform::Rotate { degrees: 180 }); + let mut want = expected[orientation as usize - 1].clone(); + want.reverse(); + assert_eq!(reds(&actual), want, "EXIF {orientation}"); + assert_eq!( + actual.dimensions(), + if orientation >= 5 { (2, 3) } else { (3, 2) } + ); + let out = f.store.resolve("session", &result).unwrap(); + assert!(!out.windows(4).any(|w| w == b"eXIf")); + assert_eq!(out[24], 8); // RGBA8, not the source's color type. + assert_eq!(out[25], 6); + } +} + +#[test] +fn fit_rounding_upscale_and_triangle_filter() { + let f = Fixture::new(); + let image = pixels(5, 3); + let file = import_pixels(&f, &image); + let (_, contain) = transformed( + &f, + &file, + Transform::Resize { + width: 8, + height: 8, + fit: Fit::Contain, + }, + ); + assert_eq!(contain.dimensions(), (8, 4)); + assert_eq!( + contain, + image::imageops::resize(&image, 8, 4, image::imageops::FilterType::Triangle) + ); + let (_, cover) = transformed( + &f, + &file, + Transform::Resize { + width: 4, + height: 5, + fit: Fit::Cover, + }, + ); + let center = image::imageops::crop_imm(&image, 1, 0, 2, 3).to_image(); + assert_eq!( + cover, + image::imageops::resize(¢er, 4, 5, image::imageops::FilterType::Triangle) + ); + let (_, stretch) = transformed( + &f, + &file, + Transform::Resize { + width: 8, + height: 8, + fit: Fit::Stretch, + }, + ); + assert_eq!( + stretch, + image::imageops::resize(&image, 8, 8, image::imageops::FilterType::Triangle) + ); + assert_ne!( + stretch, + image::imageops::resize(&image, 8, 8, image::imageops::FilterType::Nearest) + ); +} + +#[test] +fn invalid_geometry_zero_rounding_extreme_dimensions_and_scratch() { + let f = Fixture::new(); + let file = import_pixels(&f, &pixels(3, 2)); + for degrees in [0, 1, 89, 360, u32::MAX] { + assert!( + f.store + .transform("session", &file, Transform::Rotate { degrees }, None) + .is_err() + ); + } + for (width, height) in [ + (0, 1), + (1, 0), + (8193, 1), + (1, u32::MAX), + (8192, 1), + (1, 8192), + ] { + let op = Transform::Crop { + aspect_ratio: AspectRatio { width, height }, + anchor: Anchor::Center, + }; + assert!(f.store.transform("session", &file, op, None).is_err()); + } + for (width, height) in [(0, 1), (1, 0), (8193, 1), (1, u32::MAX), (8192, 8192)] { + assert!( + f.store + .transform( + "session", + &file, + Transform::Resize { + width, + height, + fit: Fit::Stretch + }, + None + ) + .is_err() + ); + } + let wide = import_pixels(&f, &pixels(8192, 1)); + assert!( + f.store + .transform( + "session", + &wide, + Transform::Resize { + width: 1, + height: 8192, + fit: Fit::Contain + }, + None + ) + .is_err() + ); + let error = f + .store + .transform( + "session", + &wide, + Transform::Resize { + width: 1, + height: 8192, + fit: Fit::Stretch, + }, + None, + ) + .unwrap_err(); + assert!(error.contains("scratch"), "{error}"); + let (_, exact) = transformed( + &f, + &wide, + Transform::Resize { + width: 8192, + height: 1, + fit: Fit::Stretch, + }, + ); + assert_eq!(exact.dimensions(), (8192, 1)); +} + +#[test] +fn export_exact_original_no_clobber_permissions_and_pre_cancel() { + let f = Fixture::new(); + let file = import_pixels(&f, &pixels(3, 2)); + let destination = f.dir.path().join("export.png"); + f.store + .export("session", &file, &destination, None) + .unwrap(); + assert_eq!( + disk::read(&destination).unwrap(), + f.store.resolve("session", &file).unwrap() + ); + assert!( + f.store + .export("session", &file, &destination, None) + .is_err() + ); + assert!( + f.store + .export("session", &file, f.dir.path(), None) + .is_err() + ); + assert!( + f.store + .export("session", &file, &f.dir.path().join("missing/out"), None) + .is_err() + ); + #[cfg(unix)] + { + use std::os::unix::fs::{PermissionsExt as _, symlink}; + assert_eq!( + disk::metadata(&destination).unwrap().permissions().mode() & 0o777, + 0o600 + ); + let dangling = f.dir.path().join("dangling"); + symlink(f.dir.path().join("absent"), &dangling).unwrap(); + assert!(f.store.export("session", &file, &dangling, None).is_err()); + assert!( + disk::symlink_metadata(dangling) + .unwrap() + .file_type() + .is_symlink() + ); + } + let controller = agentkit_core::CancellationController::new(); + let cancellation = controller.handle().checkpoint(); + controller.interrupt(); + let absent = f.dir.path().join("cancelled"); + assert!( + f.store + .export("session", &file, &absent, Some(&cancellation)) + .unwrap_err() + .contains("cancelled") + ); + assert!(!absent.exists()); + assert!( + f.store + .transform( + "session", + &file, + Transform::Rotate { degrees: 90 }, + Some(&cancellation) + ) + .unwrap_err() + .contains("cancelled") + ); +} + +#[test] +fn concurrent_exports_have_exactly_one_winner() { + let f = Fixture::new(); + let file = import_pixels(&f, &pixels(3, 2)); + let path = f.dir.path().join("winner"); + std::thread::scope(|scope| { + let a = scope.spawn(|| f.store.export("session", &file, &path, None)); + let b = scope.spawn(|| f.store.export("session", &file, &path, None)); + assert_ne!(a.join().unwrap().is_ok(), b.join().unwrap().is_ok()); + }); + assert_eq!( + disk::read(path).unwrap(), + f.store.resolve("session", &file).unwrap() + ); +} + +#[test] +fn strict_transform_deserialization() { + assert!( + serde_json::from_value::(json!({"rotate":{"degrees":90,"extra":true}})).is_err() + ); + assert!( + serde_json::from_value::(json!({"width":1,"height":1,"extra":true})).is_err() + ); + for name in [ + "center", + "top_left", + "top", + "top_right", + "left", + "right", + "bottom_left", + "bottom", + "bottom_right", + ] { + assert!(serde_json::from_value::(json!(name)).is_ok()); + } + for name in ["contain", "cover", "stretch"] { + assert!(serde_json::from_value::(json!(name)).is_ok()); + } +} + +fn png_chunk(kind: &[u8; 4], data: &[u8]) -> Vec { + let mut result = (data.len() as u32).to_be_bytes().to_vec(); + result.extend(kind); + result.extend(data); + let mut crc = !0_u32; + for byte in &result[4..] { + crc ^= u32::from(*byte); + for _ in 0..8 { + crc = (crc >> 1) ^ (0xedb88320 & 0_u32.wrapping_sub(crc & 1)); + } + } + result.extend((!crc).to_be_bytes()); + result +} + +#[test] +fn strips_text_metadata_and_rejects_expanding_png_ancillary() { + let f = Fixture::new(); + let source = import_pixels(&f, &pixels(3, 2)); + let original = f.store.resolve("session", &source).unwrap(); + let path = f.dir.path().join("metadata.png"); + let mut bytes = original.clone(); + bytes.splice(33..33, png_chunk(b"tEXt", b"comment\0private information")); + disk::write(&path, &bytes).unwrap(); + let file = f.store.import("session", &path, None).unwrap(); + assert_eq!(f.store.resolve("session", &file).unwrap(), bytes); + let (output, _) = transformed(&f, &file, Transform::Rotate { degrees: 90 }); + let out = f.store.resolve("session", &output).unwrap(); + assert!( + !out.windows(4) + .any(|w| w == b"tEXt" || w == b"iCCP" || w == b"eXIf") + ); + for kind in [b"iCCP", b"zTXt", b"iTXt"] { + let mut bytes = original.clone(); + bytes.splice(33..33, png_chunk(kind, b"not expanded or passed to codec")); + disk::write(&path, bytes).unwrap(); + assert!( + f.store + .import("session", &path, None) + .unwrap_err() + .contains("metadata") + ); + } +} + +#[test] +fn high_entropy_jpeg_transform_hits_png_output_cap_without_publication() { + let f = Fixture::new(); + // The compressed import fits 8 MiB; the lossless transformed PNG does not. + let mut state = 0x12345678_u32; + let image = image::RgbImage::from_fn(2304, 1536, |_, _| { + let mut channels = [0; 3]; + for channel in &mut channels { + state ^= state << 13; + state ^= state >> 17; + state ^= state << 5; + *channel = state as u8; + } + image::Rgb(channels) + }); + let mut bytes = Vec::new(); + image::codecs::jpeg::JpegEncoder::new_with_quality(&mut bytes, 85) + .encode_image(&image) + .unwrap(); + assert!(bytes.len() as u64 <= MAX_FILE_BYTES); + let path = f.dir.path().join("noise.jpg"); + disk::write(&path, bytes).unwrap(); + let file = f.store.import("session", &path, None).unwrap(); + let before = disk::read_dir(f.store.session_directory("session")) + .unwrap() + .count(); + let error = f + .store + .transform("session", &file, Transform::Rotate { degrees: 180 }, None) + .unwrap_err(); + assert!(error.contains("8 MiB output"), "{error}"); + assert_eq!( + disk::read_dir(f.store.session_directory("session")) + .unwrap() + .count(), + before + ); +} + +#[test] +fn jpeg_orientation_and_icc_are_not_forwarded_and_malformed_exif_is_identity() { + let f = Fixture::new(); + let image = image::RgbImage::from_fn(7, 5, |x, y| { + image::Rgb([(x * 35) as u8, (y * 50) as u8, 75]) + }); + let mut jpeg = Vec::new(); + image::codecs::jpeg::JpegEncoder::new_with_quality(&mut jpeg, 95) + .encode_image(&image) + .unwrap(); + let decoded = image::load_from_memory(&jpeg).unwrap().into_rgba8(); + for orientation in [0, 6, 9] { + let mut app1 = b"Exif\0\0".to_vec(); + app1.extend(exif(orientation)); + let app2 = b"ICC_PROFILE\0\x01\x01opaque profile is not color converted"; + let mut bytes = jpeg[..2].to_vec(); + for (marker, data) in [(0xe1, app1.as_slice()), (0xe2, app2.as_slice())] { + bytes.extend([0xff, marker]); + bytes.extend(((data.len() + 2) as u16).to_be_bytes()); + bytes.extend(data); + } + bytes.extend(&jpeg[2..]); + let path = f.dir.path().join("metadata.jpg"); + disk::write(&path, &bytes).unwrap(); + let file = f.store.import("session", &path, None).unwrap(); + let (result, actual) = transformed(&f, &file, Transform::Rotate { degrees: 180 }); + let expected = if orientation == 6 { + image::imageops::rotate270(&decoded) + } else { + image::imageops::rotate180(&decoded) + }; + assert_eq!(actual, expected); + let out = f.store.resolve("session", &result).unwrap(); + assert!(!out.windows(4).any(|w| w == b"iCCP" || w == b"eXIf")); + let export = f.dir.path().join(format!("original-{orientation}.jpg")); + f.store.export("session", &file, &export, None).unwrap(); + assert_eq!(disk::read(export).unwrap(), bytes); + } +} + +#[cfg(unix)] +const EXPORT_FAILURE_PARENT_ENV: &str = "KIT_MANAGED_EXPORT_FAILURE_PARENT_PID"; + +#[cfg(unix)] +#[test] +fn export_after_create_failure_retains_destination_and_refuses_retry() { + let output = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "--exact", + "managed_files::tests::operations::export_file_size_limit_child", + "--ignored", + "--nocapture", + "--test-threads=1", + ]) + .env(EXPORT_FAILURE_PARENT_ENV, std::process::id().to_string()) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + output.status.success(), + "export failure child failed: {}\n{stdout}\n{stderr}", + output.status + ); + assert!( + stdout.contains("managed_files::tests::operations::export_file_size_limit_child ... ok"), + "child test did not succeed: {stdout}" + ); + assert!( + stdout.contains("1 passed; 0 failed"), + "child did not run exactly one successful test: {stdout}" + ); +} + +#[cfg(unix)] +#[test] +#[ignore = "invoked only by the export failure parent in an isolated Unix process"] +fn export_file_size_limit_child() { + let parent_pid: u32 = std::env::var(EXPORT_FAILURE_PARENT_ENV) + .expect("export failure parent PID") + .parse() + .unwrap(); + assert_ne!(std::process::id(), parent_pid); + // SAFETY: getppid takes no arguments and has no memory preconditions. + assert_eq!(unsafe { libc::getppid() } as u32, parent_pid); + + // Import and resolve before reducing the process-wide file-size limit, so + // the failure is at the real export write, not fixture publication. + let f = Fixture::new(); + let file = import_pixels(&f, &pixels(3, 2)); + let original = f.store.resolve("session", &file).unwrap(); + let destination = f.dir.path().join("retained-export.png"); + const LIMIT: libc::rlim_t = 32; + assert!(original.len() > LIMIT as usize); + assert!(!destination.exists()); + let mut limit = libc::rlimit { + rlim_cur: 0, + rlim_max: 0, + }; + // SAFETY: limit is a valid writable rlimit. Only this explicitly selected + // child changes signal disposition and resource limits; the parent does not. + assert_eq!( + unsafe { libc::getrlimit(libc::RLIMIT_FSIZE, &mut limit) }, + 0 + ); + assert!(limit.rlim_cur >= LIMIT); + // SAFETY: SIG_IGN is the OS-defined disposition, not a Rust signal handler. + assert_ne!( + unsafe { libc::signal(libc::SIGXFSZ, libc::SIG_IGN) }, + libc::SIG_ERR + ); + limit.rlim_cur = LIMIT; + // SAFETY: limit points to an initialized rlimit; the hard limit is unchanged. + assert_eq!(unsafe { libc::setrlimit(libc::RLIMIT_FSIZE, &limit) }, 0); + + let error = f + .store + .export("session", &file, &destination, None) + .unwrap_err(); + assert!(error.contains(destination.to_str().unwrap()), "{error}"); + assert!( + error.contains("partial-or-complete file retained"), + "{error}" + ); + assert!(disk::metadata(&destination).unwrap().is_file()); + let retained = disk::read(&destination).unwrap(); + assert_eq!(retained, original[..LIMIT as usize]); + + // The retained partial file is still protected by create_new on retry. + let retry = f + .store + .export("session", &file, &destination, None) + .unwrap_err(); + assert!(retry.contains(destination.to_str().unwrap()), "{retry}"); + assert!( + !retry.contains("partial-or-complete file retained"), + "{retry}" + ); + assert_eq!(disk::read(&destination).unwrap(), retained); + // No restoration is needed: this is the only test in the child, which exits. +} diff --git a/src/runtime.rs b/src/runtime.rs index 287e965..e0088b9 100644 --- a/src/runtime.rs +++ b/src/runtime.rs @@ -39,9 +39,9 @@ use crate::{ ModelSelection, ProviderKind, ReasoningEffort, SelectableAdapter, SelectableSession, }, tools::{ - A2aTool, ArtifactTool, AuthTool, CloseTool, DocsTool, EditTool, ForkTool, McpTool, - Observed, PromptTool, ReadFileTool, ShellTool, SubagentTool, Subagents, SubagentsTool, - ToolSearch, observe_shared, + A2aTool, ArtifactTool, AuthTool, CloseTool, DocsTool, EditTool, ForkTool, ImageTool, + McpTool, Observed, PromptTool, ReadFileTool, ShellTool, SubagentTool, Subagents, + SubagentsTool, ToolSearch, observe_shared, }, }; @@ -1138,6 +1138,10 @@ impl Runtime { &self.root, )))) .with(Observed::new(ReadFileTool::new(self.root.clone()))) + .with(Observed::new(ImageTool::rotate(self.root.clone()))) + .with(Observed::new(ImageTool::crop(self.root.clone()))) + .with(Observed::new(ImageTool::resize(self.root.clone()))) + .with(Observed::new(ImageTool::export(self.root.clone()))) .with(Observed::new(DocsTool::new())) .with(Observed::new(ShellTool::new(self.root.clone()))) .with(Observed::new(EditTool::new(self.root.clone()))); diff --git a/src/runtime/tests/managed_files.rs b/src/runtime/tests/managed_files.rs index 5f74054..dffeaa0 100644 --- a/src/runtime/tests/managed_files.rs +++ b/src/runtime/tests/managed_files.rs @@ -233,6 +233,98 @@ async fn managed_files_select_only_returned_references_and_preserve_spilled_imag assert!(!artifact.contains("inline_bytes")); } +#[tokio::test] +async fn managed_files_transform_pipeline_delivers_only_final_image_and_replays() { + for (background, outcome) in [(false, false), (false, true), (true, true)] { + let fixture = Fixture::new(); + let output = fixture.execute( + "source = read_file({path: \"image.png\"})\nrotated = image_rotate({image: source, degrees: 90})\ncropped = image_crop({image: rotated, aspect_ratio: {width: 1, height: 1}, anchor: \"center\"})\nresized = image_resize({image: cropped, width: 4, height: 4, fit: \"contain\"})\nreceipt = export_file({file: resized, path: \"result.png\"})\nreturn {image: resized, receipt}", + Value::Null, background, outcome, + ).await.unwrap(); + let exported = std::fs::read(fixture.root.path().join("result.png")).unwrap(); + assert_image(&output, &exported); + let image = image::load_from_memory(&exported).unwrap(); + assert_eq!((image.width(), image.height()), (4, 4)); + assert_eq!( + std::fs::read(fixture.root.path().join("image.png")).unwrap(), + fixture.bytes + ); + let ToolOutput::Parts(parts) = output else { + panic!() + }; + let Part::Structured(result) = &parts[0] else { + panic!() + }; + let reference = result.value["image"].clone(); + std::fs::remove_file(fixture.root.path().join("image.png")).unwrap(); + std::fs::remove_file(fixture.root.path().join("result.png")).unwrap(); + let replay = fixture + .execute("return input", reference, background, outcome) + .await + .unwrap(); + assert_image(&replay, &exported); + } +} + +#[tokio::test] +async fn managed_files_export_receipt_never_selects_pixels() { + let fixture = Fixture::new(); + let output = fixture.execute( + "source = read_file({path: \"image.png\"})\nrotated = image_rotate({image: source, degrees: 180})\nreturn export_file({file: rotated, path: \"receipt.png\"})", + Value::Null, false, true, + ).await.unwrap(); + assert_eq!( + output, + ToolOutput::structured(json!({ + "path":fixture.root.path().canonicalize().unwrap().join("receipt.png"), + "size_bytes":std::fs::metadata(fixture.root.path().join("receipt.png")).unwrap().len(), + "status":"exported" + })) + ); + assert!(fixture.root.path().join("receipt.png").is_file()); +} + +#[tokio::test] +async fn managed_files_unused_operations_still_execute_without_delivering_images() { + let fixture = Fixture::new(); + let output = fixture.execute( + "source = read_file({path: \"image.png\"})\nunused = export_file({file: source, path: \"unused.png\"})\nreturn {done: true}", + Value::Null, false, true, + ).await.unwrap(); + assert_eq!(output, ToolOutput::structured(json!({"done":true}))); + assert_eq!( + std::fs::read(fixture.root.path().join("unused.png")).unwrap(), + fixture.bytes + ); + let error = fixture.execute( + "source = read_file({path: \"image.png\"})\nunused = image_crop({image: source, aspect_ratio: {width: 8192, height: 1}, anchor: \"center\"})\nreturn {done: true}", + Value::Null, false, true, + ).await.unwrap_err(); + assert!(error.contains("nonzero"), "{error}"); +} + +#[tokio::test] +async fn managed_files_hidden_transform_schema_rejects_invalid_inputs() { + let fixture = Fixture::new(); + for operation in [ + "image_rotate({image: source, degrees: 45})", + "image_crop({image: source, aspect_ratio: {width: 0, height: 1}, anchor: \"center\"})", + "image_crop({image: source, aspect_ratio: {width: 1, height: 1}, anchor: \"outside\"})", + "image_resize({image: source, width: 2, height: 2, fit: \"unknown\"})", + "image_resize({image: source, width: 2, height: 2, fit: \"contain\", extra: true})", + "export_file({file: source, path: \"\"})", + ] { + let script = format!("source = read_file({{path: \"image.png\"}})\nreturn {operation}"); + assert!( + fixture + .execute(&script, Value::Null, false, true) + .await + .is_err(), + "{operation}" + ); + } +} + #[tokio::test] async fn managed_files_delivery_failure_does_not_claim_rollback() { let fixture = Fixture::new(); diff --git a/src/tools/image.rs b/src/tools/image.rs new file mode 100644 index 0000000..53a4f7a --- /dev/null +++ b/src/tools/image.rs @@ -0,0 +1,298 @@ +//! Hidden managed-image operations. Only compose finalization selects pixels. +use std::path::PathBuf; + +use agentkit_core::{ToolOutput, ToolResultPart}; +use agentkit_tools_core::{ + Tool, ToolContext, ToolError, ToolName, ToolRequest, ToolResult, ToolSpec, +}; +use async_trait::async_trait; +use serde::Deserialize; +use serde_json::Value; + +use super::read_file::{file_schema, object, object_schema, positive_integer}; +use crate::managed_files::{Anchor, AspectRatio, FileReference, FileStore, Fit, Transform}; + +#[derive(Clone, Copy)] +enum Operation { + Rotate, + Crop, + Resize, + Export, +} + +#[derive(Clone)] +pub struct ImageTool { + root: PathBuf, + store: FileStore, + operation: Operation, + spec: ToolSpec, +} + +const TRANSFORM_POLICY: &str = " Consumes a session-authorized managed File, never a path. Normalizes all EXIF orientations before geometry and returns a new immutable RGBA8 PNG, stripping source metadata (not color-managed conversion). Nonanimated PNG/JPEG only. Limits: 8 MiB encoded, 8192 per dimension, 16 megapixels, 64 MiB decode, 128 MiB resize scratch and 256 MiB estimated live pixel work; otherwise-valid geometries can fail budgets. Cancellation is cooperative between stages; running codecs are not preempted. Only final returned references deliver pixels."; + +impl ImageTool { + pub fn rotate(root: PathBuf) -> Self { + Self::new( + root, + Operation::Rotate, + "image_rotate", + "Rotate 90, 180, or 270 degrees clockwise.", + object_schema([ + ("image", file_schema()), + ( + "degrees", + object([ + ("type", Value::from("integer")), + ( + "enum", + Value::Array(vec![Value::from(90), Value::from(180), Value::from(270)]), + ), + ]), + ), + ]), + ) + } + + pub fn crop(root: PathBuf) -> Self { + Self::new( + root, + Operation::Crop, + "image_crop", + "Take the largest inscribed crop with an integer-rounded aspect ratio. Floor the shortened dimension; zero fails. Anchor positions choose the retained region; centered odd remainders leave the extra pixel right/bottom.", + object_schema([ + ("image", file_schema()), + ( + "aspect_ratio", + object_schema([ + ("width", positive_integer(8192)), + ("height", positive_integer(8192)), + ]), + ), + ( + "anchor", + object([ + ("type", Value::from("string")), + ( + "enum", + Value::Array(vec![ + Value::from("center"), + Value::from("top_left"), + Value::from("top"), + Value::from("top_right"), + Value::from("left"), + Value::from("right"), + Value::from("bottom_left"), + Value::from("bottom"), + Value::from("bottom_right"), + ]), + ), + ]), + ), + ]), + ) + } + + pub fn resize(root: PathBuf) -> Self { + Self::new( + root, + Operation::Resize, + "image_resize", + "Resize with premultiplied-alpha Triangle filtering in encoded color space; upscaling is allowed. contain fits within the width/height box, flooring the shortened dimension (zero fails), without padding. cover center-crops to the integer-rounded target aspect ratio then resizes exactly, permitting rounding distortion. stretch resizes exactly without preserving aspect ratio.", + object_schema([ + ("image", file_schema()), + ("width", positive_integer(8192)), + ("height", positive_integer(8192)), + ( + "fit", + object([ + ("type", Value::from("string")), + ( + "enum", + Value::Array(vec![ + Value::from("contain"), + Value::from("cover"), + Value::from("stretch"), + ]), + ), + ]), + ), + ]), + ) + } + + pub fn export(root: PathBuf) -> Self { + Self::new( + root, + Operation::Export, + "export_file", + "Export exact bytes of a session-authorized managed File to a NEW local file. Relative paths use the working directory; absolute paths and parent symlinks follow OS permissions, not a sandbox. Parent must exist. Atomic create-new refuses existing files, directories and final symlinks; never overwrites a source. Unix creation mode is 0600. Disk-only writes commit at successful file sync. Cancellation/error after creation intentionally retains potentially partial or complete output; retry refuses that existing destination. Returns a text-only path/status receipt, not a File reference.", + object_schema([ + ("file", file_schema()), + ( + "path", + object([ + ("type", Value::from("string")), + ("minLength", Value::from(1)), + ("maxLength", Value::from(4096)), + ]), + ), + ]), + ) + } + + fn new( + root: PathBuf, + operation: Operation, + name: &str, + description: &str, + schema: Value, + ) -> Self { + let export = matches!(operation, Operation::Export); + let description = if export { + description.to_owned() + } else { + format!("{description}{TRANSFORM_POLICY}") + }; + // Deliberately effectful: transforms persist immutable objects and export + // writes user-visible bytes. Unused calls still run inside compose. + let output_schema = if export { + object_schema([ + ("path", object([("type", Value::from("string"))])), + ("size_bytes", positive_integer(8_388_608)), + ( + "status", + object([ + ("type", Value::from("string")), + ("enum", Value::Array(vec![Value::from("exported")])), + ]), + ), + ]) + } else { + file_schema() + }; + Self { + store: FileStore::new(&root), + root, + operation, + spec: ToolSpec::new(ToolName::new(name), description, schema) + .with_output_schema(output_schema), + } + } +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct RotateInput { + image: FileReference, + degrees: u32, +} +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct CropInput { + image: FileReference, + aspect_ratio: AspectRatio, + anchor: Anchor, +} +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct ResizeInput { + image: FileReference, + width: u32, + height: u32, + fit: Fit, +} +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct ExportInput { + file: FileReference, + path: String, +} + +enum Action { + Transform(FileReference, Transform), + Export(FileReference, PathBuf), +} + +fn parse(value: Value) -> Result { + serde_json::from_value(value).map_err(|error| ToolError::InvalidInput(error.to_string())) +} + +#[async_trait] +impl Tool for ImageTool { + fn spec(&self) -> &ToolSpec { + &self.spec + } + + async fn invoke( + &self, + request: ToolRequest, + context: &mut ToolContext<'_>, + ) -> Result { + let action = match self.operation { + Operation::Rotate => { + let input: RotateInput = parse(request.input)?; + Action::Transform( + input.image, + Transform::Rotate { + degrees: input.degrees, + }, + ) + } + Operation::Crop => { + let input: CropInput = parse(request.input)?; + Action::Transform( + input.image, + Transform::Crop { + aspect_ratio: input.aspect_ratio, + anchor: input.anchor, + }, + ) + } + Operation::Resize => { + let input: ResizeInput = parse(request.input)?; + Action::Transform( + input.image, + Transform::Resize { + width: input.width, + height: input.height, + fit: input.fit, + }, + ) + } + Operation::Export => { + let input: ExportInput = parse(request.input)?; + if input.path.is_empty() || input.path.len() > 4096 { + return Err(ToolError::InvalidInput( + "path must contain 1 to 4096 UTF-8 bytes".into(), + )); + } + Action::Export(input.file, self.root.join(input.path)) + } + }; + let store = self.store.clone(); + let cancellation = context.cancellation.clone(); + let session = request.session_id.0; + let value = tokio::task::spawn_blocking(move || match action { + Action::Transform(image, transform) => { + let reference = + store.transform(&session, &image, transform, cancellation.as_ref())?; + serde_json::to_value(reference).map_err(|error| error.to_string()) + } + Action::Export(file, path) => { + let size_bytes = store.export(&session, &file, &path, cancellation.as_ref())?; + Ok(object([ + ("path", Value::from(path.to_string_lossy().into_owned())), + ("size_bytes", Value::from(size_bytes)), + ("status", Value::from("exported")), + ])) + } + }) + .await + .map_err(|error| ToolError::Internal(error.to_string()))? + .map_err(ToolError::ExecutionFailed)?; + Ok(ToolResult::new(ToolResultPart::success( + request.call_id, + ToolOutput::structured(value), + ))) + } +} diff --git a/src/tools/mod.rs b/src/tools/mod.rs index df85a1f..eec151f 100644 --- a/src/tools/mod.rs +++ b/src/tools/mod.rs @@ -2,6 +2,7 @@ mod a2a; mod artifact; mod docs; mod edit; +mod image; pub(crate) mod mcp; mod observed; mod read_file; @@ -13,6 +14,7 @@ pub use a2a::A2aTool; pub use artifact::ArtifactTool; pub use docs::DocsTool; pub use edit::EditTool; +pub use image::ImageTool; pub use mcp::{AuthTool, McpTool, ToolSearch}; pub use observed::Observed; pub(crate) use observed::shared as observe_shared; diff --git a/src/tools/read_file.rs b/src/tools/read_file.rs index 264c624..4fe7889 100644 --- a/src/tools/read_file.rs +++ b/src/tools/read_file.rs @@ -24,24 +24,68 @@ impl ReadFileTool { root, spec: ToolSpec::new( ToolName::new("read_file"), - "Import a regular local PNG or JPEG image as a durable immutable File reference. Only File references reachable from the final compose return deliver pixels; intermediate references remain private. Maximum 8 MiB, 8192 pixels per dimension and 16 megapixels; animation is unsupported. Source bytes, orientation and metadata are preserved. Files are session-scoped and survive restart and source deletion; other sessions have no implicit access. This is not a text-file reader.", + "Import a regular local PNG or JPEG image as a durable immutable File reference. Only File references reachable from the final compose return deliver pixels; intermediate references remain private. Maximum 8 MiB, 8192 pixels per dimension and 16 megapixels; animation is unsupported. Source bytes, orientation and accepted metadata are preserved; PNG iCCP/zTXt/iTXt metadata is rejected to bound expansion. Files are session-scoped and survive restart and source deletion; other sessions have no implicit access. This is not a text-file reader.", object_schema([ ("path", object([("type", Value::from("string")), ("minLength", Value::from(1)), ("maxLength", Value::from(4096))])) ]), ) - .with_output_schema(object_schema([ - ("$kit", object([("type", Value::from("string")), ("enum", Value::Array(vec![Value::from("file")]))])), - ("version", object([("type", Value::from("integer")), ("enum", Value::Array(vec![Value::from(1)]))])), - ("id", object([("type", Value::from("string")), ("pattern", Value::from("^file_[0-9a-f]{64}$"))])), - ("name", object([("type", Value::from("string")), ("minLength", Value::from(1)), ("maxLength", Value::from(255))])), - ("mime_type", object([("type", Value::from("string")), ("enum", Value::Array(vec![Value::from("image/png"), Value::from("image/jpeg")]))])), - ("size_bytes", positive_integer(8_388_608)), - ("image", object_schema([("width", positive_integer(8192)), ("height", positive_integer(8192))])), - ])) + .with_output_schema(file_schema()) .with_annotations(ToolAnnotations::read_only()), } } } -fn object(fields: [(&str, Value); N]) -> Value { +pub(super) fn file_schema() -> Value { + object_schema([ + ( + "$kit", + object([ + ("type", Value::from("string")), + ("enum", Value::Array(vec![Value::from("file")])), + ]), + ), + ( + "version", + object([ + ("type", Value::from("integer")), + ("enum", Value::Array(vec![Value::from(1)])), + ]), + ), + ( + "id", + object([ + ("type", Value::from("string")), + ("pattern", Value::from("^file_[0-9a-f]{64}$")), + ]), + ), + ( + "name", + object([ + ("type", Value::from("string")), + ("minLength", Value::from(1)), + ("maxLength", Value::from(255)), + ]), + ), + ( + "mime_type", + object([ + ("type", Value::from("string")), + ( + "enum", + Value::Array(vec![Value::from("image/png"), Value::from("image/jpeg")]), + ), + ]), + ), + ("size_bytes", positive_integer(8_388_608)), + ( + "image", + object_schema([ + ("width", positive_integer(8192)), + ("height", positive_integer(8192)), + ]), + ), + ]) +} + +pub(super) fn object(fields: [(&str, Value); N]) -> Value { Value::Object(Map::from_iter( fields .into_iter() @@ -49,7 +93,7 @@ fn object(fields: [(&str, Value); N]) -> Value { )) } -fn object_schema(fields: [(&str, Value); N]) -> Value { +pub(super) fn object_schema(fields: [(&str, Value); N]) -> Value { let required = Value::Array(fields.iter().map(|(key, _)| Value::from(*key)).collect()); object([ ("type", Value::from("object")), @@ -59,7 +103,7 @@ fn object_schema(fields: [(&str, Value); N]) -> Value { ]) } -fn positive_integer(maximum: u64) -> Value { +pub(super) fn positive_integer(maximum: u64) -> Value { object([ ("type", Value::from("integer")), ("minimum", Value::from(1)),