Skip to content

feat: deliver managed images through compose - #126

Merged
danielkov merged 5 commits into
mainfrom
feat/compose-files-phase-1
Sep 8, 2026
Merged

feat: deliver managed images through compose#126
danielkov merged 5 commits into
mainfrom
feat/compose-files-phase-1

Conversation

@danielkov

@danielkov danielkov commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Keep compose as the only model-exposed tool while adding durable, session-scoped File references and a hidden read_file callable. PNG/JPEG references selected by the final Runlet return deliver typed images through native tool output or an ordinary user-image transport fallback, and render in expanded TUI tool cards.

Motivation

Implement phase 1 of the approved four-phase plan: compose image reads without exposing another top-level tool or embedding pixels in intermediate JSON. Lack of native image tool-output support must not block image delivery when ordinary user-image input is available.

Impact

The verified openai-subscription:gpt-5.4 route retains native image tool output. Other subscription models, including gpt-6-astra, and the OpenRouter/Speakeasy adapters use their ordinary user-role image encoding. Their textual tool results preserve diagnostics and direct the model to image attachments following the complete tool-result batch. This transport fallback does not imply every selected model supports vision.

The initial reader accepts nonanimated PNG and JPEG. Managed objects survive source deletion and process restart; no automatic garbage collection is introduced. Transformations, multimodal subagents, and Markdown/all-origin rendering remain later phases.

Technical details

Durable and session-authorized snapshots

A strict versioned descriptor identifies an immutable binary envelope containing authoritative metadata and a digest-verified payload. Publication requires successful disk and directory-entry durability barriers. Resolution derives its namespace from the actual session, rejects traversal and symlinks, and grants no implicit cross-session or fork access.

Bounded final-return selection

Only references reachable from the final compose return are selected. Traversal, descriptor strings, every reference occurrence, duplicate metadata, labels, encoded bytes, and decoded pixels are bounded before delivery. Both compose execution entry points share finalization, including detached completion. Errors distinguish a completed program from failed delivery so callers do not blindly repeat side effects.

Request-only fallback without duplicate tool answers

Canonical transcripts retain typed tool results. Provider adapters project non-native images into user-role attachments only in outgoing requests, after all outstanding results in the batch. Tool text, labels, diagnostics, and nested-result provenance remain available. Detached completion notifications are projected without answering their already-acknowledged tool calls again. Replay and authenticated continuations reuse retained images without rerunning compose or persisting synthetic user turns; existing image normalization and provider limits still apply.

Separate text and image presentation

The 8 KiB compose text budget excludes media payloads and reserves adjacent JSON Pointer labels before spilling ordinary JSON. TUI tool cards reuse the existing bounded image decoder/cache. Raw-output fallback suppresses image payloads without discarding surrounding diagnostic text or literal data-URL scheme mentions.

Demo

Screenshot 2026-09-08 at 13 03 21

@danielkov

Copy link
Copy Markdown
Contributor Author

Current-head local evidence for c636c54f9bb795274cffdc6c760af9ff3f4598c9:

  • cargo fmt --check passed.
  • cargo clippy --locked --all-targets --all-features -- -D warnings passed.
  • RUST_TEST_THREADS=2 cargo test --locked passed: 1,121 library tests, 33 binary tests, and all integration suites. Four library tests are ignored by the ordinary harness; the managed-file subprocess helpers are explicitly exercised by their parent tests.
  • scripts/tests/test_next_release_version.sh passed; this PR has no phase-specific manifest, lockfile, or release-version changes against current main.

Coverage includes genuine fresh-process PNG/JPEG resolution after source deletion; isolated write/sync/ENOSPC durability failures; actual task-manager detachment followed by originating-turn interruption and deferred typed-image completion; compose-to-native Responses image encoding; nested/deduplicated references and text spill; provider switching and TUI live/replay/fallback behavior.

The default highly concurrent macOS test run hit existing shell/subagent timing failures; the complete bounded-concurrency rerun passed. Provider delivery is verified through production request encoding, not a live authenticated model call.

Independent code review approved this exact SHA. That is not a submitted GitHub approval; the repository's required GitHub approval remains outstanding. No merge or auto-merge is authorized.

@kit-code-agent kit-code-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raw-only tool output can lose surrounding diagnostics when image data-URL redaction replaces an entire text value.

Comment thread src/tui/mod.rs Outdated
kit-code-agent[bot]
kit-code-agent Bot previously approved these changes Sep 8, 2026

@kit-code-agent kit-code-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. The changes look good to merge.

@kit-code-agent kit-code-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. The changes look good to merge.

@danielkov
danielkov force-pushed the feat/compose-files-phase-1 branch from 7a642d7 to 2f76d11 Compare September 8, 2026 17:10

@kit-code-agent kit-code-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. The changes look good to merge.

@danielkov
danielkov merged commit 38161a6 into main Sep 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant