Skip to content

feat(#3996): save generated media with no-resend placeholders - #4025

Open
aheritier wants to merge 4 commits into
workspace-media-foundationsfrom
workspace-media-materialization
Open

feat(#3996): save generated media with no-resend placeholders#4025
aheritier wants to merge 4 commits into
workspace-media-foundationsfrom
workspace-media-materialization

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What and why

Materialize completed media as workspace files and record owner-qualified manifest references with actual collision/MIME-adjusted names. Session JSON stores references rather than generated base64. Ordinary later turns and compaction receive bounded metadata placeholders instead of bytes. Per-item failures preserve text and successful siblings. The missing-image notice is a phrase heuristic with accepted false positives and negatives; it is not NLP, has no retry, and does not test output capability.

Part of #3996. Review this PR against its immediate parent, #4024, rather than the aggregate stack against main.

Commit inventory

Head: 9d7da808954352fc5fd7c54aeeacc8240c0bcc2d; parent SHA: 8cd7c8d6182ec1a60d9a404268f88887f19c80fa.

Validation

Build, test compilation, owning-package tests and the named fixture passed at this PR head.

Exact deterministic fixture command:

go test -v -count=1 ./pkg/cli ./pkg/runtime ./pkg/session ./pkg/model/provider/gemini -run 'TestImageGenerationWarningPreservesTextAndPrintsAfterIt|TestMaterializeGeneratedMedia|TestMediaSaveFailureReason|TestRunStream_MediaOnlyAssistantHistoryRemainsCoherent|TestRunStream_GeneratedMediaAbsentFromNextTurnHistory|TestStripGeneratedMediaTransform|TestImageGenerationTextOnlyWarning|TestImageGenerationProviderErrorDoesNotEmitWarning|TestGeneratedMediaManifest|TestConvertMessagesToGemini_GeneratedMediaPlaceholder'

Matched top-level tests: pkg/cli: 1; pkg/model/provider/gemini: 4; pkg/runtime: 28; pkg/session: 6.

Deterministic scope: the named local fixture exercises this PR boundary with disposable configuration/stores and fake or loopback providers as applicable. Every listed package ran nonzero matching top-level tests.

Deferred/live scope: Live provider, remote CI, and platform execution are not claimed by this deterministic receipt. The final stack head passed build, lint, full tests, an uncached full suite, focused race tests and documentation checks in disposable environments. Remote CI is tracked by the checks below; no new paid-provider or active-database validation was run.

@aheritier aheritier mentioned this pull request Aug 22, 2026
Open
@aheritier aheritier added area/docs Documentation changes area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/gemini Google Gemini provider support area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/sessions For features/issues/fixes related to session lifecycle (resume, persistence, export) kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 9ffaa5a to 2b15b2e Compare August 24, 2026 15:13
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 2b15b2e to 3dadcf6 Compare August 25, 2026 07:59
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 3dadcf6 to b07ac83 Compare August 25, 2026 08:16
@aheritier
aheritier force-pushed the workspace-media-materialization branch from b07ac83 to 5e896ae Compare August 25, 2026 20:24
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 5e896ae to ef641a7 Compare August 26, 2026 11:09
@aheritier
aheritier force-pushed the workspace-media-materialization branch 2 times, most recently from 3d8769a to ef641a7 Compare September 1, 2026 16:12
@aheritier
aheritier force-pushed the workspace-media-materialization branch 2 times, most recently from 7bc0365 to 41c56eb Compare September 1, 2026 19:58
aheritier

This comment was marked as resolved.

Sign in to view

@aheritier
aheritier force-pushed the workspace-media-materialization branch from 41c56eb to d8df5cd Compare September 6, 2026 16:14
@aheritier
aheritier force-pushed the workspace-media-materialization branch from d8df5cd to 94cbcfa Compare September 7, 2026 11:19
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 170abf1 to 868e798 Compare September 8, 2026 07:57
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 868e798 to 529ebfc Compare September 8, 2026 17:31
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 529ebfc to 9d7da80 Compare September 8, 2026 20:35
@aheritier aheritier changed the title feat(#3996): classify generated-media save failures into safe user-visible reasons feat(#3996): save generated media with no-resend placeholders Sep 8, 2026
@aheritier aheritier mentioned this pull request Sep 8, 2026
Open
@aheritier
aheritier marked this pull request as ready for review September 8, 2026 20:38
@aheritier
aheritier requested a review from a team as a code owner September 8, 2026 20:38
@dgageot
dgageot force-pushed the workspace-media-materialization branch from 9d7da80 to 0dac5fe Compare September 9, 2026 07:41
dgageot
dgageot previously approved these changes Sep 9, 2026
View reviewed changes
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 0dac5fe to a953251 Compare September 9, 2026 11:56
@dgageot
dgageot force-pushed the workspace-media-materialization branch from a953251 to 2569c9e Compare September 9, 2026 12:15
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 2569c9e to b985006 Compare September 9, 2026 12:56
@aheritier
aheritier force-pushed the workspace-media-materialization branch from b985006 to 16f0138 Compare September 9, 2026 13:32
@aheritier
aheritier force-pushed the workspace-media-materialization branch from 16f0138 to 7148760 Compare September 9, 2026 14:07
@aheritier
feat(#3996): materialize generated media with no-resend history
51f71b6 
Save accumulated media into the owning session's workspace, resolving
parent-session WorkingDir provenance. Persist owner-qualified workspace
references and MIME/name/size rather than generated base64 in session JSON.
Use generic naming fallback for refused provider names and per-item warnings
when no workspace/save is available; there is no alternate storage target
in this patch. Exempt media-bearing results from the empty-response error.

Replace saved generated parts in ordinary later provider turns with bounded
metadata placeholders and avoid counting mirrored text twice in compaction.
If all saves in a media-only reply fail, an empty assistant record may remain
alongside warnings; successful siblings and reply text are retained.

At a text-only stop, match phrases in the last user prompt for a nonfatal
missing-image notice. A terminal error skips this branch; current-agent
structured-output config suppresses it, but arbitrary per-call overrides
and reply content are not classified. Negated/question/context phrases can
match, capability is unchecked, and continuations are not tracked as one
submission. Characterization tests pin those limits. Use mockRuntime.events
for the independent CLI text-before-warning test, not a later helper.
@aheritier
fix(#3996): sanitize and bound generated-media metadata
dfc81b0 
Route provider display names through chat.SanitizeDisplayName and normalize
MIME metadata to bounded conservative type/subtype tokens, falling back to
application/octet-stream. This token check is not strict RFC 6838
validation. Bound formatted warnings/placeholders to 512 bytes and fields
to 128 bytes without splitting valid UTF-8 runes.

Sanitize harness interpolation and keep raw OS errors and source paths out
of warnings. Make unsupported-input stripping skip generated artifacts
independently of transform registration order. Cover hostile/overlong
metadata, fallbacks, ownerless markers, converter placeholders, harness
injection and partial-success batches.
@aheritier
feat(#3996): record generated media in a manifest trust anchor
5d6bc20 
Record owner session, final workspace-relative path, sanitized MIME and
creation time after successful writes in SQLite migration 028 and the
memory store. Validate path shapes on insertion and lookup, prune records
on session deletion, and avoid a foreign key because materialization may
precede the persisted session row. The manifest provides authorization for
file resolution; generated references still require a known root kind.

Persist collision-adjusted paths and issue bounded extension-correction
notices. Missing or unwritable roots retain per-item warnings without a
data-directory fallback. Reject unclean stored WorkingDir values. This
does not yet wire the TUI resolver or prompt-directed workspace redirection.
@aheritier
feat(#3996): classify generated-media save failures into safe reasons
bc659a1 
Replace generic materialization and manifest warnings with fixed reasons:
missing workspace provenance, unwritable or deleted locations, exhausted
collision names, refused paths, and files saved without a manifest record.
Keep the item index and bounded, sanitized MIME/display-name metadata;
never expose raw errors, absolute paths, or session IDs in these warnings.
Unclassified failures advise --debug and retry, with detailed causes kept
in debug logs. Add ErrNameExhausted to classify collision exhaustion
without echoing the requested path.
@aheritier
aheritier dismissed dgageot’s stale review September 9, 2026 15:55

The merge-base changed after approval.

@aheritier
aheritier force-pushed the workspace-media-materialization branch from 7148760 to bc659a1 Compare September 9, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dgageot dgageot dgageot approved these changes

Assignees

No one assigned

Labels

area/docs Documentation changes area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/gemini Google Gemini provider support area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/sessions For features/issues/fixes related to session lifecycle (resume, persistence, export) kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@aheritier @dgageot