Skip to content

feat(#3996): name generated media with private markers and prompt fallback - #4027

Open
aheritier wants to merge 2 commits into
workspace-media-escape-confirmfrom
workspace-media-naming
Open

feat(#3996): name generated media with private markers and prompt fallback#4027
aheritier wants to merge 2 commits into
workspace-media-escape-confirmfrom
workspace-media-naming

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

What and why

Request private [media-file: ...] markers, remove them from visible and persisted text, and apply naming precedence: marker, exact one-image user-prompt filename, provider name, then generated-N. Prompt fallback applies only when exactly one image was returned. Requested paths remain untrusted and pass through workspace-only containment, MIME correction, and collision-safe publication.

Part of #3996. Review this PR against its immediate parent, #4026, rather than the aggregate stack against main.

Commit inventory

Head: da8faa29eace36625b546c4d7103b4d08543d57e; parent SHA: 1409e8b414370be9a472cad3d889d0cc8a2316c9.

Validation

Build, test compilation, owning-package tests and the named fixture passed at this PR head.

Exact deterministic fixture command:

go test -v -count=1 ./pkg/runtime ./pkg/model/provider/gemini -run 'TestExtractExplicitOutputFilename|TestApplyUserPromptRequestedPath|TestUserPromptOfPhraseFilename|TestMarkerOverridesUserPromptFilename|TestUserPromptFilenameSkipsMultiBlobTurns|TestComparativeReferencePromptsNeverNameGeneratedMedia|TestHandleStream_MediaFileMarker|TestCreateChatCompletionStream_MediaFileInstruction'

Matched top-level tests: pkg/model/provider/gemini: 2; pkg/runtime: 8.

Deterministic scope: the named local fixture exercises this PR boundary with disposable configuration/stores and fake or loopback providers as applicable. Every listed package ran nonzero matching top-level tests.

Deferred/live scope: Live provider, remote CI, and platform execution are not claimed by this deterministic receipt. The final stack head passed build, lint, full tests, an uncached full suite, focused race tests and documentation checks in disposable environments. Remote CI is tracked by the checks below; no new paid-provider or active-database validation was run.

@aheritier aheritier added area/providers/gemini Google Gemini provider support area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the workspace-media-naming branch from 25c9d9a to c4127a5 Compare August 24, 2026 15:13
@aheritier
aheritier force-pushed the workspace-media-naming branch from c4127a5 to 9594ce2 Compare August 25, 2026 08:00
@aheritier aheritier added the status/needs-rebase PR has merge conflicts or is out of date with main label Aug 25, 2026
@aheritier

Copy link
Copy Markdown
Collaborator Author

👋 This PR has merge conflicts with the base branch. Please rebase or merge the latest base branch and resolve them. I've moved it to draft and added status/needs-rebase; it'll be picked back up automatically once the conflicts are cleared.

@aheritier
aheritier force-pushed the workspace-media-naming branch from 9594ce2 to 56fad2f Compare August 25, 2026 08:16
@aheritier
aheritier force-pushed the workspace-media-naming branch from 56fad2f to 0272095 Compare August 25, 2026 08:32
@aheritier aheritier removed the status/needs-rebase PR has merge conflicts or is out of date with main label Aug 25, 2026
@aheritier
aheritier force-pushed the workspace-media-naming branch from 0272095 to ddcbca4 Compare August 25, 2026 20:24
@aheritier
aheritier force-pushed the workspace-media-naming branch from ddcbca4 to 9e83563 Compare August 26, 2026 11:10
@aheritier
aheritier force-pushed the workspace-media-naming branch 2 times, most recently from 4968274 to 9e83563 Compare September 1, 2026 16:09
@aheritier
aheritier force-pushed the workspace-media-naming branch from 9e83563 to c7783d3 Compare September 1, 2026 16:30
@aheritier
aheritier force-pushed the workspace-media-naming branch from c7783d3 to b18b517 Compare September 1, 2026 19:58
aheritier

This comment was marked as resolved.

@aheritier
aheritier force-pushed the workspace-media-naming branch from beea6b8 to e9fd7c8 Compare September 8, 2026 06:17
@aheritier
aheritier force-pushed the workspace-media-naming branch from e9fd7c8 to 9db704a Compare September 8, 2026 07:57
@aheritier
aheritier force-pushed the workspace-media-naming branch from 9db704a to b189b04 Compare September 8, 2026 17:31
@aheritier
aheritier force-pushed the workspace-media-naming branch from b189b04 to da8faa2 Compare September 8, 2026 20:36
@aheritier aheritier changed the title feat(#3996): fall back to an explicit user-prompt filename for one unnamed image feat(#3996): name generated media with private markers and prompt fallback Sep 8, 2026
@aheritier
aheritier marked this pull request as ready for review September 8, 2026 20:39
@aheritier
aheritier requested a review from a team as a code owner September 8, 2026 20:39
@dgageot
dgageot force-pushed the workspace-media-naming branch from da8faa2 to c40f2d7 Compare September 9, 2026 07:41
dgageot
dgageot previously approved these changes Sep 9, 2026
@aheritier
aheritier force-pushed the workspace-media-naming branch from c40f2d7 to 0f4ecd5 Compare September 9, 2026 11:56
@dgageot
dgageot force-pushed the workspace-media-naming branch from 0f4ecd5 to 31c417f Compare September 9, 2026 12:15
@aheritier
aheritier force-pushed the workspace-media-naming branch from 31c417f to 19d3fc1 Compare September 9, 2026 12:56
@aheritier
aheritier force-pushed the workspace-media-naming branch from 19d3fc1 to a26a733 Compare September 9, 2026 13:32
@aheritier
aheritier force-pushed the workspace-media-naming branch from a26a733 to 9cddd6e Compare September 9, 2026 14:07
Add a naming instruction to ordinary Gemini requests whose resolved
image-output capability enables TEXT+IMAGE, including gateway, direct API,
and Vertex routes. Ask for one exact "[media-file: relative/path]" line per
image, honoring a requested filename or choosing a meaningful name, and
request one image unless multiple images or variations are requested.

Filter marker lines from live and persisted assistant text with a bounded,
chunk-split-safe parser. Pair paths positionally with media through
MediaDelta.RequestedPath, then use provider names or generated-N for
unnamed blobs; strip and ignore extra markers without dropping images.
Marker paths remain untrusted and pass through workspace containment,
extension correction, workspace redirection, and collision-safe publishing
so model-selected names cannot bypass the existing write policy.
…named image

A live image-output model can ignore the [media-file:] marker instruction
entirely, leaving a prompt like "Generate an image as sunshine.jpg" to land
as generated-1.png. Add a deterministic fallback: when a turn returns
exactly ONE media blob that marker pairing left unnamed, parse a single
unambiguous explicit output filename from the triggering user message.

The cue grammar is strict, deliberately not NLP: save (it) as / save to /
write to / output to / name it / call it / filename:=, plus bare "as" only
inside a narrow imperative output context (generation verb +
optionally-articled media noun, e.g. "Generate an image as sunshine.jpg"),
and a companion of-phrase form (generation verb + media noun +
"of <subject>" + "as <filename>") so "Generate an image of a red panda as
assets/red-panda.jpg" extracts the intended name. The of-phrase subject
cannot cross quotes, clause punctuation, or CR/LF, and any subject
containing "as", "with", or "in" is refused because those prepositions
introduce open-ended attribute phrases whose trailing "as" compares — a
false reject only costs the generic generated-N name, while a false
capture could engage the workspace-redirection policy for a merely
referenced file. Bare "called" and unanchored "as" are NOT cues, so
comparative references to existing files never extract a name. Candidates
are keyed by the filename capture's position: one occurrence matched by
both grammars counts once; distinct occurrences stay ambiguous and extract
nothing. Names may be quoted, backticked, or unquoted with a known image
extension; zero or multiple candidates extract nothing.

Precedence stays marker -> user-prompt filename -> provider display name ->
generic generated-N, and the extracted path only fills
MediaDelta.RequestedPath, so the existing untrusted-path pipeline
(MIME/extension correction, collision suffixing, workspace containment and
workspace redirection) applies unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/gemini Google Gemini provider support area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants