Skip to content

fix(js): do not append prompt when resuming generation (#4652) #4657

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
BenInvertase wants to merge 4 commits into
base: main
Choose a base branch
from
+39 −2
Draft

fix(js): do not append prompt when resuming generation (#4652) #4657

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6155d43
fix(js): skip appending prompt in generate when resume is set (#4652)
BenInvertase Feb 14, 2026
0d51d10
fix(js): skip appending user prompt in definePrompt when resume is se…
BenInvertase Feb 14, 2026
5744b1e
test(js): toGenerateRequest does not append prompt when resume is set
BenInvertase Feb 14, 2026
799bc7c
fix(js): throw INVALID_ARGUMENT when both prompt and resume are set (…
BenInvertase Feb 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions js/ai/src/generate.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,13 @@ export async function toGenerateRequest(
registry: Registry,
options: GenerateOptions
): Promise<GenerateRequest> {
if (options.prompt && options.resume) {
throw new GenkitError({
status: 'INVALID_ARGUMENT',
message:
'prompt is not supported when resume is set. The message history in messages is used instead.',
});
}
const messages: MessageData[] = [];
if (options.system) {
messages.push({
Expand All @@ -200,7 +207,7 @@ export async function toGenerateRequest(
if (options.messages) {
messages.push(...options.messages.map((m) => Message.parseData(m)));
}
if (options.prompt) {
if (options.prompt && !options.resume) {
Copy link
Contributor

@cabljac cabljac Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same as other comment in this file i think)

messages.push({
role: 'user',
content: Message.parseContent(options.prompt),
Expand Down Expand Up @@ -333,6 +340,13 @@ async function resourcesToActionRefs(
}

function messagesFromOptions(options: GenerateOptions): MessageData[] {
if (options.prompt && options.resume) {
throw new GenkitError({
status: 'INVALID_ARGUMENT',
message:
'prompt is not supported when resume is set. The message history in messages is used instead.',
});
}
const messages: MessageData[] = [];
if (options.system) {
messages.push({
Expand All @@ -343,7 +357,7 @@ function messagesFromOptions(options: GenerateOptions): MessageData[] {
if (options.messages) {
messages.push(...options.messages);
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There's an inconsistency in how messages are processed here compared to toGenerateRequest. This function pushes messages from options.messages directly, while toGenerateRequest uses Message.parseData(m) to normalize each message. The type definition for options.messages allows for lenient message formats (e.g., content as a string), which Message.parseData handles. Without parsing, this could lead to incorrectly formatted messages being sent to the model. To ensure consistency and robustness, it would be better to parse the messages here as well.

Suggested change
messages.push(...options.messages);
messages.push(...options.messages.map((m) => Message.parseData(m)));

Copy link
Contributor

@cabljac cabljac Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems preexisting? not something introduced here.

}
if (options.prompt) {
if (options.prompt && !options.resume) {
Copy link
Contributor

@cabljac cabljac Feb 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only thing i'm thinking is if a user explicitly does e.g

ai.generate({
  prompt: 'count to 10',
  messages: [...],
  resume: { respond: [...] },
})

then the prompt is silently ignored, which might be confusing -

might be better to be defensive here and do:

  if (options.prompt && options.resume) {
    throw new GenkitError({
      status: 'INVALID_ARGUMENT',
      message: 'prompt is not supported when resume is set. The message history in messages is used instead.',
    });
  }

messages.push({
role: 'user',
content: Message.parseContent(options.prompt),
Expand Down
1 change: 1 addition & 0 deletions js/ai/src/prompt.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,7 @@ async function renderUserPrompt<
promptCache: PromptCache,
renderOptions: PromptGenerateOptions<O, CustomOptions> | undefined
) {
if (renderOptions?.resume) return;
if (typeof options.prompt === 'function') {
messages.push({
role: 'user',
Expand Down
22 changes: 22 additions & 0 deletions js/ai/tests/generate/generate_test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,28 @@ describe('toGenerateRequest', () => {
},
throws: 'FAILED_PRECONDITION',
},
{
should: 'throw INVALID_ARGUMENT when both prompt and resume are set',
prompt: {
messages: [
{ role: 'user', content: [{ text: 'hi' }] },
{
role: 'model',
content: [
{ text: 'there' },
{ toolRequest: { name: 'test', input: { x: 1 } } },
],
},
],
prompt: 'count to 10',
resume: {
respond: {
toolResponse: { name: 'test', output: { done: true } },
},
},
},
throws: 'INVALID_ARGUMENT',
},
{
should: 'passes through output options',
prompt: {
Expand Down