Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .changeset/typed-video-duration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
'@tanstack/ai': major
'@tanstack/ai-fal': major
---

Add per-model typed durations for video generation.

`VideoAdapter` gains a fifth generic, `TModelDurationByName`, plus two
introspection methods on every adapter:

- `availableDurations()` — returns a `DurationOptions` tagged union
(`discrete | range | mixed | none`) describing the durations the current
model accepts.
- `snapDuration(seconds)` — coerces raw seconds to the closest valid duration
for the current model.

The `duration` field on `generateVideo({ adapter, ... })` is now per-model
typed via `VideoDurationForAdapter<TAdapter>`. For FAL, the type is derived
from `@fal-ai/client`'s `EndpointTypeMap`, so:

- `falVideo('fal-ai/kling-video/v1.6/standard/text-to-video')` → `duration?: '5' | '10'`
- `falVideo('fal-ai/veo3')` → `duration?: '4s' | '6s' | '8s'`
- `falVideo('fal-ai/minimax/video-01')` → `duration` not accepted

**Breaking**: callers passing `duration: <number>` to FAL video models must
either pass the typed string union directly or call
`adapter.snapDuration(seconds)`. Adapters that have not yet declared their
per-model duration map get a sensible default (`{ kind: 'none' }`,
`undefined`) so existing behaviour is preserved.

Builds on `@tanstack/ai-schemas` (#622); once that PR's FAL pipeline syncs
runtime constraint data, the hand-curated map in
`packages/typescript/ai-fal/src/video/video-provider-options.ts` will be
replaced with schema-derived lookups.
329 changes: 250 additions & 79 deletions README.md

Large diffs are not rendered by default.

6 changes: 4 additions & 2 deletions docs/media/image-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ All image adapters support these common options:

| Model | Supported Sizes |
|-------|----------------|
| `gpt-image-2` | `1024x1024`, `1536x1024`, `1024x1536`, `auto` |
| `gpt-image-1` | `1024x1024`, `1536x1024`, `1024x1536`, `auto` |
| `gpt-image-1-mini` | `1024x1024`, `1536x1024`, `1024x1536`, `auto` |
| `dall-e-3` | `1024x1024`, `1792x1024`, `1024x1792` |
Expand Down Expand Up @@ -138,11 +139,11 @@ const result = await generateImage({

OpenAI models support model-specific Model Options:

#### GPT-Image-1 / GPT-Image-1-Mini
#### GPT-Image-2 / GPT-Image-1 / GPT-Image-1-Mini

```typescript
const result = await generateImage({
adapter: openaiImage('gpt-image-1'),
adapter: openaiImage('gpt-image-2'),
prompt: 'A cat wearing a hat',
modelOptions: {
quality: 'high', // 'high' | 'medium' | 'low' | 'auto'
Expand Down Expand Up @@ -223,6 +224,7 @@ interface GeneratedImage {

| Model | Images per Request |
|-------|-------------------|
| `gpt-image-2` | 1-10 |
| `gpt-image-1` | 1-10 |
| `gpt-image-1-mini` | 1-10 |
| `dall-e-3` | 1 |
Expand Down
20 changes: 10 additions & 10 deletions docs/reference/classes/StreamProcessor.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ Useful for auto-continue logic
clearMessages(): void;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:419](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L419)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:420](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L420)

Clear all messages

Expand All @@ -193,7 +193,7 @@ Clear all messages
finalizeStream(): void;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1683](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1683)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1684](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1684)

Finalize the stream — complete all pending operations.

Expand Down Expand Up @@ -251,7 +251,7 @@ Get current messages
getRecording(): ChunkRecording | null;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1844](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1844)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1845](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1845)

Get the current recording

Expand All @@ -267,7 +267,7 @@ Get the current recording
getState(): ProcessorState;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1801](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1801)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1802](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1802)

Get current processor state (aggregated across all messages)

Expand Down Expand Up @@ -303,7 +303,7 @@ auto-continuation produces no content.
process(stream): Promise<ProcessorResult>;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:436](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L436)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:437](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L437)

Process a stream and emit events through handlers

Expand All @@ -325,7 +325,7 @@ Process a stream and emit events through handlers
processChunk(chunk): void;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:470](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L470)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:471](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L471)

Process a single chunk from the stream.

Expand Down Expand Up @@ -355,7 +355,7 @@ docs/chat-architecture.md#adapter-contract — Expected event types and ordering
removeMessagesAfter(index): void;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:390](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L390)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:391](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L391)

Remove messages after a certain index (for reload/retry)

Expand All @@ -377,7 +377,7 @@ Remove messages after a certain index (for reload/retry)
reset(): void;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1868](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1868)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1869](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1869)

Full reset (including messages)

Expand Down Expand Up @@ -440,7 +440,7 @@ an assistant message which can cause empty message flicker.
startRecording(): void;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1831](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1831)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1832](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1832)

Start recording chunks

Expand Down Expand Up @@ -478,7 +478,7 @@ Get the conversation as ModelMessages (for sending to LLM)
static replay(recording, options?): Promise<ProcessorResult>;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1887](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1887)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1888](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1888)

Replay a recording through the processor

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/convertMessagesToModelMessages.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ function convertMessagesToModelMessages(messages): ModelMessage<
| null>[];
```

Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:71](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L71)
Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:79](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L79)

Convert UIMessages or ModelMessages to ModelMessages

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/createReplayStream.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ title: createReplayStream
function createReplayStream(recording): AsyncIterable<AGUIEvent>;
```

Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1899](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1899)
Defined in: [packages/typescript/ai/src/activities/chat/stream/processor.ts:1900](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/stream/processor.ts#L1900)

Create an async iterable from a recording

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/generateMessageId.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ title: generateMessageId
function generateMessageId(): string;
```

Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:529](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L529)
Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:549](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L549)

Generate a unique message ID

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/modelMessageToUIMessage.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ title: modelMessageToUIMessage
function modelMessageToUIMessage(modelMessage, id?): UIMessage;
```

Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:385](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L385)
Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:394](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L394)

Convert a ModelMessage to UIMessage

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/modelMessagesToUIMessages.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ title: modelMessagesToUIMessages
function modelMessagesToUIMessages(modelMessages): UIMessage<unknown>[];
```

Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:456](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L456)
Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:465](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L465)

Convert an array of ModelMessages to UIMessages

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/normalizeToUIMessage.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ title: normalizeToUIMessage
function normalizeToUIMessage(message, generateId): UIMessage;
```

Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:506](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L506)
Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:526](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L526)

Normalize a message (UIMessage or ModelMessage) to a UIMessage
Ensures the message has an ID and createdAt timestamp
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/functions/uiMessageToModelMessages.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ function uiMessageToModelMessages(uiMessage): ModelMessage<
| null>[];
```

Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:146](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L146)
Defined in: [packages/typescript/ai/src/activities/chat/messages.ts:154](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/messages.ts#L154)

Convert a UIMessage to ModelMessage(s)

Expand Down
8 changes: 4 additions & 4 deletions docs/reference/interfaces/AgentLoopState.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: AgentLoopState

# Interface: AgentLoopState

Defined in: [packages/typescript/ai/src/types.ts:702](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L702)
Defined in: [packages/typescript/ai/src/types.ts:703](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L703)

State passed to agent loop strategy for determining whether to continue

Expand All @@ -17,7 +17,7 @@ State passed to agent loop strategy for determining whether to continue
finishReason: string | null;
```

Defined in: [packages/typescript/ai/src/types.ts:708](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L708)
Defined in: [packages/typescript/ai/src/types.ts:709](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L709)

Finish reason from the last response

Expand All @@ -29,7 +29,7 @@ Finish reason from the last response
iterationCount: number;
```

Defined in: [packages/typescript/ai/src/types.ts:704](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L704)
Defined in: [packages/typescript/ai/src/types.ts:705](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L705)

Current iteration count (0-indexed)

Expand All @@ -44,6 +44,6 @@ messages: ModelMessage<
| null>[];
```

Defined in: [packages/typescript/ai/src/types.ts:706](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L706)
Defined in: [packages/typescript/ai/src/types.ts:707](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L707)

Current messages array
8 changes: 4 additions & 4 deletions docs/reference/interfaces/ApprovalRequestedEvent.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: ApprovalRequestedEvent

# Interface: ApprovalRequestedEvent

Defined in: [packages/typescript/ai/src/types.ts:1195](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1195)
Defined in: [packages/typescript/ai/src/types.ts:1196](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1196)

Emitted when a server tool requires approval before execution. The agent
loop yields this and pauses — `structured-output.complete` will not fire
Expand All @@ -31,7 +31,7 @@ for that run. The shape is fixed by the orchestrator's tool-approval flow
optional model: string;
```

Defined in: [packages/typescript/ai/src/types.ts:1147](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1147)
Defined in: [packages/typescript/ai/src/types.ts:1148](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1148)

Model identifier for multi-model support

Expand All @@ -47,7 +47,7 @@ Model identifier for multi-model support
name: "approval-requested";
```

Defined in: [packages/typescript/ai/src/types.ts:1196](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1196)
Defined in: [packages/typescript/ai/src/types.ts:1197](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1197)

#### Overrides

Expand All @@ -63,7 +63,7 @@ CustomEvent.name
value: object;
```

Defined in: [packages/typescript/ai/src/types.ts:1197](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1197)
Defined in: [packages/typescript/ai/src/types.ts:1198](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1198)

#### approval

Expand Down
12 changes: 6 additions & 6 deletions docs/reference/interfaces/AudioGenerationOptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: AudioGenerationOptions

# Interface: AudioGenerationOptions\<TProviderOptions\>

Defined in: [packages/typescript/ai/src/types.ts:1489](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1489)
Defined in: [packages/typescript/ai/src/types.ts:1490](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1490)

Options for audio generation (music, sound effects, etc.).
These are the common options supported across providers.
Expand All @@ -24,7 +24,7 @@ These are the common options supported across providers.
optional duration: number;
```

Defined in: [packages/typescript/ai/src/types.ts:1497](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1497)
Defined in: [packages/typescript/ai/src/types.ts:1498](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1498)

Desired duration in seconds

Expand All @@ -36,7 +36,7 @@ Desired duration in seconds
logger: InternalLogger;
```

Defined in: [packages/typescript/ai/src/types.ts:1505](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1505)
Defined in: [packages/typescript/ai/src/types.ts:1506](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1506)

Internal logger threaded from the generateAudio() entry point. Adapters
must call logger.request() before the SDK call and logger.errors() in
Expand All @@ -50,7 +50,7 @@ catch blocks.
model: string;
```

Defined in: [packages/typescript/ai/src/types.ts:1493](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1493)
Defined in: [packages/typescript/ai/src/types.ts:1494](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1494)

The model to use for audio generation

Expand All @@ -62,7 +62,7 @@ The model to use for audio generation
optional modelOptions: TProviderOptions;
```

Defined in: [packages/typescript/ai/src/types.ts:1499](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1499)
Defined in: [packages/typescript/ai/src/types.ts:1500](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1500)

Model-specific options for audio generation

Expand All @@ -74,6 +74,6 @@ Model-specific options for audio generation
prompt: string;
```

Defined in: [packages/typescript/ai/src/types.ts:1495](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1495)
Defined in: [packages/typescript/ai/src/types.ts:1496](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1496)

Text description of the desired audio
10 changes: 5 additions & 5 deletions docs/reference/interfaces/AudioGenerationResult.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: AudioGenerationResult

# Interface: AudioGenerationResult

Defined in: [packages/typescript/ai/src/types.ts:1521](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1521)
Defined in: [packages/typescript/ai/src/types.ts:1522](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1522)

Result of audio generation

Expand All @@ -17,7 +17,7 @@ Result of audio generation
audio: GeneratedAudio;
```

Defined in: [packages/typescript/ai/src/types.ts:1527](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1527)
Defined in: [packages/typescript/ai/src/types.ts:1528](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1528)

The generated audio

Expand All @@ -29,7 +29,7 @@ The generated audio
id: string;
```

Defined in: [packages/typescript/ai/src/types.ts:1523](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1523)
Defined in: [packages/typescript/ai/src/types.ts:1524](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1524)

Unique identifier for the generation

Expand All @@ -41,7 +41,7 @@ Unique identifier for the generation
model: string;
```

Defined in: [packages/typescript/ai/src/types.ts:1525](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1525)
Defined in: [packages/typescript/ai/src/types.ts:1526](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1526)

Model used for generation

Expand All @@ -53,7 +53,7 @@ Model used for generation
optional usage: object;
```

Defined in: [packages/typescript/ai/src/types.ts:1529](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1529)
Defined in: [packages/typescript/ai/src/types.ts:1530](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1530)

Token usage information (if available)

Expand Down
Loading
Loading