You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to #618 / #624. OpenRouter has a dedicated async video generation API (POST /api/v1/videos) with Seedance 2.0, Veo 3.1, Wan and others — separate from the chat-completions pathway our current @tanstack/ai-openrouter image adapter uses. We should add a video adapter, and close a few image-activity gaps surfaced while reviewing #624.
POST https://openrouter.ai/api/v1/videos → 202 with polling_url; poll GET /api/v1/videos/{jobId} until completed/failed; download from unsigned_urls[0]. Optional webhooks (video.generation.completed/failed/..., HMAC-SHA256).
The media-input fields map 1:1 onto the MediaInputRole taxonomy from #624:
TanStack imageInputs role
OpenRouter field
start_frame
frame_images[] with frame_type: "first_frame"
end_frame
frame_images[] with frame_type: "last_frame"
reference / character
input_references[]
(If both frame_images and input_references are present, OpenRouter treats the request as image-to-video; references take lower priority — i2v cookbook, reference-to-video cookbook.)
Implementation notes:
Async job lifecycle is closest to the OpenAI Sora adapter (packages/ai-openai/src/adapters/video.ts) — submit + poll, streaming progress via the existing video activity job path.
Video models do not appear in plain GET /api/v1/models; use GET /api/v1/videos/models (or ?output_modalities=video). The model-meta sync script needs to learn this second endpoint.
Image inputs should pass URLs through verbatim and convert data sources to data URIs (same policy as the image adapter — no server-side fetching). Note OpenRouter warns providers can't fetch URLs behind redirects/bot checks; surface that in docs.
Unmapped size is silently dropped — SIZE_TO_ASPECT_RATIO is a fixed 10-entry table; any other size resolves to undefined and no aspect ratio is sent (packages/ai-openrouter/src/adapters/image.ts). Warn or throw instead.
Verify imageConfig key casing against the SDK/API contract — the request mixes numberOfImages (camelCase) with aspect_ratio / image_size (snake_case), and relies on OpenRouter "filtering invalid config", which would mask a miskeyed field as a permanent silent no-op. The HTTP API documents image_config with snake_case fields.
Summary
Follow-up to #618 / #624. OpenRouter has a dedicated async video generation API (
POST /api/v1/videos) with Seedance 2.0, Veo 3.1, Wan and others — separate from the chat-completions pathway our current@tanstack/ai-openrouterimage adapter uses. We should add a video adapter, and close a few image-activity gaps surfaced while reviewing #624.1. New:
openrouterVideoadapter (generateVideoactivity)OpenRouter's video API (docs, announcement):
POST https://openrouter.ai/api/v1/videos→202withpolling_url; pollGET /api/v1/videos/{jobId}untilcompleted/failed; download fromunsigned_urls[0]. Optional webhooks (video.generation.completed/failed/..., HMAC-SHA256).model,prompt,duration,resolution,aspect_ratio,size,seed,generate_audio,callback_url,provider(passthrough).The media-input fields map 1:1 onto the
MediaInputRoletaxonomy from #624:imageInputsrolestart_frameframe_images[]withframe_type: "first_frame"end_frameframe_images[]withframe_type: "last_frame"reference/characterinput_references[](If both
frame_imagesandinput_referencesare present, OpenRouter treats the request as image-to-video; references take lower priority — i2v cookbook, reference-to-video cookbook.)Implementation notes:
packages/ai-openai/src/adapters/video.ts) — submit + poll, streaming progress via the existing video activity job path.GET /api/v1/models; useGET /api/v1/videos/models(or?output_modalities=video). The model-meta sync script needs to learn this second endpoint.datasources to data URIs (same policy as the image adapter — no server-side fetching). Note OpenRouter warns providers can't fetch URLs behind redirects/bot checks; surface that in docs.videoInputs/audioInputs: not supported by the API today → keep the throw-on-nonempty convention from feat(ai): add imageInputs / videoInputs / audioInputs for image-conditioned generation #624.2. Image activity gaps (chat-completions pathway)
From the #624 review:
sizeis silently dropped —SIZE_TO_ASPECT_RATIOis a fixed 10-entry table; any other size resolves toundefinedand no aspect ratio is sent (packages/ai-openrouter/src/adapters/image.ts). Warn or throw instead.imageConfigkey casing against the SDK/API contract — the request mixesnumberOfImages(camelCase) withaspect_ratio/image_size(snake_case), and relies on OpenRouter "filtering invalid config", which would mask a miskeyed field as a permanent silent no-op. The HTTP API documentsimage_configwith snake_case fields.image_config.strength(0.0–1.0 image-to-image influence) is documented for i2i models — consider exposing it via provider options now thatimageInputslands in feat(ai): add imageInputs / videoInputs / audioInputs for image-conditioned generation #624.Definition of done
Per repo couplings (CLAUDE.md +
.agent/self-learning/coupling.json), the implementing PR must include:openrouterVideoadapter + model metadata (incl. sync-script support forGET /api/v1/videos/models)imageInputsrole mapping per the table above; throws for unsupported roles/cardinalitiesimageConfigcasing verification, optionalstrength)docs/media/video-generation.mdprovider table +docs/adapters/openrouter.mdpackages/ai/skills/ai-core/media-generation/SKILL.mdprovider tableRefs: #618, #624