-
Notifications
You must be signed in to change notification settings - Fork 23
workflows comfy #1442
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
giurgiur99
wants to merge
32
commits into
next-4
Choose a base branch
from
feat/ltx-video-ugc-template
base: next-4
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
workflows comfy #1442
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
b02905a
workflows comfy
giurgiur99 08f1e0d
fix template
giurgiur99 248fe1d
fix download
giurgiur99 dc783b0
split in two bundles
giurgiur99 233942c
workflow id
giurgiur99 613b017
multiscene test
giurgiur99 11fb2ea
multishoot v2
giurgiur99 6231b06
fix v2
giurgiur99 c0f74d5
v3 try
giurgiur99 3866a6e
voice concat
giurgiur99 989a93a
update ugc product template to include new fields necessary on dashboard
dnsi0 6e87150
concat voices too
giurgiur99 434f221
install missing services
giurgiur99 3fe722e
cut to new scene
giurgiur99 26f8dde
remove sizeGb
dnsi0 803503a
cleanup
giurgiur99 01a6c2d
Merge branch 'feat/ltx-video-ugc-template' of https://github.com/ocea…
giurgiur99 373bff2
readd sizegb and schema
giurgiur99 07ec8c7
minimax flow
giurgiur99 db1dc33
simplify
giurgiur99 4a18194
minimax h3 v2
giurgiur99 be813f1
v3 h3
giurgiur99 f8f3e0b
h3 v4
giurgiur99 c791959
image defaults
giurgiur99 c34cb01
speed improvement
giurgiur99 5486dc9
fixes
giurgiur99 fcd927f
fix box sizes
giurgiur99 2c6ac40
fix corrupt model download
giurgiur99 e83c4bb
new carachters
giurgiur99 8ff1fe6
fix tail sound
giurgiur99 93640b7
add prompt creation model
giurgiur99 8495e25
add minimax-music3 template
dnsi0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| { | ||
| "id": "ltx-video-ugc-multishot", | ||
| "name": "ComfyUI — UGC multishot reel (LTX-2.3)", | ||
| "description": "ComfyUI preloaded with LTX-2.3 for a vertical UGC reel built one shot at a time, with the same character in every shot. It runs the same two-stage generator as the product-video template (22B dev fp8, base pass then a spatial-upscaler refine pass), so shot quality matches. Usage: optionally upload a character photo as the first frame — with no photo you turn on 'No character image' and the style prompt builds the character instead. Then fill two text boxes: 'Style — every shot' holds everything that must stay identical (who the character is, wardrobe, room, lighting, palette, camera look) and you never touch it again; 'Scene — this shot' holds only what changes (action, camera move, dialogue, ambient sound). Click Run for a clip, edit the scene box, Run again. Every run also saves that clip's last frame beside it as lastframe_….png, and a Shot start dial decides how it is used: at 1.0 it is frame 0 and the shot continues seamlessly, while around 0.35 it acts as a character reference only, so the model composes a new frame and you get a hard cut to a new scene with the same character. Chaining shots this way — shot 2 then starts exactly where shot 1 ended and identity carries in pixels rather than in prose, the trick that holds a multi-shot reel together. Clips are 9:16 vertical (720×1280) at 5 seconds each (126 frames at 25 fps) with audio, saved to the bucket root; ComfyUI's save counter never overwrites, so re-takes accumulate and you pick the good ones later. Once every shot is good, switch to the Assemble reel workflow to concatenate up to 8 of them into one clip inside ComfyUI — no external editor needed. The assemble workflow keeps each clip's own audio and runs it through a voice-conversion node: LTX re-rolls the speaker on every clip, so the stitched track would otherwise change voice at each cut. It takes the reference voice from Shot 1's own audio — nothing to upload and nothing to type, since this is speech-to-speech rather than transcription — and converts every line to that one speaker, with words, timing and room tone untouched — so the picture stays in sync and the lips still match. The node pack is not in the image; the bootstrap clones it and installs its dependencies on launch, but only when the graph actually uses it, so the first launch of this template is slower and the product template is unaffected. Select a persistent-storage bucket: it holds ComfyUI's whole base directory, so the 38 GiB of weights download once and are reused, and the numbered shot clips land in the bucket root where the storage API's listFiles can see them. Needs a CUDA GPU with 48 GB+ VRAM.", | ||
| "kind": "bundle", | ||
| "service": "comfyui", | ||
| "outcome": "Build a multi-shot 9:16 UGC reel with one character and one voice throughout.", | ||
| "category": "video", | ||
| "includes": [ | ||
| { | ||
| "name": "LTX-2.3 22B dev (fp8)", | ||
| "kind": "model", | ||
| "sizeGb": 29.2, | ||
| "repoId": "Lightricks/LTX-2.3-fp8" | ||
| }, | ||
| { | ||
| "name": "Gemma-3-12B-it text encoder (fp4 mixed)", | ||
| "kind": "model", | ||
| "sizeGb": 9.5, | ||
| "repoId": "Comfy-Org/ltx-2" | ||
| }, | ||
| { | ||
| "name": "LTX-2.3 22B distilled LoRA (rank 111)", | ||
| "kind": "model", | ||
| "sizeGb": 2.7, | ||
| "repoId": "Comfy-Org/ltx-2.3" | ||
| }, | ||
| { | ||
| "name": "LTX-2.3 spatial upscaler x2", | ||
| "kind": "model", | ||
| "sizeGb": 1.0, | ||
| "repoId": "Lightricks/LTX-2.3" | ||
| }, | ||
| { | ||
| "name": "Gemma-3-12B-it abliterated LoRA (rank 64)", | ||
| "kind": "model", | ||
| "sizeGb": 0.6, | ||
| "repoId": "Comfy-Org/ltx-2" | ||
| }, | ||
| { | ||
| "name": "TTS-Audio-Suite (voice conversion for the assemble workflow)", | ||
| "kind": "customnode", | ||
| "url": "https://github.com/diodiogod/TTS-Audio-Suite" | ||
| } | ||
| ], | ||
| "image": "yanwk/comfyui-boot", | ||
| "tag": "cu130-megapak-pt211-20260803", | ||
| "exposedPorts": [ | ||
| 8188 | ||
| ], | ||
| "entrypoint": [ | ||
| "/bin/bash", | ||
| "-c" | ||
| ], | ||
| "commandFile": "comfyui-ugc-bootstrap.sh", | ||
| "workflows": [ | ||
| { | ||
| "id": "ocean_ugc_multishot", | ||
| "name": "Multishot reel — one shot per Run", | ||
| "description": "An image input plus two prompt boxes — a style box reused by every shot and a scene box you retype between runs. One Run renders one 9:16 clip and saves its last frame as lastframe_….png. The Shot start dial decides whether that image continues the take seamlessly (1.0) or acts as a character reference for a hard cut to a new scene (~0.35). With no image at all, the style prompt builds the character instead.", | ||
| "file": "workflows/ocean_ugc_multishot.json" | ||
| }, | ||
| { | ||
| "id": "ocean_ugc_assemble", | ||
| "name": "Assemble reel", | ||
| "description": "Concatenates up to 8 rendered shot clips (video + audio) into a single reel using core ComfyUI video nodes, then converts every voice in the stitched track to Shot 1's voice so the reel does not change speaker at each cut — pick clips in the LoadVideo dropdowns, bypass any slots you don't need, and Run. No upload and no script: it reads the reference voice straight off the clips you already rendered.", | ||
| "file": "workflows/ocean_ugc_assemble.json" | ||
| } | ||
| ], | ||
| "userConfigurableEnvVars": [ | ||
| { | ||
| "key": "COMFY_WORKFLOW_ID", | ||
| "validation": "^[A-Za-z0-9_-]+$" | ||
| }, | ||
| { | ||
| "key": "COMFY_WORKFLOW" | ||
| } | ||
| ], | ||
| "requiredResources": [ | ||
| { | ||
| "id": "cpu", | ||
| "min": 8, | ||
| "recommended": 16, | ||
| "unit": "cores" | ||
| }, | ||
| { | ||
| "id": "ram", | ||
| "min": 48, | ||
| "recommended": 128, | ||
| "unit": "GB" | ||
| }, | ||
| { | ||
| "id": "disk", | ||
| "min": 45, | ||
| "recommended": 85, | ||
| "unit": "GB" | ||
| }, | ||
| { | ||
| "kind": "discrete", | ||
| "type": "gpu", | ||
| "min": 1, | ||
| "recommended": 1, | ||
| "unit": "count", | ||
| "description": "CUDA GPU, 48 GB+ VRAM recommended (LTX-2.3 22B fp8 + Gemma-3-12B encoder)" | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| { | ||
| "id": "ltx-video-ugc-product", | ||
| "name": "ComfyUI — UGC product video (LTX-2.3)", | ||
| "description": "ComfyUI preloaded with LTX-2.3 for vertical short-form product video: a product photo becomes a 9:16 clip with camera motion and ambient audio, capped at the graph's 5 seconds (126 frames at 25 fps). Select a persistent-storage bucket: it holds ComfyUI's whole base directory, so the 38 GiB of weights download once and are reused, and generated clips land in the bucket root. Needs a CUDA GPU with 48 GB+ VRAM.", | ||
| "kind": "bundle", | ||
| "service": "comfyui", | ||
| "outcome": "Turn a product photo into a 9:16 clip with camera motion and ambient audio.", | ||
| "category": "video", | ||
| "includes": [ | ||
| { | ||
| "name": "LTX-2.3 22B dev (fp8)", | ||
| "kind": "model", | ||
| "sizeGb": 29.2, | ||
| "repoId": "Lightricks/LTX-2.3-fp8" | ||
| }, | ||
| { | ||
| "name": "Gemma-3-12B-it text encoder (fp4 mixed)", | ||
| "kind": "model", | ||
| "sizeGb": 9.5, | ||
| "repoId": "Comfy-Org/ltx-2" | ||
| }, | ||
| { | ||
| "name": "LTX-2.3 22B distilled LoRA (rank 111)", | ||
| "kind": "model", | ||
| "sizeGb": 2.7, | ||
| "repoId": "Comfy-Org/ltx-2.3" | ||
| }, | ||
| { | ||
| "name": "LTX-2.3 spatial upscaler x2", | ||
| "kind": "model", | ||
| "sizeGb": 1.0, | ||
| "repoId": "Lightricks/LTX-2.3" | ||
| }, | ||
| { | ||
| "name": "Gemma-3-12B-it abliterated LoRA (rank 64)", | ||
| "kind": "model", | ||
| "sizeGb": 0.6, | ||
| "repoId": "Comfy-Org/ltx-2" | ||
| } | ||
| ], | ||
| "image": "yanwk/comfyui-boot", | ||
| "tag": "cu130-megapak-pt211-20260803", | ||
| "exposedPorts": [ | ||
| 8188 | ||
| ], | ||
| "entrypoint": [ | ||
| "/bin/bash", | ||
| "-c" | ||
| ], | ||
| "commandFile": "comfyui-ugc-bootstrap.sh", | ||
| "workflows": [ | ||
| { | ||
| "id": "ocean_ugc_product", | ||
| "name": "Product → vertical clip", | ||
| "description": "Upload a product photo, get a 9:16 clip with camera motion and ambient audio.", | ||
| "file": "workflows/ocean_ugc_product.json" | ||
| } | ||
| ], | ||
| "userConfigurableEnvVars": [ | ||
| { | ||
| "key": "COMFY_WORKFLOW_ID", | ||
| "validation": "^[A-Za-z0-9_-]+$" | ||
| }, | ||
| { | ||
| "key": "COMFY_WORKFLOW" | ||
| } | ||
| ], | ||
| "requiredResources": [ | ||
| { | ||
| "id": "cpu", | ||
| "min": 8, | ||
| "recommended": 16, | ||
| "unit": "cores" | ||
| }, | ||
| { | ||
| "id": "ram", | ||
| "min": 48, | ||
| "recommended": 128, | ||
| "unit": "GB" | ||
| }, | ||
| { | ||
| "id": "disk", | ||
| "min": 40, | ||
| "recommended": 80, | ||
| "unit": "GB" | ||
| }, | ||
| { | ||
| "kind": "discrete", | ||
| "type": "gpu", | ||
| "min": 1, | ||
| "recommended": 1, | ||
| "unit": "count", | ||
| "description": "CUDA GPU, 48 GB+ VRAM recommended (LTX-2.3 22B fp8 + Gemma-3-12B encoder)" | ||
| } | ||
| ] | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Both templates set
disk.minbelow their ownincludesmanifest sum. The shared root cause is thatrequiredResources.diskwas not derived from the declared download sizes.requiredResourcesgatesSERVICE_START, so a consumer granted the minimum starts the service, pays, and then runs out of space during the download.docs/serviceTemplates/ltx-video-ugc-product.json#L81-L86: theincludesentries sum to 43.0 GB whiledisk.minis 40. Raisedisk.minabove the sum, and align the "38 GiB" figure indescription(line 4) with the manifest.docs/serviceTemplates/ltx-video-ugc-multishot.json#L92-L97: the same 43.0 GB of models plus the unsizedTTS-Audio-Suiteentry against adisk.minof 45. SetsizeGbon that entry and raisedisk.minto cover the node pack and its Hugging Face weights.📍 Affects 2 files
docs/serviceTemplates/ltx-video-ugc-product.json#L81-L86(this comment)docs/serviceTemplates/ltx-video-ugc-multishot.json#L92-L97🤖 Prompt for AI Agents