[ralyodio]

Video generation that our own fleet can actually serve. Today the video plugin only lists models with a 24 to 48 GiB VRAM floor, and the fleet has ~7 CPU-only workers, 2 AMD GPUs and one 8 GiB NVIDIA card, so no clip has ever been generated.

GPU path: Wan 2.1 T2V 1.3B on an 8 GiB card

• plugins/video/module.toml: new row wan2.1-t2v-1.3b (Wan-AI/Wan2.1-T2V-1.3B-Diffusers, Apache-2.0, 480p@16fps). vram_gib = 8 (floor with offload) and full_vram_gib = 20 (everything on the card). Rows may now carry their own job defaults; this one defaults to 33 frames of 480x272, 16 fps, 20 steps.
• runner.py: when the card is smaller than full_vram_gib, it encodes the prompt on the CPU first (the UMT5-XXL encoder is ~11 GB in bf16, and model offload would still move it onto the card whole), then enable_model_cpu_offload() plus VAE tiling. Rows without full_vram_gib behave exactly as before. It also refuses Wan frame counts that are not 4k+1 (the pipeline rounds them and the clip then fails validation after the work is paid for). Runner version 3; the sidecar records offload.
• Contract tests cover the new row, the model defaults, the 4k+1 rule and the offload policy.
• The runner image digest pin moves in a follow-up PR, after this merge publishes the new image (video-runner.yml prints the digest).

Model tags, so jobs only reach nodes that can run them

• GPU nodes now advertise c0mpute:video:model:<id> for each model their card can hold (nvidia-smi memory vs the manifest floors, with 512 MiB of slack). The /v1/videos gateway requires the tag of the model a job names, rejects unknown models with a 400, and fills omitted fields from that model's defaults.
• Role::Video is in the default worker roles. It only subscribes where it can serve.

CPU path: the same model on stable-diffusion.cpp (opt-in)

• New video_cpu.rs. It serves video.txt2vid with wan2.1-t2v-1.3b through the sd.cpp build the diffusion plugin already pins (master-1060-0b3b98d, which supports Wan via -M vid_gen; no bump needed). sd-cli writes PNG frames and ffmpeg encodes an H.264 MP4.
• Weights are pinned to repo commits and SHA-256, all Apache-2.0, 5.43 GB in total: Wan2.1-T2V-1.3B Q8_0 from off-grid-ai (converted with sd.cpp itself, 1.52 GB), UMT5-XXL encoder Q4_K_M from city96 (3.66 GB), and the Wan 2.1 VAE from Comfy-Org (0.25 GB). The sd.cpp runtime is shared with CPU images (new runtime marker), and one sd-cli job runs per node across images and clips.
• Caps are 33 frames, 480x272 (or 272x480), 20 steps, text-to-video only. It bids only while it has the RAM, the 16-thread job budget, low enough load and ffmpeg, re-checked on every offer.
• It is opt-in because a clip ties up 16 threads for ~20+ minutes: c0mpute video setup --cpu, or [video] cpu = true / C0MPUTE_VIDEO_CPU=1 to install in the background. C0MPUTE_VIDEO_CPU=0 turns it off. c0mpute video check reports both paths.
• Measured on dev2 (production, load 36-84 on 32 cores, 16-CPU capped container), 480x272x33 frames: sampling takes 52-93 s per step, so 20 steps is 20-30 minutes, and the UMT5 conditioning takes 9-35 s. RSS is 5.3 GB with every weight in RAM and 1.8 GB with --params-backend te=disk,vae=disk at the same per-step speed; that setting is what ships. All-disk was also 1.8 GB but more than twice as slow. No full clip finished on dev2. earlyoom killed all four runs (at steps 9, ~10 and 13 of 20, and the short 17-frame test) because host memory dipped below 10%; it also hit the music worker in the same window. So the VAE decode and MP4 encode have not run on real output here, and CPU video is not enabled on dev2 by this PR.

Also

• /plugins shows a model's CPU option ("or CPU: ...") and the new hardware text.
• v0.2.90.

Tests

• cargo test -p c0mpute-core (video, video_cpu, diffusion, config, capabilities, gateway e2e)
• python3 plugins/video/runner/test_runner.py

🤖 Generated with Claude Code

[github-actions]

ThreatCrush Security Scan

13 finding(s)

HIGH/CRITICAL: 1 | MEDIUM: 8 | LOW: 4

Severity	Rule	Location
HIGH	js-ssrf-outbound-request	apps/web/public/sw.js:24
MEDIUM	js-unescaped-html-sink	apps/web/src/app/blog/[slug]/page.tsx:53
MEDIUM	js-unescaped-html-sink	apps/web/src/app/layout.tsx:80
MEDIUM	sh-remote-script-execution	scripts/dev-setup.sh:25
MEDIUM	sh-eval-expansion	scripts/dev-setup.sh:35
MEDIUM	sh-remote-script-execution	scripts/install.sh:181
MEDIUM	sh-remote-script-execution	scripts/install.sh:206
MEDIUM	sh-remote-script-execution	scripts/install.sh:324
MEDIUM	sh-remote-script-execution	scripts/install.sh:366
LOW	py-ssrf-outbound-request	examples/business-strategist-agent/chat.py:37
LOW	py-ssrf-outbound-request	examples/business-strategist-agent/train.py:54
LOW	secret-generic-credential	node/crates/c0mpute-core/src/webhook.rs:1165
LOW	py-path-traversal	node/crates/c0mpute-core/tests/fixtures/fake-infernet-text:29

Snippets are redacted; ThreatCrush never prints matched credential material.