Add EdenLive

docs/source/en/_toctree.yml

@@ -349,6 +349,8 @@
         title: DiTTransformer2DModel
       - local: api/models/easyanimate_transformer3d
         title: EasyAnimateTransformer3DModel
+      - local: api/models/edenlive_transformer_3d
+        title: EdenLiveTransformer3DModel
       - local: api/models/ernie_image_transformer2d
         title: ErnieImageTransformer2DModel
       - local: api/models/flux2_transformer
@@ -729,6 +731,8 @@
         title: Cosmos
       - local: api/pipelines/cosmos3
         title: Cosmos3
+      - local: api/pipelines/edenlive
+        title: EdenLive
       - local: api/pipelines/framepack
         title: Framepack
       - local: api/pipelines/helios
@@ -798,6 +802,8 @@
       title: DPMSolverSDEScheduler
     - local: api/schedulers/singlestep_dpm_solver
       title: DPMSolverSinglestepScheduler
+    - local: api/schedulers/edenlive
+      title: EdenLiveFlowMatchEulerDiscreteScheduler
     - local: api/schedulers/edm_multistep_dpm_solver
       title: EDMDPMSolverMultistepScheduler
     - local: api/schedulers/edm_euler

docs/source/en/api/models/edenlive_transformer_3d.md

@@ -0,0 +1,47 @@
+<!-- Copyright 2026 The VAST-AI Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+-->
+
+# EdenLiveTransformer3DModel
+
+[EdenLive](https://github.com/VAST-AI-Research/EdenLive) uses a video diffusion transformer conditioned on
+text, a reference image, and DINO proxy features projected inside each transformer forward pass. The same model
+class supports bidirectional and causal attention. See the [pipeline guide](../pipelines/edenlive) for video generation.
+
+```python
+import torch
+from diffusers import EdenLiveTransformer3DModel
+
+transformer = EdenLiveTransformer3DModel.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers", subfolder="transformer", torch_dtype=torch.bfloat16
+)
+```
+
+For causal weights, load `VAST-AI/EdenLive-5B-Causal-Diffusers` or `VAST-AI/EdenLive-14B-Causal-Diffusers` with the same
+`subfolder="transformer"`. Keep model timesteps in FP32 when calling the transformer directly, including when its
+weights are BF16. Pass raw DINO patch features as `proxy_features` with shape `(batch, proxy_input_channels, frames,
+height, width)`; `proxy_channels` sets the number of projected channels. The internal projection remains in FP32
+when loaded with `from_pretrained`.
+
+## EdenLiveTransformer3DModel
+
+[[autodoc]] EdenLiveTransformer3DModel
+    - forward
+    - create_causal_state
+
+## EdenLiveCausalState
+
+The causal pipeline manages this state automatically. For custom loops, create a separate state for each CFG branch
+with `transformer.create_causal_state()`. Denoise with `cache_update=False`, then commit each clean chunk at timestep
+zero with `cache_update=True`. Call `reset()` before an independent video.
+
+[[autodoc]] EdenLiveCausalState
+    - reset

docs/source/en/api/pipelines/edenlive.md

@@ -0,0 +1,136 @@
+<!-- Copyright 2026 The VAST-AI Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+-->
+
+# EdenLive
+
+[EdenLive](https://github.com/VAST-AI-Research/EdenLive) generates video from a proxy video, a reference
+image, and a text prompt. [`EdenLivePipeline`] supports bidirectional generation; [`EdenLiveCausalPipeline`]
+generates a complete video in causal chunks and supports changing prompts between chunks.
+
+## Basic usage
+
+Prepare an RGB reference image and a proxy video with at least 49 frames sampled at 24 fps. The image provides
+appearance, while the proxy provides motion and scene structure.
+
+```python
+import torch
+from diffusers import EdenLivePipeline
+from diffusers.utils import export_to_video, load_image, load_video
+
+pipe = EdenLivePipeline.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers",
+    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
+).to("cuda")
+
+image = load_image("reference.png")
+video = load_video("proxy-24fps.mp4")
+
+frames = pipe(
+    image=image,
+    video=video,
+    prompt="A character explores a snowy landscape.",
+    num_frames=49,
+).frames[0]
+export_to_video(frames, "edenlive.mp4", fps=24)
+```
+
+Keep the proxy encoder in FP32 when loading the other models in BF16, as shown above. The transformer keeps its
+internal proxy projection in FP32 when loaded with `from_pretrained`.
+
+## Causal generation
+
+Load the causal pipeline from its dedicated repository. Both model sizes use four fixed denoising steps per chunk.
+
+```python
+from diffusers import EdenLiveCausalPipeline
+
+causal_pipe = EdenLiveCausalPipeline.from_pretrained(
+    "VAST-AI/EdenLive-5B-Causal-Diffusers",
+    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
+).to("cuda")
+```
+
+Use `prompt` for fixed text, or `chunk_prompts` to change it during a single video. At the default chunk size,
+49 frames require five prompts. Reusing the image and video above:
+
+```python
+frames = causal_pipe(
+    image=image,
+    video=video,
+    chunk_prompts=[
+        "A character walks through snow.",
+        "A character walks through snow.",
+        "The character approaches a campfire.",
+        "The character warms their hands beside the campfire.",
+        "The character rests beside the campfire.",
+    ],
+    num_frames=49,
+).frames[0]
+export_to_video(frames, "chunk-prompts.mp4", fps=24)
+```
+
+`chunk_prompts` overrides a single `prompt` and cannot be combined with `prompt_embeds`. The negative prompt remains
+fixed. A list passed as `prompt` represents a batch of independent videos. Each pipeline call starts a new video.
+
+## LightTAE decoder
+
+The full Wan VAE is the default. For faster decoding, pass the optional [`EdenLiveLightTAEDecoder`] to either pipeline:
+
+```python
+from diffusers import EdenLiveLightTAEDecoder
+
+pipe = EdenLivePipeline.from_pretrained(
+    "VAST-AI/EdenLive-5B-Diffusers",
+    decoder=EdenLiveLightTAEDecoder.from_pretrained(
+        "VAST-AI/EdenLive-5B-Diffusers", subfolder="decoder", torch_dtype=torch.bfloat16
+    ),
+    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
+).to("cuda")
+```
+
+## Notes
+
+| Checkpoint | Pipeline | Default height × width | Example fps | Steps |
+|---|---|---|---|---|
+| [5B bidirectional](https://huggingface.co/VAST-AI/EdenLive-5B-Diffusers) | `EdenLivePipeline` | 704 × 1280 | 24 | 48 |
+| [14B bidirectional](https://huggingface.co/VAST-AI/EdenLive-14B-Diffusers) | `EdenLivePipeline` | 480 × 832 | 16 | 30 |
+| [5B causal](https://huggingface.co/VAST-AI/EdenLive-5B-Causal-Diffusers) | `EdenLiveCausalPipeline` | 704 × 1280 | 24 | 4 per chunk |
+| [14B causal](https://huggingface.co/VAST-AI/EdenLive-14B-Causal-Diffusers) | `EdenLiveCausalPipeline` | 480 × 832 | 16 | 4 per chunk |
+
+- Sample proxy frames at the desired output frame rate. `num_frames` must be $4k+1$ and fit within the input video;
+  omitting it uses the largest valid prefix. Custom dimensions must be multiples of 32 for 5B or 16 for 14B.
+- Guidance defaults to 3 for bidirectional generation and 1 for causal generation. Set `guidance_scale` above one
+  to use a `negative_prompt`.
+- For CPU offload, load without `.to("cuda")` and call `enable_model_cpu_offload()`.
+- See the [example README](https://github.com/huggingface/diffusers/tree/main/examples/edenlive) for installation,
+  video resampling, CPU offload, and checkpoint conversion.
+
+## EdenLivePipeline
+
+[[autodoc]] EdenLivePipeline
+    - all
+    - __call__
+
+## EdenLiveCausalPipeline
+
+[[autodoc]] EdenLiveCausalPipeline
+    - all
+    - __call__
+
+## EdenLiveLightTAEDecoder
+
+[[autodoc]] EdenLiveLightTAEDecoder
+    - decode
+
+## EdenLivePipelineOutput
+
+[[autodoc]] pipelines.edenlive.pipeline_output.EdenLivePipelineOutput

docs/source/en/api/schedulers/edenlive.md

@@ -0,0 +1,27 @@
+<!-- Copyright 2026 The VAST-AI Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+-->
+
+# EdenLiveFlowMatchEulerDiscreteScheduler
+
+[`EdenLiveFlowMatchEulerDiscreteScheduler`] implements deterministic flow Euler sampling for EdenLive 14B
+bidirectional generation and stochastic sampling for the distilled causal checkpoints. The 5B bidirectional model
+uses [`UniPCMultistepScheduler`].
+
+Load the `scheduler` subfolder from the matching checkpoint repository. For causal generation, use
+`VAST-AI/EdenLive-5B-Causal-Diffusers` or `VAST-AI/EdenLive-14B-Causal-Diffusers`; the released causal schedules have
+four fixed steps per chunk. See the [pipeline guide](../pipelines/edenlive#causal-generation) for loading the pipeline.
+
+## EdenLiveFlowMatchEulerDiscreteScheduler
+
+[[autodoc]] EdenLiveFlowMatchEulerDiscreteScheduler
+    - set_timesteps
+    - step

examples/edenlive/README.md

@@ -0,0 +1,72 @@
+# EdenLive inference
+
+Generate video from a reference image, proxy video, and prompt. See the [pipeline guide](../../docs/source/en/api/pipelines/edenlive.md)
+for the Python API and model defaults.
+
+## Setup
+
+Install a CUDA-compatible [PyTorch](https://pytorch.org/get-started/locally/) and torchvision, then run from the
+Diffusers repository root:
+
+```bash
+pip install -e . "transformers>=4.56.0" accelerate sentencepiece ftfy av imageio imageio-ffmpeg
+```
+
+The [requirements file](requirements.txt) also lists the example dependencies.
+
+## Single GPU
+
+```bash
+python examples/edenlive/inference_edenlive.py \
+  --model VAST-AI/EdenLive-5B-Causal-Diffusers --causal \
+  --video proxy.mp4 --image reference.png \
+  --prompt "A character explores a snowy landscape." \
+  --num-frames 49
+```
+
+Use `--causal` with a causal repository, or omit it for a bidirectional repository. `--model` also accepts a local
+Diffusers directory. The script resamples the proxy video to the model's default frame rate or `--fps`.
+`--num-frames` must be $4k+1$ and fit within the resampled video; omit it to use the largest valid prefix.
+
+The default output is `edenlive.mp4`, with timing and input details in `edenlive.json`. Set `--output` to change it.
+Use `--lighttae` for faster decoding, `--vae-tiling` to reduce VAE memory use, or `--cpu-offload` to move models
+to the GPU only when needed. Run
+`python examples/edenlive/inference_edenlive.py --help` for all options.
+
+## Model repositories
+
+| Size | Bidirectional | Causal |
+|---|---|---|
+| 5B | [EdenLive-5B-Diffusers](https://huggingface.co/VAST-AI/EdenLive-5B-Diffusers) | [EdenLive-5B-Causal-Diffusers](https://huggingface.co/VAST-AI/EdenLive-5B-Causal-Diffusers) |
+| 14B | [EdenLive-14B-Diffusers](https://huggingface.co/VAST-AI/EdenLive-14B-Diffusers) | [EdenLive-14B-Causal-Diffusers](https://huggingface.co/VAST-AI/EdenLive-14B-Causal-Diffusers) |
+
+Each repository contains one complete pipeline. Select the matching repository and pipeline variant.
+
+## Changing prompts
+
+With `--causal`, add `--chunk-prompts prompts.json`. For 49 frames, supply five prompts:
+
+```json
+[
+  "A character walks through snow.",
+  "A character walks through snow.",
+  "The character approaches a campfire.",
+  "The character warms their hands beside the campfire.",
+  "The character rests beside the campfire."
+]
+```
+
+The list replaces `--prompt`, with one prompt per chunk. The negative prompt remains fixed.
+
+## Convert original checkpoints
+
+To convert a local original release instead of loading the published Diffusers weights:
+
+```bash
+python scripts/convert_edenlive_to_diffusers.py \
+  --source /path/to/EdenLive --size 5b --variant bidirectional \
+  --output /path/to/EdenLive-5B-Diffusers
+```
+
+Use `--size 14b` for 14B or `--variant causal` for causal generation, with a separate output directory for each
+combination. Each package contains one sampling variant and the optional LightTAE decoder.