ConsID-Gen

View-Consistent and Identity-Preserving Image-to-Video Generation

Mingyang Wu¹   Ashirbad Mishra²   Soumik Dey²   Shuo Xing¹   Naveen Ravipati²
Hansi Wu²   Binbin Li²   Zhengzhong Tu^1†

¹Texas A&M University    ²eBay Inc.
^†Corresponding Author

Accepted by CVPR 2026

---

Updates

• Inference/training code, ConsIDVid dataset resources, and the ConsID-Gen model checkpoint are now available.
• 2026.02: Our paper is accepted by CVPR 2026.

Released Resources

Inference/training code, dataset resources, and model checkpoints are available below.

• ConsID-Gen inference / training code
• ConsIDVid public set: https://huggingface.co/datasets/mingyang-wu/ConsIDVid
• ConsIDVid proprietary set: files and extraction instructions
• Model checkpoint: https://huggingface.co/mingyang-wu/ConsID-Gen

Dataset Download

The ConsIDVid public set is hosted on Hugging Face. The proprietary set is provided separately in this repository, as linked on the project page.

pip install -U huggingface_hub
hf download "mingyang-wu/ConsIDVid" --repo-type dataset --local-dir "./datasets/ConsIDVid"

Code: ConsID-Gen + Wan2.1 (Inference & Training)

Setup Environment

conda create -n considgen python=3.10
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
pip install -r requirements.txt

Model Download

Use this section to download the model weights required by run_inference_considgen.py.

The ConsID-Gen checkpoint is hosted at mingyang-wu/ConsID-Gen. Download model.safetensors to models/ConsID-Gen/checkpoints/model.safetensors. Wan2.1 and VGGT weights are also required.

# 1) Install Hugging Face CLI
pip install -U huggingface_hub

# 2) Download Wan2.1-Fun-1.3B-InP
mkdir -p ./models/PAI/Wan2.1-Fun-1.3B-InP
hf download "alibaba-pai/Wan2.1-Fun-1.3B-InP" --local-dir "./models/PAI/Wan2.1-Fun-1.3B-InP"

# 3) Download VGGT-1B
mkdir -p ./models/VGGT-1B
hf download "facebook/VGGT-1B" --local-dir "./models/VGGT-1B"

# 4) Download ConsID-Gen
hf download "mingyang-wu/ConsID-Gen" "model.safetensors" --local-dir "./models/ConsID-Gen/checkpoints"

Google Drive mirror for ConsID-Gen checkpoints.

Inference Example

Run single-image conditioned generation with a finetuned checkpoint:

python run_inference_considgen.py \
  --input_image_path /path/to/input_image.jpg \
  --image_dir /path/to/multi_view_images_dir \
  --prompt "A product-style close-up video with stable lighting and clean background." \
  --output_dir ./tmp \
  --checkpoint_path models/ConsID-Gen/checkpoints/model.safetensors

Training

Prepare the training dataset.

The training metadata should be a JSON list. Each sample needs:

• video: path to the training video
• prompt: text description for the video
• image_list: list of multi-view reference image paths

Example metadata structure:

[
  {
    "dataset": "example_dataset_name",
    "video": "/path/to/video.mp4",
    "prompt": "A short text description of the target video.",
    "image_list": [
      "/path/to/view_1.jpg",
      "/path/to/view_2.jpg"
    ]
  }
]

CUDA_VISIBLE_DEVICES=3 python run_train_considgen.py \
  --dataset_metadata_path ./example_metadata.json \
  --model_paths '["models/PAI/Wan2.1-Fun-1.3B-InP/diffusion_pytorch_model.safetensors","models/PAI/Wan2.1-Fun-1.3B-InP/models_t5_umt5-xxl-enc-bf16.pth","models/PAI/Wan2.1-Fun-1.3B-InP/Wan2.1_VAE.pth","models/PAI/Wan2.1-Fun-1.3B-InP/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth"]' \
  --vggt_model_path models/VGGT-1B \
  --tokenizer_path models/PAI/Wan2.1-Fun-1.3B-InP/google/umt5-xxl \
  --trainable_models dit,considgen_adapter \
  --output_path models/ConsID-Gen/example \
  --num_epochs 1 \
  --dataset_num_workers 0 \
  --wandb_mode disabled

Citation

If you use the code or any part of the datasets, then please cite this paper in your work.

@InProceedings{Wu_2026_CVPR,
    author    = {Wu, Mingyang and Mishra, Ashirbad and Dey, Soumik and Xing, Shuo and Ravipati, Naveen and Wu, Hansi and Li, Binbin and Tu, Zhengzhong},
    title     = {ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video Generation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {1853-1863}
}