Are LORAs for Ideogram4 supported? I mean are LORAs in the prompt applied to both parts of the diffusion model ("main" and "unconditional"?)

[alex-ie]

The title. I have read LORAs need to be applied to both parts of Ideogram4 for proper results. Is it done automatically or not? TIA

[stduhpf]

The unconditional model is loaded with "model.diffusion_model.uncond." prefix (instead of "model.diffusion_model.").

For LoRA weights to be applied to the unconditionnal model, they need to have the ".uncond" part in the prefix.

If you're asking if the LoRA weights that target the main diffusion model are also applied to the unconditionnal model, then no. But if the LoRA model is made with duplicated weights targetting the uncond model, then they would be applied.

[alex-ie]

Am I correct to understand that for models consisting of two files, LoRA should consist of two files too to apply to both parts (lora1.gguf / lora1.uncond.gguf)? And one of LoRA files should have suffix .uncond, and the prompt needs to include only one name : <lora:lora1:1>?

[stduhpf]

No, id didn't express myself properly.

"Inside" the lora File, there are named tensors. The names should match the tensors from the model they are apllied to.

For example if some model tensor is loaded as "model.diffusion_model.layers.0.attention.norm_q.weight", the corresponding LoRA tensors should be named something like "diffusion_model.layers.0.attention.norm_q.weight.lora_down" and "diffusion_model.layers.0.attention.norm_q.weight.lora_up".

And in the case of the unconditional model for ideogram, the model tensors name all start with "model.diffusion_model.uncond.", for example "model.diffusion_model.uncond.layers.0.attention.norm_q.weight", meaning the correspondingt tensors in the LoRA file should be named "diffusion_model.uncond.layers.0.attention.norm_q.weight.lora_down" and "diffusion_model.uncond.layers.0.attention.norm_q.weight.lora_up".

[helloe365]

Confirming from the source, since I looked into this recently: in src/model/diffusion/ideogram4.hpp the uncond model is initialized with uncond_prefix = prefix + ".uncond", so the unconditional transformer's tensors live under model.diffusion_model.uncond.* and nothing automatic copies LoRA weights across namespaces. @stduhpf's answer is right — main-model LoRA keys never touch the uncond branch.

So no, it's not done automatically, and it matters exactly in your case: you load the uncond transformer with --uncond-diffusion-model (per docs/ideogram4.md), which means the uncond side runs the base weights while the cond side gets your LoRA — half the CFG signal is coming from a different model than you trained.

If your LoRA only ships main-branch weights, the straightforward fix is duplicating the keys into the uncond namespace before use:

from safetensors.torch import load_file, save_file

sd = load_file("lora.safetensors")
out = dict(sd)
prefix = "model.diffusion_model."
for k, v in sd.items():
    if k.startswith(prefix):
        out["model.diffusion_model.uncond." + k[len(prefix):]] = v
save_file(out, "lora_with_uncond.safetensors")

Roughly doubles the file size in memory since every tensor exists twice, but that's the cost of the separate uncond model. If the LoRA was trained with the uncond keys already duplicated (some of the Ideogram4 ones are), check the key names first and skip this entirely.

If that gets the LoRA applying on both branches, marking the thread solved would help the next Ideogram4 user hunting for this.