feat: add P-H3-Image support

README.md

@@ -73,6 +73,7 @@ API and command-line option may change frequently.***
     - [Ming-Image Design](./docs/ming_image.md)
     - [PixArt](./docs/pixart.md)
     - [Iris-3B](./docs/iris.md)
+    - [P-H3-Image](./docs/p_h3_image.md)
   - [Image Edit Models](./docs/edit.md)
     - [FLUX.1-Kontext-dev](./docs/kontext.md)
     - [Qwen Image Edit series](./docs/qwen_image_edit.md)
@@ -81,6 +82,7 @@ API and command-line option may change frequently.***
     - [Mage-Flow-Edit](./docs/mage_flow.md#image-editing)
     - [LLaDA-Image Edit](./docs/llada_image.md#image-editing)
     - [Fibo 2.0 Dev / Edit Turbo](./docs/fibo2.md#editing-and-turbo)
+    - [P-H3-Image](./docs/p_h3_image.md#image-editing)
   - Video Models
     - [Wan2.1/Wan2.2](./docs/wan.md)
     - [MiniMax-H3](./docs/minimax_h3.md)

docs/p_h3_image.md

@@ -0,0 +1,64 @@
+# P-H3-Image
+
+[P-H3-Image](https://huggingface.co/PrunaAI/P-H3-Image) is Pruna AI's image model built on
+[MiniMax-H3](./minimax_h3.md): MiniMax-H3's 33B transformer generating a single frame, a
+Qwen3-VL-4B text encoder whose hidden states a text connector maps to the transformer's text
+input, and a VAE with a single-frame image decoder. It does text-to-image and edits with one to
+eight images. It is guidance-distilled: there is no CFG and no negative prompt.
+
+## Model files
+
+Pass the components separately, in MiniMax-H3's original layout:
+
+- `--diffusion-model`: the MiniMax-H3 transformer
+- `--llm`: the Qwen3-VL-4B text encoder with the text connector's tensors under `text_connector.`
+- `--vae`: the VAE with the single-frame image decoder
+
+The text connector is what identifies P-H3-Image: the same transformer without it loads as
+MiniMax-H3.
+
+The diffusers checkpoint in PrunaAI/P-H3-Image does not load directly.
+
+### Download weights
+
+- gguf: https://huggingface.co/SearchingMan/P-H3-Image-GGUF/tree/main
+    - `p_h3_image-Q4_K_M.gguf`, `p_h3_image_text_encoder-Q8_0.gguf`, `p_h3_image_vae-F16.gguf`
+
+## Text-to-image
+
+```sh
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\p_h3_image-Q4_K_M.gguf --llm ..\models\p_h3_image_text_encoder-Q8_0.gguf --vae ..\models\p_h3_image_vae-F16.gguf -p "A German Expressionist oil painting of a Berlin street at night in the manner of Ernst Ludwig Kirchner, with gaunt figures hurrying past a glowing gas lamp." -W 2048 -H 2048 --diffusion-fa -v
+```
+
+The model works best at 8 megapixels (2912x2912, 3360x2496, 3872x2176); 2 and 4 megapixels
+show more artifacts in faces, hands and small details. Width and height are rounded to
+multiples of 32.
+
+Sampling uses the checkpoint's 8-step sigma grid, `1.0, 0.984753, 0.967905, 0.951688, 0.925372,
+0.881923, 0.810083, 0.528718, 0.0`, whatever `--steps` says; pass `--sigmas` to sample another
+schedule.
+
+## Image editing
+
+Pass the image to edit, then up to seven more references, with `-r`. The prompt is the edit
+instruction; it is encoded after `Edit <Picture 1>. ` unless it already starts with that or with
+`subject_definitions:`. Refer to the other images as `<Picture 2>` and so on.
+
+```sh
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\p_h3_image-Q4_K_M.gguf --llm ..\models\p_h3_image_text_encoder-Q8_0.gguf --vae ..\models\p_h3_image_vae-F16.gguf -r street.png -p "Make it a snowy winter night: heavy snow falls across the whole scene, the pavement and rooftops are covered in fresh white snow, and the figures wear snow on their hats and shoulders. Keep the painting style, composition, figures and lamp unchanged." -W 2048 -H 2048 --diffusion-fa -v
+```
+
+Each image keeps its own size, scaled into 1 to 8 megapixels; the text encoder sees it at about
+one megapixel. `--ref-image-args` size settings do not apply. Edits work best at 4 or 8 megapixels and with an instruction that describes the
+result, including what to keep: a bare "let it snow" on a 1 megapixel canvas comes out
+oversaturated with little snow, while a full description on a 2048x2048 canvas edits cleanly.
+
+## Notes
+
+- The reference pipeline rewrites every prompt with a Qwen3.5-9B prompt upsampler first. sd.cpp
+  renders the prompt as given, so detailed prompts work best.
+- `generate_image()` runs P-H3-Image through the MiniMax-H3 video pipeline with one frame;
+  `generate_video()` with `--video-frames 1` does the same.
+- P-H3-Image is licensed under the MiniMax H3 Community License, which excludes the European
+  Union, the United Kingdom, the Republic of Korea and the United States unless MiniMax grants a
+  separate license. Its Qwen-derived parts are under Apache 2.0.

examples/server/recipes/defaults.md

@@ -73,6 +73,7 @@ For community edit adapters and other optional extensions, these are the base-mo
 | `moss-image-sft` | 3 | 20 | 1024 × 1024 | [Source](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT)  |
 | `nitrosd-realism` | 1 | 1 | 1024 × 1024 | [Source](https://huggingface.co/ChenDY/NitroFusion)  |
 | `ovis-image` | 5 | 20 | 1024 × 1024 | [Source](https://huggingface.co/ATH-MaaS/Ovis-Image-7B)  |
+| `p-h3-image` | 1 | 8 | 1024 × 1024 | [Source](https://huggingface.co/PrunaAI/P-H3-Image) Guidance-distilled; its fixed 8-step sigma grid is applied on model load. The source recommends 8 MP (default 2912 × 2912); edits work best at 4 or 8 MP. |
 | `pid` | 1 | 4 | 2048 × 2048 | [Source](https://huggingface.co/nvidia/PiD) Standalone PiD decoding at 2K; guidance is handled by the upstream generator. |
 | `pid-15` | 1 | 4 | 2048 × 2048 | [Source](https://huggingface.co/nvidia/PiD) Standalone PiD decoding at 2K; guidance is handled by the upstream generator. |
 | `pid-15-flux2` | 1 | 4 | 2048 × 2048 | [Source](https://huggingface.co/nvidia/PiD) Standalone PiD decoding at 2K; guidance is handled by the upstream generator. |

examples/server/recipes/p-h3-image.json

@@ -0,0 +1,38 @@
+{
+  "schema_version": 1,
+  "id": "p-h3-image",
+  "name": "P-H3-Image",
+  "size_multiple": 32,
+  "description": "Text-to-image, or editing with one to eight reference images; the first is the image to edit. Uses the text encoder with its text connector.",
+  "model_url": "https://huggingface.co/PrunaAI/P-H3-Image",
+  "slots": [
+    {
+      "id": "diffusion_model",
+      "required": true
+    },
+    {
+      "id": "llm",
+      "required": true,
+      "label": "P-H3-Image Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "vae",
+      "required": true,
+      "label": "P-H3-Image VAE"
+    }
+  ],
+  "defaults": {
+    "cfg_scale": 1,
+    "steps": 8,
+    "width": 1024,
+    "height": 1024,
+    "sampling_method": "euler"
+  },
+  "features": [
+    "ref_images"
+  ],
+  "disabled_features": [
+    "img2img",
+    "mask_image"
+  ]
+}

src/conditioning/conditioner.hpp

@@ -1971,11 +1971,102 @@ struct AnimaConditioner : public Conditioner {
     }
 };
 
+struct PH3ImageTextConnectorConfig {
+    int64_t in_channels  = 5 * 2560;
+    int64_t out_channels = 5120;
+
+    static PH3ImageTextConnectorConfig detect_from_weights(const String2TensorStorage& tensor_storage_map,
+                                                           const std::string& prefix) {
+        PH3ImageTextConnectorConfig config;
+        auto proj = tensor_storage_map.find(prefix + ".proj.weight");
+        if (proj == tensor_storage_map.end()) {
+            return config;
+        }
+        config.in_channels  = proj->second.ne[0];
+        config.out_channels = proj->second.ne[1];
+        LOG_VERBOSE("p_h3_image text connector: in_channels = %" PRId64 ", out_channels = %" PRId64,
+                    config.in_channels,
+                    config.out_channels);
+        return config;
+    }
+};
+
+// P-H3-Image's map from five Qwen3-VL hidden states to the channels MiniMax-H3's transformer
+// reads: standardize, project, destandardize. Ref: diffusers PH3ImageTextConnector.
+struct PH3ImageTextConnector : public GGMLBlock {
+    PH3ImageTextConnectorConfig config;
+
+    explicit PH3ImageTextConnector(const PH3ImageTextConnectorConfig& config)
+        : config(config) {
+        blocks["proj"] = std::make_shared<Linear>(config.in_channels, config.out_channels, false);
+    }
+
+    void init_params(ggml_context* ctx,
+                     const String2TensorStorage& tensor_storage_map = {},
+                     const std::string prefix                       = "") override {
+        SD_UNUSED(tensor_storage_map);
+        SD_UNUSED(prefix);
+        params["input_mean"]     = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, config.in_channels);
+        params["input_std"]      = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, config.in_channels);
+        params["output_mean"]    = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, config.out_channels);
+        params["output_std"]     = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, config.out_channels);
+        params["sink_embedding"] = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, config.out_channels);
+    }
+
+    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) {
+        auto proj = std::dynamic_pointer_cast<Linear>(blocks["proj"]);
+        x         = ggml_div(ctx->ggml_ctx, ggml_sub(ctx->ggml_ctx, x, params["input_mean"]), params["input_std"]);
+        x         = proj->forward(ctx, x);
+        x         = ggml_add(ctx->ggml_ctx, ggml_mul(ctx->ggml_ctx, x, params["output_std"]), params["output_mean"]);
+        // The first token is an attention sink, conditioned the same for every prompt.
+        auto sink = ggml_reshape_2d(ctx->ggml_ctx, params["sink_embedding"], config.out_channels, 1);
+        if (x->ne[1] == 1) {
+            return sink;
+        }
+        return ggml_concat(ctx->ggml_ctx, sink, ggml_ext_slice(ctx->ggml_ctx, x, 1, 1, x->ne[1]), 1);
+    }
+};
+
+struct PH3ImageTextConnectorRunner : public GGMLRunner {
+    PH3ImageTextConnectorConfig config;
+    PH3ImageTextConnector model;
+
+    PH3ImageTextConnectorRunner(ggml_backend_t backend,
+                                const String2TensorStorage& tensor_storage_map      = {},
+                                const std::string& prefix                           = "",
+                                std::shared_ptr<RunnerWeightManager> weight_manager = nullptr)
+        : GGMLRunner(backend, weight_manager),
+          config(PH3ImageTextConnectorConfig::detect_from_weights(tensor_storage_map, prefix)),
+          model(config) {
+        model.init(params_ctx, tensor_storage_map, prefix);
+    }
+
+    std::string get_desc() override {
+        return "p_h3_image_text_connector";
+    }
+
+    void get_param_tensors(std::map<std::string, ggml_tensor*>& tensors, const std::string& prefix) {
+        model.get_param_tensors(tensors, prefix);
+    }
+
+    sd::Tensor<float> compute(int n_threads, const sd::Tensor<float>& hidden_states) {
+        auto get_graph = [&]() -> ggml_cgraph* {
+            ggml_cgraph* gf = ggml_new_graph(compute_ctx);
+            auto runner_ctx = get_context();
+            ggml_build_forward_expand(gf, model.forward(&runner_ctx, make_input(hidden_states)));
+            return gf;
+        };
+        return restore_trailing_singleton_dims(GGMLRunner::compute(get_graph, n_threads, true),
+                                               hidden_states.dim());
+    }
+};
+
 struct LLMEmbedder : public Conditioner {
     SDVersion version;
     std::shared_ptr<Tokenizer> tokenizer;
     std::shared_ptr<LLM::LLMRunner> llm;
     std::shared_ptr<T5Runner> byt5;
+    std::shared_ptr<PH3ImageTextConnectorRunner> text_connector;
 
     LLMEmbedder(ggml_backend_t backend,
                 const String2TensorStorage& tensor_storage_map      = {},
@@ -2046,6 +2137,12 @@ struct LLMEmbedder : public Conditioner {
                 tokenizer = std::make_shared<Qwen2Tokenizer>();
             }
         }
+        if (sd_version_is_p_h3_image(version)) {
+            text_connector = std::make_shared<PH3ImageTextConnectorRunner>(backend,
+                                                                           tensor_storage_map,
+                                                                           "text_encoders.llm.text_connector",
+                                                                           weight_manager);
+        }
         if (sd_version_is_hunyuan_video(version)) {
             const std::string byt5_prefix = "text_encoders.t5xxl.transformer";
             for (const auto& [name, _] : tensor_storage_map) {
@@ -2066,6 +2163,9 @@ struct LLMEmbedder : public Conditioner {
         if (byt5) {
             byt5->get_param_tensors(tensors, "text_encoders.t5xxl.transformer");
         }
+        if (text_connector) {
+            text_connector->get_param_tensors(tensors, "text_encoders.llm.text_connector");
+        }
     }
 
     void get_param_tensor_ops(std::map<ggml_tensor*, enum ggml_op>& tensor_ops) override {
@@ -2077,6 +2177,9 @@ struct LLMEmbedder : public Conditioner {
         if (byt5) {
             byt5->set_max_graph_vram_bytes(max_vram_bytes);
         }
+        if (text_connector) {
+            text_connector->set_max_graph_vram_bytes(max_vram_bytes);
+        }
     }
 
     void set_runtime_backends(const std::vector<ggml_backend_t>& backends) override {
@@ -2141,6 +2244,9 @@ struct LLMEmbedder : public Conditioner {
         if (byt5) {
             byt5->runner_end();
         }
+        if (text_connector) {
+            text_connector->runner_end();
+        }
     }
 
     std::tuple<std::vector<int>, std::vector<float>, std::vector<float>> tokenize(std::string text,
@@ -2459,6 +2565,10 @@ struct LLMEmbedder : public Conditioner {
             prompt_template_encode_start_idx = 0;
             out_layers                       = {50};
             prompt_attn_range                = {0, 0};
+            if (text_connector) {
+                // hidden_states 13, 19, 25, 31 and the final normed one, as the connector was trained on.
+                out_layers = {13, 19, 25, 31, static_cast<int>(llm->config.num_layers) + 1};
+            }
 
             if (llm->enable_vision) {
                 const int patch_size = llm->config.vision.patch_size;
@@ -2469,14 +2579,30 @@ struct LLMEmbedder : public Conditioner {
                     int width  = static_cast<int>(image.shape()[0]);
                     int h_bar  = std::max(factor, static_cast<int>(std::round(static_cast<double>(height) / factor)) * factor);
                     int w_bar  = std::max(factor, static_cast<int>(std::round(static_cast<double>(width) / factor)) * factor);
-                    resize_image_dims(height,
-                                      width,
-                                      h_bar,
-                                      w_bar,
-                                      factor,
-                                      3136,
-                                      12845056,
-                                      RefImageResizeMode::AREA);
+                    if (sd_version_is_p_h3_image(version)) {
+                        // About one megapixel at the image's aspect ratio, as P-H3-Image's vision_pixels;
+                        // a size already on the grid and within 20% of that is kept.
+                        constexpr double vision_pixels = 1000000.0;
+                        double area                    = static_cast<double>(width) * height;
+                        if (width % factor != 0 || height % factor != 0 ||
+                            area < 0.8 * vision_pixels || area > 1.2 * vision_pixels) {
+                            double ratio = static_cast<double>(width) / height;
+                            w_bar        = std::max(factor, static_cast<int>(std::round(std::sqrt(vision_pixels * ratio) / factor)) * factor);
+                            h_bar        = std::max(factor, static_cast<int>(std::round(std::sqrt(vision_pixels / ratio) / factor)) * factor);
+                        } else {
+                            w_bar = width;
+                            h_bar = height;
+                        }
+                    } else {
+                        resize_image_dims(height,
+                                          width,
+                                          h_bar,
+                                          w_bar,
+                                          factor,
+                                          3136,
+                                          12845056,
+                                          RefImageResizeMode::AREA);
+                    }
                     auto resized = sd::ops::interpolate(
                         image,
                         std::vector<int64_t>{w_bar, h_bar, image.shape()[2], image.shape()[3]});
@@ -2564,6 +2690,15 @@ struct LLMEmbedder : public Conditioner {
                     }
                 }
             }
+            if (sd_version_is_p_h3_image(version) && !image_embeds.empty()) {
+                // An edit names the image it changes, unless the prompt already does (as the
+                // reference prompt upsampler's edit briefs do).
+                const size_t start = conditioner_params.text.find_first_not_of(" \t\n\r");
+                const auto text    = start == std::string::npos ? std::string() : conditioner_params.text.substr(start);
+                if (!starts_with(text, "Edit <Picture 1>.") && !starts_with(text, "subject_definitions:")) {
+                    prompt += "Edit <Picture 1>. ";
+                }
+            }
             prompt += conditioner_params.text;
         } else if (sd_version_is_hunyuan_video(version)) {
             prompt_template_encode_start_idx = 98;
@@ -3227,6 +3362,12 @@ struct LLMEmbedder : public Conditioner {
         if (hidden_states.empty()) {
             return {};
         }
+        if (text_connector) {
+            hidden_states = text_connector->compute(n_threads, hidden_states);
+            if (hidden_states.empty()) {
+                return {};
+            }
+        }
         std::vector<sd::Tensor<float>> extra_hidden_states_vec;
         if (sd_version_is_hunyuan_video(version) && byt5) {
             std::vector<std::string> quoted_texts;

src/model.h

@@ -70,6 +70,7 @@ enum SDVersion {
     VERSION_MOSS_IMAGE,
     VERSION_FIBO2,
     VERSION_CHROMA,
+    VERSION_P_H3_IMAGE,
     VERSION_COUNT,
 };
 
@@ -165,8 +166,13 @@ static inline bool sd_version_is_ltxav(SDVersion version) {
     return false;
 }
 
+// P-H3-Image is MiniMax-H3 generating one frame, so it takes every MiniMax-H3 path.
 static inline bool sd_version_is_minimax_h3(SDVersion version) {
-    return version == VERSION_MINIMAX_H3;
+    return version == VERSION_MINIMAX_H3 || version == VERSION_P_H3_IMAGE;
+}
+
+static inline bool sd_version_is_p_h3_image(SDVersion version) {
+    return version == VERSION_P_H3_IMAGE;
 }
 
 static inline bool sd_version_is_wan(SDVersion version) {
@@ -303,7 +309,7 @@ static inline bool sd_version_supports_video_generation(SDVersion version) {
 }
 
 static inline bool sd_version_supports_image_generation(SDVersion version) {
-    return !sd_version_supports_video_generation(version);
+    return !sd_version_supports_video_generation(version) || sd_version_is_p_h3_image(version);
 }
 
 static inline bool sd_version_uses_flux_vae(SDVersion version) {

src/model/integrations/minimax_h3.cpp

@@ -40,6 +40,12 @@ const std::vector<ModelDefinition>& minimax_h3_definitions() {
             definition.build                            = build_minimax_h3;
             result.push_back(std::move(definition));
         }
+        {
+            ModelDefinition definition = result.back();
+            definition.version         = VERSION_P_H3_IMAGE;
+            definition.name            = "P-H3-Image";
+            result.push_back(std::move(definition));
+        }
         return result;
     }();
     return definitions;