[danielhanchen]

Summary

This merges upstream master-1050-58bb3c1 into the fork and ports the carried ggml patches onto the new ggml submodule commit (135ce1e0). The current Studio pin, master-982-0518b34-uaba3a1c, is 68 upstream commits behind. Among them are leejet#2183 (the model manager reads free memory again before it gives up, also vendored on its own in #50) and leejet#2273 (faster MOSS normalization and staging).

Why now

MOSS-Image-SFT Q8_0 with Qwen3.5-9B Q8_0 through sd-server, 1024x1024, 25 steps. One Colab L4 job, both arms on the same card, with VRAM held down to 7.5 GiB free at load. Each cell ran twice, and repeats agree within 1.7 s.

Free VRAM during the render	current pin (master-982-0518b34-uaba3a1c)	this branch (7c062d2, patches applied)
7.5 GiB, nothing taken after load	95.5 s / 93.9 s (sampling 82.4 s / 80.9 s)	73.2 s / 73.2 s (sampling 59.9 s / 59.7 s)
2.0 GiB, VRAM taken after load	148.7 s / 150.4 s	87.4 s / 87.8 s (sampling 72.9 s / 72.9 s)

So this branch is about 1.3x faster on the normal path and about 1.7x faster under memory pressure. Both renders are correct images.

In a separate job, a plain upstream build of the same base (58bb3c1, no fork patches) next to this branch took 89.5 s vs 101.7 s with nothing taken, and 89.9 s vs 89.5 s at 2 GiB. Sampling took 76.1 s vs 68.2 s and 75.3 s vs 74.7 s, so the ported patches cost MOSS nothing. At 1.5 GiB free, the pin, plain upstream and this branch all fail the same way (the text encoder segment needs 1543 MB, 1534 MB is free). That case is a real shortfall, not something an engine update fixes. unslothai/unsloth#16815 makes Studio report it as out of memory.

Changes

• 64fb9d4: merge of master-1050-58bb3c1. Conflicts were resolved keeping both sides (fork-only server and workflow pieces plus upstream's new code).
• d4ed8d3: upstream moves ROPE_APPLY, SAGE_ATTN, SOL_ATTN and QUANTIZE_I8_CONVROT out of ggml and into stable-diffusion.cpp backend extensions, so several patches in scripts/unsloth/ggml-patches no longer applied:
  • 0002 (long sequence flash attention): reuses upstream's new 128x128 tile for head size 128 and keeps only the head size 64 instance.
  • 0003, 0004: context and op enum position only.
  • 0006 (cuDNN attention): the SAGE_ATTN half is dropped together with the op. cuDNN now serves FLASH_ATTN_EXT only.
  • 0007 (rope_apply PDL race fix): dropped together with the op. The extension kernel that replaces it launches without PDL.
  • 0010 (Metal ROPE_APPLY kernel): dropped. The op no longer exists and ggml's extension registry has no Metal hook, so Metal runs the unfused rope graph again.
  • 0001, 0005, 0008, 0009 and 0011 to 0013 apply unchanged.
• 7c062d2: the prebuilt workflow now also fetches thirdparty/cutlass. Upstream's CUDA build includes cmake/cutlass.cmake, which stops at configure time without the CUTLASS headers. The source stamp only fetched ggml, so every CUDA leg would have failed. The step keeps only include/ and the license, which keeps the source tarball small.
• 6f12cd9: the macOS arm64 leg moves from macos-14 to macos-15. GitHub retired the macos-14 image, so that leg was cancelled before it started (The configuration 'macos-14-arm64-us-default' is not supported). master has the same runner label. CMAKE_OSX_DEPLOYMENT_TARGET=14.0 stays, so the binaries still run on macOS 14.

Testing

• Every patch applies on 135ce1e0. The CPU and CUDA builds of ggml (with test-backend-ops) and of sd-server succeed with the patches applied.
• CUDA source build of this branch on Colab (sm_89, CUTLASS headers from the submodule): the MOSS renders in the table above.
• CPU sd-cli render of MOSS-Image-SFT Q8_0 renders with the patches applied (18.2 s) and without them (66.8 s), image saved both times.

Not verified yet

• Artifact-only prebuilt run of 7c062d2 (https://github.com/unslothai/stable-diffusion.cpp/actions/runs/42157974628): every leg built except macOS arm64. That covers Linux CPU, Vulkan, aarch64, CUDA 12 and CUDA 13; Windows CPU, Vulkan, CUDA 12 and CUDA 13; and macOS x86_64. The CUDA legs were cold builds (new ggml commit plus the CUTLASS kernels, no ccache hits) and took 1 h 18 min to about 2 h 50 min, inside the job limits. macOS arm64 was cancelled because its runner image is retired; https://github.com/unslothai/stable-diffusion.cpp/actions/runs/42178497743 rebuilds it with 6f12cd9.
• MiniMax-H3 was not rendered on this tree.
• Metal: because 0010 was dropped, MiniMax-H3 on Apple Silicon may be slower until the kernel is ported to the new extension path.
• sm100: losing the cuDNN Sage path (0006) may cost MiniMax-H3 speed on Blackwell.

Please do not merge before the prebuilt run is green. Once it is merged and the nightly publishes, Studio's DEFAULT_TAG can move to the new tag, and #50 becomes redundant.