You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-ups from the independent audit of #30 (CLEAR at 0a865ab, merged as 1694916). None block; evidence in llamadart-evidence/sd-native-pr-30/.
E5M2 not covered by tests/test_fp8_runtime.py. Dropping E5M2 from the patch's type check passes the test, yet that build aborts on an E5M2 model on MTL0. Add an E5M2 copy to the test.
auto-fit under-plans FP8 on Metal.backend_fit.cppestimate_components uses ts.nbytes() (1 byte/elem) while Metal now holds FP8 weights as F16 (2 bytes). sd_turbo-fp8-all: plan 1244 MiB, Metal registers 2583 MB. Risk: large FP8 checkpoints on memory-tight iOS devices. Options: account for the retype in the estimate, or cast in-graph with the existing FP8→F16 copy kernel to keep 1 byte/elem. Document the doubled memory meanwhile.
Retirement test is heuristic. It misses an upstream exclusion through a helper (e.g. ggml_is_type_fp8) or a kernel named …_fp8_…; cost is only carrying the patch longer.
Pre-existing, not caused by [0.17] Load FP8 weights as F16 on Metal instead of aborting #30 (same on unpatched builds), synthetic LoRA: LoRA immediate mode aborts on Metal even with an F16 base ("pre-allocated tensor … cannot run the operation", ggml-backend.cpp:1089); an FP8 base with a LoRA segfaults on CPU (immediate, and at-runtime with FP8 lora_down).
Activity
fix: fall back from immediately LoRA on Metal whatever the stored wei…
test: skip the Metal module LoRA test where Metal cannot generate
fix: apply LoRAs at runtime where a merge crashes, and size FP8 on Me…