Deterministic SIGSEGV in sequence-checkpoint init on Metal for hybrid / SWA models (Qwen3.5, Gemma 3)

[jb-ao]

Description

AddonContextSequenceCheckpointInitWorker crashes the Node process with SIGSEGV on macOS/Metal whenever a model with needsCheckpoints === true (hybrid or sliding-window architecture) evaluates its first prompt. The crash is 100% reproducible for us across two unrelated model families and survives reboots; plain-attention models (Mistral family) never crash because the checkpoint path is never taken.

Environment

• node-llama-cpp 3.19.0 (prebuilt Metal binary, libllama.metal.b9842.dylib)
• macOS 26.5.1 (Darwin 25.5.0), Apple M1, 8 GB unified memory
• Node v22.17.1
• Models (both Q4_K_M GGUF):
  • unsloth/Qwen3.5-4B-GGUF → Qwen3.5-4B-Q4_K_M.gguf (hybrid attention)
  • bartowski/google_gemma-3-4b-it-GGUF → google_gemma-3-4b-it-Q4_K_M.gguf (sliding-window attention)

Reproduction

import { getLlama, LlamaChatSession } from 'node-llama-cpp';

const llama = await getLlama();
const model = await llama.loadModel({ modelPath: 'Qwen3.5-4B-Q4_K_M.gguf' });
const context = await model.createContext({ contextSize: 4096 });
const session = new LlamaChatSession({ contextSequence: context.getSequence() });
// Any prompt long enough for generation to start; crashes when the
// initial/interval checkpoint is taken during evaluation:
await session.promptWithMeta('<a few hundred tokens of text>', { temperature: 0.2, maxTokens: 400 });
// -> process dies with SIGSEGV, no JS error

Crashes every time (6/6 across the two models on our machine), including immediately after a clean reboot with ~0 swap in use, so it is not memory pressure.

Crash stacks (from macOS DiagnosticReports)

Qwen3.5-4B (hybrid) — KERN_INVALID_ADDRESS at 0x0000000000000250:

libllama.metal.b9842.dylib  llama_context::synchronize()
libllama.metal.b9842.dylib  llama_state_seq_get_data_ext
llama-addon.node            AddonContextSequenceCheckpointInitWorker::Execute()
llama-addon.node            Napi::AsyncWorker::OnExecute(Napi::Env)

Gemma-3-4B-it (SWA) — KERN_INVALID_ADDRESS in memmove:

libsystem_platform.dylib    _platform_memmove
libllama.metal.b9842.dylib  llama_io_write_host::~llama_io_write_host()
libllama.metal.b9842.dylib  llama_context::state_seq_get_data(int, unsigned char*, unsigned long, unsigned int)
llama-addon.node            AddonContextSequenceCheckpointInitWorker::Execute()
llama-addon.node            Napi::AsyncWorker::OnExecute(Napi::Env)

Full .ips crash reports available on request (six of them: three per model).

Analysis / notes

• LlamaContextSequence.needsCheckpoints returns true for isHybrid || isRecurrent and for SWA models without swaFullCache — matching exactly the two families that crash. _takeCheckpoint() then constructs AddonContextSequenceCheckpoint and its init() ends up in llama_state_seq_get_data(_ext), which appears to be unsafe for these KV-cache types on Metal in the bundled llama.cpp build (possibly related to upstream issues about seq-state save/restore for hybrid/recurrent and iSWA caches).
• Workaround that works for SWA models: createContext({ swaFullCache: true }) — needsCheckpoints goes false and everything runs normally (verified with a full 40-generation benchmark on the Gemma model above).
• No workaround found for hybrid models: needsCheckpoints is unconditionally true for them, and both the initial checkpoint in LlamaChat and interval checkpoints reach the crashing init, so Qwen3.5-family models are currently unusable with 3.19.0 on Metal for us.

Thanks for the great library — happy to attach crash reports or test a patched build.

[giladgd]

@jb-ao I just tried reproducing this issue on both M1 Max and M5 Max but didn't get any crash, it worked as expected.
Could you please share the result of running npx -y node-llama-cpp inspect gpu on your machine?

[sroussey]

Sounds like my issue:

┌──────────────────────┬─────────────────┬───────────────────┐
│        Model         │   Multimodal?   │      Result       │
├──────────────────────┼─────────────────┼───────────────────┤
│ qwen2.5-14b-instruct │ text-only       │ ✅ 4 gens clean   │
├──────────────────────┼─────────────────┼───────────────────┤
│ qwen2.5-0.5b         │ text-only       │ ✅ clean          │
├──────────────────────┼─────────────────┼───────────────────┤
│ Qwen3.5-9B           │ vision (mmproj) │ ❌ 0x250 on gen 2 │
├──────────────────────┼─────────────────┼───────────────────┤
│ gemma-4-12b-qat      │ vision          │ ❌ 0x250          │
├──────────────────────┼─────────────────┼───────────────────┤
│ Qwen3.6-35B-A3B      │ vision          │ ❌ crash          │
└──────────────────────┴─────────────────┴───────────────────┘

generally crashing on the second generation .

"the key finding so far stands: Qwen3.5-9B is fast and accurate (100% on the hard 34k section in 134s), but multimodal GGUFs trip a 0x250 native crash on the 2nd generation"

update: for me it is more intermittent. while vision models always crash on my second run of things, even if i run things one at a time, a see a crash.

┌──────────────────┬───────┬────────┬──────┬──────┬─────────┬─────────────────────────────────────────┐
│  Section (size)  │ agree │ recall │ prec │ rows │ latency │                 status                  │
├──────────────────┼───────┼────────┼──────┼──────┼─────────┼─────────────────────────────────────────┤
│ s1_1822912 (34k) │ 100%  │ 100%   │ 100% │ 5/5  │ 132s    │ ✅ (crash on teardown, result captured) │
├──────────────────┼───────┼────────┼──────┼──────┼─────────┼─────────────────────────────────────────┤
│ s1_1848507 (4k)  │ 100%  │ 100%   │ 100% │ 9/9  │ 114s    │ ✅ (crash on teardown, result captured) │
├──────────────────┼───────┼────────┼──────┼──────┼─────────┼─────────────────────────────────────────┤
│ s1_1849470 (43k) │ 100%  │ 100%   │ 100% │ 6/6  │ 257s    │ ✅ clean exit                           │
├──────────────────┼───────┼────────┼──────┼──────┼─────────┼─────────────────────────────────────────┤
│ s1_2114227 (56k) │ —     │ —      │ —    │ —    │ —       │ ❌ crashed mid-generation, no result    │
└──────────────────┴───────┴────────┴──────┴──────┴─────────┴─────────────────────────────────────────┘

Two things stand out:
- On the 3 sections that produced results, Qwen3.5-9B is a perfect 100/100/100 — and those are the exact three where your qwen2.5-14b-instruct had title misses (John Lewis "Secretary", Frank Martire "Founder", Haldeman "Director"). The 9B got all three right. So a 9B, ~2× faster, is beating your 14B on the sections both completed.
- The crash is now clearly intermittent teardown vs mid-gen: sections 1–2 printed results then crashed on dispose; section 3 exited cleanly; only section 4 (the 56k Churchill SPAC — largest) crashed during generation, losing its result.

edit 2: i tried on both bun and node and had the same crashes.

OS: macOS 25.5.0 (arm64)
Node: 24.10.0 (arm64)

node-llama-cpp: 3.19.0
Prebuilt binaries: b9842

Metal: available

Metal device: Apple M2 Max
Metal used VRAM: 0% (384KB/51.84GB)
Metal free VRAM: 99.99% (51.84GB/51.84GB)
Metal unified memory: 51.84GB (100%)

CPU model: Apple M2 Max
Math cores: 8
Used RAM: 82.61% (52.88GB/64GB)
Free RAM: 17.38% (11.12GB/64GB)
Wired RAM: 8.21% (5.26GB/64GB)
Used swap: 67.25% (2.69GB/4GB)
Max swap size: dynamic
mmap: supported