the whole-model Vulkan driver serves deltanet hybrids: Qwen3.5-9B prefill at parity with llama.cpp, decode at 0.965x, K-quant deltanet planes native, the SPIR-V emitter's lazy operands

[borisbat]

Behavior change: on a Vulkan box with the whole-model driver, a dense deltanet hybrid now loads its deltanet planes in the file's K-quant formats (no Q8_0 transcode) and mints a new image; the first load of such a model re-mints.

Why. Qwen3.5-9B on the Vulkan tier ran at 0.05x of llama.cpp's prefill and 0.2x of its decode: the whole-model driver declined every hybrid, and the per-op rails streamed 4 GB of FFN per token through the CPU. Two GPU hangs on the way came from the SPIR-V emitter lowering ?: eagerly.

What changes.

• The whole-model resident driver serves deltanet hybrids: the token command carries recurrent layers through a fused step over per-layer device state, the window chain prefills them (conv, a sequential scan with the state in registers, the out-norm), and gated attention with partial rotary rides the batch kernels.
• The cm2 flash-attention tile stamps at head 256 with a gated epilogue; the deltanet GEMMs ride the f16-fed cm2 tiles; the beta/alpha rows live on the device as f16; the scan keeps two state columns per lane.
• The deltanet planes stay in their file formats on this driver (the loader tags them natively, the plan and arena carry the formats); a K-quant out plane declines the layer; off the f16 feed a K-quant qkv/z pair reads the Q8_K activation form.
• The window chain keeps every window past the first at the conv taps, and a lone shorter window's ring rows read as zero; the scan's idle lanes read a clamped column; the deltanet state width must be a power of two (the lane grouping's assumption); a gated model takes the cm2 flash tiles at head 256 alone.
• The dasSpirv emitter lowers ?:, && and || lazily when an operand indexes a buffer (a hoisted temp and a branch); pure operands keep OpSelect.
• Vulkan storage buffers carry the device-address usage, allocation flag and query on the coopmat2 device alone (the other creators never enable the feature), and the coopmat2 device creator names the integer dot product on the 1.3 struct.
• The runtime copies a log message before its debug agents run, so a hook on the logging context keeps it whole.

Observable behavior.

• Qwen3.5-9B UD-Q5_K_XL, 5060 Ti, same-day llama-bench control 2484 / 53.7: pp512 132 -> 2526 (1.02x), tg128 11.1 -> 51.8 (0.965x).
• test_gpu_resident_hybrid: two fixtures (Q8_0 and Q4_K_M 0.8B) hold the forced-feed bar against the all-CPU chain over four prompt shapes (one window, two, one row past a window, two tokens); the parity harness reads 40/40 token-for-token on the Q8_0 file with the driver armed.
• A hybrid prompt one row past the window, or shorter than the conv taps, prefilled resident: the conv tail read past the window plane -> the two new cells hold the bar.
• DASLLAMA_COOPMAT=mm on a hybrid (the coopmat1 device, no device addresses): the buffer creation asked for an unenabled feature -> the same ten cells pass on that device.
• The 9B window under GPU-assisted validation logs no VUID (was 80 device-address reports and one device-create report) and lands the plain run's logits.
• qk_rms_cls no longer reads 12.6 MB past its binding (the hang); the kernel-unit suite gates the scan, the beta/alpha GEMM and the flash tile at the 9B's geometry, each cell with a NaN sentinel and a poison control.

Where to look. dasllama_vulkan_prefill.das (rd_pf_recurrent, pf_run's window loop, pf_facm2), dasllama_vulkan_classes.das (DnScan, DnBaGemm, DnTail, FaCm2H256T), dasllama_gpu_resident.das (the deltanet plane formats and declines), dasllama_vulkan_common.das (has_bda), spirv_emit.das (SpirvGlobalIndexScan, lazy_merge_value).

Validation, claims, ledger

Validation

• Parity harness on the Q4_K_M 0.8B: 19/40 with the driver armed, 36/40 on the CPU alone (freeform near-ties on a 4-bit 0.8B); its instrument is the forced-feed cells, which hold a 6% bar (the K-quant chain's own CPU-vs-device noise sits flat at ~4.5% of the max logit).
• The hang forensics ran under the Khronos layer's GPU-assisted validation in safe mode; the validated numerics became evidence only after the device-address usage landed.
• Local only: the hybrid cells under DASLLAMA_COOPMAT=mm (10/10 on the coopmat1 device), the 30B-A3B Q4_K_M row (ours 606 / 63.7 vs llama.cpp 593 / 40.7 on 9/5), the per-role window profiles in followup_vulkan.md item 2, and the Vulkan serving census on the qwen35 rows (the 0.8B and the new 9B UD row, DASLLAMA_PARITY_FULL=1): with the census's new f16 session pass the resident driver's deltanet chain, the tail, the beta/alpha GEMM, the f16 router GEMV and the gated f16 head-256 flash stamp all dispatch; the ungated head-256 stamps wait for an ungated head-256 carrier in the full census, and the gated f32-output twin for an out plane off the f16 feed.
• The review round: seven checklist audits, the TDD audit, the style audit and two codex rounds on the branch; every real finding is fixed in this diff, the checklist self-review repairs included.
• The local preflight's lanes on this Windows box: tests-cpp, tests-interp and docs green; tests-jit red on two master files (tests/jit_tests/cross_target_folds.das, wasm_idot_lowering.das) whose spawn opened with a quote beside a redirect, the Windows cmd quote-stripping trap - fixed here, both green after; utils-tests red on the ast-fuzz suite alone, which hardcodes bin/daslang and cannot spawn under the MSVC layout (CI-only, every other utils suite green); tests-aot red on tests/module_cache/test_default_cache_path.das alone, an AOT-host-only red outside this diff (its children print nothing under test_aot; the -jit lane runs the same file green; CI's per-PR AOT compiles tests/language only).
• The dasllama suites under DASLLAMA_GPU=1 on the 5060 Ti: model-free 72 files green (the one red, test_gpu_tier's "auto not requested" cell, reads the knob I exported for the stocked suite and passes 11/11 in CI shape); stocked 40 files with two reds fixed here - test_chat's render-seam cell byte-compares two chains that split between the device prefill and the CPU continuation under the armed tier, so it now drops the model's device state like the scheduler cells do, and test_exe_smoke's artifact exited with STATUS_DLL_NOT_FOUND because the exe links the producing tree's two runtime DLLs and the test ran it from a temp dir with no such dir on PATH, so the test now runs it with that daslang's bin dir on PATH.

Claims - stated, not tested

• The four columns per lane scan form measured 15.3 ms against 10.9 (64 workgroups starve 36 SMs) and was reverted; the two-column form is what ships.
• The tier's kernels assume a 32-lane subgroup (followup_vulkan.md item 40e); a wave64 device runs those arms unmeasured.
• A debug agent installed from a context that dies before the process (a dastest file's context) leaves a dangling adapter in the runtime's agent list; the logger hook's probe therefore runs in a child process.
• A gated model at head 64 or 128 now takes the batch attention kernels (the gated cm2 flash stamps exist at head 256 alone); no such model is stocked, so the pick is read, not run. A deltanet state width that is not a power of two (96 passes the old % 32 guard) now declines the resident driver and asserts on the per-op chain; no such model exists.

Not done

• followup_vulkan.md item 2: the k5/k6 cm2 tiles run the deltanet GEMMs slower than the q8 tile (5 ms of the 9B window); the fused add+rms+requant twin stays off on hybrids; the scheduler's head-of-line wait; per-session device slots for the deltanet step.
• followup_vulkan.md item 40: the coopmat load stride VUID on q8_batch_mm_a_cls, debug-utils names on class pipelines, the validation-layer exit wedge, the subgroup-32 assumption.
• followup_vulkan.md item 41: the kernel-unit file's device-absent cells feint instead of skipping.
• The board cell for the hybrid route waits for decode parity.

[Copilot]

🔵 Needs a closer look

It introduces broad, high-impact changes across Vulkan serving, SPIR-V emission semantics, runtime logging, and extensive test scaffolding that warrant final human validation of end-to-end correctness and GPU safety.

Pull request overview

This PR expands daScript’s Vulkan whole-model (GPU-resident) execution path to serve Qwen3.5-class DeltaNet hybrid models efficiently and safely, while tightening correctness and validation in the SPIR-V emitter, Vulkan device/buffer setup, and runtime logging hooks.

Changes:

• Enable the Vulkan whole-model resident driver to serve DeltaNet hybrids end-to-end (prefill + decode), including gated attention / partial rotary and device-side recurrent state handling.
• Keep DeltaNet planes in their on-disk K-quant formats when the whole-model driver will be attempted (avoids Q8_0 transcode) and propagate that choice through residency/image identity.
• Fix/guard GPU correctness hazards: lazy lowering for ?:, &&, || when operands index global-rooted buffers; enable/propagate buffer-device-address usage + allocation flags + address queries where required; ensure log messages survive debug-agent temp-string churn; add/extend test coverage.

File summaries

File	Description
tutorials/integration/cpp/class_adapters_module.das.inc	Regenerated embedded tutorial payload (no logic changes).
tests/spirv/test_ternary.das	Adds coverage for lazy operand lowering behavior.
tests/spirv/test_coopmat2_fa256.das	New SPIR-V validation test for coopmat2 head-256 flash-attention tile shapes.
tests/spirv/test_census.das	Extends opcode census to include new fixtures.
tests/spirv/_spirv_common.das	Adds new SPIR-V fixtures for lazy operands and coopmat2 FA256 kernel.
tests/jit_tests/wasm_idot_lowering.das	Windows quoting fix for popen-based child spawn.
tests/jit_tests/cross_target_folds.das	Windows quoting fix for popen-based child spawn.
tests/daslib/test_logger.das	Adds child-process-based test for logger hook at field formatting.
tests/daslib/_logger_hook_child.das	Child probe used by logger hook test.
tests-cpp/small/test_to_out_agent_temp_strings.das	Minimal script compiled/executed by new C++ test.
tests-cpp/small/test_to_out_agent_temp_strings.cpp	New C++ regression test for Context::to_out message stability across agent allocations.
src/runtime/context.cpp	Copies log message before invoking debug agents to prevent corruption.
src/parser/lex.yy.h	Regenerated lexer header (line directives).
src/parser/ds_lexer.cpp	Regenerated lexer output (line directives).
plans/vulkan_hybrid_ladder.md	New plan/arc doc capturing the hybrid ladder work.
modules/REVIEW_SHADER_EMITTERS.md	Adds review rule clarifying operand/global indexing expectations in shader bodies.
modules/dasVulkan/daslib/vulkan_boost.das	Fixes feature chaining: integer dot product feature placement under Vulkan 1.3 vs standalone struct.
modules/dasVulkan/ARCHITECTURE.md	Documents subgroup/full-subgroup pairing and dot-product feature chaining constraints.
modules/dasUnitTest/unit_test.das.inc	Regenerated embedded module payload (no logic changes).
modules/dasSpirv/spirv/spirv_emit.das	Implements lazy lowering for ?:, &&, `
modules/dasSpirv/spirv/spirv_builtins.das	Adds cooperative-matrix marker types and mul-add signatures for head-256 tiles.
modules/dasSpirv/ARCHITECTURE.md	Documents operand laziness design and rationale.
modules/dasLLAMA/tests/test_vulkan_kernels.das	Expands Vulkan kernel-unit suite (FA h256 gated, dn scan/ba coverage, sentinels, updated rails).
modules/dasLLAMA/tests/test_kernel_coverage.das	Extends Vulkan kernel coverage census (adds f16 session path, adds 9B hybrid).
modules/dasLLAMA/tests/test_gpu_tier.das	Extends resident-driver contract tests (hybrid seats + dn_native tag bit).
modules/dasLLAMA/tests/test_gpu_serving_declines.das	New tests for whole-model driver decline reasons and per-layer decline text.
modules/dasLLAMA/tests/test_gpu_resident_hybrid.das	New end-to-end forced-feed parity tests for resident hybrid prefill+decode (Q8 + K-quant fixture).
modules/dasLLAMA/tests/test_exe_smoke.das	Ensures produced runtime DLLs can be found via PATH when executing from temp dir.
modules/dasLLAMA/tests/test_chat.das	Forces CPU path for render seam test to avoid mixed GPU/CPU prefill behavior under DASLLAMA_GPU=1.
modules/dasLLAMA/tests/run.das	Wires new dasLLAMA test files into suites/areas.
modules/dasLLAMA/tests/CLAUDE.md	Updates test-suite documentation for new hybrid/decline tests.
modules/dasLLAMA/REVIEW.das	Updates image layout stamp hash.
modules/dasLLAMA/REVIEW_MEASUREMENT.md	Refines measurement/provenance rules and terminology.
modules/dasLLAMA/REVIEW_GPU.md	Tightens Vulkan parity/coverage rules; documents hybrid seat install asymmetry.
modules/dasLLAMA/REVIEW_GPU_VULKAN.md	Adds Vulkan-tier checklist items for timestamp accounting + decline logging.
modules/dasLLAMA/README.md	Splits Vulkan architecture docs (adds residency companion doc).
modules/dasLLAMA/performance/model_specs.das	Adds K-quant hybrid fixture + 9B UD model spec metadata for stocked runs.
modules/dasLLAMA/followup_vulkan.md	Updates Vulkan follow-up ledger with the hybrid ladder completion and validation notes.
modules/dasLLAMA/dasllama/dasllama_vulkan_common.das	Adds dn scan/ba workgroup helpers; adds BDA enablement/alloc/query path; updates dn workspace layout and hybrid fields.
modules/dasLLAMA/dasllama/dasllama_math_vulkan.das	Plumbs dn_native config bit; installs hybrid resident seats.
modules/dasLLAMA/dasllama/dasllama_load.das	Tags DeltaNet K-quant formats when the whole-model driver is to be attempted (dn_native).
modules/dasLLAMA/dasllama/dasllama_gpu_tier.das	Extends resident-driver SPI: new prepare signature, per-layer mirror idx, hybrid seat install + hooks.
modules/dasLLAMA/dasllama/dasllama_config.das	Adds dn_native to Vulkan image identity/tagging config.
modules/dasLLAMA/dasllama/dasllama_common.das	Adjusts session state fields around resident mirror ownership tracking.
modules/dasLLAMA/ARCHITECTURE.md	Updates architecture doc index to reflect Vulkan doc split and hybrid coverage.
modules/dasLLAMA/ARCHITECTURE_GPU.md	Updates Vulkan notes (decline logging expectations, hybrid seats as Vulkan-only asymmetry).
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN.md	Moves residency sections out; extends prefill section to document recurrent-window block + h256 FA.
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_RESIDENCY.md	New doc splitting out residency plan + marks swap details.
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_DECODE.md	Adds whole-model hybrid token-command section (decode + prefill-to-decode handoff).
daslib/logger.das	Ensures logger hook’s at line number prints as decimal (not hex-formatted).

Review details

• Files reviewed: 52/54 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

One new test (tests/daslib/test_logger.das) can cascade into misleading failures because it continues file IO after a failed child-process spawn instead of returning early.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/daslib/test_logger.das:217

• After asserting the child process exit code, the test continues and unconditionally reads/removes the log file. Since equal(...) does not abort on failure, a failed spawn can cascade into a secondary failure (or exception) in read_lines(p) that hides the real cause. Guard the file IO behind a successful rc == 0 check (or return early on failure).

• Files reviewed: 53/55 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

It spans core Vulkan residency/dispatch behavior, SPIR-V codegen control-flow changes, and runtime logging internals, which warrants final human review despite strong test coverage additions.

Review details

• Files reviewed: 51/51 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

It makes large, cross-cutting changes across Vulkan residency/serving, SPIR-V emission semantics, and runtime logging, which warrants final human review despite the added test coverage.

Review details

• Files reviewed: 50/50 changed files
• Comments generated: 0 new
• Review effort level: Lite