Pipeline compilation: createPipeline traces multi-pass programs into fused, barrier-synchronized plans

[fuzzie360]

gpu.createPipeline(fn) — express a multi-kernel, multi-pass computation as a plain JavaScript orchestration function; gpu.js traces it once into a static plan and compiles the whole plan as a unit. On the webasm backend the plan fuses over one shared memory with workers advancing pass-to-pass on Atomics barriers — no main-thread round trip, no copy, per step.

const sweep = gpu.createKernel(function (u, q) { ... }, { constants: { hi }, output: [1024, 1024] });

const solve = gpu.createPipeline(function (u, q) {
  for (let s = 0; s < this.constants.sweeps; s++) {
    u = sweep(u, q);
  }
  return u;
}, { constants: { sweeps: 512 } });

const result = await solve(u0, q);   // one launch, fences inside, one readback

Design contract: docs/design/pipeline-compilation.md (trace-once semantics, handle rules with named build-time errors, automatic double-buffering — u = sweep(u, q) compiles to ONE kernel over two alternating buffers, retiring the duplicate-kernel ping-pong idiom — and the v1 exclusions, stated).

Measured (scripts/benchmark-pipeline.mjs, gauntlet jacobi/heat shapes, Apple M1 Max, checksums identical)

Workload	plain JS	webasm per-pass	pipeline generic	pipeline fused-sync	pipeline fused-threaded
jacobi 1024², 512 sweeps	1227 ms (1.63×)	1997 ms (1×)	2009 ms (0.99×)	1760 ms (1.13×)	387 ms (5.16×)
heat 1024², 1024 steps	2473 ms (2.05×)	5073 ms (1×)	4342 ms (1.17×)	3630 ms (1.40×)	890 ms (5.70×)

The rows the webasm backend used to lose to plain JavaScript are now 2.8–3.2× wins. The per-pass overheads the fusion deletes — a worker-pool round trip, argument re-upload, and a readback per step — are exactly what the gauntlet perf investigation measured as dominant on multi-pass workloads.

What ships

• Core (src/pipeline.js): frozen Proxy handles (element reads, arithmetic coercion, spread, and key enumeration all throw named messages), synchronous tracer (async/generator orchestration throws; Math.random barred; stale handles across re-traces throw), plan IR with static-liveness buffer assignment, generic executor correct on cpu / webgl / webgl2 / headlessgl / webasm, tail-serialized Promise calls with call-time argument snapshots (textures snapshot via clone()).
• webasm fused executors (src/backend/web-assembly/pipeline-executor.js): sync fusion over one shared memory (512-pass ping-pong dedupes to 3 wasm instances); threaded fusion with monotonic-generation Atomics barriers — nothing is ever reset under a laggard worker, aborts retire workers still owing acks (the only place a silently-terminated browser worker is detectable), stall backstop counts barrier arrivals as progress. Everything unfusable degrades to the generic executor with a named fallbackReason.
• Clone fidelity: plan clones inherit randomSeed, returnType, and user-pinned argumentTypes (new declaredArgumentTypes, distinct from build-inferred types) so pipelined kernels compute exactly like direct calls.
• A backend-wide fix surfaced by review: texture.clone() on a mutable kernel's output was broken everywhere — copy-on-write only ran under immutable; the GL render path now honors outstanding clone refs unconditionally.
• 89 pipeline tests + docs + index.d.ts; adversarial review ran three dimensions, 12 findings all reproduced-then-fixed, re-verified against the reviewers' own repro scripts (including 1500 back-to-back threaded runs with zero barrier faults).

WebGPU lowering: fused-encoder

Added after the initial push (the branch's last four commits): every plan step compiles against persistent storage buffers on the kernel's device — ping-pong as two static bind groups, per-step params uniforms — and per call the WHOLE plan records into one GPUCommandEncoder: argument writes, every step as a compute pass, result copies to a single staging buffer, one queue.submit, one mapAsync. Reuses WebGPUKernel's own WGSL/pipeline build; nothing forked. Seeded Math.random keeps bit-exact parity with direct calls.

Measured (scripts/benchmark-pipeline-webgpu.mjs, headed Chrome / Metal, checksums identical):

Workload	webgpu per-pass	pipeline generic	pipeline fused-encoder
jacobi 1024², 512 sweeps	43.9 ms	497.3 ms	28.3 ms (1.55×)
heat 1024², 1024 steps	59.8 ms	255.2 ms	37.0 ms (1.62×)

Honest framing: webgpu per-pass is already a strong baseline (pipeline: true calls resolve at submit, so the GPU queue pipelines all sweeps); the encoder fusion removes the remaining per-call JS/bind/submit overhead. The generic executor is the degradation path only — it syncs per step on webgpu and the docs say so. Verify-first paid off: the pipeline test matrix had never enumerated webgpu, so phase 1 put the generic executor on real WebGPU for the first time (it held; one latent Input-snapshot bug in the shared tracer was found live and fixed). The lowering's review confirmed 3 findings (oversize-argument silent zeros, result-only handle seats bypassing the resident-handle screen, Input result-shape parity) — all reproduced in headed Chrome, fixed, and pinned by regression tests.

v1 exclusions (documented in the README)

No mid-plan readback (this.check reserved), no graphical kernels or kernel maps inside plans (named errors).

Gates: Node 2833/0 · headed browser 4262/0 (webgpu pipeline rows genuinely executing) · SwiftShader failure set identical to baseline.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RUTVDFaHav3uAdN3XfZLyx

[fuzzie360]

Findings from migrating the gpu.rocks benchmark suite to createPipeline. 19 multi-pass workloads were attempted, each migration then reviewed by an independent pass that re-ran the numbers.

Correctness is not the problem — every migration passed. Several were bit-exact against the plain-JS oracle (rel 0), the rest inside 1e-4. Trace rules, unrolling, ping-pong buffer assignment and re-trace on setConstants all behaved as the design contract says. WebGPU and WebASM both improve, sometimes a lot.

The blocker is the generic executor's cost on the GL backends, and it is large enough that not one of the 19 could be adopted.

The regression

Real browser, Apple M1 Max, Chrome. Recorded published-2.22.0 numbers against this branch with the row migrated:

row	column	2.22.0	pipeline
heat	WebGPU	46 ms	30 ms	−35%
	WebGL2	56 ms	472 ms	+747%
	WebGL	63 ms	1837 ms	+2809%
	WebASM	4468 ms	3917 ms	−12%
life	WebGPU	31 ms	29 ms	−6%
	WebGL2	65 ms	226 ms	+248%
	WebGL	75 ms	731 ms	+869%
sobel	WebGPU	22 ms	22 ms	+1%
	WebGL2	20 ms	78 ms	+288%
	WebGL	47 ms	159 ms	+237%

Independently, agents measured the same shape at full size on other rows: jacobi WebGL 72.7 → 803.7 ms (11×), ising 66.5 → 984.9 ms (15×), bitonic-sort 121 → 3062 ms (25×).

Mechanism, confirmed by counting

_cloneKernel (src/pipeline.js) hard-codes immutable: true for every plan kernel on every backend:

const settings = {
  output: Array.from(kernel.output),
  pipeline: true,
  immutable: true,
  dynamicArguments: true,
};

On GL, immutable: true means each call allocates a fresh output texture and releases the previous one. Instrumenting the plan clone's own context (plan.kernels[0].clone.kernel.context) over a 200-step plan, steady state, after build:

executorKind : (generic)
clone settings: pipeline= true  immutable= true
steady-state run over 200 steps -> createTexture 198  deleteTexture 198  | per step 0.99

One texture created and destroyed per plan step. The hand-rolled ping-pong these workloads previously used allocates two textures and reuses them for the life of the kernel. heat runs 1024 steps, so a plan run does ~1024 full-size create/destroy pairs where the old code did zero — which is the 29× on its WebGL cell.

dynamicArguments: true is also forced, which gives up per-seat specialisation; I did not isolate its share.

Why this blocks adoption rather than merely costing something

A workload here exposes one gpujs() used by all five columns — WebGPU, WebGL2, WebGL, WebASM, CPU. It cannot take the fused path on WebGPU/WebASM and keep the hand-rolled loop on GL. So every migration is all-or-nothing, and each one trades a 1.2–4.7× win on two columns for a 3–29× loss on two others.

Branching on gpu.mode inside the workload was considered and rejected: five cells in one row measuring two different orchestration strategies is exactly the incomparability the suite exists to avoid.

What would unblock it

Any one of these, roughly in order of preference:

1. A fused GL executor, matching what WebGPU and WebASM already have.
2. Stop forcing immutable: true in the generic executor. The plan already knows its buffer assignment statically — the ping-pong slots are computed at compile time — so the clones could use mutable output slots and reuse them, which is what the assignment is for. This looks like the smallest fix that recovers most of the loss.
3. Failing both, recycle textures inside _executeGeneric rather than allocating per step.

Three secondary findings

backend() detection is now blind, which silently disables a safety net. Workloads probe kernel.kernel.constructor.mode to assert the backend that ran is the backend that was asked for — that is how this suite catches gpu.js silently degrading to CPU. Under a pipeline, the user's shortcut is not what executes; plan.kernels[i].clone is. Every migrated row's guard reported the requested mode regardless of what actually ran. plan.kernels[0].clone.kernel.constructor.mode works as a replacement, but the feature should probably expose the executed backend and fallbackReason on the pipeline itself, the way #868 added kernel.fallbackReason.

Threaded WebASM changes what a cell means. A migrated row reaching fused-threaded puts a multi-threaded number next to a single-threaded plain-JS baseline, and next to unmigrated rows that are single-threaded. That is a legitimate result but not a comparable one, and it is not a decision an individual workload can make. A way to pin a pipeline to the sync path — or simply a reliable way to read which executor ran — would let a benchmark keep the comparison honest.

Some rows should never be migrated, and that is fine. launch-overhead measures per-dispatch cost as the difference between the gpu.js column and a hand-written column that is already one encoder; fusing makes the two structurally identical and the difference goes to zero, deleting the measurement. residency exists to price setPipeline(true) as an opt-in, and the contract says inner kernels do not need it — so a traced version contains zero instances of the API the row measures. Both correctly declined. Worth knowing the feature has rows it should not touch.

Reproducing

The migrated workloads are kept on the gpu.rocks side rather than reverted, so the comparison can be re-run once the GL lowering changes. Each is a createPipeline rewrite of an existing hand-rolled ping-pong with the kernel bodies untouched, so they are ready-made test cases: heat (1024 steps, 1 kernel, 2 buffers), gray-scott (768 steps, 2 kernels, 4 buffers), erosion (515 steps, 7 kernels, 5 buffers), wavefront (12289 steps), topk (1753 steps).

[fuzzie360]

Landed as 0d5b105 — option 2, taken further than the comment asked, and measured on your own instrumentation method.

The generic executor no longer forces immutable: true. Static liveness was indeed the enabling fact: assignBuffers already guarantees no step reads a slot while that slot's writer renders, so the executor now clones per (kernel, seat signature, output slot) with immutable: false — each clone owns one output texture for the plan's life, which is mechanically the hand-rolled kA/kB two-texture pattern, generated. Two further layers fell out of profiling the same 200-step 1024² repro:

• dynamicArguments: true was most of the loss that remained after the churn was gone — the signature-keyed clones make every seat's type plan-static, so it is now false (per-seat specialisation restored).
• Array pipeline arguments were re-uploading per step (the 4 MB q array, 200 times); they now upload once per call through lazy identity kernels on backends where uploads cost — the upU/upQ idiom, also generated.

200-step jacobi ping-pong, 1024², headlessgl	before	after
createTexture per step (steady state)	0.99	0.00
pipeline generic	248 ms	51 ms
hand-rolled two-kernel loop (same session)	35–67 ms	—

Checksums identical throughout. Parity-or-better with the hand-rolled loop rather than 7×-slower; the 3–29× GL cells should now be re-runnable. Static shapes gained an explicit drift story (size changes rebuild the generic clones, matching the fused executors' recompile contract) and cpu results copy on readback so a held result survives the next call.

On the secondary findings: agreed on all three. Exposing the executed backend on the pipeline itself (the #868 fallbackReason treatment — it is already there, plus a backend accessor) and a way to pin the sync path for benchmark comparability are small and worth doing before merge; the launch-overhead/residency non-migrations are correct readings of what those rows measure. A fused GL executor remains the eventual right answer for option 1, but with generic at parity the urgency is gone.

[fuzzie360]

Re-ran everything against 0d5b105 (mutable statically-typed clones). It fixes it. Same 13 migrated workloads, same machine, real browser.

The mechanism is gone

Instrumenting the plan's own GL context over a 200-step plan, steady state:

                    before (3b47188)    after (0d5b105)
createTexture              198                 0
deleteTexture              198                 0
per step                  0.99             0.000
genericClones                —                 4

The regression is gone

WebGL2 and WebGL, published-2.22.0 baseline → branch before the fix → branch after:

row	GL2 2.22	GL2 was	GL2 now	GL 2.22	GL was	GL now
heat	56	472	59	63	1837	66
gray-scott	52	393	62	120	1417	167
life	65	226	70	75	731	84
wavefront	490	1272	496	484	1277	483
spectral-filter	16	73	34	39	280	36
erosion	52	97	56	114	293	119
topk	144	238	125	420	425	380
compaction	393	127	109	987	257	150

Medians against the 2.22.0 baseline, across all 13:

column	before	after
WebGL2	+160%	+8%
WebGL	+158%	+5%
WebGPU	−23%	−25%
WebASM	−4%	−7%

heat's WebGL cell went 1837 ms → 66 ms against a 63 ms baseline. The two columns the feature was for kept their wins: WebGPU median −25%, with wavefront −90% (674 → 69 ms, 12,289 passes) and compaction −70%.

Correctness held. No cell reported WRONG on any column of any of the 13 rows; the suite checksums every column against a plain-JS oracle at 1e-4 before timing it.

One residual, and it looks like a fixed per-call cost

What is left is not proportional — it is roughly constant, so it only shows on rows that were already fast. WebGL2, sorted by how fast the row was to begin with:

row               2.22 ms   now    delta
path-trace              5      8     +3
spectral-filter        16     34    +18
canny                  19     50    +31
sobel                  20     57    +37
erosion                52     56     +4
gray-scott             52     62    +10
heat                   56     59     +3
optical-flow           62     66     +4
life                   65     70     +5
reduction             128     65    -63
topk                  144    125    -19
compaction            393    109   -284
wavefront             490    496     +6

Median absolute delta is +5 ms for rows under 70 ms and −41 ms for rows over it. sobel (16 passes) and canny (12 passes) are the worst relative cases at +37 and +31 ms, while heat (1024 passes) costs +3 ms — so it is not per-step. It reads like a fixed per-pipeline-call cost, plausibly the argument sampling/upload the plan does before dispatching, which a 16-pass row cannot amortise and a 1024-pass row never notices.

Not a blocker from our side — a few ms on a benchmark row is legible where a 29× was not — but if there is a cheap win in the per-call path, sobel and canny would show it immediately.

Where this leaves adoption

Adoptable now, as far as the GL columns are concerned. Before we record a saved run and publish it, two things from the earlier comment still stand and are worth a line each:

• backend() is still blind. The suite asserts the backend that ran is the backend asked for, by reading kernel.kernel.constructor.mode — that is how it catches silent CPU degradation. Under a pipeline the user's shortcut is not what executes. plan.kernels[0].clone worked before this commit; with genericClones now holding the real writers, whatever the supported accessor should be, exposing the executed backend and fallbackReason on the pipeline would be better than either.
• Threaded WebASM comparability. A migrated row reaching fused-threaded is multi-threaded next to a single-threaded plain-JS baseline and single-threaded unmigrated rows. A way to read which executor ran, or to pin the sync path, would let us keep that column honest.

[fuzzie360]

Both blockers are addressed as supported, documented API in 2b18318 (plus 358cdd9) — and your point about reverse-engineered accessors breaking silently was validated in the most direct way possible: the backend accessor I added in 358cdd9 read plan.kernels[0].clone, which 0d5b105 had already made stale. It now derives from the executor that ran, never from plan internals.

1. Executed backend + degradation reason — supported API, README-documented:

• pipeline.backend — the mode of the kernels that actually execute. Fused kinds name their backend; generic reports its writer clones' mode; under degradation-inside-generic it reports 'cpu', restoring your webasm: pipelined kernels always degrade to cpu — 17 of 30 real GPGPU workloads never reach WebAssembly #868-style guard. (Same value convention as kernel.kernel.constructor.mode, so your existing assertion code ports directly.)
• pipeline.fallbackReason — why a fused executor declined; null while fused.

Verified end-to-end on the nastiest chain: texture argument → webasm fused declines (this screen was missing on webasm — your report's probe found a real crash there, now recompile-then-degrade with a named reason) → generic → clone itself degrades → backend === 'cpu', correct results, reason populated.

2. Executor kind + sync pinning — both:

• pipeline.executorKind ('fused-threaded' | 'fused-sync' | 'fused-encoder' | 'generic') — supported, documented.
• createPipeline(fn, { threads: false }) pins the webasm lowering to fused-sync, for exactly your column-comparability case.

3. The fixed per-call cost — partially addressed, and I need your shape to finish it. When the pipeline is quiescent, GL argument uploads now run synchronously at call time, so the upload texture IS the call-time snapshot and the deep argument copy is skipped entirely (previously every call paid copy + flatten). On my repro (16-pass 2048², typed-array rows, headlessgl) the copy was only ~2 ms, so I cannot reproduce your +37 ms locally — which makes me suspect your sobel/canny arguments are plain nested JS arrays (slow to deep-copy) or the cost sits elsewhere in the browser path. If your 20-minute sweep still shows +30 ms on sobel/canny after this commit, the shape of those rows' make() outputs is the next thing I'd want to see.

Would appreciate the 13-row re-run whenever convenient — gates on this end are green (Node 2838/0, browser 4267/0, SwiftShader baseline unchanged).

[fuzzie360]

Re-ran the 13 rows on 2b18318. Both blockers are closed on our side, and the GL columns are at parity. Also: your backend accessor going stale one commit later is the cleanest possible demonstration of why this needed to be supported API rather than something I read out of the plan — thank you for taking it seriously.

Medians vs published 2.22.0, all 13 migrated rows

column	3b47188	0d5b105	2b18318
WebGL2	+160%	+8%	+2%
WebGL	+158%	+5%	+1%
WebGPU	−23%	−25%	−25%
WebASM	−4%	−7%	−8%

Notable rows against the 2.22.0 baseline: wavefront WebGPU 674 → 67 ms (−90%, 12,289 passes), compaction −70% on WebGPU and WebGL2 and WebGL, reduction WebGL2 128 → 59 (−54%), topk WebASM 13417 → 7871 (−41%), gray-scott WebGL 120 → 118 against 1417 two commits ago.

No cell reported WRONG on any column of any row.

The new API does exactly what we needed

pipeline.backend is wired into all 13 rows now, replacing accessors that were either reading the user's shortcut (blind) or plan.kernels[0].clone (stale). Verified it reports the executor that ran — heat on the cpu column returns 'cpu' with the oracle matching to rel 0.

threads: false is set on all 13, pinning the WebASM column to fused-sync so it means one thing across migrated and unmigrated rows. That is the right default for a benchmark; it is not a criticism of the threaded path, and if we later add a threaded column it will be because executorKind lets us label it honestly rather than silently mix it in.

The residual — your instinct about argument shapes was right

Your fix helped a little (sobel 57 → 52, canny 50 → 47) but the bulk remains, and it is confined to short-plan rows:

row	passes	2.22 GL2	now	delta
sobel	16	20 ms	52 ms	+32 ms
canny	12	19 ms	47 ms	+28 ms
spectral-filter	32	16 ms	38 ms	+22 ms
path-trace	9	5 ms	8 ms	+3 ms
heat	1024	56 ms	57 ms	+1 ms

You asked for the shape of those rows' make() outputs. You were right — they are plain nested JS arrays, not typed arrays:

sobel            src: Float32Array(4194304) | rows: Array(2048) of Float32Array(2048)
canny            src: Float32Array(4194304) | rows: Array(2048) of Float32Array(2048)
spectral-filter  signal: Array(128) of Float32Array(16384) | ...
heat             u0: Float32Array(1048576)          <- flat, and costs +1 ms

sobel and canny pass rows — an Array(2048) whose elements are Float32Array(2048) — as the pipeline argument on every call (await solve(rows)). spectral-filter passes Array(128) of Float32Array(16384). heat, which passes a single flat Float32Array, costs +1 ms.

That is a clean correlation: the rows paying the cost are exactly the ones handing the pipeline an array-of-typed-arrays, and the one handing it a flat typed array pays nothing. 2048 outer elements versus 128 versus 1, against +32 / +22 / +1 ms, also lines up with per-outer-element work rather than per-byte — all three carry ~4–16 MB total.

Worth noting these rows are 2-D kernels, so the nested shape is what gpu.js wants for a [n, n] input; it is not us being careless with the argument type. If the plan can hoist an unchanged nested argument's upload the way it now does for flat ones, that would close it.

Not blocking us — we can publish at +2% GL median. Flagging it because it is the one thing left that a user with an image-shaped kernel would hit.

[fuzzie360]

Your outer-element-count correlation cracked it — and the embarrassing half is that the fix I claimed in 2b18318 was dead code. The eager-upload fast path armed on _executor === null, but the settled-degraded-to-generic sentinel is false, so it never engaged on any GL pipeline. Your sobel 57 → 52 was measuring noise around a path that never ran; the +32 ms was the per-call deep copy of Array(2048)-of-rows (2048 slices + 2048 allocations per call, much heavier in a browser than the flat memcpy heat's single Float32Array gets — hence per-outer-element, exactly as you read it).

Fixed in b85e21b: the fast path now actually arms (the sentinel is pinned by a test so a future refactor cannot re-deaden it silently), and on the quiescent sequential pattern your harness uses — await solve(rows) back-to-back — the deep copy is skipped entirely: the synchronous GL upload at call time is the call-time snapshot. Nested-rows arguments keep full call-time sampling semantics (mutate rows right after the un-awaited call and the result still reflects call-time contents — pinned by test), and an eager upload declines on size drift rather than writing out of bounds into the previous size's upload kernel (also caught by your test shape, also pinned).

What should remain on sobel/canny after this: the per-row flattenTo set inside the upload itself, which the 2.22.0 hand-rolled baseline pays identically — so those rows should land at or very near baseline. If the sweep still shows a gap beyond ~2–3 ms there, something else is in play and I want to know.

Whenever you have 20 minutes for the re-run — and thank you for the correlation table; 'scales with outer elements, not bytes' was the whole diagnosis.