Shorten backward-substitution dependency chains - #44
Merged
ChrisRackauckas merged 1 commit intoAug 8, 2026
Conversation
Profiling the backward (upper-ldiv/lower-rdiv) family against its
forward twins showed near-identical instruction counts but ~2x the
cycles per diagonal-block solve (IPC 1.5 vs 3.3): the backward kernels
emitted each column's reduction with ascending k, so the freshest
operand (A_{n+1}) came first and every remaining fma serialized behind
it, giving an O(W^2) critical path where the forward kernels' ascending
order yields O(W). Emitting the reductions with descending k (and
descending j for the cross-sub-block updates in uldiv_solve_W_u!) puts
the freshest operand last, restoring the one-fma-per-column chain.
Also re-bases the backward reduction loops to zero-based induction with
pre-offset pointers, mirroring the forward kernels' loop shape.
Measured (EPYC 7502/AVX2, Float64, nrhs=8, min times): left-upper ldiv!
vs left-lower at equal UNIT: n=48 1.39x -> 1.15x, n=64 1.30x -> 1.11x,
n=128 1.17x -> 1.06x, n>=500 ~1.02x. Vs OpenBLAS trsm the upper leg
improves from ~2.2x to ~2.4-2.7x at n in [32,256]. Forward kernels are
untouched.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Error while trying to register: Register Failed |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #44 +/- ##
==========================================
+ Coverage 85.01% 94.55% +9.54%
==========================================
Files 1 1
Lines 427 827 +400
==========================================
+ Hits 363 782 +419
+ Misses 64 45 -19 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ChrisRackauckas
marked this pull request as ready for review
August 8, 2026 08:39
ChrisRackauckas
added a commit
that referenced
this pull request
Aug 8, 2026
The backward kernels' nmuladd reduction read the already-solved trailing columns with ascending nk, so its first loads hit the block the previous kernel call had just stored, stalling on stores still draining. Walking nk descending reads the oldest columns first and the freshest block last — the mirror of the forward kernels' ascending reduction, which also ends on its most recently written block. The down-counting loop also compiles to the same shared-scaled-index address form as the forward loop (two address updates per trip instead of six). Measured (EPYC 7502/AVX2, Float64, nrhs=8, min times), left-upper ldiv! vs left-lower at equal UNIT flag, after the chain fix in #44: n=48 1.15x -> 1.10x, n=64 1.11x -> 1.06x, n=128 1.06x -> 1.04x, n=256 1.05x -> 1.02x, n=1000 1.02x -> 1.00x. Vs OpenBLAS trsm the upper leg is now 2.0-2.9x. Forward kernels untouched. 0.2.2 was registered from a0118b0 (pre-#44), so the #44 and this-PR improvements ship as 0.2.3. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #41. Performance-only; no API or semantics change; forward-substitution kernels untouched.
What changed and why
Per-solve hardware counters (isolated by differencing two iteration counts) showed the backward family executing nearly the same instructions as its forward twins but at less than half the IPC — e.g. at n=12 (a single register-blocked call, zero reduction trips): forward 1785 insns / 543 cycles (IPC 3.3) vs backward 1825 insns / 1208 cycles (IPC 1.5). Pure dependency stalls.
Root cause:
solve_AU(and the forward small kernels) emit each column's updates with ascending k, so the freshest operand (A_{n-1}) arrives last — older updates overlap its latency and the critical path is one fma per column. The backward kernels emittedfor k = n+1:Nascending, which puts the freshest operand (A_{n+1}) first, serializing the remainingN-n-1fmas behind it: anO(W²/2)chain per diagonal-block solve instead ofO(W). The same inversion existed inBdivL_small_kern[_u]!and inuldiv_solve_W_u!'s cross-sub-blockj-loop.Fix: emit those reductions with descending k/j (freshest last). Also re-based the backward reduction loops to zero-based induction with pre-offset (
gesp) pointers, matching the forward kernels' loop shape.Measured (EPYC 7502 / AVX2, Float64, nrhs=8, min times, 1 BLAS thread)
Backward vs forward at equal UNIT flag (
ldiv!transposed family, μs):Left-upper
ldiv!(non-unit) vs OpenBLAStrsm, same protocol:(#41's table had 1.9–2.3x on these points.)
Verification
TriangularSolve.jl | 148781 148781pass (Julia 1.12.4) plus Aqua/ambiguities; extended sweep (n=1..400 all remainder cases, nrhs=1..32, Float32/64, unit/non-unit, 2/3-arg, threaded, packedlu!parents to 512,BigFloatfallback): 0 failures; zero steady-state allocations.Note on versioning: 0.2.2 (from #41) is not yet registered — registration needs a JuliaSIMD org member to comment
@JuliaRegistrator register(my bot account was refused). If this merges first it can ship inside 0.2.2; if 0.2.2 is registered first, this should bump to 0.2.3.🤖 Generated with Claude Code