Skip to content

Shorten backward-substitution dependency chains - #44

Merged
ChrisRackauckas merged 1 commit into
JuliaSIMD:mainfrom
ChrisRackauckas-Claude:ldiv-upper-perf
Aug 8, 2026
Merged

Shorten backward-substitution dependency chains#44
ChrisRackauckas merged 1 commit into
JuliaSIMD:mainfrom
ChrisRackauckas-Claude:ldiv-upper-perf

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown

⚠️ Draft — please ignore until reviewed by @ChrisRackauckas.

Follow-up to #41. Performance-only; no API or semantics change; forward-substitution kernels untouched.

What changed and why

Per-solve hardware counters (isolated by differencing two iteration counts) showed the backward family executing nearly the same instructions as its forward twins but at less than half the IPC — e.g. at n=12 (a single register-blocked call, zero reduction trips): forward 1785 insns / 543 cycles (IPC 3.3) vs backward 1825 insns / 1208 cycles (IPC 1.5). Pure dependency stalls.

Root cause: solve_AU (and the forward small kernels) emit each column's updates with ascending k, so the freshest operand (A_{n-1}) arrives last — older updates overlap its latency and the critical path is one fma per column. The backward kernels emitted for k = n+1:N ascending, which puts the freshest operand (A_{n+1}) first, serializing the remaining N-n-1 fmas behind it: an O(W²/2) chain per diagonal-block solve instead of O(W). The same inversion existed in BdivL_small_kern[_u]! and in uldiv_solve_W_u!'s cross-sub-block j-loop.

Fix: emit those reductions with descending k/j (freshest last). Also re-based the backward reduction loops to zero-based induction with pre-offset (gesp) pointers, matching the forward kernels' loop shape.

Measured (EPYC 7502 / AVX2, Float64, nrhs=8, min times, 1 BLAS thread)

Backward vs forward at equal UNIT flag (ldiv! transposed family, μs):

   n  | uLower  uUpper(before) uUpper(after) | ratio before -> after
   48 |   0.68     0.96           0.78       |  1.39x -> 1.15x
   64 |   1.24     1.62           1.38       |  1.30x -> 1.11x
  128 |   4.45     5.20           4.70       |  1.17x -> 1.06x
  256 |  19.37    20.95          20.31       |  1.09x -> 1.05x
  500 |  73.77    77.40          75.43       |  1.05x -> 1.02x
 1000 | 281.57   290.25         286.08       |  1.02x -> 1.02x

Left-upper ldiv! (non-unit) vs OpenBLAS trsm, same protocol:

   n  |   BLAS   TS upper   speedup
   32 |   1.63     0.64      2.55x
   64 |   4.09     1.54      2.66x
  128 |  12.36     5.07      2.44x
  256 |  48.20    20.24      2.38x
  500 | 151.81    76.93      1.97x
 1000 | 581.25   291.33      2.00x

(#41's table had 1.9–2.3x on these points.)

Verification

  • Full test suite: TriangularSolve.jl | 148781 148781 pass (Julia 1.12.4) plus Aqua/ambiguities; extended sweep (n=1..400 all remainder cases, nrhs=1..32, Float32/64, unit/non-unit, 2/3-arg, threaded, packed lu! parents to 512, BigFloat fallback): 0 failures; zero steady-state allocations.
  • Forward kernels: code untouched by this diff; spot re-bench consistent with Add native left-upper ldiv! and right-lower rdiv! kernels #41 within noise.
  • Not verified: AVX-512/aarch64 (AVX2 machine), MKL.

Note on versioning: 0.2.2 (from #41) is not yet registered — registration needs a JuliaSIMD org member to comment @JuliaRegistrator register (my bot account was refused). If this merges first it can ship inside 0.2.2; if 0.2.2 is registered first, this should bump to 0.2.3.

🤖 Generated with Claude Code

Profiling the backward (upper-ldiv/lower-rdiv) family against its
forward twins showed near-identical instruction counts but ~2x the
cycles per diagonal-block solve (IPC 1.5 vs 3.3): the backward kernels
emitted each column's reduction with ascending k, so the freshest
operand (A_{n+1}) came first and every remaining fma serialized behind
it, giving an O(W^2) critical path where the forward kernels' ascending
order yields O(W). Emitting the reductions with descending k (and
descending j for the cross-sub-block updates in uldiv_solve_W_u!) puts
the freshest operand last, restoring the one-fma-per-column chain.
Also re-bases the backward reduction loops to zero-based induction with
pre-offset pointers, mirroring the forward kernels' loop shape.

Measured (EPYC 7502/AVX2, Float64, nrhs=8, min times): left-upper ldiv!
vs left-lower at equal UNIT: n=48 1.39x -> 1.15x, n=64 1.30x -> 1.11x,
n=128 1.17x -> 1.06x, n>=500 ~1.02x. Vs OpenBLAS trsm the upper leg
improves from ~2.2x to ~2.4-2.7x at n in [32,256]. Forward kernels are
untouched.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@JuliaRegistrator

Copy link
Copy Markdown

Error while trying to register: Register Failed
@ChrisRackauckas-Claude, it looks like you are not a publicly listed member/owner in the parent organization (JuliaSIMD).
If you are a member/owner, you will need to change your membership to public. See GitHub Help

@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.55%. Comparing base (258a420) to head (b28192b).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
+ Coverage   85.01%   94.55%   +9.54%     
==========================================
  Files           1        1              
  Lines         427      827     +400     
==========================================
+ Hits          363      782     +419     
+ Misses         64       45      -19     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review August 8, 2026 08:39
@ChrisRackauckas
ChrisRackauckas merged commit 2e68e37 into JuliaSIMD:main Aug 8, 2026
7 checks passed
ChrisRackauckas added a commit that referenced this pull request Aug 8, 2026
The backward kernels' nmuladd reduction read the already-solved trailing
columns with ascending nk, so its first loads hit the block the previous
kernel call had just stored, stalling on stores still draining. Walking
nk descending reads the oldest columns first and the freshest block last
— the mirror of the forward kernels' ascending reduction, which also
ends on its most recently written block. The down-counting loop also
compiles to the same shared-scaled-index address form as the forward
loop (two address updates per trip instead of six).

Measured (EPYC 7502/AVX2, Float64, nrhs=8, min times), left-upper ldiv!
vs left-lower at equal UNIT flag, after the chain fix in #44:
n=48 1.15x -> 1.10x, n=64 1.11x -> 1.06x, n=128 1.06x -> 1.04x,
n=256 1.05x -> 1.02x, n=1000 1.02x -> 1.00x. Vs OpenBLAS trsm the
upper leg is now 2.0-2.9x. Forward kernels untouched.

0.2.2 was registered from a0118b0 (pre-#44), so the #44 and this-PR
improvements ship as 0.2.3.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants