Skip to content

perf(mpi): 🧭 GF(2)-linear rank routing — messages per rank flat in R - #296

Open
diagonal-hamiltonian wants to merge 34 commits into
mainfrom
perf/linear-routing-on-wire
Open

diagonal-hamiltonian wants to merge 34 commits into
mainfrom
perf/linear-routing-on-wire

Conversation

@diagonal-hamiltonian

@diagonal-hamiltonian diagonal-hamiltonian commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Implements a GF(2)-linear rank hash, in the same spirit as arXiv:2506.13241 but not the same construction — see Relation to prior work below.

Rebased onto main 2026-09-01, now that #263 has merged as 364a5f50. This branch's
first six commits were #263's and are dropped; the 25 routing commits are replayed. An earlier
version of this line said merging #263 would retarget this branch unchanged — that turned out
to be wrong, because #263 picked up a review refactor (spans, and returned aggregates in place
of out-params) after it was written. See the rebase comment below for what had to be resolved.
Commit SHAs named anywhere below predate the rebase; 22 of the 25 are content-identical
under git range-diff.

Updated 2026-08-29 — ten further commits (a610f048..2f8d3dc2) collapse the d dial, take the
emit path off the hash, size the per-generator bookkeeping to the reachable window, overlap the
count round, and carry the lever onto propagate()/energy()/gradient(). Every timing table
below is labelled with the commit it was measured on.
The Leonardo ladder and the
#263-composition table predate these commits and are kept as the record of the routing change in
isolation; the three Deucalion campaigns are current.

What this does

main's MPI cost obeys

MPI time  =  collectives per rank (FIXED by the algorithm)  ×  R  ×  (R−1)  ×  cost per peer-message

#263 attacks the last factor. This attacks (R−1), which is not physics — it is a consequence of one
line.
The destination of a query was monomial_hash(M ⊕ G) % R, and monomial_hash is splitmix,
whose full avalanche is exactly what makes the exchange dense. Every earlier attempt at a sparse
transport failed because the routing stayed dense: a rank pair is non-empty if any of its
partition pairs has data, so at 8.7% partition-level occupancy the chance a rank pair is empty is ~2e-8.

Make the rank-level hash GF(2)-linear instead:

h(M) = XOR of v_i over support(M)      ⇒      h(M ⊕ G) = h(M) ⊕ h(G)

A rank owns M, so every query it emits for generator G lands on my_rank ⊕ Δ(G)fanout 1.
XOR is an involution, so that peer sends back on the same round. Nothing about the term set, the
volume moved, or the gate structure changes; only which rank a term lives on.

Routing is two-level, because the two levels have different costs: GF(2)-linear across MPI ranks,
where message count is the cost, and full-avalanche splitmix across partitions within a rank, where
fanout is free (shared memory) and only balance matters.

flat_dest(M) = rank_linear(M) * S  +  splitmix(M) % S

monoprop_ROUTING selects the arm: linear (default, fanout 1) or splitmix (the previous
behaviour, bit-identical to the base). Power-of-two R is required and a non-power-of-two R
now throws UnroutableGeometry rather than silently falling back — a silent fallback was a way to
measure the wrong arm without noticing. R == 1 takes no rank bit and so is the dense case by
construction, which is why the collective path stays.

Commits

Routing (22ad7049 and below):

route through one Router one routing::Router owns the term→slot map, replacing two inline copies of hash % P. mpi::geometry() is new: size() cannot tell an inter-rank message from an inter-partition one.
skip the exchange for identity generators a generator with empty support anticommutes with nothing, so both its exchange passes carry zero queries — but collectives fire regardless of payload. 60 of the 60-site Hubbard's 476 generators per Trotter layer are these, i.e. 12.6% of all MPI calls doing nothing.
exchange point-to-point over the peers routing can reach mpi::PeerPlan threads the fanout to HybridComm; every O(R·S²) sweep that runs serially on partition 0 shrinks to O(f·S²). check_routing_agreement() allreduces the configuration once at construction, because a rank that missed the env var would hang, not answer differently.
route GF(2)-linear by default flips the default; splitmix is the explicit opt-out.

Follow-through (a610f048..2f8d3dc2, new):

collapse the linear-bit dial to a boolean d was an integer dial where a boolean does: the shipped configuration is d = log2 R and d = 0 is the control, and the intermediate values existed only to be tested. PeerPlan::bits becomes bool sparse; monoprop_ROUTE_LINEAR_BITS, hi_mask_, the hi term and a 64-bit division on a runtime divisor all go. −139 lines, and one whole dimension out of the test matrix.
route the emit path from the generator's shift dest(M ⊕ G) was recomputing a per-generator constant per emitted term. rank(M ⊕ G) = my_rank ⊕ Δ(G) is an identity, and Δ(G) is already computed once per generator for the PeerPlan. Removes the parity-plane fold (28 loads + 28 AND + 28 XOR + 7 popcount per query at production widths) from the hot loop.
pack the send side under the count round the count block is ints — eager, and fully overlappable with packing. Split post/wait and pack between them.
replay the layer exchange point-to-point + gate the pairwise replay on routing begin_flat_exchange posted a dense Ialltoallv with no PeerPlan, so the graph-replay path got zero benefit. Now gated on a rank-uniform wire_bits derived from the routing configuration — the obvious gate (each rank counts its own non-zero legs) is rank-varying and deadlocks when the decision straddles.
name the slot window a peer plan can reach + size the query path to the peer window + narrow the staging sweeps at fanout 1 only S of the P = R·S flat slots can receive a query, yet the bookkeeping was sized and swept over P — ~18 sweeps of length 2048 per generator per pass at R=128, S=16. SlotWindow{base, count} is contiguous (that is what the boolean buys), and the dense case is the count == P value of the same path, not a second implementation.

Measured

Balance — settled by measurement, not by the dial

One instrumented run of the production point (60 sites, cutoff 10, lower_atol=2.6e-06
1,569,152,761 terms) recorded a joint histogram over every owned term at ten layers, so every
(R, S) is a marginal of one table and d=0 is its own control:

layer terms rank max/mean d=0 fanout 1 ranks used
3 21,148 1.265 1.204 128/128
5 346,334 1.047 1.051 128/128
12 151,921,485 1.003 1.002 128/128
29 1,569,152,761 1.001 1.001 128/128

Imbalance is confined to layers 1–3, which hold ≤21k of 1.57e9 terms (0.0013%). This is a property of
the map, not of the transport, so it is unchanged by the follow-through commits.

Campaign A — the full branch against #263's head

22ad70492f8d3dc2, two binaries, md5-gated. Deucalion x86, layout 8×16, 10 reps per cell
interleaved inside one allocation with the arm order flipped per rep, paired per-rep ratios then
median. All 15 timing tests survive Holm at family size 15 (p = 0.00195 or 0.0215 against a
0.05/15 = 0.0033 floor).

operation N=4 (R=32) N=8 (R=64) N=16 (R=128)
propagate[hubbard] 1.32× (0.76) 1.56× (0.64) 1.99× (0.50)
build_graph[pauli] 1.23× (0.81) 1.34× (0.75) 1.53× (0.65)
propagate[pauli] 1.26× (0.79) 1.52× (0.66) 1.84× (0.54)
energy[pauli] 1.28× (0.78) 1.39× (0.72) 4.96× (0.20)
gradient[pauli] 1.20× (0.84) 1.46× (0.68) 3.05× (0.33)

The gain grows with R in every row, which is the signature the work removed is proportional to P.

Campaign B — the lever itself, re-baselined on the current tip

monoprop_ROUTING=splitmixlinear on one binary (633df79e, a bench-only tree), so the arms
differ in nothing but the routing arm, read back per-rep from the in-process config. The harness
refuses same-md5 arms and printed its usual banner; the readback (splitmix on all 20 main files,
linear on all 20 port files) is what makes the cells valid. All 15 tests survive Holm.

operation N=4 (R=32) N=8 (R=64) N=16 (R=128)
propagate[hubbard] 2.89× (0.35) 4.23× (0.24) 8.19× (0.12)
build_graph[pauli] 2.83× (0.35) 3.94× (0.25) 5.86× (0.17)
propagate[pauli] 2.82× (0.35) 4.49× (0.22) 7.20× (0.14)
energy[pauli] 1.56× (0.64) 1.72× (0.58) 1.49× (0.67)
gradient[pauli] 1.47× (0.68) 2.34× (0.43) 2.66× (0.38)

This is a per-operation figure, not the whole-run wall the 0.32× headline below measures — the two
are not directly comparable. The engine's byte ledger also shows the graph shrinking 2.06× at N=16
(14.19 → 6.87 GiB) — expected, since the retained graph is quadratic in the flat world and fanout 1
collapses the peer set — but /usr/bin/time -v did not run in these cells, so that is a ledger
number (capacity, not resident bytes) and not a peak-RSS claim.

Campaign C — attribution

3e6ab8052f8d3dc2 isolates the windowing commits. Only 4 of 12 tests survive Holm at family
size 12, so read the rest as unresolved:

operation N=4 N=8 N=16 Holm
propagate[pauli] 1.24× (0.80) 1.48× (0.68) 1.90× (0.53) survives at all three
build_graph[pauli] 1.17× (0.85) 1.29× (0.78) 1.43× (0.70) survives at N=8 only
energy[pauli] 0.97× 1.02× 1.04× none
gradient[pauli] 0.94× 0.99× 1.00× none

Subtracting C from A: propagate's gain is the windowing, and energy/gradient's 4.96×/3.05×
at N=16 is not
— it comes from the commits below 3e6ab805, of which the point-to-point graph
replay is the only one touching that path. This also explains the apparent energy 1.04× in C: the
replay commit had already taken that cell from 4589 ms to 883 ms, leaving the windowing nothing to win.

Strong ladder (Leonardo DCGP, measured at 27da5fad — routing only, no follow-through)

3 arms × 2 layouts × 5 rungs × 2 reps, interleaved inside one allocation per rung, ITAC on every cell:

N R main this ratio main msgs/rank/layer this
1 8 200.8 s 188.8 s 0.94 19,992 1,274
2 16 113.6 s 98.7 s 0.87 42,840 1,350
4 32 67.6 s 49.8 s 0.74 88,536 1,388
8 64 47.9 s 25.3 s 0.53 179,928 1,395
16 128 46.5 s 15.0 s 0.32 362,712 1,397

Strong efficiency 27% → 79%; on 4×28, 31% → 82%. Messages per rank per layer are flat in R
(1,274 → 1,397, +10% across a 16× increase in R) against main's +18×, i.e. 260× fewer at R=128.
The residual +10% is the self-peer effect (shift 0 ⇒ the peer is me ⇒ memcpy, no message), which
saturates. main's reversal is gone: 4×28 turns over at n8→n16 (45.6 → 48.0 s) while this keeps
falling (28.2 → 17.2 s). Term count was 1,569,152,761 on every arm at every flat world.

The same lever applies to the Pauli path with no code change (a Pauli in symplectic form is a vector
in F₂^2n and multiplication is XOR of those vectors up to phase). On the 127-qubit heavy-hex kicked
Ising, fanout 1 gives 1.039 max/mean at R=128 against splitmix's 1.052. The identity-generator skip
does not transfer: those were an artifact of Hubbard's interaction gates.

Correctness

Routing changes which rank owns a term, so cross-rank miss indices mint in a different order and
reductions reassociate. The bar is term count exact, expectation value within a few ULP, with
bit-identity required on the splitmix arm.

  • splitmix is bit-identical to the base at every (world, S) pair tested — same term count and
    the same 64 bits of expectation value.
  • Linear routing: term count exact in every configuration, worst deviation 1 ULP.
  • 309/309 ctest at 2f8d3dc2, and 593/593 MPI tests at flat worlds 2, 32 (two layouts) and
    256. routing_tests is 17 cases, hybrid_comm_tests 20, plus a new flat_exchange_tests (6) for
    the point-to-point replay — including the shift identity route(M⊕G) == route(M) ⊕ Δ(G), fanout
    exactly 1, window() agreeing with peer()/contains() in both the sparse and dense cases, the
    non-power-of-two throw, and the derived wire plan over an empty partition-0 row.

Collapsing d removes tests by construction, so the surviving cases were checked to still reach
both routers rather than assumed to.

Scope

Routing/transport only — no gate fusion; gates stay atomic and there is still exactly one exchange
per generator per pass. The dense path stays, because R == 1 routes through it.

Net +1,694 / −664 over the ten new commits and +3,231 / −430 for the branch, of which
HybridComm.h is +331 / −127. The earlier claim of "~120 lines" was measured before the
follow-through; the windowing replaced the parallel O(P) indexing but did not shrink the file, and
the only net deletion in the stack is the d collapse at −139. What actually got simpler is the
per-exchange serial work (O(R·S²)O(f·S²)), the per-generator bookkeeping (O(P)O(S)), one
Router where routing was two inline expressions, and a boolean where there was a four-valued dial.

Composing with #263 (measured at 27da5fad, before the follow-through)

4 arms × 8×14 × {4, 8, 16} nodes × 2 reps, interleaved in one allocation per rung:

N R main #263 alone this alone both vs this alone msgs/rank/layer KB/rank/layer
4 32 67.0 s 53.2 s 50.1 s 40.7 s -19% 88,536 -> 1,388 852k -> 368k
8 64 52.1 s 54.7 s* 26.2 s 22.5 s -14% 179,928 -> 1,395 433k -> 187k
16 128 44.7 s 40.9 s 14.4 s 13.8 s -4% 362,712 -> 1,397 221k -> 96k

* one cold rep (42.0 / 67.4 s); the #263-vs-main read at n8 is not usable.

The two levers compose in the mechanism: the stacked arm carries this branch's message count
unchanged and roughly half its wire volume at every rung. Routing moves messages, #263 moves
bytes, and neither undoes the other. The wall gain is sub-additive and shrinks with R, which is what
fanout 1 implies: per-peer cost is multiplied by 1 rather than by (R−1).

At 27da5fad the residual was 90–91% MPI_Waitall at a thread spread of 1.02–1.04 — latency
everyone pays alike — which is what the follow-through commits attack.

Two questions this raises, answered

Should the splitmix arm be removed? No. It is the bit-identity control for a change that moves
term ownership, it is the opt-out arm every ratio in Campaign B is measured against, and it is the
recovery path when the generator shifts span only ρ < d dimensions and 2^d − 2^ρ ranks would sit
empty (which report_routing_coverage_ warns about). Removing it would also not remove splitmix:
SplitmixHash is the hash for OperatorIndex and MonomialMap and is untouched by routing.

Should the linear hash be used elsewhere — e.g. for OperatorIndex? No, and this is the one place
it must not go. Table is open-addressing with linear probing. A hash of the form ⊕_{i∈supp} v_i
is exactly 2-independent, not 3-independent: if M₃ = M₁ ⊕ M₂ then h(M₃) is determined by the
other two. Pătraşcu–Thorup showed 2-independence is insufficient for linear probing, and monoprop's
key set is precisely XOR-generated — the scan inserts M ⊕ G for a fixed small generator set — so
the dependent triples are the workload, not a corner case. Keep fold_hash/spread as they are. (For
the same reason, an XOR set checksum cancels duplicates and is not a safe agreement digest.)

One free variable is left unused: linear_basis is drawn at random, but the generator list is
replicated and known at construction, so the basis could be fitted so the shifts provably span
F₂^d — turning the coverage warning into a guarantee. Tracked as a follow-up, not done here.

Relation to prior work

The PR description points at arXiv:2506.13241 (Broers, Sun &
Yunoki, Scalable Simulation of Quantum Many-Body Dynamics with Or-Represented Quantum Algebra;
Phys. Rev. Applied 26, 024046). Worth being precise, because it is not the same hash, and the
difference is the whole result.

ORQA's distribution map (their eq. 12) is

f(I) = [ Σ_j I^(j,k) ] mod N

— the 2n-bit multi-index cut into k-bit blocks, each read as an integer, summed over ℤ_N. That is
additive over the integers mod N. The gate acts by XOR, and addition mod N carries, so the two
operations do not commute. Their eq. 13 makes the consequence explicit:

f(I⊻J) = m + Σ_{j=1..2|J|} ±2^(J_j mod k)   mod N

The signs depend on the current bits of I, so the destination is not determined by J alone: they
bound it at 2^(2|J|+1) processes (17 in their setting). That is a large reduction in fanout, and
it is why the paper needs a stochastic perturbation (their eq. 14) to recover load balance afterwards.

This PR takes the hash linear over the same group the gate acts by. Monomials under a gate form
(F₂^{2n}, ⊕), and h(M) = ⊕_{i ∈ supp(M)} v_i is a homomorphism of it, so h(M ⊕ G) = h(M) ⊕ h(G)
is an identity, not a bound. Three things follow that eq. 13 does not give:

  • Fanout is exactly 1, independent of |supp G| and of R — not 2^(2|J|+1), and not a function
    of the gate weight at all.
  • The pairing is an involution, so a fixed generator partitions the ranks into disjoint pairs that
    each side derives with no communication. The response retraces the query for free.
  • Balance is structural: the fibres h⁻¹(r) are cosets of ker h, all of size 2^(2n-d), so a
    uniformly drawn monomial is balanced by construction. No perturbation term is needed — measured rank
    max/mean 1.001 at R=128 from layer 5 on.

Two further differences: our routing is two-level where ORQA is flat; and they move updates with
one-sided RMA (MPI_Put) rather than the Isend/Irecv pairs here. Two things they have that we do
not, both tracked as follow-ups: the RMA transport, and scale — they report strong scaling to 2^17
processes on Fugaku against the R=128 measured above.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added configurable monomial routing with splitmix and linear modes.
    • Added optional deterministic routing seeds through environment settings.
    • Improved MPI communication by selecting optimized pairwise exchanges for suitable network layouts.
    • Added support for efficient sparse peer communication and partition-aware routing.
  • Bug Fixes

    • Added clearer startup validation for incompatible configurations and inconsistent routing settings.
    • Improved diagnostics when routing coverage is incomplete.
  • Documentation

    • Documented routing configuration, requirements, diagnostics, and validation behavior.

@github-actions

Copy link
Copy Markdown

Docs preview: https://pr-296.monoprop-docs.pages.dev

@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.70%. Comparing base (c5e88c8) to head (290112c).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #296   +/-   ##
=======================================
  Coverage   97.70%   97.70%           
=======================================
  Files          14       14           
  Lines         742      742           
  Branches       98       98           
=======================================
  Hits          725      725           
  Misses         12       12           
  Partials        5        5           
Flag Coverage Δ
cpp 97.70% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves MPI scalability by changing term routing so that (when the MPI rank count is a power of two) each generator’s cross-rank queries route to a predictable XOR-derived peer set, enabling sparse point-to-point exchanges and keeping messages-per-rank approximately flat in R. It also adds correctness/consistency checks and new test coverage to ensure routing and exchange behavior stays aligned across the codebase.

Changes:

  • Introduce routing::Router for a two-level routing scheme (GF(2)-linear across MPI ranks, splitmix within-rank partitions) and route all ownership decisions through it.
  • Add mpi::PeerPlan and integrate sparse peer exchange into HybridComm / MPICompat alltoallv paths, plus skip exchange work for identity generators.
  • Expand tests to pin routing identities, scan/find-rank agreement under both routers, and correctness of sparse peer delivery.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
cpp/monoprop/detail/mpi/Routing.h New routing::Router implementing two-level (linear+splitmix) routing with env-configured defaults.
cpp/monoprop/detail/mpi/MPIUtils.h Route ownership via Router, add router_for(), and add a cross-rank routing-agreement check.
cpp/monoprop/detail/mpi/Comm.h Add mpi::PeerPlan to describe sparse reachable peers for point-to-point exchange.
cpp/monoprop/detail/mpi/MPICompat.h Thread PeerPlan through alltoallv helpers and add sparse point-to-point fallback for S==1 MPI-only comms.
cpp/monoprop/detail/mpi/MPICompat.cpp Extend alltoall_counts to support sparse plans, including MPI-only point-to-point counts exchange.
cpp/monoprop/detail/mpi/HybridComm.h Add sparse-plan support to count/payload phases (plan-aware packing, exchange, and scatter).
cpp/monoprop/detail/mpi/MPICompat.h / .cpp Add geometry() split (ranks vs partitions) and use it to drive routing/plan decisions.
cpp/monoprop/detail/mpi/CMakeLists.txt Register new Routing.h header in build sources.
cpp/monoprop/detail/evolution/layer_build/Scan.h Emit query destinations via Router (single source of truth for ownership).
cpp/monoprop/detail/evolution/layer_build/Engine.h Derive PeerPlan per generator from Router and use it in query/response exchanges; skip exchanges for identity generators.
cpp/monoprop/detail/monomial_propagator/MonomialPropagator.inl Seed initial ownership using Router and enforce routing agreement at construction.
cpp/tests/routing_tests.cpp New tests pinning routing invariants (bit-identical d=0, GF(2) linearity, shift identity, fanout, defaults).
cpp/tests/mpi_utils_tests.cpp Update scan/find-rank agreement test to validate both routing modes and pass Router through.
cpp/tests/hybrid_comm_tests.cpp Add tests ensuring sparse plans deliver only to plan peers on both Hybrid and plain-MPI paths.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cpp/monoprop/detail/mpi/MPIUtils.h Outdated
Comment thread cpp/monoprop/detail/mpi/Routing.h Outdated
Comment thread cpp/monoprop/detail/mpi/Comm.h Outdated
Comment thread cpp/tests/routing_tests.cpp
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 29, 2026
diagonal-hamiltonian added a commit that referenced this pull request Aug 29, 2026
PR #296 made rank routing GF(2)-linear and converted the graph BUILD path,
but replay still posted MPI_Ialltoallv over all R ranks -- so propagate(),
which Hubbard calls 29 times per build_graph, paid a full collective to move
one peer's worth of doubles. post_flat_alltoallv now counts the legs carrying
a payload and, at or below num_ranks/4 of them, posts Irecv/Isend pairs over
those legs instead.

No plan and no count round are needed: derive_exchange_layout already hands
both sides the same array, so what a rank sends a peer IS that peer's recv
count and both ends drop the same legs on the same value. sparse_pairwise
drives it unchanged, with a dense PeerPlan walking [0, R) and posting only the
non-zero legs; the self slot stays a memcpy. Its request vector moves into the
Ticket, which now drains it in wait() the way PendingAlltoallv does -- resized
once and indexed, never push_back'ed, because MPI holds those pointers until
the wait.

The branch is RANK-LOCAL, and that is a precondition rather than a proof: a
rank choosing the collective waits forever on ranks that chose point-to-point.
The default routing (linear, d = log2 R) gives fanout 1, so every rank's row
holds at most one active leg and no row can straddle the budget. splitmix
routing (monoprop_ROUTING=splitmix, or an explicit d) does not, and a layer
whose per-rank partner counts land near num_ranks/4 can split the branch --
documented on flat_exchange_prefers_pairwise, not fixed here.

layer_exchange_participates is unchanged for the same reason: the symmetric
layout does let every rank agree on whether IT transfers anything, but the
collective arm is still reachable, so skipping the round at local total 0
would strand it.

Assisted-by: ClaudeCode:claude-opus-5
@robertodr
robertodr force-pushed the pr/query-wire-v3 branch 3 times, most recently from 3098f5c to 5068659 Compare September 1, 2026 11:03
diagonal-hamiltonian added a commit that referenced this pull request Sep 1, 2026
PR #296 made rank routing GF(2)-linear and converted the graph BUILD path,
but replay still posted MPI_Ialltoallv over all R ranks -- so propagate(),
which Hubbard calls 29 times per build_graph, paid a full collective to move
one peer's worth of doubles. post_flat_alltoallv now counts the legs carrying
a payload and, at or below num_ranks/4 of them, posts Irecv/Isend pairs over
those legs instead.

No plan and no count round are needed: derive_exchange_layout already hands
both sides the same array, so what a rank sends a peer IS that peer's recv
count and both ends drop the same legs on the same value. sparse_pairwise
drives it unchanged, with a dense PeerPlan walking [0, R) and posting only the
non-zero legs; the self slot stays a memcpy. Its request vector moves into the
Ticket, which now drains it in wait() the way PendingAlltoallv does -- resized
once and indexed, never push_back'ed, because MPI holds those pointers until
the wait.

The branch is RANK-LOCAL, and that is a precondition rather than a proof: a
rank choosing the collective waits forever on ranks that chose point-to-point.
The default routing (linear, d = log2 R) gives fanout 1, so every rank's row
holds at most one active leg and no row can straddle the budget. splitmix
routing (monoprop_ROUTING=splitmix, or an explicit d) does not, and a layer
whose per-rank partner counts land near num_ranks/4 can split the branch --
documented on flat_exchange_prefers_pairwise, not fixed here.

layer_exchange_participates is unchanged for the same reason: the symmetric
layout does let every rank agree on whether IT transfers anything, but the
collective arm is still reachable, so skipping the round at local total 0
would strand it.

Assisted-by: ClaudeCode:claude-opus-5
@diagonal-hamiltonian
diagonal-hamiltonian force-pushed the perf/linear-routing-on-wire branch from 2f8d3dc to 84bdf3e Compare September 1, 2026 13:09
@github-actions github-actions Bot added dependencies Pull requests that update a dependency file ci labels Sep 1, 2026
robertodr pushed a commit that referenced this pull request Sep 1, 2026
#324)

🤖 _AI text below_ 🤖

## Summary

The C++ `mpi` ctest variants spawn their own `mpiexec -n <n>` from
inside the test, so the `--map-by :OVERSUBSCRIBE` that the neighbouring
Python steps pass on the command line never reaches them — only the
environment can. The variable both justfile recipes already export,
`OMPI_MCA_rmaps_base_oversubscribe`, is the **OpenMPI 4** spelling and
OpenMPI 5 ignores it.

## Checklist

- [x] Tests added or updated to cover the changes — n/a; this is CI
configuration. Verified by direct measurement against OpenMPI 5.0.8
(table above). Note it cannot be proven by this PR's own CI, because
`main` runs the sweep at 2 ranks where the bug is invisible; #296
exercises it at 4.
- [x] Documentation updated (docstrings, `docs/`, `CONTRIBUTING.md`) if
needed — n/a
- [x] `CHANGELOG` / release notes updated if applicable — n/a

## AI/LLM disclosure

- [x] I used the following tool to help write this PR description:
Claude Code (claude-opus-5)
- [x] I used the following tool to generate or modify code: Claude Code
(claude-opus-5)

---------

Signed-off-by: Roberto Di Remigio Eikås <robertodr@users.noreply.github.com>
robertodr pushed a commit that referenced this pull request Sep 1, 2026
PR #296 made rank routing GF(2)-linear and converted the graph BUILD path,
but replay still posted MPI_Ialltoallv over all R ranks -- so propagate(),
which Hubbard calls 29 times per build_graph, paid a full collective to move
one peer's worth of doubles. post_flat_alltoallv now counts the legs carrying
a payload and, at or below num_ranks/4 of them, posts Irecv/Isend pairs over
those legs instead.

No plan and no count round are needed: derive_exchange_layout already hands
both sides the same array, so what a rank sends a peer IS that peer's recv
count and both ends drop the same legs on the same value. sparse_pairwise
drives it unchanged, with a dense PeerPlan walking [0, R) and posting only the
non-zero legs; the self slot stays a memcpy. Its request vector moves into the
Ticket, which now drains it in wait() the way PendingAlltoallv does -- resized
once and indexed, never push_back'ed, because MPI holds those pointers until
the wait.

The branch is RANK-LOCAL, and that is a precondition rather than a proof: a
rank choosing the collective waits forever on ranks that chose point-to-point.
The default routing (linear, d = log2 R) gives fanout 1, so every rank's row
holds at most one active leg and no row can straddle the budget. splitmix
routing (monoprop_ROUTING=splitmix, or an explicit d) does not, and a layer
whose per-rank partner counts land near num_ranks/4 can split the branch --
documented on flat_exchange_prefers_pairwise, not fixed here.

layer_exchange_participates is unchanged for the same reason: the symmetric
layout does let every rank agree on whether IT transfers anything, but the
collective arm is still reachable, so skipping the round at local total 0
would strand it.

Assisted-by: ClaudeCode:claude-opus-5
@robertodr
robertodr force-pushed the perf/linear-routing-on-wire branch from 84bdf3e to f5201e6 Compare September 1, 2026 13:57
Base automatically changed from pr/query-wire-v3 to main September 1, 2026 14:23
diagonal-hamiltonian added a commit that referenced this pull request Sep 1, 2026
PR #296 made rank routing GF(2)-linear and converted the graph BUILD path,
but replay still posted MPI_Ialltoallv over all R ranks -- so propagate(),
which Hubbard calls 29 times per build_graph, paid a full collective to move
one peer's worth of doubles. post_flat_alltoallv now counts the legs carrying
a payload and, at or below num_ranks/4 of them, posts Irecv/Isend pairs over
those legs instead.

No plan and no count round are needed: derive_exchange_layout already hands
both sides the same array, so what a rank sends a peer IS that peer's recv
count and both ends drop the same legs on the same value. sparse_pairwise
drives it unchanged, with a dense PeerPlan walking [0, R) and posting only the
non-zero legs; the self slot stays a memcpy. Its request vector moves into the
Ticket, which now drains it in wait() the way PendingAlltoallv does -- resized
once and indexed, never push_back'ed, because MPI holds those pointers until
the wait.

The branch is RANK-LOCAL, and that is a precondition rather than a proof: a
rank choosing the collective waits forever on ranks that chose point-to-point.
The default routing (linear, d = log2 R) gives fanout 1, so every rank's row
holds at most one active leg and no row can straddle the budget. splitmix
routing (monoprop_ROUTING=splitmix, or an explicit d) does not, and a layer
whose per-rank partner counts land near num_ranks/4 can split the branch --
documented on flat_exchange_prefers_pairwise, not fixed here.

layer_exchange_participates is unchanged for the same reason: the symmetric
layout does let every rank agree on whether IT transfers anything, but the
collective arm is still reachable, so skipping the round at local total 0
would strand it.

Assisted-by: ClaudeCode:claude-opus-5
@diagonal-hamiltonian
diagonal-hamiltonian force-pushed the perf/linear-routing-on-wire branch from f5201e6 to 5ada3da Compare September 1, 2026 15:51
@diagonal-hamiltonian

Copy link
Copy Markdown
Collaborator Author

🤖 AI text below 🤖

Rebased onto main after #263 merged

#263 merged as 364a5f50, so this branch's first six commits are now upstream and have been
dropped. The 25 routing commits are replayed onto main. By git range-diff against the old
stack, 22 of the 25 are identical and three differ, all three only by the span conversion:
commits 1 (route through one Router) and 16 (route the emit path from the generator's shift)
change one line each, SelfQueryStage::push(pos, k, phase) -> push(pos, phase); commit 22 is
the substantive adaptation described below.

The note at the top of this description was wrong, and I have corrected it: it said merging
#263 would retarget this branch "unchanged". It didn't, because #263 picked up a review refactor
after that line was written — pointer+length pairs became std::span, and two out-params became
returned aggregates (QueryWire::Decoded, MergedPartner). This branch had 67 call sites of the
converted APIs, so the rebase needed a real adaptation, not a retarget.

What the rebase had to resolve

21 of the 25 commits replayed clean. One — perf(evolution): size the per-generator query path to the peer window — conflicted in five files, because it rewrites exactly the hot path #263's review
had just re-typed. Every hunk was the same shape, routing's window semantics against main's span
API, with both sides wanted:

file resolution
Resolve.h kept main's positions_at and this branch's sender_index/sender_slot; the deserialize loop keeps its window indexing (si, buf) and reads through the Decoded-returning read_query
Scan.h this branch's at_slot(r_prime) re-basing door, main's span-taking push
Engine.h this branch's window loop, including counts(R, 0) — the explicit zero-init is load-bearing, since the loop writes only in-window slots
mpi_utils_tests.cpp kq/bucket (the outer loop now owns k) through the span read_positions
sparse_resolve_tests.cpp the window-indexed buf ref, span push

Two decisions worth surfacing, both places where taking this branch's side verbatim would have
silently undone a merged review outcome:

  • Three asserts stayed deleted. size the per-generator query path re-spells three asserts for
    the window API (queries_r.at_slot(my_rank).empty(), ls.size() == self_stage_.size(), and the
    src_idx_r[wi].size() == count_queries(...) walk). Checked against the pre-review head by text:
    all three are perf(evolution): ⚡ carry cross-rank queries as a width-adaptive record #263 asserts that the review removed. They are not reintroduced. The five asserts
    now in Engine.h beyond main's two are this branch's own window/router invariants and are
    untouched.
  • Resolve.h needed <cassert> restored. main dropped the include when its last assert went;
    this branch has two of its own O(1) window-consistency preconditions there, which the rebase
    otherwise left without a declaration.

Gates

Rebased head, one dev-x86 node, foss/2025b, monoprop_ENABLE_MPI=ON, MAX_NUM_MODES=1024,
installed _core.abi3.so md5 3f7e1818d2c7ced26076a9a351bb4843:

  • ctest -L unit309/309, rc=0
  • ctest -L serial306/306, rc=0

And the multi-rank sweep on two dev-x86 nodes, which is the gate that matters here since every
conflict was in the multi-rank query path:

  • ctest -L serial 306/306, ctest -L mpi 2/2
  • Python suite --with-mpi, 600 passed at each of R x S = 2x1, 2x16, 4x8 and 16x16 (world 2 to
    256; every R a power of two, which this branch now requires rather than falling back silently)
  • ALL SUITES PASSED, job exit 0:0

monoprop_link_export_probe passed here (2.31 s). That case is documented as red on main
itself (#313, MPI_Comm_size before MPI_Init). I have not rebuilt main in the same
configuration to confirm, and this branch's monoprop_EXPORT commit exports mpi::geometry — a
function this branch adds — so it is not obviously the cause. Flagging the observation, not
claiming the fix.

No re-measurement, and why

The span conversion is precisely what was A/B'd for #263 before it merged: 10 interleaved reps per
cell in one allocation, ratios divided per rep then median, Holm-corrected across six cells at two
layouts. No cell resolved as slower; peak VmHWM 1.00x / 1.01x. The routing logic in this branch is
untouched by the rebase, and every timing table below is still labelled with the commit it was
measured on.

diagonal-hamiltonian and others added 23 commits September 5, 2026 09:15
Under a sparse plan begin_alltoallv copied the caller's known_recv_counts
verbatim, zeroing only the self slot, while alltoall_counts already masked
the counts it exchanged. A non-zero count for a rank outside the peer set
therefore sized recv_buffer for bytes no Irecv ever writes, and wait_into
handed the caller uninitialised memory with no error. PeerPlan::contains
answers membership as a low-bits equality, so the mask costs no allocation.
publish_recv_rows_ takes the plan for the same reason: it summed over every
rank while its one reader masked to peers.

The self slot is a copy rather than a message and its two counts are each
other's transpose, so both point-to-point loops now assert that instead of
reading the send buffer through the recv count -- reachable whenever a
generator's shift is zero, which happens in essentially every layer.

Ranks that all agree on a WRONG shift stay symmetric and never hang; they
drop the blocks outside the peer set silently. pack_count_matrix_ asserts
the non-peer remainder is empty, and Comm.h no longer claims a deadlock is
the only failure mode.

Routing knobs move into EnvConfig.h, which already owned this job: raw
strtol left monoprop_ROUTE_LINEAR_BITS=abc parsing as 0, turning linear
routing off with no diagnostic. Unparseable and out-of-range values now
throw. monoprop_ROUTING's documented default was also backwards -- linear
has been the default since 40041c1.

check_routing_agreement reduces two independent digests rather than one:
allreduce_sum is the only collective here and a sum is not an equality
test. Partitions join the digest, since S enters Router::dest and two ranks
differing only in S agreed before and still routed apart.

Drops the flat-world find_rank overload. It had no production caller left
and answered splitmix during a linear run, which is precisely the silent
ownership split its own comment warned about.

Assisted-by: ClaudeCode:claude-opus-5
The sparse plan arrived as a second copy of each dense path: four
point-to-point loops differing only in member-vs-local request storage,
byte-vs-typed pointers and the tag, two byte-identical scatters, two
recv-column fills around one loop nest, and eleven plan.dense() branches.
Five branches remain and all five earn it -- four are the collectives
themselves, where R-1 Isends would be a regression against an Alltoall,
and one is the known-recv-count mask, which the dense path must not pay.

Pairwise.h also gives the four MPI tags one home. They were bare magic
numbers in three files, and the reason Engine.h's two exchange rounds may
share one on a single communicator -- non-overtaking within
(src, dst, tag, comm), the query round's Waitall preceding any round-2
post, and both ends skipping a zero-count leg on the same value by
transpose -- was written nowhere. It is load-bearing, so it is written down.

Peers materialise once per verb rather than being recomputed S*f times
inside the staging loops, and exchange_payload_ takes the extent its
callers already hold instead of asking MPI for it per call.

Net line count rises: the extracted helper is a new file, and four copies
collapsing into one is the point rather than the arithmetic. Routing.h's
derivation moves to the parallelism docs page, keeping only the invariant
that Scan.h and find_rank must agree.

Assisted-by: ClaudeCode:claude-opus-5
routing::gf2_rank called itself the coverage diagnostic and had no caller
outside its own test, so the check it describes never ran. Linear routing
reaches only the subspace the per-generator shifts span: at rank rho below
linear_bits, 2^d - 2^rho ranks receive nothing all run and the imbalance
looks like slow peers rather than a routing property.

Not beside check_routing_agreement, where it belongs conceptually -- at
construction the gate list does not exist yet. It runs once at the top of
the gate loop, where the generators first arrive, and only under linear
routing, so splitmix and non-power-of-two geometries pay nothing.

A warning rather than a throw: every term still lands on exactly one owner,
so the answer is right and only the balance is not. One COMMROUTE line per
rank, in COMMPLACE's shape. An out-of-range gate index leaves the report
unwritten so it cannot pre-empt build_evolve_result_'s per-gate throw.

Assisted-by: ClaudeCode:claude-opus-5
The skip tested gen.any() after the scan had already run. An identity
generator anticommutes with nothing, so the scan returns on its empty
fold-column set having produced no query, no cosine block and no swept
coefficient -- work that was being done to discover it was not needed.
Hoisting the test above it also skips the cos-block concatenation.

LayerBuildEngine construction stays: its ctor sizes the caller-owned
matched scratch, which is reported as matched_scratch_bytes, and skipping
it would move that telemetry when a propagator's first gate is identity.
The ctor is O(R); everything expensive is now behind the test.

Assisted-by: ClaudeCode:claude-opus-5
dest() walked the monomial's set bits, loading a basis vector and XOR-ing
per bit -- a data-dependent chain ~20-28 long under the production cutoff,
per term, in the hottest loop here. monomial_hash beside it is a single mix
of one word, so the new hash dominated the destination, not the old one.

Only the low d bits survive the mask, so transpose: plane j holds bit j of
every basis vector, and bit j of the image is parity(popcount(M & plane_j)).
Folding the words with XOR before the popcount is the same parity, so it is
d popcounts rather than d per word -- 14 branch-free ops at R=128 over 120
slots. The planes key on the seed and the width alone, never the geometry,
so one table serves every Router and dest() binds a pointer at construction
instead of meeting the static-init guard per term.

Bit-identical, which is the acceptance criterion and not an aspiration:
routing_transposed_basis_is_bit_identical_to_the_bit_walk pins dest() and
rank_shift() against an independent reference of the old walk over 100k
monomials across several (R, S, d), and asserts the comparison count so it
cannot pass vacuously.

The constructor is private now: a router with linear bits has to come
through for_modes, because reading planes bound at another width would be
silent. splitmix stays width-free -- with d = 0 no plane is ever read.

Assisted-by: ClaudeCode:claude-opus-5
The dense branch posted MPI_Ialltoallv and completed it in wait_into while
the sparse branch waited inline, so the two transports differed in a way
nothing in the signature showed. The handle now carries the request set and
waits with the rest; MPI reads send_buffer and recv_buffer until those
complete, and both move with the handle, so the pointers stay good.

No win is claimed: both consumers call wait_into immediately, so there is
nothing to overlap yet. This is the seam that a later overlap needs, and
one path fewer to reason about.

Sizing no longer sweeps [0, R) three times per exchange: counts and their
prefix fold into one pass, and the known-recv mask copies the f peer blocks
into a freshly zeroed array rather than copying all R and zeroing the
remainder. At R=128 this is noise; at 4096, against ~1,400 exchanges per
rank per layer, it is not.

Assisted-by: ClaudeCode:claude-opus-5
Both sparse cases pinned plan.count(R) == 1, so every `for k in [0, f)` in
the transport had only ever run one iteration -- and the rank list defaulted
to 2, where a plan with any linear bits can resolve only one peer, so no
amount of local testing would have reached f > 1 either. Adding 4 to the
list is what makes the multi-peer cases runnable at all.

Four gaps closed: several peers, where peer-ordered blocks interleave with
the [0, R) prefix sums in the staging sizers; an empty leg, so the zero-count
skip is taken on one side only -- the asymmetry that deadlocks; skip_self
under shift 0, which is the self-peer slot; and two rounds back to back on
one communicator, which is the pattern Pairwise.h's non-overtaking argument
claims is safe, now asserted rather than argued.

The scan/find_rank floors count per router. They were written for one loop
and kept when a second router was added, so each arm's floor was really the
pair's. Measured, all six routers agree on a total of 387 while the split
runs 196/191 at R=2 to 335/52 at R=8 -- the partner count belongs to the
operator and the gate, and routing only moves a partner between the
cross-rank and self-owned side. That invariance is the assertion now; the
floors only catch a scan that emitted nothing.

Assisted-by: ClaudeCode:claude-opus-5
The GF(2)-linear rank routing supported any d in [0, log2 R], but only d = 0
(monoprop_ROUTING=splitmix) and d = log2 R (the shipped default) were ever run;
the intermediate values existed only to be tested. Router now carries a mode
flag: linear takes every rank bit from the hash, splitmix takes none.

A rank count that is not a power of two no longer falls back to d = 0 silently
under linear routing -- it raises routing::UnroutableGeometry at Router
construction. R = 1 is a power of two, takes no rank bit, and stays dense, so
every single-rank run keeps the collective transport.

PeerPlan follows: {bool sparse; int shift} with one peer, me ^ shift, which
removes the 1 << bits signed-shift hazard entirely rather than narrowing it.
monoprop_ROUTE_LINEAR_BITS and its parser are gone; monoprop_ROUTING and
monoprop_ROUTE_SEED are unchanged, and d = 0 stays bit-identical to
monomial_hash(M) % P.

BREAKING CHANGE: monoprop_ROUTE_LINEAR_BITS is removed, and a non-power-of-two
MPI rank count now raises under the default linear routing instead of falling
back to the dense all-to-all.

Assisted-by: ClaudeCode:claude-opus-5
rank(M^G) == rank(M) ^ rank_shift(G) is exact under linear routing, and
build_layer already derives rank_shift(G) once per gate for the PeerPlan --
which is to say the linear planes Scan.h evaluated for every emitted query
were recomputing a per-generator constant. Router::dest_from_shift takes the
rank bits from this rank's own slot XOR that shift, leaving only the
partition index per term; splitmix has no such identity and falls back to
dest() bit for bit.

Two strength reductions in dest() ride along, both blocked only by parts_
being a runtime member. `q % S` becomes `q & (S - 1)` when S is a power of
two, which every production layout is (S in 1,2,4,8,16); and at S == 1 the
partition index is 0 for every term, so monomial_hash is not evaluated at
all. Both are identities, not approximations.

Per term at the production geometry the emit path was
  tzcnt; log2(R) x (kW loads + kW ands + kW xors + popcnt + shift/or);
  imul; splitmix mix64; 64-bit divq
and is now
  shrx; xor; imul; splitmix mix64; and.
The plane loop, its popcounts and the division are gone; the mix64 stays
because the partition index still needs it (and goes too at S == 1).

Ownership must not move by a term, so the identity is asserted rather than
argued: a debug assert at the fast path against dest(), and a sweep in
routing_tests over nine geometries and both modes, including S == 1, S = 3
and S = 14 so the mask path and the division path are both covered.

dest_from_shift requires that the local operator hold only terms this rank
owns -- the precondition mpi::PeerPlan already carries, since it sends every
query for a gate to me ^ shift. The scan/find_rank agreement test was
feeding one rank the whole operator, which no rank ever holds; it now
distributes by find_rank and runs every rank. The six routers still agree
on a total of 387 partners, and the split becomes all-encoded under linear
routing (shift 1, 3, 7) against 212/175 to 343/44 under splitmix.

Assisted-by: ClaudeCode:claude-opus-5
exchange_count_blocks_ posted its Isend/Irecv pairs and MPI_Waitall'd on
them in the same breath, inside the partition-0 serial section between B1
and B2, while pack_send_ -- the only real per-partition work in the verb --
did not start until after B2. The count block is S*S ints, 1 KB at S=16: an
eager message that needs no cooperation from the peer, so nothing about it
justified blocking the packing behind it.

Split into post_count_blocks_ / wait_count_blocks_, with the wait moved to
where the counts are first genuinely read. That reader is fill_recv_col_ via
block_sum_, not the send-side sizing: size_staging_send_ works off the rows
each partition published before B1, and pack_send_ off that sizing, so both
run with the round in flight. The recv-side sizing and the payload exchange
follow the wait in the B3->B4 window, and the per-partition extraction of
recv_counts / recv_displs moves past B4, which is the first point at which
counts_recv_ exists. Still four syncs; no byte moves differently and no
delivery order changes.

The dense arm is untouched by construction -- MPI_Alltoall is blocking and
cannot be split, so plan.dense() completes inside the post and leaves
count_posted_ at zero, making the wait a no-op. Only the sparse arm splits.

The count requests live in their own count_reqs_, never the payload's reqs_.
The current ordering drains the count round before exchange_payload_ posts,
so one vector would in fact be safe today; separate storage is what keeps
Pairwise.h's resize-once-then-index rule from turning into a use-after-
realloc if the wait is ever moved again.

Three cases: the same peer-masked layout through the dense and the sparse
arm, compared element for element on both against each other and against
the tags, so a dropped or torn count block changes a length; the self-peer
plan at shift 0, where the count round posts nothing at all and the wait is
the no-op path; and a zero-count peer, whose payload legs are skipped while
its count block still travels, followed by an all-silent round over the
first's staging high-water bytes.

Assisted-by: ClaudeCode:claude-opus-5
PR #296 made rank routing GF(2)-linear and converted the graph BUILD path,
but replay still posted MPI_Ialltoallv over all R ranks -- so propagate(),
which Hubbard calls 29 times per build_graph, paid a full collective to move
one peer's worth of doubles. post_flat_alltoallv now counts the legs carrying
a payload and, at or below num_ranks/4 of them, posts Irecv/Isend pairs over
those legs instead.

No plan and no count round are needed: derive_exchange_layout already hands
both sides the same array, so what a rank sends a peer IS that peer's recv
count and both ends drop the same legs on the same value. sparse_pairwise
drives it unchanged, with a dense PeerPlan walking [0, R) and posting only the
non-zero legs; the self slot stays a memcpy. Its request vector moves into the
Ticket, which now drains it in wait() the way PendingAlltoallv does -- resized
once and indexed, never push_back'ed, because MPI holds those pointers until
the wait.

The branch is RANK-LOCAL, and that is a precondition rather than a proof: a
rank choosing the collective waits forever on ranks that chose point-to-point.
The default routing (linear, d = log2 R) gives fanout 1, so every rank's row
holds at most one active leg and no row can straddle the budget. splitmix
routing (monoprop_ROUTING=splitmix, or an explicit d) does not, and a layer
whose per-rank partner counts land near num_ranks/4 can split the branch --
documented on flat_exchange_prefers_pairwise, not fixed here.

layer_exchange_participates is unchanged for the same reason: the symmetric
layout does let every rank agree on whether IT transfers anything, but the
collective arm is still reachable, so skipping the round at local total 0
would strand it.

Assisted-by: ClaudeCode:claude-opus-5
The leg-count budget was a rank-LOCAL predicate over a rank-varying quantity,
so two ranks could land on opposite sides of it: one enters MPI_Ialltoallv and
waits forever on the other, which posted point-to-point. The floor at 1 made
the default routing safe by accident (fanout 1 gives every row 0 or 1 legs) and
left splitmix and small-d configurations able to hang. Numeric tuning cannot
fix that, so the budget is gone.

The transport now keys on `wire_bits`, the resolved linear-bit count when the
routing gives fanout 1, derived once in Evolution.cpp from
routing::linear_bits_for -- the same number check_routing_agreement allreduces
at construction and throws on. That makes it rank-uniform by construction, and
it is also the actual reason the legs are empty. Any other geometry passes 0
and keeps today's collective. Exchange.h learns no routing: it takes an int.

Kind::Hybrid gets it too, which is the layout that matters -- 8 ranks/node x 16
partitions went through the dense collective with no plan at all. The wire plan
cannot come from the call site: only partition 0 reaches MPI, and its own row
may be the empty one while a sibling holds the rank's only traffic. So
partition 0 derives it in the B1->B2 window, where the published recv rows give
the first view wider than one partition, and an empty rank resolves to the self
peer rather than to dense -- keeping the branch a function of the gate alone.
Asserted lossless against the send rows. Only the wire is narrowed; the serial
O(R*S^2) staging sweeps still walk every rank, which needs the per-generator
shift the recorded graph does not carry.

sparse_pairwise takes an `active_legs` upper bound, so a dense plan over a
one-leg layout no longer sizes its request vector at 2R (64 KB per exchange at
R=4096, one malloc/free each). The Kind::Mpi arm keeps the DENSE plan on
purpose: it walks all R and posts the non-zero legs, so no derived shift can
drop a block there.

Assisted-by: ClaudeCode:claude-opus-5
Four breaks a clean three-way merge did not surface, because each side edited a
different line:

- `PeerPlan{.bits=}` in `derived_wire_plan_` and three `hybrid_comm_tests` cases,
  written against the int dial the routing commit replaced with `.sparse`.
- `routing::linear_bits_for`, deleted with the dial but still the replay
  transport's gate. Restored through `Router::bits_for`, which IS the private
  constructor, so the resolution and the non-power-of-two throw cannot drift.
- `sparse_count`, dropped as the deleted fanout-2 case's only helper; the count
  round's new cases had since become a second user.
- The fanout-2 sweep in `..._split_count_round_matches_the_dense_arm`: a sparse
  plan is fanout 1 by construction, so `f > 1` is no longer expressible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Under GF(2)-linear routing a generator's queries all land on one peer rank, so the
reachable flat slots are that rank's S partitions -- one contiguous run of the P=R*S
world instead of all of it. SlotWindow names the run, WindowIndex is its re-based index
(a distinct type: a flat slot used as one would otherwise stay in bounds and address the
wrong peer), and WindowVec is a vector over the run whose only flat-slot door asserts
membership.

PeerPlan::window derives it in one expression per field; dense is its count == P value,
not a second case. No caller yet.

Assisted-by: ClaudeCode:claude-opus-5
Under linear routing a generator's queries all land on one peer rank, so the reachable
destinations are that rank's S slots, not the P=R*S world. Every per-generator per-slot
structure from the scan to the wire was still allocated and swept over all P: at R=128,
S=16 that is length-2048 arrays with 16 live entries, built twice per generator for ~416
non-identity generators over 29 layers.

The six FusedScanResult arrays, the engine's queries_r / src_idx_r / src_val_r /
combined_qv_, the probe's goff / sender ids, the resolver's responses, and
begin_alltoallv's counts / pack / prefix / unpack sweeps are now window-length.
begin_alltoallv derives the window from the plan alone, so dense is the count == P value
of the same expression rather than a second arm; a caller may still hand it a whole [P]
array, which the window then masks.

Re-basing is safe by construction rather than by review: WindowVec::at_slot is the only
place a flat slot becomes an index, and it asserts membership, while operator[] takes a
WindowIndex so a bare flat slot will not compile. Self is inside the window only when the
rank shift is zero, which resolve_self_queries now branches on and asserts against an
empty self stage. GraphSink::acc stays flat [P] -- build_layer_storage_unified is
P-shaped -- so the sink turns the window index back into a slot.

MPICompat could not be deferred: the scan's arrays are moved into the engine and thence
onto the wire with no seam that does not cost a P-allocation to bridge.

Assisted-by: ClaudeCode:claude-opus-5
The sizing that partition 0 runs between B1 and B2 still swept the whole P=R*S world
while S-1 partitions parked at the barrier, even though every sweep that writes these
tables and every sweep that reads them walks the same peer set. col_sum_ (twice per
send sizing) and recv_col_ are now zeroed over the peer slots only, and the two
displacement prefixes walk peers_ instead of all R -- peers_ is ascending in both arms,
so the prefix takes the same value at every peer as the full one, a non-peer
contributing zero.

publish_recv_rows_ runs on every partition, not just 0: it now zeroes the row and sums
only the peers' blocks, replacing R*S adds with R stores and one block's worth. The row
stays fully written because derived_wire_plan_ reads all of it.

The [R] count and displacement arrays are still zeroed in full: MPI_Alltoallv reads
every entry on the dense arm, and the zeros are what make the narrowed prefix exact.

Assisted-by: ClaudeCode:claude-opus-5
alltoallv's derive_wire_bits path had no test and no library caller. It exists because
only partition 0 reaches MPI while its own row may be the empty one, so the peer set has
to be read off every partition's published recv rows; reading partition 0's alone
resolves to the self peer, whose legs are all zero, and every block is dropped with no
hang to show for it.

The case puts the rank's only traffic on partition S-1 with partition 0 sending and
receiving nothing, and checks the payload arrives carrying its source's global id -- so
it fails on a plan that names the wrong peer as well as on one that names none.

Assisted-by: ClaudeCode:claude-opus-5
Routing reaches geometry() from headers -- find_rank and router_for in MPIUtils.h,
the engine's coverage report, and begin_alltoallv, which derives the peer window from
it -- so the symbol now crosses the hidden-visibility boundary that
link_export_probe.x links against, and the probe failed to link without this. Every
other MPICompat free function a public template chain reaches already carries the
attribute.

Assisted-by: ClaudeCode:claude-opus-5
find_rank lost its rank-count overload when routing moved behind
routing::Router -- deliberately, since that overload would answer
splitmix during a linear run. routing_tests.cpp was updated with it;
this call site was not, and it took every build job in CI down.

Router::splitmix(slots) is bit-identical to the monomial_hash % slots
this used to call (pinned by routing_splitmix_is_bit_identical_to_hash_mod_p),
so the partition property under test is unchanged -- and unlike a linear
router it accepts the non-power-of-two slots = 3 case.

Assisted-by: Pi:claude-opus-5
@robertodr
robertodr force-pushed the perf/linear-routing-on-wire branch from f666c5d to aa4f8bf Compare September 5, 2026 07:15
diagonal-hamiltonian added a commit that referenced this pull request Sep 7, 2026
…the follow-through tip

The 2x2 composite loses its (a)-(d) lettering: columns are now headed "Strong scaling" and
"Weak scaling", so the lower-right panel says what it is instead of making the reader count
panels and decode a caption. Rows are the quantity, columns the family, which is the
arrangement that lets neighbouring axes mean the same thing.

Sharing follows from that. All four panels span 128-8192 cores, so one x axis serves them
all -- cores labels on the bottom row, the nodes axis on the top. The efficiency row shares
y as well, being one quantity on one 0-100% scale, and its right panel drops tick labels
that were a verbatim copy of the left's.

The wall-time row deliberately does NOT share y. Strong spans 6.9-454 s and weak 10-217 s,
and clip_y sizes each panel to its own data; a shared range let the narrower weak limits win
and pushed the tallest strong curves (437 s, 454 s) off the top of the frame. A figure that
hides two of its own measurements is worse than one with two y scales.

Data re-measured on the #296 follow-through tip (`_core.so` ce208a74, worktree a204dc13):
239 reps over 45 rungs, zero failing their gate. At 8192 cores strong efficiency reads
48/81/99% against the previous 14/39/74%, and the strong-scaling reversal is gone -- the
1.6B curve kept descending to 64 nodes instead of turning up at 32.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci cpp dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants