Load-balance, active-box windowing, block-structured AMR#1628
Open
sbryngelson wants to merge 357 commits into
Open
Load-balance, active-box windowing, block-structured AMR#1628sbryngelson wants to merge 357 commits into
sbryngelson wants to merge 357 commits into
Conversation
# Conflicts: # src/simulation/m_ibm.fpp
… (fold regression)
… (fold regression)
…_markers+ghost_points across the host pointer swap (was CUDA_ERROR_ILLEGAL_ADDRESS in s_ibm_setup_fine on device)
…_markers+ghost_points across the host pointer swap (was CUDA_ERROR_ILLEGAL_ADDRESS in s_ibm_setup_fine on device)
… fine-block IB recompute at fluid-ghost sub-time
…lerance + support-matrix docs
… fine-block IB recompute at fluid-ghost sub-time
…lerance + support-matrix docs
…coverage (static/subcycle/moving-IB restart_check) + doc
…ay meshes (serial verified; parallel mirrors sim reader)
…-poly override_tol 5e-9 + lint fixes (abort-msg '...', case.md sys_size ref)
…s + non-poly tolerance # Conflicts: # toolchain/mfc/test/cases.py
…n 3 IB kernels + OMP attach/detach ACC-parity (validated CPU+ACC+nvfortran-OMP+amdflang-OMP, ghost pts 276 all)
…CC/nvfortran-OMP/amdflang-OMP)
…kers%sf) on the IB kernels alone fixes IB-AMR on OpenMP (validated nvfortran-OMP 3/3); keeps the shared macro untouched
…itialized Newton guess -> NaN under reldebug NaN-init; mirrors coarse s_compute_q_T_sf)
…ess tensor (MFlowCode#1633); skip chaotic mibm_periodic_collision golden
… two-body static IB+AMR golden F980C769
…dent FD coefficients on fine swap/restore; add 1D hypoelastic AMR golden ACE05393
…t_damage, THINC (int_comp), and body forces under subcycle
… on the coarse grid (fine advance skips coarse-index spatials), startup abort on support/block overlap; golden 65C375B4
…port stays coarse (tags suppressed, candidate boxes clipped clear, internal assert); golden 2FC423D3
…pb/mv side-state (piecewise-constant prolong, own rhs scratch, restrict with moments), rank-span guard; golden BCBA6E74
sbryngelson
marked this pull request as ready for review
July 5, 2026 15:49
…model amr.md and case.md described a mirror fine decomposition and a dedicated s_mpi_sendrecv_amr_fine_halo routine that does not exist anywhere in src/ - the code implements single-owner blocks assigned by Morton-SFC chains-on-chains work balancing at every regrid (with migration), coupled through P2P coarse<->fine gather/scatter, with a same-level fine-fine seam halo for tiled adjacent blocks. The PR description's matching phrase was updated in the same pass. Half-extent cap corrected: it is half the GLOBAL extent (amr_maxc), not half a rank's subdomain.
s_amr_assign_block_owners balanced blocks by pure geometric fine cell count, ignoring the calibrated cost model merged alongside it - two equal-size blocks weighed the same whether one held an immersed body or phase-change hot spots. Add s_amr_block_cost: each rank sums the load-weight model (base 1 + K_ib per IB-marked cell + K_pc per phase-change Newton iteration when that diagnostic array is live) over its coarse cells inside each block's footprint, one MPI_ALLREDUCE makes the vector replicated, and the chains-on-chains assignment runs on cost x ref_ratio**(level*dims) in real arithmetic (fixed loop order on replicated inputs -> identical on every rank). With no cost signals the weight reduces to the footprint count - geometry-only behavior. K_* constants move to m_constants (shared with m_load_weight; avoids a module cycle). Ownership only relocates blocks - field values are owner-independent (np=1 == np=2 element-exact) - and the full AMR suite passes. The Lagrangian cloud is excluded from blocks by construction, so K_bub does not apply here.
…; document rebalance-by-restart and the swap-serialization ceiling The static split weighted planes by the first advection variable, but m_load_weight's own calibration note records that EE-bubble source cost is flat across a 2500x void range - the alpha marginal is not a load signal there. Use uniform marginals for bubbles_euler (the AMR fine-work injection still moves planes); Lagrangian and multi-fluid cases keep the probe. amr.md gains the cost-weighted-ownership note, the checkpoint-and-restart rebalancing recipe (s_load_balance_rebalance runs at every startup), and a scaling note on the sequential per-rank block advance.
…he overlap boxes, drop full-field host pulls
… full block volume s_amr_fill_fine_ghosts, its multi-fluid closure pass, s_amr_lerp_fine_ghosts, and both pbmv twins looped the full buffered fine extent with an interior-skip branch to touch only the O(surface) ghost shell - x3 per lockstep step, x6 subcycled. A shared pure helper (s_amr_build_ghost_slabs) decomposes the shell into up to 2*num_dims disjoint face slabs (x slabs full-transverse; y slabs x-restricted; z slabs x,y-restricted; union exactly the old mask's complement of the interior), and each kernel launches per slab with the identical per-cell body - ~6x fewer iterations at 64^2/buff 4 and no divergent mask branch. Full AMR suite (54) byte-identical; LoadBalance golden passes.
# Conflicts: # .typos.toml # src/simulation/m_checker.fpp
…ockstep contracts explicitly s_restrict_one_var lost its last caller when the scatter path went device-side - delete it (-43 LOC). The two remaining element-exact twin pairs (s_amr_restrict_overwrite_device / s_amr_restrict_pack_device and the pbmv pair) each gain an explicit TWIN cross-reference comment stating the lockstep-edit contract: same child-sum loop order, arithmetic, and casts, byte-identically, or owner-local and scattered coarse cells diverge. Kept as plain Fortran routines (no whole-routine macro emission). Full AMR suite passes on the merged tree.
sbryngelson
changed the title
…ubcycle seam tiling, and the load-diagnostic writers
…it the s_amr_regrid driver into per-phase subroutines (pure code motion)
…ortran build break) The s_amr_regrid phase split made old_np an intent(out) dummy of s_amr_regrid_stash_migrate while two BLOCK-scope declarations (cnt(old_np), getk(old_np)) use it as an array bound - F2018 bars intent(out) dummies from restricted (specification) expressions. Local gfortran tolerated it; the CI GNU lanes (coverage, ubuntu, macos, Phoenix cpu, convergence) all reject it at build. Mirror the value into a routine-scope local (np_l) captured at the same point and size the block arrays with that. Regrid goldens pass unchanged.
… same-level block intersection The gather/scatter coupling scanned all ranks (s_amr_rank_coarse_range + box intersection) per block per RK stage. The coarse decomposition is static and block boxes change only at regrid, so cache rank-ascending per-block gather and scatter lists (amr_ovl_gather/scatter + counts), rebuilt on the same dirty flag as the seam pairs - flagged before the regrid rebuild so its own gathers consume fresh lists. Message order and matching unchanged (lists preserve the ascending scan order). s_amr_check_seam_topology additionally aborts when two SAME-level blocks intersect: different levels legitimately nest, tiling emits disjoint tiles, and the L1 IB pass merges overlapping boxes, but the child IB body-bbox expansion has no overlap-merge pass - intersecting children would restrict and reflux the shared cells twice, silently breaking conservation. Full AMR suite (57) + LoadBalance + diagnostics goldens pass.
…king the full block volume" This reverts commit 4557a27.
…abort on same-level block intersection" This reverts commit af2d830.
…k only the overlap boxes, drop full-field host pulls" This reverts commit 276fb37.
…stale-bounds corruption) The seed-once guard from 2153da6 assumed the device copy of ab_int was dead weight for non-active_box runs. Under AMR the fine advance swaps idwint to the fine-block bounds every stage, and CCE OpenACC resolves the convert kernel's loop bounds through the PRESENT device copy of this module variable - the stale COARSE bounds on the swapped fine grid drove out-of-range device accesses and a deterministic spurious wave at every fine-block edge. This is why every AMR golden (and only AMR goldens) failed on Frontier CCE gpu-acc while CCE cpu/gpu-omp and NVHPC acc stayed green (they evaluate bounds host-side), and why reverting the slab/gather/cache perf commits never moved the failing numbers. Restore the unconditional per-call device update (6 integers) and document the constraint.
…ck only the overlap boxes, drop full-field host pulls" This reverts commit ab43047.
… abort on same-level block intersection" This reverts commit 0c01a4f.
…sking the full block volume" This reverts commit 61b71e4.
Maintainability review follow-up (comment-only, no behavior change): (1) Correct three comments that asserted falsehoods - amr_isect_lo/hi frame is level-dependent (L0 for level-1, parent-fine for level>=2, not a hard-coded 2x); the 'multi-level not yet landed / slots always 1' phrases (multi-level and multi-block ship in this PR); and the L0-reflux 'remaining piece' note (level>=2 blocks reflux to their PARENT via s_amr_reflux_to_parent, implemented). (2) Document the swap contract at the sw_* declarations and in common-pitfalls.md - grid-derived module state a kernel reads on the fine grid must be swapped or refreshed, device copy too (the ab_int regression class); flag amr_rvw as the next candidate; add an AMR note at s_compute_rhs. (3) Complete the TWIN lockstep convention on the q<->pb/mv axis (12 pairs), the coarse RK stage combination (s_tvd_rk <-> the two fine RK updaters), and the freg<->creg capture policy (the 'energy only when not viscous' rule now names all four sites).
The np=1 branches of s_amr_gather_coarse_patch and its pb/mv twin inlined a device kernel byte-identical to s_amr_gather_own_box_device / s_amr_gather_own_box_pbmv_device, which the np>1 path already calls. Replace the inline copies with the shared calls (-30 LOC), removing a drift surface. No behavior change - the surviving kernel is the one already exercised (verified: 21C71558/476AA3A4/1CBACEB5/BCBA6E74/DDD79C8B pass).
s_amr_igr_swap_sigma seeds the fine block's sigma Dirichlet data from the OWNER's LOCAL coarse jac, clamped to the owner's buffer bounds. Unlike q_cons that seed is not P2P-gathered, so a block whose footprint or ghost shell crosses a rank boundary reads clamped edge values instead of the neighbour rank's sigma - a silent wrong answer with zero coverage. Gate to num_procs = 1 (runtime-only, no case_validator mirror) until the sigma seed is distributed. IGR AMR goldens run ppn=1 and are unaffected (6C20B752/660FFBFE pass).
First user-facing example for block-structured AMR: a dense droplet in uniform flow, its interface tracked by a dynamically-regridded, subcycled 2:1 block while smooth regions stay coarse. Exercises the four required AMR settings (amr + initial block, amr_regrid_int > 0, amr_subcycle) plus amr_cluster_eff for the closed-interface case. Validated and smoke-run 150 steps / 15 regrids, clean (no cap, no NaN).
Group the refinement-ratio parameter with the amr_ family so it surfaces under ./mfc.sh params amr and cannot become a breaking rename after merge (reviewer's last fix-before-merge item). Pure mechanical rename of the namelist parameter and its Fortran variable across 8 source files, the param registry/validator/tests, and the AMR docs; the test trace label and golden field values are unchanged (no golden regeneration). Rebuilt all 3 targets; AMR/IGR goldens pass incl. DB9BF199 (the ref_ratio-4 case) and multi-level restart.
Privatize four symbols with no external caller (s_amr_swap_to_fine, s_amr_restore_coarse, s_amr_fill_fine_ghosts, amr_dt_fine) so the audited swap call-sites become a compiler guarantee; narrow m_start_up's bare 'use m_amr' to the five symbols it actually references. t_level stays public (it backs the externally-consumed amr_slots). Build + AMR goldens pass.
amr_block_beg/end note they are 0-based inclusive and required when amr = T; amr_tag_eps states the actual tagging criterion (max axial |rho(i+1)-rho(i-1)|/(2 rho_i)); add amr.md to the lint_docs freshness set (it passes clean).
75AD6885 (multi-level static block) had byte-identical case mods to 4AF96C49 (multi-level restart); restart_check runs the same straight-run golden compare before the restart roundtrip (test.py:547 then :578), so 4AF96C49 fully subsumes it. Removed the stanza + golden dir and folded its description into the surviving test.
Flip restart_check on the existing 3D static-block golden (476AA3A4): the restart reader validates the fine extent per axis, so a z-extent slip would pass every non-restart 3D golden. Reuses the straight-run golden (no new golden); the midpoint roundtrip passes, confirming 3D multi-axis AMR restart round-trips correctly.
Terse pass over the comments in the PR's new AMR/load-balance modules (m_amr, m_amr_regrid, m_amr_restart, m_amr_registers, m_active_box, m_load_balance, m_load_weight, m_sfc_partition, m_rank_timing, m_box): cut filler, redundant restatement, and obvious-code narration, and re-flow the mangled nest_children comments. Comment-only (~37% fewer words, net -124 lines) - every invariant, index/frame convention, compiler-quirk note, TWIN lockstep marker, and trap warning is preserved in substance. Build + 12 AMR/IGR/MHD/load-balance/active-box goldens bit-identical.
Terse pass over the PR-ADDED comments in m_ibm, m_checker, m_time_steppers, and m_rhs (pre-existing files - only the PR's own AMR / fine-IB / swap / gate comments were edited; verified no pre-existing comment or PROHIBIT message string was modified). Cut filler and run-ons while preserving every gate rationale, compiler-quirk note (CCE lib-4425 module separation, declare-target/present-table, ab_int device-copy, AMD defaultmap/ROCm), TWIN marker, and trap warning. Comment-only; build + 8 AMR/IB/IGR/subcycle/QBMM goldens bit-identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An opt-in, default-off family of performance features and the measurement infrastructure they rest on. With all flags at their defaults the only touched production path is
s_mpi_decompose_computational_domain, refactored through the newm_boxmodule (byte-identical; covered by the existing suite).m_box(partition arithmetic),m_load_weight/load_weight_wrt(per-cell load-weight field + imbalance metric),m_sfc_partition/sfc_partition_wrt(Morton-SFC predicted-imbalance diagnostic),m_load_balance/load_balance(weighted static decomposition at init; AMR-fine-work-aware),m_rank_timing/rank_time_wrt(per-rank compute-time diagnostic).m_active_box/active_box: restricts reconstruction/Riemann/RK windows to a light-cone-grown box around non-ambient flow; strict-subset golden-tested.hybrid_wenoandhybrid_riemann(+hybrid_weno_eps,hybrid_smooth_flux): linear-optimal weights / central-or-Rusanov flux in smooth cells, full WENO/HLLC at flagged discontinuities (Jameson sensor, stencil-dilated, per-level under AMR).m_amr+m_amr_registers: two-level 2:1 refined block hierarchy; conservative restriction and conservative-linear prolongation with physics-specific closures; per-stage flux registers with Berger–Colella refluxing; Berger–Rigoutsos multi-block dynamic regrid; optional dt/2 subcycling; multi-rank (single-owner blocks assigned by Morton-SFC work balancing at each regrid, with migration; blocks may span rank seams via P2P coarse↔fine gather/scatter; same-level seam halo; distributed registers); restart (both IO modes, regridded-layout persistence); AMR-aware post-processing (fine blocks visualizable as Silo overlay domains); GPU-resident fine level on both OpenACC and OpenMP offload.Full algorithm and user documentation:
docs/documentation/amr.md(support matrix enforced at runtime by the checker — unsupported combinations abort with named messages, never silently).AMR physics support matrix (abridged; authoritative table in amr.md)
Supported and golden-tested: single- and multi-fluid (5-eq,
mpp_lim) · 6-eq with per-block pressure relaxation · viscous (refluxed) · phase change (relax) · chemistry incl. species diffusion · Euler–Euler bubbles (polytropic/non-polytropic, mono/polydisperse, QBMM incl. non-polytropic with per-blockpb/mvside-state; dynamic regrid + subcycle) · acoustic sources (coarse-grid support with regrid exclusion) · immersed boundaries (multi-body, static or prescribed-motion, incl. dynamic regrid with body-containment expansion and per-substage guards) · 2D axisymmetric (per-block WENO-coefficient recompute) · stretched grids (exact parent-bisection ghost coordinates + per-swap coefficient recompute) · hybrid WENO/Riemann sensors (per-level) · Lagrangian bubbles (cloud excluded from blocks; two-way coupling on the coarse grid; regrid clips around the moving cloud) ·active_box(blocks contained in the growing window; agrees with plain AMR to ~1e-14) · IGR (restriction-only coupling: fine sigma solve seeded/Dirichlet-bounded by the coarse solve; documented truncation-order seam, exact free-stream) · 1D MHD/RMHD (div(B)=0 by construction in 1D; HLL and HLLD, incl. relativistic).Gated with named aborts (documented rationale): surface tension (seam force imbalance is structural — three fixes attempted and diagnosed in amr.md) · 2D/3D MHD (attempted and measured: the c/f seam is a continuous O(1) div(B) source GLM cleaning cannot remove — needs constrained-transport-class B prolongation/reflux) · hyperelasticity · 3D cylindrical (global azimuthal filter) · force-driven IB (
moving_ibm=2) · STL bodies · Riemann-extrapolation BCs (bc=-4) ·amr_subcycleunder IGR · stretched grids with Lagrangian/IB-regrid (uniform-spacing index conversions).Validation evidence
Known issues (all non-gating or in progress)
continue-on-error): an intermittent post-detected NaN on the two Lagrangian+AMR goldens. Exhaustively unreproducible off GitHub's runners — the exact failing stack (NVHPC 24.3 SDK,-tp=px -Kieee, HPC-X MPI, and the CI docker image itself under apptainer) passes elsewhere, as do native/zen2 builds; 24.5+ green. Documented at the golden definitions.Review guide
The commit history is arc-ordered (active-box → load-weight → SFC → weighted decomposition → rank timing → hybrid → m_box → AMR rungs → physics envelope → CI/GPU hardening); reviewing by arc is much easier than by file. The AMR arc builds stepwise: static hierarchy → restriction/prolongation → fine advance → refluxing → regrid → subcycling → multi-rank → GPU → each physics rung with its own validation. Commit messages carry the validation evidence for their change (measured defects, golden UUIDs, repro details for CI fixes).
All parameters ship default-off with
case_validatorentries, runtime checker gates, andcase.md/amr.mddocumentation.