Fix quadratic cost in adaptive mesh refinement's coarse/fine cell-map construction - #5326
Merged
Conversation
… construction adaptive_parent_child_cell_maps resolved each fine cell's coarse parent with DMLabelGetValue, called once per fine cell. DMLabelGetValue should resolve a point in O(1); instead it scans every stratum of the label linearly. set_adaptive_parent_label gives every coarse cell its own stratum, so the loop was effectively O(nfine * ncoarse). Walk the label by stratum instead (DMLabelGetStratumSize/GetStratumIS, one call per coarse cell), which uses PETSc's O(1) value->stratum hash map and touches each fine cell exactly once: O(nfine + ncoarse). Also wrap refine_marked_elements's previously-unattributed sub-steps in PETSc.Log.Event so -log_view attributes time to the call that spent it instead of lumping it into refine_marked_elements's own self-time. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pbrubeck
commented
Aug 5, 2026
The recorded multigrid iteration counts and convergence plot predate #5215's DMPlexTransform-based mesh adaptivity; regenerate both against current refine_marked_elements output. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Describe why walking by stratum is fast rather than reciting the removed point-wise loop's complexity. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pbrubeck
commented
Aug 5, 2026
connorjward
approved these changes
Aug 6, 2026
Contributor
Author
|
Thanks. CI failures are known. Merging |
leo-collins
pushed a commit
that referenced
this pull request
Aug 6, 2026
… construction (#5326) * Fix quadratic cost in adaptive mesh refinement's coarse/fine cell-map construction
haldaas
added a commit
to haldaas/firedrake
that referenced
this pull request
Aug 13, 2026
Submesh(mesh, comm=COMM_SELF) cost O(P^2) in the number of mesh points P. On three ranks a UnitSquareMesh(181, 181) needed 8.9 seconds to build one serial submesh. Four times more cells cost about ten times more time. Every preconditioner that builds a subdomain matrix through firedrake.preconditioners.matis pays this cost, because local_mesh builds one serial submesh per process. The cost was in submesh_correct_entity_classes. A submesh on COMM_SELF has no neighbour, so it has no ghost points and no owned points, and every point is core. The function set that state one point at a time. DMLabelSetValue moves the stratum into a hash set and destroys its index set. The DMLabelHasPoint of the next iteration reads the hash set back, sorts it, and rebuilds the index set and the bit array. Each point therefore cost O(k log k) in the size k of the core stratum so far. Set the three strata in bulk instead. This is the same fix as the one in PR firedrakeproject#5326 for the coarse and fine cell maps. submesh_create also built a temporary label that marked every cell, and passed it to DMPlexFilter. DMPlexFilter selects every cell by itself when it gets no label, so the label was unnecessary. Submesh now leaves label_name as None for a codim-0 submesh of the whole mesh. A codim-1 submesh still passes the "depth" label, because DMPlexFilter with no label selects cells, not facets. Measured on three ranks, for submesh_correct_entity_classes alone: cells before after 717 34 ms 0.12 ms 2787 320 ms 0.43 ms 11059 3214 ms 1.85 ms 22023 8907 ms 3.80 ms The whole submesh build at 11059 cells falls from 3256 ms to 71 ms. Both paths produce the same submesh. On UnitSquareMesh(128, 128) with three ranks, the submesh built with a label and the submesh built without one have the same chart, the same subpoint index set, and the same pyop2_core, pyop2_owned, pyop2_ghost and Face Sets strata. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pbrubeck
added a commit
that referenced
this pull request
Aug 18, 2026
… construction (#5326) * Fix quadratic cost in adaptive mesh refinement's coarse/fine cell-map construction
pbrubeck
added a commit
that referenced
this pull request
Aug 19, 2026
* Make serial submesh construction linear in the number of points Submesh(mesh, comm=COMM_SELF) cost O(P^2) in the number of mesh points P. On three ranks a UnitSquareMesh(181, 181) needed 8.9 seconds to build one serial submesh. Four times more cells cost about ten times more time. Every preconditioner that builds a subdomain matrix through firedrake.preconditioners.matis pays this cost, because local_mesh builds one serial submesh per process. The cost was in submesh_correct_entity_classes. A submesh on COMM_SELF has no neighbour, so it has no ghost points and no owned points, and every point is core. The function set that state one point at a time. DMLabelSetValue moves the stratum into a hash set and destroys its index set. The DMLabelHasPoint of the next iteration reads the hash set back, sorts it, and rebuilds the index set and the bit array. Each point therefore cost O(k log k) in the size k of the core stratum so far. Set the three strata in bulk instead. This is the same fix as the one in PR #5326 for the coarse and fine cell maps. submesh_create also built a temporary label that marked every cell, and passed it to DMPlexFilter. DMPlexFilter selects every cell by itself when it gets no label, so the label was unnecessary. Submesh now leaves label_name as None for a codim-0 submesh of the whole mesh. A codim-1 submesh still passes the "depth" label, because DMPlexFilter with no label selects cells, not facets. Measured on three ranks, for submesh_correct_entity_classes alone: cells before after 717 34 ms 0.12 ms 2787 320 ms 0.43 ms 11059 3214 ms 1.85 ms 22023 8907 ms 3.80 ms The whole submesh build at 11059 cells falls from 3256 ms to 71 ms. Both paths produce the same submesh. On UnitSquareMesh(128, 128) with three ranks, the submesh built with a label and the submesh built without one have the same chart, the same subpoint index set, and the same pyop2_core, pyop2_owned, pyop2_ghost and Face Sets strata. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Pablo Brubeck <brubeck@protonmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
adaptive_parent_child_cell_mapsresolved each fine cell's coarse parent with PETSc'sDMLabelGetValue, called once per fine cell.DMLabelGetValueshould resolve a point in O(1); instead it scans every stratum of the label linearly until it finds the one containing the queried point.set_adaptive_parent_labelgives every coarse cell its own stratum value, so on an adaptively refined mesh the per-fine-cell loop was effectively O(nfine · ncoarse), not O(nfine). Profilingadaptive_multigrid.pyunder-log_viewshowedrefine_marked_elementsdominated not by PETSc'srefine_sbrmesh transform itself, but by time attributed to this uninstrumented loop.adaptive_parent_child_cell_mapsnow walks the label by stratum instead, withDMLabelGetStratumSize/DMLabelGetStratumIS, one call per coarse cell. That uses PETSc's O(1) value→stratum hash map and touches each fine cell exactly once, so the loop is O(nfine + ncoarse). Output is unchanged — cross-checked bit-for-bit against the original point-wise result on refined netgen meshes.On the
AdaptiveMeshHierarchydemo (first 10 Dörfler-marked refinement levels of an L-shaped netgen mesh),adaptive_parent_child_cell_maps's share of time insiderefine_marked_elementsdrops from 59% to 8%, andrefine_marked_elements's own total drops by 55%. The dominant cost left insiderefine_marked_elementsis now legitimately PETSc's own mesh-refinement transform.Collateral changes
_adapt_marked_cellsandrefine_marked_elements(firedrake/adapt.py) gainedPETSc.Log.Eventwrappers around each of their previously-unattributed sub-steps (label marking,set_adaptive_parent_label, theadaptLabeltransform,Mesh()construction,adaptive_parent_child_cell_maps, re-marking, Netgen re-curving). Without them,-log_viewattributed all of this work torefine_marked_elements's own self-time rather than the call that actually spent it, which is what obscured this cost in the first place.The
adaptive_multigriddemo's recorded multigrid iteration counts and convergence plot predated #5215's DMPlexTransform-based mesh adaptivity; both are regenerated here against currentrefine_marked_elementsoutput.Tests
tests/firedrake/multigrid/test_adaptive_multigrid.pyalready exercisesadaptive_parent_child_cell_mapsacross firedrake/netgen meshes and process counts; it passes unchanged at nprocs 1/2/4.🤖 Generated with Claude Code