Skip to content

Redistributed Submesh and MeshHierarchy - #5270

Open
pbrubeck wants to merge 4 commits into
pbrubeck/adaptive-multigridfrom
pbrubeck/mesh-redistribution
Open

Redistributed Submesh and MeshHierarchy#5270
pbrubeck wants to merge 4 commits into
pbrubeck/adaptive-multigridfrom
pbrubeck/mesh-redistribution

Conversation

@pbrubeck

@pbrubeck pbrubeck commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Stack

This is PR 2 of 3: #5288 (adaptive-multigrid) -> this PR (mesh-redistribution) -> #5287 (assign-submesh-restricted).

Summary

Mesh redistribution is now a Submesh of the mesh it was redistributed from, rather than an ad-hoc
redist attribute bolted onto the mesh.

  • Submesh(mesh, redistribute=True) repartitions instead of inheriting the parent's parallel
    distribution, for both subdomain_id=None (the whole mesh, as needed by a redistributed
    MeshHierarchy) and a proper subdomain (co-dimension 0 only). The child records a
    submesh_point_sf pushing the parent's plex points onto its own; submesh_shares_distribution
    tells consumers (assembly, interpolation) whether two related meshes still share a distribution.
  • Entity orientations are preserved across redistribution: the child inherits the parent's universal
    vertex numbering (dmcommon.submesh_vertex_numbering) and quadrilateral/hexahedral cell
    orientations (dmcommon.submesh_cell_orientations), so cell closures are ordered identically on
    both sides and RT/BDM/N1curl/RTCF/RTCE data transfers exactly instead of picking up sign/permutation
    flips.
  • Assembling or interpolating across a redistributed mesh and its parent raises NotImplementedError,
    since entity maps are only defined when the two share a distribution; use Function.assign instead.
  • MeshHierarchy (redistribute=True by default) and refine_marked_elements/HierarchyBase.adapt
    redistribute a refined/adapted mesh via Submesh whenever it would otherwise leave empty ranks.
    firedrake/mg/interface.py (prolong/restrict/inject) and firedrake/mg/utils.py
    (transfer_mesh, set_dm_refine_level, set_level) route transfers through the pre-redistribution
    mesh, then assign onto the redistributed one.

Testing

Redistribution round-trips exactly (<= 1e-16) for tri/quad/tet/hex x CG/DG/RT/BDM/N1curl/RTCF/RTCE on
1-3 ranks, including curved (P2) and periodic (DG1) coordinate fields
(tests/firedrake/submesh/test_submesh_assign.py). tests/firedrake/multigrid/test_redist_mesh.py
covers uniform and adaptive MeshHierarchy redistribution (2/3/4 ranks); test_submesh_assemble.py
covers the NotImplementedError paths. All pass at every process count their markers declare.

🤖 Generated with Claude Code

@pbrubeck
pbrubeck requested a review from connorjward July 21, 2026 17:22
@pbrubeck
pbrubeck marked this pull request as ready for review July 22, 2026 07:17
Comment thread firedrake/mg/mesh.py
Comment thread tests/firedrake/multigrid/test_redist_mesh.py
Comment thread tests/firedrake/multigrid/test_redist_mesh.py
Comment thread tests/firedrake/multigrid/test_redist_mesh.py
@connorjward

Copy link
Copy Markdown
Contributor

Is this ready for review? You seem to still be working on it.

@pbrubeck

Copy link
Copy Markdown
Contributor Author

Is this ready for review? You seem to still be working on it.

It is ready you can review it, but ideally we should merge #5213 and #5215 first.

I'm adding the review suggestions from the meeting, and I have a genuine question: should we introduce a new redistribute kwarg or should we pass it through the distribution_parameters?

Comment thread firedrake/redist.py Outdated
from pyop2.mpi import MPI


class RedistributedMeshTransfer:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move this to Function.assign()? Maybe we can do it more generalically for Submesh, RestrictedFunctionSpace, and RestributedMesh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think so.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now can redistribute a Mesh/Submesh. I am only adding support for assign across the submesh and its parent, which is what redistributed multigrid needs.

@pbrubeck pbrubeck changed the title Mesh redistribution support (split from #5215) Submesh(mesh, redistribute=True) Jul 28, 2026
@pbrubeck pbrubeck changed the title Submesh(mesh, redistribute=True) Redistributed Submesh and MeshHierarchy Jul 28, 2026
Comment thread firedrake/mg/mesh.py Outdated
@pbrubeck
pbrubeck changed the base branch from pbrubeck/mg-redist to pbrubeck/adaptive-multigrid July 30, 2026 16:37
@pbrubeck
pbrubeck force-pushed the pbrubeck/mesh-redistribution branch from 8e51000 to f4b9c56 Compare July 30, 2026 16:38
@pbrubeck pbrubeck added the base:main Run this PR using a main (dev) build label Jul 30, 2026
@pbrubeck
pbrubeck force-pushed the pbrubeck/adaptive-multigrid branch from af6c7df to 1a70084 Compare July 31, 2026 13:29
@pbrubeck
pbrubeck force-pushed the pbrubeck/mesh-redistribution branch 2 times, most recently from a55c288 to 2a1e2f6 Compare July 31, 2026 13:38
Comment thread firedrake/mesh.py Outdated
Comment thread firedrake/mesh.py Outdated
Comment thread firedrake/mesh.py Outdated
@pbrubeck
pbrubeck force-pushed the pbrubeck/adaptive-multigrid branch 2 times, most recently from 1c2b268 to 994901b Compare August 8, 2026 12:15
pbrubeck and others added 4 commits August 8, 2026 13:37
An adaptive refinement can leave some ranks with no cells of their
own when the marked region is small or clustered, and the same can
happen to a uniform MeshHierarchy level on a coarse base mesh. Teach
Submesh to redistribute a mesh onto a balanced partition while
inheriting the parent's universal vertex numbering and cell
orientations, so that the redistributed mesh's nodes stay in
one-to-one correspondence with the parent's.

MeshHierarchy and refine_marked_elements grow a redistribute= option
that, when a level would otherwise have empty ranks, replaces it with
such a redistributed Submesh. The transfer operators run on the
parent-owned mesh the cell maps were built from and then Assigner
moves the result across the point SF relating parent and submesh, via
a new Assigner._assign_redistributed path.
Co-authored-by: Pablo Brubeck <brubeck@protonmail.com>
Split the sentences that stack clauses, and say what invalidates the
inherited entity classification rather than what stops holding.
Submesh() builds a submesh it does not redistribute with the no-op
distribution parameters, and then overwrites them with the parameters of the
parent. Say which of the two the condition tests, and why the overwrite is
right.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

base:main Run this PR using a main (dev) build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants