Conversation
_set_desired_longitude_range decided whether to wrap by asking
lon.max() > 180. On a dask-backed coordinate that reduction is a compute,
and Grid.__init__ calls it -- so opening a chunked grid read and reduced
every longitude array before the caller had asked for anything, and did it
again on every isel and every copy(), each of which builds a new Grid.
The wrap is now xr.where((lon > 180) | (lon < -180), (lon + 180) % 360 - 180,
lon): elementwise, lazy, chunk-parallel, no reduction.
Measured on a synthetic 4M-node UGRID file, chunked at 500k nodes, best of 5:
dask computes wall tracemalloc peak
open_grid (before) 3 23.3ms 12.2 MB
open_grid (after) 2 16.2ms 1.0 MB
isel (before) 2
isel (after) 1
The computes that remain are a separate site on connectivity rather than
coordinates: _standardize_connectivity's conn.isnull().any() in io/_ugrid.py,
reached twice on the read path, and _slice_face_indices in grid/slice.py
materializing the connectivity it slices by. Neither is touched here.
Three behavioural differences, all from doing this per element rather than
per array.
* In-range longitudes are now left exactly alone. (lon + 180) - 180 does
not round-trip, so wrapping the whole array perturbed values that were
already in range by up to 3e-14 degrees -- in outCSne30, 2.1182935e-14
became 2.8421709e-14. On the elements that do need wrapping the two forms
are bit-identical.
* Longitudes below -180 are normalized. The old test was on the maximum
alone, so it reached the negative tail only when the same array also held
a value above 180.
* Both endpoints are kept, so the interval is the closed [-180, 180].
Folding 180.0 to -180.0 would match _xyz_to_lonlat_deg, which wraps
unconditionally into the half-open interval, but it breaks
antimeridian_face_indices: that reads a face as crossing from the span of
its longitudes, and a face with one vertex at 180 and the rest near -170
goes from a span of 350 to a span of 10 and disappears. Caught by
test_antimeridian_point_on and
test_to_geodataframe_preserves_antimeridian_faces.
Each variable is wrapped at most once, keyed on the xr.Variable object.
Without that, edge_lat -- which calls this on every property access,
outside its populate guard -- would stack a where layer onto the graph per
access. Keying on the Variable makes the memo
self-invalidating: assigning into _ds replaces that object, so a repopulated
or user-assigned coordinate is wrapped again.
The two Exodus round-trip tests compared a grid against its own
lon -> xyz -> lon reload with assert_allclose(rtol=1e-8), and passed only
because the old whole-array wrap applied to the original the identical
perturbation the reload applies. With the original left alone, the reload's
own error is exposed, and rtol is the wrong instrument for it twice over: a
longitude near zero has no magnitude for a relative tolerance to measure
against (outCSne30 nodes 4372, 4749, 7e-15 degrees apart), and longitude is
periodic, so a node on or one ulp short of the antimeridian reads 180.0 on
the original and -180.0 on the reload -- the same meridian, scored as a
360-degree error (179 nodes of outRLL1deg; outCSne30 nodes 3966, 5155).
Those two assertions now compare the difference modulo 360 to an absolute
tolerance, still ERROR_TOLERANCE. The helper still catches a 1e-7 shift and
still rejects an antipode.
Tier 0.2 of the chunked refactor plan.
Test suite: 962 passed, 1 skipped. test_plot_with_features fails identically
before and after (matplotlib figure size, unrelated).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The elementwise wrap is the right shape for a dask-backed coordinate and the
wrong one for an array already in memory. There was never a compute to defer
on that path, only a scan, and elementwise costs a pass plus ~18 bytes per
node of temporaries -- two bool masks, the arithmetic, the result -- where a
reduction costs a pass and allocates nothing.
That matters because every open_grid/open_dataset call in the benchmark suite
is eager; none pass chunks=. So the previous commit, measured there, was a
regression and nothing else.
Measured on a synthetic 4M-node UGRID file, best of 5. The wrap in isolation:
in range 0..360
old reduction 4.6ms 0MB 36.8ms 64MB
elementwise only 20.9ms 72MB 38.1ms 72MB
elementwise + guard 9.3ms 0MB 42.7ms 72MB
and through eager open_grid, where the file read dominates and the peak does
not move at all (180.0 MB in every arm):
in range 0..360
base (cmd/nogil) 145.5ms 181.9ms
elementwise only 164.4ms 186.1ms
elementwise + guard 147.9ms 185.9ms
_lon_within_range is a guard, not a decision: when it is true the wrap is the
identity on every element, so skipping it cannot change a value.
test_eager_fast_path_agrees_with_the_where_element_for_element asserts that
directly against the unguarded expression rather than assuming it, over five
inputs including both endpoints, the negative tail and NaN.
The reductions are the thing that made the old code compute, so they are
allowed only where there is nothing to defer -- da.chunks is None. A
dask-backed array skips the guard entirely, which
test_guard_is_skipped_for_dask_backed_arrays pins by asserting the guard does
compute when handed one. The chunked numbers are unchanged: 2 graph
executions, 16.0ms, 1.0 MB peak.
`and` short-circuits, so an array that does need wrapping usually pays a
single max -- the same reduction the old code paid -- before falling through.
The 0..360 column above is that extra max: ~4.6ms on a 190ms open_grid.
Test suite: 973 passed, 1 skipped. test_plot_with_features fails identically
before and after (matplotlib figure size, unrelated).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every other open_grid in the suite is eager, so none of it can see what
Grid construction does to a dask-backed grid -- which is the path the lazy
longitude wrap changed, and the path the rest of the chunked refactor will
keep changing. OpenGridChunked adds time_open_grid and
track_peakmem_open_grid, parametrized over the oQU 480km and 120km meshes
already registered in helpers/_fixtures.py.
Chunks are held at N_CHUNKS=4 per grid dimension, so the graph is the same
shape at both resolutions and only the data under it grows. The file is read
directly rather than through CachedFixtures, because reading it is the
subject.
On MPAS this is where the lazy wrap matters most. node_lon, edge_lon and
face_lon all exist at construction, so the old max() > 180 check ran three
computes per open; the branch runs none. Measured with this benchmark's own
setup, best of 15, against HEAD's tree with coordinates.py taken from
cmd/nogil:
time tracemalloc peak
base branch base branch
480km 60.5ms 56.5ms 3.27MB 3.27MB
120km 42.3ms 38.0ms 3.53MB 4.02MB
The 120km peak reads higher on the branch, and it is not data. With gc
disabled, building the where graph allocates ~1.9 MB of transient objects,
nearly all in inspect.signature via dask/xarray op dispatch -- the same at
both resolutions and with a single chunk, so it does not scale with the
grid. At 480km it sits under the HDF5 read's own high-water mark; at 120km
the netCDF3 read is cheap enough that it becomes the peak. After a
gc.collect() the branch retains ~30 kB more than base, which is the extra
graph layers. Worth knowing for later steps: this benchmark sees
graph-construction overhead, not only bytes read.
Three things the numbers above depend on:
* Compare each resolution to its own history, not to the other. The two
files are different formats -- oQU480.grid.nc is netCDF4/HDF5,
oQU120.grid.nc is netCDF3 -- and the HDF5 open costs more, so 480km reads
slower than 120km despite a sixteenth of the data.
* Each open gets a fresh copy of the chunks dict. match_chunks_to_ugrid
(core/utils.py) writes the source-format dimension names into the
dict it is handed, so a reused one gives every sample after the first a
different argument.
* The warning "The specified chunks separate the stored chunks" is
filtered. oQU480 stores layerThickness, ssh and zMid as one chunk of all
1791 cells, so any n_face chunking splits them and xarray warns once per
open -- about data variables the grid reader drops.
Checked by calling the class the way asv does (setup(param), then the
time_/track_ methods) under -W error::UserWarning. Not run through asv
itself: its discovery subprocess cannot import uxarray from the uxarray
conda env, where the package is not installed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmdupuis3
added
scalability
ASV BenchmarkingBenchmark Comparison ResultsBenchmarks that have improved:
Benchmarks that have stayed the same:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1792
Overview
_set_desired_longitude_rangedoesif da.max() > 180on a lazy reduction, three times (node_lon, edge_lon, face_lon). Re-fires on everyGridconstruction, so everyiseland everycopy()pays it again.PR Checklist
General
Testing & Benchmarking
Documentation and Examples
docs/api.rst; internal (private) function names start with an underscore (_)AI Disclosure
AI Usage: Claude Opus 5.5