perf(geometry): instrument geometry-parse memory, and write down where it goes

[jcschaff]

Follow-on to #2021 / #2022. That pair asks "should an oversized image be refused"; this asks
why it costs so much in the first place, which is the question Jim raised.

No production code changes. This is the measurement plus the write-up
(docs/geometry-memory-notes.md); the optimisations are separate decisions.

The headline contradicts #2021

#2021 described eleven parses of a 61,920,000 pixel geometry exhausting a 1000 MB heap.
Measured at the same pixel count (396³ = 62,099,136):

phase                                        peak     retained     B/px    allocated     B/px      ms
0. synthetic pixels (byte[N])             69.9 MB      60.0 MB     1.01      59.2 MB     1.00      75
1. new VCImageUncompressed                69.9 MB        808 B     0.00       1.0 KB     0.00     110
2. new Geometry(name,image) [setImage]    69.9 MB      71.8 KB     0.00     829.3 KB     0.01      14
3. createSampledImage                    130.0 MB      60.0 MB     1.01      59.2 MB     1.00     156
4. RegionImage regions only (dim 0)      452.1 MB     240.1 MB     4.05     492.9 MB     8.32     472
5. RegionImage + surfaces (dim 3)      1,431.5 MB     450.3 MB     7.60   1,349.5 MB    22.79    1103
TOTAL (peak=max, rest=sum)             1,431.5 MB     810.4 MB    13.68   1,961.6 MB    33.12    1930

One parse peaks at ~1.4 GB against a 1000 MB heap. Not eleven — one. The pods were never
going to survive that request; the repetition only decided how far it got first.

Why eleven, and why that is the biggest lever

Xmlproducer.java:1416 writes the full <Geometry> element — image included — inside each
<SimulationContext>, and XmlReader.java:6024 parses each one independently. The prod request
(GET /api/v0/biomodel/101963252/simulation/98916046, from the PetalBot crawler) logged 15
getting simulation context rep lines and 11 image warnings.

So the largest available win is not in the geometry algorithms at all — it is parsing the
geometry once per document. 11× on every number in that table, plus an ~11× cut in VCML size
for multi-SimulationContext spatial models.

What is actually on the heap

jcmd GC.class_histogram at prod scale:

structure	instances	bytes
int[]	2,121	497.7 MB
byte[]	22,869	130.4 MB
MembraneEdgeNeighbor	1,920,800	61.5 MB
MembraneElementIdentifier	1,919,138	46.1 MB
Node / Node[] / ArrayList / SurfAndFace / Quadrilateral	~480 k each	~75 MB

Two things fall out:

• The 497 MB of int[] is mapImageIndexToLinkRegion — one int per pixel, 236.9 MB each —
  against a measured 16,706 link regions, which fit in two bytes. RegionImage:86-146 already
  carries a commented-out CompactUnsignedIntStorage that does exactly this byte→short→int
  promotion. Someone reached this conclusion before; it never got wired in.
• 480,200 quadrilaterals carry ~180 MB of objects — ~375 bytes per quad, where a quad's
  geometric content is four node indices (16 bytes).

The instrument

GeometryMemoryProfiler reports three numbers per phase because they answer three different
questions: peak (does the JVM survive), retained (how many concurrent requests fit),
allocated (collector pressure).

Two details that were needed to make the numbers trustworthy:

• A warmup pass runs first and is discarded. Without it the first measured phase absorbs class
  loading and JIT — the initial draft reported 7.9 MB retained and 41 MB allocated for
  new VCImageUncompressed, which actually costs 808 bytes.
• Retention holds every intermediate on purpose. An earlier draft let them fall out of scope
  and reported the resulting collection as a memory saving.

A census step reads the label arrays by reflection — no accessor added to production code — to
report the actual link-region count, which is what grounds the int→short claim rather than
assuming it.

I also deleted a hardcoded "bytes per polygon" constant I had initially put in the census: it was
a guess about object layout dressed up as a measurement. Those bytes now come from the histogram.

Opportunities (full detail in the doc, ordered by size)

1. Parse the geometry once per document — 11×, touches no geometry code.
2. Do not generate surfaces on XML read. XmlReader:2090 runs precomputeAll on every parse;
   an API request that serializes a model back out never uses surfaces. Note XmlReader:2086
   already parses the document's stored <SurfaceDescription> into a variable named dummy and
   discards it, immediately before recomputing surfaces from scratch.
3. Narrow the per-pixel label arrays — 237 MB → 118 MB via the promotion scheme already
   written and commented out.
4. Struct-of-arrays for the surface — ~375 B/quad → ~28 B/quad, but it ripples through
   SurfaceCollection, TaubinSmoothing, the STL exporter and the VTK path. Worth costing first.
5. Slab-wise streaming region finding — the algorithm is already two-pass connected components
   and the surface pass already streams two z-planes of nodes, so the structure is largely there.
6. Two small ones: createSampledImage copying a full byte[] when it could alias the original;
   getShortEncodedRegionIndexImage building a byte[2N] (124 MB here) that callers could stream.

Where an idea is not an unconditional win, the doc says so — the per-region BitSet
alternative (also commented out, at RegionImage:880-885) is numDistinctRegions × N bits, which
is 23 MB for a clean 3-region segmentation and catastrophic for a noisy one. That is very likely
why it was commented out, and it should be chosen at runtime from the measured region count if
it is used at all.

One piece of dead weight found, deliberately not fixed here

FloodFill2DLine (RegionImage:213) has an inverted guard —
if (sp + 4 < MAXDEPTH_TIMES_4) throw new RuntimeException("stack overflow") — which would throw
whenever there is room. It never fires because the code is unreachable: its only caller
calculateRegions3Dfaster is invoked solely from a commented-out line (RegionImage:554), inside
calculateRegions, itself commented out at RegionImage:407. Worth deleting so nobody loses time
on it, but it is not a live defect and does not belong in this PR.

Verification

Every line-number citation was checked against this branch (they are master +#1997; note
GeometrySpec shifts ~+26 lines once #2022 lands, which the doc records). mvn test-compile -pl vcell-core is clean. The profiler is a main in test sources with no @Test, so CI does not run
it.

Reproducing the prod-scale figure needs -Xmx6g and, once #2022 lands, -Dvcell.geometry.imageSizeLimit
raised — the ceiling refuses the image, which is exactly what it is for. That refusal firing during
this work was the first live confirmation that #2022 behaves as designed.

Refs #2021, #2022

🤖 Generated with Claude Code

https://claude.ai/code/session_018kr8SbzXtwW3gMVUgMfDDt

[jcschaff]

Converted to draft. Superseded for now by the planning document in #2025, which folds in this branch's measurements and corrects one claim from its notes file (XML read does not rebuild surfaces — getGeometrySurfaceDescription mutates the geometry it is passed and only the return value goes into the variable named dummy).

Worth saying: the profiler in here is the piece most worth keeping regardless of which direction is chosen. Every number in #2025 came from it, and all four corrections in its section 4 were only findable because it existed.