Support pulse-height tallies in shared secondary mode - #4089
Open
GuySten wants to merge 5 commits into
Open
Conversation
GuySten
marked this pull request as ready for review
August 31, 2026 18:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support pulse-height tallies with the shared secondary bank
Removes the restriction that forced
shared_secondary_bankoff whenever apulse-height tally was present, and implements the deferred per-history scoring
described in the TODO that guard carried.
Problem
A pulse-height tally scores once per source history, in the bin containing the
total energy that history's entire particle tree deposited in a cell. In the
default transport modes the whole tree is carried by a single
Particle, sopht_storage_already holds the per-history total whenevent_death()runs.Under the shared secondary bank each generation is transported as a fresh set of
Particleobjects, load balanced across MPI ranks between generations. Threethings broke:
pht_storage_is a per-Particlemember reset ininitialize_particle_track,so every descendant's deposition was dropped from its root's total.
event_death()scored unconditionally, so each fragment of one physical pulsewas scored as a separate pulse, shredding the spectrum toward low energy.
implemented on this path.
event_revive_from_secondary()skipped it on thegrounds that
create_secondary()had already done it;create_secondary()contained no pulse-height code. The compensating pre-add in
sample_secondary_photons()still ran unconditionally.Approach
Every particle carries a root index: the dense global index of its primary in
the source bank, in
[0, n_particles). It is propagated throughSourceSite, soit survives bank sorting and MPI migration.
Each
Particlestill accumulates its own fragment as before. At death thefragment is staged in a per-thread buffer keyed by root index rather than scored.
Once the generation loop drains, contributions are routed by
MPI_Alltoallvtothe rank owning each root, summed, and scored once per history.
Root index rather than
parent_id/progeny_idbecause those identify theimmediate parent and would require walking a chain through generations that have
already been discarded. It is also dense, so it indexes an array directly and its
owner rank is a binary search in the phase-1 partition.
simulation::phase1_work_indexsnapshots that partition, becausecalculate_work()overwriteswork_indexon every secondary generation.Changes
particle_data.h—SourceSite::root_index,ParticleData::root_index_and accessorsparticle.cpp— propagate the root index; implement the parent-side subtraction increate_secondary(); routeevent_death()to staging in shared modetallies/pulse_height.{h,cpp}(new) — staging buffers, owner exchange, deferred scoringsimulation.cpp/simulation.h—phase1_work_index; init and finalize hooks in bothshared drivers
tally_scoring.cpp—score_pulse_height_tally()takes the energy vector as an argumentinstead of reading
p.pht_storage()initialize.cpp— removecheck_pulse_height_compatibility()Testing
New tests in
tests/unit_tests/test_pulse_height.py:normalized per source particle, so the tally summed over all bins must be exactly
1.0. Per-track scoring pushes this above one; a dropped history pushes it below.
compute_particle_id()andcompute_transport_seed()both branch on the sharedbank, so the two modes sample different random number streams. Mean deposited
energy and per-bin shape are compared against combined standard errors.
Notes for reviewers
tests/regression_tests/pulse_heightalready parametrizes over shared/local, but its stored
shared_*results arebyte-identical to the local ones because the guard was silently disabling the
feature. Run with
--updateand check the regenerated spectra against the localones before committing them.
only exercised with more than one process.
on thread scheduling and on which rank a descendant landed on, so results differ
at round-off between thread counts. Given that
sort_bank()exists specificallyfor reproducibility, this is a regression in kind, though only in the last bits.
A deterministic variant (stage per generation, sort by root index, fold in order)
costs a sort per generation and can be added behind a flag if wanted.
track; in a typical detector problem most histories never reach a pulse-height
cell. The per-rank totals array is
8 * n_owned_roots * n_pulse_height_cellsbytes.
Checklist
I have followed the style guidelines for Python source files (if applicable)I have made corresponding changes to the documentation (if applicable)