Skip to content

TPC: fix pair swap when reading CalArray<PadFlags> written by the cus… - #16124

Merged
alcaliva merged 1 commit into
AliceO2Group:devfrom
matthias-kleiner:tpc-padflags-streamer-fix
Nov 16, 2026
Merged

alcaliva merged 1 commit into
AliceO2Group:devfrom
matthias-kleiner:tpc-padflags-streamer-fix

Conversation

@matthias-kleiner

Copy link
Copy Markdown
Contributor

…tom streamer

The custom member streamer for CalArray::mData writes the count N/2 followed by N single shorts, but reads N/2 ints and reinterprets each as a pair of flags (the layout written by ROOT < 6.33). Every object written by the streamer itself (ROOT >= 6.33) therefore comes back with each pair of adjacent pads swapped. This affects all IDC_PadStatusMap objects written since the streamer was introduced (e.g. 2026 data, ROOT 6.36.04): the dead channel map used in reconstruction marks the neighbour of an isolated bad pad instead of the pad itself. Objects written with ROOT < 6.33 (e.g. 2025, ROOT 6.32.06) are read correctly.

The class version did not change, so the reader now selects the layout by the ROOT version that wrote the buffer (TBuffer::GetVersionOwner(); the current ROOT if the buffer has no file): < 6.33 as before, >= 6.33 N shorts. The writer is unchanged, so existing objects stay readable and no re-upload is needed.

Add a ROOT I/O round-trip unit test for CalDet, which fails with the old reader (349440 of 524160 flags differ) and passes with the new one.

…tom streamer

The custom member streamer for CalArray<PadFlags>::mData writes the count N/2
followed by N single shorts, but reads N/2 ints and reinterprets each as a pair
of flags (the layout written by ROOT < 6.33). Every object written by the
streamer itself (ROOT >= 6.33) therefore comes back with each pair of adjacent
pads swapped. This affects all IDC_PadStatusMap objects written since the
streamer was introduced (e.g. 2026 data, ROOT 6.36.04): the dead channel map
used in reconstruction marks the neighbour of an isolated bad pad instead of
the pad itself. Objects written with ROOT < 6.33 (e.g. 2025, ROOT 6.32.06) are
read correctly.

The class version did not change, so the reader now selects the layout by the
ROOT version that wrote the buffer (TBuffer::GetVersionOwner(); the current ROOT
if the buffer has no file): < 6.33 as before, >= 6.33 N shorts. The writer is
unchanged, so existing objects stay readable and no re-upload is needed.

Add a ROOT I/O round-trip unit test for CalDet<PadFlags>, which fails with the
old reader (349440 of 524160 flags differ) and passes with the new one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for e4426e4 at 2026-11-14 12:43:

## sw/BUILD/O2-sim-challenge-test-latest/log
./sim-challenge.log[ERROR] SHUTTING DOWN DUE TO SIGNALED EXIT IN COMPONENT 208
./sim-challenge.log[12:43:12][ERROR] Some error/interrupt occurred on socket during receive
./sim.log[ERROR] SHUTTING DOWN DUE TO SIGNALED EXIT IN COMPONENT 208
./sim.log[12:43:12][ERROR] Some error/interrupt occurred on socket during receive


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ echo 'Found 0 errors and 0 warnings.'
Found 0 errors and 0 warnings.
++ [[ 0 -gt 0 ]]
++ [[ 0 -gt 0 ]]
++ [[ 0 -gt 0 ]]
++ mkdir -p /sw/INSTALLROOT/972e3dbce16c8cacf06a769e117521a56b4d28c6/slc9_x86-64/o2checkcode/1.0-local58/etc/modulefiles
++ alibuild-generate-module
+ HAS_BIN=
+ HAS_LIB=
+ HAS_CMAKE=
+ HAS_ROOT=
+ '[' 0 '!=' 0 ']'
+ cat
+ printf 'if ![ is-loaded '\''BASE/1.0'\'' ] {\n module load BASE/1.0\n}'
+ echo 'BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release'
BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release
+ FULL_BUILD_REQUIRES='googlebenchmark pytorch_cpuinfo ninja-fortran O2-customization defaults-release FairCMakeModules Alice-GRID-Utils MPFR alibuild-recipe-tools date json-c bz2 ninja double-conversion CMake UUID Python-modules-list safe_int'
+ echo 'FULL_BUILD_REQUIRES=googlebenchmark pytorch_cpuinfo ninja-fortran O2-customization defaults-release FairCMakeModules Alice-GRID-Utils MPFR alibuild-recipe-tools date json-c bz2 ninja double-conversion CMake UUID Python-modules-list safe_int'
FULL_BUILD_REQUIRES=googlebenchmark pytorch_cpuinfo ninja-fortran O2-customization defaults-release FairCMakeModules Alice-GRID-Utils MPFR alibuild-recipe-tools date json-c bz2 ninja double-conversion CMake UUID Python-modules-list safe_int
++ env
++ cut -f1 -d=
++ grep -v '^DEFAULT_'
++ grep -v PKGREVISION
++ grep -v ALIBUILD_RECIPE_TOOLS
++ grep REVISION
++ sed -e s/_REVISION//
+ for x in $(env | cut -f1 -d= | grep -v "^DEFAULT_" | grep -v PKGREVISION | grep -v ALIBUILD_RECIPE_TOOLS | grep REVISION | sed -e 's/_REVISION//')
++ eval 'echo $LIBFFI_REVISION'
+++ echo 14
+ REVISION_VALUE=14
++ eval 'echo $LIBFFI_VERSION'
+++ echo v3.2.1-alice1
+ VERSION_VALUE=v3.2.1-alice1
++ eval 'echo $LIBFFI_ROOT'
+++ echo /sw/slc9_x86-64/libffi/v3.2.1-alice1-14
+ ROOT_PATH_VALUE=/sw/slc9_x86-64/libffi/v3.2.1-alice1-14
+ '[' -z 14 ']'
+ echo 'googlebenchmark pytorch_cpuinfo ninja-fortran O2-customization defaults-release FairCMakeModules Alice-GRID-Utils MPFR alibuild-recipe-tools date json-c bz2 ninja double-conversion CMake UUID Python-modules-list safe_int'
+ tr '[:lower:]' '[:upper:]'
[0 more errors; see full log]

Full log here.

@sawenzel

Copy link
Copy Markdown
Collaborator

I will take a look at this. Please wait before merging.

@sawenzel
sawenzel requested review from ktf and sawenzel November 15, 2026 21:39
@sawenzel

Copy link
Copy Markdown
Collaborator

Thanks for tracking this down. I agree with the diagnosis: the write branch of the streamer, which I carried over in #14830 without a round-trip test, never produced the old ROOT layout, so objects written through it come back with adjacent pads swapped. The fix reads both layouts correctly for the CCDB objects we have.

Since this code runs in the async tags, the backport matters. I believe it is needed for all tags on ROOT 6.36 that read maps written since the streamer went live, i.e. 2026 data and MC anchored to it. Tags reading only 2025 or earlier maps are not affected, and on ROOT 6.32 the streamer is not compiled in.

One request before merging: please use GetVersionOwner() % 1000000. ROOT adds 1000000 to the stored version for files above 2 GB, so a large file written with 6.32 would be taken for the new layout and read swapped. With that, we can merge now.

The remaining points can follow later, and I propose to track them in a JIRA ticket:

  • Tests: only the new branch is covered at the moment. The old int layout should be asserted as well, which needs either the two read paths as separate functions or a small reference file written with 6.32. In addition, the n % 3 pattern leaves one pair in three identical; a value derived from the pad index would catch every swap.
  • Format: I would recommend making it describe itself, because the two layouts are identical in count and length and the reader can only tell them apart by the ROOT version of the file. This could be a tagged write, or going back to the true old layout together with a one-time re-upload of the affected maps.
  • A theoretical case: a map written with ROOT 6.36 while the streamer was not active (builds between 2025-10-16 and 2025-11-19, or TPC_PADFLAGS_STREAMER_OFF) has ROOT's native layout with count N, which neither branch reads. I do not know that such an object exists. A check that mData.size() matches the number of pads of the subset would at least make it fail loudly.
  • Cosmetics: the header comment still says that we write the old format, which contradicts the new paragraph below it.

@sawenzel

Copy link
Copy Markdown
Collaborator

@alcaliva

@alcaliva
alcaliva merged commit 7449ca5 into AliceO2Group:dev Nov 16, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

5 participants