Skip to content

BUG: Guard division-by-zero NaN and UB-cast sites from issue 6575#6589

Open
hjmjohnson wants to merge 10 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:bug-nan-guards-6575
Open

BUG: Guard division-by-zero NaN and UB-cast sites from issue 6575#6589
hjmjohnson wants to merge 10 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:bug-nan-guards-6575

Conversation

@hjmjohnson

Copy link
Copy Markdown
Member

Guards the easy division-by-zero → NaN → undefined-behavior sites catalogued in #6575: ten fixes (items B2, B3, B4, B9, B11, B12, B20, B23, B27, B29), one commit per item, with seven new GoogleTests. Five of the tests demonstrably fail on the pre-fix code; the full test suites of all nine touched modules pass (235/235 locally).

Per-item summary
Item Site Guard
B2 N4BiasFieldCorrectionImageFilter::SharpenImage Independent min/max scans; degenerate bin range → identity mapping (was NaN bin index → UB int cast → unchecked indexing)
B3 SLICImageFilter Empty cluster keeps its previous centroid (was 0/0 NaN centroid → UB rounding)
B4 WienerDeconvolutionImageFilter functor Pf == Pn → zero output (was complex division by zero; the magnitude guard passes on inf)
B9 AdaptiveEqualizationHistogram Zero intensity range → identity (was NaN everywhere on constant input)
B11 STAPLEImageFilter Initialize p/q; keep previous estimate on a zero denominator (was NaN on all-background inputs)
B12 LabelOverlapMeasuresImageFilter FalsePositiveError guard now tests the actual FP+TN denominator (was NaN for a whole-image label)
B20 BoxSigmaImageFilter (itkBoxUtilities.h, both body and edge paths) Single-sample window → 0 (was sqrt(0/0) NaN at radius 0)
B23 StructureTensorImageFilter (AnisotropicDiffusionLBR) Zero maximum trace → no rescale (was ×inf → NaN tensors → UB negative step count downstream)
B27 PatchBasedDenoisingImageFilter Clamp the update to the representable pixel range before the cast (negative float → unsigned is UB)
B29 GaussianRandomSpatialNeighborSubsampler Bound-check the variate in floating point before the integer conversion (negative float → unsigned is UB)
Test coverage and verification notes

New GoogleTests: N4 constant-image (BiasCorrection — new GTest driver entry), Wiener functor zero-SNR (Deconvolution), AdaptiveHistogramEqualization constant-image + LabelOverlapMeasures whole-image label (ImageStatistics), BoxSigma radius-0 (Smoothing), STAPLE all-background (ImageCompare — new GTest driver block), StructureTensor constant-image (AnisotropicDiffusionLBR).

Fail-before/pass-after was verified locally by rebuilding each test against the pre-fix sources: the AHE, LabelOverlap, BoxSigma, STAPLE, and StructureTensor tests fail before the fix. The N4 and Wiener tests pass pre-fix on arm64/macOS because the underlying UB (NaN→int conversion, libc complex division) happens to be benign there — they pin the now-defined behavior and would catch regressions under UBSan or on other platforms.

Untested-by-design: B3 (an empty SLIC cluster is not deterministically constructible from the public API), B27 and B29 (UB-only guards with no observable defined-behavior change on this platform).

Findings and analysis by the #6575 reporter; see that ledger for the full derivations.

A constant included region made histogramSlope zero, so the histogram
bin index became 0.0/0.0 = NaN whose float-to-int conversion is
undefined behavior feeding unchecked indexing. Independently, the
else-if min/max scan never assigned binMinimum when the traversed
values were strictly increasing.

Make the scans independent and degrade sharpening to an identity
mapping when the intensity range is degenerate.

Issue InsightSoftwareConsortium#6575, item B2.
A cluster ending an iteration with zero members divided its
accumulator by zero, leaving NaN spatial components whose later
float-to-integer rounding is undefined behavior feeding image reads.

Issue InsightSoftwareConsortium#6575, item B3.
Pn / (Pf - Pn) divided by zero when the input power at a frequency
equaled the noise power spectral density constant; the magnitude
guard passes on the resulting infinity and NaN reached the output.
Return zero for such frequencies: zero estimated signal power means
nothing to recover.

Issue InsightSoftwareConsortium#6575, item B4.
…tity

A constant image gave a zero intensity range, so the normalization
divided by zero and every output pixel became NaN.

Issue InsightSoftwareConsortium#6575, item B9.
All-background inputs zero the weight image, making the sensitivity
denominator 0.0/0.0 = NaN with no guard. Initialize the estimates and
keep the previous value when a denominator vanishes.

Issue InsightSoftwareConsortium#6575, item B11.
The degenerate-value guard tested the source voxel count while the
division is by FP + TN, so a label covering the whole image passed the
guard and produced 0.0/0.0 = NaN.

Issue InsightSoftwareConsortium#6575, item B12.
A radius-0 window has one sample, so the unbiased variance divided by
zero and every output pixel became sqrt(0.0/0.0) = NaN.

Issue InsightSoftwareConsortium#6575, item B20.
A constant input has zero tensors, so the unit-maximum-trace rescale
divided by zero and multiplied the tensors into NaN, later driving an
undefined float-to-int conversion in the diffusion step count.

Issue InsightSoftwareConsortium#6575, items B23 and part of the B23 chain.
The GAUSSIAN fidelity branch and the pure-smoothing path can drive the
update negative; casting that to an unsigned pixel type is undefined
behavior.

Issue InsightSoftwareConsortium#6575, item B27.
A normal variate is unbounded below and the float-to-unsigned
conversion of a negative value is undefined behavior; the rejection
loop relied on the wrapped result.

Issue InsightSoftwareConsortium#6575, item B29.
@github-actions github-actions Bot added type:Bug Inconsistencies or issues which will cause an incorrect result under some or all circumstances type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct area:Filtering Issues affecting the Filtering module area:Segmentation Issues affecting the Segmentation module area:Numerics Issues affecting the Numerics module labels Jul 11, 2026
@hjmjohnson hjmjohnson marked this pull request as ready for review July 11, 2026 11:37
@greptile-apps

greptile-apps Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR hardens several ITK numeric edge cases. The main changes are:

  • Added guards for zero-range and single-sample image operations.
  • Preserved finite outputs in STAPLE, SLIC, Wiener, and structure-tensor degenerate cases.
  • Clamped or pre-checked values before risky numeric casts.
  • Added focused GoogleTests for the corrected behavior.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code. The new guards match the degenerate-case behavior described by the affected filters, and the new header dependencies fit the repository's C++ standard baseline.

No files need attention.

T-Rex T-Rex Logs

What T-Rex did

  • The prerequisite probe was run to check tool availability, and it confirmed missing tooling with exit code 1 and per-tool missing statuses.
  • An initial shell-wrapping attempt was carried out, revealing all tools missing before the wrapper failed on /bin/sh substitution syntax.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
Modules/Filtering/BiasCorrection/include/itkN4BiasFieldCorrectionImageFilter.hxx Adds an identity path for degenerate histogram ranges in N4 sharpening.
Modules/Filtering/Denoising/include/itkPatchBasedDenoisingImageFilter.hxx Clamps update components before converting them back to the output pixel type.
Modules/Filtering/ImageCompare/include/itkSTAPLEImageFilter.hxx Initializes p/q estimates and keeps the prior estimate when a denominator is zero.
Modules/Filtering/ImageStatistics/include/itkLabelOverlapMeasuresImageFilter.hxx Checks the actual false-positive denominator before computing the metric.
Modules/Filtering/Smoothing/include/itkBoxUtilities.h Returns zero sigma for single-sample body and edge windows.
Modules/Numerics/Statistics/include/itkGaussianRandomSpatialNeighborSubsampler.hxx Rejects out-of-bound Gaussian variates before converting them to integer indices.
Modules/Segmentation/SuperPixel/include/itkSLICImageFilter.hxx Keeps the previous centroid when a SLIC cluster has no assigned pixels.

Reviews (1): Last reviewed commit: "BUG: Reject out-of-bound Gaussian variat..." | Re-trigger Greptile

@hjmjohnson hjmjohnson requested a review from dzenanz July 11, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Filtering Issues affecting the Filtering module area:Numerics Issues affecting the Numerics module area:Segmentation Issues affecting the Segmentation module type:Bug Inconsistencies or issues which will cause an incorrect result under some or all circumstances type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant