[robert3005]

Summary

Several aggregate kernels handle nulls by walking runs of valid values one at a time, or by zipping each value with its validity bit through an iterator. When nulls are scattered, so runs are only one or two values long, that is slow, and the cost depends heavily on how the nulls are spread.

This PR adds a masked reduction lane kernel. It folds every lane and reads null lanes as an identity of the fold, then applies it to every aggregate kernel where it measurably helps. The result is a branch-free reduction that costs the same whatever the validity density.

Changes

• vortex-compute: new IndexedFoldExt::fold_masked (lane_kernels/fold.rs).
  • Folds every lane of an IndexedSource into an accumulator, reading null lanes as a caller-chosen identity.
  • Tests each lane against a constant bit of its 32-lane half of the validity word, instead of shifting the word by the lane index. Baseline x86-64 has no per-lane variable shift, so the shift version doesn't vectorize; the constant-bit version does.
  • Skips 32-lane halves with no valid lane.
• Integer min/max (min_max/primitive.rs): null lanes read as a valid value from the array, which leaves both extrema unchanged.
• Float min/max (min_max/primitive.rs), on both the all-valid and nullable paths:
  • The fold runs over total-order keys: integers derived from the float bits that order exactly as total_cmp does, which is how total_cmp is implemented.
  • The reduction is therefore plain integer min/max, and the result is bit-for-bit what the previous total_cmp reduction returned, NaN placement included.
  • NaN values are skipped or kept as before.
• Decimal min/max (min_max/decimal.rs): null lanes read as a valid value.
• Float nan_count (nan_count/primitive.rs): null lanes read as zero, which isn't NaN.
• Sum (sum/primitive.rs):
  • Integers narrower than 64 bits read null lanes as zero. They are summed in the existing SUM_CHUNK blocks with one overflow check per block, as on the all-valid path.
  • Floats read null lanes as zero and add the values in the same order as before. The accumulator starts at +0.0, so it is never -0.0, the one value adding zero would change; the result is unchanged.
  • 64-bit integers keep the per-run path, since they check every element for overflow.

Left unchanged, because the fold measured no better:

• Decimal sums: their per-element checked add dominates the cost.
• Grouped (per-list) sums: on small groups with long valid runs, setting up the fold per group cost more than it saved. For example, grouped i32 sum with clustered nulls went from 6.3 µs to 7.3 µs.

Not applicable: is_sorted with nulls depends on where the nulls fall, so it isn't a fold.

Tests:

• The kernel: a mask that doesn't start on a word boundary, and all-null input.
• Total-order keys: ordering matches total_cmp for f16/f32/f64, and the round trip preserves the bits.
• Null rows holding infinities, NaN or the type's extreme values must not reach integer min/max, float min/max (in both NaN modes), integer sum or float sum, across a full 64-row word and the partial word after it.

Benches:

• cargo bench -p vortex-compute --bench fold_masked: the kernel against arrow-rs's nullable min/sum.
• New cargo bench -p vortex-array --bench aggregate_nullable: the aggregate path with nulls scattered at random. The existing benches only have clustered nulls, which is the run-walking path's best case.

aggregate_nullable medians, 15,000 rows, 50% valid, scattered:

	before	after
f64 min/max	82.2 µs	40.0 µs
f64 min/max, all valid	32.7 µs	28.1 µs
decimal (i64) min/max	65.4 µs	29.5 µs
f64 NaN count	20.4 µs	6.4 µs
f64 sum	68.9 µs	58.8 µs

The "before" column here is this PR's first commit, so it already includes the integer fold.

Integer kernels, measured on a separate branch through the aggregate's accumulate on 15,000 nullable i32s (that bench is not in this PR):

	50% valid	90%	95%	97%	99%
min/max before	~45 µs	~22 µs	~15 µs	~11 µs	~6.7 µs
min/max after	~8–9 µs	~8–9 µs	~8–9 µs	~8–9 µs	~8–9 µs
sum before	~49 µs	~20 µs	~12 µs	~8.4 µs	~4.8 µs
sum after	~5.2–6.0 µs	~5.2–6.0 µs	~5.2–6.0 µs	~5.2–6.0 µs	~5.2–6.0 µs

At 99% valid, integer min/max is now slightly slower (~8–9 µs against ~6.7 µs), because the run walk had very few runs there.

Kernel medians from fold_masked, 15,000 i32s, the same at every density:

	fold_masked	arrow-rs nullable
min	7.0 µs	7.9–8.2 µs
min and max in one pass	7.8 µs	—
sum (i32 → i64, overflow-checked)	5.6 µs	3.9 µs (wraps in i32)

Checks run:

• cargo clippy -p vortex-compute -p vortex-array --all-targets --all-features -- -D warnings
• cargo test -p vortex-compute (33 passed)
• cargo test -p vortex-array --lib (5,155 passed)

🤖 Generated with Claude Code

https://claude.ai/code/session_01X1HnweRGxeDWgQwXXM6f5s

[codspeed]

Merging this PR will regress 3 benchmarks

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 29 improved benchmarks
❌ 3 regressed benchmarks
✅ 2272 untouched benchmarks
🆕 58 new benchmarks
⏩ 270 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	min_max_decimal_i256[ScatteredNulls]	673.9 µs	1,000.9 µs	-32.68%
❌	Walltime	threshold_avx2[80]	2.6 µs	3 µs	-12.91%
❌	Walltime	starts_with_avx512[8, VarBinView]	68.5 µs	76.2 µs	-10.1%
⚡	CPU Simulation	sum_i32_nulls_scattered	691 µs	98.8 µs	×7
⚡	CPU Simulation	sum_v2_i32_nulls_scattered	734.4 µs	124.2 µs	×5.9
⚡	CPU Simulation	max_i32_nulls_scattered	768.1 µs	138.6 µs	×5.5
⚡	CPU Simulation	integer_stats_all_nullable	2,434 µs	516.6 µs	×4.7
⚡	CPU Simulation	integer_stats_nullable	1,791.5 µs	562.7 µs	×3.2
⚡	CPU Simulation	cast_i32_to_u32[8192]	391.5 µs	123.5 µs	×3.2
⚡	CPU Simulation	min_max[msp_i32_p18_nullable]	225.6 µs	114.6 µs	+96.87%
⚡	CPU Simulation	sum_v2_f64	206 µs	135.4 µs	+52.18%
⚡	CPU Simulation	sum_f64	205.5 µs	135.2 µs	+51.97%
⚡	CPU Simulation	sum_f64_nulls_clustered	212.1 µs	140 µs	+51.5%
⚡	CPU Simulation	sum_v2_f64_nulls_clustered	212.4 µs	140.5 µs	+51.13%
⚡	CPU Simulation	max_f64	282 µs	188.9 µs	+49.34%
⚡	CPU Simulation	sum[canonical]	381.9 µs	266 µs	+43.56%
⚡	CPU Simulation	min_max[gap_msp_i16_p10]	105.5 µs	75.8 µs	+39.17%
⚡	CPU Simulation	sum_i64	189.5 µs	136.4 µs	+38.95%
⚡	CPU Simulation	sum_v2_i64	191.5 µs	138.1 µs	+38.63%
⚡	CPU Simulation	min_max[msp_i32_p18]	128.6 µs	94 µs	+36.8%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing rk/fold-masked (2e8bff5) with develop (c84e489)}

Footnotes

1. 270 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩