Skip to content

Move generic SIMD filtering to vortex-buffer - #10930

Open
robert3005 wants to merge 3 commits into
developfrom
rk/buffer-filter
Open

robert3005 wants to merge 3 commits into
developfrom
rk/buffer-filter

Conversation

@robert3005

@robert3005 robert3005 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

The filter kernels for fixed-width values (SIMD compress, byte compress, run copy, index gather, and the bitmap walk) do not depend on arrays. This change moves them from vortex-array to a new vortex_buffer::filter module, so that crates that use only buffers can filter values without vortex-array.

@robert3005 robert3005 added the changelog/break A breaking API change label Oct 28, 2026
@codspeed

codspeed Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 8.04%

3 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 3 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 10 improved benchmarks
❌ 7 regressed benchmarks
✅ 2253 untouched benchmarks
⏩ 370 skipped benchmarks1
🗄️ 30 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation filter_powerlaw_by_random[10000] 531.1 ns 1,179.2 ns -54.96%
❌ Walltime mul_u64_nonnull_neon 20.5 µs 39.7 µs -48.4%
❌ CPU Simulation random_i8[0.01] 98.8 µs 147.2 µs -32.89%
❌ CPU Simulation filter_list_primitive_wide[EdgeSpanning] 829.2 µs 1,185.2 µs -30.04%
❌ CPU Simulation filter_list_primitive_wide[Repeated] 1.9 ms 2.3 ms -16.17%
❌ CPU Simulation take_filter_primitive_small_uncached_random_mask_random_indices[4096, 10] 19.9 µs 23.4 µs -15.11%
❌ Walltime starts_with_avx2[8, VarBinView] 73 µs 85.5 µs -14.67%
⚡ CPU Simulation bench_compare_varbin[(50000, 2)] 162.8 µs 133.2 µs +22.22%
⚡ CPU Simulation bench_compare_varbin[(50000, 64)] 164.5 µs 134.9 µs +21.93%
⚡ CPU Simulation bench_compare_sliced_dict_varbinview[(7500, 10000)] 228.7 µs 200.7 µs +13.97%
⚡ CPU Simulation bench_compare_sliced_dict_varbinview[(10000, 10000)] 235 µs 206.8 µs +13.61%
⚡ CPU Simulation bench_compare_sliced_dict_varbinview[(1000, 10000)] 90.3 µs 79.9 µs +12.93%
⚡ Walltime starts_with_avx2[64, VarBin] 62.3 µs 55.8 µs +11.82%
⚡ Walltime add_constant_shapes_neon[(32768, PerRowNullableConstant)] 16.3 µs 14.7 µs +10.85%
⚡ CPU Simulation filter_list_primitive_short[Repeated] 152.3 µs 137.6 µs +10.66%
⚡ Walltime add_constant_shapes_neon[(32768, ConstantPerRow)] 15.7 µs 14.3 µs +10.22%
⚡ Walltime add_constant_shapes_neon[(32768, PerRowConstant)] 15.7 µs 14.3 µs +10.16%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/buffer-filter (d6bd74a) with develop (9bf403a)

Open in CodSpeed

Footnotes

  1. 370 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 30 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

robert3005 and others added 2 commits October 28, 2026 16:58
Signed-off-by: Robert Kruszewski <github@robertk.io>
Add vortex_buffer::filter::filter_records, which filters records whose byte
width is known only at run time. Power-of-two widths use filter_buffer, and
other widths copy each run of the selection. The fixed-width array filter
now uses it instead of its own bitmap loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005
robert3005 enabled auto-merge (squash) October 28, 2026 18:17
@robert3005
robert3005 disabled auto-merge October 28, 2026 18:19
In `filter_records`, LLVM kept the per-word closures of byte compress and
the bitmap walk out of line, because the word walk calls them from three
places. Sparse masks then made one call for each mask word. Mark the
closures `inline(always)`.

Signed-off-by: Robert Kruszewski <github@robertk.io>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant