Skip to content

Avoid a per-run zip_eq call in run-end bool decoding #10962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
connortsui20 merged 1 commit into from
Oct 29, 2026
Merged

Avoid a per-run zip_eq call in run-end bool decoding #10962

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions vortex-array/src/arrays/runend/decompress_bool.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
//! Uses an adaptive strategy that pre-fills the buffer with the majority value
//! (0s or 1s) and only fills the minority runs, minimizing work for skewed distributions.

use itertools::Itertools;
use vortex_buffer::BitBuffer;
use vortex_buffer::BitBufferMut;
use vortex_error::VortexResult;
Expand All @@ -31,13 +30,27 @@ use crate::validity::Validity;
const PREFILL_RUN_THRESHOLD: usize = 32;

/// Decodes run-end encoded boolean values into a flat `BoolArray`.
///
/// # Panics
///
/// Panics if `ends` and `values` have different lengths.
pub fn runend_decode_bools(
ends: PrimitiveArray,
values: BoolArray,
offset: usize,
length: usize,
ctx: &mut ExecutionCtx,
) -> VortexResult<ArrayRef> {
// NB: The decode loops pair run ends with values through `zip`, which stops at the shorter
// input, so this check must stay. Do not switch the loops to `itertools::zip_eq`: `ZipEq::next`
// has no `#[inline]`, so rustc can place it in another codegen unit, and the loop then calls it
// out of line once per run.
assert_eq!(
ends.len(),
values.len(),
"run ends and values must have the same length"
);
Comment on lines +44 to +52

@connortsui20 connortsui20 Oct 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should upstream this to itertools?


let validity = values
.as_ref()
.validity()?
Expand Down Expand Up @@ -171,7 +184,7 @@ fn decode_bool_non_nullable(
let mut decoded = BitBufferMut::full(prefill, length);
let mut current_pos = 0usize;

for (end, value) in run_ends.zip_eq(values.iter()) {
for (end, value) in run_ends.zip(values.iter()) {
if end > current_pos && value != prefill {
// SAFETY: current_pos < end <= length == decoded.len()
unsafe { decoded.fill_range_unchecked(current_pos, end, value) };
Expand Down Expand Up @@ -203,7 +216,7 @@ fn decode_bool_nullable(
let mut decoded_validity = BitBufferMut::full(prefill_valid, length);
let mut current_pos = 0usize;

for (end, (value, is_valid)) in run_ends.zip_eq(values.iter().zip(validity_mask.iter())) {
for (end, (value, is_valid)) in run_ends.zip(values.iter().zip(validity_mask.iter())) {
if end > current_pos {
// SAFETY: current_pos < end <= length == decoded.len() == decoded_validity.len()
if is_valid != prefill_valid {
Expand Down
Loading