perf: optimize spark_ceil (3x faster)#4926
Conversation
spark_ceil (3x faster)
mbutrovich
left a comment
There was a problem hiding this comment.
First pass, thanks @andygrove!
| _ => div_ceil(x, div), | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
ceil.rs:97-254 tests only small unscaled values (max 12999), which all take the new i64 fast path. The entire point of this PR is the branch at ceil.rs:85, and the fallback arm (_ => div_ceil(x, div) at ceil.rs:93) has zero direct coverage. A regression that broke only the wide path would pass CI.
Suggested change: add an array test with a Decimal128(38, s) input whose unscaled values exceed i64::MAX (positive and negative), asserting the ceil result. Example: unscaled 20_000_000_000_000_000_000 (> i64::MAX) at scale 6 targeting Decimal128(38, 0), plus a negative sibling and an exact multiple, so both the fast and fallback arms plus the negative-remainder branch are covered in one suite.
| precision: u8, | ||
| scale: i8, | ||
| f: &dyn Fn(i128) -> i128, | ||
| f: impl Fn(i128) -> i128, |
There was a problem hiding this comment.
utils.rs:60 now takes f: impl Fn(i128) -> i128, but the array call site at ceil.rs:51 still passes &f. This compiles because &F: Fn via the blanket impl, so F monomorphizes to &closure and dispatch is static, but it is now a stray reference that obscures the intent of the change and adds a pointer indirection the PR was trying to remove.
Suggested change: pass the closure by value at ceil.rs:51: make_decimal_array(array, precision, scale, f).
| a: &Option<i128>, | ||
| precision: u8, | ||
| scale: i8, | ||
| f: &dyn Fn(i128) -> i128, |
There was a problem hiding this comment.
utils.rs:49 still declares f: &dyn Fn(i128) -> i128 and ceil.rs:68 passes &f. The PR's stated rationale is "the per-element rounding closure is monomorphized rather than called through a &dyn Fn vtable," but that only happened for the array path. The scalar path is a single element so the perf cost is irrelevant, however the two helpers now have inconsistent signatures for no reason, which is a maintenance trap.
Suggested change: change make_decimal_scalar at utils.rs:49 to f: impl Fn(i128) -> i128 and pass f by value at ceil.rs:68, matching make_decimal_array. This keeps the two helpers symmetric.
Which issue does this PR close?
N/A
Rationale for this change
Optimize existing expression.
What changes are included in this PR?
Decimal ceil now uses 64-bit hardware division for values/divisors that fit in i64 instead of the 128-bit division intrinsic, and the per-element rounding closure is monomorphized rather than called through a
&dyn Fnvtable.How are these changes tested?
Existing tests.
Benchmark (criterion):
Full criterion output: