Skip to content

Decimal Add/Sub kernels#8724

Open
mhk197 wants to merge 5 commits into
developfrom
mk/decimal-add-sub
Open

Decimal Add/Sub kernels#8724
mhk197 wants to merge 5 commits into
developfrom
mk/decimal-add-sub

Conversation

@mhk197

@mhk197 mhk197 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Adds shared infrastructure for decimal arithmetic and native Add/Sub kernels.

Operand alignment

The execution kernel requires both operands to have the same precision and scale (nullability may differ). Addition and subtraction can then operate directly and exactly on the unscaled integers because both values use the same power-of-ten factor.

Vortex's optional coercion pass can align operands with different decimal dtypes before execution. Query-engine integrations may instead insert their own planner casts. If neither layer aligns the operands, the kernel rejects them rather than performing an implicit per-kernel rescale.

Result precision

Adding or subtracting two p-digit integers can require one additional carry digit. This PR follows the Arrow/Hive result-type rule for already aligned operands, capped at Vortex's maximum decimal precision:

decimal(p, s) +/- decimal(p, s) -> decimal(min(p + 1, 76), s)

The scale remains unchanged because the operation is performed on integers at the same scale. For example:

decimal(2, 0): 60 + 60 = 120 -> decimal(3, 0)
decimal(5, 2): 123.45 + 0.55 = 124.00 -> decimal(6, 2)

Widening the logical precision can also widen the native storage used for the result. The kernel executes at the smallest native width capable of representing the result dtype, including the important decimal(18, s) -> decimal(19, s) (i64 -> i128) and decimal(38, s) -> decimal(39, s) (i128 -> i256) transitions. Inputs may already use a wider physical representation; they are converted into the selected working width before arithmetic.

Why precision 76 is the limit

Vortex's widest decimal storage is a signed 256-bit integer. Its positive limit is:

2^255 - 1 ~= 5.79 * 10^76

That is enough to represent every 76-digit decimal integer, whose largest value is 10^76 - 1, but not every 77-digit integer, whose largest value would be 10^77 - 1. Some 77-digit values do fit in i256; that is not sufficient for a decimal(77, s) type, because a declared precision must support the type's entire range. Therefore the largest Vortex decimal dtype is decimal(76, s).

At precision 76 the result dtype cannot widen to precision 77, so it stays at precision 76 and the kernel explicitly checks the mathematical result against the precision-76 bound. Let M = 10^76 - 1:

M + 0 -> succeeds
M + 1 -> 10^76, which requires 77 digits -> decimal overflow

This is a logical decimal-precision check, not merely an i256 overflow check. In fact, even M + M still fits physically in i256, but it is outside the declared precision-76 range and must be rejected. Computing in i256 first and then checking the decimal bound cleanly distinguishes the physical representation from the logical dtype contract.

Nulls, constants, and errors

  • Output nullability is the union of the input nullabilities, while a lane is valid only when both input lanes are valid.
  • Validity is combined and materialized once. Arithmetic runs only on valid lanes, so arbitrary values underneath null lanes cannot trigger an overflow error.
  • Any native-width overflow or result-precision overflow on a valid lane returns an error for the operation.
  • Constant and all-null operands are handled directly instead of first materializing full decimal buffers. Constant folding is preserved, and empty Add/Sub inputs produce a correctly widened empty dtype without evaluating a constant value.
  • Decimal Mul/Div remain unsupported because they require separate result-type and rescaling rules.

Implementation

  • Splits numeric execution into numeric/{mod,primitive,decimal,tests}.rs, mirroring the compare implementation.
  • Canonicalizes encoded decimal operands through execute::<DecimalArray>, while extracting constant and null operands directly.
  • Uses a checked lane loop at the result's working width, followed by an explicit precision-bound check.
  • Shares widened_buffer from arrays/decimal between comparison and numeric execution.

Coverage

The conformance suite exercises Decimal, Chunked, and DecimalByteParts arrays, including:

  • Add/Sub with array and constant operands.
  • Carry-digit widening and native-width boundary transitions.
  • Mixed physical storage widths.
  • Nullable overflow cases, verifying that invalid lanes are skipped.
  • Empty and all-null inputs.
  • Precision-76 success and overflow boundaries.

binary_ops also includes non-null i64 and nullable i128 decimal Add benchmarks.

mhk197 added 3 commits July 10, 2026 17:54
Mechanical move mirroring the compare split: dtype-generic lane machinery
stays in numeric/mod.rs (bounds relaxed to T: Default so wider decimal types
can reuse it), primitive-specific code moves to numeric/primitive.rs, tests
to numeric/tests.rs.

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Add and Sub over decimal arrays sharing a decimal dtype, applied to the
unscaled stored integers (exact at a shared scale) in a working width wide
enough that in-precision inputs cannot spuriously overflow. Precision
violations error only on valid lanes, matching primitive semantics. Mul and
Div need rescaling and are gated off until follow-up PRs. widened_buffer
moves to arrays/decimal so compare and numeric share it.

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
The conformance harness now accepts decimal arrays, checking Add/Sub results
element-wise against DecimalScalar::checked_binary_numeric for representative
constants. Wire it into the decimal, chunked, and decimal-byte-parts compute
tests, and add decimal Add cases to the binary_ops benchmark.

Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@codspeed-hq

codspeed-hq Bot commented Jul 11, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 1660 untouched benchmarks
🆕 2 new benchmarks
⏩ 47 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation add_decimal_i128_nullable N/A 3.5 ms N/A
🆕 Simulation add_decimal_i64_nonnull N/A 1.9 ms N/A

Comparing mk/decimal-add-sub (3a56ba9) with develop (6b27c45)

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@mhk197 mhk197 added changelog/feature A new feature changelog/performance A performance improvement labels Jul 14, 2026
mhk197 added 2 commits July 14, 2026 15:53
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@mhk197 mhk197 marked this pull request as ready for review July 15, 2026 16:53
@mhk197 mhk197 changed the title Add native decimal Add/Sub kernels Decimal Add/Sub kernels Jul 15, 2026
@mhk197 mhk197 removed the changelog/performance A performance improvement label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant