Skip to content

Optimize WideDecimalBinaryExpr: skip the null-masking pass in non-ANSI mode when nothing overflows #4943

Description

@andygrove

Part of #4936.

Problem

WideDecimalBinaryExpr implements wide (128-bit intermediate) decimal + - *, which runs on the hot path of most TPC-DS queries. It computes the result in a single fused try_binary pass (native/spark-expr/src/math_funcs/wide_decimal_binary_expr.rs), writing i128::MAX as an overflow sentinel in non-ANSI mode. It then unconditionally runs null_if_overflow_precision to convert sentinels into nulls:

let result = if eval_mode != EvalMode::Ansi {
    result.null_if_overflow_precision(p_out)
} else {
    result
};

null_if_overflow_precision is a second full pass that allocates a new array, and it runs even when no value overflowed, which is the common case.

Proposed change

Run the masking pass only when a sentinel is actually present, mirroring what was done for DecimalRescaleCheckOverflow in #4938:

let result = if eval_mode != EvalMode::Ansi && result.values().contains(&i128::MAX) {
    result.null_if_overflow_precision(p_out)
} else {
    result
};

contains short-circuits at the first sentinel, so the overflow path is unaffected while the common no-overflow path skips the extra allocation. ANSI mode never produces a sentinel (it errors during the try_binary pass) and is unchanged.

Output stays bit-identical: when no sentinel is present, null_if_overflow_precision would have nulled nothing, so skipping it yields the same array.

Notes

This is the non-ANSI counterpart to the CheckOverflow ANSI fast path in #4937. A review of the other decimal expressions found no remaining ANSI-detection asymmetry: DecimalRescaleCheckOverflow (#4938), WideDecimalBinaryExpr, and the decimal casts already do their ANSI overflow check inside a single cheap pass.

A criterion benchmark (no-overflow, no-overflow-with-nulls, sparse-overflow, dense-overflow, and an ANSI shape) should accompany the change to confirm the win and that overflow shapes do not regress.

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions