Skip to content

[codegen] Support normalized key for timestamp with local time zone#8821

Open
wombatu-kun wants to merge 1 commit into
apache:masterfrom
wombatu-kun:todo/normalized-key-timestamp-ltz
Open

[codegen] Support normalized key for timestamp with local time zone#8821
wombatu-kun wants to merge 1 commit into
apache:masterfrom
wombatu-kun:todo/normalized-key-timestamp-ltz

Conversation

@wombatu-kun

Copy link
Copy Markdown
Contributor

Purpose

Extend the normalized-key sort fast path to TIMESTAMP WITH LOCAL TIME ZONE, a direct follow-up to #8759 which added the non-compact TIMESTAMP support. LocalZonedTimestampType is physically stored by the same Timestamp runtime class (millisecond + nanoOfMillisecond) and sorts by the same Timestamp.compareTo, but it was excluded from supportNormalizedKey, so sorting by such a column fell back to the per-field RecordComparator instead of the binary normalized key.

The runtime writer SortUtil.putTimestampNormalizedKey already supports any precision, so the change only whitelists the extra type root in SortCodeGenerator (supportNormalizedKey, getNormalizeKeyLen, prefixGetFromBinaryRow and getter), reusing DataTypeChecks.getPrecision the same way GenerateUtils already does for the record comparator path.

Micro-benchmark: sort 1,000,000 rows by a single timestamp key through BinaryInMemorySortBuffer (best of 10 iterations after 3 warmup, in-memory). Sorting a TIMESTAMP column is the control and stays flat, confirming the delta is the path switch and not measurement noise. key is the generated normalized-key length; 0B means the column was excluded and the whole sort ran through the comparator.

case before after speedup
TIMESTAMP WITH LOCAL TIME ZONE(9) non-compact 1418 ns/row, 705 K/s (key=0B, fallback) 384 ns/row, 2606 K/s (key=13B) 3.70x
TIMESTAMP WITH LOCAL TIME ZONE(3) compact 1307 ns/row, 765 K/s (key=0B, fallback) 367 ns/row, 2722 K/s (key=9B) 3.56x
TIMESTAMP(9) control 367 ns/row 398 ns/row ~1x
TIMESTAMP(3) control 388 ns/row 376 ns/row ~1x

Tests

NormalizedKeyComputerTest is parameterized over both timestamp roots and covers: key metadata (compact 9 bytes, non-compact 13 bytes, fully-determining), sub-millisecond ordering, compareKey agreeing with Timestamp.compareTo over all value pairs, and an end-to-end sort through BinaryInMemorySortBuffer. It also asserts that a TIMESTAMP WITH LOCAL TIME ZONE column and a TIMESTAMP column produce byte-for-byte identical normalized keys for the same value.

Extend the normalized-key sort fast path to TIMESTAMP WITH LOCAL TIME ZONE, mirroring the non-compact TIMESTAMP support added in apache#8759. LocalZonedTimestampType shares the same Timestamp runtime representation (millisecond + nanoOfMillisecond) and the same compareTo ordering as TimestampType, but was excluded from supportNormalizedKey, so sorting by such a column fell back to the per-field RecordComparator instead of the binary normalized key.

The runtime writer SortUtil.putTimestampNormalizedKey already handles any precision, so this change only whitelists the extra type root in SortCodeGenerator (supportNormalizedKey, getNormalizeKeyLen, prefixGetFromBinaryRow and getter), reusing DataTypeChecks.getPrecision the same way GenerateUtils already does for the record comparator path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant