[codegen] Support normalized key for timestamp with local time zone#8821
Open
wombatu-kun wants to merge 1 commit into
Open
[codegen] Support normalized key for timestamp with local time zone#8821wombatu-kun wants to merge 1 commit into
wombatu-kun wants to merge 1 commit into
Conversation
Extend the normalized-key sort fast path to TIMESTAMP WITH LOCAL TIME ZONE, mirroring the non-compact TIMESTAMP support added in apache#8759. LocalZonedTimestampType shares the same Timestamp runtime representation (millisecond + nanoOfMillisecond) and the same compareTo ordering as TimestampType, but was excluded from supportNormalizedKey, so sorting by such a column fell back to the per-field RecordComparator instead of the binary normalized key. The runtime writer SortUtil.putTimestampNormalizedKey already handles any precision, so this change only whitelists the extra type root in SortCodeGenerator (supportNormalizedKey, getNormalizeKeyLen, prefixGetFromBinaryRow and getter), reusing DataTypeChecks.getPrecision the same way GenerateUtils already does for the record comparator path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Extend the normalized-key sort fast path to
TIMESTAMP WITH LOCAL TIME ZONE, a direct follow-up to #8759 which added the non-compactTIMESTAMPsupport.LocalZonedTimestampTypeis physically stored by the sameTimestampruntime class (millisecond + nanoOfMillisecond) and sorts by the sameTimestamp.compareTo, but it was excluded fromsupportNormalizedKey, so sorting by such a column fell back to the per-fieldRecordComparatorinstead of the binary normalized key.The runtime writer
SortUtil.putTimestampNormalizedKeyalready supports any precision, so the change only whitelists the extra type root inSortCodeGenerator(supportNormalizedKey,getNormalizeKeyLen,prefixGetFromBinaryRowandgetter), reusingDataTypeChecks.getPrecisionthe same wayGenerateUtilsalready does for the record comparator path.Micro-benchmark: sort 1,000,000 rows by a single timestamp key through
BinaryInMemorySortBuffer(best of 10 iterations after 3 warmup, in-memory). Sorting aTIMESTAMPcolumn is the control and stays flat, confirming the delta is the path switch and not measurement noise.keyis the generated normalized-key length;0Bmeans the column was excluded and the whole sort ran through the comparator.TIMESTAMP WITH LOCAL TIME ZONE(9)non-compactTIMESTAMP WITH LOCAL TIME ZONE(3)compactTIMESTAMP(9)controlTIMESTAMP(3)controlTests
NormalizedKeyComputerTestis parameterized over both timestamp roots and covers: key metadata (compact 9 bytes, non-compact 13 bytes, fully-determining), sub-millisecond ordering,compareKeyagreeing withTimestamp.compareToover all value pairs, and an end-to-end sort throughBinaryInMemorySortBuffer. It also asserts that aTIMESTAMP WITH LOCAL TIME ZONEcolumn and aTIMESTAMPcolumn produce byte-for-byte identical normalized keys for the same value.