Repository navigation
Conversation
DateBinFunctionColumnTransformer.convertToTimestamp added Instant.toEpochMilli() and LocalDateTime.getNano() together. The former already contains the whole milliseconds of the second and the latter is the full nanosecond-of-second, so the milliseconds were counted twice and month/year bins started that many milliseconds late. Convert from getEpochSecond() and getNano() instead so each part is counted once. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
a1f5cd5 to
2bca3ad
Compare
JackieTien97
left a comment
There was a problem hiding this comment.
Thanks for the fix. The root cause is right: toEpochMilli() already includes the milliseconds of the second, so adding getNano() on top counted them twice. Converting from getEpochSecond() + getNano() fixes it for ms, us and ns.
I compared the old and new conversion against an exact BigInteger reference, including pre-1970 instants (where the old code was off by even more), and ran DateBinFunctionTest on 2bca3ad: 12/12 pass. I also agree with your note on #19264: Instant.ofEpochSecond normalizes a negative nano adjustment, so convertToLocalDateTime doesn't throw for negative timestamps.
Two inline comments: an overflow edge case in the rewritten lines, and a suggested pre-1970 test.
One more, not tied to a diff line: the commit message ends with Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>, but the root AGENTS.md says "Do NOT add Co-Authored-By trailer to commit messages." Could you drop it from the commit? It can also be removed when squash-merging.
TimeUnit.SECONDS.toMillis, toMicros and toNanos already saturate, but the sub-second part was added with an unchecked add, so a month bin past the long range wrapped to a large negative value. Use saturatedAdd so the bin end stays at the long bound. Also cover pre-1970 month bins with a sub-second origin, where getEpochSecond() rounds down and getNano() stays non-negative.
JackieTien97
left a comment
There was a problem hiding this comment.
Thanks for the quick update. Both inline comments are addressed in 755a66d: the conversion now saturates at the long bounds instead of wrapping, and the pre-1970 cases are covered.
I re-ran it locally on 755a66d: spotless passes, DateBinFunctionTest passes 13/13, and the probe cases past the long range (ms and ns) now return Long.MAX_VALUE for both the bin end and nextDateBin.
One leftover for the merge: the first commit (2bca3ad) still carries the Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> trailer, which AGENTS.md asks not to add, so please drop it from the squash commit message.
Description
Fixes #19264.
DateBinFunctionColumnTransformer.convertToTimestampcomputed the result astoEpochMilli() + getNano().toEpochMilli()already contains the wholemilliseconds of the second, and
getNano()is the full nanosecond-of-second(0–999_999_999), not the remainder below one millisecond. The intra-second
milliseconds were therefore counted twice, so month/year bins started that many
milliseconds late. For example
date_bin(1mo, time, 500)returned1000for asource of
500and2678401000for2678400500.The fix converts from
Instant.getEpochSecond()andInstant.getNano(), so eachpart of the instant is counted exactly once, for
ms,usandnsprecision.Hour and day bins do not use this helper and are unaffected. Month bins with a
whole-second origin are also unaffected because
getNano()is 0 for them.How to reproduce
Before:
1000and2678401000. After:500and2678400500.Tests
Added cases in
DateBinFunctionTestfor the issue's scenario, a 2024 origin,dateBinStartEnd,nextDateBin, andus/nsprecision with a sub-secondorigin.
🤖 Generated with Claude Code