Skip to content

Fix library UDF time units h and d overflowing at nanosecond precision - #19289

Open
PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix-udf-parsetime-ns
Open

PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix-udf-parsetime-ns

Conversation

@PDGGK

@PDGGK PDGGK commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Description

Util.parseTime in library-udf returns wrong values for the units h and d when timestamp_precision is ns. Both constants are multiplied as int and become long only at the last factor, so the product overflows first (Util.java:303, :306):

unit returned correct
1h -694,967,296,000 3,600,000,000,000
1d 500,654,080,000 86,400,000,000,000

So INTEGRAL rejects 'unit'='1h' as an unknown unit and returns 172.57 times the correct value for 'unit'='1d', and RESAMPLE rejects 'every'='1h'. The other library functions that read a duration through parseTime get the same values. I came across this while measuring INTEGRAL for the dev@ thread on its requirements.

Change

The first factor is now long (60L * ..., 24L * ...), so the whole product is computed in long. The other units, and all units at ms and us precision, already fit in int before the L factor, so their values do not change.

#17871 also changes parseTime and UDAFIntegral, but not these two constants. The negative-timestamp case of INTEGRAL that I mentioned on dev@ is handled there (hasLastValue), so it is not part of this PR.

Verification

  • New UtilParseTimeTest: 1h, 1d and 2d at ns, and every unit from 1ms to 1d compared across ms, us and ns. Both tests fail on master c6e194d with expected:<3600000000000> but was:<-694967296000> and pass with the change. All 21 tests in library-udf pass, and so does spotless:check.
  • On apache/iotdb:2.0.11-standalone with timestamp_precision=ns, with the library-udf jar built from v2.0.11 (its parseTime is the same as on master) without and with the change. A jar built from master bundles a newer TsFile than the 2.0.11 server, so I used the release sources. Data: s1 = 1.0 at time 0 and one day later.
query v2.0.11 with the change
integral(s1, 'unit'='1s') 86400.0 86400.0
integral(s1, 'unit'='1m') 1440.0 1440.0
integral(s1, 'unit'='1h') 701: ... Unknown time unit input 24.0
integral(s1, 'unit'='1d') 172.57424527530065 1.0
integral(s1, 'unit'='2d') 86.28712263765033 0.5
  • resample(s1, 'every'='1h') on two points at 0 h and 2 h: v2.0.11 fails with 701: ... gap should be a time period whose unit is ms, s, m, h, d.; with the change it returns the three rows at 0 h, 1 h and 2 h.

This PR has:

  • been self-reviewed.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.

At nanosecond precision, Util.parseTime multiplied the hour and day
constants as int and widened to long only at the last factor, so the
product overflowed first. "1h" became negative and was rejected as an
unknown unit, and "1d" became 500,654,080,000 ns, which made INTEGRAL
with unit '1d' return 172.57 times the correct value. Start both
products with a long factor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant