fix: DATE values encoded as days-since-epoch instead of INT32 yyyyMMdd - #15
Merged
Conversation
IoTDB's DATE wire format is an INT32 encoded as year*10000 + month*100 + day (e.g. 2026-07-13 -> 20260713), matching the Java client's DateUtils.parseDateExpressionToInt and the C# client. The Node.js client was writing and reading days-since-epoch, so DATE values written via tablets were corrupted and unreadable by other clients (and vice versa). Adjudicated against a live IoTDB 2.0.6 server: a row inserted via SQL literal '2026-07-13' reads back as INT32 20260713 on the wire. Changes: - Add shared parseDateToInt / parseIntToDate utils in src/utils/DataTypes.ts (UTC calendar components, matching Java LocalDate semantics) and export them - Session.serializeColumn case 9: write yyyyMMdd (was days-since-epoch) - FastSerializer.serializeDateColumn: same fix on the fast path - Session.deserializeColumn case 9 (TSQueryDataSet): decode yyyyMMdd -> Date - ColumnDecoder Int32ArrayColumnDecoder: decode DATE (type 9) columns to Date - Session.parseTsBlock: DATE columns arrive typed as INT32 (1) in the TsBlock header; convert them to Date using the metadata dataTypeList - Unit tests: yyyyMMdd round-trip, exact wire bytes for 2026-07-13 (0x01352769), TsBlock decode; updated existing DATE serialization tests Verified: tablet-written Date and SQL-literal date now read back identical against a live IoTDB 2.0.6 (both 2026-07-13T00:00:00.000Z).
HTHou
reviewed
Jul 13, 2026
HTHou
left a comment
HTHou
left a comment
Contributor
There was a problem hiding this comment.
I found three correctness and public-contract issues in the DATE wire-format change. Details are inline.
Addresses three review comments from HTHou on PR #15: 1. parseTsBlock DATE conversion now respects columnIndex2TsBlockColumnIndexList. dataTypeList is ordered by LOGICAL response columns while physical TsBlock columns may be deduplicated or reordered, so converting physical column i based on dataTypes[i] could convert the wrong column. The mapping (logical -> physical, -1 = time column) is now passed into parseQueryResult from both call sites (executeQueryStatement initial batch and SessionDataSet.fetchNextBatch) and inverted into per-physical-column logical types. DATE conversion only applies when a physical column's logical type is unambiguously DATE; conflicting duplicate mappings log a warning and skip conversion instead of crashing. Absent mapping keeps identity behavior. 2. Docs updated to the yyyyMMdd contract: docs/data-types.md and the English/Chinese tree user guides no longer describe numeric DATE as days since epoch; they now document the INT32 yyyyMMdd encoding with examples (20240101 for 2024-01-01). 3. parseDateToInt/parseIntToDate now validate like the Java client's DateUtils/LocalDate: invalid Date objects (NaN time), non-integer or non-finite numbers, years outside 1000-9999, and impossible calendar dates (e.g. 20230229, month 13, day 0) throw a descriptive Error instead of being silently normalized or serialized as garbage. null/undefined handling is unchanged (callers filter nulls first). Unit tests added: non-identity mapping TsBlock conversion (crafted TsBlock buffers with INT32 wire columns + DATE logical type), dedup and conflicting-mapping cases, and validation boundary tests (leap day 20240229 ok, 20230229/20241301/20240100/year 999/year 10000/NaN Date/non-integer/legacy days-epoch 19723 all throw).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #14
Problem
The Node.js client serializes
DATE(TSDataType 9) values as days since Unix epoch and deserializes them the same way. IoTDB's actual DATE wire format is an INT32 encoded asyear*10000 + month*100 + day(e.g.2026-07-13→20260713), as implemented by:DateUtils.parseDateExpressionToInt/parseIntToDate(iotdb-core)'2026-07-13'reads back as raw i3220260713on the wireConsequences of the old behavior: dates written via
insertTabletfrom Node.js were stored as small integers (~20655 for 2026), which other clients decode as year-0002 nonsense; dates written by any other client (~2026xxxx) decoded in Node.js as dates around year 57000.Fix
parseDateToInt/parseIntToDateinsrc/utils/DataTypes.ts(exported from the package index). Calendar components are taken in UTC, matching JavaLocalDatesemantics (no timezone math).Session.serializeColumn(tablet write),FastSerializer.serializeDateColumn(fast write path),Session.deserializeColumn(TSQueryDataSet read),ColumnDecoder/parseTsBlock(TsBlock read).parseTsBlocknow converts such columns toDateobjects usingdataTypeList, so query results returnDateinstead of raw integers.nullon read).Verification
20260713→01 35 27 69big-endian), TsBlock column decode. 124/124 pass.Date('2026-07-13')written via binary tablet and a'2026-07-13'SQL literal now read back identical (2026-07-13T00:00:00.000Z); previously the tablet row was corrupted.npm run buildand existing e2e data-type suites pass against a live server.