From b1effec2e39cfaaedfd5620ae51e308afb3b32de Mon Sep 17 00:00:00 2001 From: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com> Date: Thu, 10 Sep 2026 13:25:47 -0500 Subject: [PATCH 1/2] docs: fix stale `TIMESTAMP WITH TIME ZONE` comment, document the mapping The comment on the `SQLDataType::Timestamp` arm in `planner.rs` promises `Timestamp`, but the expression it describes is an `Option` that is `None` by default. That comment went stale in https://github.com/apache/datafusion/pull/18359, which changed `datafusion.execution.time_zone` from `String` to `Option` and dropped the `Some(...)` wrapper without touching the two lines above it. Correct the comment; the behavior itself is under discussion in https://github.com/apache/datafusion/issues/25166 and is left unchanged. `docs/source/user-guide/sql/data_types.md` had no row for `TIMESTAMP WITH TIME ZONE` / `TIMESTAMPTZ` at all. Add one, along with the optional `(p)` precision that both it and `TIMESTAMP` accept, and note that the timezone component comes from `datafusion.execution.time_zone`, which is unset by default. Co-Authored-By: Claude Opus 5 --- datafusion/sql/src/planner.rs | 8 +++++++- docs/source/user-guide/sql/data_types.md | 26 ++++++++++++++++++------ 2 files changed, 27 insertions(+), 7 deletions(-) diff --git a/datafusion/sql/src/planner.rs b/datafusion/sql/src/planner.rs index a3e6d75fdbfac..bb7347e6396d2 100644 --- a/datafusion/sql/src/planner.rs +++ b/datafusion/sql/src/planner.rs @@ -775,7 +775,13 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> { { // Timestamp With Time Zone // INPUT : [SQLDataType] TimestampTz + [Config] Time Zone - // OUTPUT: [ArrowDataType] Timestamp + // OUTPUT: [ArrowDataType] Timestamp + // + // Note that the configured time zone is an `Option` that is + // unset by default, so by default this yields the + // timezone-naive `Timestamp`. Whether that + // should remain the behavior is tracked in + // https://github.com/apache/datafusion/issues/25166 self.context_provider.options().execution.time_zone.clone() } else { // Timestamp Without Time zone diff --git a/docs/source/user-guide/sql/data_types.md b/docs/source/user-guide/sql/data_types.md index 2e75fce6c7aa9..6dd2fb603b576 100644 --- a/docs/source/user-guide/sql/data_types.md +++ b/docs/source/user-guide/sql/data_types.md @@ -105,12 +105,26 @@ The maximum supported precision for `DECIMAL` types is 76. ## Date/Time Types -| SQL DataType | Arrow DataType | -| ------------ | :------------------------------- | -| `DATE` | `Date32` | -| `TIME` | `Time64(Nanosecond)` | -| `TIMESTAMP` | `Timestamp(Nanosecond, None)` | -| `INTERVAL` | `Interval(IntervalMonthDayNano)` | +| SQL DataType | Arrow DataType | +| ------------------------------------------------------------------ | :------------------------------- | +| `DATE` | `Date32` | +| `TIME` | `Time64(Nanosecond)` | +| `TIMESTAMP` or `TIMESTAMP(p)` | `Timestamp(unit, None)` | +| `TIMESTAMPTZ` or `TIMESTAMP WITH TIME ZONE`, optionally with `(p)` | `Timestamp(unit, tz)` | +| `INTERVAL` | `Interval(IntervalMonthDayNano)` | + +`unit` is determined by the optional precision `p`, which must be `0`, `3`, `6` +or `9` for `Second`, `Millisecond`, `Microsecond` or `Nanosecond` respectively. +Any other precision is rejected. When `p` is omitted the unit is `Nanosecond`. + +`tz` is the value of the [`datafusion.execution.time_zone`] setting. That +setting is unset by default, so with the default configuration +`TIMESTAMPTZ` and `TIMESTAMP WITH TIME ZONE` map to `Timestamp(unit, None)`, +the same timezone-naive type as `TIMESTAMP`. Whether that should remain the +mapping is an open question tracked in [issue #25166]. + +[`datafusion.execution.time_zone`]: ../configs.md +[issue #25166]: https://github.com/apache/datafusion/issues/25166 ## Boolean Types From 095e6d3a311600a086700ac6ef38412cd9a0b331 Mon Sep 17 00:00:00 2001 From: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com> Date: Thu, 10 Sep 2026 18:02:26 -0500 Subject: [PATCH 2/2] docs: state the timestamptz default in the table, not two paragraphs later Review feedback on this PR: the table cell said `Timestamp(unit, tz)` while the default configuration produces `Timestamp(unit, None)`, and the row directly above it said `None`. So the table on its own taught the opposite of what happens, and the correction sat two paragraphs below. People scan tables. Also softens nothing: issue 25166 carries the `bug` label and states an invariant, so "an open question" was the wrong register. It is a known bug, and the text now says so and names what PostgreSQL and DuckDB do instead. Adds `TIMESTAMP WITHOUT TIME ZONE`, which works but was absent, and states where the precision goes, which differs between the two spellings. Moves the precision paragraph so it no longer appears to cover `DATE`, `TIME` and `INTERVAL`, none of which accept one. Every mapping re-verified against a `datafusion-cli` build. Co-Authored-By: Claude Opus 5 --- docs/source/user-guide/sql/data_types.md | 45 +++++++++++++++--------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/docs/source/user-guide/sql/data_types.md b/docs/source/user-guide/sql/data_types.md index 6dd2fb603b576..9d2fe3e6e8060 100644 --- a/docs/source/user-guide/sql/data_types.md +++ b/docs/source/user-guide/sql/data_types.md @@ -105,23 +105,34 @@ The maximum supported precision for `DECIMAL` types is 76. ## Date/Time Types -| SQL DataType | Arrow DataType | -| ------------------------------------------------------------------ | :------------------------------- | -| `DATE` | `Date32` | -| `TIME` | `Time64(Nanosecond)` | -| `TIMESTAMP` or `TIMESTAMP(p)` | `Timestamp(unit, None)` | -| `TIMESTAMPTZ` or `TIMESTAMP WITH TIME ZONE`, optionally with `(p)` | `Timestamp(unit, tz)` | -| `INTERVAL` | `Interval(IntervalMonthDayNano)` | - -`unit` is determined by the optional precision `p`, which must be `0`, `3`, `6` -or `9` for `Second`, `Millisecond`, `Microsecond` or `Nanosecond` respectively. -Any other precision is rejected. When `p` is omitted the unit is `Nanosecond`. - -`tz` is the value of the [`datafusion.execution.time_zone`] setting. That -setting is unset by default, so with the default configuration -`TIMESTAMPTZ` and `TIMESTAMP WITH TIME ZONE` map to `Timestamp(unit, None)`, -the same timezone-naive type as `TIMESTAMP`. Whether that should remain the -mapping is an open question tracked in [issue #25166]. +| SQL DataType | Arrow DataType | +| ---------------------------------------------------------- | :--------------------------------------------------------- | +| `DATE` | `Date32` | +| `TIME` | `Time64(Nanosecond)` | +| `TIMESTAMP`, `TIMESTAMP(p)`, `TIMESTAMP WITHOUT TIME ZONE` | `Timestamp(unit, None)` | +| `TIMESTAMPTZ(p)`, `TIMESTAMP(p) WITH TIME ZONE` | `Timestamp(unit, None)` by default — see the warning below | +| `INTERVAL` | `Interval(IntervalMonthDayNano)` | + +:::{warning} +`TIMESTAMPTZ` and `TIMESTAMP WITH TIME ZONE` do **not** give a timezone-aware +type by default. The zone comes from the +[`datafusion.execution.time_zone`] setting, and that setting is unset unless you +set it. So `'2024-01-01T12:00:00Z'::timestamptz` gives `Timestamp(unit, None)` +by default, and DataFusion discards the `Z`. + +This is a known bug. See [issue #25166]. PostgreSQL and DuckDB always give an +aware type here, because their session time zone always has a value. + +Set `datafusion.execution.time_zone` to make these types aware. `'UTC'` is a +good value. +::: + +`unit` comes from the optional precision `p` on the two `TIMESTAMP` rows. Use +`0`, `3`, `6` or `9` for `Second`, `Millisecond`, `Microsecond` or `Nanosecond`. +DataFusion rejects each other value of `p`. If you omit `p`, the unit is +`Nanosecond`. The precision goes after `TIMESTAMPTZ`, but before +`WITH TIME ZONE`: use `TIMESTAMPTZ(3)` or `TIMESTAMP(3) WITH TIME ZONE`. +`DATE`, `TIME` and `INTERVAL` do not accept a precision. [`datafusion.execution.time_zone`]: ../configs.md [issue #25166]: https://github.com/apache/datafusion/issues/25166