diff --git a/datafusion/sql/src/planner.rs b/datafusion/sql/src/planner.rs index a3e6d75fdbfac..bb7347e6396d2 100644 --- a/datafusion/sql/src/planner.rs +++ b/datafusion/sql/src/planner.rs @@ -775,7 +775,13 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> { { // Timestamp With Time Zone // INPUT : [SQLDataType] TimestampTz + [Config] Time Zone - // OUTPUT: [ArrowDataType] Timestamp + // OUTPUT: [ArrowDataType] Timestamp + // + // Note that the configured time zone is an `Option` that is + // unset by default, so by default this yields the + // timezone-naive `Timestamp`. Whether that + // should remain the behavior is tracked in + // https://github.com/apache/datafusion/issues/25166 self.context_provider.options().execution.time_zone.clone() } else { // Timestamp Without Time zone diff --git a/docs/source/user-guide/sql/data_types.md b/docs/source/user-guide/sql/data_types.md index 2e75fce6c7aa9..9d2fe3e6e8060 100644 --- a/docs/source/user-guide/sql/data_types.md +++ b/docs/source/user-guide/sql/data_types.md @@ -105,12 +105,37 @@ The maximum supported precision for `DECIMAL` types is 76. ## Date/Time Types -| SQL DataType | Arrow DataType | -| ------------ | :------------------------------- | -| `DATE` | `Date32` | -| `TIME` | `Time64(Nanosecond)` | -| `TIMESTAMP` | `Timestamp(Nanosecond, None)` | -| `INTERVAL` | `Interval(IntervalMonthDayNano)` | +| SQL DataType | Arrow DataType | +| ---------------------------------------------------------- | :--------------------------------------------------------- | +| `DATE` | `Date32` | +| `TIME` | `Time64(Nanosecond)` | +| `TIMESTAMP`, `TIMESTAMP(p)`, `TIMESTAMP WITHOUT TIME ZONE` | `Timestamp(unit, None)` | +| `TIMESTAMPTZ(p)`, `TIMESTAMP(p) WITH TIME ZONE` | `Timestamp(unit, None)` by default — see the warning below | +| `INTERVAL` | `Interval(IntervalMonthDayNano)` | + +:::{warning} +`TIMESTAMPTZ` and `TIMESTAMP WITH TIME ZONE` do **not** give a timezone-aware +type by default. The zone comes from the +[`datafusion.execution.time_zone`] setting, and that setting is unset unless you +set it. So `'2024-01-01T12:00:00Z'::timestamptz` gives `Timestamp(unit, None)` +by default, and DataFusion discards the `Z`. + +This is a known bug. See [issue #25166]. PostgreSQL and DuckDB always give an +aware type here, because their session time zone always has a value. + +Set `datafusion.execution.time_zone` to make these types aware. `'UTC'` is a +good value. +::: + +`unit` comes from the optional precision `p` on the two `TIMESTAMP` rows. Use +`0`, `3`, `6` or `9` for `Second`, `Millisecond`, `Microsecond` or `Nanosecond`. +DataFusion rejects each other value of `p`. If you omit `p`, the unit is +`Nanosecond`. The precision goes after `TIMESTAMPTZ`, but before +`WITH TIME ZONE`: use `TIMESTAMPTZ(3)` or `TIMESTAMP(3) WITH TIME ZONE`. +`DATE`, `TIME` and `INTERVAL` do not accept a precision. + +[`datafusion.execution.time_zone`]: ../configs.md +[issue #25166]: https://github.com/apache/datafusion/issues/25166 ## Boolean Types