Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion datafusion/sql/src/planner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -775,7 +775,13 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
{
// Timestamp With Time Zone
// INPUT : [SQLDataType] TimestampTz + [Config] Time Zone
// OUTPUT: [ArrowDataType] Timestamp<TimeUnit, Some(Time Zone)>
// OUTPUT: [ArrowDataType] Timestamp<TimeUnit, Time Zone>
//
// Note that the configured time zone is an `Option` that is
// unset by default, so by default this yields the
// timezone-naive `Timestamp<TimeUnit, None>`. Whether that
// should remain the behavior is tracked in
// https://github.com/apache/datafusion/issues/25166
self.context_provider.options().execution.time_zone.clone()
} else {
// Timestamp Without Time zone
Expand Down
37 changes: 31 additions & 6 deletions docs/source/user-guide/sql/data_types.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,12 +105,37 @@ The maximum supported precision for `DECIMAL` types is 76.

## Date/Time Types

| SQL DataType | Arrow DataType |
| ------------ | :------------------------------- |
| `DATE` | `Date32` |
| `TIME` | `Time64(Nanosecond)` |
| `TIMESTAMP` | `Timestamp(Nanosecond, None)` |
| `INTERVAL` | `Interval(IntervalMonthDayNano)` |
| SQL DataType | Arrow DataType |
| ---------------------------------------------------------- | :--------------------------------------------------------- |
| `DATE` | `Date32` |
| `TIME` | `Time64(Nanosecond)` |
| `TIMESTAMP`, `TIMESTAMP(p)`, `TIMESTAMP WITHOUT TIME ZONE` | `Timestamp(unit, None)` |
| `TIMESTAMPTZ(p)`, `TIMESTAMP(p) WITH TIME ZONE` | `Timestamp(unit, None)` by default — see the warning below |
| `INTERVAL` | `Interval(IntervalMonthDayNano)` |

:::{warning}
`TIMESTAMPTZ` and `TIMESTAMP WITH TIME ZONE` do **not** give a timezone-aware
type by default. The zone comes from the
[`datafusion.execution.time_zone`] setting, and that setting is unset unless you
set it. So `'2024-01-01T12:00:00Z'::timestamptz` gives `Timestamp(unit, None)`
by default, and DataFusion discards the `Z`.

This is a known bug. See [issue #25166]. PostgreSQL and DuckDB always give an
aware type here, because their session time zone always has a value.

Set `datafusion.execution.time_zone` to make these types aware. `'UTC'` is a
good value.
:::

`unit` comes from the optional precision `p` on the two `TIMESTAMP` rows. Use
`0`, `3`, `6` or `9` for `Second`, `Millisecond`, `Microsecond` or `Nanosecond`.
DataFusion rejects each other value of `p`. If you omit `p`, the unit is
`Nanosecond`. The precision goes after `TIMESTAMPTZ`, but before
`WITH TIME ZONE`: use `TIMESTAMPTZ(3)` or `TIMESTAMP(3) WITH TIME ZONE`.
`DATE`, `TIME` and `INTERVAL` do not accept a precision.

[`datafusion.execution.time_zone`]: ../configs.md
[issue #25166]: https://github.com/apache/datafusion/issues/25166

## Boolean Types

Expand Down