docs: document the TIMESTAMP WITH TIME ZONE type mapping and fix a stale comment

[adriangb]

Which issue does this PR close?

This PR closes no issue. It changes documentation and one comment, and it changes no behaviour.

• Related to TIMESTAMP WITH TIME ZONE can resolve to a timezone-naive type, and casting to it discards an existing timezone #25166

Rationale for this change

What a user hits

There is no bug to reproduce. There is a mapping a user cannot look up. To find out what TIMESTAMP WITH TIME ZONE produces, a user must run it:

-- default configuration: datafusion.execution.time_zone is unset
SELECT arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);
+---------------+
| Timestamp(ns) |
+---------------+

The type has no timezone. Set the session timezone and the same expression gains one:

SET datafusion.execution.time_zone = 'America/New_York';
SELECT arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);
+-----------------------------------+
| Timestamp(ns, "America/New_York") |
+-----------------------------------+

docs/source/user-guide/sql/data_types.md holds the SQL-to-Arrow type table. Three things were absent from it:

• any row for TIMESTAMPTZ or TIMESTAMP WITH TIME ZONE
• any note that the result depends on datafusion.execution.time_zone
• any note that TIMESTAMP(p) is accepted, and which values of p are valid

A user who reads the page learns none of the above.

The comment in datafusion/sql/src/planner.rs was worse than absent. It stated the opposite of the code:

// OUTPUT: [ArrowDataType] Timestamp<TimeUnit, Some(Time Zone)>
self.context_provider.options().execution.time_zone.clone()

The expression is an Option<String> that is None by default, so the arm produces Timestamp(unit, None) on a default install.

What changes for a user

The type table gains the two absent spellings and the precision rules, so a user can look the mapping up instead of a run of arrow_typeof. The page also states that the timezone component comes from a session setting that is unset by default, and it links the open issue on that default.

No behaviour changes.

The technical detail

The comment went stale in #18359, which changed datafusion.execution.time_zone from String to Option<String> with a None default. That PR dropped the Some(...) wrapper from the expression as a mechanical consequence of the type change, and left the two comment lines above it untouched.

Whether the timezone-naive default is right is under discussion in #25166. This PR only makes the documentation match the current code. It deliberately changes no behaviour.

Field research

The table is exactly where a user who moves from another engine looks, so I measured both reference engines.

PostgreSQL 17.11 (postgres:17):

=> CREATE TABLE t(b TIMESTAMPTZ, c TIMESTAMP WITH TIME ZONE);
=> SELECT column_name, data_type FROM information_schema.columns WHERE table_name='t';
 b | timestamp with time zone
 c | timestamp with time zone

=> SELECT pg_typeof('2024-01-01'::timestamptz);
 timestamp with time zone

DuckDB 1.5.2:

D SELECT typeof('2024-01-01 00:00:00'::TIMESTAMPTZ);
 TIMESTAMP WITH TIME ZONE
D SELECT typeof('2024-01-01'::TIMESTAMP WITH TIME ZONE);
 TIMESTAMP WITH TIME ZONE

Both engines always resolve the type to a zone-aware type. Neither has an unset session timezone, so neither can produce a naive result from this SQL type. DataFusion produces a naive result on a default install. That divergence is the subject of #25166.

The precision rules diverge too:

• PostgreSQL accepts p from 0 to 6. TIMESTAMP(1) and TIMESTAMP(4) are valid. TIMESTAMP(9) raises a warning and reduces p to 6.
• DuckDB accepts p from 0 to 9 and rounds up to the next storage unit. TIMESTAMP(1) gives timestamp_ms, TIMESTAMP(4) gives timestamp and TIMESTAMP(7) gives timestamp_ns.
• DataFusion accepts 0, 3, 6 and 9 alone, and rejects every other value.

What changes are included in this PR?

• datafusion/sql/src/planner.rs: correct the SQLDataType::Timestamp comment to say Timestamp<TimeUnit, Time Zone>, note that the configured time zone is an Option that is unset by default, and point at TIMESTAMP WITH TIME ZONE can resolve to a timezone-naive type, and casting to it discards an existing timezone #25166. There is no code change.
• docs/source/user-guide/sql/data_types.md: add the TIMESTAMP WITH TIME ZONE and TIMESTAMPTZ row to the Date/Time Types table, document the optional (p) precision and the units it selects, and state that the timezone component comes from datafusion.execution.time_zone, which is unset by default, with a link to the open issue.

What is the testing strategy for this PR?

There are no tests. This is a comment fix plus a documentation fix, with no behaviour change.

Every mapping in the docs was verified against a cargo build --bin datafusion-cli build of this branch rather than read off the source:

> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP);    -- Timestamp(ns)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(0)); -- Timestamp(s)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(3)); -- Timestamp(ms)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(6)); -- Timestamp(µs)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(9)); -- Timestamp(ns)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(1)); -- Error: Unsupported SQL type TIMESTAMP(1)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(2)); -- Error: Unsupported SQL type TIMESTAMP(2)

-- default configuration (datafusion.execution.time_zone unset)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);                          -- Timestamp(ns)
> select arrow_typeof(CAST('2000-01-01T00:00:00' AS TIMESTAMP WITH TIME ZONE));     -- Timestamp(ns)
> select arrow_typeof(CAST('2000-01-01T00:00:00' AS TIMESTAMP(6) WITH TIME ZONE));  -- Timestamp(µs)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ(3));                       -- Timestamp(ms)

> SET datafusion.execution.time_zone = 'America/New_York';
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);                          -- Timestamp(ns, "America/New_York")
> select arrow_typeof(CAST('2000-01-01T00:00:00' AS TIMESTAMP(3) WITH TIME ZONE));  -- Timestamp(ms, "America/New_York")
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP);                            -- Timestamp(ns)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(0));                         -- Timestamp(s)

The DDL path gives the same answers:

> CREATE TABLE t(a TIMESTAMP, b TIMESTAMPTZ, c TIMESTAMP(6), d TIMESTAMP(0) WITH TIME ZONE);
> describe t;
 a | Timestamp(ns)
 b | Timestamp(ns)
 c | Timestamp(µs)
 d | Timestamp(s)

./ci/scripts/doc_prettier_check.sh, cargo fmt --all and cargo clippy --all-targets -- -D warnings all pass. The Sphinx docs build cleanly. Its one warning is the pre-existing absent generated _static/data/deps.svg, which is unrelated to this change.

Are there any user-facing changes?

Documentation only. There are no behaviour changes and no API changes.

Merge order

There is no file-level conflict with any PR in flight. #25175 adds only datafusion/sqllogictest/test_files/datetime/timestamps_timezone.slt, and this PR touches planner.rs and data_types.md.

The two do share a dependency. Sections 0 to 4 of #25175 pin the same timezone-naive result that the new paragraph describes. A fix for #25166 must update this page and that file in one change.

🤖 Generated with Claude Code

[adriangb]

Self-review QA pass on my own PR. I built datafusion-cli from this branch and checked every mapping claim against it. Two problems need a fix. Four are smaller.

1. The text calls a defect an open question (must fix)

The new paragraph ends:

Whether that should remain the mapping is an open question tracked in [issue #25166].

#25166 carries the bug label. Its "Expected behavior" section states an invariant, not a preference:

TIMESTAMP WITH TIME ZONE never resolves to Timestamp(_, None).

"An open question" reads as a design debate with two defensible sides. The issue does not claim that. It claims a defect with a traceable cause in #18359.

This is the whole risk on this PR. I withdrew #25162 for one reason: reference text turns a defect into semantics that a reader must learn. This paragraph repeats a softer form of the same error. A reader comes away with "naive by default is the rule I must learn", not "this mapping is disputed".

Fix: name it a defect. For example, "The current mapping is a known defect, tracked in [issue #25166]." One word carries the whole signal.

2. The table row alone teaches the opposite of the default (must fix)

The new row says:

| TIMESTAMPTZ or TIMESTAMP WITH TIME ZONE, optionally with (p) | Timestamp(unit, tz) |

Measured on this branch, default configuration:

> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);
Timestamp(ns)

The cell promises a zone. The default gives none. The prose below corrects it, but a table is the part people scan. The TIMESTAMP row directly above says Timestamp(unit, None), so the contrast inside the table reads as "this one is zone-aware". That is exactly backwards for a default install.

Fix: put the default in the cell, such as Timestamp(unit, tz), or Timestamp(unit, None) by default.

3. The precision paragraph appears to cover TIME, but it does not

The paragraph starts "unit is determined by the optional precision p" and sits under the whole table. The table includes a TIME row. TIME(p) is not accepted at all:

> select arrow_typeof('00:00:00'::TIME(0));
Error: This feature is not implemented: Unsupported SQL type TIME(0)
> select arrow_typeof('00:00:00'::TIME(3));
Error: This feature is not implemented: Unsupported SQL type TIME(3)

Scope the paragraph to the two TIMESTAMP rows.

4. "optionally with (p)" hides where (p) goes

The two spellings put the precision in different places. Both work, and the row does not say so:

> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ(3));                      -- Timestamp(ms)
> select arrow_typeof(CAST('2000-01-01T00:00:00' AS TIMESTAMP(6) WITH TIME ZONE)); -- Timestamp(µs)

TIMESTAMPTZ(p) takes it at the end. TIMESTAMP(p) WITH TIME ZONE takes it in the middle. A reference table must show both forms.

5. TIMESTAMP WITHOUT TIME ZONE is absent

The PR adds every spelling of the zone-aware form but not the explicit naive form, which also works:

> select arrow_typeof(CAST('2000-01-01' AS TIMESTAMP WITHOUT TIME ZONE));
Timestamp(ns)

Add it to the TIMESTAMP row.

6. The table is where a migrant looks, and the divergence is not there

I measured both reference engines rather than quote their docs.

PostgreSQL 17.11:

=> CREATE TABLE t(b TIMESTAMPTZ, c TIMESTAMP WITH TIME ZONE);
=> SELECT column_name, data_type FROM information_schema.columns WHERE table_name='t';
 b | timestamp with time zone
 c | timestamp with time zone
=> SELECT pg_typeof('2024-01-01'::timestamptz);
 timestamp with time zone

DuckDB 1.5.2:

D SELECT typeof('2024-01-01 00:00:00'::TIMESTAMPTZ);
 TIMESTAMP WITH TIME ZONE

Both engines always resolve the type to a zone-aware type. Neither has an unset session zone. DataFusion resolves it to a zone-naive type on a default install. A user who moves from either engine reads this table first, so the divergence belongs in it, in one clause.

The precision rules also diverge, which is worth one line for the same reason:

• PostgreSQL accepts p from 0 to 6, so TIMESTAMP(1) and TIMESTAMP(4) are valid. TIMESTAMP(9) warns and reduces to 6.
• DuckDB accepts 0 to 9 and rounds up to the next storage unit, so TIMESTAMP(1) gives timestamp_ms and TIMESTAMP(7) gives timestamp_ns.
• DataFusion accepts only 0, 3, 6 and 9, and rejects the rest.

What I verified, and it is correct

Every mapping claim in the diff holds. Measured on a cargo build --bin datafusion-cli build of this branch, not read off the source.

Default configuration (datafusion.execution.time_zone is NULL):

> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP);                          -- Timestamp(ns)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(0));                       -- Timestamp(s)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(3));                       -- Timestamp(ms)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(6));                       -- Timestamp(µs)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(9));                       -- Timestamp(ns)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(1));                       -- Error: Unsupported SQL type TIMESTAMP(1)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(2));                       -- Error: Unsupported SQL type TIMESTAMP(2)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);                        -- Timestamp(ns)
> select arrow_typeof(CAST('2000-01-01T00:00:00' AS TIMESTAMP WITH TIME ZONE));   -- Timestamp(ns)
> select arrow_typeof('2000-01-01'::DATE);                                        -- Date32
> select arrow_typeof('00:00:00'::TIME);                                          -- Time64(ns)
> select arrow_typeof(INTERVAL '1' DAY);                                          -- Interval(MonthDayNano)

With the setting present:

> SET datafusion.execution.time_zone = 'America/New_York';
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMPTZ);                        -- Timestamp(ns, "America/New_York")
> select arrow_typeof(CAST('2000-01-01T00:00:00' AS TIMESTAMP(3) WITH TIME ZONE)); -- Timestamp(ms, "America/New_York")
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP);                          -- Timestamp(ns)
> select arrow_typeof('2000-01-01T00:00:00'::TIMESTAMP(0));                       -- Timestamp(s)

The DDL path agrees:

> CREATE TABLE t(a TIMESTAMP, b TIMESTAMPTZ, c TIMESTAMP(6), d TIMESTAMP(0) WITH TIME ZONE);
> describe t;
 a | Timestamp(ns)
 b | Timestamp(ns)
 c | Timestamp(µs)
 d | Timestamp(s)

Other checks:

• ./ci/scripts/doc_prettier_check.sh passes. The table rows are aligned at 105 columns each, so the diff only looks ragged.
• ../configs.md is the link style the neighbouring pages already use (explain.md, ddl.md, format_options.md).
• The planner.rs comment now matches the expression below it. It is a comment only, so no behaviour moves.

One adjacent nit, pre-existing

The untouched INTERVAL row says Interval(IntervalMonthDayNano). arrow_typeof prints Interval(MonthDayNano). The PR already edits this table, so the row can be corrected in the same commit.

Merge order

No conflict applies. #25175 adds only datafusion/sqllogictest/test_files/datetime/timestamps_timezone.slt, and this PR touches planner.rs and data_types.md, so the two do not overlap at file level.

There is a shared dependency worth a note in the body. Section 0 to 4 of #25175 pins the same naive result that this paragraph describes. A fix for #25166 must update both places in one change.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.91%. Comparing base (da89c7c) to head (095e6d3).
⚠️ Report is 142 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25171      +/-   ##
==========================================
+ Coverage   81.60%   81.91%   +0.31%     
==========================================
  Files        1123     1132       +9     
  Lines      408898   421117   +12219     
  Branches   408898   421117   +12219     
==========================================
+ Hits       333670   344951   +11281     
- Misses      55625    55775     +150     
- Partials    19603    20391     +788     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.