Skip to content

Fix hourly time labels dropping an hour on DST transition days - #6626

Open
jaideeppyne wants to merge 1 commit into
plausible:masterfrom
jaideeppyne:fix-hourly-time-labels-dst
Open

Fix hourly time labels dropping an hour on DST transition days#6626
jaideeppyne wants to merge 1 commit into
plausible:masterfrom
jaideeppyne:fix-hourly-time-labels-dst

Conversation

@jaideeppyne

Copy link
Copy Markdown

Changes

On a day that a site's timezone moves its clocks forward, the hourly graph silently loses the last hour of the day.

time_labels/1 built the time:hour labels by adding 0..n hours to the naive start of the range, where n came from DateTime.diff(last, first, :hour). That diff is absolute, so a 23 hour day yields 23 labels ending at 22:00. ClickHouse buckets the rows with toStartOfHour(toTimeZone(timestamp, tz)), which does produce a 23:00 bucket, so that row has no label to land on. The same range also emits a label for 02:00, an hour that never happened. Fall-back days get the opposite: 25 labels for 24 buckets, the last one belonging to the next day.

New York, 2024-03-10, before:

["2024-03-10 00:00:00", "2024-03-10 01:00:00", "2024-03-10 02:00:00", ..., "2024-03-10 22:00:00"]

after:

["2024-03-10 00:00:00", "2024-03-10 01:00:00", "2024-03-10 03:00:00", ..., "2024-03-10 23:00:00"]

I found this with a property check: every hour bucket ClickHouse can produce for a range must have exactly one label, and no label may exist for an hour that doesn't. I used Tzdata as the oracle for what the local hours of a day really are. Sweeping all 596 zones over every day from 2015 to 2030 gives 3,483,024 (zone, day) pairs; 6,284 of them were wrong before, 0 after. Both directions of the bug show up in every zone that observes DST, and Pacific/Chatham and Antarctica/Casey were also wrong because their transitions aren't on the hour.

time:minute already handles this correctly, and the day/week/month labels are pure Date arithmetic, so this only touches time:hour.

Scope I did not cover: a sub-day window that straddles a backwards transition in a zone whose transitions aren't on the hour can still miss one bucket. In practice that is Pacific/Chatham on its April switch, reachable via "Last 24 Hours" or a custom API range. Fixing it properly needs the bucket-to-range overlap test rather than a naive hour walk, which is a lot more code. Happy to add it if you want it covered.

The three new tests fail on master and pass here. I also ran test/plausible/stats/, main_graph_test.exs and test/plausible_web/controllers/api/external_stats_controller/ before and after: identical results, no new failures.

I used an AI assistant to help write the property-check harness and this patch. I ran everything and checked the results myself.

Tests

  • Automated tests have been added

Changelog

  • Entry has been added to changelog

Documentation

  • This change does not need a documentation update

Dark mode

  • This PR does not change the UI

time_labels/1 built the time:hour labels by adding a fixed number of hours
to the naive start of the range, with the count taken from the absolute
length of the range. ClickHouse buckets these rows with
toStartOfHour(toTimeZone(timestamp, tz)), so on a day that gains or loses
an hour the two disagree: the last hour of a spring-forward day gets no
label and its traffic drops off the graph, and a nonexistent hour is
labelled instead.

Walk the local hours of the range and keep the ones that occur in the
site timezone.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants