Fix hourly time labels dropping an hour on DST transition days - #6626
Open
jaideeppyne wants to merge 1 commit into
Open
Fix hourly time labels dropping an hour on DST transition days#6626jaideeppyne wants to merge 1 commit into
jaideeppyne wants to merge 1 commit into
Conversation
time_labels/1 built the time:hour labels by adding a fixed number of hours to the naive start of the range, with the count taken from the absolute length of the range. ClickHouse buckets these rows with toStartOfHour(toTimeZone(timestamp, tz)), so on a day that gains or loses an hour the two disagree: the last hour of a spring-forward day gets no label and its traffic drops off the graph, and a nonexistent hour is labelled instead. Walk the local hours of the range and keep the ones that occur in the site timezone.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
On a day that a site's timezone moves its clocks forward, the hourly graph silently loses the last hour of the day.
time_labels/1built thetime:hourlabels by adding0..nhours to the naive start of the range, wherencame fromDateTime.diff(last, first, :hour). That diff is absolute, so a 23 hour day yields 23 labels ending at 22:00. ClickHouse buckets the rows withtoStartOfHour(toTimeZone(timestamp, tz)), which does produce a 23:00 bucket, so that row has no label to land on. The same range also emits a label for 02:00, an hour that never happened. Fall-back days get the opposite: 25 labels for 24 buckets, the last one belonging to the next day.New York, 2024-03-10, before:
after:
I found this with a property check: every hour bucket ClickHouse can produce for a range must have exactly one label, and no label may exist for an hour that doesn't. I used
Tzdataas the oracle for what the local hours of a day really are. Sweeping all 596 zones over every day from 2015 to 2030 gives 3,483,024 (zone, day) pairs; 6,284 of them were wrong before, 0 after. Both directions of the bug show up in every zone that observes DST, andPacific/ChathamandAntarctica/Caseywere also wrong because their transitions aren't on the hour.time:minutealready handles this correctly, and the day/week/month labels are pureDatearithmetic, so this only touchestime:hour.Scope I did not cover: a sub-day window that straddles a backwards transition in a zone whose transitions aren't on the hour can still miss one bucket. In practice that is
Pacific/Chathamon its April switch, reachable via "Last 24 Hours" or a custom API range. Fixing it properly needs the bucket-to-range overlap test rather than a naive hour walk, which is a lot more code. Happy to add it if you want it covered.The three new tests fail on master and pass here. I also ran
test/plausible/stats/,main_graph_test.exsandtest/plausible_web/controllers/api/external_stats_controller/before and after: identical results, no new failures.I used an AI assistant to help write the property-check harness and this patch. I ran everything and checked the results myself.
Tests
Changelog
Documentation
Dark mode