e2e flake: Freeze clock in instance metrics e2e tests#3274
Merged
Conversation
The date range picker tests derive a default "Last hour" range from the real wall clock, which makes them fail near midnight UTC in two ways: the range straddles the day boundary (fixed defensively in #3269), and the calendar opens on the range start month, so the "Today" cell is missing when the run crosses a month boundary (June 30 -> July 1). Freeze the clock to a fixed mid-month noon so neither depends on when CI runs, and revert the #3269 time-juggling that is no longer needed.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fix in #3269 was a hack. This is a better fix, and covers a second case I didn't learn about until an hour ago.
🤖 Summary
The metrics page defaults to a "Last hour" range derived from the wall clock, so what the tests see depends on when they run. Two failure modes:
23:xxyesterday, end00:xxtoday). The test collapses both dates to today, which keeps the inherited times, so start23:xxnow lands after end00:xx, the range reads invalid before the test has done anything, and the "should be valid" assertion fails. This is what Fix time-of-day flake in date range picker e2e test #3269 worked around by setting the times explicitly.00:xxon the 1st =23:xxon the last day of the previous month) is in a different month than today. The calendar opens on the start's month, so today's cell isn't in the visible grid,getByRole('button', { name: /Today/ })finds nothing, and the click times out after 60s. This is what failed just now (June 30 → July 1): Playwright (chrome), all three browsers, tests running at 00:03 UTC.No user impact either way — near midnight the calendar just opens on the start month, which is normal. The problem is just what the tests expect to see when that happens.
Fix: Freeze the clock to a fixed mid-month noon with
page.clock.setFixedTime(same trick as the visual regression suite), so neither the range times nor the visible month depend on when CI runs. With that in place the #3269 time-juggling is no longer needed, so I reverted it back to the simpler form.