Skip to content

feat: Guided Tours E2E Tests#2415

Open
camielvs wants to merge 1 commit into
06-11-feat_guided_tours_completion_metricsfrom
06-12-feat_guided_tours_e2e_tests
Open

feat: Guided Tours E2E Tests#2415
camielvs wants to merge 1 commit into
06-11-feat_guided_tours_completion_metricsfrom
06-12-feat_guided_tours_e2e_tests

Conversation

@camielvs

@camielvs camielvs commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

This PR is 100% AI generated.

End-to-end tests that replicate and verify the exact flow of every guided tour, so a UI change that breaks a tour is caught here (and points at the tour + step to fix) rather than shipping a broken tour.

Rather than four bespoke specs, this is a single data-driven driver that reads each tour's real JSON definition and walks it the way a user would:

  • tests/e2e/tours/tourDriver.ts — runTour(page, tour) opens the tour, then for each step: asserts the step's spotlight anchor resolves in the DOM (the core regression signal), performs the gating interaction (or clicks Next for passive steps), and waits for the tour to advance (synchronised on the ?step=N URL the tour writes). It finishes on the completion step. A handler exists for every interaction type — drag-from-library, edge connection, task select, set-argument, undock/redock window, subgraph navigate/unpack, marquee multi-select, create-subgraph, and the full secrets flow.
  • Specs — one thin spec per tour: navigating-editor, first-pipeline, subgraphs, using-secrets. Each loads the shipped tour JSON via loadTour(...), so the test tracks the real tour and new steps are covered automatically (modulo a brand-new interaction type needing a handler).
  • eslint.config.js — teach playwright/expect-expect that runTour asserts internally.

Because the driver asserts each step's data-tour* anchor, removing or renaming an anchor a tour depends on fails the matching step by design.

The using-secrets spec exercises the no-backend path from #2405 / #2406: it fails the health ping and /api/secrets so the tour's in-memory mock backend serves the secret steps. The tour runs fully hands-on with normal gating, no /api/secrets calls fire, and the Submit Run confirm is asserted disabled (real run not wired in mock mode).

Related Issue and Pull requests

Closes https://github.com/Shopify/oasis-frontend/issues/644

Sits on top of the guided-tour stack (#2405 / #2406 and the framework/tour PRs below them).

Type of Change

  • New feature

Checklist

  • I have tested this does not break current pipelines / runs functionality
  • I have tested the changes on staging

Test Instructions

  • pnpm test:e2e tests/e2e/tours/ — all four tour specs pass (run in parallel against a local dev server).
  • The using-secrets spec drives the no-backend path: it routes the health ping and /api/secrets to fail, so the tour's mockBackend activates and the in-memory mock serves the secret steps. It asserts no /api/secrets calls fire and that Submit Run stays disabled in mock mode.

Additional Comments

The centered tour popover overlaps a few dialog controls (Add Secret / submit confirm / the "Use Secret" lock button). The driver dispatches the click directly to those elements (clickThrough) so React's handler still fires — the only spot where it doesn't use a real mouse click.

@github-actions

github-actions Bot commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown

🎩 Preview

A preview build has been created at: 06-12-feat_guided_tours_e2e_tests/d723728

This was referenced Jun 12, 2026
Open
Open
Open
Open
@camielvs camielvs added the #gsd:50583 Learning Hub label Jun 12, 2026 — with Graphite App
This was referenced Jun 12, 2026
Open
Open
@camielvs Graphite App

camielvs commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@camielvs camielvs mentioned this pull request Jun 12, 2026
Open
3 tasks
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 586e87d to 6bffb2c Compare June 12, 2026 23:36
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from f7adb0c to f9ccfa0 Compare June 12, 2026 23:36
@camielvs camielvs mentioned this pull request Jun 13, 2026
Open
3 tasks
camielvs
camielvs commented Jun 15, 2026
View reviewed changes
Comment thread tests/e2e/tours/tourDriver.ts Outdated
camielvs
camielvs commented Jun 15, 2026
View reviewed changes
Comment thread tests/e2e/tours/tourDriver.ts Outdated
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from f9ccfa0 to c1361f2 Compare June 15, 2026 17:55
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 6bffb2c to 18c2dd3 Compare June 15, 2026 17:55
@camielvs camielvs mentioned this pull request Jun 15, 2026
Open
8 tasks
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 18c2dd3 to 0e1b241 Compare June 15, 2026 18:47
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from 088f063 to ef6a2d0 Compare June 15, 2026 18:59
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 0e1b241 to 7f552ac Compare June 15, 2026 18:59
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from ef6a2d0 to b583f84 Compare June 15, 2026 19:21
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch 2 times, most recently from 3b6725f to cf9670d Compare June 15, 2026 19:32
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from b583f84 to cf0a129 Compare June 15, 2026 19:32
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from cf9670d to 9a1a38d Compare June 15, 2026 19:49
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from 03beccf to d725087 Compare June 15, 2026 20:13
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 9a1a38d to ad996b1 Compare June 15, 2026 20:13
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 9954dd1 to 68c9054 Compare June 16, 2026 01:15
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from 173e3d0 to fb6be72 Compare June 16, 2026 01:26
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 68c9054 to dc367c2 Compare June 16, 2026 01:26
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from fb6be72 to afe48e8 Compare June 16, 2026 17:19
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch 2 times, most recently from 20ce5e1 to 5120618 Compare June 16, 2026 18:35
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from 467fa3c to 9073078 Compare June 16, 2026 19:12
@camielvs camielvs marked this pull request as ready for review June 16, 2026 19:33
@camielvs camielvs requested a review from a team as a code owner June 16, 2026 19:33
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 9073078 to be78ffa Compare June 17, 2026 17:24
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 5120618 to 8c80eaf Compare June 17, 2026 17:24
This was referenced Jun 17, 2026
Open
Open
Open
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 8c80eaf to 612ade1 Compare June 19, 2026 02:25
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from be78ffa to 30f2eac Compare June 19, 2026 02:25
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 612ade1 to f048e6a Compare June 19, 2026 18:49
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch 2 times, most recently from b8fa57d to 84979cf Compare June 19, 2026 21:48
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from f048e6a to 9585c4f Compare June 19, 2026 21:48
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 84979cf to a795168 Compare June 26, 2026 00:20
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 9585c4f to 96f0bf2 Compare June 26, 2026 00:20
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from a795168 to 02bc46f Compare June 26, 2026 00:37
@camielvs camielvs force-pushed the 06-11-feat_guided_tours_completion_metrics branch from 96f0bf2 to 1632bef Compare June 26, 2026 00:37
@camielvs @claude
feat: Guided Tours E2E Tests
d723728 
Data-driven driver that walks each tour's JSON: asserts every step's
data-tour anchor, performs the gating interaction (or clicks Next), and
verifies advancement, finishing on the completion step. One thin spec per
tour (navigating-editor, first-pipeline, subgraphs, using-secrets).

The using-secrets spec drives the no-backend path: it fails the health ping
and /api/secrets so the tour's in-memory mock backend handles the secret
steps, runs them fully hands-on with normal gating, and asserts the Submit
Run confirm stays disabled in mock mode.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@camielvs camielvs force-pushed the 06-12-feat_guided_tours_e2e_tests branch from 02bc46f to d723728 Compare June 26, 2026 00:53
maxy-shpfy
maxy-shpfy reviewed Jun 26, 2026
View reviewed changes
Comment thread tests/e2e/tours/tourDriver.ts
return { x: box.x + box.width / 2, y: box.y + Math.min(box.height / 2, 200) };
}

async function performInteraction(

@maxy-shpfy maxy-shpfy Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This is an AI-generated code review comment.

This PR adds a new 548-line driver with FTA 72.15 and cyclomatic complexity 65, well over the strong-flag thresholds. Most of the complexity is concentrated in the interaction dispatcher and broad canvas/window/secrets helper set, so every future tour interaction will have to modify this already-large file and switch. Because this file is new in this PR, the static-analysis regression belongs here.

Suggestion: Keep runTour small and split handlers into focused modules, for example canvasInteractions.ts, windowInteractions.ts, and secretsInteractions.ts, wired through a typed Record<InteractionName, Handler> or small dispatcher. That should bring file size and cyclomatic complexity below the project thresholds and make adding a new tour interact…

Rule: static-analysis skill: FTA > 60, cyclomatic complexity > 20, and file size > 400 lines are strong flags; project-conventions: prefer focused helpers and reduce nesting/large functi

maxy-shpfy
maxy-shpfy reviewed Jun 26, 2026
View reviewed changes
Comment thread tests/e2e/tours/tourDriver.ts
* the real tour the app ships, not a copy that can drift.
*/
export function loadTour(fileName: string): TourDef {
return JSON.parse(readFileSync(join(TOURS_DIR, fileName), "utf8")) as TourDef;

@maxy-shpfy maxy-shpfy Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This is an AI-generated code review comment.

JSON.parse returns any, and the direct as TourDef assertion bypasses the type safety the surrounding comments are trying to preserve. A malformed tour JSON can compile as TourDef and then fail later inside the runner with a less direct error.

Suggestion: Parse to unknown and add a small assertTourDef/type guard, or validate with the existing zod dependency, so missing id, non-array steps, or invalid step shapes fail at load time with the tour filename included.

Rule: typescript-standards: avoid unsafe type assertions; prefer type guards/runtime validation for external data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maxy-shpfy maxy-shpfy maxy-shpfy left review comments

Assignees

No one assigned

Labels

#gsd:50583 Learning Hub

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@camielvs @maxy-shpfy