Skip to content

fix(e2e): stabilize Linux UI tests on small Xvfb display#1003

Merged
wenytang-ms merged 2 commits intomainfrom
fix/stabilize-e2e-tests
May 6, 2026
Merged

fix(e2e): stabilize Linux UI tests on small Xvfb display#1003
wenytang-ms merged 2 commits intomainfrom
fix/stabilize-e2e-tests

Conversation

@wenytang-ms
Copy link
Copy Markdown
Contributor

@wenytang-ms wenytang-ms commented May 6, 2026

Why CI Run #25417136374 Failed

The Linux-UI workflow on the run in question reported ❌ 27/31 for Java Dependency — File Operations and ❌ 15/17 for Java Dependency — Project Explorer. After downloading the screenshots artifact and reading the autotest source, I found three independent root causes, none of which was the extension code itself.

1. Sticky pane-header intercepts clicks in the Java Projects tree

The javaProjectExplorer view is contributed inside the standard Explorer container (see package.json "views": { "explorer": [...] }). On the CI Xvfb display of 1024x768 with the default sidebar layout (Maven explorer expanded, Outline + Timeline visible, plus VS Code's new Chat auxiliary bar on the right) the JAVA PROJECTS section ends up with ~140 px of vertical room. Its sticky pane-header (<div class="pane-header expanded" aria-label="Java Projects Section">) sits right on top of the first tree row.

Playwright finds the my-app treeitem but the actual click is consumed by the section header. From the job log:

- waiting for getByRole('treeitem', { name: 'my-app' }).locator('a').first()
- locator resolved to <a class="label-name">…</a>
- attempting click action
- <h3 class="title" custom-hover="true">Java Projects</h3> from
    <div ... class="pane-header expanded" aria-label="Java Projects Section">…</div>
    subtree intercepts pointer events

The same effect causes the clickTreeItemAction my-app New... step to register its click on the Collapse All codicon in the section header (the hitTarget log shows aria-label="Collapse All" instead of New...), which dismisses the inline-action popover and breaks the rest of the create-package flow (select-package, select-source-folder-2, enter-package-name all time out waiting for a Quick Pick that was never opened).

This effect gets worse between iterations because link-with-editor auto-expands the JAVA PROJECTS tree to follow the active editor — after App2.java is created, my-app's children push it right under the header.

2. Unsupported action syntax in java-dep-project-explorer.yaml

The plan used expand <name> tree item for three steps. autotest's ActionResolver only recognizes expandTreeItem <name> (no spaces, camelCase). The fallback path runs the string through the command palette, which silently dismisses unknown commands — so the tree was never actually expanded:

⚠️  No pattern match for: "expand my-app tree item" — trying as command palette
✅ [expand-project] expand my-app tree item (748ms)        ← false success
…
❌ [verify-package] wait 1 seconds (16135ms)
   → Tree item "com.mycompany.app" did not appear within 15s

The same is true for expand src/main/java tree item and expand com.mycompany.app tree item. The verify-revealed step at the end of the plan only passed by luck — open file AppToRename.java triggered link-with-editor and pre-expanded the tree.

3. Hidden command-palette commands

java.view.package.linkWithFolderExplorer, unlinkWithFolderExplorer and revealInProjectExplorer are all registered with "when": false on their commandPalette menu contribution:

"commandPalette": [
  { "command": "java.view.package.linkWithFolderExplorer",     "when": "false" },
  { "command": "java.view.package.unlinkWithFolderExplorer",   "when": "false" },
  { "command": "java.view.package.revealInProjectExplorer",    "when": "false" }
]

So run command Java: Reveal in Java Project Explorer could not invoke the command. The screenshot 34_verify-revealed_after.png shows VS Code instead opening "Select the project type" (the Java: Create Java Project picker) — fuzzy match found a different command. Again, the verify step only passed because App was already visible in the tree from previous link-with-editor expansions.

Fix in this PR

File Change
.github/workflows/linuxUI.yml Bump Xvfb resolution from 1024x7681920x1080. Gives the Java Projects pane real estate and removes the pane-header overlap class of failures.
test/e2e-plans/java-dep-file-operations.yaml Add startup steps to executeVSCodeCommand workbench.action.closeAuxiliaryBar, collapseSidebarSection OUTLINE, collapseSidebarSection TIMELINE, collapseWorkspaceRoot. Between the create-class and create-package iterations: close the editor that link-with-editor opened and re-collapse the workspace root, so my-app stays away from the sticky header. Same collapseWorkspaceRoot is repeated before each subsequent test (rename, delete) for the same reason.
test/e2e-plans/java-dep-project-explorer.yaml Replace expand <name> tree item with expandTreeItem <name>. Replace the three palette-hidden commands with executeVSCodeCommand <id>. Same startup closeAuxiliaryBar + collapse steps.

Both plans pass npx autotest validate.

How to verify

This PR will trigger the same linuxUI.yml workflow. Expected:

  • All ~31 file-operations steps pass (no more 5 s locator.click timeouts on my-app).
  • All 17 project-explorer steps pass (com.mycompany.app and App visible after explicit expansion; link/unlink/reveal actually invoked).

If the CI still flakes on a specific step we'll have stable, deterministic logs (no more silent fallbacks) and the screenshots will reflect what was actually attempted.

Recommended follow-ups (separate PRs in microsoft/javaext-autotest)

These would harden the framework so similar bugs are caught at write time instead of in CI:

  1. Strict mode: when ActionResolver falls back to the command palette and VS Code reports no matching command, fail the step instead of silently succeeding. The current behavior masked all 6 of these issues.
  2. collapseTreeItem <name> — there's expandTreeItem (no-op if expanded) but no symmetric collapse, so plans currently rely on View: Collapse All or full re-launches.
  3. Click stability: in clickTreeItem / clickTreeItemAction, after scrollIntoViewIfNeeded() also check document.elementFromPoint() against the target row; if the hit element belongs to a sticky pane-header, scroll the inner list further before clicking. This would make plans tolerant of small viewports without requiring callers to manually free space.
  4. Consider documenting in AGENTS.md that hidden palette commands need executeVSCodeCommand.

The Linux-UI workflow was failing 6 steps across the two e2e plans on
the 1024x768 Xvfb display used in CI. Three independent root causes:

1. Sticky pane-header click interception. The JAVA PROJECTS view is
   rendered inside the Explorer sidebar. With OUTLINE/TIMELINE/MAVEN
   sections also visible, the Java Projects pane has very little
   vertical space, and its sticky pane-header sits right on top of the
   first tree row. Playwright finds the my-app treeitem but the click
   is consumed by the section header (the Playwright call log shows the
   pane-header subtree intercepts pointer events). Fix: bump Xvfb to
   1920x1080 and explicitly close the auxiliary (Chat) bar plus
   collapse OUTLINE, TIMELINE and the workspace root before
   interacting with the Java Projects tree.

2. Unsupported action syntax in java-dep-project-explorer.yaml. The
   strings "expand my-app tree item", "expand src/main/java tree item"
   and "expand com.mycompany.app tree item" do not match any pattern in
   autotest's ActionResolver (only "expandTreeItem <name>" is
   recognized). They silently fell back to the command palette and
   no-op'd, so the tree never actually expanded and the subsequent
   verifyTreeItem checks for "com.mycompany.app" and "App" timed out.
   Fix: use the "expandTreeItem <name>" form.

3. Hidden command-palette commands. java.view.package.linkWith
   FolderExplorer, unlinkWithFolderExplorer and revealInProjectExplorer
   all have "when": false on their commandPalette menu contribution and
   cannot be invoked via the command palette. They were no-ops in the
   previous plan (the reveal-in-project-explorer step was even opening
   the wrong "Create Java Project" picker). Fix: invoke them by command
   id with "executeVSCodeCommand".

Also close the editor opened by the create-class step before the
create-package step, so link-with-editor doesn't auto-expand the tree
and push my-app under the sticky header again.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…k/reveal

Once autotest >=0.6.6 ships clickViewTitleAction and contextMenuOnEditorTab,
exercise the actual VS Code UI affordances instead of dispatching the
extension commands by id:

- unlinkWithFolderExplorer / linkWithFolderExplorer → clickViewTitleAction
  on the JAVA PROJECTS pane title bar (overflow menu when sync is on/off).
- revealInProjectExplorer → right-click the editor tab and pick
  'Reveal in Java Project Explorer' from editor/title/context.

These were palette-hidden ('when: false') and therefore previously fell
back to executeVSCodeCommand. Verified locally end-to-end: 21/21 steps
pass on the project-explorer plan and 39/39 on file-operations with the
locally-built autotest.

closeAuxiliaryBar still uses executeVSCodeCommand intentionally — it is
a built-in VS Code helper (not extension UI under test) and now dispatches
through a real keybinding rather than the broken driver.executeCommand
path.
@wenytang-ms wenytang-ms merged commit 578059f into main May 6, 2026
6 checks passed
@wenytang-ms wenytang-ms deleted the fix/stabilize-e2e-tests branch May 6, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants