Decide the Selenium CDP screencast from the session capabilities

.changeset/selenium-cdp-decline.md

@@ -0,0 +1,5 @@
+---
+'@wdio/selenium-devtools': patch
+---
+
+Decide the screencast's CDP mode from the session capabilities. A session that advertises no CDP endpoint (Safari, Appium, Firefox) now goes straight to screenshot polling with a plain log line, instead of calling `createCDPConnection` and logging the error it threw (`debuggerAddress.match is not a function` on Safari and Appium).

CLAUDE.md

@@ -400,7 +400,7 @@ Documented divergences from the conventions above. They exist today as debt to b
 - **One capture per action means two driver round trips per action, and the platform decides which half is expensive.** Measured on Appium 3.7.0 / UiAutomator2 (emulator-5554, Android API 37, 1080×2424): `GET /screenshot` 1.18–2.22 s (steady ~1.20 s) at 1.86 MB, `GET /source` (page-source XML) 0.09–0.53 s at 40 KB, `window/rect` 0.015 s — the screenshot dominates by ~10× on Android. **iOS is unmeasured**; the issue claims the split inverts there, and the native example (`examples/wdio/mocha/wdio.native.conf.ts`) is what makes it measurable. Measured per action: two captures 2.41 s against one capture 1.19 s; end-to-end on the native example 19.1 s against 12.4 s, with live mode (no per-action capture) at 5.9 s. That is the case for the `beforeCommand` capture, and the reason an eager post-action capture is not worth re-adding: it cost a second capture per action and existed only to be patched by a `readyState` poll that could not tell a document that had not navigated yet from one that was loading.
 - **Locator generation from a native page source must stay linear, and iOS is where it did not.** Every native action snapshot builds element locators from the page-source XML (`core/locators`), and each uniqueness check was a `//*[@name="…"]` XPath run by the `xpath` library over the whole document, several per element. Measured on the iOS Settings page source tiled up: 38 KB 82 ms, 188 KB 1.4 s, 752 KB 23.5 s, which is what a reporter saw as trace mode adding 1.5 min to a 2 min 20 s iOS test while Android, with smaller flatter trees, added 10 s. `locators/xpath-index.ts` indexes the document once per `generateAllElementLocators` walk (`withXPathIndex`) and answers `//*`, `//Tag` and `//Tag[@a="x" and …]` from it: 15 ms, 49 ms, 268 ms for the same three, identical elements. It is scoped to the walk so a caller of the exported `evaluateXPath` that edits its document is never answered stale, and it declines a namespaced document, where XPath name tests resolve namespaces and the index does not. The page-source request itself remains: 0.75-1.2 s per action on XCUITest even for Settings, against 0.09-0.5 s on UiAutomator2.
 - A filmstrip poll against a native session stacks requests: `setInterval` never waits for its async handler, a native screenshot takes ~1.2 s against the 200 ms default, and a serialised driver serves that queue ahead of the test's own commands — measured, a **15 ms command took 4.5–7.8 s**. `ScreencastRecorderBase`'s `#pollInFlight` latch bounds it to one outstanding shot, and `#pollGeneration` stops a shot orphaned by `stop()` from appending after the stop or clearing its successor's latch.
-- **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection`, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop.
+- **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection` and the capabilities read that decides whether to call it, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop.
 - **Only a command that can change the page is captured** (`shared/trace-actions.ts` `skipsCapture`, a deny-list: reads, waits, lookups and assertions are skipped, anything unlisted captures; the ACTION_MAP reads come from the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit.
 - The next test's first pre-capture used to be stamped at the previous test's last-action timestamp — the log it scans is run-long, so a test boundary was invisible to it. That slot already holds the previous test's finalize capture, and the richer-screenshot merge could replace it: with a `reloadSession` between the tests the row replayed the post-reload page. The capture now stamps `Date.now()` when the scanned timestamp predates `#currentTestStartWallTime` (0 without per-test hooks, so the standalone path is unchanged) — the same rule that makes a session's first capture its initial frame. Per-test slices were never affected: `flushTest` runs inside `afterTest`, before the next test's commands.
 

README.md

@@ -114,7 +114,7 @@ Want a portable artifact instead of a live UI (CI / agent diffing)? Switch to **
 - **Automatic Video Recording**: Captures a continuous `.webm` video of the browser session alongside the existing snapshot and DOM mutation views
 - **Per-framework modes**:
   - **WebdriverIO**: CDP push mode for Chrome/Chromium (efficient, no per-command overhead); polling fallback for other browsers
-  - **Selenium WebDriver**: CDP push mode via `selenium-webdriver/bidi`; polling fallback otherwise
+  - **Selenium WebDriver**: CDP push mode when the session advertises a CDP endpoint (Chromium); polling otherwise, Safari and Appium included
   - **Nightwatch.js**: Polling mode (Nightwatch doesn't expose a stable CDP escape hatch); works on every browser Nightwatch supports
 - **Per-Session Videos**: Each browser session (including sessions created by `browser.reloadSession()`) produces its own recording, selectable from a dropdown in the UI
 - **Smart Trimming**: Leading blank frames before the first URL navigation are automatically removed so videos start at the first meaningful page action

packages/selenium-devtools/README.md

@@ -274,7 +274,7 @@ DevTools.configure({
 
 Detailed sub-options: `quality` (0–100 JPEG quality, default 70), `maxWidth`/`maxHeight` (frame size, default 1280×720), `captureFormat` (`'jpeg'` or `'png'`), `pollIntervalMs` (used for non-Chrome browsers; default 200ms).
 
-Uses Chrome DevTools Protocol push mode where available; falls back to screenshot polling for Firefox / Safari with no config change.
+Uses Chrome DevTools Protocol push mode when the session's capabilities advertise a CDP endpoint (`se:cdp`, or a `debuggerAddress` under `goog:chromeOptions` / `ms:edgeOptions`), which local and Grid Chromium sessions do, or when `SELENIUM_REMOTE_URL` points at a Grid; every other session (Firefox, Safari, Appium) goes straight to screenshot polling with no config change. A CDP connection that fails anyway also falls back to polling.
 
 #### `headless` — hide the test browser window
 **Default:** `false` (the test browser is visible). Set to `true` to run the **test** browser without a window — useful for CI servers or when the popping window is annoying. The dashboard window is unaffected and still opens.

packages/selenium-devtools/src/helpers/cdp-endpoint.ts

@@ -0,0 +1,36 @@
+import type { CapGet } from './driverMetadata.js'
+
+function vendorDebuggerAddress(vendorOptions: unknown): string | undefined {
+  if (!vendorOptions || typeof vendorOptions !== 'object') {
+    return undefined
+  }
+  const address = (vendorOptions as { debuggerAddress?: unknown })
+    .debuggerAddress
+  return typeof address === 'string' && address ? address : undefined
+}
+
+/**
+ * Why `createCDPConnection` cannot reach a CDP endpoint for this session, or
+ * `undefined` when it can. Mirrors how selenium-webdriver resolves the
+ * endpoint (`se:cdp`, then the Chromium vendor options' `debuggerAddress`,
+ * or the `SELENIUM_REMOTE_URL` grid route), since without one it throws
+ * `debuggerAddress.match is not a function` rather than declining.
+ */
+export function cdpUnavailableReason(capGet: CapGet): string | undefined {
+  // Recent selenium-webdriver refuses CDP for Firefox, and Firefox never served Page.startScreencast.
+  if (capGet('browserName') === 'firefox') {
+    return 'CDP is not supported for Firefox'
+  }
+  if (process.env.SELENIUM_REMOTE_URL) {
+    return undefined
+  }
+  const seCdp = capGet('se:cdp')
+  if (typeof seCdp === 'string' && seCdp) {
+    return undefined
+  }
+  const vendorOptions = capGet('goog:chromeOptions') || capGet('ms:edgeOptions')
+  if (vendorDebuggerAddress(vendorOptions)) {
+    return undefined
+  }
+  return 'no CDP endpoint in the session capabilities'
+}

packages/selenium-devtools/src/helpers/driverMetadata.ts

@@ -24,7 +24,7 @@ export interface DriverMetadataResult {
   metadata: Record<string, unknown> | undefined
 }
 
-type CapGet = (k: string) => unknown
+export type CapGet = (k: string) => unknown
 
 /**
  * A plain bag from selenium's `Capabilities`, whose data lives in a private
@@ -50,7 +50,7 @@ function serializeCapabilities(capabilities: unknown): Record<string, unknown> {
   return caps?.serialize?.() ?? (capabilities as Record<string, unknown>) ?? {}
 }
 
-function makeCapGet(capabilities: unknown): CapGet {
+export function makeCapGet(capabilities: unknown): CapGet {
   return (k: string) => {
     const caps = capabilities as
       | {

packages/selenium-devtools/src/screencast.ts

@@ -8,6 +8,8 @@ import {
 } from '@wdio/devtools-core'
 import { BLANK_FRAME_THRESHOLD_BYTES } from './constants.js'
 import { getDriverOriginals } from './driverPatcher.js'
+import { cdpUnavailableReason } from './helpers/cdp-endpoint.js'
+import { makeCapGet } from './helpers/driverMetadata.js'
 import type { SeleniumDriverLike } from './types.js'
 
 const log = logger('@wdio/selenium-devtools:ScreencastRecorder')
@@ -139,8 +141,30 @@ export class ScreencastRecorder extends ScreencastRecorderBase<SeleniumDriverLik
     return cdp
   }
 
+  /** Declines before calling: without an endpoint selenium-webdriver throws a TypeError rather than declining. */
+  async #cdpDeclineReason(): Promise<string | undefined> {
+    const driver = this.driver
+    if (typeof driver?.getCapabilities !== 'function') {
+      return undefined
+    }
+    // selenium-webdriver answers getCapabilities from the session it already holds, so no round trip.
+    const capabilities = await withTimeout(
+      driver.getCapabilities(),
+      SCREENCAST_HANDSHAKE_TIMEOUT_MS,
+      undefined
+    )
+    return capabilities === undefined
+      ? undefined
+      : cdpUnavailableReason(makeCapGet(capabilities))
+  }
+
   protected override async tryStartCdp(): Promise<boolean> {
     try {
+      const declined = await this.#cdpDeclineReason()
+      if (declined) {
+        log.info(`${declined}; polling`)
+        return false
+      }
       const cdp = await this.#openCdpConnection()
       if (!cdp) {
         log.warn('CDP connection unavailable — falling back to polling')

packages/selenium-devtools/tests/screencast.test.ts

@@ -1,4 +1,4 @@
-import { describe, it, expect, vi } from 'vitest'
+import { describe, it, expect, vi, afterEach } from 'vitest'
 import { ScreencastRecorder } from '../src/screencast.js'
 import { getDriverOriginals } from '../src/driverPatcher.js'
 import type { SeleniumDriverLike } from '../src/types.js'
@@ -137,3 +137,106 @@ describe('ScreencastRecorder — CDP handshake ceiling', () => {
     }
   })
 })
+
+function capabilities(bag: Record<string, unknown>) {
+  const map = new Map(Object.entries(bag))
+  return { get: (k: string) => map.get(k), keys: () => map.keys() }
+}
+
+async function startWith(
+  caps: Record<string, unknown>,
+  createCDPConnection: SeleniumDriverLike['createCDPConnection']
+): Promise<TestScreencastRecorder> {
+  const driver: SeleniumDriverLike = {
+    executeScript: () => Promise.resolve(null),
+    getCapabilities: async () => capabilities(caps),
+    createCDPConnection
+  }
+  getDriverOriginals().takeScreenshot = vi.fn(async () => 'frame-1')
+  const r = new TestScreencastRecorder({ pollIntervalMs: 50 })
+  await r.start(driver)
+  return r
+}
+
+describe('ScreencastRecorder: CDP decided from the session capabilities', () => {
+  afterEach(() => {
+    getDriverOriginals().takeScreenshot = undefined
+    vi.unstubAllEnvs()
+  })
+
+  it.each([
+    ['desktop Safari', { browserName: 'Safari', platformName: 'mac' }],
+    [
+      'Appium native',
+      { platformName: 'Android', 'appium:automationName': 'UiAutomator2' }
+    ],
+    // The shape that made selenium-webdriver throw: vendor options with no debuggerAddress.
+    [
+      'Appium Chrome',
+      {
+        browserName: 'chrome',
+        platformName: 'Android',
+        'goog:chromeOptions': {}
+      }
+    ],
+    ['Firefox', { browserName: 'firefox', 'moz:debuggerAddress': true }]
+  ])('%s polls without calling createCDPConnection', async (_name, caps) => {
+    vi.stubEnv('SELENIUM_REMOTE_URL', '')
+    const createCDPConnection = vi.fn()
+    const r = await startWith(caps, createCDPConnection)
+    expect(createCDPConnection).not.toHaveBeenCalled()
+    expect(r.isRecording).toBe(true)
+    expect(r.frames[0].data).toBe('frame-1')
+    await r.stop()
+  })
+
+  it.each([
+    [
+      'local Chrome',
+      {
+        browserName: 'chrome',
+        'goog:chromeOptions': { debuggerAddress: 'localhost:9222' }
+      }
+    ],
+    [
+      'local Edge',
+      {
+        browserName: 'MicrosoftEdge',
+        'ms:edgeOptions': { debuggerAddress: 'localhost:9223' }
+      }
+    ],
+    [
+      'Grid Chrome',
+      { browserName: 'chrome', 'se:cdp': 'ws://grid:4444/session/1/se/cdp' }
+    ]
+  ])('%s tries CDP', async (_name, caps) => {
+    vi.stubEnv('SELENIUM_REMOTE_URL', '')
+    const createCDPConnection = vi.fn(async () => ({ execute: vi.fn() }))
+    const r = await startWith(caps, createCDPConnection)
+    expect(createCDPConnection).toHaveBeenCalledWith('page')
+    await r.stop()
+  })
+
+  it('a session under SELENIUM_REMOTE_URL tries CDP, since selenium-webdriver routes it through the grid', async () => {
+    vi.stubEnv('SELENIUM_REMOTE_URL', 'http://grid:4444/wd/hub')
+    const createCDPConnection = vi.fn(async () => ({ execute: vi.fn() }))
+    const r = await startWith({ browserName: 'chrome' }, createCDPConnection)
+    expect(createCDPConnection).toHaveBeenCalledTimes(1)
+    await r.stop()
+  })
+
+  it('a CDP connection that still fails falls back to polling', async () => {
+    vi.stubEnv('SELENIUM_REMOTE_URL', '')
+    const createCDPConnection = vi.fn(async () => {
+      throw new Error('ECONNREFUSED')
+    })
+    const r = await startWith(
+      { 'goog:chromeOptions': { debuggerAddress: 'localhost:9222' } },
+      createCDPConnection
+    )
+    expect(createCDPConnection).toHaveBeenCalledTimes(1)
+    expect(r.isRecording).toBe(true)
+    expect(r.frames[0].data).toBe('frame-1')
+    await r.stop()
+  })
+})