Add mobile replay harness

[maciejfs]

Adds scripts/mobile/: a set of Node scripts that drive any iOS app on a physical device and then check whether the FS replay looks right.

iOS only for now. Android is not supported yet and will come in a follow-up.

How it works

You write a goal JSON file that describes what screen to navigate to, what taps to perform, and what the replay should contain. The runner reads that file plus env config, connects to the device through Lidar live tools (or Appium directly), drives the goal, and writes snapshots and a report. Separate scripts then fetch the FS replay, pull out observations, and validate them against the goal.

Nothing is hardcoded for any particular app. All app-specific config comes from the goal JSON and .env.local.

What's in here

• run-lidar-live-ios.mjs: drives the app through Lidar MCP live tools
• run-goal.mjs: drives the app through Appium/WebDriverIO
• run-local-lidar-ios.mjs: builds and starts a local Lidar from your mn checkout, starts Appium, runs the goal, and writes the FS replay URL when the run finishes
• extract-session-url.mjs: pulls the iOS NSUserDefaults plist through Appium and writes the FS replay URL (used by the runner above, also runnable on its own)
• validate-goal-artifacts.mjs: checks device artifacts match the goal
• fetch-subtext-review-evidence.mjs: fetches replay evidence from Subtext MCP
• capture-subtext-review-observations.mjs: extracts structured observations from replay evidence
• validate-replay-observations.mjs: compares observations to goal expectations
• goals/example.json: documented goal skeleton you copy and edit for your app
• appium-layer.mjs, device-e2e-common.mjs: shared Appium/device plumbing

How to test

1. Copy scripts/mobile/mobile.env.example to scripts/mobile/.env.local and fill in your values.
2. Write a goal JSON for your app (or use an existing one locally).
3. Make sure node is on your PATH:

cd ~/src/subtext
export PATH="$HOME/src/fsdev/tools/node/bin:$PATH"

4. Run:

node scripts/mobile/run-local-lidar-ios.mjs

That builds Lidar, starts Appium, connects to the device, runs the goal, and writes output to tmp/. Takes about 3 minutes.

5. Wait ~90s for the replay to ingest, then validate:

export MOBILE_OUT_DIR=$(cat scripts/mobile/tmp/.last-run-dir)
node scripts/mobile/validate-goal-artifacts.mjs
node scripts/mobile/fetch-subtext-review-evidence.mjs
node scripts/mobile/capture-subtext-review-observations.mjs
node scripts/mobile/validate-replay-observations.mjs

For a remote Lidar server, set LIDAR_IOS_MCP_URL and run node scripts/mobile/run-lidar-live-ios.mjs.

See README.md in scripts/mobile/ for the full details.

Example .env.local (for internal FullStoryTestingApp testing)

FULLSTORY_API_KEY=<your key from app.fullstory.com>
MOBILE_BUNDLE_ID=com.fullstory.FullStoryTestingApp
MOBILE_UDID=<your device udid, from: xcrun devicectl list devices>
MOBILE_DEVICE_NAME=<your device name, e.g. "Maciej's iPhone 14">
LOCAL_MCP_ORG_ID=KWH
LOCAL_MCP_EMAIL=you@fullstory.com
MOBILE_GOAL_EXPECTATIONS=./scripts/mobile/goals/images.json
MOBILE_FULLSTORY_APP_HOST=https://app.staging.fullstory.com

The images.json goal file is not committed (gitignored), but you can create one by copying goals/example.json and filling in the navigation steps for the Images screen in FullStoryTestingApp.

MOBILE_FULLSTORY_APP_HOST is only needed for non-prod orgs (staging or EU). It defaults to https://app.fullstory.com.

Related

Lidar iOS live tool support: https://github.com/cowpaths/mn/pull/104208