From 8719bc1e2afd5082750f6e8198358bb125261490 Mon Sep 17 00:00:00 2001 From: Mike Pitre <12040919+mikepitre@users.noreply.github.com> Date: Tue, 13 Oct 2026 23:44:13 -0400 Subject: [PATCH 1/4] docs(expo): add the CLI output reference and the screen advice from clerk-ios clerk-ios's skill has a reference for what the CLI prints: the JSON shapes, the exit codes, and a table of error codes. The Expo skill had one sentence on exit codes. `references/output.md` now has the same content, with the Expo meaning of each code: two kinds of lanes, the steps of the test app's build, and the run on the other platform that holds the application. `SKILL.md` now says that `screen` right after `up` does not show what a spec sees, and how to read a missing node. The feature map gets the baseline preconditions that clerk-ios lists. Co-Authored-By: Claude Opus 5.5 --- .changeset/expo-verify-parity-skill.md | 2 + .claude/skills/verify-clerk-expo/SKILL.md | 4 +- .../verify-clerk-expo/features/README.md | 8 +++ .../verify-clerk-expo/references/output.md | 49 +++++++++++++++++++ 4 files changed, 62 insertions(+), 1 deletion(-) create mode 100644 .changeset/expo-verify-parity-skill.md create mode 100644 .claude/skills/verify-clerk-expo/references/output.md diff --git a/.changeset/expo-verify-parity-skill.md b/.changeset/expo-verify-parity-skill.md new file mode 100644 index 00000000000..a845151cc84 --- /dev/null +++ b/.changeset/expo-verify-parity-skill.md @@ -0,0 +1,2 @@ +--- +--- diff --git a/.claude/skills/verify-clerk-expo/SKILL.md b/.claude/skills/verify-clerk-expo/SKILL.md index 3f88a2b1191..bf3dff0dde5 100644 --- a/.claude/skills/verify-clerk-expo/SKILL.md +++ b/.claude/skills/verify-clerk-expo/SKILL.md @@ -11,7 +11,7 @@ The tests are an ordinary e2e project. `e2e.config.ts` and `specs/` run under `n No change to `@clerk/expo` or `@clerk/expo-native-components` UI or auth behavior is done until a simulator or emulator shows the changed behavior, on each platform the change touches. A `run` on the test app is one way to show it. An app of your own that exercises the change is just as good. -Run every command from the repo root. In the prose below, `doctor`, `up`, `run`, `screen`, `attach`, and `down` are verbs of that CLI. The tests and the CLI are the Node package `integration/expo-native/`, and paths that begin `specs/`, `src/`, `test/`, or `.verify/` are inside it. `features/` and `references/` are beside this file. Every verb but `attach` takes `--platform ios|android`, and iOS is the default. Every verb takes `--json` and then prints one `{ "ok": ... }` object. Exit codes are 0 for success, 1 for a failing spec, 2 for a usage error, and 3 for a failed precondition. Every error prints a `fix` line. +Run every command from the repo root. In the prose below, `doctor`, `up`, `run`, `screen`, `attach`, and `down` are verbs of that CLI. The tests and the CLI are the Node package `integration/expo-native/`, and paths that begin `specs/`, `src/`, `test/`, or `.verify/` are inside it. `features/` and `references/` are beside this file. Every verb but `attach` takes `--platform ios|android`, and iOS is the default. Every error prints a `fix` line. Every verb takes `--json`, and [CLI output](references/output.md) has its shape, the exit codes, and the error codes. The test app links `@clerk/expo`, `@clerk/expo-native-components`, `@clerk/expo-biometrics`, and `@clerk/expo-google-signin` from the workspace. It does not install `@clerk/expo-passkeys`, so the skill cannot verify passkeys. @@ -168,6 +168,8 @@ The specs under `specs/features//` are committed and cover the feature A failing spec prints `FAIL`, the first assertion message, and the path of its failure page, and `run` exits 1. The failure page (`.verify/runs//e2e/failures/*.md`) lists every step, the screen tree at the failure, and a screenshot. A run in which no test executed, such as a `--grep` that matches nothing, fails with `NO_SPECS` and exits 3, because it proved nothing. A run in which every selected spec is limited to the other platform is the one exception. It exits 0, prints `skip` for each spec, and its `next` line says `nothing ran: every selected spec was left out by platform, so the run proves nothing to post`. Read that line, because the exit code alone looks like a pass. +Right after `up` the CLI has not launched the test app, and `screen` opens it without the verify inputs that `host.launch` passes, so the screen is not one a spec sees. To look at a real screen, first run a spec that launches the app and taps to it. When a node may be missing, assert `await expect(locator).toBeVisible()` before you tap it. That failure reads `observed: no node (match count 0)`, and a failed tap says only `LOCATOR_NOT_FOUND`. + With `--retries`, a test that fails and then passes prints `flaky` and the error of its failed attempt, and a `flaky` line under the results counts such tests. `run.json` records the test as `flaky` with its `attempts`, never as passed, and `run` exits 0. The failure page of the failed attempt and the files of every attempt stay in the run directory. A test that fails on every attempt prints `FAIL`, and `run` exits 1. ### Agent steps diff --git a/.claude/skills/verify-clerk-expo/features/README.md b/.claude/skills/verify-clerk-expo/features/README.md index 87f3f4d71f8..4ea19c0288f 100644 --- a/.claude/skills/verify-clerk-expo/features/README.md +++ b/.claude/skills/verify-clerk-expo/features/README.md @@ -2,6 +2,14 @@ This directory is the maintained source for verifying the user-facing behavior of `@clerk/expo` through the `expo-native` test app in `integration/templates/expo-native`. On a local device the test app runs as a Debug dev client that loads its JS, and `packages/expo` with it, from a local Metro server. On a remote device it runs as a standalone Release build of a pushed commit, with the JS embedded. Read this index before driving the app, then use the matching feature file as the recipe. Every recipe runs through `integration/expo-native/bin/control-clerk-expo` and the specs under `specs/features//`. +## Baseline preconditions + +`SKILL.md` under Launch has the setup and the commands. These facts hold for every recipe here, on both platforms. + +- The standard settings have every auth method and signed-in feature that these recipes use. The `settings` check of `doctor` compares the live instance with the standard file once the worktree holds an application. +- Every launch gets a new `verifyStorageScope`, so no spec inherits a session from another spec. A launch with `keepStorage: true` reuses the scope of the launch before it in the same test. +- `down` deletes this worktree's application, and with it every user and organization a run created, including users created through the sign-up form. + ## Test users and sign-in `.claude/skills/verify-clerk-expo/SKILL.md` has the rules for signing in. These are the test identities and the identifiers a spec needs. diff --git a/.claude/skills/verify-clerk-expo/references/output.md b/.claude/skills/verify-clerk-expo/references/output.md new file mode 100644 index 00000000000..33c71e60b31 --- /dev/null +++ b/.claude/skills/verify-clerk-expo/references/output.md @@ -0,0 +1,49 @@ +# CLI output + +## Text output + +A verb prints progress lines to stderr while it works and its result to stdout when it ends. An error goes to stderr as two lines: `error`, the error code, and the message, then `fix:` and what to do. + +## JSON output + +Every verb takes `--json`. The verb then prints no progress lines, and prints one JSON object to stdout. + +On success the object is `{ "ok": true, "verb": "", ... }`. The remaining keys depend on the verb. `src/core/types.ts` defines them as `DoctorReport`, `UpResult`, `RunResult`, `ScreenResult`, `AttachResult`, and `DownResult`. + +On an error the object is `{ "ok": false, "error": { "code", "message", "fix", "retryable" } }`. + +`doctor --json` prints its report even when a check fails, with `ok: false` and no `error`. Each check has `id`, `ok`, `detail`, and a `fix` when there is one. A warning has `ok: true` and `state: "warning"`. A check that did not run has `ok: true` and `state: "not-run"`. + +`attach --json` prints one of two shapes. With `via: "gh"`, this machine edited the description, and the keys are `prUrl`, `posted`, and `alreadyPosted`. With `via: "runner"`, the evidence was handed to the session's runner, and the keys are `pr`, `handedOff`, `because`, `sessionRun`, and `sessionRunUrl`. + +`down --json` and `down --dry-run --json` print different keys. A dry run prints `dryRun: true`, `wouldRelease`, `wouldDelete`, `wouldStop`, and `keptRuns`. A real `down` prints `dryRun: false`, `released`, `deletedApplications`, `stoppedProcesses`, and `keptRuns`. Each entry of `wouldDelete` is `{ "kind": "application", "name" }`, and each entry of `deletedApplications` is `{ "name" }`. Neither lists users or organizations. A `down --platform

` that keeps the application for the other lane lists no application. + +## Exit codes + +| Code | Meaning | +| ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | The verb succeeded. For `run`, no spec failed. A spec that passed on a retry is `flaky` and does not fail the run. A run in which every selected spec is limited to the other platform also exits 0, so read its `next` line | +| 1 | `run` finished and at least one spec failed or was interrupted | +| 2 | The error code is `USAGE` | +| 3 | Any other error, or `doctor` has a failing check | + +## Error codes + +`retryable` is true for `POOL_FULL`, `DEVICE_BUSY`, `LEASE_LOST`, and `RATE_LIMITED`. For those, the same command can succeed later with no other change. + +| Code | Meaning | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `USAGE` | The command line, a spec's declaration, or an environment variable is malformed | +| `NOT_READY` | A precondition is missing, such as Metro that does not serve the current code. The fix names it | +| `POOL_FULL` | All four iOS lanes, or both Android lanes, on this machine are in use | +| `DEVICE_BUSY` | Another command in this worktree is driving the device, or a run on the other platform is using this worktree's application | +| `LEASE_LOST` | The leased device or the remote session is gone | +| `BUILD_FAILED` | A step of the test app's build failed, such as `turbo build`, `expo prebuild`, `xcodebuild`, or `gradlew`. The message names the step and has the last 15 lines of its output. With the remote backend, also that the app sources have uncommitted changes, that GitHub does not have HEAD, or that the session could not build the commit | +| `KEYS_MISSING` | No Platform API credential works on this machine, or `AI_GATEWAY_API_KEY_FILE` names a file that is missing, empty, or readable by other users | +| `INSTANCE_MISCONFIGURED` | The Clerk instance does not show the settings a spec needs | +| `NOT_TEST_IDENTITY` | Reserved. No command raises it. A run whose `app.log` names a user that the run did not create fails `attach` with `EVIDENCE_UNSAFE` | +| `NO_SPECS` | The selection matched no spec, or no selected spec ran | +| `E2E_CRASHED` | e2e exited without a readable report. Read `e2e.log` in the run directory | +| `EVIDENCE_UNSAFE` | The run cannot be attached, or a path is not safe to delete | +| `UNSUPPORTED` | This machine cannot do what the command asked, such as `--backend local` for a device it cannot run | +| `RATE_LIMITED` | Clerk's Platform API kept answering 429 | From 8a24d6f91ea0b7c79a23ec66c268df36ab7aeb01 Mon Sep 17 00:00:00 2001 From: Mike Pitre <12040919+mikepitre@users.noreply.github.com> Date: Wed, 14 Oct 2026 00:00:15 -0400 Subject: [PATCH 2/4] docs(expo): say when down keeps the application, and what doctor shows down --platform keeps the application while the other platform holds a lease. Only the tunnel and Clerk egress checks print blocked:, and a 403 on push means this machine's credential may not push. Co-Authored-By: Claude Opus 5.5 --- .claude/skills/verify-clerk-expo/features/README.md | 2 +- .claude/skills/verify-clerk-expo/references/remote.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.claude/skills/verify-clerk-expo/features/README.md b/.claude/skills/verify-clerk-expo/features/README.md index 4ea19c0288f..adf704bcd22 100644 --- a/.claude/skills/verify-clerk-expo/features/README.md +++ b/.claude/skills/verify-clerk-expo/features/README.md @@ -8,7 +8,7 @@ This directory is the maintained source for verifying the user-facing behavior o - The standard settings have every auth method and signed-in feature that these recipes use. The `settings` check of `doctor` compares the live instance with the standard file once the worktree holds an application. - Every launch gets a new `verifyStorageScope`, so no spec inherits a session from another spec. A launch with `keepStorage: true` reuses the scope of the launch before it in the same test. -- `down` deletes this worktree's application, and with it every user and organization a run created, including users created through the sign-up form. +- `down` deletes this worktree's application, and with it every user and organization a run created, including users created through the sign-up form. `down --platform

` keeps the application while the other platform still holds a lease in this worktree. It deletes the application when the other platform holds no lease. ## Test users and sign-in diff --git a/.claude/skills/verify-clerk-expo/references/remote.md b/.claude/skills/verify-clerk-expo/references/remote.md index c2d7c7376ba..16b7ce395ee 100644 --- a/.claude/skills/verify-clerk-expo/references/remote.md +++ b/.claude/skills/verify-clerk-expo/references/remote.md @@ -125,13 +125,13 @@ If the description has no evidence a few minutes after `down`, read the newest r Node ignores `HTTPS_PROXY` unless it is told to honor it, so the CLI decides. It goes through the proxy when the direct path to GitHub does not work, which means that it cannot connect or that GitHub rejects the machine's token on it. In that case the CLI reruns itself with the proxy in effect for itself, e2e, and agent-device. A machine that reaches GitHub directly stays direct. `doctor` prints the choice and the reason in `remote-env`. -A host that the environment blocks shows in `doctor` as `blocked:` and what answered. The fix is to add the host to the allowed domains of the environment that the CLI runs in. +The `tunnel-egress` and `clerk-egress` checks in `doctor` show a host that the environment blocks as `blocked:` and what answered, or as `no response from `. When `api.github.com` does not answer, the `github-rest` check shows `api.github.com did not answer`. In each case the fix is to add the host to the allowed domains of the environment that the CLI runs in. ## GitHub access A session uses GitHub in two ways. REST starts, reads, and ends sessions. `git push` gets a commit that you make to GitHub, so that the session can build it. -Claude Code cloud sessions are one example of a sandbox where the two differ. REST there uses the session user's own access. `git push` needs the Claude GitHub App installed on the repository, and a 403 on push means that it is not. The sandbox's `GH_TOKEN` is a placeholder that only the sandbox's proxy turns into a real credential, which is the case above where GitHub rejects the token on a direct connection. +Claude Code cloud sessions are one example of a sandbox where the two differ. REST there can use a different credential from `git push`, so REST can work while a push is refused. A 403 on push means that this machine's GitHub credential may not push to the repository. In a cloud sandbox that credential belongs to the sandbox's GitHub integration, and an administrator of the organization gives the integration access to the repository. The sandbox's `GH_TOKEN` is a placeholder that only the sandbox's proxy turns into a real credential, which is the case above where GitHub rejects the token on a direct connection. ## The Backend API in a cloud environment From 7c3c64d8d31e108a92a61f2aec7ec04cd5e1c739 Mon Sep 17 00:00:00 2001 From: Mike Pitre <12040919+mikepitre@users.noreply.github.com> Date: Wed, 14 Oct 2026 00:05:02 -0400 Subject: [PATCH 3/4] docs(verify): say what fixes a github-rest failure in the remote notes The allowed-domains fix applies to the two egress checks. The github-rest check's own fix is to make api.github.com reachable. Co-Authored-By: Claude Opus 5.5 --- .claude/skills/verify-clerk-expo/references/remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/skills/verify-clerk-expo/references/remote.md b/.claude/skills/verify-clerk-expo/references/remote.md index 16b7ce395ee..180b0c0043a 100644 --- a/.claude/skills/verify-clerk-expo/references/remote.md +++ b/.claude/skills/verify-clerk-expo/references/remote.md @@ -125,7 +125,7 @@ If the description has no evidence a few minutes after `down`, read the newest r Node ignores `HTTPS_PROXY` unless it is told to honor it, so the CLI decides. It goes through the proxy when the direct path to GitHub does not work, which means that it cannot connect or that GitHub rejects the machine's token on it. In that case the CLI reruns itself with the proxy in effect for itself, e2e, and agent-device. A machine that reaches GitHub directly stays direct. `doctor` prints the choice and the reason in `remote-env`. -The `tunnel-egress` and `clerk-egress` checks in `doctor` show a host that the environment blocks as `blocked:` and what answered, or as `no response from `. When `api.github.com` does not answer, the `github-rest` check shows `api.github.com did not answer`. In each case the fix is to add the host to the allowed domains of the environment that the CLI runs in. +The `tunnel-egress` and `clerk-egress` checks in `doctor` show a host that the environment blocks as `blocked:` and what answered, or as `no response from `. When `api.github.com` does not answer, the `github-rest` check shows `api.github.com did not answer`. For the two egress checks, the fix is to add the host to the allowed domains of the environment that the CLI runs in. For `github-rest`, the fix is to make `api.github.com` reachable from that environment; the check's fix line also says what GitHub access a cloud session needs. ## GitHub access From f2dd9c545d0d463647bb13e612f5d4c58dabcece Mon Sep 17 00:00:00 2001 From: Mike Pitre <12040919+mikepitre@users.noreply.github.com> Date: Wed, 14 Oct 2026 08:30:41 -0400 Subject: [PATCH 4/4] docs(verify): say what a github-rest failure can mean The check reports any failed request as api.github.com did not answer, timeouts included, so an allowed domain is the first thing to check, not the only one. Same change as clerk/clerk-ios#696. Co-Authored-By: Claude Opus 5.5 --- .claude/skills/verify-clerk-expo/references/remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/skills/verify-clerk-expo/references/remote.md b/.claude/skills/verify-clerk-expo/references/remote.md index 180b0c0043a..c2735b6ad2d 100644 --- a/.claude/skills/verify-clerk-expo/references/remote.md +++ b/.claude/skills/verify-clerk-expo/references/remote.md @@ -125,7 +125,7 @@ If the description has no evidence a few minutes after `down`, read the newest r Node ignores `HTTPS_PROXY` unless it is told to honor it, so the CLI decides. It goes through the proxy when the direct path to GitHub does not work, which means that it cannot connect or that GitHub rejects the machine's token on it. In that case the CLI reruns itself with the proxy in effect for itself, e2e, and agent-device. A machine that reaches GitHub directly stays direct. `doctor` prints the choice and the reason in `remote-env`. -The `tunnel-egress` and `clerk-egress` checks in `doctor` show a host that the environment blocks as `blocked:` and what answered, or as `no response from `. When `api.github.com` does not answer, the `github-rest` check shows `api.github.com did not answer`. For the two egress checks, the fix is to add the host to the allowed domains of the environment that the CLI runs in. For `github-rest`, the fix is to make `api.github.com` reachable from that environment; the check's fix line also says what GitHub access a cloud session needs. +The `tunnel-egress` and `clerk-egress` checks in `doctor` show a host that the environment blocks as `blocked:` and what answered, or as `no response from `. When `api.github.com` does not answer, the `github-rest` check shows `api.github.com did not answer`. For the two egress checks, the fix is to add the host to the allowed domains of the environment that the CLI runs in. For `github-rest`, which reports any failed request this way, timeouts included, first check that `api.github.com` is an allowed domain; if it already is, look at the network or the proxy; the check's fix line also says what GitHub access a cloud session needs. ## GitHub access