test(expo): publish for a session that started on a commit of the base branch

.changeset/expo-verify-session-start-on-base.md

@@ -0,0 +1,2 @@
+---
+---

.claude/skills/verify-clerk-expo/SKILL.md

@@ -219,15 +219,15 @@ The first `attach` of a platform adds its block after the description. A later `
 
 On a machine whose `gh` can attach, `attach` edits the description itself and prints `posted`. That needs gh 2.99.0 or newer, whose `gh pr edit` has `--attach`. It reads the description first. When the description already holds the block that this `attach` would write, it uploads nothing and prints `already posted`. When the block there is of another run or another choice of files, or there is none, it uploads the files and writes the block.
 
-`attach` always tries `gh pr edit --attach` itself first, on every kind of machine, and hands off only when that cannot work. On a machine whose `gh` cannot attach, such as a cloud sandbox, it hands the files of a remote run to the session's runner and prints `handed off`. With `--before`, the files of the before run go with them, from that run's directory in this checkout. Run it before `down`. The evidence reaches the description after `down` ends the session, when the `verify-attach` workflow publishes it, in the same tables. Its line says that the session reported the result, links the session's run, and names the account that started the session and the commit the run was made at. With `--before`, the line says that the session reports the before run, and it links that run's commit. That workflow publishes only to the open pull request of the session's branch, and only when the commit the session started on and the commit of the run are both commits of that pull request. The commit of the before run must be the commit of the run or a commit before it in its history, such as the one the branch started from. So open the pull request before `up`. A later push does not lose the evidence: the line then says that the pull request has newer commits. To show the newer commit, `run` and `attach` again, which replaces the block. `attach` checks the pull request first, and fails with the reason when the workflow would refuse it. Each platform has its own session, so `attach` the run of each platform before the `down` that ends its session.
+`attach` always tries `gh pr edit --attach` itself first, on every kind of machine, and hands off only when that cannot work. On a machine whose `gh` cannot attach, such as a cloud sandbox, it hands the files of a remote run to the session's runner and prints `handed off`. With `--before`, the files of the before run go with them, from that run's directory in this checkout. Run it before `down`. The evidence reaches the description after `down` ends the session, when the `verify-attach` workflow publishes it, in the same tables. Its line says that the session reported the result, links the session's run, and names the account that started the session and the commit the run was made at. With `--before`, the line says that the session reports the before run, and it links that run's commit. That workflow publishes only to the open pull request of the session's branch, and only when the commit of the run is a commit of that pull request. The commit the session started on must be one too, or a commit that the base branch holds, such as the one the branch came from. The commit of the before run must be the commit of the run or a commit before it in its history. A later push does not lose the evidence: the line then says that the pull request has newer commits. To show the newer commit, `run` and `attach` again, which replaces the block. `attach` checks the pull request first, and fails with the reason when the workflow would refuse it. Each platform has its own session, so `attach` the run of each platform before the `down` that ends its session.
 
 When this machine cannot attach and there is no session to hand the files to, or the hand-off fails, `attach` fails and says which. Then name the run id in the PR and say that the evidence was not attached. A run on a local device has no session, so its fix is a newer `gh`. [Remote devices](references/remote.md) has what the repository needs before a hand-off can be published, and its limits.
 
 `attach` writes only to an open pull request, and fails on one that is closed or merged. It refuses a run that is tainted, that has a failing spec or no passing one, or whose `app.log` names a user that the run did not create. A before run can have failing specs. `attach` refuses a before run that is tainted, that was never sealed, or whose `app.log` names a user that the run did not create.
 
-On a local device, work in this order: `run` the spec on the build without your change when the change is to something that exists, make the change, commit, `run` the spec, `git push`, open the pull request, `attach`, then `down`. Commit before the second `run`, so that the line `attach` writes names a commit the pull request holds. In the pull request, name the run id and the specs you ran, and say what you did not run.
+Work in this order, with one `up` for both runs: make the branch and `git push` it, `up`, `run` the spec without your change when it changes something that exists, make the change, commit, `git push`, `run` the spec again, open the pull request, `attach` with the before run as `--before <run-id>`, `down`. Open the pull request after the last run, so that it is not left open without its evidence. Commit before the second `run`, so that the line `attach` writes names a commit the pull request holds. In the pull request, name the run id and the specs you ran, and say what you did not run.
 
-On a remote session, make the before run before you push the change. A session builds the commit it was started on and later commits of the branch, and never an earlier one. When the branch has a pushed commit that does not hold the change, such as the commit that adds the spec, one session does both runs: open the pull request, `up`, `run` the spec, push the change, `run` the spec again, `attach` with `--before`, then `down`. When the change is the only commit, it takes two sessions, because the workflow publishes only for a session that started on a commit of the pull request: push the branch, `up`, `run` the spec, `down`, push the change, open the pull request, `up`, `run` the spec again, `attach` with `--before`, then `down`.
+On a local device `attach` writes the description itself, and no run needs a push. A remote session builds only the commit it was started on and later commits of the branch, so its before run comes before the push of the change. Its evidence reaches the description about a minute after `down`.
 
 Attach the run of your own change. Run your new or changed spec on its own and attach that run, so the screenshots or the video show only what the change is about. You do not owe a regression run: PR CI (`.github/workflows/verify-e2e.yml`) runs every golden spec on the pull request and reports them there. If you ran other golden specs anyway, cite that run's id in the PR and leave its video in `.verify/runs/`.
 

.claude/skills/verify-clerk-expo/references/remote.md

@@ -97,14 +97,16 @@ The workflow publishes only when all of these hold. Otherwise it prints the reas
 - The pull request that `--pr` names is open, and its branch is in this repository, not in a fork.
 - Its branch is the branch the session was started on.
 - The commit the run was made at is one of the pull request's commits.
-- The commit that branch had when `up` started the session is one of them too. GitHub lists the first 250 commits of a pull request, and the workflow reads no further. When a longer pull request has either commit past the first 250, the workflow says so and publishes nothing. Squash or rebase the branch to 250 commits or fewer, or attach from a machine whose `gh` can attach.
+- The commit that branch had when `up` started the session is one of them too, or a commit that the base branch of the pull request holds, such as the commit the branch came from. The workflow asks GitHub to compare that commit with the base of the pull request. GitHub lists the first 250 commits of a pull request, and the workflow reads no further. When a longer pull request has either commit past the first 250, the workflow says so and publishes nothing. Squash or rebase the branch to 250 commits or fewer, or attach from a machine whose `gh` can attach.
 - With `--before`, the commit of the before run is the commit of the run or a commit before it in its history. The workflow asks GitHub to compare the two commits.
 - The hand-off has at least one file.
 - The description has the two evidence comments of the platform once each, or neither.
 - A description without the comments does not end inside a code fence that is never closed.
 
 `attach` checks the same rules before it sends anything, and fails with the reason.
 
+So one session does both runs: push the branch, `up`, `run` for the before run, push the change, `run` again, open the pull request, `attach` with `--before <run-id>`, `down`. Open the pull request after the last run, so that it is not left open without its evidence, which reaches the description about a minute after `down`.
+
 A before run is the build without the change, so its commit is often the one the branch started from, which is no commit of the pull request. The workflow therefore does not look for that commit among the commits of the pull request. It asks GitHub whether that commit is the commit of the run or a commit before it in its history, and it publishes nothing when the answer is no. After a rebase that rewrote the commit of the before run, `run` the spec again on a commit that the branch still has, and pass that run as `--before`. The workflow cannot check that the files of the before run came from a build of that commit, which is why the line says that the session reports the before run.
 
 The manifest of a hand-off has a version, and the workflow publishes only the version of the script on the default branch. A branch whose verify code is older than the default branch's hands off a manifest that the workflow refuses. The refusal names both versions in the run of `verify-attach`, and nothing on the pull request shows it. Merge the default branch into the branch before you `attach`, then `down`, `up`, `run`, and `attach`. A session takes only the version of its own code. When the checkout changed version after `up`, `attach` fails, names both versions, and says to `down` and `up`.

integration/expo-native/src/core/MANIFEST

@@ -35,9 +35,9 @@ cf6aa68be73e4abaf3b63d5d84c2e9eee6197412f607ec53d0f05f7b058db420  src/core/insta
 a0ee9e4bfd34d4659685108e5e54eb1ec2aff6248236c358e6d3b14b2e2ff6a0  src/core/ledgers.ts
 afab94a970286c40d6857b03ae4fa39ec59786274beacf6f1e77f6530bf46c54  src/core/manifest.ts
 7294bbab3d892f0d9aee1e863f84e32a7f5dc3c467a7b6177be7e4ff2f29fecc  src/core/publish.ts
-76de0f98013b8a5d011dee8cf51cce63fcb3d7e68acfee51945fa5cecf2e7763  src/core/remote/backend.ts
+83a0a19bc3cf6ce0c1435f6624204a17060000064cf6ca49b8fa39143d7871b6  src/core/remote/backend.ts
 37fc42707be7d8bd5f0313a664a8465287ebb0970c520c0e0617f6ff6c9a614e  src/core/remote/github.ts
-cbd837689f2e0eff48205709a8c99508a0bcd0bea76f2647a93faa8866a76ae0  src/core/remote/handoff.ts
+d9f7e8d2f69960aba668dc2e18d6d61481c80967459922ee9fe33265b27c1735  src/core/remote/handoff.ts
 2ec436b2c53b34269e0b62ab54e41e4d9a55146cccc46f0f31591db05cdbf5c1  src/core/remote/preflight.ts
 bb69fe1f3581942285894c0901d02889c08b4293e4d9db77934fe3cdd2d4a399  src/core/remote/protocol.ts
 5a4ee0e7ed20feaeb711b1527083299b432d3fc0b7f428b636678a8a2f57581f  src/core/remote/recipe.ts

integration/expo-native/src/core/remote/backend.ts

@@ -31,6 +31,15 @@ async function pullRequestCommits(hub: GitHub, pr: number): Promise<readonly str
   return shas;
 }
 
+async function baseHolds(hub: GitHub, pr: number, base: unknown, commit: string): Promise<boolean | 'unknown'> {
+  if (typeof base !== 'string' || !/^[0-9a-f]{40}$/.test(base)) return false;
+  const compared = await hub.api('GET', `/compare/${commit}...${base}?per_page=1`);
+  if (compared.status === 404) return 'unknown';
+  if (compared.status !== 200) throw new VerifyFailure('NOT_READY', `could not compare the commit the session started on with the base branch of PR #${pr}: ${apiMessage(compared)}`, 'retry {cli} attach');
+  const status = (compared.json as { readonly status?: unknown } | null)?.status;
+  return status === 'ahead' || status === 'identical';
+}
+
 export function remoteBackend(settings: RemoteSettings, deps: RemoteDeps = { env: process.env, runner: run }): DeviceBackend<RemoteLease> {
   const { platform } = settings;
   let opened: Promise<GitHub> | undefined;
@@ -245,10 +254,11 @@ export function remoteBackend(settings: RemoteSettings, deps: RemoteDeps = { env
       if (session.status !== 200) throw new VerifyFailure('NOT_READY', `could not read run ${lease.providerRef}: ${apiMessage(session)}`, `open ${runUrl(lease.providerRef)}`);
       if (pull.status !== 200) throw new VerifyFailure('NOT_READY', `could not read PR #${bundle.pr} of ${settings.repo}: ${apiMessage(pull)}`, 'pass the number of an open pull request of this branch as --pr');
       const started = session.json as { readonly head_branch: string; readonly head_sha: string };
-      const pr = pull.json as { readonly state: string; readonly body: string | null; readonly commits: number; readonly head: { readonly ref: string; readonly sha: string; readonly repo: { readonly full_name: string } | null } };
-      const mismatch = pullRequestMismatch(
+      const pr = pull.json as { readonly state: string; readonly body: string | null; readonly commits: number; readonly base?: { readonly sha?: unknown }; readonly head: { readonly ref: string; readonly sha: string; readonly repo: { readonly full_name: string } | null } };
+      const mismatch = await pullRequestMismatch(
         { state: pr.state, headRepo: pr.head.repo?.full_name ?? null, headRef: pr.head.ref, commits: await pullRequestCommits(hub, bundle.pr), commitCount: pr.commits },
         { repo: settings.repo, headBranch: started.head_branch, startedOn: started.head_sha, runCommit: bundle.summary.commit },
+        () => baseHolds(hub, bundle.pr, pr.base?.sha, started.head_sha),
       );
       if (mismatch !== null) throw wouldNot(mismatch.why, mismatch.fix);
       if (bundle.before !== null) {

integration/expo-native/src/core/remote/handoff.ts

@@ -187,22 +187,32 @@ export interface PullRequestMismatch {
 }
 
 const ITS_OWN_PULL_REQUEST =
-  "it accepts only the open pull request of the session's branch, and only when the commit the session started on and the commit of the run are both commits of that pull request; git push, then {cli} run again and attach, or after a rewrite of the branch's history {cli} down and {cli} up first";
+  "it accepts only the open pull request of the session's branch, and only when the commit of the run is a commit of that pull request and the commit the session started on is one too or is a commit of its base branch; git push, then {cli} run again and attach, or after a rewrite of the branch's history {cli} down and {cli} up first";
+const A_COMMIT_GITHUB_FINDS = '{cli} down, then {cli} up on a commit of the pull request or of its base branch, run again and attach';
 const FEWER_COMMITS = `it can check a commit only against the first ${MOST_COMMITS_GITHUB_LISTS} commits of a pull request; squash or rebase the branch to ${MOST_COMMITS_GITHUB_LISTS} commits or fewer, git push, then {cli} down, {cli} up, run again and attach, or run {cli} attach on a machine whose gh can attach`;
 
-export function pullRequestMismatch(pull: PullRequestView, session: { readonly repo: string; readonly headBranch: string; readonly startedOn: string; readonly runCommit: string }): PullRequestMismatch | null {
+export async function pullRequestMismatch(
+  pull: PullRequestView,
+  session: { readonly repo: string; readonly headBranch: string; readonly startedOn: string; readonly runCommit: string },
+  baseHoldsStart: () => Promise<boolean | 'unknown'>,
+): Promise<PullRequestMismatch | null> {
   const refused = (why: string): PullRequestMismatch => ({ why, fix: ITS_OWN_PULL_REQUEST });
   if (pull.state !== 'open') return refused(`it is ${pull.state}`);
   if (pull.headRepo !== session.repo) return refused(`its branch is in ${pull.headRepo ?? 'a deleted fork'}, not in ${session.repo}`);
   if (pull.headRef !== session.headBranch) return refused(`its branch is ${pull.headRef} and the session runs on ${session.headBranch}`);
-  for (const [what, commit] of [['the session started on', session.startedOn], ['the run was made at', session.runCommit]] as const) {
-    if (pull.commits.includes(commit)) continue;
-    if (pull.commitCount > MOST_COMMITS_GITHUB_LISTS) {
-      return { why: `it has ${pull.commitCount} commits, GitHub lists only the first ${MOST_COMMITS_GITHUB_LISTS}, and ${what} ${commit.slice(0, 12)}, which is not among those`, fix: FEWER_COMMITS };
-    }
-    return refused(`${what} ${commit.slice(0, 12)}, which is not one of its commits`);
+  const startHeld = pull.commits.includes(session.startedOn) || (await baseHoldsStart());
+  if (startHeld === 'unknown') return { why: `the session started on ${session.startedOn.slice(0, 12)}, which GitHub does not find in ${session.repo}`, fix: A_COMMIT_GITHUB_FINDS };
+  const notHeld =
+    !startHeld
+      ? { commit: `the session started on ${session.startedOn.slice(0, 12)}`, among: 'one of its commits or a commit of its base branch' }
+      : !pull.commits.includes(session.runCommit)
+        ? { commit: `the run was made at ${session.runCommit.slice(0, 12)}`, among: 'one of its commits' }
+        : null;
+  if (notHeld === null) return null;
+  if (pull.commitCount > MOST_COMMITS_GITHUB_LISTS) {
+    return { why: `it has ${pull.commitCount} commits, GitHub lists only the first ${MOST_COMMITS_GITHUB_LISTS}, and ${notHeld.commit}, which is not among those`, fix: FEWER_COMMITS };
   }
-  return null;
+  return refused(`${notHeld.commit}, which is not ${notHeld.among}`);
 }
 
 const sha256File = (path: string): string => createHash('sha256').update(readFileSync(path)).digest('hex');