Skip to content

Nothing automated runs the spec 250 acceptance suites: 40 e2e tests and the vendored contract's source both live outside CI #269

Description

@pseudoseed

The shape

Same as #265, one level up. #265 is that root npm test filters to @cluesmith/codev, so nothing
local runs the frozen apps/client suite. This is the same sentence about spec 250's own evidence.

Two things, both outside any automated run

1. The 40 spec-250 e2e tests run nowhere automated. 32 Playwright tests across
packages/codev/src/__tests__/e2e/spec-250-{hierarchy,gate,tiling,approval}.spec.ts, plus 8 vitest
e2e in spec-250-t3code-approval.e2e.test.ts. They need a running fork server, so they sit behind
playwright.spec250.config.ts and a T3_NODE fixture and are excluded from the ordinary suite.

Every one of criteria 1, 2, 3, 4, 5, 5b and 7 rests on them. They are the only thing that
demonstrates the feature works, and they execute exactly when someone remembers to run them by hand.

The failure mode is not theoretical: with T3_NODE unset the Playwright suite reports 32 skipped
and exits 0
— the fixture refusing to start the fork server, not a pass. A CI job that ran them
without that variable would be green and prove nothing.

2. The vendored contract's source is a private repository. packages/types/src/t3/pin.json
carries contractSource: "fork" and forkRepo: https://github.com/pseudoseed/t3code. Anything that
regenerates or verifies the contract needs a checkout of a private repo at a pinned commit, so
generate.mjs --check and t3-server.mjs verify cannot run in a CI job that has only this
repository. Today they run on one laptop.

Why it matters more than an ordinary coverage gap

The whole spec is an argument that a private fork is maintainable — the rebase drill, the pin, the
refresh cycle. That argument is only true while somebody is checking, and nothing checks
automatically. A drift in the fork, or an upstream change that breaks regeneration, surfaces the
next time a human happens to run the drill.

Not a small fix, which is why it is filed rather than done

It needs at least: a CI runner with credentials for pseudoseed/t3code, a job that starts the fork
server, and a guard that makes "the fork stack was unavailable" a failure in CI while it stays a
skip locally. That last part is the interesting half — the current skip-not-pass behaviour is right
for a laptop and wrong for CI, and they need telling apart rather than one of them changed.

Found by

Spec 250, review phase — raised by the claude lane's PR review of #266. Ruled out of scope for that
PR by the architect.

Refs #265

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions