Skip to content

Adapt to the @rstest/core 1.0 reporter contract: status literals, payload fields, onTestRunStart payload #115

Closed

Description

@fi3ework

Background

web-infra-dev/rstest#1875 (draft, feat/reporter-contract) stabilizes the reporter contract for @rstest/core 1.0. It is a deliberate pre-1.0 break, and this extension resolves @rstest/core from the user's project, so a project that upgrades to core 1.0 while the extension stays on today's code degrades silently: SUPPORT_MATRIX['@rstest/core'] = '>=0.12.0' is satisfied by 1.0.0, no version-mismatch status fires, and the status switches below simply stop matching.

Filed from the rstest PR side, per its AGENTS rule that a change touching this API surface must be linked in both repos in the same PR cycle.

Upstream changes (verified against feat/reporter-contract @ dd061a1d)

  • TestResultStatus: pass | fail | skip | todo → passed | failed | skipped | todo.
  • API TestRunResult['status'] (now the shared TestRunStatus): pass | fail | error → passed | failed | error. The value is computed by the host and carried on TestRunEndPayload; api/result.ts no longer derives it.
  • All reporter hooks return MaybePromise<void>; events fire in arrival order and the host joins pending hooks before onTestFileResult (per file) and onTestRunEnd (whole run).
  • onTestRunStart(payload) now receives { files: { testPath, project, testId }[] }. Implementations that take no argument stay assignable.
  • Required payload additions: TestCaseInfo / TestSuiteInfo / TestResult gain fullName and relativeTestPath; TestFileInfo gains relativeTestPath; TestFileResult gains relativeTestPath and a required summary (TestRunSummary.tests also gains flaky).
  • parseErrorStacktrace in packages/core/src/utils/error.ts changed: stackIgnores is now RegExp[] (/(^|\/)node:/, /<anonymous>/), file:// paths are unwrapped, sourcemapped frames keep the mapped methodName, and the post-sourcemap filter now always runs (a custom ignore list no longer disables it).
  • CLI flags are unchanged; the blob format version is bumped (the editor does not use the blob reporter or merge-reports — verified).
  • The main @rstest/core entry re-exports the new public types (TestRunStatus, TestRunStartPayload, TestRunEndPayload, TestRunSummary, Duration, MaybePromise, …).

Impact in this repo

  1. Status switches silently no-op against 1.0:
    • packages/vscode/src/stacks/test/testRunReporter.ts — onTestFileResult and onTestCaseResult switch on pass / fail / skip. With 1.0 values no case matches, so file and case items are never marked passed/failed/skipped and errors never reach the run.
    • packages/vscode/src/stacks/test/worker/index.ts — result.status === 'fail' guard for "failed without test-level failures" stops firing, so an operation-level failure (coverage threshold, report error) can be swallowed.
  2. Old literals in unit tests: packages/vscode/tests/stacks/test/master.test.ts fake run() results ('pass' / 'fail', several sites) and the 0.12.0 default in its writeCoreInstall helper; packages/vscode/tests/versionCheck.test.ts pins the matrix strings.
  3. Vendored core internals need a re-sync: packages/vscode/src/stacks/test/vendored/coreInternals.ts. Its header says the stack-frame filtering is kept byte-identical because it decides the editor's failure location, so the 1.0 error.ts changes belong in the next sync. (The editor calls parseErrorStacktrace({ stack }) without getSourcemap; the file:// unwrap, the regex set and the unconditional post-filter still apply.)
  4. Support matrix / dependency wiring:
    • packages/vscode/src/shared/versionCheck.ts: SUPPORT_MATRIX['@rstest/core'] = '>=0.12.0'. Semver-wise 1.0 passes, so nothing gates the break.
    • packages/vscode/package.json devDependency @rstest/core: ^0.12.4; the E2E fixture roots pin 0.12.4 in four package.json files under packages/vscode/e2e/.
    • packages/vscode/AGENTS.md → stacks/test "Tracked upstream state" points at rstest packages/vscode e1607861; that reference should move to the commit that ships 1.0 support.
    • If the floor is raised, SUPPORT_MATRIX.rstack = '>=0.7.6' moves with it: its rationale is "the first rstack release pinning @rstest/core ~0.12.0", so the new floor needs the first rstack release pinning 1.0 (cross-repo, rstack-cli).

Compatibility decision needed

  • A. Keep >=0.12.0 and normalize at the boundary. Map legacy literals (pass/fail/skip → passed/failed/skipped) in one reporter-side normalizer; 1.0 is otherwise additive (new required fields, new hooks signature, new run-start payload), so the rest is forward-compatible. Bump the devDependency to 1.x to compile against the real types, and keep the normalizer total for unknown values so a future literal change cannot silently half-work. 0.12 projects keep working after the extension auto-updates; no rstack-cli release needed for the floor.
  • B. Raise the floor to >=1.0.0. Update the matrix, its tests, the devDependency, the fixtures and the rstack floor (needs an rstack release pinning 1.0). Cleaner, but projects still on 0.12 get an unsupported-version status, and the frozen standalone extension (^0.12.0 only, see Rstest extension sunset checklist #75) is no longer a fallback.

Recommendation: A. The extension updates itself, @rstest/core is project-resolved, and 0.12 projects will exist for a long time after 1.0 lands.

Proposed work (option A)

  1. Add one normalizer (e.g. stacks/test/shared/status.ts or beside TestRunReporter) for test-result and run-result statuses; use it in both reporter switches and in the worker's run()-result guard.
  2. Bump @rstest/core devDependency to the 1.0 line once published so type-checking sees the real literals; before that, test against a local build via rstack.rstest.corePath.
  3. Re-sync vendored/coreInternals.ts against the 1.0 error.ts.
  4. Update unit tests and fakes to the new literals and required payload fields, keeping legacy literals covered through the normalizer.
  5. Bump E2E fixture pins; point at least one fixture at a 1.0 core so both lines are exercised (or widen the Renovate range and rely on the rstack toolchain group).
  6. Update the AGENTS.md tracked-upstream note and the support-matrix comments/tests.
  7. Verify: pnpm lint && pnpm test:unit, then pnpm test:e2e:rstest. rstackjs/rstack-ecosystem-ci#99 builds @rstest/core from rstest main and runs this repo's test:unit + lint, so opening the adaptation PR in the same cycle as #1875 gives the cross-repo signal before the release.

Out of scope (checked)

  • Blob reporter / merge-reports format: not used here.
  • reporters accepting module specifiers: the editor passes reporter instances, unaffected.
  • listTests(): unchanged for callers (the collector's raw types changed internally).

References

Activity

  1. fi3ework commented on Nov 5, 2026

    @fi3ework
    MemberAuthor

    Closing: rstest 0.13 compatibility landed in #127.

    Verified by rstack-ecosystem-ci run 41989912558: @rstest/core built from web-infra-dev/rstest 74810e20 (includes web-infra-dev/rstest#1875) against this repo at 7159835 — install, build, test:unit, lint (type-check) and the rstest E2E slice (including the bridged rstack fixture) all pass. The other 0.13 contract changes (onTestRunStart payload, MaybePromise hooks, added payload fields, TestRunEndPayload.status) need no code change here.

    Left for routine follow-up once 0.13.0 is on npm (not blocking compatibility):

    • Renovate bump of the @rstest/core devDependency and E2E fixture pins.
    • Update the stacks/test tracked-upstream SHA in packages/vscode/AGENTS.md.
    • Optional re-sync of parseErrorStacktrace in vendored/coreInternals.ts (vendored, so independent of the user's core version).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions