Skip to content

Make pnpm test:node reliable under whole-corpus concurrency #761

Description

@taras

Story

As a contributor, I want pnpm test:node to complete the entire Node test
corpus reliably, so the documented local suite does not fail because unrelated
test files compete with a long-running executable-document test.

Current gap

On the clean #748 implementation revision, the unsharded command failed twice:
packages/cli/tests/plan.test.ts exceeded its inner 180-second document timeout
after about 330 seconds and 250 seconds of suite execution. The same file passed
in CI's shard shape, alone in its own process with
--test-concurrency=1, in 125.7 seconds.

The observed distinction is load, not yet a settled cause. The corpus now also
contains a rule test that starts Oxlint subprocesses, but no controlled
base-versus-head comparison establishes that this addition alone crossed the
threshold. CI remains green because its seven Node shards run files serially in
separate processes; the broken surface is the documented unsharded local
command.

Contract

Measure the whole-corpus and isolated shapes on one fixed revision, then make
the unsharded runner keep enough execution budget for document tests to exercise
their own timeout semantics. The correction may bound file concurrency, isolate
subprocess-heavy tests, or adjust a timeout only when the measurements show that
the timeout no longer represents the behavior it was written to test.

Keep runtime discovery and CI shard semantics intact. Every applicable file
still runs exactly once, a numeric failure does not hide later files, and full
failure output remains visible. Do not make the suite pass by excluding tests or
narrowing their behavior.

Acceptance

  • pnpm test:node completes successfully on at least three consecutive runs of
    one fixed revision on the representative local host.
  • packages/cli/tests/plan.test.ts still exercises its 180-second document
    timeout behavior rather than receiving an arbitrary blanket extension.
  • The seven Node shard selections still run each assigned file serially in its
    own process and cover the complete applicable corpus once.
  • A focused regression or controlled stress comparison distinguishes the chosen
    correction from the prior load-sensitive behavior.

Out of scope

  • Changing Deno or Bun suite execution without evidence of the same problem.
  • Increasing CI shard counts; those follow the separate measured five-run rule.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions