Skip to content

test(global): a test run owns a temporary folder of its own, which its teardown checks and removes - #284

Merged
rami-hatoum merged 4 commits into
mainfrom
test/a-test-run-owns-its-temporary-folder
Oct 22, 2026
Merged

rami-hatoum merged 4 commits into
mainfrom
test/a-test-run-owns-its-temporary-folder

Conversation

@rami-hatoum

Copy link
Copy Markdown
Contributor

What this changes

The suites leaked. On a machine that runs them, the temporary directory held 18 GB of folders the tests had made and never removed, thousands under each package's prefix, and the server suite left one ledger folder per run and, under load, a background runtime still running. Every helper removes what it makes on onTestFinished or in a finally, which a worker that dies, a hook that times out or a child that is killed never reaches, and nothing ever looked.

Now each test run owns a temporary folder of its own.

  • A shared global setup, in vitest.shared.ts so every package has it, makes one folder under the machine's temporary directory, named for the run, and sets TMPDIR to it before any worker starts, so os.tmpdir() in every worker and in every process a test spawns answers that folder. The helpers needed no change for this.
  • Its teardown always removes the folder whole, so a run can no longer leave anything behind, and before it does it looks: an entry left there is a test that did not clean up, and the run fails naming each one, so a leak is a red gate rather than a surprise on someone's disk. Two runs on one machine have two folders and never name each other's files.
  • Under the server, the teardown also reads each left home's runtime.log for a runtime started in the background with no ending, ends it only while ps shows the instance the log named, SIGTERM then SIGKILL, and names it in the failure. The per-test hook that did this reads the log through the same function.

The gate then showed three leaks, each fixed at its root:

  • The identity package's key test that reads an env file never removed the folder it wrote the file in.
  • The server's computation test over MCP started a server twice and stopped only the second, which left one ledger folder per run.
  • A model process of a prediction function wrote the model library's telemetry files into its temporary folder, which the fix that turns that telemetry off removes; this change builds on it.

turbo.json lists the setup's files under globalDependencies, so a change to them invalidates every package's cached test results, including those of packages that do not depend on operations.

Tests

  • The folder is made where the run is set up, named for the run, is the temporary folder of the run and of a process it starts, and is gone after the teardown.
  • A teardown that finds entries fails naming them in order and removes the folder all the same; two runs set up at once name only their own.
  • A process still running in a home a test left is ended as the teardown looks, before the folder is removed, and named with what it ran in; the server's reader finds a started runtime with no ending in a home's log and nothing in a home whose runtime ended.
  • The four server shards and their merge, three concurrent runs of one suite and a run of three projects in one process all leave no folder behind.

Checklist

  • The PR title is a conventional commit with a workspace scope, such as feat(reasoning): .... It becomes the squash commit and the changelog entry.
  • pnpm check passes locally, with every file at 100% coverage.
  • I've signed the Contributor License Agreement (the CLA bot asks on your first PR).

rami-hatoum and others added 4 commits October 22, 2026 08:30
…r it wrote the file in

The test of an unquoted compact JSON line in an env file made an api-keys- folder
under the temporary directory and never removed it, so each run of the suite left
one there. It now removes it once the test finished.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ts, so none leaves a ledger

Its tests kept the server of onAlpha in one variable that afterEach stopped, and the test of
the definition tools called onAlpha twice, so its first server ran on unstopped and left its
auto-runtime-server- folder, the ledger with its 1.1 MB WAL, under the temporary directory
on every run of the suite. onAlpha now stops the server it starts once the test finished.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s teardown checks and removes

The helpers that make a folder remove it on onTestFinished or in a finally, which a worker that
dies, a hook that times out or a child that is killed never reaches, so the suites left their
folders under the machine's temporary directory. vitest.shared.ts now names a global setup,
aFolderOfTheTestRun of @on.auto/operations, that makes auto-runtime-test-run-<project>-<random>
in the temporary folder and sets TMPDIR to it before any worker starts, so tmpdir() answers it
in every worker and in every process a test starts with the environment it inherits. Its
teardown lists what the tests left there, ends any process the setup's runningIn finds still
running in an entry, fails the run naming each entry and each process, and removes the folder
whatever happened. Two runs at once have two folders.

The ending of a process, SIGTERM and SIGKILL after stoppedWithinMs, moves from test-processes.ts
to process-endings.ts, which imports nothing of Vitest, since a global setup runs in Vitest's own
process. turbo.json counts both files among its global dependencies, as it counts
vitest.shared.ts, since the tests of every package run under them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eft running, and names it

A runtime a relay starts with start --background --instance <id> runs in a process group of its
own, and runtimesStartedInTheBackgroundEndedWithTheTest ends it in onTestFinished, which a worker
that dies or a hook that times out never reaches, so under load such a runtime ran on after the
run. The server's vitest.config.ts now names src/testing/test-run/test-run-folder.ts in place of
the shared global setup, through sharedConfigSetUpBy of vitest.shared.ts: aFolderOfTheTestRun
over runtimesStillRunningInTheBackgroundIn, which reads the runtime.log of each home left in the
run's folder for a start with no end, as the hook of a test does. A process is ended, SIGTERM and
SIGKILL after stoppedWithinMs, only while ps shows it runs under the instance the log named, and
the failure of the run names it with its home.

The reading of the log moves to runtimes-in-the-background.ts, which imports nothing of Vitest,
and the hook of a test ends what it finds with endedWhileItRuns of @on.auto/operations/testing.
@on.auto/operations/test-run-folder is the entry a package's own global setup imports. The check
that a process the log says ended, or one that runs as another, is left alone is the teardown's
test now, beside the reading the hook and the teardown share, and the relay's test keeps to the
relay.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@rami-hatoum
rami-hatoum force-pushed the test/a-test-run-owns-its-temporary-folder branch from 9b3d71e to 9206dc3 Compare October 22, 2026 07:36
@rami-hatoum
rami-hatoum merged commit 00e5cc7 into main Oct 22, 2026
20 checks passed
@rami-hatoum
rami-hatoum deleted the test/a-test-run-owns-its-temporary-folder branch October 22, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant