Skip to content

Run the test suite in parallel with pytest-xdist - #861

Merged
derek73 merged 2 commits into
masterfrom
feat/parallel-tests
Oct 24, 2026
Merged

derek73 merged 2 commits into
masterfrom
feat/parallel-tests

Conversation

@derek73

@derek73 derek73 commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • Parallel by default. -n auto --dist worksteal in pyproject's addopts. On a 12-core Mac the full suite went from 110s serial to 35s (measured 2026-10-23). worksteal because the property grids are a few long tests among 23,000 short ones. With --dist loadfile, test_properties.py alone set the wall time (66s).
  • Benchmark guards stay serial. The clock-timed scaling tests can't share the CPU: under -n auto, maiden_clause read 6.4× against its 6.0 bound on a tree that passes serially. tests/v2/test_benchmark.py is marked serial. tests/conftest.py deselects serial tests in a parallel run's workers and prints a summary line saying how to run them: uv run pytest -n0 -m serial. -n0 still runs everything in one process.
  • CI runs two passes. The parallel suite, then -n0 -m serial. The coverage job adds --cov-append to the second pass, and the ja-extra job keeps NAMEPARSER_REQUIRE_CLOCK_GUARDS on both passes.
  • Fixes a hidden order dependence that parallel runs exposed. tools/vocab/build.py read _lexicon.py from ROOT at first use. Three test_vocab.py tests repoint ROOT at a fixture tree, so each passed only after an earlier test had warmed the cache, and failed when run alone. The path is now bound at import, like VOCAB and OUTPUT. A new test fails against the old loader.
  • Docs. AGENTS.md and CONTRIBUTING.md give both commands. AGENTS.md gains a gotcha on cache-warming order dependence, and drops the stale "runs every test twice" description of the conftest fixture.

Verification

  • uv run pytest: 23,162 passed, 125 skipped, 3 xfailed in 35s.
  • uv run pytest -n0 -m serial: 39 passed.
  • uv run pytest -n0: 23,201 passed, which is the two passes combined.
  • Also clean under -n 3 --dist load and -n 11, with every test module run on its own.
  • Coverage: parallel plus the appended serial pass covers 4,625 of 4,669 lines, identical to one serial --cov run. That serial run took 4:45 under the 3.11 line tracer; the parallel one took 1:20.
  • ruff check and mypy are clean.

Not verified locally: the ja-extra job's parallel pass, since namedivider isn't installed here, and CI's 4-vCPU runners, where the gain will be smaller than on 12 cores.

🤖 Generated with Claude Code

derek73 and others added 2 commits October 23, 2026 21:38
tools/vocab/build.py read nameparser/_lexicon.py from ROOT at first use,
and three tests in tests/v2/test_vocab.py repoint ROOT at a fixture tree
holding no such file. Each passed only because an earlier test had
already filled the cache, and failed when run alone (exit 3 for 1 or 2).
The path is now bound at import, like VOCAB and OUTPUT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`-n auto --dist worksteal` in addopts: 110s serial to 35s on this
machine (12 cores; measured 2026-10-23). worksteal because the property
grids are a few long tests among 23,000 short ones.

The clock-timed scaling guards in tests/v2/test_benchmark.py cannot
share the CPU with other workers: maiden_clause read 6.4x against its
6.0 bound under -n auto. The module is marked `serial`, tests/conftest.py
deselects serial tests in a parallel run's workers and says so in the
summary, and CI runs them in a second `-n0 -m serial` pass (with
--cov-append in the coverage job). `-n0` still runs everything in one
process.

AGENTS.md and CONTRIBUTING.md give both commands; AGENTS.md also drops
the stale "runs every test twice" description of the conftest fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.05%. Comparing base (04f806d) to head (15337b9).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #861   +/-   ##
=======================================
  Coverage   99.05%   99.05%           
=======================================
  Files          47       47           
  Lines        4669     4669           
=======================================
  Hits         4625     4625           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73
derek73 merged commit 204d9f4 into master Oct 24, 2026
12 checks passed
@derek73
derek73 deleted the feat/parallel-tests branch October 24, 2026 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant