TST: Pin test dependencies to exact versions#2230
Conversation
…toring Pin all unpinned test dependency specifiers in [dependency-groups] to exact versions (==) across cuda_bindings, cuda_core, and cuda_pathfinder. Also add pip ecosystem entries to dependabot.yml so PRs are opened when newer versions become available on PyPI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mdboom
added
test
github-actions
Bot
added
CI/CD
| "pytest-repeat", | ||
| "pytest-randomly", | ||
| "pyglet>=2.1.9", | ||
| "matplotlib==3.10.9; python_version < '3.15'", |
There was a problem hiding this comment.
WDYT about e.g.
"matplotlib>=3.5.0,<=3.10.9; python_version < '3.15'",
so that we don't lock in other packages to one very specific version? — I realize it's only for testing, but I wonder if e.g. numba or rapids testing could be affected by our choices here.
There was a problem hiding this comment.
Why would they test in the same venv as ours? If they wanted to test against a prerelease of cuda_bindings, only the library dependencies would apply, not the test dependencies.
There was a problem hiding this comment.
I think the only concern here is that /this/ list is self-consistent, for example, the numpy and matplotlib versions specified here are compatible (which they are).
There was a problem hiding this comment.
Why would the test in the same venv as ours?
I think I've seen that (SWQA testing), although digging up what I have in mind isn't easy.
I'm OK to be simple/strict first and see what happens.
|
rwgk
left a comment
rwgk
left a comment
There was a problem hiding this comment.
I think we also need to update the pixi.toml files as part of this PR, or at least explicitly call them out as follow-up work.
Right now this PR freezes the test dependencies in the package pyproject.toml dependency groups, but the pixi environments still have broad specs for the same test tools. That means some CI/developer test paths can still float even after this PR lands.
The main ones I noticed are:
cuda_bindings/pixi.tomlcuda_core/pixi.tomlcuda_pathfinder/pixi.toml
If the goal is to stop unexpected pytest-side dependency changes from putting us into fire-fighting mode, the pixi-managed test environments should follow the same pinning policy as the pyproject.toml test groups, or we should document why they intentionally remain floating.
| "matplotlib; python_version < '3.15'", | ||
| "numpy", |
There was a problem hiding this comment.
Could we keep the previous lower bounds here rather than making these bare dependencies?
What I would do:
- Restore the lower bounds, because they encode the minimum supported/tested floor.
- Add upper bounds to prevent surprise future releases from breaking CI.
- Split NumPy by Python version if one cap cannot span the full Python matrix.
- Add a short comment for any Python-version split so it is clear this is compatibility-driven, not arbitrary.
Policy-wise, I think lower bounds and upper bounds serve different purposes: lower bound = minimum supported/tested floor; upper bound or exact pin = CI stability cap. I do not think we should lose the former while adding the latter.
For example, roughly this shape, informed by the CI/branch history so far:
"matplotlib>=3.5.0,<=3.10.9; python_version < '3.15'",
"numpy>=1.21.1,<=2.2.6; python_version < '3.11'",
"numpy>=1.21.1,<=2.4.6; python_version >= '3.11'",
There was a problem hiding this comment.
In general, this makes sense. There is no need for the two numpy entries, since the lack of availability of anything greater than 2.2.6 on Python 3.10 will automatically enforce that first rule.
One of the reasons I'm sort of anti-pixi is that it uses the conda (forge) packaging ecosystem which lags behind PyPI. It's not actually possible to pin these to the same versions right now, for example, because many of them have yet to be released. This creates a whole other set of fire-fighting surprises. We should either (a) use pixi everywhere, including in CI or (b) move to PyPI-based packaging everywhere and use This is a separate issue from needing to support both ecosystems for our users -- we do need to test on both there. |
mdboom
changed the title
2 participants
This is in response to #2227 causing an unwelcome surprise when pytest was updated. It was fine at the time, but this kind of thing gets really annoying during release time.
There is no reason not to pin our test dependencies, and let dependabot handle updating them so we can deal with any breakages when we want to, not to urgently unblock CI.
Summary
[dependency-groups]specifiers to exact==versions incuda_bindings,cuda_core, andcuda_pathfinderso CI runs against a known-good set of packagespipecosystem entries to.github/dependabot.yml(one per subproject) so Dependabot opens PRs when newer PyPI versions are availableNotes
cythonandsetuptoolsare intentionally left as range specifiers (>=3.2,<3.3and>=80) since they are build-time dependencies with looser compatibility constraintsml-dtypesincuda_coreis left as>=0.5.4,<0.6.0for the same reasonTest plan
🤖 Generated with Claude Code