BLD/DEV: use meson-python, add a basic Pixi workspace

[lucascolley]

If I'm to be the one to tackle something like data-apis/array-api-extra#583 for this repo, this PR will help keep me sane :)

Clearly plenty more work to do to get this up to par with array-api-extra, but I think that merging something like this would be a good start.

[lucascolley]

dynamic versioning does not play well with pixi-build-python right now. I have no experience with setuptools-scm so not sure if this is acceptable

[ev-br]

Not sure TBH. One one hand, the version is manually set in __init__, so it looks like we're not using dynamic versioning. Or maybe this "dynamic" setting in pyproject.toml makes it pick the version from __init__.
There's also a bit of magic around trusted publishing and I've no idea how this works (other than push a tag and it just works).

I suppose the way to check it is make a change in a branch, then try making a release to a testpypi (requires a manual editing of the relevant GH workflow) and see if this still works.

My personal prio for now is to get ready for 2025.12 spec release. That, among other things, requires a lot of manual testing locally, of matching branches of multiple projects. My workflow involves a lot of

$ pushd ../array-api-tests
$ git co matching_branch
$ run tests
# swear quietly
$ popd 
# fix
$ pip install -e .
$ pushd ../array-api-tests
# run tests again
# swear loudly
# fix it in -tests
# repeat until convergence

not sure how that translates to the pixified workflow (swearing is optional), and I'd prefer to do a single thing at a time.

[lucascolley]

yeah, this could be a whole lot nicer with Pixi but it will need some hours of setup. I'll let you swear loudly for now :)

[ev-br]

I'm pretty sure swearing is invariant w.r.t. tooling. Am open to an experimental test of this hypothesis after 1.14 ships. Don't know which part of flipping between branches and projects can be streamlined, but am always open to pleasant surprises.

[rgommers]

Using an explicit version number is fine. Opinions obviously vary, but I'd consider it an improvement.

Versioning is unrelated to trusted publising.

[ev-br]

Let's maybe revive this if you're interested @lucascolley ? Sorry it took me long to get back to this.

I trust you on anything pixi. If dynamic versioning gets in the way, I'd be happy to see the back of it, static is indeed much nicer.

What needs to be done if we change the versioning:

• purge any mentions of setuptools_scm and tool.setuptools.dynamic from pyproject.toml
• update the releasing docs to mention that the version needs to be bumped in both __init__.py and pyproject.toml it currently says "update the version in __init__.py"
• cut a dummy release locally (python -m build . && twine check --strict dist/
• make a dummy TestPyPI release to verify it all works

[lucascolley]

❯ pixi exec -w python-build python -m build .
...
❯ pixi exec twine check --strict dist/*.tar.gz dist/*.whl
Checking dist/array_api_compat-1.13.0-py3-none-any.whl: PASSED
Checking dist/array_api_compat-1.13.0.tar.gz: PASSED

• make a dummy TestPyPI release to verify it all works

do you want me to do this?

[lucascolley]

I implemented the switch from setuptools to meson-python

[ev-br]

Agreed that the change to meson-python is nice.

I actually wonder about vendoring scenarios. When e.g. SciPy vendors array-api-compat, it has to list the sources in the meson.build file. Now that array-api-compat has its meson.build file with the listing, can scipy reuse it? This is not blocking this PR of course, am just thinking out loud.

make a dummy TestPyPI release to verify it all works

Thinking about it, to test trusted publishing it needs to be done from main/tag etc. Presumably trusted publishing is decoupled enough, so that it's more work than justified now. So checking that a release wheel looks reasonable locally is probably sufficient.

[lucascolley]

Yes, this is the exact context within which I started this work: https://github.com/scipy/scipy/blob/main/meson.build#L216-L246

[ev-br]

Tried test-driving it with my usual workflow:

$ pip install -e . --no-build-isolation
$ pushd ../array-api-tests
$ python
>>> from array_api_compat import torch as xp
...
ModuleNotFoundError: No module named 'array_api_compat.common'

In fact, non-editable installs fail in the same way as well. What gives?

[lucascolley]

Hmm the problem is reproducible even in the default Pixi environment via pixi run python, I'm baffled at the minute.

[lucascolley]

The problem was that I had a dodgy workaround for the symlink. Hopefully now this works.

[lucascolley]

@rgommers can you see what I'm doing wrong with the symlink here? Logs like https://github.com/data-apis/array-api-compat/actions/runs/23195083314/job/67401163424?pr=373 seem to show that it is broken.

I've tried poking around with meson introspect but I can't figure it out.

[rgommers]

Symlinks aren't supported in wheels, so this looks unfamiliar and is probably not what you want.

It is possible to have a symlink in the source tree, but it gets replaced by a copy in an sdist/wheel.

[lucascolley]

9120e06 (this PR) gets rid of the symlink. However, this does not seem to work for editable builds. Is there something in Meson's Python module that can mimic configure_file(copy: true) like this?

For context, it seems like pointing to the same file twice with py.install_sources means that it is only installed once.

[lucascolley]

despite my sunk-cost into making this work for wheels only, we should probably just give up on the symlink and use a Pixi task to populate the directory at test-time, like https://github.com/data-apis/array-api-extra/blob/1c5bb9d65f75132196650de2ca966e2fc36ac15c/pyproject.toml#L146. I think we are pushing meson-python slightly past its limits by trying to mimic the symlink.

I had hoped that we could merge this without me having to convert CI over to using Pixi yet, but maybe not...