Skip to content

Upgrade cibuildwheel to v4.3.3 and test wheels on Python 3.15 - #147

Open
mbUSC wants to merge 4 commits into
singlestore-labs:mainfrom
mbUSC:cibuildwheel-v4
Open

mbUSC wants to merge 4 commits into
singlestore-labs:mainfrom
mbUSC:cibuildwheel-v4

Conversation

@mbUSC

@mbUSC mbUSC commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

cibuildwheel v2.16.5 predates Python 3.15 and can't build or test on it. Upgrade to v4.3.3 and add cp315 to CIBW_BUILD. The wheel is abi3, so it is still built only once, on cp39; cibuildwheel reuses it for cp315 and only runs the tests there.

Side effects of going from v2 to v4, from cibuildwheel's changelog:

  • Linux wheels are built on manylinux_2_28 instead of manylinux2014, so they need glibc 2.28 or newer. CentOS 7 and Amazon Linux 2 users would fall back to the sdist or the pure-Python wheel. Setting CIBW_MANYLINUX_{X86_64,AARCH64}_IMAGE to manylinux2014 keeps the old baseline.
  • Each abi3 wheel is checked with abi3audit after repair, and a failure fails the build.
  • Windows wheels are repaired with delvewheel, which bundles any DLLs the extension links against.
  • Wheels are built with build instead of pip wheel, in environments without setuptools and wheel preinstalled. Both come from pyproject.toml's build-system.requires instead.

Note

Medium Risk
Changes affect how all release wheels are built and which Linux glibc versions they support; a misconfigured manylinux pin or failed cp315 tests would block publishes.

Overview
Publish workflow bumps pypa/cibuildwheel from v2.16.5 to v4.3.3 so release builds can exercise Python 3.15. CIBW_BUILD is now cp39-* cp315-*: the abi3 wheel is still compiled once on 3.9, while 3.15 reuses that wheel and reruns the live-cluster pytest suite on the newest supported interpreter.

Because cibuildwheel v3+ defaults to manylinux_2_28, the workflow pins CIBW_MANYLINUX_X86_64_IMAGE and CIBW_MANYLINUX_AARCH64_IMAGE to manylinux2014 so Linux wheels keep the older glibc 2.17 baseline and older distros still install the C extension instead of silently falling back to pure Python.

accel.c marks the local UTF-8 helper _PyUnicode_AsUTF8 as static, limiting its linkage scope (relevant under stricter abi3 wheel checks in newer cibuildwheel).

Reviewed by Cursor Bugbot for commit 2876d2f. Bugbot is set up for automated code reviews on this repo. Configure here.

cibuildwheel v2.16.5 predates Python 3.15 and can't build or test on
it. Upgrade to v4.3.3 and add cp315 to CIBW_BUILD. The wheel is abi3,
so it is still built only once, on cp39; cibuildwheel reuses it for
cp315 and only runs the tests there.

Side effects of going from v2 to v4, from cibuildwheel's changelog:
- Linux wheels are built on manylinux_2_28 instead of manylinux2014,
  so they need glibc 2.28 or newer. CentOS 7 and Amazon Linux 2 users
  would fall back to the sdist or the pure-Python wheel. Setting
  CIBW_MANYLINUX_{X86_64,AARCH64}_IMAGE to manylinux2014 keeps the old
  baseline.
- Each abi3 wheel is checked with abi3audit after repair, and a
  failure fails the build.
- Windows wheels are repaired with delvewheel, which bundles any DLLs
  the extension links against.
- Wheels are built with `build` instead of `pip wheel`, in environments
  without setuptools and wheel preinstalled. Both come from
  pyproject.toml's build-system.requires instead.
@mbUSC mbUSC mentioned this pull request Oct 23, 2026
mbUSC pushed a commit to mbUSC/singlestoredb-python that referenced this pull request Oct 28, 2026
Python 3.15 removed zipimport.zipimporter.load_module(), which the
external function server used to load a zipped environment file. Load
it with find_spec()/exec_module() instead. Those methods only exist
from Python 3.10, so 3.9 keeps using load_module(); that branch can be
removed when 3.9 support is dropped.

Use asyncio.get_running_loop() in AwaitableUvicornServer rather than
get_event_loop(). Both callers already create it inside a running
event loop, so behaviour is unchanged.

Add 3.15 to the smoke-test and pre-commit matrices. The cibuildwheel
upgrade needed to build and test wheels on 3.15 is in singlestore-labs#147.

Tested locally on 3.15.0 against SingleStore 9.1.1 (management tests
excluded): 967 passed, 19 skipped, 1 failed, the same results as 3.14.
The failure, TestServerProcessMode::test_process_mode_worker_handles_request,
also fails on 3.14 on macOS and is unrelated to this change.
@kesmit13

Copy link
Copy Markdown
Collaborator

I ran this branch locally with cibuildwheel v4.3.3 (Linux x86_64, cp39-* cp315-*). It needs two changes before it can merge.

1. accel.c: make _PyUnicode_AsUTF8 static

As it stands the build fails at the new abi3audit step:

_singlestoredb_accel.abi3.so: "is_abi3": false, "non_abi3_symbols": ["_PyUnicode_AsUTF8"]
cibuildwheel: error: Audit command failed: abi3audit --strict --report ...

_PyUnicode_AsUTF8 isn't a CPython function. It's our own helper at accel.c:374, and it's declared without static, so it's exported from the .so. abi3audit sees an exported _Py* symbol and treats it as a call outside the stable ABI. v2.16.5 never ran abi3audit, which is why this hasn't shown up before. abi3audit runs on every platform's abi3 wheel, so the Windows and macOS builds would fail the same way.

The helper is only used inside accel.c, so the fix is one word:

-char *_PyUnicode_AsUTF8(PyObject *unicode) {
+static char *_PyUnicode_AsUTF8(PyObject *unicode) {

The name can stay. It's there so the helper can be swapped for CPython's own function once the limited-API baseline reaches 3.10. Note that the 3.10 limited-API function is PyUnicode_AsUTF8AndSize(obj, NULL), not PyUnicode_AsUTF8, and that it returns a borrowed buffer cached on the string object rather than a copy we own. The swap would have to drop the matching free() calls and make sure each source object outlives the stored pointers.

2. publish.yml: keep the glibc 2.17 baseline

cibuildwheel v3+ builds Linux wheels on manylinux_2_28 by default, which raises the minimum glibc from 2.17 to 2.28. Customers on CentOS/RHEL 7, Amazon Linux 2 or Ubuntu 18.04 wouldn't get an error: pip would skip the binary wheel and install our pure-Python py3-none-any wheel, so they'd lose the C accelerator without any warning. The extension is plain C with no need for a newer glibc or toolchain, so manylinux_2_28 gains us nothing. Pin the old image in the "Build and test wheels" env: block:

          # cibuildwheel v3+ builds on manylinux_2_28 (glibc 2.28) by default.
          # Pin manylinux2014 (glibc 2.17) so CentOS/RHEL 7, Amazon Linux 2 and
          # Ubuntu 18.04 still get the C extension instead of falling back to
          # the pure-Python wheel without any warning.
          CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
          CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014

i686 is already skipped by CIBW_SKIP, and musllinux isn't built. Using the manylinux2014 name means cibuildwheel supplies the image version it has pinned, so it keeps getting updated as cibuildwheel is upgraded. v4.3.3 pins the 2026.10.21 build, which includes CPython 3.15.

manylinux2014 is based on CentOS 7, which is end-of-life. We should plan to move to manylinux_2_28 once pypa stops rebuilding the image, and call it out in the release notes when we do.

Local result with both changes

singlestoredb-1.18.1-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
abi3audit: passed
ACCEL-OK 3.9.25  ...
Found previously built wheel ... compatible with cp315-manylinux_x86_64. Skipping build step...
ACCEL-OK 3.15.0 ... [GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]

The test command here only imports _singlestoredb_accel; it doesn't run test_basics against a database. Not covered locally: aarch64, Windows, macOS, and the database tests. CI will cover those, and will also confirm that the aarch64 manylinux2014 image has cp315.

To reproduce:

CIBW_BUILD="cp39-* cp315-*" CIBW_ARCHS_LINUX=x86_64 \
CIBW_SKIP="pp* *-musllinux* *-manylinux_i686" \
CIBW_MANYLINUX_X86_64_IMAGE=manylinux2014 \
CIBW_TEST_COMMAND='python -c "import sys, _singlestoredb_accel; print(sys.version)"' \
uvx cibuildwheel==4.3.3 --platform linux --output-dir wheelhouse

Your Name and others added 2 commits October 29, 2026 00:54
abi3audit, which cibuildwheel v4 runs on abi3 wheels, rejected the
extension because the non-static helper exported a _Py* symbol.

cibuildwheel v3+ defaults to manylinux_2_28; pin manylinux2014 to keep
the glibc 2.17 baseline so older distros still get the C extension.
@mbUSC

mbUSC commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

@kesmit13 good catch, thank you!

  1. _PyUnicode_AsUTF8 is now static, so it's no longer exported and abi3audit should pass.
  2. Pinned CIBW_MANYLINUX_X86_64_IMAGE / CIBW_MANYLINUX_AARCH64_IMAGE to manylinux2014 to keep the glibc 2.17 baseline.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants