Add compile-time RustPython backend with centralized backend dispatch

[smarcd]

Summary

This PR adds a compile-time RustPython backend to PyO3 and centralizes backend selection behind backend modules instead of scattered backend-specific cfg branches.

What is included

• a compile-time backend split for pyo3, pyo3-ffi, and pyo3-macros-backend
• a working RustPython backend integrated through that split
• backend-boundary enforcement via xtask check-backend-boundary
• CPython-preserving refactors that move backend selection out of frontend modules
• CI/doc fixes needed after the backend refactor
• adaptation to current upstream RustPython tip

Validation

CPython:

• cargo check -p pyo3
• representative downstream package reruns remained green (blake3, rpds, jiter)

RustPython:

• cargo check --no-default-features --features macros,runtime-rustpython -p pyo3
• cargo test --no-default-features --features macros,runtime-rustpython -p pyo3 --test test_gc
• in-tree runtime tests and downstream/package smoke work were used during development to keep the backend honest

Notes

• This is a large draft PR. The main goal is to make the architecture and validation surface reviewable early.
• The next follow-up area is broader ecosystem validation, especially packages that depend on numpy-line crates.
• If preferred, I can also split follow-up cleanup/CI-only commits after initial review.

[alex]

It would probably be better to start with an issue describing what the goal is and how you plan to approach it. Right now the size of this PR makes it completely impossible to review. It looks like right now it involves duplicating significant portions of the library, which is a red flag to me that this is going to be a long term maintenance cost.

[smarcd]

The basic idea is splitting PyO3 into front and backend with the frontend being the "user facing" and the backend targeting different Pythons - i.e. CPython and RustPython.

The duplication you see is probably the current CPython-specific backend code that was moved to a new home.

[codspeed-hq]

Merging this PR will degrade performance by 44.68%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 2 regressed benchmarks
✅ 102 untouched benchmarks
⏩ 1 skipped benchmark¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	extract_str_extract_fail	2 µs	2.3 µs	-16.01%
❌	extract_btreeset	17.7 ms	32 ms	-44.68%
⚡	extract_hashset	23.2 ms	18.7 ms	+24.23%

---

_{Comparing smarcd:rustpython-backend-upstream-tip (e5623b4) with main (999560a)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

[davidhewitt]

While RustPython support is laudable, this is a huge effort to embark on that will increase massively the maintenance burden for the PyO3 team. There is no obvious concensus that going this route is the way that makes most sense, e.g. RustPython has been recently looking at supporting the Python limited API.

I completely second @alex that there should be clear agreement on both the goals and the solution, I am unenthusiastic about reviewing anything of this size without that achieved first.