Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 95 additions & 46 deletions peps/pep-0783.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Post-History: `02-Apr-2025 <https://discuss.python.org/t/86862>`__,
Abstract
========

This PEP proposes a new platform tag series ``pyodide`` for binary Python package
distributions for the Pyodide Python runtime.
This PEP proposes a new platform tag series ``pyemscripten`` for binary Python
package distributions for the Pyodide Python runtime.

`Emscripten <https://emscripten.org/>`__ is a complete open-source compiler
toolchain. It compiles C/C++ code into WebAssembly/JavaScript executables, for
Expand Down Expand Up @@ -51,32 +51,40 @@ This creates friction both for package maintainers and for users.
Rationale
=========

Emscripten uses a variant of musl libc. The Emscripten compiler makes no ABI
stability guarantees between versions. Many Emscripten updates are ABI
compatible by chance, and the Rust Emscripten target behaves as if the ABI were
stable with only `occasional negative consequences
<https://github.com/rust-lang/rust/issues/131467>`__.
When Emscripten builds an application, it builds it as a free-standing program,
including the entire operating system. Emscripten primarily targets the use case
of fully static programs. When dynamic linking is used, the primary target use
case is bundle splitting and lazy loading, where the dynamic libraries are built
at the same time as the application.

There are several linker flags that adjust the Emscripten ABI, so Python
packages built to run with Emscripten must make sure to match the ABI-sensitive
linker flags used to compile the interpreter to avoid load-time or run-time
errors. The Emscripten compiler continuously fixes bugs and adds support for new
web platform features. Thus, there is significant benefit to being able to
update the ABI.
As a result of that, the Emscripten compiler makes no ABI stability guarantees
between versions. Many Emscripten updates are ABI compatible by chance, and the
Rust Emscripten target behaves as if the ABI were stable with only `occasional
negative consequences <https://github.com/rust-lang/rust/issues/131467>`__.

There are several linker flags that adjust the Emscripten ABI or system
libraries. Python packages built to run with Emscripten must make sure to match
the ABI-sensitive linker flags used to compile the interpreter to avoid
load-time or run-time errors. The Emscripten compiler continuously fixes bugs
and adds support for new web platform features. Thus, there is significant
benefit to being able to update the ABI.

In order to balance the ABI stability needs of package maintainers with the ABI
flexibility to allow the platform to move forward, Pyodide plans to adopt a new
ABI for each feature release of Python.
ABI for each feature release of Python which we call
``pyemscripten_${YEAR}_${PATCH}``.

The Pyodide team also coordinates the ABI flags that Pyodide uses with the
Emscripten ABI that Rust supports in order to ensure that we have support for
the many popular Rust packages. Historically, most of the work for this has
been related to unwinding ABIs. See for instance `this Rust Major Change
Proposal <https://github.com/rust-lang/compiler-team/issues/801>`__.

The ``pyodide`` platform tags only apply to Python interpreters compiled and
linked with the same version of Emscripten as Pyodide, with the same
ABI-sensitive flags.
The ``pyemscripten`` platform has nothing specifically to do with Python and
indeed can be used by any program that uses the appropriate version of
Emscripten and the appropriate link flags. In particular, ``pyemscripten``
platform tags can be used by Python interpreters compiled and linked with the
specified version of Emscripten and with the specified ABI-sensitive flags.


Specification
Expand All @@ -86,15 +94,15 @@ The platform tags will take the form:

.. code-block:: text

pyodide_${PYTHON_MAJOR_MINOR}_${PATCH}_wasm32
pyemscripten_${YEAR}_${PATCH}_wasm32

Each one of these will be used with a specified Python version. For example, the
platform tag ``pyodide_314_0`` will be used with Python 3.14.
platform tag ``pyemscripten_2026_0`` will be used with Python 3.14.

Emscripten Wheel ABI
--------------------

The specification of the ``pyodide_<abi>`` platform includes:
The specification of the ``pyemscripten_<abi>`` platform includes:

* Which version of the Emscripten compiler is used
* What libraries are statically linked with the interpreter
Expand All @@ -113,7 +121,7 @@ The Pyodide ABIs are fully specified in the `Pyodide Platform ABI

The ``pyodide build`` tool knows how to create wheels that match the Pyodide
ABI. Unlike with manylinux wheels, there is no need for a Docker container to
build the ``pyodide_<abi>`` wheels. All that is needed is a Linux machine and
build the ``pyemscripten_<abi>`` wheels. All that is needed is a Linux machine and
appropriate versions of Python, Node.js, and Emscripten.

It is possible to validate a wheel by installing and importing it into the
Expand All @@ -123,12 +131,12 @@ sandboxing guarantees, doing this produces no security risks.
Determining the ABI version
---------------------------

The Pyodide ABI version is stored in the ``PYODIDE_ABI_VERSION`` config variable
The ABI version is stored in the ``PYEMSCRIPTEN_ABI_VERSION`` config variable
and can be determined via:

.. code-block:: python

pyodide_abi_version = sysconfig.get_config_var("PYODIDE_ABI_VERSION")
pyemscripten_abi_version = sysconfig.get_config_var("PYEMSCRIPTEN_ABI_VERSION")


To generate the list of compatible tags, one can use the following code:
Expand All @@ -138,9 +146,9 @@ To generate the list of compatible tags, one can use the following code:
from packaging.tags import cpython_tags, _generic_platforms

def _emscripten_platforms() -> Iterator[str]:
pyodide_abi_version = sysconfig.get_config_var("PYODIDE_ABI_VERSION")
if pyodide_abi_version:
yield f"pyodide_{pyodide_abi_version}_wasm32"
pyemscripten_abi_version = sysconfig.get_config_var("PYEMSCRIPTEN_ABI_VERSION")
if pyemscripten_abi_version:
yield f"pyemscripten_{pyemscripten_abi_version}_wasm32"
yield from _generic_platforms()

emscripten_tags = cpython_tags(platforms=_emscripten_platforms())
Expand All @@ -154,14 +162,14 @@ Package Installers

Installers should use the ``_emscripten_platforms()`` function shown above to
determine which platforms are compatible with an Emscripten build of CPython. In
particular, the Pyodide ABI version is exposed via
``sysconfig.get_config_var("PYODIDE_ABI_VERSION")``.
particular, the ABI version is exposed via
``sysconfig.get_config_var(" PYEMSCRIPTEN_ABI_VERSION")``.

Package Indexes
---------------

Package indexes SHOULD accept any wheel whose platform tag matches
the regular expression ``pyodide_[0-9]+_[0-9]+_wasm32``.
the regular expression ``pyemscripten_[0-9]+_[0-9]+_wasm32``.


Dependency Specifier Markers
Expand Down Expand Up @@ -196,30 +204,71 @@ There are no security implications in this PEP.
Rejected Ideas
==============

Alternative Options for the Platform Tag
----------------------------------------
A Custom Interpreter Tag For Pyodide
------------------------------------

* ``pyodide_${YEAR}_${PATCH}`` -- This avoids potential confusion between e.g.,
``314_0`` and Python ``3.14.0``. On the other hand, it makes the link between
ABI and Python minor version ambiguous.
We don't need a custom interpreter tag for Pyodide because Pyodide is CPython.
While we do apply a few minor patches, they have no affect on the interpeter ABI
and our long term goal is to upstream everything.

* ``pyodide_${MAJOR}_${MINOR}_${ABI_PATCH}`` -- this would handle a future
two-digit major version, but the ABI patch version would be too easily
confused with a Python patch version e.g., ``3_14_0`` seems to imply Python
``3.14.0`` specifically rather than Python 3.14.x.
Alternative Options for the Platform Tag
----------------------------------------

* No ABI patch version -- we hope never to need the patch version, but it's
good to be prepared for unforseen problems.
``emscripten_${EMSCRIPTEN_VERSION}``
It is tempting to use ``emscripten`` as the platform tag because the
``pyemscripten`` platform has nothing specifically to do with Python and
indeed can be used by any program that uses the appropriate version of
Emscripten and the appropriate link flags. But
``emscripten_${EMSCRIPTEN_VERSION}`` is too vague by itself because the ABI
also depends on various linker flags.

There are other communities which have similar problems and would also benefit
from a centralized standard for "Long Term Service" ABIs that the whole
ecosystem could use. However, the Emscripten team have so far not been willing
to provide a this standard since they consider dynamic linking an unusual use
case. Thus it is left for our ecosystem to solve the problem itself. The
platform tag should contains some indication of this.

``pyemscripten_${PYTHON_MAJOR_MINOR}_${PATCH}``
This would make it clearer which Python version is meant for use with each
ABI, but it leads to conceptual confusion since the platform has nothing to do
with Python.

``pyodide_...``
For now the platform is defined by Pyodide so this connection would be made
clearer by calling the platform ``pyodide``. But the capabilities of the
platform are tied to what Emscripten supports not on what Pyodide supports so
the platform tag should be focused on Emscripten. The ``pyemscripten`` tag is
also more forwards compatible to a future where the definition of the platform
moves upstream of Pyodide.

No ABI patch version
We hope never to need the patch version, but it's good to be prepared for
unforseen problems.

How to Teach This
=================

For Pyodide users, we recommend the `Pyodide documentation on installing
packages <https://pyodide.org/en/stable/usage/loading-packages.html>`__.

For package maintainers, we recommend the `Pyodide documentation on building and
testing packages
<https://pyodide.org/en/stable/development/building-and-testing-packages.html>`__.
Fo Pyodide Users
----------------
We recommend the `Pyodide documentation on installing packages
<https://pyodide.org/en/stable/usage/loading-packages.html>`__. We will make a
table showing which ``pyemscripten`` ABI each Pyodide version is compatible with.

For Package Maintainers
-----------------------

We recommend the `Pyodide documentation on building and testing packages
<https://pyodide.org/en/stable/development/building-packages-from-source.html>`__.
The Scientific Python community is also working on
`a spec <https://github.com/scientific-python/specs/pull/388/>`_
which describes to package maintainers how to maintain web-based interactive
documentation using Emscripten-based Python.

Generally cibuildwheel is the easiest way to build and test a package for use
with Pyodide. Maintainers can also use ``pyodide-build`` directly to build a
package. Rust packages that use Maturin as their build system can be built
directly with Maturin since it has native support for cross builds.

Reference Implementation
========================
Expand Down
Loading