diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index ab7e0efee10..27dda896edc 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -714,6 +714,7 @@ peps/pep-0836.rst @savannahostrowski @Fidget-Spinner @brandtbucher peps/pep-0837.rst @serhiy-storchaka peps/pep-0838.rst @AlexWaygood peps/pep-0840.rst @jeremyhylton @gvanrossum +peps/pep-0842.rst @ZeroIntensity # ... peps/pep-2026.rst @hugovk # ... diff --git a/peps/pep-0842.rst b/peps/pep-0842.rst new file mode 100644 index 00000000000..0b58c072bfe --- /dev/null +++ b/peps/pep-0842.rst @@ -0,0 +1,575 @@ +PEP: 842 +Title: Module Exports +Author: Peter Bierma +Discussions-To: Pending +Status: Draft +Type: Standards Track +Created: 25-Jul-2026 +Python-Version: 3.16 +Post-History: `24-Jul-2026 `__ + + +Abstract +======== + +This PEP proposes an ``__export__`` variable that modules can define to +limit visibility and access to variables from outside the module. + +For example: + +.. code-block:: python + + # spam.py + __export__ = ["Public"] + + class Public: + ... + + class Private: + ... + + +.. code-block:: pycon + + >>> import spam + >>> spam.Public + + >>> spam.Private + Traceback (most recent call last): + File "", line 1, in + spam.Private + ImportError: 'Private' is not exported by 'spam' + + +Motivation +========== + + +Private names can be difficult to disambiguate on their own +----------------------------------------------------------- + +Imagine that a developer wants to define a private class in +their module. Their first instinct might be to simply define their class +as such: + +.. code-block:: python + + # spam.py + class Helper: + ... + +However, this comes with no indication that ``Helper`` is supposed to be +internal to the module, so users may accidentally begin using and relying +on it. In fact, Python's interactive :func:`help` function will even include +``Helper`` in its output next to everything else in the module. + + +Prefixed names are less maintainable +------------------------------------ + +When developing a Python module, it is common to prefix a name with ``_`` +to denote that it is private. So, as a solution to the above problem, +the developer prefixes the name with ``_``: + +.. code-block:: python + + # spam.py + class _Helper: + ... + + +Now, it's clear to users that the name is internal, at the expense of the name +being (subjectively) less readable and requiring more keystrokes by the maintainer. + +In addition, it can be difficult to remember where names need to be prefixed. +To put this issue into perspective, imagine that a developer wants to import +some other modules in their code: + +.. code-block:: python + + # spam.py + import argparse + import asyncio + import tabnanny + +In the above example, the ``spam`` module will have ``argparse``, ``asyncio``, +and ``tabnanny`` as seemingly public attributes. In practice, this is not good +for a maintainer, because maintainers may want to remove and change imports as +they please, so these attributes should not be treated as public APIs. + +Python's standard library currently sidesteps this problem through a note in +the backwards compatibility policy (:pep:`387`) that states that imported +modules are not considered public APIs and may change at any time, but +unfortunately, users are unable to determine this without directly reading +the backwards compatibility policy, which is not a common thing to do. +The solution to this is to also prefix every imported name with ``_``: + +.. code-block:: python + + # spam.py + import argparse as _argparse + import asyncio as _asyncio + import tabnanny as _tabnanny + +This brings us back to the original problem: this sprinkles the +code with extra underscores, and puts mental overhead on the developer +by requiring them to remember to prefix their imports with ``_``. + +Ideally, users shouldn't be tempted to reach for private names from modules in +the first place. + + +Specification +============= + + +.. _pep-842-export-requirements: + +``__export__`` rules +-------------------- + +Object requirements +******************* + +When defined in a module's global scope, ``__export__`` must be assigned to an +object that implements :meth:`~object.__contains__` or :meth:`~object.__iter__` +such that :class:`str` objects can be checked for containment on it. In other +words, the expression ``str_instance in __export__`` should not raise an exception. +In practice, this means that ``__export__`` will typically be a :class:`tuple` or a +:class:`list` object: + +.. code-block:: python + + __export__ = ["name1", "name2", "name3"] + __export__ = ("name1", "name2", "name3") + __export__ = {"name1", "name2", "name3"} + +Again, however, the only requirement for ``__export__`` is that the :keyword:`in` +operator is valid on it for instances of ``str``. For example, some more exotic +types are also valid assignments for ``__export__``: + +.. code-block:: python + + __export__ = {"name": 0} + # '"name" in __export__' is valid, so this is okay + + +Item requirements +***************** + +It is not required that the strings inside ``__export__`` are actually names defined +in the module (because it is not required for ``__export__`` to be a +:class:`~collections.abc.Sequence` or similar, so there is no way to validate all +values in ``__export__``), though there is no practical reason to do so. +For example, the following is valid (as in, it will not generate an exception +at runtime), with one caveat: + +.. code-block:: python + + __export__ = ["does not exist"] + +The caveat is that this will raise an exception when used with a wildcard import +(``from module import *``), because ``__all__`` is implicitly set by ``__export__``; +see :ref:`pep-842-implicit-all`. + + +Module attribute access +----------------------- + +When ``__export__`` is present in a module's globals, all access to attributes +present on the module object will also check if the attribute name is present +in ``__export__`` (via ``__contains__`` or through iteration, as specified previously). +If the attribute name is not present in ``__export__``, then an :exc:`ImportError` +is raised. For example: + +.. code-block:: python + + # spam.py + a = 42 + b = 24 + + __export__ = ["a"] + +.. code-block:: pycon + + >>> import spam + >>> spam.a + 42 + >>> spam.b + Traceback (most recent call last): + File "", line 1, in + spam.b + ImportError: 'b' is not exported by 'spam' + + +Dunder names +************ + +This does not apply to :term:`dunder` names; attributes such as :attr:`~object.__dict__` +and :attr:`~module.__file__` will always be accessible on the module through +attribute access, even if they are not included in the module's ``__export__``. +For example: + +.. code-block:: python + + # spam.py + __export__ = [] + +.. code-block:: pycon + + >>> import spam + >>> spam.__name__ + 'spam' + + +Lazy imports +************ + +:ref:`Lazy imports ` that are not listed in ``__export__`` +will not be reified upon being accessed outside the module. For example: + +.. code-block:: python + + # spam.py + lazy import json + + __export__ = [] + +.. code-block:: pycon + + >>> import spam, sys + >>> assert 'json' in sys.lazy_modules + >>> spam.json + Traceback (most recent call last): + File "", line 1, in + spam.json + ImportError: 'json' is not exported by 'spam' + >>> # json is still lazy and has not been resolved + >>> assert 'json' in sys.lazy_modules + + +Module ``__getattr__`` functions +-------------------------------- + +The behavior of ``__export__`` cannot be overridden by a module's +:meth:`~module.__getattr__` function, as ``__getattr__`` functions are only +invoked for undefined names on modules. However, in cases where a module +``__getattr__`` is invoked, ``__export__`` has no effect. For example: + +.. code-block:: python + + # spam.py + __export__ = ["exported"] + + exported = 42 + + def __getattr__(name): + if name == "exported": + # This is never triggered! + raise ImportError() + + if name == "hello": + # "hello" is never put through the __export__ filter + return 42 + + raise AttributeError(f"{__name__!r} has no attribute {name!r}") + +.. code-block:: pycon + + >>> import spam + >>> spam.__export__ + ["exported"] + >>> spam.exported + 42 + >>> spam.hello + 42 + + +``__dir__`` behavior +-------------------- + +On a module with ``__export__``, the module's :meth:`~module.__dir__` function +will be modified to exclude names that are not in the module's ``__export__``. +As with attributes, this behavior does not apply to dunder names; those will always +be included in the output of ``dir()``, regardless of whether the names are included +in ``__export__``. For example: + +.. code-block:: python + + # spam.py + class Public: + ... + + class Private: + ... + + __export__ = ["Public"] + +.. code-block:: pycon + + >>> import spam + >>> dir(spam) + ['Public', '__builtins__', '__doc__', '__export__', '__file__', '__loader__', '__name__', '__package__', '__spec__'] + + +User-defined module ``__dir__`` functions +***************************************** + +If a module defines its own ``__dir__`` method, it takes precedence +over this behavior. It is up to the implementer of ``__dir__`` to exclude +names that are not present in ``__export__``. For example: + +.. code-block:: python + + # spam.py + a = 42 + b = 24 + + __export__ = ['a'] + + def __dir__(): + return list(globals().keys()) + +.. code-block:: pycon + + >>> import spam + >>> dir(spam) + [..., 'a', 'b'] + +It is worth noting that there are real consequences for including unexported +names in custom ``__dir__`` functions. For example, :func:`help` can no longer +be used with the above module: + +.. code-block:: pycon + + >>> import spam + >>> help(spam) + 'b' is not exported by 'spam' + + +.. _pep-842-implicit-all: + +Implicit ``__all__`` definitions +-------------------------------- + +If a module defines ``__export__`` but does not define :attr:`~module.__all__`, +the ``__all__`` will be assigned to ``__export__``. To visualize: + +.. code-block:: python + + # spam.py + a = 42 + b = 24 + c = 'c' + + __export__ = ['a', 'b'] + # __all__ is implicitly set to ['a', 'b'], so 'c' will not be included in + # wildcard imports. + +.. code-block:: pycon + + >>> from spam import * + >>> a + 42 + >>> b + 24 + >>> c + Traceback (most recent call last): + File "", line 1, in + c + NameError: name 'c' is not defined + +This means that the +:ref:`previously specified requirements ` for +``__export__`` are not exhaustive, as ``__export__`` in this case must also +be a valid ``__all__``. For example, including a name that does not exist in +``__export__`` will break wildcard imports: + +.. code-block:: python + + # spam.py + a = 42 + __export__ = ['a', 'noexist'] + + +.. code-block:: pycon + + >>> from spam import * + Traceback (most recent call last): + File "", line 1, in + from spam import * + AttributeError: module 'spam' has no attribute 'noexist' + + +Semantic implementation +----------------------- + +For a module, defining ``__export__`` is roughly equivalent to adding the +following code: + +.. code-block:: python + + __all__ = __export__ + + def _is_dunder_name(name): + return (len(name) > 4) and name.startswith("__") and name.endswith("__") + + def __getattribute__(name): + try: + value = globals()[name] + except KeyError: + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") from None + + if _is_dunder_name(name): + return value + + if name not in __export__: + raise ImportError(f"{name!r} is not exported by {__name__!r}") + + return value + + _module = sys.modules[__name__] + # This is a spooky magic function -- pretend it exists for example's sake + patch(_module, '__getattribute__', __getattribute__) + + def __dir__(): + names = [] + for name in globals().keys(): + if (name in __export__) or _is_dunder_name(name): + names.append(name) + return names + + +Rationale +========= + + +``__export__`` is not a secure access modifier +---------------------------------------------- + +This PEP does not aim to be a secure mechanism for preventing access to +private attributes in modules. In fact, bypassing ``__export__`` is trivial; +simply access ``mod.__dict__['attr_name']`` instead of ``mod.attr_name`` at +runtime. + +This is by design. Python does not include access modifiers as a language +feature for a reason. To `quote `__ Eric Smith: +"Access to internals of other classes is a feature when you need it". This PEP +does not intend to change this convention, nor should it be interpreted as an +indication that Python is tending toward the direction of true access modifiers. + +Instead, the intention of this PEP is to improve clarity when inspecting modules +at runtime, which should, in turn, improve the maintainer experience of Python +modules in the long term. + + +Backwards Compatibility +======================= + +This PEP has the potential to break users who were already defining global +variables called ``__export__``. That said, the Python language reference +:ref:`explicitly forbids ` users from doing this in the first place. + + +Security Implications +===================== + +This PEP has no known security implications. + + +How to Teach This +================= + +``__export__`` will be documented as part of the language standard. + +To help adoption, it will be recommended that users define both ``__all__`` +and ``__export__`` in their modules. This allows code on Python 3.16+ to get +the proper export behavior, while older versions still keep their ``__all__`` +attribute. In practice, this should look something like this: + +.. code-block:: python + + __all__ = ["hovercraft"] + __export__ = __all__ + ["eels"] + + +Reference Implementation +======================== + +A reference implementation of this PEP can be found +`here `__. + + +Rejected Ideas +============== + + +Reuse ``__all__`` for exports +----------------------------- + +Instead of adding a new ``__export__`` variable, an alternative was to +reuse ``__all__`` for names. + +This was ultimately decided against because it seemed clear that there were +cases where a name could be in ``__export__``, but not in ``__all__``. The +primary example for this case was with static typing. For example, a module +may define several type aliases that would pollute a namespace if used with +a wildcard import, so the developer chooses to not include them in ``__all__``, +but users of static typing will still want access to these type aliases for +annotating their own code. + + +Add new ``export`` syntax +------------------------- + +Rather than simply defining exported names in a global variable, it was +`suggested `__ to take this proposal a +step further and add a true ``export`` (soft) keyword to Python that would +effectively auto-generate an ``__export__`` variable at runtime, like so: + +.. code-block:: python + + export class Foo: + ... + + export MY_CONST = 42 + + # __export__ would now be set to ['Foo', 'MY_CONST'] + + +This is feasible, and may very well become part of Python someday, but the timing +did not feel right. Adding new syntax to Python requires a lot of demand and +community feedback, and it was unclear whether there was enough demand for this +feature to warrant new syntax. + +That said, it is expected that if this PEP is accepted, libraries will take +advantage of ``__export__`` to build APIs that replicate the proposed ``export`` +syntax. Third-party solutions and widespread adoption would make it much clearer +that new syntax is the best choice for Python in the long run. + + +Open Issues +=========== + +TBD. + + +Acknowledgements +================ + +Thanks to Hugo van Kemenade and Savannah Ostrowski for `inspiring +`__ the idea +behind this PEP. + + +Change History +============== + +TBD. + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.