Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,9 @@ Tools/c-analyzer/ @ericsnowcurrently
Tools/check-c-api-docs/ @ZeroIntensity

# Fuzzing
Modules/_xxtestfuzz/ @ammaraskar
Modules/_xxtestfuzz/ @python/fuzzers
Lib/test/test_xxtestfuzz.py @python/fuzzers
.github/workflows/reusable-cifuzz.yml @python/fuzzers

# Limited C API & Stable ABI
Doc/c-api/stable.rst @encukou
Expand Down
8 changes: 0 additions & 8 deletions Doc/c-api/float.rst
Original file line number Diff line number Diff line change
Expand Up @@ -224,11 +224,6 @@ endian processor, or ``0`` on little endian processor.
Return value: ``0`` if all is OK, ``-1`` if error (and an exception is set,
most likely :exc:`OverflowError`).

There are two problems on non-IEEE platforms:

* What this does is undefined if *x* is a NaN or infinity.
* ``-0.0`` and ``+0.0`` produce the same bytes string.

.. c:function:: int PyFloat_Pack2(double x, char *p, int le)

Pack a C double as the IEEE 754 binary16 half-precision format.
Expand Down Expand Up @@ -256,9 +251,6 @@ Return value: The unpacked double. On error, this is ``-1.0`` and
:c:func:`PyErr_Occurred` is true (and an exception is set, most likely
:exc:`OverflowError`).

Note that on a non-IEEE platform this will refuse to unpack a bytes string that
represents a NaN or infinity.

.. c:function:: double PyFloat_Unpack2(const char *p, int le)

Unpack the IEEE 754 binary16 half-precision format as a C double.
Expand Down
61 changes: 45 additions & 16 deletions Doc/c-api/memory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,11 @@ The following function sets, modeled after the ANSI C standard, but specifying
behavior when requesting zero bytes, are available for allocating and releasing
memory from the Python heap.

The :ref:`default memory allocator <default-memory-allocators>` uses the
:ref:`pymalloc memory allocator <pymalloc>`.
In the GIL-enabled build (default build) the
:ref:`default memory allocator <default-memory-allocators>` uses the
:ref:`pymalloc memory allocator <pymalloc>`, whereas in the
:term:`free-threaded build`, the default is the
:ref:`mimalloc memory allocator <mimalloc>` instead.

.. warning::

Expand All @@ -215,6 +218,11 @@ The :ref:`default memory allocator <default-memory-allocators>` uses the

The default allocator is now pymalloc instead of system :c:func:`malloc`.

.. versionchanged:: 3.13

In the :term:`free-threaded <free threading>` build, the default allocator
is now :ref:`mimalloc <mimalloc>`.

.. c:function:: void* PyMem_Malloc(size_t n)

Allocates *n* bytes and returns a pointer of type :c:expr:`void*` to the
Expand Down Expand Up @@ -340,7 +348,9 @@ memory from the Python heap.
the :ref:`Customize Memory Allocators <customize-memory-allocators>` section.

The :ref:`default object allocator <default-memory-allocators>` uses the
:ref:`pymalloc memory allocator <pymalloc>`.
:ref:`pymalloc memory allocator <pymalloc>`. In the
:term:`free-threaded <free threading>` build, the default is the
:ref:`mimalloc memory allocator <mimalloc>` instead.

.. warning::

Expand Down Expand Up @@ -420,23 +430,24 @@ Default Memory Allocators

Default memory allocators:

=============================== ==================== ================== ===================== ====================
Configuration Name PyMem_RawMalloc PyMem_Malloc PyObject_Malloc
=============================== ==================== ================== ===================== ====================
Release build ``"pymalloc"`` ``malloc`` ``pymalloc`` ``pymalloc``
Debug build ``"pymalloc_debug"`` ``malloc`` + debug ``pymalloc`` + debug ``pymalloc`` + debug
Release build, without pymalloc ``"malloc"`` ``malloc`` ``malloc`` ``malloc``
Debug build, without pymalloc ``"malloc_debug"`` ``malloc`` + debug ``malloc`` + debug ``malloc`` + debug
=============================== ==================== ================== ===================== ====================
=================================== ======================= ==================== ====================== ======================
Configuration Name PyMem_RawMalloc PyMem_Malloc PyObject_Malloc
=================================== ======================= ==================== ====================== ======================
Release build ``"pymalloc"`` ``malloc`` ``pymalloc`` ``pymalloc``
Debug build ``"pymalloc_debug"`` ``malloc`` + debug ``pymalloc`` + debug ``pymalloc`` + debug
Release build, without pymalloc ``"malloc"`` ``malloc`` ``malloc`` ``malloc``
Debug build, without pymalloc ``"malloc_debug"`` ``malloc`` + debug ``malloc`` + debug ``malloc`` + debug
Free-threaded build ``"mimalloc"`` ``mimalloc`` ``mimalloc`` ``mimalloc``
Free-threaded debug build ``"mimalloc_debug"`` ``mimalloc`` + debug ``mimalloc`` + debug ``mimalloc`` + debug
=================================== ======================= ==================== ====================== ======================

Legend:

* Name: value for :envvar:`PYTHONMALLOC` environment variable.
* ``malloc``: system allocators from the standard C library, C functions:
:c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`.
* ``pymalloc``: :ref:`pymalloc memory allocator <pymalloc>`.
* ``mimalloc``: :ref:`mimalloc memory allocator <mimalloc>`. The pymalloc
allocator will be used if mimalloc support isn't available.
* ``mimalloc``: :ref:`mimalloc memory allocator <mimalloc>`.
* "+ debug": with :ref:`debug hooks on the Python memory allocators
<pymem-debug-hooks>`.
* "Debug build": :ref:`Python build in debug mode <debug-build>`.
Expand Down Expand Up @@ -733,9 +744,27 @@ The mimalloc allocator

.. versionadded:: 3.13

Python supports the mimalloc allocator when the underlying platform support is available.
mimalloc "is a general purpose allocator with excellent performance characteristics.
Initially developed by Daan Leijen for the runtime systems of the Koka and Lean languages."
Python supports the `mimalloc <https://github.com/microsoft/mimalloc/>`__
allocator when the underlying platform support is available.
mimalloc is a general purpose allocator with excellent performance
characteristics, initially developed by Daan Leijen for the runtime systems
of the Koka and Lean languages.

Unlike :ref:`pymalloc <pymalloc>`, which is optimized for small objects (512
bytes or fewer), mimalloc handles allocations of any size.

In the :term:`free-threaded <free threading>` build, mimalloc is the default
and **required** allocator for the :c:macro:`PYMEM_DOMAIN_MEM` and
:c:macro:`PYMEM_DOMAIN_OBJ` domains. It cannot be disabled in free-threaded
builds. The free-threaded build uses per-thread mimalloc heaps, which allows
allocation and deallocation to proceed without locking in most cases.

In the default (non-free-threaded) build, mimalloc is available but not the
default allocator. It can be selected at runtime using
:envvar:`PYTHONMALLOC`\ ``=mimalloc`` (or ``mimalloc_debug`` to include
:ref:`debug hooks <pymem-debug-hooks>`). It can be disabled at build time
using the :option:`--without-mimalloc` configure option, but this option
cannot be combined with :option:`--disable-gil`.

tracemalloc C API
=================
Expand Down
20 changes: 11 additions & 9 deletions Doc/library/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -606,17 +606,18 @@ are always available. They are listed here in alphabetical order.
.. warning::

This function executes arbitrary code. Calling it with
user-supplied input may lead to security vulnerabilities.
untrusted user-supplied input will lead to security vulnerabilities.

The *source* argument is parsed and evaluated as a Python expression
(technically speaking, a condition list) using the *globals* and *locals*
mappings as global and local namespace. If the *globals* dictionary is
present and does not contain a value for the key ``__builtins__``, a
reference to the dictionary of the built-in module :mod:`builtins` is
inserted under that key before *source* is parsed. That way you can
control what builtins are available to the executed code by inserting your
own ``__builtins__`` dictionary into *globals* before passing it to
:func:`eval`. If the *locals* mapping is omitted it defaults to the
inserted under that key before *source* is parsed.
Overriding ``__builtins__`` can be used to restrict or change the available
names, but this is **not** a security mechanism: the executed code can
still access all builtins.
If the *locals* mapping is omitted it defaults to the
*globals* dictionary. If both mappings are omitted, the source is
executed with the *globals* and *locals* in the environment where
:func:`eval` is called. Note, *eval()* will only have access to the
Expand Down Expand Up @@ -671,7 +672,7 @@ are always available. They are listed here in alphabetical order.
.. warning::

This function executes arbitrary code. Calling it with
user-supplied input may lead to security vulnerabilities.
untrusted user-supplied input will lead to security vulnerabilities.

This function supports dynamic execution of Python code. *source* must be
either a string or a code object. If it is a string, the string is parsed as
Expand Down Expand Up @@ -702,9 +703,10 @@ are always available. They are listed here in alphabetical order.

If the *globals* dictionary does not contain a value for the key
``__builtins__``, a reference to the dictionary of the built-in module
:mod:`builtins` is inserted under that key. That way you can control what
builtins are available to the executed code by inserting your own
``__builtins__`` dictionary into *globals* before passing it to :func:`exec`.
:mod:`builtins` is inserted under that key.
Overriding ``__builtins__`` can be used to restrict or change the available
names, but this is **not** a security mechanism: the executed code can
still access all builtins.

The *closure* argument specifies a closure--a tuple of cellvars.
It's only valid when the *object* is a code object containing
Expand Down
4 changes: 4 additions & 0 deletions Doc/library/stdtypes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3202,6 +3202,10 @@ The conversion types are:
| | character in the result. | |
+------------+-----------------------------------------------------+-------+

For floating-point formats, the result should be correctly rounded to a given
precision ``p`` of digits after the decimal point. The rounding mode matches
that of the :func:`round` builtin.

Notes:

(1)
Expand Down
14 changes: 11 additions & 3 deletions Doc/using/cmdline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1085,6 +1085,13 @@ conflict.
* ``pymalloc_debug``: same as ``pymalloc`` but also install debug hooks.
* ``mimalloc_debug``: same as ``mimalloc`` but also install debug hooks.

.. note::

In the :term:`free-threaded <free threading>` build, the ``malloc``,
``malloc_debug``, ``pymalloc``, and ``pymalloc_debug`` values are not
supported. Only ``default``, ``debug``, ``mimalloc``, and
``mimalloc_debug`` are accepted.

.. versionadded:: 3.6

.. versionchanged:: 3.7
Expand All @@ -1094,12 +1101,13 @@ conflict.
.. envvar:: PYTHONMALLOCSTATS

If set to a non-empty string, Python will print statistics of the
:ref:`pymalloc memory allocator <pymalloc>` every time a new pymalloc object
arena is created, and on shutdown.
:ref:`pymalloc memory allocator <pymalloc>` or the
:ref:`mimalloc memory allocator <mimalloc>` (whichever is in use)
every time a new object arena is created, and on shutdown.

This variable is ignored if the :envvar:`PYTHONMALLOC` environment variable
is used to force the :c:func:`malloc` allocator of the C library, or if
Python is configured without ``pymalloc`` support.
Python is configured without both ``pymalloc`` and ``mimalloc`` support.

.. versionchanged:: 3.6
This variable can now also be used on Python compiled in release mode.
Expand Down
3 changes: 3 additions & 0 deletions Doc/using/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,9 @@ also be used to improve performance.
Disable the fast :ref:`mimalloc <mimalloc>` allocator
(enabled by default).

This option cannot be used together with :option:`--disable-gil`
because the :term:`free-threaded <free threading>` build requires mimalloc.

See also :envvar:`PYTHONMALLOC` environment variable.

.. option:: --without-pymalloc
Expand Down
1 change: 1 addition & 0 deletions Include/cpython/pystate.h
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ struct _ts {
_PyStackChunk *datastack_chunk;
PyObject **datastack_top;
PyObject **datastack_limit;
_PyStackChunk *datastack_cached_chunk;
/* XXX signal handlers should also be here */

/* The following fields are here to avoid allocation during init.
Expand Down
10 changes: 9 additions & 1 deletion Include/internal/pycore_floatobject.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ extern "C" {

/* runtime lifecycle */

extern void _PyFloat_InitState(PyInterpreterState *);
extern PyStatus _PyFloat_InitTypes(PyInterpreterState *);
extern void _PyFloat_FiniType(PyInterpreterState *);

Expand Down Expand Up @@ -42,6 +41,15 @@ extern double _Py_parse_inf_or_nan(const char *p, char **endptr);

extern int _Py_convert_int_to_double(PyObject **v, double *dbl);

/* Should match endianness of the platform in most (all?) cases. */

#ifdef DOUBLE_IS_BIG_ENDIAN_IEEE754
# define _PY_FLOAT_BIG_ENDIAN 1
# define _PY_FLOAT_LITTLE_ENDIAN 0
#else
# define _PY_FLOAT_BIG_ENDIAN 0
# define _PY_FLOAT_LITTLE_ENDIAN 1
#endif

#ifdef __cplusplus
}
Expand Down
3 changes: 1 addition & 2 deletions Include/internal/pycore_pymath.h
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,7 @@ extern void _Py_set_387controlword(unsigned short);
// (extended precision), and we don't know how to change
// the rounding precision.
#if !defined(DOUBLE_IS_LITTLE_ENDIAN_IEEE754) && \
!defined(DOUBLE_IS_BIG_ENDIAN_IEEE754) && \
!defined(DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754)
!defined(DOUBLE_IS_BIG_ENDIAN_IEEE754)
# define _PY_SHORT_FLOAT_REPR 0
#endif

Expand Down
8 changes: 2 additions & 6 deletions Include/internal/pycore_runtime_init.h
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ extern "C" {
#include "pycore_debug_offsets.h" // _Py_DebugOffsets_INIT()
#include "pycore_dtoa.h" // _dtoa_state_INIT()
#include "pycore_faulthandler.h" // _faulthandler_runtime_state_INIT
#include "pycore_floatobject.h" // _py_float_format_unknown
#include "pycore_floatobject.h" // _py_float_format_*
#include "pycore_function.h"
#include "pycore_hamt.h" // _PyHamt_BitmapNode_Type
#include "pycore_import.h" // IMPORTS_INIT
Expand Down Expand Up @@ -84,10 +84,6 @@ extern PyTypeObject _PyExc_MemoryError;
.stoptheworld = { \
.is_global = 1, \
}, \
.float_state = { \
.float_format = _py_float_format_unknown, \
.double_format = _py_float_format_unknown, \
}, \
.types = { \
.next_version_tag = _Py_TYPE_VERSION_NEXT, \
}, \
Expand Down Expand Up @@ -233,4 +229,4 @@ extern PyTypeObject _PyExc_MemoryError;
#ifdef __cplusplus
}
#endif
#endif /* !Py_INTERNAL_RUNTIME_INIT_H */
#endif /* !Py_INTERNAL_RUNTIME_INIT_H */
12 changes: 0 additions & 12 deletions Include/internal/pycore_runtime_structs.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,6 @@ struct _pymem_allocators {
PyObjectArenaAllocator obj_arena;
};

enum _py_float_format_type {
_py_float_format_unknown,
_py_float_format_ieee_big_endian,
_py_float_format_ieee_little_endian,
};

struct _Py_float_runtime_state {
enum _py_float_format_type float_format;
enum _py_float_format_type double_format;
};

struct pyhash_runtime_state {
struct {
#ifndef MS_WINDOWS
Expand Down Expand Up @@ -270,7 +259,6 @@ struct pyruntimestate {
} audit_hooks;

struct _py_object_runtime_state object_state;
struct _Py_float_runtime_state float_state;
struct _Py_unicode_runtime_state unicode_state;
struct _types_runtime_state types;
struct _Py_time_runtime_state time;
Expand Down
1 change: 0 additions & 1 deletion Include/pymacconfig.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
#undef SIZEOF_UINTPTR_T
#undef SIZEOF_PTHREAD_T
#undef WORDS_BIGENDIAN
#undef DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754
#undef DOUBLE_IS_BIG_ENDIAN_IEEE754
#undef DOUBLE_IS_LITTLE_ENDIAN_IEEE754
#undef HAVE_GCC_ASM_FOR_X87
Expand Down
12 changes: 10 additions & 2 deletions Lib/multiprocessing/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -1085,14 +1085,22 @@ def wait(object_list, timeout=None):

Returns list of those objects in object_list which are ready/readable.
'''
object_list = list(object_list)

if not object_list:
if timeout is None:
while True:
time.sleep(1e6)
elif timeout > 0:
time.sleep(timeout)
return []

if timeout is None:
timeout = INFINITE
elif timeout < 0:
timeout = 0
else:
timeout = int(timeout * 1000 + 0.5)

object_list = list(object_list)
waithandle_to_obj = {}
ov_list = []
ready_objects = set()
Expand Down
Loading
Loading