gh-131565: Implement ctypes.util.dllist() in the _ctypes extension

[serhiy-storchaka]

ctypes.util.dllist() returned only the main executable on NetBSD, because NetBSD's dl_iterate_phdr() enumerates only the link-map group of the calling object. Called through ctypes, the caller is libffi's closure trampoline, which belongs to no object, so nothing else was reported (verified on NetBSD 10: a C helper .so calling dl_iterate_phdr() reports all objects, while the ctypes path reports 1).

Implement dllist() as a small C helper in the _ctypes extension, so the caller is _ctypes and all loaded shared libraries are reported. The pure-Python dl_iterate_phdr() wrapper in ctypes.util is replaced by from _ctypes import dllist. Apple platforms keep their dyld-based implementation (the C helper is gated out there).

Verified on NetBSD 10 (now 24 objects, was 1), Linux (20), OpenBSD 7 (23) and FreeBSD 15 (26); test_dllist passes on all four.

Fixes the root cause investigated in gh-131565. CC @WardBrian

🤖 Generated with Claude Code

[johnslavik]

(Tagging Peter for his interest in ctypes.)

[WardBrian]

This patch looks reasonable to me. I find the comments claude generates to be a bit too verbose, but that's personal preference.

Nice find on the netbsd trampoline behavior. That's an edge case I never would have considered. Arguably the fact that this can't be implemented in pure python makes it's inclusion in the standard library even more useful

[serhiy-storchaka]

dl_iterate_phdr() is called with the GIL held on purpose: CPython already calls dlopen() with the GIL held, both in py_dl_open() above and in Python/dynload_shlib.c, so the lock order is GIL first, dynamic loader lock second, and this follows it.

Acquiring the GIL in the callback would reverse that order — the callback runs while the loader lock is held — and could deadlock against a thread which is in dlopen() with the GIL. Releasing the GIL is only safe if the callback does not touch Python at all: copy the names while iterating and build the list after dl_iterate_phdr() returns. It only walks the already loaded objects, so I do not think it is worth the complication.

As for the return value, dl_iterate_phdr() returns whatever the last callback invocation returned. It has no error channel of its own and does not document setting errno, and our callback returns non-zero only after setting an exception, so PyErr_Occurred() carries the same information without depending on the callback's convention.

(This answer was written by Claude.)

[ZeroIntensity]

Let's add a comment then. It's unclear to me as to why dlopen is called with the GIL, though.

[ZeroIntensity]

I'm not a huge fan of the PyErr_Occurred thing, but I won't block the PR on it. LGTM.

[serhiy-storchaka]

Thank you for your review.

[miss-islington-app]

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-154882 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-154883 is a backport of this pull request to the 3.14 branch.