Segfault with greenlet 3.4.0 and gevent monkeypatches

[godlygeek]

We've noticed segfaults after main with an atexit handler. A minimal reproducer is something like:

$ python3.12 -m venv .venv
$ source .venv/bin/activate
$ pip install greenlet==3.4.0 gevent
$ python -c 'import atexit; atexit.register(lambda: print(urllib.request.urlopen("http://example.com"))); from gevent import monkey; monkey.patch_all(); import urllib.request; print(urllib.request.urlopen("http://example.com"))'

This doesn't segfault with older versions of greenlet, and seems to be caused by greenlet switching to using atexit handlers for its cleanup.

[godlygeek]

This might be the same issue as #506 - filelock does use an atexit handler - but I'm not certain so I opened a separate issue.

[jamadden]

Thanks! That makes sense --- the example atexit handler is using gevent, which is using greenlet, which has already been cleaned up. I was afraid something like this might happen but I didn't have any test cases (here or gevent) that produced it. In retrospect, I should have written one just like this. I also think you're right about 506.

The workaround is to 'import greenlet' first, before you install your atexit handler.

Another workaround is to disable some of gevent's C accelerators using the GEVENT_PURE_PYTHON environment variable.

The fix might be in gevent - its C code needs to be prepared for getcurrent to return NULL (the pure-python version raises AttributeError as expected).

[godlygeek]

In our case, we can't easily reorder the imports to import greenlet first. Setting aside the minimal reproducer, the real atexit handler we actually hit this from is doing telemetry for us. It's installed by a .pth hook by our company's managed Python interpreters, and exists to report a list of all of the modules that were imported by each script when it exits (to allow us to sort out who's using what in places where many different packages are installed into the same environment).

So, the atexit handler is installed before the user's code ever begins running, and we have no idea that gevent is going to get imported or that it's monkeypatches are going to get installed.

I'm inclined to agree that the right fix here is probably in gevent. Perhaps it might be as simple as gevent installing an atexit handler before it does import greenlet, and having that atexit handler undo the monkeypatches as part of interpreter teardown...

[godlygeek]

If we have to, I'm sure there's other tricks we can play here - like instead of reporting the imported modules from the same Python process, spawning a new one. Higher overhead, but perhaps that's a cost we need to pay to isolate us from the user's code monkeypatching stuff out from underneath us.

But, if this is the same issue as the filelock one, it's not just our weird telemetry atexit handler that's breaking! 🙂