Skip to content

Skip the function finalizer once HA has closed the event loop - #897

Merged
craigbarratt merged 2 commits into
custom-components:masterfrom
Zedas030:finalizer-closed-loop
Nov 18, 2026
Merged

craigbarratt merged 2 commits into
custom-components:masterfrom
Zedas030:finalizer-closed-loop

Conversation

@Zedas030

@Zedas030 Zedas030 commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

FunctionDecoratorManager registers a weakref.finalize on the function so a running manager gets stopped when the function goes away. weakref.finalize also runs from atexit by default, and at that point HA has already closed the loop, so a manager that was still RUNNING calls hass.async_create_task() on a closed loop and leaves the "Event loop is closed" traceback from #880 (plus the "Task was destroyed but it is pending" line for the coroutine that never ran).

The finalizer now has atexit = False, since there is nothing left to stop after shutdown. It also hands the stop off with hass.loop.call_soon_threadsafe(), because the cyclic GC can run it in any thread (including HA's executor threads), and that call raises RuntimeError once the loop is closed. That covers the function being collected after the loop closed but before the interpreter exits. Normal deletion while HA is running still stops the manager.

Added tests in test_decorator_manager.py: deleting the function of a running manager stops it, a running manager whose loop is closed doesn't call async_create_task, the finalizer is not registered for atexit, and deleting the function in a worker thread stops the manager through the loop without HA's thread-safety warning. The closed-loop and worker-thread tests fail without the change. Full test suite passes. I haven't reproduced the shutdown traceback on a real HA install, it's intermittent for the reporter.

Fixes #880

The weakref finalizer on a function's EvalFuncVar also ran from atexit, after HA had closed the loop, so a manager still RUNNING called hass.async_create_task() on a closed loop and logged a traceback at shutdown. Don't run it at exit, and return early when the loop is already closed. Fixes #880.

@craigbarratt craigbarratt left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good. Not running the finalizer at exit and skipping it once the loop is closed address #880.

Optional: make the finalizer thread-safe

There's a related problem that was already there before this PR. Many pyscript objects are referenced in cycles, so they're freed by the cyclic garbage collector, which runs in whichever thread triggers it, including HA's executor threads. The finalizer then calls hass.async_create_task() off the event loop. I reproduced this by deleting a running manager's function in an executor job and calling gc.collect() there. HA raised its thread-safety error (Detected that custom integration 'pyscript' calls hass.async_create_task from a thread other than the event loop ...), and the manager was never stopped. It also leaves a small gap where the loop can close between the is_closed() check and async_create_task().

Handing off with call_soon_threadsafe fixes both, because it's safe from any thread and raises RuntimeError if the loop is already closed:

        def stop_deleted_func():
            if self.status is DecoratorManagerStatus.RUNNING:
                self.hass.async_create_task(self.safe_await(self.stop()))

        def on_func_var_deleted():
            # garbage collection can run this in any thread, so hand off to the loop
            try:
                self.hass.loop.call_soon_threadsafe(stop_deleted_func)
            except RuntimeError:
                # once HA has closed the loop at shutdown there is nothing left to stop
                pass

Your atexit = False stays as is. I tried this locally: your three tests pass unchanged, and this test (which fails on the current branch) passes:

@pytest.mark.asyncio
async def test_function_decorator_manager_finalizer_in_worker_thread(hass, caplog):
    """Deleting the function in a worker thread should stop the manager via the loop."""
    DecoratorManager.hass = hass
    holder = [DummyEvalFuncVar()]
    manager = FunctionDecoratorManager(DummyAstCtx(), holder[0])
    manager.update_status(DecoratorManagerStatus.RUNNING)

    def drop():
        holder.clear()
        gc.collect()

    await hass.async_add_executor_job(drop)
    await hass.async_block_till_done()

    assert manager.status is DecoratorManagerStatus.STOPPED
    assert "thread other than the event loop" not in caplog.text

It's up to you whether to add this here. If you'd rather keep this PR focused, just say so and I'll merge it as is and add the thread-safety change in a follow-up commit.

Comment thread custom_components/pyscript/decorator.py Outdated
def on_func_var_deleted():
if self.status is DecoratorManagerStatus.RUNNING:
# once HA has closed the loop at shutdown there is nothing left to stop
if self.status is DecoratorManagerStatus.RUNNING and not self.hass.loop.is_closed():

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional (see the review summary): this can run in any thread that triggers garbage collection, and the loop can close between this check and async_create_task(). Handing off with hass.loop.call_soon_threadsafe(...) and catching RuntimeError covers both.

The cyclic garbage collector can run the weakref finalizer in any thread, including HA's executor threads, where calling hass.async_create_task() trips HA's thread-safety check and the manager is never stopped. Schedule the stop through loop.call_soon_threadsafe() instead, which also raises RuntimeError once the loop is closed, so the separate is_closed() check and its race go away. As suggested in review.
@Zedas030

Copy link
Copy Markdown
Contributor Author

Thanks for digging into that! I've added your change and the worker-thread test in a second commit. Full suite passes (199), and the new test fails without the change.

@craigbarratt
craigbarratt merged commit 2ae0c15 into custom-components:master Nov 18, 2026
6 checks passed
@craigbarratt

Copy link
Copy Markdown
Member

Looks good - thanks for the PR!

@Zedas030
Zedas030 deleted the finalizer-closed-loop branch November 18, 2026 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Shutdown traceback: weakref finalizer schedules stop() on closed event loop

2 participants