Retain strong refs to update-processing tasks in AsyncTeleBot#2588
Retain strong refs to update-processing tasks in AsyncTeleBot#2588Dramex wants to merge 1 commit intoeternnoir:masterfrom
Conversation
asyncio only keeps weak references to tasks returned from create_task, so a fire-and-forget task with no external reference may be garbage collected mid-execution, silently dropping updates. This adds a per-bot _pending_tasks set that holds each processing task until it completes, mirroring the pattern documented in the asyncio docs. Closes eternnoir#2572
There was a problem hiding this comment.
Pull request overview
This PR fixes a potential asyncio GC hazard in AsyncTeleBot polling by retaining strong references to fire-and-forget tasks created for update processing, and adds a regression test to ensure tasks are tracked and cleaned up.
Changes:
- Add
self._pending_tasksto keep strong references to in-flight update-processing tasks. - Track each
asyncio.create_task(self.process_new_updates(...))in_process_pollingand discard it on completion. - Add a self-contained unit test covering task tracking and cleanup behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
telebot/async_telebot.py |
Retains strong refs to background tasks created during polling and discards them when done. |
tests/test_async_telebot.py |
Adds regression test validating _pending_tasks tracking during task execution and cleanup after completion. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| await bot._process_polling(non_stop=True, interval=0, timeout=0) | ||
| # Allow the fire-and-forget task to finish plus one yield for the | ||
| # add_done_callback discard to run. | ||
| await process_completed.wait() |
There was a problem hiding this comment.
The test awaits process_completed.wait() with no timeout. If the regression reappears (or the stubbed polling loop changes), this can hang the entire test run. Consider wrapping the wait in asyncio.wait_for(..., timeout=...) so the test fails fast instead of blocking indefinitely.
| await process_completed.wait() | |
| await asyncio.wait_for(process_completed.wait(), timeout=1) |
| # Retain a strong reference so the task isn't GC'd mid-execution. | ||
| task = asyncio.create_task(self.process_new_updates(updates)) | ||
| self._pending_tasks.add(task) | ||
| task.add_done_callback(self._pending_tasks.discard) |
There was a problem hiding this comment.
The done callback currently only discards the task from _pending_tasks. If process_new_updates raises, the task’s exception is never retrieved/observed, which can lead to noisy Task exception was never retrieved warnings. Consider extending the done-callback to call task.exception() (and possibly log via _handle_exception/logger) before discarding, so failures aren’t silently ignored.
| # Strong references to background tasks created via asyncio.create_task(). | ||
| # asyncio only keeps weak references, so unreferenced tasks can be GC'd | ||
| # mid-execution; see https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task | ||
| self._pending_tasks: set = set() |
There was a problem hiding this comment.
self._pending_tasks is annotated as a bare set, which loses the element type in this typed package. Consider using a parameterized type like set[asyncio.Task] (or set[asyncio.Task[None]] if appropriate) so static type checkers and IDEs can reason about what’s stored here.
| self._pending_tasks: set = set() | |
| self._pending_tasks: set[asyncio.Task[Any]] = set() |
Closes #2572.
What
AsyncTeleBot._process_pollingdispatches incoming update batches viaasyncio.create_task(...)without retaining a reference to the resultingTask. Per the Python asyncio docs, the event loop only keeps weak references to tasks, so an unreferenced task may be garbage-collected mid-execution and updates silently dropped.Change
self._pending_tasks: settoAsyncTeleBot.__init__to hold strong references to in-flight background tasks._process_polling, captured theTaskreturned byasyncio.create_task(...), added it to_pending_tasks, and attachedtask.add_done_callback(self._pending_tasks.discard)so the set self-cleans on completion.This is the exact pattern suggested by the asyncio docs and matches the minimal fix proposed by the reporter (@fshp971).
Why this approach vs the alternatives
@All-The-Foxes suggested two alternatives in the thread:
process_new_updatesis fast.awaitprocess_new_updatesdirectly — simplest, but changes concurrency semantics (polling would block on handler execution). That's a behavior change for existing bots.This PR takes the minimal-risk route: preserves existing fire-and-forget semantics while fixing the GC hazard. Happy to rework toward either alternative if that's the preferred direction.
Tests
Added
tests/test_async_telebot.pywith a regression test that:_process_pollingwith stubbedget_updates/process_new_updates/get_me/close_session_pending_tasksduring execution_pending_tasksis emptied after the task completes (discard callback fires)Full test run:
49 passed, 72 skipped(skipped tests requireTOKEN/CHAT_IDenv vars, unchanged from before this PR).