Taskiq version
0.11.17
Python version
Python 3.11
OS
macOS
What happened?
When encountering a task/a message that can't be found, the TaskIQ receiver logs a warning and then returns (None), see receiver.py
However, this leaves the task/message unacked and forever pending in limbo.
Clearly, if the task can't be found this is not going anywhere and either the broker and/or the result backend should in one way or another be informed.
In our case: we use RabbitMQ as broker and had a handful of "not found" tasks (the reason has been fixed, doesn't really matter) around. They were no longer in queue (due to pre-fetch), so inspecting the message queue or purging it did not help. Terminating the consumer only put those messages back into the queue very briefly, before the re-spawned consumer picked them up again and nothing was gained. (For RabbitMQ specifically, these messages should have been nacked. )
Independent of the exact broker and implementation however this behaviour is problematic: it leaves messages in an undefined state.
Relevant log output
WARNING:receiver.py:119: task "project:dont_ask_me_how_this_happened_task" is not found. Maybe you forgot to import it?
Broker initialization code
broker = AioPikaBroker(
url=broker_config.to_url(
vhost=app_config.name
),
queue_name=app_config.name,
delayed_message_exchange_plugin=True,
max_priority=5,
qos=3,
)
Taskiq version
0.11.17
Python version
Python 3.11
OS
macOS
What happened?
When encountering a task/a message that can't be found, the TaskIQ receiver logs a warning and then returns (None), see receiver.py
However, this leaves the task/message unacked and forever pending in limbo.
Clearly, if the task can't be found this is not going anywhere and either the broker and/or the result backend should in one way or another be informed.
In our case: we use RabbitMQ as broker and had a handful of "not found" tasks (the reason has been fixed, doesn't really matter) around. They were no longer in queue (due to pre-fetch), so inspecting the message queue or purging it did not help. Terminating the consumer only put those messages back into the queue very briefly, before the re-spawned consumer picked them up again and nothing was gained. (For RabbitMQ specifically, these messages should have been
nacked. )Independent of the exact broker and implementation however this behaviour is problematic: it leaves messages in an undefined state.
Relevant log output
Broker initialization code