Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: 2026-03-31 00:32:38,382 ERROR [pulsar.messaging.bind_amqp][[manager=_default_]-[action=postprocess]-[job=163778]] Failure to publish Pulsar state change for job_id 163778.
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: Traceback (most recent call last):
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/pulsar/messaging/bind_amqp.py", line 74, in bind_on_status_change
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: pulsar_exchange.publish("status_update", payload)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/pulsar/client/amqp_exchange.py", line 236, in publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: producer.publish(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/kombu/messaging.py", line 186, in publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return _publish(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/kombu/connection.py", line 556, in _ensured
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return fun(*args, **kwargs)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/kombu/messaging.py", line 208, in _publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return channel.basic_publish(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/channel.py", line 1791, in _basic_publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: self.connection.drain_events(timeout=0)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/connection.py", line 525, in drain_events
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: while not self.blocking_read(timeout):
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/connection.py", line 531, in blocking_read
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return self.on_inbound_frame(frame)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/method_framing.py", line 53, in on_frame
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: callback(channel, method_sig, buf, None)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/connection.py", line 537, in on_inbound_method
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return self.channels[channel_id].dispatch_method(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/abstract_channel.py", line 156, in dispatch_method
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: listener(*args)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/channel.py", line 293, in _on_close
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: raise error_for_code(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: amqp.exceptions.PreconditionFailed: Basic.publish: (406) PRECONDITION_FAILED - message size 17306945 is larger than configured max size 16777216
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: Exception in thread [manager=_default_]-[action=postprocess]-[job=163778]:
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: Traceback (most recent call last):
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: self.run()
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/usr/lib/python3.9/threading.py", line 892, in run
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: self._target(*self._args, **self._kwargs)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/pulsar/managers/stateful.py", line 229, in do_postprocess
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: self.__state_change_callback(final_status, job_id)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/pulsar/messaging/bind_amqp.py", line 74, in bind_on_status_change
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: pulsar_exchange.publish("status_update", payload)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/pulsar/client/amqp_exchange.py", line 236, in publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: producer.publish(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/kombu/messaging.py", line 186, in publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return _publish(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/kombu/connection.py", line 556, in _ensured
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return fun(*args, **kwargs)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/kombu/messaging.py", line 208, in _publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return channel.basic_publish(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/channel.py", line 1791, in _basic_publish
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: self.connection.drain_events(timeout=0)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/connection.py", line 525, in drain_events
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: while not self.blocking_read(timeout):
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/connection.py", line 531, in blocking_read
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return self.on_inbound_frame(frame)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/method_framing.py", line 53, in on_frame
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: callback(channel, method_sig, buf, None)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/connection.py", line 537, in on_inbound_method
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: return self.channels[channel_id].dispatch_method(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/abstract_channel.py", line 156, in dispatch_method
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: listener(*args)
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: File "/opt/pulsar/venv/lib/python3.9/site-packages/amqp/channel.py", line 293, in _on_close
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: raise error_for_code(
Mar 31 00:32:38 pulsar-re.grid.cesnet.cz pulsar[723660]: amqp.exceptions.PreconditionFailed: Basic.publish: (406) PRECONDITION_FAILED - message size 17306945 is larger than configured max size 16777216
