Skip to content

Start the fanout worker when only the fanout queue is configured #1389

Description

@dahlia

Motivation

FederationImpl._startQueueInternal() starts with a guard that predates the fanout queue:

if (this.inboxQueue == null && this.outboxQueue == null) return;

FederationQueueOptions.fanout was added in 1.5.0 (#220), but the guard was not updated. A federation configured with queue: { fanout: ... } therefore returns before the fanout branch, while ContextImpl.sendActivityInternal() still enqueues the FanoutMessage afterward. Nothing calls fanoutQueue.listen(), so the messages stay in the backend unprocessed.

The failure hides because small sends do not use the fanout queue. With the default fanout: "auto" and fewer than five inboxes (FANOUT_THRESHOLD), or with fanout: "skip", activities are delivered directly because outboxQueue is null. Adding inbox or outbox to the configuration accidentally starts the fanout worker too, which makes the bug look intermittent.

Nothing is logged either. The early return precedes logger.debug("Starting a fanout task worker."), and _startQueueAutomatically() only logs rejections. Federation.startQueue(data, { queue: "fanout" }) has the same problem: it resolves successfully without starting anything, which is easy to mistake for a healthy worker during startup.

Reproduction

Configure a federation with only a fanout queue, leaving manuallyStartQueue at its default:

const federation = createFederation<void>({
  kv: new MemoryKvStore(),
  queue: { fanout: new InProcessMessageQueue() },
});

Then start the worker explicitly:

await federation.startQueue(undefined, { queue: "fanout" });

or send an activity through the fanout path:

await ctx.sendActivity(sender, recipients, activity, { fanout: "force" });

fanoutQueue.listen() is never called. The following check fails on the affected versions; the listener resolves immediately so the check terminates on fixed versions too:

let listenCalls = 0;
const fanout: MessageQueue = {
  enqueue: () => Promise.resolve(),
  listen() {
    listenCalls++;
    return Promise.resolve();
  },
};

const federation = createFederation<void>({
  kv: new MemoryKvStore(),
  queue: { fanout },
  manuallyStartQueue: true,
});
await federation.startQueue(undefined, { queue: "fanout" });

assertEquals(listenCalls, 1); // fails: the actual value is 0

Expected behavior

startQueue() without a queue selector, or with { queue: "fanout" }, should call fanoutQueue.listen() once. Automatic startup after a fanout enqueue should do the same. FederationStartQueueOptions.queue documents that omitting the selector starts the inbox, outbox, and fanout queues.

Fix

Include fanoutQueue in the guard:

-    if (this.inboxQueue == null && this.outboxQueue == null) return;
+    if (
+      this.inboxQueue == null && this.outboxQueue == null &&
+      this.fanoutQueue == null
+    ) {
+      return;
+    }

Deleting the guard also works because the rest of the function is null-safe and would await an empty promise list; keeping the fanout check preserves the fast path.

Fix 2.0-maintenance first, then merge forward through 2.1-maintenance, 2.2-maintenance, and 2.3-maintenance. 2.4-maintenance and main need no change: b1cd1c372 widened the guard for the task queue and afdaa6136 replaced it with a role list, both first released in 2.4.0.

Add regression tests with only queue: { fanout }: one asserting that startQueue(..., { queue: "fanout" }) starts the listener, and one asserting the same for automatic startup after sendActivity(..., { fanout: "force" }). Both fail on the affected branches. Add a changelog fragment for the next 2.0.x release.

Scope

Affected versions are 2.0.x (including the unreleased 2.0.36), 2.1.x through 2.1.30, 2.2.x through 2.2.19, and 2.3.x through 2.3.14. 2.4.0 and later are not affected.

Cloudflare Workers deployments that set manuallyStartQueue: true and consume messages through Federation.processQueuedTask() are unaffected, since they never rely on _startQueueInternal() for fanout.

1.5.0 through 1.11.x contain the same guard as well, although the 1.x maintenance branches have not been updated since January 2026.

Metadata

Metadata

Assignees

Labels

Type

Fields

Priority

None yet

Effort

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions