Skip to content

Conversation

@yuandrew
Copy link
Contributor

@yuandrew yuandrew commented Nov 13, 2025

What was changed

Make worker_registry aware of WorkerTaskType, to allow multiple workers without overlapping types to be registered on the same client/NS/TQ

Why?

We want to support this to minimize the breakage for users, since previously this scenario did not return an error. This is likely primarily used in tests or maybe code organization.

Checklist

  1. Closes

  2. How was this tested:

Added new tests

  1. Any docs updates needed?

Note

Enables registering workflow/activity/nexus-only workers on the same client/namespace/task queue when their task types don’t overlap, updates slot selection to use task types, and adds validations/tests.

  • Worker Registry (crates/client/src/worker_registry):
    • Store per-worker metadata via RegisteredWorkerInfo including build_id and WorkerTaskTypes.
    • Permit multiple workers on the same namespace/task_queue if their task_types don’t overlap (same build_id allowed if non-overlapping); reject empty capabilities.
    • Reserve WFT slots only from workers with enable_workflows.
    • Extend ClientWorker trait with worker_task_types().
  • Common (crates/common/src/worker.rs):
    • Introduce WorkerTaskTypes::overlaps_with helper; validate WorkerConfig.task_types and related invariants.
  • Core (crates/sdk-core/src/worker/mod.rs):
    • Pass config.task_types into client worker registration; implement worker_task_types() in ClientWorkerRegistrator.
    • Gate pollers/managers by task_types (workflows/activities/nexus) where applicable.
  • Client Raw (crates/client/src/raw.rs):
    • Tests updated to provide WorkerTaskTypes when registering workers for eager start path.
  • Tests:
    • Add coverage for coexistence of workflow-only/activity-only/nexus-only workers, rejection of overlapping/empty capabilities, and WFT reservation ignoring non-workflow workers.

Written by Cursor Bugbot for commit 3e0b754. This will update automatically on new commits. Configure here.

@yuandrew yuandrew requested a review from a team as a code owner November 13, 2025 01:55
});

// Determine worker capabilities based on configuration
let capabilities = temporalio_client::WorkerCapabilities {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% confident this is correct in how we detect workflow/nexus only workers

Copy link
Member

@cretz cretz Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrmm, I don't think langs tell Core today whether it will ever ask for workflow or Nexus work. This may be new (required) worker options. Never before has Core cared ahead of time which poll calls were going to be made.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually kinda want to switch to that model anyway. The whole implicit polling thing complicates shutdown and switching to an up-front model would make some nasty code go away (and actually save some resources too).

The only complication would be if we ever expect allowing people to dynamically start polling for a certain kind when they weren't before... but... I'm not sure that's a scenario I think matters at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only complication would be if we ever expect allowing people to dynamically start polling for a certain kind when they weren't before... but... I'm not sure that's a scenario I think matters at all.

That could potentially be something we allow users to configure from server commands?

But would it be that be that complicated? I guess that would mean more code is kept in callbacks and more mutex/atomic variables to track state for worker commands and shutdown and such

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good point. We can but I'm honestly not sure if we should. More knobs and complexity for unclear benefit. Though, maybe, it's kind of the same mechanism as worker "pause" which people have asked for.

I'm kinda worried about pausing/resuming polling (mostly for workflows) because shutdown is already really hard to get right and this is like extra-special shutdown.

But, regardless, I don't want you to deal with any of this right now. IMO it makes sense to add info about whether or not the worker polls for the 3 task kinds. For now, I'm perfectly fine with requiring lang to say up-front if the worker should start life polling for which task types. It can easily do that by just checking what is registered, like it does now to decide if it should make poll calls at all. If it calls poll on something it didn't say should be turned on, just immediately return PollError::Shutdown

All the other stuff we can address later

@yuandrew yuandrew force-pushed the worker-registry-workflow-activity branch from 2b33733 to 3e0b754 Compare November 17, 2025 20:35
@yuandrew yuandrew requested a review from cretz November 17, 2025 20:37
{
for existing_worker_info in existing_workers {
if existing_worker_info.build_id.as_ref() == build_id.as_ref()
&& task_types.overlaps_with(&existing_worker_info.task_types)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only concern here would be if langs do not start setting this value right away and therefore it'd look like it overlaps when it doesn't but I think the removal of the default from #1059 ensures that won't happen, so we're good.

@yuandrew yuandrew merged commit e0f86d7 into temporalio:master Nov 17, 2025
47 of 50 checks passed
@yuandrew yuandrew deleted the worker-registry-workflow-activity branch November 17, 2025 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants