Skip to content

Add workers.celery.labels & workers.kubernetes.labels#65030

Open
Miretpl wants to merge 2 commits intoapache:mainfrom
Miretpl:add-workers-celery-k8s-labels
Open

Add workers.celery.labels & workers.kubernetes.labels#65030
Miretpl wants to merge 2 commits intoapache:mainfrom
Miretpl:add-workers-celery-k8s-labels

Conversation

@Miretpl
Copy link
Copy Markdown
Contributor

@Miretpl Miretpl commented Apr 10, 2026

related: #28880

This PR introduces two new fields: workers.celery.labels and workers.kubernetes.labels. The workers.labels field is now deprecated.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds executor-specific worker label configuration to the Airflow Helm chart by introducing workers.celery.labels and workers.kubernetes.labels, and deprecating the legacy workers.labels key to support configuring Celery and Kubernetes workers independently (notably for CeleryKubernetesExecutor use cases).

Changes:

  • Introduce workers.celery.labels and workers.kubernetes.labels in values + JSON schema; mark workers.labels as deprecated.
  • Update Helm templates for Kubernetes worker artifacts / pod-template-file to prefer workers.kubernetes.labels with fallback to deprecated workers.labels.
  • Expand Helm tests to cover labels coming from both the deprecated location and the new executor-specific locations.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
helm-tests/tests/helm_tests/airflow_core/test_worker.py Parametrizes label tests to validate deprecated vs executor-specific label behavior across worker resources.
helm-tests/tests/helm_tests/airflow_core/test_worker_sets.py Extends worker-set label overwrite coverage when labels are defined under workers.celery.
helm-tests/tests/helm_tests/airflow_core/test_pdb_worker.py Updates PDB label tests to cover deprecated + workers.celery.labels scenarios.
helm-tests/tests/helm_tests/airflow_aux/test_pod_template_file.py Updates pod-template-file label tests to cover deprecated + workers.kubernetes.labels scenarios.
chart/values.yaml Adds new values keys and annotates workers.labels as deprecated.
chart/values.schema.json Adds schema support for the new keys and updates the deprecated key’s description.
chart/templates/workers/worker-kubernetes-serviceaccount.yaml Uses workers.kubernetes.labels (fallback to deprecated workers.labels) for Kubernetes worker ServiceAccount labels.
chart/templates/NOTES.txt Adds a deprecation warning for workers.labels.
chart/newsfragments/65030.significant.rst Announces the deprecation and replacement keys.
chart/files/pod-template-file.kubernetes-helm-yaml Uses workers.kubernetes.labels (fallback to deprecated workers.labels) for pod-template-file labels.
chart/docs/customizing-labels.rst Adjusts examples in label customization docs (but currently drops the worker example).

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Deprecates workers.labels in the Helm chart in favor of two new, more specific settings: workers.celery.labels and workers.kubernetes.labels, and updates chart templates/tests/docs accordingly.

Changes:

  • Add workers.celery.labels and workers.kubernetes.labels to values.yaml and values.schema.json, and emit a deprecation warning for workers.labels.
  • Update Helm templates to prefer the new component-specific labels (falling back to deprecated workers.labels).
  • Expand Helm unit tests to validate label precedence/behavior across worker-related resources.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
helm-tests/tests/helm_tests/airflow_core/test_worker_sets.py Adds test coverage for workers.celery.labels behavior in worker sets.
helm-tests/tests/helm_tests/airflow_core/test_worker.py Parameterizes multiple label-source scenarios to validate precedence for worker resources.
helm-tests/tests/helm_tests/airflow_core/test_pdb_worker.py Updates PDB tests to cover new label configuration behavior.
helm-tests/tests/helm_tests/airflow_aux/test_pod_template_file.py Updates pod-template-file tests for workers.kubernetes.labels precedence.
chart/values.yaml Introduces new values keys and marks workers.labels as deprecated.
chart/values.schema.json Adds schema entries for the new label keys and marks workers.labels as deprecated.
chart/templates/workers/worker-kubernetes-serviceaccount.yaml Switches label merge source to prefer workers.kubernetes.labels (fallback to deprecated).
chart/templates/NOTES.txt Adds a deprecation warning message for workers.labels.
chart/newsfragments/65030.significant.rst Documents the deprecation of workers.labels in release notes.
chart/files/pod-template-file.kubernetes-helm-yaml Switches label merge source to prefer workers.kubernetes.labels (fallback to deprecated).
chart/docs/customizing-labels.rst Removes worker label example but does not yet show the new worker label keys.

@Miretpl Miretpl force-pushed the add-workers-celery-k8s-labels branch 2 times, most recently from a22d226 to 285e778 Compare April 12, 2026 12:54
@Miretpl Miretpl force-pushed the add-workers-celery-k8s-labels branch from 285e778 to 5d1b47d Compare April 12, 2026 12:56
@Miretpl Miretpl marked this pull request as ready for review April 12, 2026 12:57
@jscheffl jscheffl added the backport-to-chart/v1-2x-test Automatic backport to chart 1.2x maintenance branch label Apr 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart backport-to-chart/v1-2x-test Automatic backport to chart 1.2x maintenance branch kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants