Add workers.celery.labels & workers.kubernetes.labels#65030
Add workers.celery.labels & workers.kubernetes.labels#65030Miretpl wants to merge 2 commits intoapache:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds executor-specific worker label configuration to the Airflow Helm chart by introducing workers.celery.labels and workers.kubernetes.labels, and deprecating the legacy workers.labels key to support configuring Celery and Kubernetes workers independently (notably for CeleryKubernetesExecutor use cases).
Changes:
- Introduce
workers.celery.labelsandworkers.kubernetes.labelsin values + JSON schema; markworkers.labelsas deprecated. - Update Helm templates for Kubernetes worker artifacts / pod-template-file to prefer
workers.kubernetes.labelswith fallback to deprecatedworkers.labels. - Expand Helm tests to cover labels coming from both the deprecated location and the new executor-specific locations.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| helm-tests/tests/helm_tests/airflow_core/test_worker.py | Parametrizes label tests to validate deprecated vs executor-specific label behavior across worker resources. |
| helm-tests/tests/helm_tests/airflow_core/test_worker_sets.py | Extends worker-set label overwrite coverage when labels are defined under workers.celery. |
| helm-tests/tests/helm_tests/airflow_core/test_pdb_worker.py | Updates PDB label tests to cover deprecated + workers.celery.labels scenarios. |
| helm-tests/tests/helm_tests/airflow_aux/test_pod_template_file.py | Updates pod-template-file label tests to cover deprecated + workers.kubernetes.labels scenarios. |
| chart/values.yaml | Adds new values keys and annotates workers.labels as deprecated. |
| chart/values.schema.json | Adds schema support for the new keys and updates the deprecated key’s description. |
| chart/templates/workers/worker-kubernetes-serviceaccount.yaml | Uses workers.kubernetes.labels (fallback to deprecated workers.labels) for Kubernetes worker ServiceAccount labels. |
| chart/templates/NOTES.txt | Adds a deprecation warning for workers.labels. |
| chart/newsfragments/65030.significant.rst | Announces the deprecation and replacement keys. |
| chart/files/pod-template-file.kubernetes-helm-yaml | Uses workers.kubernetes.labels (fallback to deprecated workers.labels) for pod-template-file labels. |
| chart/docs/customizing-labels.rst | Adjusts examples in label customization docs (but currently drops the worker example). |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Deprecates workers.labels in the Helm chart in favor of two new, more specific settings: workers.celery.labels and workers.kubernetes.labels, and updates chart templates/tests/docs accordingly.
Changes:
- Add
workers.celery.labelsandworkers.kubernetes.labelstovalues.yamlandvalues.schema.json, and emit a deprecation warning forworkers.labels. - Update Helm templates to prefer the new component-specific labels (falling back to deprecated
workers.labels). - Expand Helm unit tests to validate label precedence/behavior across worker-related resources.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| helm-tests/tests/helm_tests/airflow_core/test_worker_sets.py | Adds test coverage for workers.celery.labels behavior in worker sets. |
| helm-tests/tests/helm_tests/airflow_core/test_worker.py | Parameterizes multiple label-source scenarios to validate precedence for worker resources. |
| helm-tests/tests/helm_tests/airflow_core/test_pdb_worker.py | Updates PDB tests to cover new label configuration behavior. |
| helm-tests/tests/helm_tests/airflow_aux/test_pod_template_file.py | Updates pod-template-file tests for workers.kubernetes.labels precedence. |
| chart/values.yaml | Introduces new values keys and marks workers.labels as deprecated. |
| chart/values.schema.json | Adds schema entries for the new label keys and marks workers.labels as deprecated. |
| chart/templates/workers/worker-kubernetes-serviceaccount.yaml | Switches label merge source to prefer workers.kubernetes.labels (fallback to deprecated). |
| chart/templates/NOTES.txt | Adds a deprecation warning message for workers.labels. |
| chart/newsfragments/65030.significant.rst | Documents the deprecation of workers.labels in release notes. |
| chart/files/pod-template-file.kubernetes-helm-yaml | Switches label merge source to prefer workers.kubernetes.labels (fallback to deprecated). |
| chart/docs/customizing-labels.rst | Removes worker label example but does not yet show the new worker label keys. |
a22d226 to
285e778
Compare
285e778 to
5d1b47d
Compare
related: #28880
This PR introduces two new fields:
workers.celery.labelsandworkers.kubernetes.labels. Theworkers.labelsfield is now deprecated.Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.