Feat: Enable celery workers and beat

CHANGELOG.md

@@ -4,6 +4,8 @@
 
 ### Added
 
+- Support for Async Queries via Celery. Two new roles, `worker` and `beat` ([#724]).
+- Support for redis broker and results backend for celery workers in the CRD ([#724]).
 - BREAKING: Add required CLI argument and env var to set the image repository used to construct final product image names: `IMAGE_REPOSITORY` (`--image-repository`), eg. `oci.example.org/my/namespace` ([#726]).
 
 ### Changed
@@ -25,6 +27,7 @@
 [#719]: https://github.com/stackabletech/superset-operator/pull/719
 [#721]: https://github.com/stackabletech/superset-operator/pull/721
 [#722]: https://github.com/stackabletech/superset-operator/pull/722
+[#724]: https://github.com/stackabletech/superset-operator/pull/724
 [#726]: https://github.com/stackabletech/superset-operator/pull/726
 
 ## [26.3.0] - 2026-03-16

deploy/helm/chart_testing.yaml

@@ -1,3 +1,4 @@
+---
 remote: origin
 target-branch: main
 chart-dirs:

deploy/helm/superset-operator/templates/clusterrole-operator.yaml

@@ -64,6 +64,18 @@ rules:
     resourceNames:
       - {{ include "operator.name" . }}-clusterrole
   # StatefulSet created per role group. Applied via SSA and tracked for orphan cleanup.
+  - apiGroups:
+      - apps
+    resources:
+      - deployments
+    verbs:
+      - get
+      - create
+      - delete
+      - list
+      - patch
+      - update
+      - watch
   - apiGroups:
       - apps
     resources:

docs/modules/superset/pages/usage-guide/celery-async-queries.adoc

@@ -0,0 +1,43 @@
+= Async Queries via Celery
+:description: Schedule Apache Superset SQL queries on Celery workers.
+
+In order to support long running queries that exceed the web requests timeout, an asynchronous backend has to be configured.
+This requires two more roles to be configured besides the existing `nodes`. The (celery) `workers` and a `beat` for scheduling.
+
+The beat role is limited to `1` or `0` replicas only. Replicas greater than one are ignored and set to `1`.
+This avoids multiple beat instances scheduling the same tasks at the same time.
+
+If no beat or scheduling is required, async queries keep work without a beat instance.
+
+The following example shows additional required settings to enable async queries:
+
+[source,yaml]
+----
+spec:
+  clusterConfig:
+    celeryBroker:
+      redis:
+        host: superset-redis-master
+        port: 6379
+        credentialsSecretName: superset-redis-broker-credentials
+    celeryResultsBackend:
+      redis:
+        host: superset-redis-master
+        port: 6379
+        credentialsSecretName: superset-redis-results-backend-credentials
+  nodes:
+    roleGroups:
+      default:
+        replicas: 1
+  workers:
+    roleGroups:
+      default:
+        replicas: 2
+  beat: # optional
+    roleGroups:
+      default:
+        # Only 1 or 0 instances possible.
+        replicas: 1
+----
+
+This is not a complete example. The main `nodes` role for the webserver and e.g. the metadata database are missing.

docs/modules/superset/pages/usage-guide/database-connections.adoc

@@ -3,8 +3,11 @@
 
 Superset requires a metadata database for storing slices, connections, tables, dashboards and other metadata.
 The actual connection string is calculated by the operator so that the user does not need to remember the exact structure.
+Using async queries with celery workers also requires a broker and a results backend.
 
-== Typed connections
+== Metadata database
+
+=== PostgreSQL
 
 [source,yaml]
 ----
@@ -19,7 +22,7 @@ spec:
 <1> A reference to one of the supported database backends (e.g. `postgresql`).
 <2> A reference to a Secret which must contain the two keys `username` and `password`.
 
-== Generic connections
+=== Generic connections
 
 Alternatively, these connections can also be defined in full in a referenced Secret:
 
@@ -33,3 +36,75 @@ spec:
 ----
 
 <1> A reference to a Secret which must contain the single key `connectionUrl` e.g. `postgresql://superset:superset@superset-postgresql/superset`
+
+== Broker
+
+The broker queue is required to schedule or pass queries on the celery workers.
+
+=== Redis
+
+A redis broker can be configured in the `clusterConfig`:
+
+[source,yaml]
+----
+spec:
+  clusterConfig:
+    celeryBroker:
+      redis:
+        host: redis-master
+        port: 6379
+        credentialsSecretName: superset-redis-credentials
+----
+
+=== Generic connections
+
+Alternatively, these connections can also be defined in full in a referenced Secret:
+
+[source,yaml]
+----
+spec:
+  clusterConfig:
+    celeryBroker:
+      generic:
+        connectionUrlSecretName: superset-redis-broker-url # <1>
+----
+
+<1> A reference to a Secret which must contain the single key `connectionUrl` e.g. `redis://:redis@redis-master/0`
+
+== Results Backend
+
+Celery workers store their results in the configured results backend.
+
+=== Redis
+
+A redis broker can be configured in the `clusterConfig`:
+
+[source,yaml]
+----
+spec:
+  clusterConfig:
+    celeryResultsBackend:
+      redis:
+        host: redis-master
+        port: 6379
+        credentialsSecretName: superset-redis-credentials
+----
+
+=== S3
+
+Currently not supported.
+
+=== Generic connections
+
+Alternatively, these connections can also be defined in full in a referenced Secret:
+
+[source,yaml]
+----
+spec:
+  clusterConfig:
+    celeryResultsBackend:
+      generic:
+        connectionUrlSecretName: superset-redis-results-backend-url # <1>
+----
+
+<1> A reference to a Secret which must contain the single key `connectionUrl` e.g. `redis://:redis@redis-master/0`