canary-environment: Fix permissions

test/canary-environment/README.md

@@ -172,8 +172,8 @@ steady state) doesn't land on one replica:
 |---------|-------|-------|
 | `qa_canary_environment_storage` | tpch + pg_cdc + mysql_cdc sources and their source-tables | |
 | `qa_canary_environment_upsert` | the Kafka loadgen `customer`/`sales` sources (and the `customer_tbl`/`sales_tbl` source-tables, which ingest on their source's cluster) | **disk-backed** — the UPSERT key→value state spills to RocksDB on local disk, which is what keeps the cold backlog re-read from OOMing |
-| `qa_canary_environment_sinks` | all sinks | isolates sink backfill from source snapshots |
-| `qa_canary_environment_compute` | materialized views + indexes | |
+| `qa_canary_environment_sinks` | all sinks, plus the testdrive-managed `public_table.table_mv` + its index | isolates sink backfill from source snapshots |
+| `qa_canary_environment_compute` | the model materialized views + indexes | blue/green-swapped by `promote` |
 
 They are pre-created manually (with environment-appropriate sizes) and not
 declared in the project, so `mz-deploy` never resizes them; `stage` clones the
@@ -182,6 +182,17 @@ their own — they ingest on their source's cluster — so a source's placement
 determines its tables' placement. The use of the `quickstart` cluster is
 strongly discouraged.
 
+`public_table.table_mv` lives on `qa_canary_environment_sinks` rather than
+`qa_canary_environment_compute`, even though it is a materialized view.
+`promote` swaps the compute cluster (`stage` clones it, `promote` swaps the
+clone in and drops the old cluster `CASCADE`). The model MVs survive because the
+deploy re-creates them in the clone every time, but `table_mv` is
+testdrive-managed and created once, so a swap would drop it (and its index, and
+the sinks reading it) with no recreation. That is exactly what broke
+`test/canary-load` once. Only clusters referenced by the blue/green MVs/views
+are staged and swapped; sinks are applied in place, so their cluster is never
+swapped and `table_mv` is safe there alongside the two sinks that read it.
+
 # Creating the objects
 
 To create or re-create the database objects:

test/canary-environment/mzcompose.py

@@ -424,11 +424,21 @@ def workflow_create(c: Composition, parser: WorkflowArgumentParser) -> None:
             > CREATE TABLE IF NOT EXISTS public_table.table (c INT)
             > GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE public_table.table TO "infra+qacanaryload@materialize.io"
 
+            # table_mv and its index go on qa_canary_environment_sinks, NOT the
+            # shared qa_canary_environment_compute. compute is mz-deploy's
+            # blue/green target: `promote` clones it, swaps the clone in, and
+            # drops the old cluster CASCADE. The model MVs survive because the
+            # deploy re-creates them in the clone every time, but table_mv is
+            # testdrive-managed and created once, so a swap would drop it (and its
+            # index, and the sinks reading it) with no recreation. Only clusters
+            # referenced by the blue/green MVs/views are staged and swapped; the
+            # sinks cluster is not (sinks are applied in place), so table_mv is
+            # safe there, alongside the two sinks that already read it.
             > CREATE MATERIALIZED VIEW IF NOT EXISTS public_table.table_mv
-              IN CLUSTER qa_canary_environment_compute
+              IN CLUSTER qa_canary_environment_sinks
               AS SELECT max(c) FROM public_table.table
             > CREATE INDEX IF NOT EXISTS table_mv_idx
-              IN CLUSTER qa_canary_environment_compute
+              IN CLUSTER qa_canary_environment_sinks
               ON public_table.table_mv (max)
             > GRANT ALL PRIVILEGES ON TABLE public_table.table_mv TO "infra+bot@materialize.com", "infra+qacanaryload@materialize.io"