From 421dbab591c525e3b032391ee53413a025a3d287 Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Thu, 5 Nov 2026 12:51:50 +0700 Subject: [PATCH 1/2] docs: a mask on a whole table, schema or catalog masks every column below it Co-Authored-By: Claude Opus 5.5 --- .../data-governance/data/masks-and-filters.md | 10 ++++- .../engines-and-policy/migrate.md | 1 + platform/deploy-operate/upgrade.md | 44 +++++++++++++++++++ 3 files changed, 53 insertions(+), 2 deletions(-) diff --git a/platform/data-governance/data/masks-and-filters.md b/platform/data-governance/data/masks-and-filters.md index dfd8c7d..2fd798a 100644 --- a/platform/data-governance/data/masks-and-filters.md +++ b/platform/data-governance/data/masks-and-filters.md @@ -76,6 +76,12 @@ A SQL boolean expression over the table's columns, for example `region = 'us'` o | **Path** | **Scope**: catalog (required), schema, table, and column for a mask. | A row filter by path needs catalog, schema and table together, and never a column; the dialog shows no column field and blocks saving until the three are set. A line below the fields reads *Applies to demo_warehouse.analytics.sessions*. | | **Classification** | **Classification**: pick a classification. | The rule applies to every table or column carrying the tag. The tag is stored by its key, shown as *stored as personal-data-pii*. **Change** picks another. A tag rule is catalog-unscoped. | +A mask by **Path** that leaves **Column** empty masks every column below the path: every column of the table, of every +table in the schema, or of every table in the catalog. On Trino and StarRocks it compiles to a masking policy with `*` at +each level left empty (since platform #NNN). A mask on a whole catalog leaves the catalog's own `information_schema` +alone, so `SHOW TABLES`, `SHOW COLUMNS`, `DESCRIBE` and JDBC metadata calls keep returning real names. The engine's +**Policies** tab lists such a mask as, for example, `wh.!information_schema.*.*` (`!` marks an excluded value). + ### Exempt The principals the rule does not apply to. The picker offers **user** and **group** only, and no **Exclude** list, because Ranger's mask and row-filter items cannot express a role or an exclusion. Egret refuses both at compile time. @@ -90,7 +96,7 @@ A StarRocks table has two hidden columns, `_file` (the object-store path of each mask, egret refuses them to a reader who does not see the real value in every masked column of the table. That reader's `SELECT *` on the table fails with error 5204 on `_file`. Name the columns you need, or read through a view over the table: a view shows the masked values and no path. A reader who sees the real value in every masked column can still use -`SELECT *`. +`SELECT *`. This covers every table that a mask on a whole table, schema or catalog reaches. A change to the masks takes effect on the next compile of the profile. To apply it at once, compile the profile with `POST /api/control-plane/perm/compile/{profile_id}`. @@ -143,7 +149,7 @@ The local platform admin `admin@egret.local` got the same masked values: platfor - A rule that selects by path is copied to the new name when its table or column is renamed, and the original stays on the old name; a rule that selects by tag follows because classifications do. A rule that selects by a pattern the new name escapes refuses the rename. See [Renames and drops](./grants.md#renames-and-drops). - A row filter or custom mask expression that names a renamed column is not rewritten (KI-321). - On official StarRocks images, masks and row filters do not reach a table's Iceberg metadata tables (`orders$files`, `orders$partitions`, …), which StarRocks serves to any login without asking Ranger, so column bounds, partition values and counts are readable regardless of rules (KI-340); they do not apply to prepared statements, so a Data API endpoint with parameters returns rows and values in the clear (KI-345); they do not apply to the HTTP SQL API or Arrow Flight SQL (KI-346), nor to `FILES()`, which reads the data files directly (KI-347); a materialized view stores its rows past its creator's masks and row filters (KI-348); and `EXPLAIN COSTS` shows a filtered person each column's lowest and highest value over the rows the filter hides (KI-350). Egret's patched image `ghcr.io/egret-data/egret-starrocks-fe` `4.1.7-egret.4`, the default for new engines, fixes all of them; on an official image, do not give StarRocks logins to people who must not see the values. -- On Trino, a table's metadata tables (`orders$files`, `orders$partitions`, …) follow its masks, row filters and denies (since platform #821): a person a mask, row filter or deny on the table restricts gets *Access Denied* on every one of them, and a person it exempts or does not restrict reads them as before. A tag rule or clearance does the same for each table that carries the classification. Profiles that egret's permission model does not compile get no such deny (KI-341). +- On Trino, a table's metadata tables (`orders$files`, `orders$partitions`, …) follow its masks, row filters and denies (since platform #821), including a mask on the table's schema or catalog (since platform #NNN): a person a mask, row filter or deny on the table restricts gets *Access Denied* on every one of them, and a person it exempts or does not restrict reads them as before. A tag rule or clearance does the same for each table that carries the classification. Profiles that egret's permission model does not compile get no such deny (KI-341). - On StarRocks, `SELECT *` on an Iceberg table shows the hidden `_file` and `_pos` columns to a user with a row filter (KI-337). List the columns instead. ## Further reading diff --git a/platform/data-governance/engines-and-policy/migrate.md b/platform/data-governance/engines-and-policy/migrate.md index d41adce..0398044 100644 --- a/platform/data-governance/engines-and-policy/migrate.md +++ b/platform/data-governance/engines-and-policy/migrate.md @@ -81,6 +81,7 @@ If the profiles changed since the preview, a migration reports *Migrating found - Migration is proven for Trino with plugin 2.5.0 or 2.8.0 and StarRocks with 2.8.0 or 2.9.0. For another plugin version the tab refuses. - A policy naming a group or role in a way the model cannot express is **Blocked**. - A policy that names a tag egret has no tag for is **Blocked**: the migration never creates tags. Create a tag whose key and name are both the policy's tag, or keep the policy hand-written (KI-326). +- A hand-written masking policy that names no `column` level, or stops higher, masks nothing on Trino or StarRocks and is **Blocked**. To mask every column below its path, set the missing levels to `*` in the policy editor; that takes effect at once, and the policy can then be migrated. A policy with a gap (for example, a catalog and a table but no schema) is told to name the missing level, or to set only that level to `*`. - A hand-written tag access policy is **Blocked** and never becomes a clearance; recreate it as a classification clearance (KI-327). - Policies egret's own plumbing writes (engine delegation, admin access, query limits and the policies **Seed default policies** adds) are **Locked** and are not rewritten (KI-328). diff --git a/platform/deploy-operate/upgrade.md b/platform/deploy-operate/upgrade.md index e287b40..e412441 100644 --- a/platform/deploy-operate/upgrade.md +++ b/platform/deploy-operate/upgrade.md @@ -501,6 +501,50 @@ There is no migration and no API change. differs, recompile the profile first ([`POST /api/control-plane/perm/compile/{service_id}`](../reference/rest-api/perm.md)) to bring them in line, then migrate again. +### Masks on a whole table, schema or catalog take effect (platform #NNN) + +Before platform #NNN, a mask rule by path that left **Column** empty compiled without the column level, and Trino and +StarRocks applied it to no column. Those columns came back unmasked although the rule was listed as live. From #NNN the +rule masks every column below its path, as the rule dialog says. + +There is no migration. At its first start the new version recompiles each policy profile that still serves such a mask, +and the plugins download the new policies once. Later starts skip profiles already repaired. + +**Before upgrading**, run `GET /api/control-plane/perm/overlaps` (or open each engine's **Overlaps** tab). It lists every +pair of masks or row filters stored in a profile's download that can decide the same request two ways. Resolve every +pair in which one member is a mask by path with an empty **Column**: delete one of the two, or narrow one so they no +longer meet. Such a pair has no effect today, because one member masks nothing. After the upgrade, recompiling the +profile would make the pair real, so egret refuses the recompile: + +- the profile keeps serving its previous compile, so its whole-table, schema and catalog masks stay **inert**; +- every write that recompiles the profile is refused with a 409 conflict naming the pair until the pair is resolved. + That includes grants, rules, catalog grants, workspace membership and role changes on the profile's catalogs, and + **any classification (tag) rule write in the installation**, which recompiles every profile; +- the server logs the refusal at error level at every start. On the Governance overview, **Needs attention** lists a + **Critical** finding, *The last compile of 1 profile failed; engines keep the previous compile. The refusal names rules + A and B.*, with the profile's line reading *Refused at boot: masks with no column stay inert (previous compile + served); writes recompiling this profile are refused until these rules are resolved.* and links to both rules. + +The pair's coordinate on the **Overlaps** tab does not say which member has an empty Column; open each rule to see. If +the recompile fails because of a database error or a lock timeout, the line reads *Boot recompile failed: masks with no +column stay inert (previous compile served); the next boot retries.*, and writes are not blocked by it. Any other +refusal, such as an expansion ceiling, shows the same line but keeps refusing writes until it is fixed, as it did +before the upgrade. + +**What readers notice after the upgrade:** + +- columns under a whole-table, schema or catalog mask are masked for everyone the rule does not exempt; +- on Trino, the metadata tables (`$files`, `$partitions`, `$snapshots`, …) of every table under a schema or catalog mask + are denied to the same people; +- on StarRocks, `SELECT *` on those tables is refused to them (error 5204); +- a rule migrated earlier from a hand-written masking policy that named no column now masks; the original never did. + Review migrated mask rules with an empty Column. + +A mask on a whole catalog does not apply to the catalog's `information_schema`. The engine's **Policies** tab lists it +as, for example, `wh.!information_schema.*.*` (`!` marks an excluded value). A mask on a whole table lists as +`wh.sales.orders.*`. A hand-written masking policy that names no column is left as it is and still masks nothing; +migrating it is refused until its missing levels are set (see [Migrate policies](../data-governance/engines-and-policy/migrate.md)). + ## Rolling back There is no supported in-place rollback. Restore the database from the backup taken in step 2, then start the previous From 98d7d0a5e4ef35c5a2680fa145b4a9c5dce8939a Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Thu, 5 Nov 2026 12:52:41 +0700 Subject: [PATCH 2/2] docs: name platform #837 Co-Authored-By: Claude Opus 5.5 --- platform/data-governance/data/masks-and-filters.md | 4 ++-- platform/deploy-operate/upgrade.md | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/platform/data-governance/data/masks-and-filters.md b/platform/data-governance/data/masks-and-filters.md index 2fd798a..85e05eb 100644 --- a/platform/data-governance/data/masks-and-filters.md +++ b/platform/data-governance/data/masks-and-filters.md @@ -78,7 +78,7 @@ A SQL boolean expression over the table's columns, for example `region = 'us'` o A mask by **Path** that leaves **Column** empty masks every column below the path: every column of the table, of every table in the schema, or of every table in the catalog. On Trino and StarRocks it compiles to a masking policy with `*` at -each level left empty (since platform #NNN). A mask on a whole catalog leaves the catalog's own `information_schema` +each level left empty (since platform #837). A mask on a whole catalog leaves the catalog's own `information_schema` alone, so `SHOW TABLES`, `SHOW COLUMNS`, `DESCRIBE` and JDBC metadata calls keep returning real names. The engine's **Policies** tab lists such a mask as, for example, `wh.!information_schema.*.*` (`!` marks an excluded value). @@ -149,7 +149,7 @@ The local platform admin `admin@egret.local` got the same masked values: platfor - A rule that selects by path is copied to the new name when its table or column is renamed, and the original stays on the old name; a rule that selects by tag follows because classifications do. A rule that selects by a pattern the new name escapes refuses the rename. See [Renames and drops](./grants.md#renames-and-drops). - A row filter or custom mask expression that names a renamed column is not rewritten (KI-321). - On official StarRocks images, masks and row filters do not reach a table's Iceberg metadata tables (`orders$files`, `orders$partitions`, …), which StarRocks serves to any login without asking Ranger, so column bounds, partition values and counts are readable regardless of rules (KI-340); they do not apply to prepared statements, so a Data API endpoint with parameters returns rows and values in the clear (KI-345); they do not apply to the HTTP SQL API or Arrow Flight SQL (KI-346), nor to `FILES()`, which reads the data files directly (KI-347); a materialized view stores its rows past its creator's masks and row filters (KI-348); and `EXPLAIN COSTS` shows a filtered person each column's lowest and highest value over the rows the filter hides (KI-350). Egret's patched image `ghcr.io/egret-data/egret-starrocks-fe` `4.1.7-egret.4`, the default for new engines, fixes all of them; on an official image, do not give StarRocks logins to people who must not see the values. -- On Trino, a table's metadata tables (`orders$files`, `orders$partitions`, …) follow its masks, row filters and denies (since platform #821), including a mask on the table's schema or catalog (since platform #NNN): a person a mask, row filter or deny on the table restricts gets *Access Denied* on every one of them, and a person it exempts or does not restrict reads them as before. A tag rule or clearance does the same for each table that carries the classification. Profiles that egret's permission model does not compile get no such deny (KI-341). +- On Trino, a table's metadata tables (`orders$files`, `orders$partitions`, …) follow its masks, row filters and denies (since platform #821), including a mask on the table's schema or catalog (since platform #837): a person a mask, row filter or deny on the table restricts gets *Access Denied* on every one of them, and a person it exempts or does not restrict reads them as before. A tag rule or clearance does the same for each table that carries the classification. Profiles that egret's permission model does not compile get no such deny (KI-341). - On StarRocks, `SELECT *` on an Iceberg table shows the hidden `_file` and `_pos` columns to a user with a row filter (KI-337). List the columns instead. ## Further reading diff --git a/platform/deploy-operate/upgrade.md b/platform/deploy-operate/upgrade.md index e412441..5dda4fc 100644 --- a/platform/deploy-operate/upgrade.md +++ b/platform/deploy-operate/upgrade.md @@ -501,10 +501,10 @@ There is no migration and no API change. differs, recompile the profile first ([`POST /api/control-plane/perm/compile/{service_id}`](../reference/rest-api/perm.md)) to bring them in line, then migrate again. -### Masks on a whole table, schema or catalog take effect (platform #NNN) +### Masks on a whole table, schema or catalog take effect (platform #837) -Before platform #NNN, a mask rule by path that left **Column** empty compiled without the column level, and Trino and -StarRocks applied it to no column. Those columns came back unmasked although the rule was listed as live. From #NNN the +Before platform #837, a mask rule by path that left **Column** empty compiled without the column level, and Trino and +StarRocks applied it to no column. Those columns came back unmasked although the rule was listed as live. From #837 the rule masks every column below its path, as the rule dialog says. There is no migration. At its first start the new version recompiles each policy profile that still serves such a mask,