fix(metadata-protocol)!: a package-less write onto a packaged view, dashboard, report, translation or email template is refused, naming the .custom pair (ADR-0146 D11)

.changeset/28576-packageless-overlay-write-closed.md

@@ -0,0 +1,52 @@
+---
+'@objectstack/metadata-protocol': minor
+'@objectstack/spec': patch
+'@objectstack/skills': patch
+'@object-ui/app-shell': patch
+---
+
+fix(metadata-protocol)!: a write naming no package onto an installed package's view, dashboard, report, translation or email template is refused where `os migrate packageless-overlays` can move its row, naming the base's `.custom` pair and the command
+
+Clause-②: yes (narrowing) — the metadata save, publish, delete and rollback doors refuse a write that names no package onto an item of the five overlay types that one installed read-only package ships, where the ceremony would classify the row (or the first write's body) `movable` and its move can succeed; they accepted it before as a package-unbound Setup row. Every row the ceremony lists and does not move keeps that write. One write is widened: a rollback of the `.custom` pair's own declared row of an item its base ships (#28644 item 1) is judged as the pair's declared override, where the store sealed it as an artifact override before, on every type the pair can override (a page or flow pair row included).
+
+Metadata-protocol: narrowed (the `allowEnvironmentOverlay` flag of `view`, `dashboard`, `report`, `translation` and `email_template` no longer opens a package-less write onto a packaged item the ceremony can move; it stays the predicate the migration ceremony, the read of an unmigrated row and every row the ceremony leaves use)
+
+Skills-docs: updated here — `skills/objectstack-platform/SKILL.md`'s ADR-0146 paragraph said "a package-unbound Setup row still wins over the base's until D11"; it now says such a row from before D11 wins over the base's until `os migrate packageless-overlays` moves it, and that its package-less save, reset and rollback are refused once the command can move it. To stay under the bundle's token ceiling, the same paragraph drops "the base's is unchanged" after the export sentence (which "replaces the base's document whole" already implies) and the aside that the other overridable types wait for their doors (the sentence already names the six types the read honours today).
+
+<!-- adr-0087: registered packageless-overlay-write-closed -->
+
+**BREAKING** write narrowing at the metadata doors (`@objectstack/metadata-protocol`), shipped as `minor` during the launch window.
+
+**Upgrade, then run the ceremony.** The command ships in this release. On each deployment with Setup customizations of a package's items, upgrade, run `os migrate packageless-overlays` (preview) with the flags and environment the deployment boots with (`--preset` and `--dev` mean what they mean to `os serve`), then `os migrate packageless-overlays --apply`. Nothing breaks at the upgrade: between the upgrade and `--apply`, every customization is served as before, and the package-less save, reset, publish and rollback of one the command can move are refused, naming the command. Done when the preview lists no movable row.
+
+The preview lists five dispositions; only `movable` rows are moved, and every other row keeps its package-less write:
+
+- `movable`: `--apply` moves it into the pair. A move that ends `failed` keeps the row as it was; fix what the error names and re-run. When the error names the pair (it cannot be ensured, or it refuses the row) or the base's missing version, the row also keeps its package-less write and may be left.
+- `multi_shipped` (several packages ship the name): delete it naming no package, then save it into the pair of the package it customizes if it is still wanted (the pair's override is then served at that base's address, `?package=<base>`; a read naming no package is not pinned to it while several packages ship the name).
+- `no_pair` (the base's id forms no pair): nothing to do.
+- `foreign_declaration` (its `overrides` names another package): remove the key with a save naming no package, then re-run: the move declares the right override. (A package-less save that keeps a corrected `overrides` is refused by the override write gate.)
+- `shadowed` (the pair already holds the item; the pair's row is served when it declares the override, otherwise this row still is): nothing to do to keep the pair's declared override; or restore the package original (`DELETE …?package=<base>.custom`) and re-run to move this one.
+
+### Migration: FROM → TO
+
+FROM, a customization of the installed package `com.acme.crm`'s view written with no package:
+
+```http
+PUT /api/v1/meta/view/account_list
+{ "name": "account_list", "label": "My accounts" }
+```
+
+TO, the same customization written into the base's paired `.custom`, declaring what it overrides:
+
+```http
+PUT /api/v1/meta/view/account_list?package=com.acme.crm.custom
+{ "name": "account_list", "label": "My accounts", "overrides": { "package": "com.acme.crm", "basedOn": "1.4.0" } }
+```
+
+To restore the package original, FROM `DELETE /api/v1/meta/view/account_list` TO `DELETE /api/v1/meta/view/account_list?package=com.acme.crm.custom`. `basedOn` is the base's installed `manifest.version`. The pair is created by `os migrate packageless-overlays --apply`, or by Studio's Customize.
+
+- **Now refused.** A save (draft or publish mode), a publish, a delete of the active item, or a rollback whose restored row is package-less, with no `?package=` (or an empty one), of a `view`, `dashboard`, `report`, `translation` or `email_template` that exactly one installed read-only package ships, where the item's stored package-less row in that state (or, with none, the saved body) is one the ceremony classifies `movable`, the pair can be ensured (the base has a namespace and a SemVer version), and the override write gate admits that row into the pair. This covers an unmigrated `movable` row, and the first package-less write of an item the pair already holds (a moved one). Each answers `403 NOT_OVERRIDABLE`, writes nothing, and its message names the item, the shipping package, the pair (`?package=<base>.custom`), the `overrides` declaration and `os migrate packageless-overlays`; a rollback's message says to save a version from before the move into the pair, declaring it. The metadata item page shows that message as sent when its URL holds no `?package=`. The refusal comes before every gate that reads a stored row's lock, so it is the same with or without `overrides` in the body.
+- **Still open.** Every write naming `?package=<base>.custom` (the pair's save, publish, delete and rollback, judged by the override write gate as before); a rollback of the pair's own declared row, which is judged as the pair's override; a write of a name no installed package ships (environment metadata); and every package-less write the ceremony would not move, as before this release: a name several read-only packages ship (`multi_shipped`); a base whose id forms no pair (`no_pair`); a base whose pair cannot be ensured (no namespace, or a version that is not SemVer); a stored package-less row whose `overrides` names another package (`foreign_declaration`) or beside the pair's row of the item (`shadowed`; its package-less writes are addressed to the pair's row as before, so its package-less delete restores the package original); and an `email_template` locale the base does not ship (the pair refuses it, #28777; the ceremony's move of it is #28904). Discarding a draft stays open.
+- **Unchanged.** A package-less row stored before this release is still served exactly as before, by every read. The ceremony still moves it: it writes into the pair, never package-less.
+- **Studio.** The read-only item notice no longer links to the package-less Setup overlay (`@object-ui/app-shell`).
+- **Removed with it.** #28413's interim allowance, by which a package-less save that landed on a moved row kept that row's `overrides` block, is gone: such a save is refused like any other.

packages/cli/src/commands/migrate/packageless-overlays.ts

@@ -58,17 +58,23 @@ function reasonOf(row: CensusRow): string {
   switch (row.disposition) {
     case 'multi_shipped':
       return `several packages ship this name (${row.shippers.join(', ')}), so it overlays none of them for certain. `
-        + 'Left as it is: move it by hand into the pair of the package it customizes, or delete it.';
+        + 'Left as it is: delete it (a delete naming no package removes it while no package holds a row of '
+        + 'this name), then, to keep the customization, save it into the pair of the package it customizes '
+        + '(served then at that package\'s address, ?package=<base>).';
     case 'no_pair':
       return `'${row.base}' has no paired .custom package id, so there is nowhere to move it. Left as it is.`;
     case 'foreign_declaration':
       return `its body already carries an \`overrides\` key that does not name '${row.base}'. Left as it is: `
-        + 'correct or remove the key, then re-run.';
+        + 'remove the key with a save naming no package, then re-run: the move declares the right override.';
     case 'shadowed':
-      return `'${row.pair}' already ${row.shadowDeclares ? 'overrides this item (a customization made in Studio)' : 'holds a row of this name'}, `
-        + 'and that row is the one served — this row is never merged into it. Two ways out: keep the Studio '
-        + `customization and delete this package-less row (id ${row.id}), or use "Restore package original" in `
-        + `Studio to delete '${row.pair}''s row and re-run this command to move this one.`;
+      return (row.shadowDeclares
+        ? `'${row.pair}' already overrides this item (a customization made in Studio), and that row is the one `
+          + 'served — this row is never merged into it. Left as it is: to keep the Studio customization, nothing '
+          + 'is owed (this row is not served while that one stands); to make this row the customization instead, '
+        : `'${row.pair}' already holds a row of this name that declares no override, so this row is still the one `
+          + 'served, and the ceremony does not move it beside that row. Left as it is: to have it moved, ')
+        + `use "Restore package original" in Studio to delete '${row.pair}''s row, then re-run this command to `
+        + 'move this one.';
     default:
       return '';
   }

packages/metadata-protocol/src/migrations/overlay-index.ts

@@ -501,6 +501,11 @@ function overlayDiscriminatorOf(type: string): { field: string; canonical: strin
     return ITEM_KEY_DISCRIMINATORS[PLURAL_TO_SINGULAR[type] ?? type];
 }
 
+/** The body field that names `type`'s bundle member (`email_template`: `locale`), or `undefined` for a type with none. */
+export function overlayMemberField(type: string): string | undefined {
+    return overlayDiscriminatorOf(type)?.field;
+}
+
 /**
  * The value {@link OVERLAY_MEMBER_COLUMN} holds for the member a caller names.
  *

packages/metadata-protocol/src/protocol-customization-package.ts

@@ -128,7 +128,7 @@ export interface CustomizationPackageHost {
 }
 
 /** The registry slice the door and the predicate read. */
-interface PairRegistry {
+export interface PairRegistry {
     getPackage(id: string): PairPackageRow | undefined;
     getNamespaceOwners?(namespace: string): string[];
 }
@@ -289,34 +289,41 @@ export async function ensureCustomizationPackage(
     }
 }
 
-/** One judgement of the registry, and the create when the pair is absent. */
-async function ensurePairOnce(
-    this: CustomizationPackageHost,
+/**
+ * [ADR-0146 D11, #28576] Why the ensure door would refuse to ensure `baseId`'s
+ * pair on this registry now, or `undefined` when it would answer the existing
+ * pair or create it. ONE reading of the door's registry-decidable refusals:
+ * {@link ensurePairOnce} throws the error this answers, and the close of the
+ * package-less write (`packagelessOverlayWriteRefusal`) leaves a name's
+ * package-less write open where it answers one — `os migrate
+ * packageless-overlays` could not move such a row (its step 1 fails). Pure
+ * over the registry; it writes nothing.
+ */
+export function customizationPairEnsureRefusal(
+    engine: ObjectStackProtocolImplementation['engine'],
     registry: PairRegistry,
     baseId: string,
     pairId: string,
-): Promise<EnsureCustomizationPackageResponse> {
+): Error | undefined {
     const base = registry.getPackage(baseId);
     if (!base) {
-        throw refusal(404, `Package '${baseId}' not found: a paired customization package is ensured for an installed `
+        return refusal(404, `Package '${baseId}' not found: a paired customization package is ensured for an installed `
             + 'base.');
     }
     // ADR-0070 D2's one predicate. A writable base is the customer's own and
     // takes their changes directly; the pair exists for a base the customer
     // cannot author into.
-    if (isWritablePackage(this.engine, baseId)) {
-        throw refusal(409, `Package '${baseId}' is writable: customize it directly. A paired customization package `
+    if (isWritablePackage(engine, baseId)) {
+        return refusal(409, `Package '${baseId}' is writable: customize it directly. A paired customization package `
             + `('${pairId}') holds the changes to a read-only base — one provided by code or an installed app `
             + '(ADR-0146 D3).');
     }
     const namespace = customizationNamespace(base.manifest?.namespace);
     if (!namespace.ok) {
-        throw refusal(409, `Cannot ensure '${pairId}': ${noPairNamespaceClause(baseId, base.manifest?.namespace, namespace.reason)} `
+        return refusal(409, `Cannot ensure '${pairId}': ${noPairNamespaceClause(baseId, base.manifest?.namespace, namespace.reason)} `
             + '(ADR-0146 D3).');
     }
-
-    // Idempotent: an existing pair is returned unchanged, or refused naming
-    // what it lacks — ⛔ never adopted, never rewritten.
+    // An existing pair is returned unchanged, or refused naming what it lacks.
     const existing = registry.getPackage(pairId);
     if (existing) {
         const lacks: string[] = [];
@@ -326,24 +333,16 @@ async function ensurePairOnce(
         }
         const declaredRange = declaredRangeOn(existing.manifest, baseId);
         if (declaredRange === undefined) lacks.push(`a dependency on '${baseId}' in manifest.dependencies`);
-        if (lacks.length > 0 || declaredRange === undefined) {
-            throw refusal(409, `Package '${pairId}' is installed but is not the paired customization package of `
+        if (lacks.length > 0) {
+            return refusal(409, `Package '${pairId}' is installed but is not the paired customization package of `
                 + `'${baseId}': it lacks ${lacks.join(', and ')}. It is not adopted or rewritten — delete it, or `
                 + 'install a manifest over it that declares both, then ensure the pair again (ADR-0146 D3).');
         }
-        return {
-            basePackageId: baseId,
-            packageId: pairId,
-            namespace: namespace.value,
-            dependencyRange: declaredRange,
-            created: false,
-            writable: isWritablePackage(this.engine, pairId),
-        };
+        return undefined;
     }
-
     const range = customizationDependencyRange(base.manifest?.version);
     if (!range.ok) {
-        throw refusal(409, `Cannot ensure '${pairId}': package '${baseId}' is at version `
+        return refusal(409, `Cannot ensure '${pairId}': package '${baseId}' is at version `
             + `'${String(base.manifest?.version)}', which is not a SemVer 2.0.0 version, so the pair's dependency `
             + 'range — the major of the base\'s version — cannot be declared (ADR-0146 D3).');
     }
@@ -352,12 +351,44 @@ async function ensurePairOnce(
     // would read.
     const owner = registry.getNamespaceOwners?.(namespace.value)?.find((id) => id !== pairId);
     if (owner !== undefined) {
-        throw refusal(422, `Cannot ensure '${pairId}': its namespace '${namespace.value}' is already owned by package `
+        return refusal(422, `Cannot ensure '${pairId}': its namespace '${namespace.value}' is already owned by package `
             + `'${owner}'. The pair's namespace is the base's followed by '_custom' and has no alternative; uninstall `
             + `'${owner}' or move it off that namespace, then ensure the pair again (ADR-0146 D3).`,
         'NAMESPACE_CONFLICT');
     }
+    return undefined;
+}
 
+/** One judgement of the registry, and the create when the pair is absent. */
+async function ensurePairOnce(
+    this: CustomizationPackageHost,
+    registry: PairRegistry,
+    baseId: string,
+    pairId: string,
+): Promise<EnsureCustomizationPackageResponse> {
+    // Every refusal the registry decides, read once ({@link customizationPairEnsureRefusal}).
+    const refused = customizationPairEnsureRefusal(this.engine, registry, baseId, pairId);
+    if (refused) throw refused;
+    const base = registry.getPackage(baseId)!;
+    const namespace = customizationNamespace(base.manifest?.namespace);
+    if (!namespace.ok) throw new Error('unreachable: the namespace was judged above');
+
+    // Idempotent: an existing pair is returned unchanged — ⛔ never adopted,
+    // never rewritten (its refusal is above).
+    const existing = registry.getPackage(pairId);
+    if (existing) {
+        return {
+            basePackageId: baseId,
+            packageId: pairId,
+            namespace: namespace.value,
+            dependencyRange: declaredRangeOn(existing.manifest, baseId)!,
+            created: false,
+            writable: isWritablePackage(this.engine, pairId),
+        };
+    }
+
+    const range = customizationDependencyRange(base.manifest?.version);
+    if (!range.ok) throw new Error('unreachable: the version was judged above');
     const baseName = typeof base.manifest?.name === 'string' && base.manifest.name !== '' ? base.manifest.name : baseId;
     const manifest = {
         id: pairId,

packages/metadata-protocol/src/protocol-environment-rows.ts

@@ -241,6 +241,24 @@ export function shippingPackageIds(shipped: readonly unknown[]): ReadonlySet<str
     return shippers;
 }
 
+/**
+ * [ADR-0146 D11] The READ-ONLY code packages that ship a name, sorted: the
+ * {@link shippingPackageIds} of `shipped` less every package
+ * `isWritablePackage` answers for — a writable package's item is edited in
+ * place (ADR-0070) and has no pair (ADR-0146 D3), and the pair itself is
+ * writable. ONE reading for both halves of D11: the bases
+ * `os migrate packageless-overlays` classifies a package-less row against,
+ * and the bases whose package-less write the doors close
+ * (`packagelessOverlayWriteRefusal`), so the close never reaches a name the
+ * ceremony would call environment metadata, nor misses one it would move.
+ */
+export function readOnlyShippingPackages(
+    shipped: readonly unknown[],
+    isWritablePackage: (packageId: string) => boolean,
+): string[] {
+    return [...shippingPackageIds(shipped)].filter((id) => !isWritablePackage(id)).sort();
+}
+
 /**
  * [#28409, ADR-0146 D8 items 1–2] The one code package that ships a name, or
  * `undefined` when none or several do: the case in which