diff --git a/.changeset/29373-export-withheld-maps.md b/.changeset/29373-export-withheld-maps.md new file mode 100644 index 00000000000..8c298da51ca --- /dev/null +++ b/.changeset/29373-export-withheld-maps.md @@ -0,0 +1,16 @@ +--- +'@objectstack/spec': minor +--- + +feat(spec): the package export manifest declares `withheldMaps`, the maps it withheld whole, and an unmanaged import accepts it + +Clause-②: yes (widening) +Metadata-protocol: added + +`GET /packages/:id/export` writes each item as a served read returns it, so a connector's header and environment-variable maps are already withheld whole by the connector redactor and the exported item simply lacks them. An import elsewhere could not tell such a map from one the package never had. The export contract now has a place to say so: + +- **`PackageExportManifestSchema` gains an optional fixed key `withheldMaps`**, a list of `{ type, name, path }`: `type` is `connector`, `name` is the connector item's name, and `path` is one of the four positions the redactor withholds whole, spelled as it reports them: `providerConfig.headers`, `providerConfig.defaultHeaders`, `providerConfig.transport.headers`, `providerConfig.transport.env`. Positions only: no header or variable name is listed, and an entry carrying any other key is refused. It is declared ahead of its producer: no export writes it and no install door reads it yet, and its describe opens `[PLANNED — not yet served]` until the runtime half serves it. Until then, an export's lack of the key says nothing about what was withheld. +- **`UNMANAGED_INSTALL_KEY_DISPOSITIONS` classifies `withheldMaps` as `door`**, so an unmanaged import accepts an export that carries it instead of refusing the whole package for an unrecognised key. The runtime half has the door read it to name each incomplete map; the new base never carries it. +- **The export's `label` is classified `door` too.** The export contract has always declared it, but the table never classified it, so an unmanaged import refused any export that carried one. The base's install manifest declares no `label`, so the base is named by the request's `targetName` or the source `name`, as before. + +The table's totality test now reads the export's fixed keys as a third schema a package body is read with, beside `ManifestSchema` and the stack definition, so a fixed key the export gains reddens a spec test on the PR that adds it. diff --git a/content/docs/references/api/package-lifecycle.mdx b/content/docs/references/api/package-lifecycle.mdx index 2b163f98475..af6ac1be5ab 100644 --- a/content/docs/references/api/package-lifecycle.mdx +++ b/content/docs/references/api/package-lifecycle.mdx @@ -43,6 +43,13 @@ declared a VERSION rollback against the live COMMIT-rollback path; `RollbackToPackageCommitResponseSchema` below is the true contract, authored after that retirement per the ruling's sequencing (3A). +Two members are not transcriptions but DECLARED AHEAD OF THEIR PRODUCER +(ADR-0049), each describe opening `[PLANNED — not yet served]` until the +change that serves it drops the marker: the `nav` arm of +`GetPackageCustomizationsResponseSchema.contributions` (#27928) and +`PackageExportManifestSchema.withheldMaps` (#28640 ruling ③A, served by +#29342). Their own docblocks say what is not served yet. + **Source:** `packages/spec/src/api/package-lifecycle.zod.ts` @@ -316,7 +323,7 @@ A field or flow another package contributes to one of the base's objects. | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **packages** | `({ id: string; name: string; version: string; label?: string } & Record)[]` | ✅ | Every package whose registry row's `source` is not `code` (the marketplace and authored packages, a `.custom` pair among them), enabled or not, each once and each as `GET /packages/:id/export` answers it, dependencies first: an entry comes after every entry its package's `manifest.dependencies` names, so a `.custom` pair comes after its base. Install the entries in this order; never re-sort them. No entry carries an enabled state, so every entry installs enabled. Empty when every package of the deployment is registered by its own code. | +| **packages** | `({ id: string; name: string; version: string; label?: string; … } & Record)[]` | ✅ | Every package whose registry row's `source` is not `code` (the marketplace and authored packages, a `.custom` pair among them), enabled or not, each once and each as `GET /packages/:id/export` answers it, dependencies first: an entry comes after every entry its package's `manifest.dependencies` names, so a `.custom` pair comes after its base. Install the entries in this order; never re-sort them. No entry carries an enabled state, so every entry installs enabled. Empty when every package of the deployment is registered by its own code. | ### Nested Shape: `PackageExportBundle.packages[number]` @@ -326,6 +333,7 @@ A field or flow another package contributes to one of the base's objects. | **name** | `string` | ✅ | The exported package's machine name. | | **version** | `string` | ✅ | The exported package's version. | | **label** | `string` | optional | Display label, when the package declares one. | +| **withheldMaps** | `{ type: 'connector'; name: string; path: Enum<'providerConfig.headers' \| 'providerConfig.defaultHeaders' \| …> }[]` | optional | [PLANNED — not yet served] Declared ahead of its producer (ADR-0049): no export writes this list yet and no install door reads it yet. Every map the export withheld WHOLE because a served read never carries it: a connector's header and environment-variable maps. The item is written with that map absent, so once served an import reads this list to tell an incomplete map from one the package never had, and names each one for an operator to re-enter. Positions only: no header or variable name is listed. Absence says nothing about what was withheld when the exporter predates this key; from an exporter that writes it, absence means nothing was withheld. | --- @@ -340,6 +348,15 @@ A field or flow another package contributes to one of the base's objects. | **name** | `string` | ✅ | The exported package's machine name. | | **version** | `string` | ✅ | The exported package's version. | | **label** | `string` | optional | Display label, when the package declares one. | +| **withheldMaps** | `{ type: 'connector'; name: string; path: Enum<'providerConfig.headers' \| 'providerConfig.defaultHeaders' \| …> }[]` | optional | [PLANNED — not yet served] Declared ahead of its producer (ADR-0049): no export writes this list yet and no install door reads it yet. Every map the export withheld WHOLE because a served read never carries it: a connector's header and environment-variable maps. The item is written with that map absent, so once served an import reads this list to tell an incomplete map from one the package never had, and names each one for an operator to re-enter. Positions only: no header or variable name is listed. Absence says nothing about what was withheld when the exporter predates this key; from an exporter that writes it, absence means nothing was withheld. | + +### Nested Shape: `PackageExportManifest.withheldMaps[number]` + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **type** | `'connector'` | ✅ | Metadata type of the item whose map was withheld. | +| **name** | `string` | ✅ | Name of the item whose map was withheld. | +| **path** | `Enum<'providerConfig.headers' \| 'providerConfig.defaultHeaders' \| …>` | ✅ | Dot path of the withheld map inside the item. | --- diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/api.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/api.md index 77495e7d445..58421963d0c 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/api.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/api.md @@ -19,4 +19,4 @@ hand-patch a number here** — fix the code or the verdict and regenerate. | Dir | Sites | |---|---| -| `api/` | 489 | +| `api/` | 490 | diff --git a/packages/spec/authorable-surface/api.json b/packages/spec/authorable-surface/api.json index ff3d9739939..03c12f79ac8 100644 --- a/packages/spec/authorable-surface/api.json +++ b/packages/spec/authorable-surface/api.json @@ -1306,6 +1306,7 @@ "api/PackageExportManifest:label", "api/PackageExportManifest:name", "api/PackageExportManifest:version", + "api/PackageExportManifest:withheldMaps", "api/PackageInstallManifest:capabilities [RETIRED]", "api/PackageInstallManifest:configuration [RETIRED]", "api/PackageInstallManifest:contributes", diff --git a/packages/spec/src/api/package-lifecycle.test.ts b/packages/spec/src/api/package-lifecycle.test.ts index 8bb51ac72ed..a92cf59e852 100644 --- a/packages/spec/src/api/package-lifecycle.test.ts +++ b/packages/spec/src/api/package-lifecycle.test.ts @@ -25,6 +25,7 @@ import { GetPackageCustomizationsResponseSchema, } from './package-lifecycle.zod'; import { PackagePublishResultSchema as SystemPackagePublishResultSchema } from '../system/metadata-persistence.zod'; +import { CONNECTOR_CREDENTIAL_MAP_PATHS, redactConnectorItem } from '../integration/connector-credential-redaction'; import { InstalledPackageSourceSchema } from '../kernel/package-registry.zod'; describe('the ruled re-export of PackagePublishResultSchema into the `/api` namespace', () => { @@ -197,7 +198,7 @@ describe('RollbackToPackageCommitResponseSchema declares the COMMIT-rollback bod }); }); -describe('PackageExportManifestSchema declares the four fixed keys and stays open', () => { +describe('PackageExportManifestSchema declares the five fixed keys and stays open', () => { /** A verbatim-shaped capture of a real `assemblePackageManifest` return. */ const realResponse = { id: 'com.example.crm', @@ -214,7 +215,7 @@ describe('PackageExportManifestSchema declares the four fixed keys and stays ope if (result.success) expect(result.data).toEqual(realResponse); }); - it('requires exactly the four fixed keys — a manifest with no items still parses', () => { + it('requires only the three identity keys — a manifest with no items still parses', () => { const bare = { id: 'com.example.empty', name: 'empty', version: '1.0.0' }; expect(PackageExportManifestSchema.safeParse(bare).success).toBe(true); }); @@ -224,6 +225,57 @@ describe('PackageExportManifestSchema declares the four fixed keys and stays ope }); }); +describe('PackageExportManifestSchema.withheldMaps names each map the export withheld whole (#28640 ruling ③A)', () => { + const identity = { id: 'com.example.crm', name: 'example-crm', version: '2.1.0' }; + /** The connector item as the export writes it: served, so its header map is already absent. */ + const servedConnector = { name: 'stripe', label: 'Stripe', type: 'api', provider: 'rest', providerConfig: { baseUrl: 'https://api.stripe.example' } }; + + it('parses an export that lists a withheld map and PRESERVES it — the item body stays as served', () => { + const manifest = { + ...identity, + connectors: [servedConnector], + withheldMaps: [{ type: 'connector', name: 'stripe', path: 'providerConfig.headers' }], + }; + const result = PackageExportManifestSchema.safeParse(manifest); + expect(result.success).toBe(true); + if (result.success) expect(result.data).toEqual(manifest); + }); + + it('is declared ahead of its producer: the describe opens with the planned marker and names the ADR, never a tracker id', () => { + const description = PackageExportManifestSchema.shape.withheldMaps.description ?? ''; + expect(description).toMatch(/^\[PLANNED — not yet served\] /); + expect(description).toContain('ADR-0049'); + expect(description).not.toMatch(/#\d{3,}/); + }); + + it('is optional — an export that withheld nothing, or one from before the key, still parses', () => { + expect(PackageExportManifestSchema.safeParse(identity).success).toBe(true); + }); + + it('spells each position exactly as the connector redactor reports it — the same four maps, nothing else', () => { + const declared = [...PackageExportManifestSchema.shape.withheldMaps.unwrap().element.shape.path.options]; + // The redactor's own list of the maps it withholds whole … + expect(declared).toEqual(CONNECTOR_CREDENTIAL_MAP_PATHS.map((p) => p.join('.'))); + // … and the spelling its `redactedKeys` names them by, read off a real redaction. + const map = { Authorization: 'x' }; + const { redactedKeys } = redactConnectorItem({ + ...servedConnector, + providerConfig: { headers: map, defaultHeaders: map, transport: { headers: map, env: { TOKEN: 'x' } } }, + }); + expect([...redactedKeys].sort()).toEqual([...declared].sort()); + }); + + it.each([ + ['a position outside the four', { type: 'connector', name: 'stripe', path: 'auth.token' }], + ['a position spelled without its providerConfig root', { type: 'connector', name: 'stripe', path: 'defaultHeaders' }], + ['a type no export withholds a map of yet', { type: 'webhook', name: 'hook', path: 'providerConfig.headers' }], + ['a list of the withheld header names — maps are withheld whole', { type: 'connector', name: 'stripe', path: 'providerConfig.headers', keys: ['Authorization'] }], + ['an entry with no item name', { type: 'connector', path: 'providerConfig.headers' }], + ])('refuses %s', (_label, entry) => { + expect(PackageExportManifestSchema.safeParse({ ...identity, withheldMaps: [entry] }).success).toBe(false); + }); +}); + describe('PackageExportBundleSchema declares the whole-deployment export as composed per-package exports (#28914)', () => { /** * The bundle the card's done criteria name: a base, its `.custom` pair and a diff --git a/packages/spec/src/api/package-lifecycle.zod.ts b/packages/spec/src/api/package-lifecycle.zod.ts index ef904ecdaed..1ff8889028f 100644 --- a/packages/spec/src/api/package-lifecycle.zod.ts +++ b/packages/spec/src/api/package-lifecycle.zod.ts @@ -37,6 +37,13 @@ * declared a VERSION rollback against the live COMMIT-rollback path; * `RollbackToPackageCommitResponseSchema` below is the true contract, * authored after that retirement per the ruling's sequencing (3A). + * + * Two members are not transcriptions but DECLARED AHEAD OF THEIR PRODUCER + * (ADR-0049), each describe opening `[PLANNED — not yet served]` until the + * change that serves it drops the marker: the `nav` arm of + * `GetPackageCustomizationsResponseSchema.contributions` (#27928) and + * `PackageExportManifestSchema.withheldMaps` (#28640 ruling ③A, served by + * #29342). Their own docblocks say what is not served yet. */ import { z } from 'zod'; @@ -175,16 +182,56 @@ export const RollbackToPackageCommitResponseSchema = lazySchema(() => z.object({ * (`runtime/src/domains/packages.ts`) builds the key set DYNAMICALLY from the * metadata type registry — one plural key per type present * (`manifest[plural] = items.map(clean)`), partitioning `views` per #5320 — - * plus the four fixed keys below. Only the fixed keys are pinnable; + * plus the five fixed keys below. Only the fixed keys are pinnable; * enumerating the registry here would freeze this contract against future * metadata types, so everything else deliberately falls to the open * catch-all. Freezes nothing. + * + * `withheldMaps` (#28640 ruling ③A) names each map position the export + * withheld whole. The export reads SERVED items, so the connector redactor + * (`CONNECTOR_CREDENTIAL_MAP_PATHS`, `integration/connector-credential-redaction.ts`) + * has already dropped each of its four header / env maps, and the item is + * written with the map simply absent; this list is how an import tells an + * incomplete map from one that was never written. Positions only — never a + * header or variable name — so the credential heuristic stays the + * redactor's, not a contract of the export. The path vocabulary is the + * redactor's own `redactedKeys` spelling of those four positions, and + * `package-lifecycle.test.ts` holds the two equal. + * + * DECLARED AHEAD OF ITS PRODUCER (ADR-0049): `withheldMaps` is declared + * before any export writes it and before any install door reads it. Until the + * server half (#29342) makes `assemblePackageManifest` write the list and both + * install doors name each incomplete map, no export carries it; its describe + * opens `[PLANNED — not yet served]`, and #29342 drops the marker in the change + * that serves it (the pin is in `package-lifecycle.test.ts`). Its `door` row in + * `UNMANAGED_INSTALL_KEY_DISPOSITIONS` is live today: it keeps the unmanaged + * import from refusing an export that carries the key. */ export const PackageExportManifestSchema = lazySchema(() => z.object({ id: z.string().describe('The exported package\'s id.'), name: z.string().describe('The exported package\'s machine name.'), version: z.string().describe('The exported package\'s version.'), label: z.string().optional().describe('Display label, when the package declares one.'), + // Declared ahead of its producer: #29342 writes and reads it, and drops the + // `[PLANNED — not yet served]` marker in that change. + withheldMaps: z.array(z.object({ + type: z.literal('connector').describe('Metadata type of the item whose map was withheld.'), + name: z.string().describe('Name of the item whose map was withheld.'), + path: z.enum([ + 'providerConfig.headers', + 'providerConfig.defaultHeaders', + 'providerConfig.transport.headers', + 'providerConfig.transport.env', + ]).describe('Dot path of the withheld map inside the item.'), + }).strict()).optional().describe( + '[PLANNED — not yet served] Declared ahead of its producer (ADR-0049): no export writes this list ' + + 'yet and no install door reads it yet. Every map the export withheld WHOLE because a served read ' + + 'never carries it: a connector\'s header and environment-variable maps. The item is written with ' + + 'that map absent, so once served an import reads this list to tell an incomplete map from one the ' + + 'package never had, and names each one for an operator to re-enter. Positions only: no header or ' + + 'variable name is listed. Absence says nothing about what was withheld when the exporter predates ' + + 'this key; from an exporter that writes it, absence means nothing was withheld.', + ), }).catchall(z.unknown().describe( 'One key per metadata type present in the package (plural spelling, e.g. ' + '`objects`, `views`), each an array of cleaned item bodies. The key set ' diff --git a/packages/spec/src/kernel/unmanaged-install.test.ts b/packages/spec/src/kernel/unmanaged-install.test.ts index 7afe4b6da74..280263aa5b7 100644 --- a/packages/spec/src/kernel/unmanaged-install.test.ts +++ b/packages/spec/src/kernel/unmanaged-install.test.ts @@ -6,13 +6,15 @@ * authoring refusal and by the install door's import planner * (`@objectstack/metadata-protocol`). * - * Held TOTAL here, beside both schemas a package body is read with, so the PR - * that adds a key to `ManifestSchema` or to the stack definition is the PR that - * reddens — not a later install that refuses the key with no reason on record. + * Held TOTAL here, beside the three schemas a package body is read with, so the + * PR that adds a key to `ManifestSchema`, to the stack definition or to the + * package export's fixed keys is the PR that reddens — not a later install that + * refuses the key with no reason on record. */ import { describe, expect, it } from 'vitest'; import { ManifestSchema } from './manifest.zod'; import { ObjectStackDefinitionSchema } from '../stack.zod'; +import { PackageExportManifestSchema } from '../api/package-lifecycle.zod'; import { PLURAL_TO_SINGULAR } from '../meta-spelling/manifest-collection-spelling'; import { ASSEMBLED_VIEW_ITEMS_KEY } from '../ui/assembled-views.zod'; import { @@ -27,8 +29,10 @@ const collections = new Set([...Object.keys(PLURAL_TO_SINGULAR), ASSEMBLED_VIEW_ const tombstoned = (schema: unknown) => /^\[REMOVED\]/.test(String((schema as { description?: unknown } | undefined)?.description ?? '')); const stackShape = (ObjectStackDefinitionSchema as unknown as { shape: Record }).shape; +/** The export's FIXED keys — its open catch-all carries the collections, judged by the registry. */ +const exportShape = PackageExportManifestSchema.shape; -describe('the unmanaged-install key classification is total against both schemas a package body is read with', () => { +describe('the unmanaged-install key classification is total against the three schemas a package body is read with', () => { it('classifies every live ManifestSchema key', () => { const missing = Object.entries(ManifestSchema.shape) .filter(([key, schema]) => !collections.has(key) && !tombstoned(schema) && !(key in UNMANAGED_INSTALL_KEY_DISPOSITIONS)) @@ -41,8 +45,13 @@ describe('the unmanaged-install key classification is total against both schemas expect(missing).toEqual([]); }); - it('classifies nothing neither schema declares — a stale row is a lie about a key', () => { - const declared = new Set([...Object.keys(ManifestSchema.shape), ...Object.keys(stackShape)]); + it('classifies every fixed key of the package export — the body an import elsewhere is handed', () => { + const missing = Object.keys(exportShape).filter((key) => !(key in UNMANAGED_INSTALL_KEY_DISPOSITIONS)); + expect(missing).toEqual([]); + }); + + it('classifies nothing no schema declares — a stale row is a lie about a key', () => { + const declared = new Set([...Object.keys(ManifestSchema.shape), ...Object.keys(stackShape), ...Object.keys(exportShape)]); expect(Object.keys(UNMANAGED_INSTALL_KEY_DISPOSITIONS).filter((key) => !declared.has(key))).toEqual([]); }); @@ -55,6 +64,14 @@ describe('the unmanaged-install key classification is total against both schemas expect(UNMANAGED_INSTALL_KEY_DISPOSITIONS.data).toBe('door'); }); + it('leaves the export\'s `withheldMaps` and `label` to the door — an export is importable as written', () => { + // `withheldMaps` (#28640 ruling ③A) is read by the door to name each incomplete map; the + // export's `label` has no slot on the base's closed install manifest. Either one left + // unclassified refuses every export that carries it as an unrecognised key. + expect(UNMANAGED_INSTALL_KEY_DISPOSITIONS.withheldMaps).toBe('door'); + expect(UNMANAGED_INSTALL_KEY_DISPOSITIONS.label).toBe('door'); + }); + it('refuses exactly code, uncopyable content and an artifact envelope', () => { expect([...UNMANAGED_INSTALL_REFUSING_DISPOSITIONS].sort()).toEqual(['code', 'envelope', 'uncopyable']); }); diff --git a/packages/spec/src/kernel/unmanaged-install.ts b/packages/spec/src/kernel/unmanaged-install.ts index 509b9461533..132ea0de80b 100644 --- a/packages/spec/src/kernel/unmanaged-install.ts +++ b/packages/spec/src/kernel/unmanaged-install.ts @@ -17,10 +17,12 @@ * metadata type's own `allowRuntimeCreate` flag in * `DEFAULT_METADATA_TYPE_REGISTRY`, the flag the save door reads. * - * `unmanaged-install.test.ts` holds the table TOTAL against both schemas a - * package body is read with — `ManifestSchema` and `ObjectStackDefinitionSchema` - * — so a key either schema gains reddens a spec test on the PR that adds it, - * instead of being refused at an install door with no reason on record. + * `unmanaged-install.test.ts` holds the table TOTAL against the three schemas a + * package body is read with — `ManifestSchema`, `ObjectStackDefinitionSchema` + * and the fixed keys of `PackageExportManifestSchema` (`@objectstack/spec/api`), + * the body `GET /packages/:id/export` writes for an import elsewhere — so a key + * any of them gains reddens a spec test on the PR that adds it, instead of + * being refused at an install door with no reason on record. */ /** @@ -39,7 +41,8 @@ * - `code` — refused: code, or a descriptor of a code-bearing artifact. * - `uncopyable` — refused: package content the environment ledger has no row for. * - `envelope` — refused: it describes an artifact, not one package. - * - `door` — not the copy's: read or applied by the install door, and + * - `door` — not the copy's: read or applied by the install door (or, + * for an export's display `label`, set aside by it), and * never written onto the base. * - `deployment` — not the copy's: deployment configuration no install applies. */ @@ -62,8 +65,8 @@ export const UNMANAGED_INSTALL_REFUSING_DISPOSITIONS: ReadonlySet> = { // ── The package's identity ── @@ -112,6 +115,17 @@ export const UNMANAGED_INSTALL_KEY_DISPOSITIONS: Readonly