diff --git a/.changeset/connector-entry-nested-undeclared-key-refused.md b/.changeset/connector-entry-nested-undeclared-key-refused.md new file mode 100644 index 00000000000..b72b639bbdc --- /dev/null +++ b/.changeset/connector-entry-nested-undeclared-key-refused.md @@ -0,0 +1,31 @@ +--- +'@objectstack/spec': minor +--- + +feat(spec)!: an authored connector entry refuses a key `retryConfig` or an `actions` element does not declare + +Clause-②: no (narrowing) +Metadata-protocol: narrowed + + + +**BREAKING** accept-set narrowing on an authored connector entry (`DeclarativeConnectorEntrySchema`), at every door that parses it: `defineConnector()`, `defineStack({ connectors })`, `os validate` / `os compile`, the metadata plugin's artifact load at boot and on reload, and the metadata save door `PUT /api/v1/meta/connector/:name`. + +- **Before.** The entry refused an undeclared top-level key, but its two nested shapes were plain objects: an undeclared key inside `retryConfig` or inside an `actions` element parsed green and meant nothing to the runtime. +- **Now.** Each is refused by name, in one `unrecognized_keys` issue located at the container (`retryConfig`, `actions.N`) that lists every undeclared key; at the save door `422 INVALID_METADATA`, and nothing is stored. A credential-shaped key (`secret`, `token`, `apiKey`, `password`, `credentialRef`, …) meets the entry's own prescription (`auth.credentialRef`); an execution binding on an action (`method`, `path`, `url`) is refused with ADR-0097 §5's: an action carries none. +- **Unchanged.** The published open shapes — `ConnectorSchema`, `RetryConfigSchema`, `ConnectorActionSchema` — keep their posture: the def a plugin hands to `registerConnector`, the actions a provider derives, and the retry policy the runtime parses at materialization. A row stored before this refusal keeps working: the runtime parses its policy with the open `RetryConfigSchema`, which drops the undeclared key, and the read withholds it. +- **A save over such a row.** An explicit `retryConfig` replaces the stored one whole (the maintainer's ruling): a save that sends `retryConfig` — the served form, `{}`, any object — does not carry the stored row's undeclared retry key forward, and its answer names each key it dropped (advisory rule `withheld-key-dropped`). A save that leaves `retryConfig` out removes it, as before. An undeclared key in an `actions` element stays at rest through a save of the served form. + +## FROM → TO + +```ts +// before — parsed green; `secret` was read by nothing +retryConfig: { strategy: 'exponential_backoff', maxAttempts: 3, secret: '…' }, +actions: [{ key: 'get_invoice', label: 'Get invoice', method: 'GET', path: '/invoices/{id}' }], + +// after — refused at retryConfig and at actions.0; keep the declared keys only +retryConfig: { strategy: 'exponential_backoff', maxAttempts: 3 }, +actions: [{ key: 'get_invoice', label: 'Get invoice', effect: 'read' }], +``` + +A credential moves to the secrets layer under the name `auth.credentialRef` gives; an execution binding has no home on an action (a provider-bound instance derives its actions from the upstream). The upgrade guide entry is `connector-entry-nested-undeclared-key-refused`. diff --git a/.changeset/connector-entry-url-credential-refused.md b/.changeset/connector-entry-url-credential-refused.md new file mode 100644 index 00000000000..6753401c546 --- /dev/null +++ b/.changeset/connector-entry-url-credential-refused.md @@ -0,0 +1,34 @@ +--- +'@objectstack/spec': minor +--- + +feat(spec)!: an authored connector URL holds no credential part, and the connector read judges every `providerConfig` and `auth` URL by the same walk + +Clause-②: no (narrowing) +Metadata-protocol: narrowed + + + +**BREAKING** accept-set narrowing on an authored connector instance (`DeclarativeConnectorEntrySchema`), at every door that runs its refinement: `defineStack({ connectors })`, `os validate` / `os compile`, the metadata plugin's artifact load at boot and on reload, and the metadata save door `PUT /api/v1/meta/connector/:name`. + +- **Before.** A URL under `providerConfig` or `auth` parsed green whatever it held. +- **Now.** A URL holding a userinfo (`user@` or `user:password@` before the host) or a credential query parameter with a value (`api_key`, `token`, `access_token`, `key`, `secret`, `signature` and the rest of the connector redactor's list, matched case-insensitively on the decoded name, plus the instance's own `auth.paramName`) is refused: one issue per URL, located at the URL (`providerConfig.baseUrl`, `providerConfig.transport.url`, `providerConfig.spec`, `auth.tokenUrl`, `auth.authorizationUrl`, `auth.resource`, and any other URL at any depth under `providerConfig`, array elements included), naming the parts it holds — never the URL — and prescribing `auth: { type, credentialRef }`; at the save door `422 INVALID_METADATA`, and nothing is stored. There is no rule per provider: a URL is recognised by its value. +- **One judgement, one walk, both doors.** The write door and the read redactor run one walk over an entry's URLs with one definition of a credential part, so the door refuses exactly the URLs the read rewrites or withholds. A URL inside a header or environment map (`providerConfig.headers`, `defaultHeaders`, `transport.headers`, `transport.env`) is judged by neither: the read withholds those maps whole. +- **Read.** The built-in `connector` redactor rewrites the credential parts out of every URL that walk returns, the `auth` endpoints (`tokenUrl`, `authorizationUrl`, `resource`) as well as `providerConfig`, the rest of each URL served byte for byte and the position named in `redactedKeys`. +- **Unchanged.** `ConnectorSchema` — the def a plugin hands to `registerConnector` — keeps its posture. The runtime materializes a stored row as it is stored and never re-judges it, so a connector stored before this refusal keeps dispatching with its stored URL; a save of its served form keeps the stored URL (the metadata protocol's carry-forward), and an edit of the served URL stores exactly the edit. + +## FROM → TO + +```ts +// before — parsed green; the secret lived in the URL +{ name: 'billing_api', label: 'Billing API', type: 'api', provider: 'rest', + providerConfig: { baseUrl: 'https://billing.example.com/v1?api_key=…' }, + auth: { type: 'none' } }, + +// after — refused at providerConfig.baseUrl; the URL is credential-free and the key is a reference +{ name: 'billing_api', label: 'Billing API', type: 'api', provider: 'rest', + providerConfig: { baseUrl: 'https://billing.example.com/v1' }, + auth: { type: 'api-key', credentialRef: 'BILLING_API_KEY', paramName: 'api_key' } }, +``` + +Set the secret under the name `credentialRef` gives (an environment variable in the open tier, or Setup → Connector credentials) and delete it from the URL. There is deliberately no automatic rewrite: the value in the URL may be the only copy of the secret. The upgrade guide entry is `connector-entry-url-credential-refused`. diff --git a/content/docs/references/api/protocol.mdx b/content/docs/references/api/protocol.mdx index 6dc761fcc64..c0eb3d90a04 100644 --- a/content/docs/references/api/protocol.mdx +++ b/content/docs/references/api/protocol.mdx @@ -2874,7 +2874,7 @@ The authoring surface of a metadata write. Closed: `copilot` is an AI assistant' | **seq** | `integer` | ✅ | Monotonic sequence number of the metadata event this write appended to the item history (sys_metadata_history.event_seq). Orders writes; unlike `version` it is not an OCC token. | | **state** | `Enum<'draft' \| 'active'>` | ✅ | Lifecycle the body was written into: "draft" when the request asked for draft mode (`?mode=draft`), otherwise "active" (published and live). A draft is staged only — it is not served to the runtime until published. | | **projectionApplied** | `{ success: boolean; error?: string }` | optional | Outcome of the awaited ADR-0094 mutation projector — the post-persist step that materializes this metadata into its derived data-plane read model (e.g. `datasource` → its live connection pool). Present ONLY when a projector is registered for this metadata type, which is why it is optional: its absence means "no projector ran", never "the projection failed". Best-effort by design — a projector failure is reported here and logged, never thrown, so a caller that needs the read model to be live must check `projectionApplied.success` rather than rely on the 200. | -| **advisories** | `{ rule: string; path: string; where: string; message: string; … }[]` | optional | Non-gating findings from the runtime authoring gate — the same shared author-time rules `os validate` / `os build` / `os lint` run, applied to this body on its way to `active`. The write SUCCEEDED; these are what the gate has to say about it anyway (closing D3). Present ONLY when at least one advisory was raised — an empty array is never emitted, so a clean save's response bytes are unchanged and absence means "nothing to report", never "the gate did not run". Advisory by construction: every entry has `severity` `warning` or `info`, because an `error` finding refuses the write and arrives as the 422 `invalid_metadata` envelope instead of here. A caller that ignores this key behaves exactly as before. Runtime-only: the CLI surfaces the same findings on its own stdout, and a Studio / MCP / AI author has no CLI at all, which is the gap this key exists to close. The gate runs on both write doors (D1), and both report: `POST /meta/:type/:name/publish` carries the same key on `PublishMetaItemResponseSchema`. The save also reports here what it did to the row at rest that the author did not write, on a draft save too: one `warning` with rule `withheld-key-dropped` for each stored position it dropped because the type's read redactor withholds it and marks it not carried forward (a connector's undeclared `auth` keys, which no door accepts and nothing reads). Its `path` names where the value sat at rest, in the same spelling (`connectors[0].auth.token`). | +| **advisories** | `{ rule: string; path: string; where: string; message: string; … }[]` | optional | Non-gating findings from the runtime authoring gate — the same shared author-time rules `os validate` / `os build` / `os lint` run, applied to this body on its way to `active`. The write SUCCEEDED; these are what the gate has to say about it anyway (closing D3). Present ONLY when at least one advisory was raised — an empty array is never emitted, so a clean save's response bytes are unchanged and absence means "nothing to report", never "the gate did not run". Advisory by construction: every entry has `severity` `warning` or `info`, because an `error` finding refuses the write and arrives as the 422 `invalid_metadata` envelope instead of here. A caller that ignores this key behaves exactly as before. Runtime-only: the CLI surfaces the same findings on its own stdout, and a Studio / MCP / AI author has no CLI at all, which is the gap this key exists to close. The gate runs on both write doors (D1), and both report: `POST /meta/:type/:name/publish` carries the same key on `PublishMetaItemResponseSchema`. The save also reports here what it did to the row at rest that the author did not write, on a draft save too: one `warning` with rule `withheld-key-dropped` for each stored position it dropped because the type's read redactor withholds it and marks it not carried forward (a connector's undeclared `auth` keys and the undeclared keys of its `retryConfig`, which no door accepts and nothing reads). Its `path` names where the value sat at rest, in the same spelling (`connectors[0].auth.token`). | | **message** | `string` | optional | | ### Nested Shape: `SaveMetaItemResponse.projectionApplied` diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/integration.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/integration.md index 7ad938eb5e4..e45ff2b2b54 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/integration.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/integration.md @@ -19,4 +19,4 @@ hand-patch a number here** — fix the code or the verdict and regenerate. | Dir | Sites | |---|---| -| `integration/` | 4 | +| `integration/` | 6 | diff --git a/packages/cli/test/package-union-readers.test.ts b/packages/cli/test/package-union-readers.test.ts index 10157c36b7a..06c6cf9a8b0 100644 --- a/packages/cli/test/package-union-readers.test.ts +++ b/packages/cli/test/package-union-readers.test.ts @@ -95,7 +95,8 @@ const app = { name: 'unr_app', label: 'Union', navigation: [{ id: 'nav_tickets', const ticketZh = { 'zh-CN': { objects: { unr_ticket: { label: '工单', pluralLabel: '工单', fields: { title: { label: '标题' } } } }, apps: { unr_app: { label: '联合', navigation: { nav_tickets: { label: '工单' } } } } } }; const noteZh = { 'zh-CN': { objects: { unr_note: { label: '笔记', fields: { name: { label: '名称' } } } } } }; -const conn = { name: 'unr_conn', label: 'Conn', type: 'api', retryConfig: { zzzUnionProbeKey: 1 } }; +const api = { name: 'unr_notes_api', path: '/unr/notes', method: 'GET', type: 'object_operation', + objectParams: { object: 'unr_note', operation: 'find', zzzUnionProbeKey: 1 } }; `; const CONFIGS: Record = { @@ -119,13 +120,15 @@ export default composeStacks([svc, appStack], { manifest: 'preserve' }); i18nOne: `${PIECES} export default defineStack({ manifest: appManifest, objects: [ticket, note], apps: [app], i18n: I18N, translations: [ticketZh, noteZh] } as any); `, - // An undeclared connector key. \`strict: false\` keeps it past the producer's + // An undeclared authoring key. \`strict: false\` keeps it past the producer's // parse, so the doors' own pre-parse walk is the only carrier into \`--json\`. - // It sits one level down, in \`retryConfig\` (a strip-mode shape the walk still - // visits): the connector entry's own root is closed, so a top-level key there - // is refused by the artifact load rather than warned about. + // It sits one level down, in an API endpoint's \`objectParams\` (a strip-mode + // shape the walk still visits): every collection root is closed, so a + // top-level key is refused by the artifact load rather than warned about. It + // was a connector's \`retryConfig\` until the authored connector entry closed + // its nested shapes too, after which the load refuses that key instead. keyTwo: `${PIECES} -const svc = defineStack({ manifest: svcManifest, objects: [note], connectors: [conn] } as any, { strict: false }); +const svc = defineStack({ manifest: svcManifest, objects: [note], apis: [api] } as any, { strict: false }); const appStack = defineStack({ manifest: appManifest, objects: [ticket], apps: [app] } as any); export default composeStacks([svc, appStack], { manifest: 'preserve' }); `, @@ -262,6 +265,6 @@ describe('#22238 — the undeclared-authoring-key walk reads the package union', it('`os validate --json` carries the finding in `warnings` for a two-package app', () => { const run = r.validateTwo; expect(run.code, `${run.stdout}\n${run.stderr}`).toBe(0); - expect(named(run)).toEqual([expect.stringContaining('connectors.unr_conn.retryConfig.zzzUnionProbeKey')]); + expect(named(run)).toEqual([expect.stringContaining('apis.unr_notes_api.objectParams.zzzUnionProbeKey')]); }); }); diff --git a/packages/metadata-protocol/src/metadata-redaction.connector-undeclared-keys.test.ts b/packages/metadata-protocol/src/metadata-redaction.connector-undeclared-keys.test.ts index 5d52e269f1f..3e1483212bb 100644 --- a/packages/metadata-protocol/src/metadata-redaction.connector-undeclared-keys.test.ts +++ b/packages/metadata-protocol/src/metadata-redaction.connector-undeclared-keys.test.ts @@ -2,8 +2,10 @@ // // The write-path inverse of the connector read projection, for the keys a // connector does not declare: a save of the served form keeps what the read -// withheld (top level, `retryConfig`, and an `actions` element addressed by -// its `key`), and a container the author removed is never minted back. +// withheld at the top level and in an `actions` element addressed by its +// `key`; a container the author removed is never minted back; and an +// undeclared key of an object `retryConfig` is not carried forward — a save +// that sends `retryConfig` replaces the stored one whole (ruling 6593804783). // // Every value below is an obviously-fake placeholder. @@ -40,9 +42,9 @@ describe('carry-forward of the keys a connector does not declare', () => { expect(body.actions[0]).toEqual({ key: 'get_invoice', label: 'Get invoice' }); }); - it('a save of the served form keeps every withheld value, the edited label included', () => { + it('a save of the served form keeps every carried-forward withheld value, the edited label included', () => { const out = carryForwardRedactedValues('connector', { ...served(), label: 'Vendor catalog (edited)' }, stored); - expect(out).toEqual({ ...stored, label: 'Vendor catalog (edited)' }); + expect(out).toEqual({ ...stored, label: 'Vendor catalog (edited)', retryConfig: { strategy: 'exponential_backoff' } }); }); it('an `actions` element is followed by its `key` when the author reorders the list', () => { @@ -90,6 +92,35 @@ describe('carry-forward of the keys a connector does not declare', () => { }); }); +// [ruling 6593804783] An undeclared key of an object `retryConfig` is withheld +// AND not carried forward: a save that sends `retryConfig` replaces the stored +// container whole, and the drop is reported. +describe('an undeclared `retryConfig` key is not carried forward', () => { + it('a save of the served form drops it, names it, and still carries the top-level and `actions` values', () => { + const { item, dropped } = carryForwardRedactedValuesReportingDrops('connector', served(), stored); + expect(item).toEqual({ ...stored, retryConfig: { strategy: 'exponential_backoff' } }); + expect(dropped).toEqual(['retryConfig.secret']); + }); + + it('`retryConfig: {}` replaces the stored container whole', () => { + const { item, dropped } = carryForwardRedactedValuesReportingDrops('connector', { ...served(), retryConfig: {} }, stored); + expect((item as Record).retryConfig).toEqual({}); + expect(dropped).toEqual(['retryConfig.secret']); + }); + + it('a body that removes `retryConfig` removes it, as before, and says so', () => { + const body = served(); + delete body.retryConfig; + const { item, dropped } = carryForwardRedactedValuesReportingDrops('connector', body, stored); + expect(item).not.toHaveProperty('retryConfig'); + expect(dropped).toEqual(['retryConfig.secret']); + }); + + it('the runtime gate is never told it will be restored', () => { + expect(redactedPathsCarriedForward('connector', served(), stored).sort()).toEqual(['actions.0.token', 'apiKey']); + }); +}); + // [#28640 ②B] An undeclared `auth` key is withheld AND not carried forward: a // save of the served form drops it, and the drop is reported. describe('an undeclared `auth` key is not carried forward', () => { @@ -136,8 +167,10 @@ describe('an undeclared `auth` key is not carried forward', () => { expect(dropped).toEqual(['auth.token']); }); - it('CONTROL: a row with no undeclared `auth` key reports no drop', () => { - const { dropped } = carryForwardRedactedValuesReportingDrops('connector', served(), stored); + it('CONTROL: a row with no undeclared `auth` or `retryConfig` key reports no drop', () => { + const atRest = { ...stored, retryConfig: { strategy: 'exponential_backoff' } }; + const body = JSON.parse(JSON.stringify(redactMetadataItem('connector', atRest))); + const { dropped } = carryForwardRedactedValuesReportingDrops('connector', body, atRest); expect(dropped).toEqual([]); }); }); diff --git a/packages/qa/dogfood/test/connector-meta-door-entry-and-url-hardening.dogfood.test.ts b/packages/qa/dogfood/test/connector-meta-door-entry-and-url-hardening.dogfood.test.ts index 11ac81af6d9..7c918b0179b 100644 --- a/packages/qa/dogfood/test/connector-meta-door-entry-and-url-hardening.dogfood.test.ts +++ b/packages/qa/dogfood/test/connector-meta-door-entry-and-url-hardening.dogfood.test.ts @@ -14,8 +14,10 @@ // system context, the only way such a row is written), is served with those // parts withheld, to a member and to an administrator, in the item, list and // layered reads; the stored row itself is never touched. -// - CARRY-FORWARD: a save of the served form keeps the stored URL whole, and -// the instance still dispatches with it. +// - CARRY-FORWARD: on a row stored before the save door refused a credential +// part in a URL (seeded through the system context), a save of the served +// form keeps the stored URL whole, and the instance still dispatches with +// it: the runtime materializes the stored row as it is, never re-judged. // // Every credential below is an obviously-fake placeholder, and every assertion // on one is its absence from what a door served, or its presence at rest. @@ -248,11 +250,16 @@ describe('connector metadata doors: undeclared entry keys refused; URL credentia describe('CARRY-FORWARD: a save of the served form keeps the stored URL, and dispatch still uses it', () => { it('the served URL drops the credential parameter; the re-saved row keeps it; the upstream receives it', async () => { const carriedUrl = `${upstreamUrl}/v1?api_key=${PLACEHOLDERS.carried}®ion=eu`; - const put = await stack.apiAs(admin, 'PUT', `/meta/connector/${CARRIED}`, instance(CARRIED, { - providerConfig: { baseUrl: carriedUrl }, - })); + // A row stored before the save door refused a credential part in a URL: + // saved disabled, so nothing materializes before the seeded form is the + // row, then the system context writes that form over it. + const put = await stack.apiAs(admin, 'PUT', `/meta/connector/${CARRIED}`, instance(CARRIED, { enabled: false })); expect(put.status, await put.clone().text()).toBe(200); - await waitReady(CARRIED); + const [row] = await stored(CARRIED); + await ql.update('sys_metadata', { + id: row.id, + metadata: JSON.stringify(instance(CARRIED, { providerConfig: { baseUrl: carriedUrl } })), + }, { context: SYSTEM }); const read = await stack.apiAs(admin, 'GET', `/meta/connector/${CARRIED}`); const served = itemOf(await read.json()); diff --git a/packages/qa/dogfood/test/connector-meta-door-undeclared-keys-withheld.dogfood.test.ts b/packages/qa/dogfood/test/connector-meta-door-undeclared-keys-withheld.dogfood.test.ts index d32b5c784c4..f2cdd31d6b4 100644 --- a/packages/qa/dogfood/test/connector-meta-door-undeclared-keys-withheld.dogfood.test.ts +++ b/packages/qa/dogfood/test/connector-meta-door-undeclared-keys-withheld.dogfood.test.ts @@ -3,16 +3,18 @@ // The connector metadata doors, booted on the showcase with its own connector // plugins — keys a connector does not declare are never served: // -// - READ: a stored row carrying top-level keys the connector does not declare -// (seeded through the system context, the only way such a row is written -// now), an instance whose `retryConfig` and a descriptor whose `actions` -// element carry an undeclared key (both accepted by the save door), are +// - READ: a stored row carrying top-level keys the connector does not declare, +// an instance whose `retryConfig` and a descriptor whose `actions` element +// carry an undeclared key (each seeded through the system context, the only +// way such a row is written now: the save door refuses all three), are // served with those keys withheld and every declared key as stored — to a // member and to an administrator, in the item, layered, published and list // reads, and on the published read of a package published twice (whose // second snapshot nests the first). The stored rows are never touched. // - CARRY-FORWARD: a save of the served form succeeds and keeps every stored -// value; a `retryConfig` the author removes is removed. +// value at the top level and in an `actions` element; a `retryConfig` sent +// back as served replaces the stored one whole, and one the author removes +// is removed. // - CONTROL: an instance that names its secret with `credentialRef` saves, // materializes and dispatches upstream with the reference resolved. // @@ -154,12 +156,21 @@ describe('connector metadata doors: keys a connector does not declare are withhe }), }, { context: SYSTEM }); - const retry = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY}`, instance(RETRY, { - retryConfig: { strategy: 'exponential_backoff', maxAttempts: 2, secret: PLACEHOLDERS.retry }, - })); - expect(retry.status, await retry.clone().text()).toBe(200); - const desc = await stack.apiAs(admin, 'PUT', `/meta/connector/${DESCRIPTOR}`, descriptor); - expect(desc.status, await desc.clone().text()).toBe(200); + // The nested rows: a clean save, then the system context writes the + // undeclared nested key over it (the save door refuses it now). + const seedOver = async (name: string, clean: Record, seeded: Record) => { + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${name}`, clean); + expect(res.status, await res.clone().text()).toBe(200); + const [seedRow] = await stored(name); + await ql.update('sys_metadata', { id: seedRow.id, metadata: JSON.stringify(seeded) }, { context: SYSTEM }); + }; + await seedOver( + RETRY, + instance(RETRY, { retryConfig: { strategy: 'exponential_backoff', maxAttempts: 2 } }), + instance(RETRY, { retryConfig: { strategy: 'exponential_backoff', maxAttempts: 2, secret: PLACEHOLDERS.retry } }), + ); + const { token: _token, ...cleanAction } = descriptor.actions[0]!; + await seedOver(DESCRIPTOR, { ...descriptor, actions: [cleanAction, descriptor.actions[1]] }, descriptor); }, 180_000); afterAll(async () => { @@ -263,11 +274,11 @@ describe('connector metadata doors: keys a connector does not declare are withhe ]); }); - it('a `retryConfig` the author removes is removed; one sent back as served keeps its stored key', async () => { + it('a `retryConfig` sent back as served replaces the stored one whole; one the author removes is removed', async () => { const served = authorable((await read(admin, RETRY)).item); - const kept = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY}`, { ...served, label: 'CUK retry (edited)' }); - expect(kept.status, await kept.clone().text()).toBe(200); - expect(bodyOf((await stored(RETRY))[0]).retryConfig.secret).toBe(PLACEHOLDERS.retry); + const replaced = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY}`, { ...served, label: 'CUK retry (edited)' }); + expect(replaced.status, await replaced.clone().text()).toBe(200); + expect(bodyOf((await stored(RETRY))[0]).retryConfig).toEqual({ strategy: 'exponential_backoff', maxAttempts: 2 }); const { retryConfig: _removed, ...withoutRetry } = served; const cleared = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY}`, withoutRetry); diff --git a/packages/qa/dogfood/test/connector-meta-door-url-and-nested-keys.dogfood.test.ts b/packages/qa/dogfood/test/connector-meta-door-url-and-nested-keys.dogfood.test.ts new file mode 100644 index 00000000000..ce6b510bf3b --- /dev/null +++ b/packages/qa/dogfood/test/connector-meta-door-url-and-nested-keys.dogfood.test.ts @@ -0,0 +1,415 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// The connector metadata doors, booted on the showcase with its own connector +// plugins — an authored connector keeps no credential part in a URL and no key +// its nested shapes do not declare: +// +// - WRITE: the save door refuses, with the `422 INVALID_METADATA` envelope +// located at the position, a URL holding a userinfo or a credential query +// parameter — under `providerConfig` (`baseUrl`, `transport.url`, a URL +// `spec`, a URL inside an inline `spec` document) and under `auth` +// (`tokenUrl`, `authorizationUrl`) — and a key `retryConfig` or an +// `actions` element does not declare. Nothing is stored, and an existing +// row is left as it was. +// - CONTROL: an inline `spec` document with credential-free URLs saves as +// written; an instance with credential-free URLs that names its secret with +// `credentialRef` saves, materializes and dispatches upstream with the +// reference resolved; a descriptor with declared action and retry keys saves. +// - READ: a stored row whose `auth` endpoints carry a userinfo (seeded through +// the system context, the only way such a row is written now) is served +// with the userinfo withheld and every declared key as stored, to a member +// and to an administrator, in the item, layered and list reads. The stored +// row is never touched. +// - CARRY-FORWARD: a save of the served form keeps the stored values; an +// author's own edit of a served URL is stored as the author wrote it. An +// explicit `retryConfig` — `{}` or the served form — replaces the stored +// one whole: a key it does not declare is gone from the next read of the +// stored row, and the answer names it, while a withheld header map on the +// same row is kept (CONTROL). A `retryConfig` the author leaves out is +// removed with its stored keys, as before (CONTROL), and one stored as a +// non-object is still kept at rest by a save that leaves it out (CONTROL). +// +// Every credential below is an obviously-fake placeholder, and every assertion +// on one is its absence from what a door served or stored, or its presence at +// rest on a seeded row. + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { createServer, type Server } from 'node:http'; +import type { AddressInfo } from 'node:net'; +import { type VerifyStack } from '@objectstack/verify'; +import { bootShowcase } from './showcase-boot.js'; + +const SYSTEM = { isSystem: true, positions: [], permissions: [] }; + +const PLACEHOLDERS = { + urlPassword: 'TEST_PLACEHOLDER_CUN_URL_PASSWORD', + urlToken: 'TEST_PLACEHOLDER_CUN_URL_TOKEN', + urlQuery: 'TEST_PLACEHOLDER_CUN_URL_QUERY', + nested: 'TEST_PLACEHOLDER_CUN_NESTED', + header: 'TEST_PLACEHOLDER_CUN_HEADER', + authEndpoint: 'TEST_PLACEHOLDER_CUN_AUTH_ENDPOINT', + resolvedRef: 'TEST_PLACEHOLDER_CUN_RESOLVED_REFERENCE', +} as const; +const NEVER_SERVED = Object.values(PLACEHOLDERS).filter((p) => p !== PLACEHOLDERS.resolvedRef); + +/** The reference every instance here names; set in this process's environment before any save. */ +const REF = 'CUN_CONTROL_CREDENTIAL_REF'; +const CONTROL = 'cun_control'; +const DESCRIPTOR = 'cun_descriptor'; +const EXISTING = 'cun_existing'; +const OAUTH_ROW = 'cun_oauth_stored'; +const RETRY_ROW = 'cun_retry_stored'; +const RETRY_EMPTY = 'cun_retry_empty'; +const RETRY_OMIT = 'cun_retry_omit'; +const RETRY_MALFORMED = 'cun_retry_malformed'; + +const rowsOf = (r: any): any[] => (Array.isArray(r) ? r : Array.isArray(r?.records) ? r.records : Array.isArray(r?.value) ? r.value : []); +const bodyOf = (row: any): Record => + typeof row?.metadata === 'string' ? JSON.parse(row.metadata) : (row?.metadata ?? {}); +const itemOf = (json: any): Record => (json?.item ?? json?.data?.item ?? json?.data ?? json) as Record; +const holdsAPlaceholder = (v: unknown): boolean => { + const text = typeof v === 'string' ? v : JSON.stringify(v ?? null); + return NEVER_SERVED.some((p) => text.includes(p)); +}; +/** A served item as a client sends it back: the read decorations dropped. */ +const authorable = (served: Record): Record => + Object.fromEntries(Object.entries(served).filter(([k]) => !k.startsWith('_'))); + +describe('connector metadata doors: an authored connector keeps no credential in a URL and no undeclared nested key', () => { + let stack: VerifyStack; + let admin: string; + let member: string; + let ql: any; + let automation: any; + let upstream: Server; + let upstreamUrl: string; + const received: string[] = []; + let previousRef: string | undefined; + + const stored = async (name: string) => + rowsOf(await ql.find('sys_metadata', { where: { type: 'connector', name }, context: SYSTEM })); + + const instance = (name: string, over: Record = {}) => ({ + name, + label: `CUN ${name}`, + type: 'api', + description: 'A provider-bound instance for the URL and nested-key pins.', + provider: 'rest', + providerConfig: { baseUrl: `${upstreamUrl}/v1` }, + auth: { type: 'bearer', credentialRef: REF }, + ...over, + }); + + const oauth2 = (name: string, over: Record = {}) => instance(name, { + auth: { + type: 'oauth2', + grant: 'authorization_code', + authorizationUrl: 'https://auth.vendor.example.com/authorize', + tokenUrl: 'https://auth.vendor.example.com/token', + clientId: 'cun-client', + credentialRef: REF, + ...over, + }, + }); + + /** A minimal inline OpenAPI document whose one server is `serverUrl`. */ + const inlineSpec = (serverUrl: string) => ({ + openapi: '3.0.3', + info: { title: 'CUN inline', version: '1.0.0' }, + servers: [{ url: serverUrl }], + paths: {}, + }); + + const descriptor = { + name: DESCRIPTOR, + label: 'CUN descriptor', + type: 'saas', + description: 'A catalog-only descriptor for the nested-key pins.', + actions: [{ key: 'get_invoice', label: 'Get invoice', effect: 'read', inputSchema: { type: 'object' } }], + }; + + const read = async (token: string, name: string) => { + const res = await stack.apiAs(token, 'GET', `/meta/connector/${name}`); + const text = await res.text(); + expect(res.status, text.slice(0, 200)).toBe(200); + return { text, item: itemOf(JSON.parse(text)) }; + }; + + /** One refused save: the 422 envelope, located at `at`, echoing no credential. */ + const expectRefused = async (name: string, body: Record, at: string) => { + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${name}`, body); + const text = await res.text(); + expect(res.status, text.slice(0, 300)).toBe(422); + const json = JSON.parse(text); + expect(json.code ?? json.error?.code).toBe('INVALID_METADATA'); + const paths = (json.issues ?? json.error?.issues ?? []).map((i: any) => (Array.isArray(i.path) ? i.path.join('.') : i.path)); + expect(paths, text.slice(0, 300)).toContain(at); + expect(holdsAPlaceholder(text), 'the refusal echoes no credential').toBe(false); + }; + + const waitReady = async (name: string): Promise => { + const deadline = Date.now() + 15_000; + let last = ''; + while (Date.now() < deadline) { + const res = await stack.apiAs(admin, 'GET', '/automation/connectors'); + const json = (await res.json()) as any; + const found = (json?.data?.connectors ?? []).find((c: any) => c?.name === name); + last = JSON.stringify(found ?? null); + if (found?.state === 'ready') return; + await new Promise((r) => setTimeout(r, 100)); + } + throw new Error(`connector ${name} never reached ready: ${last}`); + }; + + beforeAll(async () => { + upstream = createServer((req, res) => { + received.push(req.url ?? ''); + res.writeHead(200, { 'Content-Type': 'application/json' }); + res.end('{"ok":true}'); + }); + await new Promise((r) => upstream.listen(0, '127.0.0.1', () => r())); + upstreamUrl = `http://127.0.0.1:${(upstream.address() as AddressInfo).port}`; + previousRef = process.env[REF]; + process.env[REF] = PLACEHOLDERS.resolvedRef; + + stack = await bootShowcase({}); + ql = await stack.kernel.getServiceAsync('objectql'); + automation = await stack.kernel.getServiceAsync('automation'); + admin = await stack.signIn(); + member = await stack.signUp('cun-member@connector-url-nested.test'); + + const existing = await stack.apiAs(admin, 'PUT', `/meta/connector/${EXISTING}`, instance(EXISTING)); + expect(existing.status, await existing.clone().text()).toBe(200); + + // Rows stored before the save door refused them: a clean save, then the + // system context writes the credential parts / undeclared key over it. + const seedOver = async (name: string, clean: Record, seeded: Record) => { + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${name}`, clean); + expect(res.status, await res.clone().text()).toBe(200); + const [row] = await stored(name); + await ql.update('sys_metadata', { id: row.id, metadata: JSON.stringify(seeded) }, { context: SYSTEM }); + }; + await seedOver(OAUTH_ROW, oauth2(OAUTH_ROW), oauth2(OAUTH_ROW, { + tokenUrl: `https://cun:${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/token`, + authorizationUrl: `https://${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/authorize?region=eu`, + })); + // Each also holds a withheld header map, carried forward: the control on the same save. + for (const name of [RETRY_ROW, RETRY_EMPTY, RETRY_OMIT]) { + await seedOver(name, instance(name, { retryConfig: { maxAttempts: 2 } }), instance(name, { + providerConfig: { baseUrl: `${upstreamUrl}/v1`, defaultHeaders: { Authorization: `Bearer ${PLACEHOLDERS.header}` } }, + retryConfig: { maxAttempts: 2, secret: PLACEHOLDERS.nested }, + })); + } + // Saved disabled: the pin is the door's answer, not a materialization of a policy no parse accepts. + await seedOver( + RETRY_MALFORMED, + instance(RETRY_MALFORMED, { enabled: false }), + instance(RETRY_MALFORMED, { enabled: false, retryConfig: PLACEHOLDERS.nested }), + ); + }, 180_000); + + afterAll(async () => { + await stack?.stop?.(); + await new Promise((r) => (upstream ? upstream.close(() => r()) : r())); + if (previousRef === undefined) delete process.env[REF]; + else process.env[REF] = previousRef; + }); + + describe('WRITE: a credential part in a URL, or an undeclared nested key, is refused and nothing is stored', () => { + const cases: Array<[label: string, name: string, body: () => Record, at: string]> = [ + ['a userinfo password in `providerConfig.baseUrl`', 'cun_w_userinfo', + () => instance('cun_w_userinfo', { providerConfig: { baseUrl: `https://u:${PLACEHOLDERS.urlPassword}@api.vendor.example.com/v1` } }), 'providerConfig.baseUrl'], + ['a userinfo token in `providerConfig.baseUrl`', 'cun_w_usertoken', + () => instance('cun_w_usertoken', { providerConfig: { baseUrl: `https://${PLACEHOLDERS.urlToken}@api.vendor.example.com/v1` } }), 'providerConfig.baseUrl'], + ['`api_key` in the `providerConfig.baseUrl` query', 'cun_w_api_key', + () => instance('cun_w_api_key', { providerConfig: { baseUrl: `https://api.vendor.example.com/v1?api_key=${PLACEHOLDERS.urlQuery}` } }), 'providerConfig.baseUrl'], + ['`token` in the `providerConfig.baseUrl` query', 'cun_w_token', + () => instance('cun_w_token', { providerConfig: { baseUrl: `https://api.vendor.example.com/v1?page=2&token=${PLACEHOLDERS.urlQuery}` } }), 'providerConfig.baseUrl'], + ['`access_token` in the `providerConfig.baseUrl` query', 'cun_w_access_token', + () => instance('cun_w_access_token', { providerConfig: { baseUrl: `https://api.vendor.example.com/v1?access_token=${PLACEHOLDERS.urlQuery}` } }), 'providerConfig.baseUrl'], + ['the instance\'s own `auth.paramName` in the `providerConfig.baseUrl` query', 'cun_w_param_name', + () => instance('cun_w_param_name', { + providerConfig: { baseUrl: `https://api.vendor.example.com/v1?vendor_key=${PLACEHOLDERS.urlQuery}` }, + auth: { type: 'api-key', credentialRef: REF, paramName: 'vendor_key' }, + }), 'providerConfig.baseUrl'], + ['a userinfo password in an mcp `providerConfig.transport.url`', 'cun_w_mcp', + () => ({ name: 'cun_w_mcp', label: 'CUN mcp', type: 'api', provider: 'mcp', + providerConfig: { transport: { kind: 'http', url: `https://u:${PLACEHOLDERS.urlPassword}@mcp.vendor.example.com/mcp` } } }), 'providerConfig.transport.url'], + ['`token` in an openapi URL `providerConfig.spec`', 'cun_w_openapi', + () => ({ name: 'cun_w_openapi', label: 'CUN openapi', type: 'api', provider: 'openapi', + providerConfig: { spec: `https://api.vendor.example.com/openapi.json?token=${PLACEHOLDERS.urlQuery}` }, + auth: { type: 'bearer', credentialRef: REF } }), 'providerConfig.spec'], + ['`token` in a URL inside an inline openapi `providerConfig.spec` document', 'cun_w_inline_spec', + () => ({ name: 'cun_w_inline_spec', label: 'CUN inline spec', type: 'api', provider: 'openapi', + providerConfig: { spec: inlineSpec(`https://api.vendor.example.com/v1?token=${PLACEHOLDERS.urlQuery}`) }, + auth: { type: 'bearer', credentialRef: REF } }), 'providerConfig.spec.servers.0.url'], + ['a userinfo password in `auth.tokenUrl`', 'cun_w_token_url', + () => oauth2('cun_w_token_url', { tokenUrl: `https://c:${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/token` }), 'auth.tokenUrl'], + ['a userinfo in `auth.authorizationUrl`', 'cun_w_authz_url', + () => oauth2('cun_w_authz_url', { authorizationUrl: `https://${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/authorize` }), 'auth.authorizationUrl'], + ['an undeclared key inside `retryConfig`', 'cun_w_retry', + () => instance('cun_w_retry', { retryConfig: { maxAttempts: 2, secret: PLACEHOLDERS.nested } }), 'retryConfig'], + ['an undeclared key inside an `actions` element', 'cun_w_action', + () => ({ ...descriptor, name: 'cun_w_action', actions: [{ key: 'list', label: 'List', token: PLACEHOLDERS.nested }] }), 'actions.0'], + ]; + + it.each(cases)('%s', async (_label, name, body, at) => { + await expectRefused(name, body(), at); + expect(await stored(name), `${name}: nothing stored`).toEqual([]); + }); + + it('on an existing row, the refused save leaves the row as it was', async () => { + const before = bodyOf((await stored(EXISTING))[0]); + await expectRefused(EXISTING, instance(EXISTING, { + providerConfig: { baseUrl: `https://u:${PLACEHOLDERS.urlPassword}@api.vendor.example.com/v1` }, + }), 'providerConfig.baseUrl'); + await expectRefused(EXISTING, instance(EXISTING, { retryConfig: { secret: PLACEHOLDERS.nested } }), 'retryConfig'); + expect(bodyOf((await stored(EXISTING))[0])).toEqual(before); + expect(holdsAPlaceholder(await stored(EXISTING))).toBe(false); + }); + }); + + describe('CONTROL: credential-free URLs and declared nested keys save and work', () => { + it('an instance naming its secret with `credentialRef` saves, materializes ready, and dispatches with the reference resolved', async () => { + const put = await stack.apiAs(admin, 'PUT', `/meta/connector/${CONTROL}`, instance(CONTROL, { + providerConfig: { baseUrl: `${upstreamUrl}/v1?region=eu&token=` }, + retryConfig: { strategy: 'exponential_backoff', maxAttempts: 2 }, + })); + expect(put.status, await put.clone().text()).toBe(200); + await waitReady(CONTROL); + const handler = automation.resolveConnectorAction(CONTROL, 'request'); + expect(handler, `${CONTROL} is materialized with its request action`).toBeTypeOf('function'); + const before = received.length; + const out = await handler({ method: 'GET', path: '/probe' }, {}); + expect(out?.status, JSON.stringify(out)).toBe(200); + expect(received.length).toBe(before + 1); + expect(received[received.length - 1]).toMatch(/^\/v1\/probe/); + }); + + it('an inline openapi document whose URLs carry no credential part saves as written', async () => { + // Saved disabled: the pin is the door's answer, not a materialization. + const body = { + name: 'cun_inline_spec_clean', label: 'CUN inline spec (clean)', type: 'api', provider: 'openapi', enabled: false, + providerConfig: { spec: inlineSpec('https://api.vendor.example.com/v1?region=eu') }, + auth: { type: 'bearer', credentialRef: REF }, + }; + const res = await stack.apiAs(admin, 'PUT', '/meta/connector/cun_inline_spec_clean', body); + expect(res.status, await res.clone().text()).toBe(200); + expect(bodyOf((await stored('cun_inline_spec_clean'))[0]).providerConfig).toEqual(body.providerConfig); + }); + + it('a descriptor with declared action keys, and an oauth2 instance with credential-free endpoints, save', async () => { + const desc = await stack.apiAs(admin, 'PUT', `/meta/connector/${DESCRIPTOR}`, descriptor); + expect(desc.status, await desc.clone().text()).toBe(200); + const oauth = await stack.apiAs(admin, 'PUT', '/meta/connector/cun_oauth_clean', oauth2('cun_oauth_clean')); + expect(oauth.status, await oauth.clone().text()).toBe(200); + expect(bodyOf((await stored(DESCRIPTOR))[0]).actions).toEqual(descriptor.actions); + }); + }); + + describe('READ: a stored `auth` endpoint\'s userinfo is withheld from every reader; declared keys are served', () => { + it('guard the guard: the stored row holds the seeded userinfo', async () => { + const row = bodyOf((await stored(OAUTH_ROW))[0]); + expect(row.auth.tokenUrl).toContain(PLACEHOLDERS.authEndpoint); + expect(row.auth.authorizationUrl).toContain(PLACEHOLDERS.authEndpoint); + }); + + it('the item, layered and list reads, for a member and for an administrator', async () => { + for (const token of [member, admin]) { + const { text, item } = await read(token, OAUTH_ROW); + expect(holdsAPlaceholder(text)).toBe(false); + expect(item.auth).toEqual({ + type: 'oauth2', + grant: 'authorization_code', + authorizationUrl: 'https://auth.vendor.example.com/authorize?region=eu', + tokenUrl: 'https://auth.vendor.example.com/token', + clientId: 'cun-client', + credentialRef: REF, + }); + for (const path of [`/meta/connector/${OAUTH_ROW}/layers`, '/meta/connector', '/meta/connectors']) { + const res = await stack.apiAs(token, 'GET', path); + const body = await res.text(); + expect(res.status, `${path}: ${body.slice(0, 200)}`).toBe(200); + expect(body.includes(OAUTH_ROW), `${path} names ${OAUTH_ROW}`).toBe(true); + expect(holdsAPlaceholder(body), `${path} withholds the userinfo`).toBe(false); + } + } + }); + + it('the stored row is never redacted: redaction is a serving act', async () => { + expect(bodyOf((await stored(OAUTH_ROW))[0]).auth.tokenUrl).toContain(PLACEHOLDERS.authEndpoint); + }); + }); + + describe('CARRY-FORWARD: the served form saves and keeps the stored values; an explicit edit is the author\'s word', () => { + it('a save of the served oauth2 row succeeds and keeps each stored endpoint', async () => { + const served = authorable((await read(admin, OAUTH_ROW)).item); + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${OAUTH_ROW}`, { ...served, label: 'CUN oauth (edited)' }); + expect(res.status, await res.clone().text()).toBe(200); + const atRest = bodyOf((await stored(OAUTH_ROW))[0]); + expect(atRest.label).toBe('CUN oauth (edited)'); + expect(atRest.auth.tokenUrl).toBe(`https://cun:${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/token`); + expect(atRest.auth.authorizationUrl).toBe(`https://${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/authorize?region=eu`); + }); + + it('an author\'s edit of a served URL is stored exactly as written', async () => { + const served = authorable((await read(admin, OAUTH_ROW)).item); + const edited = { ...served, auth: { ...served.auth, tokenUrl: 'https://auth.vendor.example.com/oauth2/token' } }; + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${OAUTH_ROW}`, edited); + expect(res.status, await res.clone().text()).toBe(200); + const atRest = bodyOf((await stored(OAUTH_ROW))[0]); + expect(atRest.auth.tokenUrl).toBe('https://auth.vendor.example.com/oauth2/token'); + expect(atRest.auth.authorizationUrl).toBe(`https://${PLACEHOLDERS.authEndpoint}@auth.vendor.example.com/authorize?region=eu`); + }); + + const HEADERS_AT_REST = () => ({ Authorization: `Bearer ${PLACEHOLDERS.header}` }); + + it('`retryConfig: {}` replaces the stored one whole: the next read of the stored row no longer has the undeclared key', async () => { + expect(bodyOf((await stored(RETRY_EMPTY))[0]).retryConfig, 'guard the guard: the seeded row holds the key') + .toEqual({ maxAttempts: 2, secret: PLACEHOLDERS.nested }); + const served = authorable((await read(admin, RETRY_EMPTY)).item); + expect(served.retryConfig).toEqual({ maxAttempts: 2 }); + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY_EMPTY}`, { ...served, retryConfig: {} }); + const text = await res.text(); + expect(res.status, text.slice(0, 300)).toBe(200); + // The save names the key it did not carry forward. + expect(JSON.parse(text).advisories, text.slice(0, 600)).toEqual([ + expect.objectContaining({ rule: 'withheld-key-dropped', severity: 'warning', path: 'connectors[0].retryConfig.secret' }), + ]); + expect(text.includes(PLACEHOLDERS.nested), 'the answer names the key, never its value').toBe(false); + const atRest = bodyOf((await stored(RETRY_EMPTY))[0]); + expect(atRest.retryConfig).toEqual({}); + // CONTROL, the same save: a withheld position outside `retryConfig` is still carried forward. + expect(atRest.providerConfig.defaultHeaders).toEqual(HEADERS_AT_REST()); + }); + + it('a save of the served form replaces the stored `retryConfig` whole too', async () => { + const served = authorable((await read(admin, RETRY_ROW)).item); + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY_ROW}`, { ...served, label: 'CUN retry (edited)' }); + expect(res.status, await res.clone().text()).toBe(200); + const atRest = bodyOf((await stored(RETRY_ROW))[0]); + expect(atRest.label).toBe('CUN retry (edited)'); + expect(atRest.retryConfig).toEqual({ maxAttempts: 2 }); + expect(atRest.providerConfig.defaultHeaders).toEqual(HEADERS_AT_REST()); + }); + + it('CONTROL: a save that leaves `retryConfig` out removes it with its stored keys, as before', async () => { + const { retryConfig: _removed, ...withoutRetry } = authorable((await read(admin, RETRY_OMIT)).item); + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY_OMIT}`, withoutRetry); + expect(res.status, await res.clone().text()).toBe(200); + const atRest = bodyOf((await stored(RETRY_OMIT))[0]); + expect(atRest).not.toHaveProperty('retryConfig'); + expect(atRest.providerConfig.defaultHeaders).toEqual(HEADERS_AT_REST()); + }); + + it('CONTROL: a `retryConfig` stored as a non-object is withheld whole and still carried forward by a save that leaves it out', async () => { + const served = authorable((await read(admin, RETRY_MALFORMED)).item); + expect(served).not.toHaveProperty('retryConfig'); + const res = await stack.apiAs(admin, 'PUT', `/meta/connector/${RETRY_MALFORMED}`, { ...served, label: 'CUN retry malformed (edited)' }); + expect(res.status, await res.clone().text()).toBe(200); + const atRest = bodyOf((await stored(RETRY_MALFORMED))[0]); + expect(atRest.label).toBe('CUN retry malformed (edited)'); + expect(atRest.retryConfig).toBe(PLACEHOLDERS.nested); + }); + }); +}); diff --git a/packages/spec/scripts/build-schemas-check-mode.deleted-baseline.test.ts b/packages/spec/scripts/build-schemas-check-mode.deleted-baseline.test.ts index b7adcbb5bb8..ca21e71e98d 100644 --- a/packages/spec/scripts/build-schemas-check-mode.deleted-baseline.test.ts +++ b/packages/spec/scripts/build-schemas-check-mode.deleted-baseline.test.ts @@ -95,13 +95,17 @@ const DELETED_UNREACHABLE = 'api/SessionResponse:zzOverCollected4650'; /** #17356's pin. A def whose ONLY root is an UNREGISTERED KIND — `connector`, * bound in `UNREGISTERED_KIND_SCHEMAS` by #6245 and deliberately absent from - * `listMetadataTypeSchemaTypes()`. `integration/RetryConfig` sits two hops - * from that root (`connector.retryConfig`, unwrapped once through `optional`), - * and `stack.connectors[]` / `PUT /api/v1/meta/connector/:name` both parse a - * real metadata document through it. (The def was `integration/DataSyncConfig`, - * the measured specimen below, until protocol 18 retired it whole with - * `connector.syncConfig` — ADR-0049; `RetryConfig` is the connector's other - * open block on the same root at the same depth.) + * `listMetadataTypeSchemaTypes()`. `integration/DeclarativeConnectorEntry` is + * that root's own def — the schema `stack.connectors[]` and + * `PUT /api/v1/meta/connector/:name` parse a real metadata document against — + * so the eager graph reaches it by identity. (The def was + * `integration/DataSyncConfig`, the measured specimen below, until protocol 18 + * retired it whole with `connector.syncConfig` — ADR-0049; then + * `integration/RetryConfig`, two hops down at `connector.retryConfig`, until the + * authored entry parsed that block through its own closed twin. The twin shares + * `RetryConfig`'s field instances, so the gate still reaches the open def, but + * through its derived-clone bridge — a different reading from the root-graph + * hit this pin asserts in the eager regime.) * * Until #17356 the gate built its roots from `listMetadataTypeSchemaTypes()` * alone, so this def read `null` and check (c) proof 2 WAIVED a bare deletion @@ -115,7 +119,7 @@ const DELETED_UNREACHABLE = 'api/SessionResponse:zzOverCollected4650'; * ever sees a key the build STOPPED emitting, and the def is judged by its * DEF half (`key.slice(0, key.indexOf(':'))`), so a synthetic leaf under the * real def runs the identical code path as the real deletion did. */ -const DELETED_VIA_UNREGISTERED_KIND_DEF = 'integration/RetryConfig'; +const DELETED_VIA_UNREGISTERED_KIND_DEF = 'integration/DeclarativeConnectorEntry'; const DELETED_VIA_UNREGISTERED_KIND = `${DELETED_VIA_UNREGISTERED_KIND_DEF}:zzOnlyRootIsAnUnregisteredKind17356`; @@ -491,7 +495,8 @@ describe('build-schemas.ts — deleted baseline lines must prove themselves (#46 // `listMetadataTypeSchemaTypes()`, which per #6245 deliberately does not // enumerate `UNREGISTERED_KIND_SCHEMAS`. `connector` lives there, so the // BFS never started from it, `integration/DataSyncConfig` (then this - // fixture's def; `integration/RetryConfig` since protocol 18) read `null`, + // fixture's def; `integration/DeclarativeConnectorEntry`, the connector + // root's own def, since the entry closed its nested blocks) read `null`, // and a bare deletion of one of its baseline lines was waived as // over-collection — for a def `stack.connectors[]` parses on every boot. // The gate now enumerates its own reachability root union; the KIND diff --git a/packages/spec/src/api/protocol.zod.ts b/packages/spec/src/api/protocol.zod.ts index 744b13f7428..db936c67d74 100644 --- a/packages/spec/src/api/protocol.zod.ts +++ b/packages/spec/src/api/protocol.zod.ts @@ -1234,7 +1234,8 @@ export const SaveMetaItemResponseSchema = lazySchema(() => z.object({ + 'author did not write, on a draft save too: one `warning` with rule ' + '`withheld-key-dropped` for each stored position it dropped because the ' + 'type\'s read redactor withholds it and marks it not carried forward ' - + '(a connector\'s undeclared `auth` keys, which no door accepts and nothing ' + + '(a connector\'s undeclared `auth` keys and the undeclared keys of its ' + + '`retryConfig`, which no door accepts and nothing ' + 'reads). Its `path` names where the value sat at rest, in the same ' + 'spelling (`connectors[0].auth.token`).', ), diff --git a/packages/spec/src/integration/connector-credential-redaction.test.ts b/packages/spec/src/integration/connector-credential-redaction.test.ts index 5ba6764f497..e6ab490e1bf 100644 --- a/packages/spec/src/integration/connector-credential-redaction.test.ts +++ b/packages/spec/src/integration/connector-credential-redaction.test.ts @@ -265,6 +265,43 @@ describe('the connector redactor — credentials inside a URL under `providerCon }); }); +describe('the connector redactor — credentials inside a URL under `auth`', () => { + const oauth2 = (over: Record) => ({ + type: 'oauth2', grant: 'authorization_code', clientId: 'client-1', credentialRef: 'V_CLIENT_SECRET', + authorizationUrl: 'https://auth.vendor.example/authorize', tokenUrl: 'https://auth.vendor.example/token', ...over, + }); + + it('rewrites the userinfo out of `auth.tokenUrl` and `auth.authorizationUrl`, keeping the rest byte for byte', () => { + const { item: served, redactedKeys } = redactConnectorItem({ + ...base, + auth: oauth2({ + tokenUrl: `https://c:${P(60)}@auth.vendor.example:8443/oauth/token?region=eu`, + authorizationUrl: `https://${P(61)}@auth.vendor.example/authorize`, + }), + }); + expect(redactedKeys).toEqual(['auth.authorizationUrl', 'auth.tokenUrl']); + expect(served.auth).toEqual(oauth2({ + tokenUrl: 'https://auth.vendor.example:8443/oauth/token?region=eu', + authorizationUrl: 'https://auth.vendor.example/authorize', + })); + expect(holds(served, P(60), P(61))).toBe(false); + }); + + it('strips a credential query parameter from `auth.resource`', () => { + const { item: served, redactedKeys } = redactConnectorItem({ + ...base, + auth: oauth2({ resource: `https://api.vendor.example/?audience=v1&token=${P(62)}` }), + }); + expect(redactedKeys).toEqual(['auth.resource']); + expect((served.auth as Record).resource).toBe('https://api.vendor.example/?audience=v1'); + }); + + it('declared `auth` URLs with no credential part are untouched', () => { + const item = { ...base, auth: oauth2({ resource: 'https://api.vendor.example/' }) }; + expect(redactConnectorItem(item)).toEqual({ item, redactedKeys: [] }); + }); +}); + describe('the connector redactor — top-level keys the entry does not declare', () => { it.each([ ['a credential-shaped scalar', { apiKey: P(50) }, ['apiKey']], @@ -419,8 +456,9 @@ describe('the connector redactor — purity and the untouched remainder', () => }); }); -// [#28640 ②B, ruling 6566912252] Which withheld positions a save does NOT carry -// forward: the undeclared `auth` keys, and nothing else. +// [#28640 ②B, ruling 6566912252; ruling 6593804783] Which withheld positions a +// save does NOT carry forward: the undeclared `auth` keys and the undeclared +// keys of an object `retryConfig`, and nothing else. describe('the connector redactor — withheld and not carried forward', () => { it('names every undeclared `auth` key in `notCarriedForwardKeys` as well as in `redactedKeys`', () => { const { item: served, redactedKeys, notCarriedForwardKeys } = redactConnectorItem({ @@ -459,7 +497,6 @@ describe('the connector redactor — withheld and not carried forward', () => { ...base, apiKey: P(73), authentication: { type: 'api-key', key: P(74) }, - retryConfig: { strategy: 'exponential_backoff', secret: P(75) }, actions: [{ key: 'get_invoice', label: 'Get invoice', token: P(76) }], providerConfig: { baseUrl: `https://u:${P(77)}@v.example`, @@ -471,11 +508,42 @@ describe('the connector redactor — withheld and not carried forward', () => { }); expect(result.redactedKeys).toEqual([ 'actions.0.token', 'apiKey', 'authentication.key', 'providerConfig.baseUrl', 'providerConfig.defaultHeaders', - 'providerConfig.headers', 'providerConfig.transport.env', 'providerConfig.transport.headers', 'retryConfig.secret', + 'providerConfig.headers', 'providerConfig.transport.env', 'providerConfig.transport.headers', ]); expect(result).not.toHaveProperty('notCarriedForwardKeys'); }); + it('names every undeclared key of an object `retryConfig` in `notCarriedForwardKeys` as well as in `redactedKeys`', () => { + const { item: served, redactedKeys, notCarriedForwardKeys } = redactConnectorItem({ + ...base, retryConfig: { strategy: 'exponential_backoff', maxAttempts: 2, secret: P(75), token: P(85) }, + }); + expect(redactedKeys).toEqual(['retryConfig.secret', 'retryConfig.token']); + expect(notCarriedForwardKeys).toEqual(['retryConfig.secret', 'retryConfig.token']); + expect(served.retryConfig).toEqual({ strategy: 'exponential_backoff', maxAttempts: 2 }); + }); + + it('a cleared undeclared `retryConfig` key is withheld and not carried forward too', () => { + const { redactedKeys, notCarriedForwardKeys } = redactConnectorItem({ ...base, retryConfig: { maxAttempts: 2, secret: null } }); + expect(redactedKeys).toEqual(['retryConfig.secret']); + expect(notCarriedForwardKeys).toEqual(['retryConfig.secret']); + }); + + it('CONTROL: a `retryConfig` that is not an object is withheld whole and still carried forward', () => { + const result = redactConnectorItem({ ...base, retryConfig: P(86) }); + expect(result.redactedKeys).toEqual(['retryConfig']); + expect(result).not.toHaveProperty('notCarriedForwardKeys'); + }); + + it('beside an `actions` element\'s undeclared key, only the `retryConfig` key is marked', () => { + const { redactedKeys, notCarriedForwardKeys } = redactConnectorItem({ + ...base, + retryConfig: { strategy: 'exponential_backoff', secret: P(87) }, + actions: [{ key: 'get_invoice', label: 'Get invoice', token: P(88) }], + }); + expect(redactedKeys).toEqual(['actions.0.token', 'retryConfig.secret']); + expect(notCarriedForwardKeys).toEqual(['retryConfig.secret']); + }); + it('CONTROL: `auth` that is not an object is withheld whole and still carried forward', () => { const result = redactConnectorItem({ ...base, auth: P(81) }); expect(result.redactedKeys).toEqual(['auth']); diff --git a/packages/spec/src/integration/connector-credential-redaction.ts b/packages/spec/src/integration/connector-credential-redaction.ts index 37b16d48e4d..fa79044245a 100644 --- a/packages/spec/src/integration/connector-credential-redaction.ts +++ b/packages/spec/src/integration/connector-credential-redaction.ts @@ -58,8 +58,8 @@ * one came from outside them, and what it holds cannot be judged key by * key: an `auth: ''` is the secret itself. Withheld whole, fail * closed, rather than served as stored. - * 5. **The credential parts of every URL under `providerConfig`** — rewritten, - * not removed: the userinfo (`https://user:pass@host`, and the + * 5. **The credential parts of every URL under `providerConfig` and + * `auth`** — rewritten, not removed: the userinfo (`https://user:pass@host`, and the * `https://@host` form several HTTP APIs read as a token, so the * user part goes too) and each credential query parameter * ({@link CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES}, plus the @@ -70,10 +70,16 @@ * It covers `providerConfig.baseUrl` (`rest`, `openapi`), * `providerConfig.transport.url` (`mcp` over http) and a URL * `providerConfig.spec` (`openapi`), and any other URL a provider keeps at - * an OBJECT depth of `providerConfig`: a string is judged when it starts - * with `scheme://`. - * 6. **An array under `providerConfig` that holds such a URL, WHOLE**, and - * **a key under `providerConfig` whose own name carries a dot that holds + * an OBJECT depth of `providerConfig`; and the endpoints an instance-auth + * kind declares (`auth.tokenUrl`, `auth.authorizationUrl`, `auth.resource`) + * and any URL a kind declares later. A string is judged when it starts + * with `scheme://`. What a credential part is, and the walk that finds + * every URL holding one, live in ONE place, + * `./connector-url-credentials.ts`, which the write door's refinement + * runs too: an authored URL holding one is refused, so only a row stored + * before that refusal can carry one to this read. + * 6. **An array under `providerConfig` or `auth` that holds such a URL, + * WHOLE**, and **a key under them whose own name carries a dot that holds * such a URL at any depth, WHOLE.** The carry-forward cannot address an * array element by position, nor a dotted key segment by segment (its * dotted name is indistinguishable from a nested path once `redactedKeys` @@ -115,6 +121,19 @@ * judged. An element is never removed on its own, which would shift every * later index; the carry-forward keeps the stored value at rest. * + * **A key inside an object `retryConfig`: withheld and NOT carried + * forward** (the maintainer's ruling 6593804783): each is named in + * `notCarriedForwardKeys` as well as in `redactedKeys`, as item 1's are. + * The authored `retryConfig` is closed, so every door refuses such a key, + * and nothing at runtime reads one: the instance materialization parses + * the stored policy with the open {@link RetryConfigSchema}, which drops + * it, and hands the provider only the parsed policy. So a save that sends + * `retryConfig` — the served form, `{}`, any object — replaces the stored + * container whole, and the save names each key it dropped; a save that + * leaves `retryConfig` out removes the container, as it always did. An + * `actions` element's key, and a `retryConfig` that is not an object + * (withheld whole), stay carried forward. + * * ## Cleared forms, and what is never touched * * A position holding nothing — `null`, `''`, or an empty map — is served as @@ -137,72 +156,36 @@ import { ConnectorInstanceAuthSchema } from '../shared/connector-auth.zod'; import { ConnectorActionSchema, DeclarativeConnectorEntrySchema, RetryConfigSchema } from './connector.zod'; import { METADATA_READ_DECORATIONS } from '../kernel/metadata-read-decorations'; -import { CREDENTIAL_URL_QUERY_PARAM_NAMES } from '../data/driver/common.zod'; import { redactUrlCredentialQueryParams } from '../data/datasource-credential-redaction'; +import { + CONNECTOR_CREDENTIAL_MAP_PATHS, + CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES, + URL_USERINFO_RE, + connectorCredentialQueryParamsOf, + connectorUrlCredentialPositionOf, + findConnectorUrlCredentials, +} from './connector-url-credentials'; import { objectShapeOf, wrapperChainOf, zodDefOf } from '../kernel/metadata-credential-fields'; import type { MetadataRedactionResult } from '../kernel/metadata-type-redaction'; /** The map positions under `providerConfig` withheld whole (see the module note, item 3). */ -export const CONNECTOR_CREDENTIAL_MAP_PATHS: readonly (readonly string[])[] = Object.freeze([ - ['providerConfig', 'headers'], - ['providerConfig', 'defaultHeaders'], - ['providerConfig', 'transport', 'headers'], - ['providerConfig', 'transport', 'env'], -]); +export { CONNECTOR_CREDENTIAL_MAP_PATHS }; /** * The query parameters whose value a connector URL is read as carrying a - * credential in (see the module note, item 5): the datasource set - * (`password`, `authToken`) and the names HTTP APIs conventionally take a key, - * token or signature under. Matched case-insensitively on the decoded name - * (`credentialQueryParamOf`). A name here that holds no secret on some API - * costs that API's served URL the parameter, never its stored value: the - * carry-forward keeps it on a save of the served form. - */ -export const CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES: readonly string[] = Object.freeze([ - ...CREDENTIAL_URL_QUERY_PARAM_NAMES, - 'access_token', - 'api_key', - 'api-key', - 'apikey', - 'auth_token', - 'client_secret', - 'id_token', - 'key', - 'passwd', - 'refresh_token', - 'secret', - 'sig', - 'signature', - 'token', - 'x-api-key', -]); - -/** `scheme://` — the strings item 5 judges. */ -const URL_SCHEME_RE = /^[a-z][a-z0-9+.\-]*:\/\//i; - -/** - * `scheme://userinfo@` — the whole userinfo. Every class excludes `/?#`, so the - * `@` must sit in the authority; the greedy class backtracks to the LAST `@` - * before the path, the userinfo boundary RFC 3986 defines (the reading - * `URL_USERINFO_RE` in the datasource module documents). + * credential in (see the module note, item 5) — declared once, beside the + * walk the write door shares (`./connector-url-credentials.ts`). A name here + * that holds no secret on some API costs that API's served URL the parameter, + * never its stored value: the carry-forward keeps it on a save of the served + * form. */ -const URL_USERINFO_RE = /^([a-z][a-z0-9+.\-]*:\/\/)[^/?#]*@/i; +export { CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES }; /** One URL with its userinfo and credential query parameters removed; the input when there is none. */ function redactConnectorUrl(value: string, params: readonly string[]): string { return redactUrlCredentialQueryParams(value.replace(URL_USERINFO_RE, '$1'), params); } -/** The query parameter names item 5 strips for `item`: the fixed set plus its own `auth.paramName`. */ -function credentialQueryParamsOf(item: Record): readonly string[] { - const auth = item.auth; - const own = isPlainRecord(auth) && auth.type === 'api-key' && typeof auth.paramName === 'string' && auth.paramName !== '' - ? auth.paramName - : undefined; - return own === undefined ? CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES : [...CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES, own]; -} - /** One instance-auth kind as the union declares it: its literal-valued keys, and every key it declares. */ interface DeclaredAuthKind { /** Each key whose schema is a literal, with the values it accepts (`type`, `grant`, `scheme`). */ @@ -354,19 +337,17 @@ function nestedKeys(): { retryConfig: ReadonlySet; action: ReadonlySet): string[][] { const declared = nestedKeys(); const paths: string[][] = []; const retryConfig = item.retryConfig; - if (isPlainRecord(retryConfig)) { - for (const key of Object.keys(retryConfig)) if (!declared.retryConfig.has(key)) paths.push(['retryConfig', key]); - } else if (!isClearedForm(retryConfig)) { - paths.push(['retryConfig']); - } + if (!isPlainRecord(retryConfig) && !isClearedForm(retryConfig)) paths.push(['retryConfig']); const actions = item.actions; if (Array.isArray(actions) && actions.every(isPlainRecord)) { actions.forEach((action, index) => { @@ -380,6 +361,19 @@ function undeclaredNestedKeysOf(item: Record): string[][] { return paths; } +/** + * Item 8, the half NOT carried forward: every key an object `retryConfig` + * does not declare, as item-relative paths — withheld whatever its value (see + * the module note). `[]` when `retryConfig` is not an object + * ({@link undeclaredNestedKeysOf} withholds that whole, carried forward). + */ +function undeclaredRetryConfigKeysOf(item: Record): string[][] { + const retryConfig = item.retryConfig; + if (!isPlainRecord(retryConfig)) return []; + const declared = nestedKeys().retryConfig; + return Object.keys(retryConfig).filter((key) => !declared.has(key)).map((key) => ['retryConfig', key]); +} + /** * Item 1: every key the item's instance-auth kind does not declare, as * item-relative paths — withheld whatever its value, and not carried forward @@ -415,64 +409,43 @@ function withValueAt(item: Record, path: readonly string[], val return { ...item, [head]: withValueAt(item[head] as Record, rest, value) }; } -/** Whether `value` is a URL whose credential parts {@link redactConnectorUrl} removes. */ -function isCredentialUrl(value: string, params: readonly string[]): boolean { - return URL_SCHEME_RE.test(value) && redactConnectorUrl(value, params) !== value; -} - -/** Whether `value` holds, at any depth, a URL {@link isCredentialUrl} judges (item 6). */ -function holdsCredentialUrl(value: unknown, params: readonly string[]): boolean { - if (typeof value === 'string') return isCredentialUrl(value, params); - if (Array.isArray(value)) return value.some((element) => holdsCredentialUrl(element, params)); - if (isPlainRecord(value)) return Object.values(value).some((child) => holdsCredentialUrl(child, params)); - return false; -} - /** - * Items 5 and 6: every URL under `providerConfig`, at object depth, whose - * credential parts {@link redactConnectorUrl} removes — the path and the - * rewritten value — and every array there holding such a URL, withheld whole - * (`value` absent). + * Items 5 and 6: every URL the write door's own walk returns + * ({@link findConnectorUrlCredentials} — the one traversal both doors run), at + * the position {@link connectorUrlCredentialPositionOf} names. A URL reached + * through object keys alone is rewritten in place — the path and the value + * {@link redactConnectorUrl} leaves. An array holding such a URL, and a key + * whose own name carries a dot holding one at any depth, cannot be addressed + * part for part, so each is withheld WHOLE (`value` absent), once however many + * URLs it holds: fail closed, and the carry-forward keeps the stored value at + * rest by the whole-string fallback. Item 5's "avoid silent deletion" yields to + * item 7's fail-closed there. */ function credentialUrlRewritesOf( item: Record, ): Array<{ path: string[]; value?: string }> { const rewrites: Array<{ path: string[]; value?: string }> = []; - const providerConfig = item.providerConfig; - if (!isPlainRecord(providerConfig)) return rewrites; - const params = credentialQueryParamsOf(item); - const walk = (node: Record, prefix: readonly string[]): void => { - for (const [key, value] of Object.entries(node)) { - const path = [...prefix, key]; - // A key whose own name carries a dot cannot be addressed segment by - // segment (its dotted name is indistinguishable from a nested path once - // `redactedKeys` joins on dots), so it is never rewritten in place or - // descended into. Fail closed instead: if it holds a credential URL at - // any depth, withhold the key WHOLE — the carry-forward keeps the stored - // value at rest by the whole-string fallback. Item 5's "avoid silent - // deletion" yields to item 7's fail-closed here. - if (key.includes('.')) { - if (holdsCredentialUrl(value, params)) rewrites.push({ path }); - continue; - } - if (typeof value === 'string') { - if (isCredentialUrl(value, params)) rewrites.push({ path, value: redactConnectorUrl(value, params) }); - } else if (Array.isArray(value)) { - if (holdsCredentialUrl(value, params)) rewrites.push({ path }); - } else if (isPlainRecord(value)) { - walk(value, path); - } + const withheld = new Set(); + const params = connectorCredentialQueryParamsOf(item); + for (const found of findConnectorUrlCredentials(item)) { + const position = connectorUrlCredentialPositionOf(found.path); + if (position.rewrite) { + rewrites.push({ path: position.path, value: redactConnectorUrl(found.url, params) }); + continue; } - }; - walk(providerConfig, ['providerConfig']); + const key = JSON.stringify(position.path); + if (withheld.has(key)) continue; + withheld.add(key); + rewrites.push({ path: position.path }); + } return rewrites; } /** * The built-in `connector` redactor: withholds the positions the module note - * lists, names each in `redactedKeys` (dotted, sorted) and item 1's in - * `notCarriedForwardKeys` too, and returns the input by reference when there - * is nothing to withhold. + * lists, names each in `redactedKeys` (dotted, sorted) and item 1's and an + * object `retryConfig`'s undeclared keys in `notCarriedForwardKeys` too, and + * returns the input by reference when there is nothing to withhold. */ export function redactConnectorItem(item: Record): MetadataRedactionResult { let served = item; @@ -486,9 +459,10 @@ export function redactConnectorItem(item: Record): MetadataReda served = withoutKeyAt(served, path); redactedKeys.push(path.join('.')); } - // Item 1: an undeclared `auth` key, whatever its value, withheld and not - // carried forward — a save of the served form drops it. - for (const path of undeclaredAuthKeysOf(item)) { + // Item 1, and item 8's keys of an object `retryConfig`: an undeclared key, + // whatever its value, withheld and not carried forward — a save of the + // served form drops it. + for (const path of [...undeclaredRetryConfigKeysOf(item), ...undeclaredAuthKeysOf(item)]) { const { present } = readAt(served, path); if (!present) continue; served = withoutKeyAt(served, path); diff --git a/packages/spec/src/integration/connector-entry-nested-keys.test.ts b/packages/spec/src/integration/connector-entry-nested-keys.test.ts new file mode 100644 index 00000000000..c26ea216150 --- /dev/null +++ b/packages/spec/src/integration/connector-entry-nested-keys.test.ts @@ -0,0 +1,170 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// An authored connector entry (`DeclarativeConnectorEntrySchema`, behind +// `defineConnector()`, `defineStack({ connectors })`, `os validate` / +// `os compile`, the artifact load and `PUT /meta/connector/:name`) refuses a +// key its two nested shapes do not declare — inside `retryConfig`, and inside +// each `actions` element — as it refuses one at its own top level. Every +// refusal is pinned by its issue `code`, `path` and `keys`, with a CONTROL +// accept beside it. The open carriers a plugin's connector def and the runtime +// use (`ConnectorSchema`, `RetryConfigSchema`, `ConnectorActionSchema`) keep +// their posture, pinned as controls. +// +// Every value below is an obviously-fake placeholder, and the refusal must +// never echo it. + +import { describe, it, expect } from 'vitest'; +import { ZodError } from 'zod'; +import { + ConnectorActionSchema, + ConnectorSchema, + DeclarativeConnectorEntrySchema, + RetryConfigSchema, + defineConnector, +} from './connector.zod'; +import { ObjectStackDefinitionSchema } from '../stack.zod'; + +const PLACEHOLDER = 'TEST_PLACEHOLDER_NOT_A_SECRET'; + +const INSTANCE = { + name: 'vendor', + label: 'Vendor', + type: 'api', + provider: 'rest', + providerConfig: { baseUrl: 'https://vendor.example.com' }, + auth: { type: 'bearer', credentialRef: 'VENDOR_TOKEN' }, +} as const; + +const DESCRIPTOR = { name: 'erp', label: 'ERP', type: 'saas' } as const; + +const RETRY = { strategy: 'exponential_backoff', maxAttempts: 2, initialDelayMs: 500, jitter: false } as const; +const ACTION = { key: 'get_invoice', label: 'Get invoice', description: 'One invoice.', effect: 'read', inputSchema: { type: 'object' } } as const; + +/** The one issue of a refused parse, as `[code, path, keys]`, and its message. */ +function refusal(input: unknown): { located: [string, string, unknown]; message: string } { + const result = DeclarativeConnectorEntrySchema.safeParse(input); + expect(result.success, 'the entry must be refused').toBe(false); + expect(result.error!.issues, JSON.stringify(result.error!.issues)).toHaveLength(1); + const issue = result.error!.issues[0]!; + return { located: [issue.code, issue.path.join('.'), (issue as { keys?: unknown }).keys], message: issue.message }; +} + +describe('an authored connector entry refuses a key its nested shapes do not declare', () => { + it.each([ + ['an instance with every declared `retryConfig` key', { ...INSTANCE, retryConfig: RETRY }], + ['a descriptor with every declared action key', { ...DESCRIPTOR, actions: [ACTION, { key: 'list', label: 'List' }] }], + ['an empty `retryConfig` (clears the policy)', { ...INSTANCE, retryConfig: {} }], + ] as const)('CONTROL: %s parses', (_label, entry) => { + expect(DeclarativeConnectorEntrySchema.safeParse(entry).success).toBe(true); + }); + + it.each([ + ['a credential-shaped key', { secret: PLACEHOLDER }, ['secret']], + ['a misspelled policy key', { maxRetries: 4 }, ['maxRetries']], + ['several keys', { secret: PLACEHOLDER, timeoutMs: 100 }, ['secret', 'timeoutMs']], + ] as const)('`retryConfig` with %s is refused at `retryConfig`', (_label, extra, keys) => { + const { located, message } = refusal({ ...INSTANCE, retryConfig: { ...RETRY, ...extra } }); + expect(located).toEqual(['unrecognized_keys', 'retryConfig', keys]); + expect(message).not.toContain(PLACEHOLDER); + }); + + it.each([ + ['a credential-shaped key', { token: PLACEHOLDER }, ['token']], + ['an execution binding', { method: 'GET', path: '/invoices/{id}' }, ['method', 'path']], + ] as const)('an `actions` element with %s is refused at that element', (_label, extra, keys) => { + const { located, message } = refusal({ ...DESCRIPTOR, actions: [{ key: 'list', label: 'List' }, { ...ACTION, ...extra }] }); + expect(located).toEqual(['unrecognized_keys', 'actions.1', keys]); + expect(message).not.toContain(PLACEHOLDER); + }); + + it('inside a stack, the issue is located under the entry', () => { + const result = ObjectStackDefinitionSchema.safeParse({ + manifest: { id: 'com.example.app', name: 'app', version: '1.0.0', type: 'app' }, + connectors: [{ ...INSTANCE, retryConfig: { ...RETRY, secret: PLACEHOLDER } }], + }); + expect(result.success).toBe(false); + expect(result.error!.issues.map((issue) => issue.path.join('.'))).toContain('connectors.0.retryConfig'); + }); + + it('`defineConnector()` refuses it too, and accepts the declared keys', () => { + let thrown: unknown; + try { + defineConnector({ ...DESCRIPTOR, actions: [{ ...ACTION, token: PLACEHOLDER } as never] }); + } catch (error) { + thrown = error; + } + expect(thrown, 'defineConnector must refuse the undeclared action key').toBeInstanceOf(ZodError); + expect((thrown as ZodError).issues.map((issue) => [issue.code, issue.path.join('.'), (issue as { keys?: unknown }).keys])) + .toEqual([['unrecognized_keys', 'actions.0', ['token']]]); + expect(() => defineConnector({ ...DESCRIPTOR, actions: [ACTION] })).not.toThrow(); + }); + + it('CONTROL: the open carriers keep their posture (an undeclared nested key is dropped, not refused)', () => { + const def = ConnectorSchema.parse({ ...DESCRIPTOR, retryConfig: { ...RETRY, secret: PLACEHOLDER }, actions: [{ ...ACTION, token: PLACEHOLDER }] }); + expect(JSON.stringify(def)).not.toContain(PLACEHOLDER); + expect(RetryConfigSchema.safeParse({ ...RETRY, secret: PLACEHOLDER }).success).toBe(true); + expect(ConnectorActionSchema.safeParse({ ...ACTION, token: PLACEHOLDER }).success).toBe(true); + }); + + describe('the prescription each refusal carries', () => { + const CREDENTIAL = ['connector metadata never holds a credential', 'auth: { type, credentialRef }']; + const BINDING = ['an action carries no execution binding (ADR-0097 §5)', 'derives its actions from the upstream']; + + it.each(['secret', 'token', 'apiKey', 'key', 'password', 'credentials', 'credentialRef'])( + 'a credential-shaped `%s` inside `retryConfig` meets the entry\'s credential prescription', + (name) => { + const { located, message } = refusal({ ...INSTANCE, retryConfig: { ...RETRY, [name]: PLACEHOLDER } }); + expect(located).toEqual(['unrecognized_keys', 'retryConfig', [name]]); + for (const needle of CREDENTIAL) expect(message).toContain(needle); + expect(message).not.toContain(PLACEHOLDER); + }, + ); + + it.each(['secret', 'token', 'apiKey', 'password', 'credentialRef'])( + 'a credential-shaped `%s` inside an `actions` element meets the entry\'s credential prescription', + (name) => { + const { located, message } = refusal({ ...DESCRIPTOR, actions: [{ ...ACTION, [name]: PLACEHOLDER }] }); + expect(located).toEqual(['unrecognized_keys', 'actions.0', [name]]); + for (const needle of CREDENTIAL) expect(message).toContain(needle); + expect(message).not.toContain(PLACEHOLDER); + }, + ); + + it.each(['method', 'path', 'url'])('an execution binding `%s` on an action meets ADR-0097 §5\'s prescription', (name) => { + const { located, message } = refusal({ ...DESCRIPTOR, actions: [{ ...ACTION, [name]: 'x' }] }); + expect(located).toEqual(['unrecognized_keys', 'actions.0', [name]]); + for (const needle of BINDING) expect(message).toContain(needle); + }); + + it('CONTROL: an action\'s own `key` is its declared machine name, never a misplaced credential', () => { + expect(DeclarativeConnectorEntrySchema.safeParse({ ...DESCRIPTOR, actions: [ACTION] }).success).toBe(true); + }); + }); + + it('the closed nested shapes declare exactly the open shapes\' keys, as the same field instances', () => { + const entry = DeclarativeConnectorEntrySchema.safeParse({ ...INSTANCE, retryConfig: RETRY }); + expect(entry.success).toBe(true); + const shapeOf = (schema: unknown) => (schema as { shape: Record }).shape; + const unwrapped = (schema: unknown): unknown => { + let node = schema as { unwrap?: () => unknown; shape?: unknown }; + while (node.shape === undefined && typeof node.unwrap === 'function') node = node.unwrap() as typeof node; + return node; + }; + const entryShape = shapeOf(DeclarativeConnectorEntrySchema); + const retryTwin = shapeOf(unwrapped(entryShape.retryConfig)); + const actionTwin = shapeOf(unwrapped(unwrapped(entryShape.actions))); + const retryKeys = Object.keys(shapeOf(RetryConfigSchema)).sort(); + const actionKeys = Object.keys(shapeOf(ConnectorActionSchema)).sort(); + expect(Object.keys(retryTwin).sort()).toEqual(retryKeys); + expect(Object.keys(actionTwin).sort()).toEqual(actionKeys); + // The same instances, so the authorable-surface gate reaches the open defs through the twins. + for (const key of retryKeys) expect(retryTwin[key], `retryConfig.${key}`).toBe(shapeOf(RetryConfigSchema)[key]); + for (const key of actionKeys) expect(actionTwin[key], `actions[].${key}`).toBe(shapeOf(ConnectorActionSchema)[key]); + // Every declared key is accepted by the closed entry … + const everyRetry = Object.fromEntries(retryKeys.map((k) => [k, (RetryConfigSchema.parse({}) as Record)[k]])); + expect(DeclarativeConnectorEntrySchema.safeParse({ ...INSTANCE, retryConfig: everyRetry }).success).toBe(true); + const everyAction = { ...ACTION, outputSchema: { type: 'object' } }; + expect(Object.keys(everyAction).sort()).toEqual(actionKeys); + expect(DeclarativeConnectorEntrySchema.safeParse({ ...DESCRIPTOR, actions: [everyAction] }).success).toBe(true); + }); +}); diff --git a/packages/spec/src/integration/connector-entry-url-credentials.test.ts b/packages/spec/src/integration/connector-entry-url-credentials.test.ts new file mode 100644 index 00000000000..21b8ac99bb5 --- /dev/null +++ b/packages/spec/src/integration/connector-entry-url-credentials.test.ts @@ -0,0 +1,226 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// An authored connector entry (`DeclarativeConnectorEntrySchema`, behind +// `defineStack({ connectors })`, `os validate` / `os compile`, the artifact load +// and `PUT /meta/connector/:name`) keeps no credential part in a URL: under +// `providerConfig` (any key, any depth, array elements included) and under +// `auth` (`tokenUrl`, `authorizationUrl`, `resource`), a URL holding a userinfo +// or a credential query parameter is refused, one issue per URL, located at +// the URL. Every refusal has a CONTROL accept beside it, so a refusal that fired +// on everything could not pass, and the last block pins that the write door +// refuses exactly what the read redactor rewrites. +// +// Every value below is an obviously-fake placeholder, and the refusal must +// never echo it. + +import { describe, it, expect } from 'vitest'; +import { ConnectorSchema, DeclarativeConnectorEntrySchema } from './connector.zod'; +import { redactConnectorItem } from './connector-credential-redaction'; +import { ObjectStackDefinitionSchema } from '../stack.zod'; + +const PLACEHOLDER = 'TEST_PLACEHOLDER_NOT_A_SECRET'; + +const rest = (baseUrl: string, auth: Record = { type: 'bearer', credentialRef: 'VENDOR_TOKEN' }) => ({ + name: 'vendor', + label: 'Vendor', + type: 'api', + provider: 'rest', + providerConfig: { baseUrl }, + auth, +}); + +const mcp = (url: string) => ({ + name: 'tools', + label: 'Tools', + type: 'api', + provider: 'mcp', + providerConfig: { transport: { kind: 'http', url } }, +}); + +const openapi = (spec: string) => ({ + name: 'billing', + label: 'Billing', + type: 'api', + provider: 'openapi', + providerConfig: { spec }, + auth: { type: 'bearer', credentialRef: 'BILLING_TOKEN' }, +}); + +const oauth2 = (over: Record) => rest('https://api.vendor.example.com', { + type: 'oauth2', + grant: 'authorization_code', + authorizationUrl: 'https://auth.vendor.example.com/authorize', + tokenUrl: 'https://auth.vendor.example.com/token', + clientId: 'client-1', + credentialRef: 'VENDOR_CLIENT_SECRET', + ...over, +}); + +/** Every issue of a refused parse, as `[code, path]`, and the messages. */ +function refusals(input: unknown): { located: Array<[string, string]>; messages: string[] } { + const result = DeclarativeConnectorEntrySchema.safeParse(input); + expect(result.success, 'the entry must be refused').toBe(false); + const issues = result.error!.issues; + return { + located: issues.map((issue) => [issue.code, issue.path.join('.')]), + messages: issues.map((issue) => issue.message), + }; +} + +const accepts = (input: unknown): void => { + const result = DeclarativeConnectorEntrySchema.safeParse(input); + expect(result.success, JSON.stringify(result.success ? null : result.error.issues)).toBe(true); +}; + +describe('an authored connector URL holds no credential part', () => { + describe('CONTROL: a URL with no credential part is accepted at every position', () => { + it.each([ + ['a rest baseUrl', rest('https://api.vendor.example.com/v1')], + ['a baseUrl with an ordinary query', rest('https://api.vendor.example.com/v1?region=eu&page=2')], + ['a baseUrl whose credential parameter is empty', rest('https://api.vendor.example.com/v1?token=&api_key=')], + ['a baseUrl whose credential name sits in the fragment', rest(`https://api.vendor.example.com/v1#token=${PLACEHOLDER}`)], + ['an mcp http transport url', mcp('https://mcp.vendor.example.com/mcp')], + ['an openapi spec URL', openapi('https://api.vendor.example.com/openapi.json')], + ['an openapi spec that is a path, not a URL', openapi('./specs/billing.yaml')], + ['oauth2 endpoints and resource', oauth2({ resource: 'https://api.vendor.example.com/' })], + ['a credentialRef-only instance', rest('https://api.vendor.example.com', { type: 'api-key', credentialRef: 'VENDOR_KEY', paramName: 'key' })], + ] as const)('%s', (_label, entry) => accepts(entry)); + }); + + describe('REFUSED: a userinfo or a credential query parameter, at each URL position', () => { + it.each([ + ['a userinfo password in the baseUrl', rest(`https://user:${PLACEHOLDER}@api.vendor.example.com/v1`), 'providerConfig.baseUrl'], + ['a userinfo token in the baseUrl', rest(`https://${PLACEHOLDER}@api.vendor.example.com/v1`), 'providerConfig.baseUrl'], + ['`api_key` in the baseUrl query', rest(`https://api.vendor.example.com/v1?api_key=${PLACEHOLDER}`), 'providerConfig.baseUrl'], + ['`token` in the baseUrl query', rest(`https://api.vendor.example.com/v1?region=eu&token=${PLACEHOLDER}`), 'providerConfig.baseUrl'], + ['`access_token` in the baseUrl query', rest(`https://api.vendor.example.com/v1?access_token=${PLACEHOLDER}`), 'providerConfig.baseUrl'], + ['a credential name spelled in another case', rest(`https://api.vendor.example.com/v1?Access_Token=${PLACEHOLDER}`), 'providerConfig.baseUrl'], + ['a percent-encoded credential name', rest(`https://api.vendor.example.com/v1?api%5Fkey=${PLACEHOLDER}`), 'providerConfig.baseUrl'], + [ + 'the instance\'s own `auth.paramName` in the baseUrl query', + rest(`https://api.vendor.example.com/v1?vendor_key=${PLACEHOLDER}`, { type: 'api-key', credentialRef: 'VENDOR_KEY', paramName: 'vendor_key' }), + 'providerConfig.baseUrl', + ], + ['a userinfo password in an mcp transport url', mcp(`https://user:${PLACEHOLDER}@mcp.vendor.example.com/mcp`), 'providerConfig.transport.url'], + ['`token` in an openapi spec URL', openapi(`https://api.vendor.example.com/openapi.json?token=${PLACEHOLDER}`), 'providerConfig.spec'], + ['a userinfo password in `auth.tokenUrl`', oauth2({ tokenUrl: `https://c:${PLACEHOLDER}@auth.vendor.example.com/token` }), 'auth.tokenUrl'], + ['`client_secret` in `auth.tokenUrl`', oauth2({ tokenUrl: `https://auth.vendor.example.com/token?client_secret=${PLACEHOLDER}` }), 'auth.tokenUrl'], + ['a userinfo in `auth.authorizationUrl`', oauth2({ authorizationUrl: `https://c:${PLACEHOLDER}@auth.vendor.example.com/authorize` }), 'auth.authorizationUrl'], + ['`key` in `auth.resource`', oauth2({ resource: `https://api.vendor.example.com/?key=${PLACEHOLDER}` }), 'auth.resource'], + ] as const)('%s', (_label, entry, at) => { + const { located, messages } = refusals(entry); + expect(located).toEqual([['custom', at]]); + expect(messages[0]).toContain(`\`${at}\``); + expect(messages[0]).toContain('auth: { type, credentialRef }'); + expect(messages[0]).not.toContain(PLACEHOLDER); + }); + + it('a URL at any depth under `providerConfig`, an array element and a dotted key included', () => { + const entry = { + ...rest('https://api.vendor.example.com'), + providerConfig: { + baseUrl: 'https://api.vendor.example.com', + mirrors: ['https://eu.vendor.example.com', `https://us.vendor.example.com?sig=${PLACEHOLDER}`], + 'x.callback': `https://hooks.vendor.example.com?secret=${PLACEHOLDER}`, + nested: { deeper: { url: `https://u:${PLACEHOLDER}@deep.vendor.example.com` } }, + }, + }; + expect(refusals(entry).located).toEqual([ + ['custom', 'providerConfig.mirrors.1'], + ['custom', 'providerConfig.x.callback'], + ['custom', 'providerConfig.nested.deeper.url'], + ]); + }); + + it('one issue per URL, naming every credential part it holds', () => { + const entry = oauth2({ tokenUrl: `https://c:${PLACEHOLDER}@auth.vendor.example.com/token?token=${PLACEHOLDER}&key=${PLACEHOLDER}` }); + const { located, messages } = refusals(entry); + expect(located).toEqual([['custom', 'auth.tokenUrl']]); + expect(messages[0]).toContain('a userinfo'); + expect(messages[0]).toContain('the `token` query parameter'); + expect(messages[0]).toContain('the `key` query parameter'); + }); + + it('inside a stack, the issue is located under the entry', () => { + const result = ObjectStackDefinitionSchema.safeParse({ + manifest: { id: 'com.example.app', name: 'app', version: '1.0.0', type: 'app' }, + connectors: [rest(`https://api.vendor.example.com?token=${PLACEHOLDER}`)], + }); + expect(result.success).toBe(false); + const paths = result.error!.issues.map((issue) => issue.path.join('.')); + expect(paths).toContain('connectors.0.providerConfig.baseUrl'); + }); + }); + + it('CONTROL: a URL inside a header or environment map is left to the map\'s own rules (the read withholds the map whole)', () => { + accepts({ + ...rest('https://api.vendor.example.com'), + providerConfig: { baseUrl: 'https://api.vendor.example.com', headers: { Referer: `https://u:${PLACEHOLDER}@app.example.com` } }, + }); + accepts({ + name: 'local_tools', label: 'Local tools', type: 'api', provider: 'mcp', + providerConfig: { transport: { kind: 'stdio', command: 'tools', env: { UPSTREAM_URL: `https://u:${PLACEHOLDER}@db.example.com` } } }, + }); + }); + + it('CONTROL: the open runtime carrier `ConnectorSchema` keeps its posture', () => { + expect(ConnectorSchema.safeParse(rest(`https://api.vendor.example.com?token=${PLACEHOLDER}`)).success).toBe(true); + }); + + describe('the write door refuses exactly what the read redactor rewrites', () => { + const urls = [ + 'https://api.vendor.example.com/v1', + 'https://api.vendor.example.com/v1?page=2', + 'https://api.vendor.example.com/v1?token=', + `https://user:${PLACEHOLDER}@api.vendor.example.com/v1`, + `https://${PLACEHOLDER}@api.vendor.example.com/v1`, + `https://api.vendor.example.com/v1?apikey=${PLACEHOLDER}`, + `https://api.vendor.example.com/v1?page=2&signature=${PLACEHOLDER}#token=x`, + `https://api.vendor.example.com/v1#api_key=${PLACEHOLDER}`, + ]; + it.each(urls)('%s', (url) => { + for (const entry of [rest(url), oauth2({ tokenUrl: url })]) { + const refused = !DeclarativeConnectorEntrySchema.safeParse(entry).success; + const rewritten = redactConnectorItem(entry as Record).redactedKeys.length > 0; + expect(refused, `${url}: refused ⇔ rewritten`).toBe(rewritten); + } + }); + + it('at every depth, one walk: each refused URL is withheld by the read, at the position it can address', () => { + const entry = { + ...rest('https://api.vendor.example.com'), + providerConfig: { + baseUrl: `https://u:${PLACEHOLDER}@api.vendor.example.com`, + mirrors: [ + 'https://eu.vendor.example.com', + `https://us.vendor.example.com?sig=${PLACEHOLDER}`, + `https://ap.vendor.example.com?token=${PLACEHOLDER}`, + ], + 'x.callback': { url: `https://hooks.vendor.example.com?secret=${PLACEHOLDER}` }, + nested: { deeper: { url: `https://u:${PLACEHOLDER}@deep.vendor.example.com` }, plain: 'https://plain.vendor.example.com' }, + headers: { Referer: `https://u:${PLACEHOLDER}@app.example.com` }, + }, + }; + expect(refusals(entry).located.map(([, path]) => path)).toEqual([ + 'providerConfig.baseUrl', + 'providerConfig.mirrors.1', + 'providerConfig.mirrors.2', + 'providerConfig.x.callback.url', + 'providerConfig.nested.deeper.url', + ]); + const { item: served, redactedKeys } = redactConnectorItem(entry); + // An array and a dotted key are withheld whole, once; the header map is + // withheld whole by its own rule (item 3), the one position the write + // door leaves to the map's own rules. + expect(redactedKeys).toEqual([ + 'providerConfig.baseUrl', + 'providerConfig.headers', + 'providerConfig.mirrors', + 'providerConfig.nested.deeper.url', + 'providerConfig.x.callback', + ]); + expect(JSON.stringify(served)).not.toContain(PLACEHOLDER); + expect((served.providerConfig as Record).nested.plain).toBe('https://plain.vendor.example.com'); + }); + }); +}); diff --git a/packages/spec/src/integration/connector-url-credentials.ts b/packages/spec/src/integration/connector-url-credentials.ts new file mode 100644 index 00000000000..a2b5fc4d72c --- /dev/null +++ b/packages/spec/src/integration/connector-url-credentials.ts @@ -0,0 +1,197 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The credential parts of a URL a connector entry keeps — the ONE definition + * both connector doors judge by. Internal: imported by the write door's + * refinement (`DeclarativeConnectorEntrySchema` in `./connector.zod.ts`) and by + * the read redactor (`./connector-credential-redaction.ts`), never exported on + * its own. + * + * ## The URLs judged + * + * Every string that starts with `scheme://` under the two places an entry + * keeps a URL ({@link CONNECTOR_URL_ROOTS}): + * + * - `providerConfig` — the provider's own vocabulary: `baseUrl` (`rest`, + * `openapi`), `transport.url` (`mcp` over http), a URL `spec` (`openapi`), + * and any other URL a provider keeps there; + * - `auth` — the endpoints an instance-auth kind declares: `tokenUrl`, + * `authorizationUrl` and `resource` (`oauth2`), and any URL a kind declares + * later. + * + * No list of provider keys: a URL is recognised by its value, so a provider + * that keeps a URL under a new key is judged with nothing to edit here. + * + * ## The credential parts + * + * - **the userinfo**, whole: `https://user:pass@host`, and `https://@host`, + * the form several HTTP APIs read as a token; + * - **each credential query parameter** with a non-empty value: the names in + * {@link CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES}, plus the + * `auth.paramName` an `api-key` instance itself names. A pair is judged by + * `credentialQueryParamOf` (`data/driver/common.zod.ts`), the boundary + * definition the datasource doors share: the query runs from the first `?` + * to the fragment, names match case-insensitively on the percent-decoded + * key, and an empty value carries nothing. + * + * ## One traversal, two doors + * + * {@link findConnectorUrlCredentials} is the only walk over an entry's URLs. + * The write door's refinement refuses each URL it returns, at that URL's path; + * the read redactor rewrites each one it can address and withholds the rest + * whole ({@link connectorUrlCredentialPositionOf}). One definition and one + * walk, so the two cannot disagree about which URLs carry a credential part. + * Neither judges a URL inside a header or environment map + * ({@link CONNECTOR_CREDENTIAL_MAP_PATHS}): the read withholds such a map + * whole. + */ + +import { CREDENTIAL_URL_QUERY_PARAM_NAMES, urlCredentialQueryParams } from '../data/driver/common.zod'; + +/** + * The header and environment maps a provider sends upstream — withheld WHOLE by + * the read redactor (a map's entries cannot be told apart), so neither door + * judges the URLs inside one: the read never rewrites inside a map, and the + * write leaves a map's entries to the authoring lint that judges them. + */ +export const CONNECTOR_CREDENTIAL_MAP_PATHS: readonly (readonly string[])[] = Object.freeze([ + ['providerConfig', 'headers'], + ['providerConfig', 'defaultHeaders'], + ['providerConfig', 'transport', 'headers'], + ['providerConfig', 'transport', 'env'], +]); + +/** The entry keys under which a connector keeps URLs (see the module note). */ +export const CONNECTOR_URL_ROOTS: readonly string[] = Object.freeze(['providerConfig', 'auth']); + +/** + * The query parameters whose value a connector URL is read as carrying a + * credential in: the datasource set (`password`, `authToken`) and the names + * HTTP APIs conventionally take a key, token or signature under. Matched + * case-insensitively on the decoded name (`credentialQueryParamOf`). + */ +export const CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES: readonly string[] = Object.freeze([ + ...CREDENTIAL_URL_QUERY_PARAM_NAMES, + 'access_token', + 'api_key', + 'api-key', + 'apikey', + 'auth_token', + 'client_secret', + 'id_token', + 'key', + 'passwd', + 'refresh_token', + 'secret', + 'sig', + 'signature', + 'token', + 'x-api-key', +]); + +/** `scheme://` — the strings judged as URLs. */ +export const URL_SCHEME_RE = /^[a-z][a-z0-9+.\-]*:\/\//i; + +/** + * `scheme://userinfo@` — the whole userinfo. Every class excludes `/?#`, so the + * `@` must sit in the authority; the greedy class backtracks to the LAST `@` + * before the path, the userinfo boundary RFC 3986 defines. + */ +export const URL_USERINFO_RE = /^([a-z][a-z0-9+.\-]*:\/\/)[^/?#]*@/i; + +const isPlainRecord = (v: unknown): v is Record => + !!v && typeof v === 'object' && !Array.isArray(v); + +/** The query parameter names judged for `item`: the fixed set plus its own `auth.paramName`. */ +export function connectorCredentialQueryParamsOf(item: Record): readonly string[] { + const auth = item.auth; + const own = isPlainRecord(auth) && auth.type === 'api-key' && typeof auth.paramName === 'string' && auth.paramName !== '' + ? auth.paramName + : undefined; + return own === undefined ? CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES : [...CONNECTOR_CREDENTIAL_URL_QUERY_PARAM_NAMES, own]; +} + +/** Whether `path` is one of {@link CONNECTOR_CREDENTIAL_MAP_PATHS}. */ +function isCredentialMapPath(path: ReadonlyArray): boolean { + return CONNECTOR_CREDENTIAL_MAP_PATHS.some( + (mapPath) => mapPath.length === path.length && mapPath.every((segment, i) => segment === path[i]), + ); +} + +/** What credential parts one URL holds. */ +export interface ConnectorUrlCredentialParts { + /** The URL carries a userinfo (`user@`, `user:password@`). */ + userinfo: boolean; + /** The credential query parameters it carries with a value, as the judged name spells them. */ + params: string[]; +} + +/** The credential parts `value` holds, or `undefined` when it is not a URL or holds none. */ +export function connectorUrlCredentialPartsOf( + value: string, + params: readonly string[], +): ConnectorUrlCredentialParts | undefined { + if (!URL_SCHEME_RE.test(value)) return undefined; + const userinfo = URL_USERINFO_RE.test(value); + const found = urlCredentialQueryParams(value, params); + return userinfo || found.length > 0 ? { userinfo, params: found } : undefined; +} + +/** One URL holding credential parts, located from the entry root. */ +export interface ConnectorUrlCredential extends ConnectorUrlCredentialParts { + /** The URL's position: object keys, and an array element's index. */ + path: Array; + /** The URL itself — what the read redactor rewrites. ⛔ Never echoed by the write door's refusal. */ + url: string; +} + +/** + * Every URL under {@link CONNECTOR_URL_ROOTS} that holds a credential part, at + * any depth — inside arrays and under keys whose own name carries a dot too — + * except inside the maps of {@link CONNECTOR_CREDENTIAL_MAP_PATHS}, whatever a + * map position holds. The one traversal both doors run (see the module note): + * the write door refuses every URL returned, the read redactor withholds each + * at its {@link connectorUrlCredentialPositionOf}. + */ +export function findConnectorUrlCredentials(item: Record): ConnectorUrlCredential[] { + const params = connectorCredentialQueryParamsOf(item); + const found: ConnectorUrlCredential[] = []; + const walk = (value: unknown, path: Array): void => { + if (isCredentialMapPath(path)) return; + if (typeof value === 'string') { + const parts = connectorUrlCredentialPartsOf(value, params); + if (parts) found.push({ path, url: value, ...parts }); + } else if (Array.isArray(value)) { + value.forEach((element, index) => walk(element, [...path, index])); + } else if (isPlainRecord(value)) { + for (const [key, child] of Object.entries(value)) walk(child, [...path, key]); + } + }; + for (const root of CONNECTOR_URL_ROOTS) { + if (isPlainRecord(item[root])) walk(item[root], [root]); + } + return found; +} + +/** + * Where the read redactor withholds a URL {@link findConnectorUrlCredentials} + * returned. The served body's withheld positions are named in `redactedKeys`, + * joined on dots, and the save-side carry-forward re-grafts the stored value by + * that name, so a position must be addressable by it: object keys without a + * dot are. The first hop along the path that is not — an array element (the + * carry-forward cannot address one by index), or a key whose own name carries + * a dot (indistinguishable from a nested path once joined) — ends the position + * there, and everything under it is withheld whole: the ARRAY itself, or the + * dotted KEY. `rewrite` is true when no such hop exists, and the URL is then + * rewritten in place, its credential parts removed. + */ +export function connectorUrlCredentialPositionOf( + path: ReadonlyArray, +): { path: string[]; rewrite: boolean } { + for (let i = 1; i < path.length; i += 1) { + const hop = path[i]!; + if (typeof hop === 'number') return { path: path.slice(0, i) as string[], rewrite: false }; + if (hop.includes('.')) return { path: path.slice(0, i + 1) as string[], rewrite: false }; + } + return { path: [...path] as string[], rewrite: true }; +} diff --git a/packages/spec/src/integration/connector.zod.ts b/packages/spec/src/integration/connector.zod.ts index 877c2a1f7d2..898e98ff3bc 100644 --- a/packages/spec/src/integration/connector.zod.ts +++ b/packages/spec/src/integration/connector.zod.ts @@ -166,6 +166,7 @@ import { strictObject } from '../shared/strict-object'; // ============================================================================ import { lazySchema } from '../shared/lazy-schema'; +import { findConnectorUrlCredentials, type ConnectorUrlCredential } from './connector-url-credentials'; // ============================================================================ // REMOVED: connector-attached sync — `syncConfig` and `fieldMappings` @@ -318,9 +319,73 @@ export const ConnectorRetryStrategySchema = lazySchema(() => z.enum([ export type ConnectorRetryStrategy = z.input; /** - * Retry Configuration + * What a credential-shaped key meets on an authored connector entry. A + * connector entry holds no secret at any key: a provider-bound instance names + * its credential with `auth.credentialRef` and the value lives in the secrets + * layer (ADR-0097 §3, ADR-0143 D3). + */ +const ENTRY_CREDENTIAL_KEY_GUIDANCE = + 'Not authorable on a connector entry — connector metadata never holds a credential. A provider-bound ' + + 'instance names its secret with `auth: { type, credentialRef }`, and the administrator sets the value ' + + 'under that name (an environment variable in the open tier, or Setup → Connector credentials).'; + +/** + * The credential-shaped keys an authored connector entry answers with + * {@link ENTRY_CREDENTIAL_KEY_GUIDANCE} — at its top level and inside each of + * its closed nested shapes (`retryConfig`, an `actions` element), so a secret + * misplaced one level down meets the same prescription. + */ +const ENTRY_CREDENTIAL_KEYS_GUIDANCE: Readonly> = Object.freeze({ + apiKey: ENTRY_CREDENTIAL_KEY_GUIDANCE, + key: ENTRY_CREDENTIAL_KEY_GUIDANCE, + token: ENTRY_CREDENTIAL_KEY_GUIDANCE, + accessToken: ENTRY_CREDENTIAL_KEY_GUIDANCE, + secret: ENTRY_CREDENTIAL_KEY_GUIDANCE, + clientSecret: ENTRY_CREDENTIAL_KEY_GUIDANCE, + password: ENTRY_CREDENTIAL_KEY_GUIDANCE, + credentials: ENTRY_CREDENTIAL_KEY_GUIDANCE, + credentialRef: ENTRY_CREDENTIAL_KEY_GUIDANCE, +}); + +/** + * What an execution binding written on an authored connector action meets: an + * action carries none (ADR-0097 §5). A provider-bound instance derives its + * actions from the upstream; a catalog descriptor's actions describe, and + * dispatch nothing. */ -export const RetryConfigSchema = lazySchema(() => z.object({ +const ENTRY_ACTION_EXECUTION_BINDING_GUIDANCE = + 'Not authorable on a connector action — an action carries no execution binding (ADR-0097 §5). A ' + + 'provider-bound instance derives its actions from the upstream (the OpenAPI document, or the MCP ' + + "server's tools), and a catalog descriptor's actions describe an integration without dispatching."; + +/** + * `make()`'s result, built on the first call and handed to every later one — so + * two carriers built from it share each field INSTANCE, not only its spelling. + */ +function sharedFields(make: () => T): () => T { + let built: { value: T } | undefined; + return () => (built ??= { value: make() }).value; +} + +/** + * The retry configuration's declared keys — one declaration, two carriers: the + * open {@link RetryConfigSchema} and the closed retry shape of an AUTHORED + * connector entry ({@link AuthoredRetryConfigSchema}). Built ONCE, so both + * carriers hold the same field instances: the closed twin is module-private + * and emits no def of its own, and the authorable-surface gate + * (`computeSurfaceReachability()`, `scripts/build-schemas.ts`) recognises the + * open `integration/RetryConfig` as authorable through the twin by exactly + * that identity (its derived-clone bridge). Built per carrier instead, the + * open def was reached by no metadata root and a bare deletion from this + * declaration read as unreachable. + * + * The cost `connectorBaseShape` below avoids at the top level — the guidance + * census pairing the open def with the closed declaration's `guidance` — is + * read here and does not bite: that pairing is consulted only for a deleted + * baseline key the guidance NAMES, and every key the twin's guidance names + * (credential-shaped keys) is one the open def has never declared. + */ +const retryConfigShape = sharedFields(() => ({ /** * Retry strategy */ @@ -362,6 +427,31 @@ export const RetryConfigSchema = lazySchema(() => z.object({ jitter: z.boolean().optional().default(true).describe('Add jitter to retry delays'), })); +/** + * Retry Configuration — the open shape: the retry policy a plugin's connector + * def carries (`ConnectorSchema`), and the shape the runtime parses a policy + * with (`resolveConnectorFetchPolicy`, the instance materialization). An + * AUTHORED entry's `retryConfig` is the closed {@link AuthoredRetryConfigSchema}. + */ +export const RetryConfigSchema = lazySchema(() => z.object(retryConfigShape())); + +/** + * The CLOSED retry shape of an authored connector entry + * ({@link DeclarativeConnectorEntrySchema}): the keys {@link RetryConfigSchema} + * declares — the same field instances ({@link retryConfigShape}) — refusing + * every other, a credential-shaped key with the entry's own prescription. An + * authored entry is stored as the body the save door accepts (ADR-0005 + * appendix (c), ruling B), so on an open shape an undeclared key here parsed + * green, was kept, and meant nothing to the runtime. Module private: the + * published carrier stays {@link RetryConfigSchema}, which the runtime keeps + * parsing a stored policy with. + */ +const AuthoredRetryConfigSchema = lazySchema(() => strictObject({ + surface: "this connector entry's `retryConfig`", + history: 'Until this shape was closed, an undeclared key here was accepted without effect.', + guidance: { ...ENTRY_CREDENTIAL_KEYS_GUIDANCE }, +}, retryConfigShape())); + export type RetryConfig = z.input; /** Post-parse shape of {@link RetryConfig} — defaults applied, transforms run (ADR-0122). */ export type RetryConfigParsed = z.infer; @@ -792,9 +882,13 @@ export const ConnectorActionEffectSchema = lazySchema(() => z.enum([ export type ConnectorActionEffect = z.input; /** - * Connector Action Definition + * A connector action's declared keys — one declaration, two carriers: the open + * {@link ConnectorActionSchema} and the closed action shape of an AUTHORED + * connector entry ({@link AuthoredConnectorActionSchema}). Built ONCE, so both + * carriers hold the same field instances, for the reason + * {@link retryConfigShape} records. */ -export const ConnectorActionSchema = lazySchema(() => z.object({ +const connectorActionShape = sharedFields(() => ({ key: z.string().describe('Action key (machine name)'), label: z.string().describe('Human readable label'), description: z.string().optional(), @@ -815,6 +909,36 @@ export const ConnectorActionSchema = lazySchema(() => z.object({ */ effect: ConnectorActionEffectSchema.optional(), })); + +/** + * Connector Action Definition — the open shape: the actions a plugin's + * connector def carries and a provider derives (`ConnectorSchema`). An + * AUTHORED entry's `actions` elements are the closed + * {@link AuthoredConnectorActionSchema}. + */ +export const ConnectorActionSchema = lazySchema(() => z.object(connectorActionShape())); + +/** + * The CLOSED action shape of an authored connector entry + * ({@link DeclarativeConnectorEntrySchema}): the keys + * {@link ConnectorActionSchema} declares — the same field instances — refusing + * every other, for the reason {@link AuthoredRetryConfigSchema} records. A + * credential-shaped key meets the entry's own prescription; an execution + * binding — `method`, `path`, `url` — meets ADR-0097 §5's: an action carries + * none. Module private: the published carrier stays {@link ConnectorActionSchema}. + */ +const AuthoredConnectorActionSchema = lazySchema(() => strictObject({ + surface: 'this connector action', + history: 'Until this shape was closed, an undeclared key here was accepted without effect.', + guidance: { + // `key` is the action's own declared machine name here, not a misplaced + // credential, so the entry's credential table is taken without it. + ...Object.fromEntries(Object.entries(ENTRY_CREDENTIAL_KEYS_GUIDANCE).filter(([name]) => name !== 'key')), + method: ENTRY_ACTION_EXECUTION_BINDING_GUIDANCE, + path: ENTRY_ACTION_EXECUTION_BINDING_GUIDANCE, + url: ENTRY_ACTION_EXECUTION_BINDING_GUIDANCE, + }, +}, connectorActionShape())); export type ConnectorAction = z.input; // `ConnectorTriggerSchema` / `ConnectorTrigger` (`key`, `label`, `description`, @@ -835,8 +959,14 @@ export type ConnectorAction = z.input; * pair the OPEN `Connector` def with the closed declaration too, and charge it * prescriptions an open shape never delivers (the `shared/` ledger row's * one-declaration, two-doors finding). + * + * `authored` selects the closed nested shapes the closed entry is built with + * ({@link AuthoredRetryConfigSchema}, {@link AuthoredConnectorActionSchema}), + * so the entry refuses an undeclared key inside `retryConfig` and inside each + * `actions` element as it does at its own top level; the open carrier keeps + * the open nested shapes. */ -const connectorBaseShape = () => ({ +const connectorBaseShape = (authored = false) => ({ /** * Machine name (snake_case) */ @@ -921,7 +1051,7 @@ const connectorBaseShape = () => ({ ), /** Zapier-style Capabilities */ - actions: z.array(ConnectorActionSchema).optional(), + actions: z.array(authored ? AuthoredConnectorActionSchema : ConnectorActionSchema).optional(), /** * `triggers` — RETIRED (ADR-0049 enforce-or-remove). A connector trigger never @@ -987,7 +1117,7 @@ const connectorBaseShape = () => ({ /** * Retry configuration */ - retryConfig: RetryConfigSchema.optional().describe('Retry configuration'), + retryConfig: (authored ? AuthoredRetryConfigSchema : RetryConfigSchema).optional().describe('Retry configuration'), /** * `connectionTimeoutMs` — RETIRED (ADR-0049 enforce-or-remove). A bounded, @@ -1101,17 +1231,6 @@ const connectorBaseShape = () => ({ */ const ConnectorBaseSchema = lazySchema(() => z.object(connectorBaseShape())); -/** - * What a credential-shaped key meets on an authored connector entry. A - * connector entry holds no secret at any key: a provider-bound instance names - * its credential with `auth.credentialRef` and the value lives in the secrets - * layer (ADR-0097 §3, ADR-0143 D3). - */ -const ENTRY_CREDENTIAL_KEY_GUIDANCE = - 'Not authorable on a connector entry — connector metadata never holds a credential. A provider-bound ' - + 'instance names its secret with `auth: { type, credentialRef }`, and the administrator sets the value ' - + 'under that name (an environment variable in the open tier, or Setup → Connector credentials).'; - /** * What a provider setting written at the top level of an entry meets: it * belongs under `providerConfig`, which the provider validates at @@ -1156,15 +1275,7 @@ const DeclarativeConnectorEntryBaseSchema = lazySchema(() => strictObject({ retry: 'retryConfig', }, guidance: { - apiKey: ENTRY_CREDENTIAL_KEY_GUIDANCE, - key: ENTRY_CREDENTIAL_KEY_GUIDANCE, - token: ENTRY_CREDENTIAL_KEY_GUIDANCE, - accessToken: ENTRY_CREDENTIAL_KEY_GUIDANCE, - secret: ENTRY_CREDENTIAL_KEY_GUIDANCE, - clientSecret: ENTRY_CREDENTIAL_KEY_GUIDANCE, - password: ENTRY_CREDENTIAL_KEY_GUIDANCE, - credentials: ENTRY_CREDENTIAL_KEY_GUIDANCE, - credentialRef: ENTRY_CREDENTIAL_KEY_GUIDANCE, + ...ENTRY_CREDENTIAL_KEYS_GUIDANCE, baseUrl: ENTRY_PROVIDER_CONFIG_KEY_GUIDANCE, url: ENTRY_PROVIDER_CONFIG_KEY_GUIDANCE, spec: ENTRY_PROVIDER_CONFIG_KEY_GUIDANCE, @@ -1172,7 +1283,7 @@ const DeclarativeConnectorEntryBaseSchema = lazySchema(() => strictObject({ headers: ENTRY_PROVIDER_CONFIG_KEY_GUIDANCE, defaultHeaders: ENTRY_PROVIDER_CONFIG_KEY_GUIDANCE, }, -}, connectorBaseShape())); +}, connectorBaseShape(true))); /** * Core connector configuration — the authorable shape behind the ruled @@ -1245,6 +1356,30 @@ export function defineConnector(config: z.input): Connec return DefineConnectorSchema.parse(config); } +/** + * What an authored connector URL holding a credential part meets at every door + * that runs the {@link DeclarativeConnectorEntrySchema} refinement: one issue + * per URL, located at the URL, naming the parts it holds — never the URL or + * its value — and the reference to use instead. What a credential part is — + * the userinfo, a credential query parameter — and the walk that finds every + * URL holding one are defined once, in `./connector-url-credentials.ts`, which + * the connector read redactor runs too. No rule per provider: a URL is + * recognised by its value, wherever under `providerConfig` or `auth` it sits. + */ +function connectorUrlCredentialRefusal(name: string, found: ConnectorUrlCredential): string { + const at = found.path.map(String).join('.'); + const parts = [ + ...(found.userinfo ? ['a userinfo (`user@` or `user:password@` before the host)'] : []), + ...found.params.map((param) => `the \`${param}\` query parameter`), + ]; + return `Connector instance '${name}' keeps a credential in the URL at \`${at}\`: ${parts.join(' and ')}. ` + + 'Connector metadata never holds a credential, in a URL or anywhere else. Remove it from the URL and name the ' + + "secret with `auth: { type, credentialRef }` — `bearer` for a token, `api-key` for a key (`paramName` when the " + + 'API takes it as a query parameter, which the provider then adds to every request), `basic` for a user and ' + + 'password — and set the value under that name: an environment variable in the open tier, or Setup → Connector ' + + 'credentials.'; +} + /** * A declarative `connectors:` **stack entry** (ADR-0097) — {@link ConnectorSchema} * plus the cross-field rules that apply only when a connector is *authored inside @@ -1290,6 +1425,10 @@ export function defineConnector(config: z.input): Connec * - A provider-bound instance must NOT author `actions` — the provider derives * them from the upstream (OpenAPI document / MCP `tools/list`); authoring both * the instance and its actions reintroduces drift (§5 non-goals). + * - A URL under an instance's `providerConfig` or `auth` holds no credential + * part — no userinfo, no credential query parameter — refused at the URL + * (`connectorUrlCredentialRefusal`, judged by the walk the read redactor + * runs too); the secret is named with `auth.credentialRef`. * * `triggers` used to be the second key of that last rule, refused with the * reason that the provider derives triggers too. That reason was untrue — no @@ -1354,6 +1493,15 @@ export const DeclarativeConnectorEntrySchema = lazySchema(() => } return; } + // An authored URL holds no credential part, under `providerConfig` or + // `auth`, at any depth (see `connectorUrlCredentialRefusal`). + for (const found of findConnectorUrlCredentials(entry as Record)) { + ctx.addIssue({ + code: 'custom', + path: found.path, + message: connectorUrlCredentialRefusal(entry.name, found), + }); + } if (entry.actions && entry.actions.length > 0) { ctx.addIssue({ code: 'custom', diff --git a/packages/spec/src/kernel/metadata-authoring-lint.test.ts b/packages/spec/src/kernel/metadata-authoring-lint.test.ts index 38dd7a3f6c7..dbf0db0572f 100644 --- a/packages/spec/src/kernel/metadata-authoring-lint.test.ts +++ b/packages/spec/src/kernel/metadata-authoring-lint.test.ts @@ -261,14 +261,18 @@ describe('the object/field unknown-key warnings survive the generalization', () // // The open-collection fixture was `views` until #10039 established that the // parse had been REFUSING those keys all along, then `connectors` until its - // root closed too. No open ROOT is left, so the open half is now a nested - // strip site under a closed root — `connectors[].retryConfig`, which the - // walk still visits per node — carried alongside two closed collections so - // the "must not contaminate" property still has something to be true about. + // root closed too. No open ROOT is left, so the open half is a nested strip + // site under a closed root, which the walk still visits per node. It was + // `connectors[].retryConfig` until the authored connector entry closed its + // nested shapes as well; it is now `apis[].objectParams`, and the connector + // fixture stays as a third closed collection (its hand-off is asserted + // below), so the "must not contaminate" property still has something to be + // true about. const findings = lintAuthoring({ objects: [{ name: 'a', label: 'A', indexes: [{ fields: ['a'], zzz: 1 }] }], views: [{ name: 'v', object: 'a', zzz: 1 }], connectors: [{ name: 'c', retryConfig: { zzz: 1 } }], + apis: [{ name: 'e', objectParams: { zzz: 1 } }], }); // NOT deduped. This assertion used to run through a `new Set(...)` because // the walk emitted every union-root finding twice (#10039) — a mask over a @@ -276,7 +280,7 @@ describe('the object/field unknown-key warnings survive the generalization', () // is asserted now, so a re-introduced duplicate fails here rather than // being absorbed on the way to the assertion. expect(findings.map((f) => `${f.surface}:${f.path}`).sort()).toEqual([ - 'connector:connectors.c.retryConfig.zzz', + 'api:apis.e.objectParams.zzz', ]); // The hand-off half for the view fixture: not unreported — rejected, by the // strict container that made the old warning a contradiction. @@ -289,6 +293,14 @@ describe('the object/field unknown-key warnings survive the generalization', () name: 'a', label: 'A', fields: { x: { type: 'text', label: 'X' } }, indexes: [{ fields: ['a'], zzz: 1 }], }).success).toBe(false); + // The hand-off half for the connector fixture: refused at `retryConfig`. + const connector = ObjectStackDefinitionSchema.safeParse({ + manifest: { id: 'com.example.t', version: '1.0.0', type: 'app', name: 'T' }, + connectors: [{ name: 'c', label: 'C', type: 'api', retryConfig: { zzz: 1 } }], + }); + expect(connector.success).toBe(false); + expect(connector.success ? [] : connector.error.issues.map((i) => [i.code, i.path.join('.')])) + .toContainEqual(['unrecognized_keys', 'connectors.0.retryConfig']); }); it('hands off to the parse for the roots that closed', () => { @@ -605,15 +617,16 @@ describe('one voice, and a single report per record', () => { // The key is unique to this test: `warnedUnknownAuthoringKeys` is a // MODULE-level set, so a path any other test already warned on would make // this one read zero warnings and pass vacuously. - // A nested strip site (`retryConfig`) since the connector root closed — - // see 'reports across collections in one walk' above. - const connectorStack = { - manifest: { ...MANIFEST, id: 'com.example.connectors' }, - connectors: [{ name: 'c1', retryConfig: { zzz_once_probe_key: 1 } }], + // A nested strip site (`apis[].objectParams`) since the connector root and + // then its nested shapes closed — see 'reports across collections in one + // walk' above. + const apiStack = { + manifest: { ...MANIFEST, id: 'com.example.apis' }, + apis: [{ name: 'e1', objectParams: { zzz_once_probe_key: 1 } }], }; - const findings = lintAuthoring(connectorStack); + const findings = lintAuthoring(apiStack); expect(findings.map((f) => `${f.surface}:${f.path}`)).toEqual([ - 'connector:connectors.c1.retryConfig.zzz_once_probe_key', + 'api:apis.e1.objectParams.zzz_once_probe_key', ]); const warned: string[] = []; @@ -622,7 +635,7 @@ describe('one voice, and a single report per record', () => { try { // `strict: false` skips the parse; the pre-parse warning pass runs either // way, which is the door under test here. - defineStack({ ...viewStack, ...connectorStack } as never, { strict: false }); + defineStack({ ...viewStack, ...apiStack } as never, { strict: false }); } finally { console.warn = realWarn; } diff --git a/packages/spec/src/migrations/entries/semantic/18.connector-entry-nested-undeclared-key-refused.ts b/packages/spec/src/migrations/entries/semantic/18.connector-entry-nested-undeclared-key-refused.ts new file mode 100644 index 00000000000..53caa89c6fe --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.connector-entry-nested-undeclared-key-refused.ts @@ -0,0 +1,31 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +export const entry: SemanticMigration = { + id: 'connector-entry-nested-undeclared-key-refused', + surface: 'connector entries AUTHORED through `defineConnector()`, defineStack `connectors:`, `os validate` / ' + + '`os compile`, the artifact load and `PUT /meta/connector/:name` — a key `retryConfig` or an `actions` element ' + + 'does not declare', + replacement: 'keep only the keys `RetryConfigSchema` declares inside `retryConfig` (`strategy`, `maxAttempts`, ' + + '`initialDelayMs`, `maxDelayMs`, `backoffMultiplier`, `retryableStatusCodes`, `retryOnNetworkError`, `jitter`) ' + + 'and only the keys `ConnectorActionSchema` declares inside each `actions` element (`key`, `label`, ' + + '`description`, `inputSchema`, `outputSchema`, `effect`). A credential goes into the secrets layer under the ' + + 'name `auth.credentialRef` gives and the key is deleted; an execution binding (`method`, `path`, `url`) has no ' + + 'home on an action (ADR-0097 §5) — a provider-bound instance derives its actions from the upstream.', + reason: + 'The entry closed its top level; its two nested shapes were still plain objects, so an undeclared key inside ' + + '`retryConfig` or an `actions` element parsed green at every door and meant nothing to the runtime. Each is ' + + 'refused now, by name, in one `unrecognized_keys` issue located at the container (`retryConfig`, `actions.N`) ' + + 'that lists every undeclared key. There is deliberately no mechanical rewrite, for the top-level entry\'s ' + + 'reason: an undeclared key may be a misplaced setting or the only copy of a secret. The open shapes ' + + '(`ConnectorSchema`, `RetryConfigSchema`, `ConnectorActionSchema` — the def a plugin hands to ' + + '`registerConnector`, and the policy the runtime parses) are unchanged. A row stored before this refusal keeps ' + + 'working: the read withholds the undeclared key. An explicit `retryConfig` in a save replaces the stored one ' + + 'whole, so its undeclared key is not kept and the save names it on its answer; an `actions` element\'s ' + + 'undeclared key is kept at rest by a save of the served form.', + acceptanceCriteria: + 'Every authored connector entry parses through `DeclarativeConnectorEntrySchema` with no `unrecognized_keys` ' + + 'issue at `retryConfig` or at an `actions` element; each secret that sat at such a key is set under its ' + + '`credentialRef` name.', +}; diff --git a/packages/spec/src/migrations/entries/semantic/18.connector-entry-url-credential-refused.ts b/packages/spec/src/migrations/entries/semantic/18.connector-entry-url-credential-refused.ts new file mode 100644 index 00000000000..c81fb3b565f --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.connector-entry-url-credential-refused.ts @@ -0,0 +1,31 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +export const entry: SemanticMigration = { + id: 'connector-entry-url-credential-refused', + surface: 'connector entries AUTHORED through defineStack `connectors:`, `os validate` / `os compile`, the artifact ' + + 'load and `PUT /meta/connector/:name` — a URL under `providerConfig` (`baseUrl`, `transport.url`, a URL `spec`, ' + + 'any other URL a provider keeps there) or under `auth` (`tokenUrl`, `authorizationUrl`, `resource`) that holds a ' + + 'userinfo or a credential query parameter', + replacement: 'keep every authored connector URL credential-free and name the secret with `auth: { type, ' + + 'credentialRef }`: `bearer` for a token, `api-key` for a key (with `paramName` when the API takes it as a query ' + + 'parameter, which the provider then adds to every request), `basic` for a user and password. Set the value under ' + + 'the name `credentialRef` gives — an environment variable in the open tier, or Setup → Connector credentials — ' + + 'and delete it from the URL.', + reason: + 'Connector metadata holds no credential at any position, and the read already withheld the credential parts of ' + + 'a URL under `providerConfig`. The write door now holds the same line: a URL holding a userinfo (`user@` or ' + + '`user:password@`) or a credential query parameter (the datasource names, the conventional HTTP key and token ' + + 'names, and the instance\'s own `auth.paramName`) is refused with `422 INVALID_METADATA`, one issue per URL ' + + 'located at the URL, naming the parts it holds. The judgement and the walk are the read redactor\'s own, so the ' + + 'door refuses exactly what the read withholds, `auth` endpoints and `providerConfig` alike. A URL inside a header ' + + 'or environment map is judged by neither (the read withholds those maps whole). There is deliberately no ' + + 'mechanical rewrite: the value in the URL may be the only copy of the secret, and where it goes next is the ' + + 'administrator\'s act. A row stored before this refusal keeps working: a save of the served form keeps the ' + + 'stored URL, and an edit of the served URL stores the edit.', + acceptanceCriteria: + 'Every authored connector entry parses through `DeclarativeConnectorEntrySchema` with no issue located at a ' + + 'URL under `providerConfig` or `auth`; each secret that sat in a URL is set under its `credentialRef` name and ' + + 'the instance still materializes (`GET /api/v1/automation/connectors` reports it `ready`).', +};