feat(service-datasource): a datasource naming a connector instance in external.connector borrows its credential, and its driver presents it per request (ADR-0015 D7)

.changeset/27746-datasource-borrows-connector-credential.md

@@ -0,0 +1,28 @@
+---
+'@objectstack/types': patch
+'@objectstack/service-automation': patch
+'@objectstack/service-datasource': patch
+'@objectstack/driver-odata': patch
+---
+
+feat: a datasource that names a connector instance in `external.connector` borrows that instance's credential at runtime, and its driver presents it per request
+
+Clause-②: yes
+Metadata-protocol: none
+
+The runtime half of ADR-0015 amendment D7. A datasource over a vendor API that names a declared connector instance now reads with that instance's credential — the one set under Setup → Connector credentials (or the instance's environment variable, or the host's resolver) — so one credential serves the connector's flow actions and the datasource's virtual objects:
+
+```ts
+connectors: [{ name: 'sap_s4', type: 'api', provider: 'openapi', providerConfig: { spec: './sap-openapi.json' },
+  auth: { type: 'bearer', credentialRef: 'sap_token' } }],
+datasources: [{ name: 'sap_orders', driver: 'sap_orders', config: {},
+  schemaMode: 'external', external: { connector: 'sap_s4' } }],
+// served by createODataDriverPlugin({ name: 'sap_orders', serviceRoot: '…' }), its auth left at 'none'
+```
+
+- **`@objectstack/service-automation`** registers a `connector-credential-source` service that lends a declared instance's credential (bearer, API key, basic), resolved through the same chain the connector itself uses. A credential set or changed through the credential door is picked up by the datasource's next request, with no restart.
+- **`@objectstack/service-datasource`** borrows it when such a datasource connects (a plugin driver the engine already adopted from its `driver.<name>` service is judged at that connect too), is probed (Test connection) or is saved from the admin door, and hands the driver factory a credential handle on `DatasourceConnectionSpec.credential`. A datasource whose instance is not declared, is a catalog descriptor, has no `auth` credential or is disabled — or a stack with no automation service to lend it — is refused, fail-closed, with a reason naming the fix: it never connects to fail its first read with an upstream 401.
+- **`@objectstack/driver-odata`** presents it: a datasource it serves that names `external.connector` borrows the handle at its first request and sends every request through `callWithBorrowedCredential`, in the form the instance's `auth` declares (bearer, basic, or an API key as a header or query parameter). An upstream's error text is stripped of every spelling of the presented credential before it is thrown (`withoutBorrowedCredential`), so an API key sent as a query parameter never reaches an error body through a service that echoes the request URI. `external.credentialsRef` keeps its path; both channels at once, or an `auth.scheme` set on the plugin beside the connector, refuse the request.
+- **`@objectstack/types`** carries the contract (`IConnectorCredentialSource`, `ConnectorCredentialHandle`, `CONNECTOR_CREDENTIAL_SOURCE_SERVICE`) and `callWithBorrowedCredential`, the one rule a plugin driver presents a borrowed credential by: refreshed when it has expired, refreshed once and retried on a `401`, and a `503 SERVICE_UNAVAILABLE` naming the instance when the fresh one is refused too.
+
+Nothing changes for a datasource that binds its own secret with `external.credentialsRef`. The handle answers the grant-bearing kinds (OAuth2 and the app-credential exchange) through the same interface when connector instances can declare them; a driver written against it today needs no change then.

packages/drivers/driver-odata/README.md

@@ -65,10 +65,38 @@ export default defineStack({
 | `name` | `odata` | The datasource name, the driver name and its `driver.<name>` service suffix. |
 | `serviceRoot` | — | The service root URL (http or https, no credentials, no query). |
 | `version` | `4.0` | The protocol version. Only the v4 dialect ships. |
-| `auth` | `{ scheme: 'none' }` | `none`, `bearer`, or `basic` with a `username`. The secret is the datasource's `external.credentialsRef`. |
+| `auth` | `{ scheme: 'none' }` | `none`, `bearer`, or `basic` with a `username`. The secret is the datasource's `external.credentialsRef`. Leave it `none` when the datasource borrows a connector instance's credential (below). |
 | `pageSize` | `500` | Rows asked for per server page (`Prefer: odata.maxpagesize`). |
 | `requestTimeoutMs` | `30000` | Per-request timeout. |
 
+## Borrowing a connector instance's credential
+
+When a connector instance already authorizes the vendor, the datasource can name it in
+`external.connector` instead of binding a secret of its own (ADR-0015 amendment D7): one
+credential, set under Setup → Connector credentials, serves the connector's flow actions and
+the datasource's records.
+
+```typescript
+connectors: [{
+  name: 'erp_api', type: 'api', provider: 'openapi', providerConfig: { spec: './erp-openapi.json' },
+  auth: { type: 'bearer', credentialRef: 'erp_token' },
+}],
+datasources: [{
+  name: 'erp', driver: 'erp', schemaMode: 'external', config: {},
+  external: { connector: 'erp_api' },                // no credentialsRef: exactly one channel
+}],
+```
+
+The plugin is created with `auth` left at `none`: the instance's `auth` decides how the credential
+is sent (a bearer token, basic auth, or an API key as a header or a query parameter). It borrows
+from the `connector-credential-source` service, which `@objectstack/service-automation`
+registers, at the first request, and presents it on every request through
+`callWithBorrowedCredential`: refreshed when it has expired, refreshed once and retried on a
+`401`, and refused with a `503` naming the instance when the fresh one is refused too. A stack with
+no automation service, an instance that cannot lend (not declared, disabled, no `auth`), an `auth`
+scheme set on the plugin beside the connector, and a `credentialsRef` beside it all refuse the
+request instead of sending it.
+
 ## What is pushed down
 
 `$filter`: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$nin`, `$null`,

packages/drivers/driver-odata/package.json

@@ -37,6 +37,7 @@
   "dependencies": {
     "@objectstack/core": "workspace:*",
     "@objectstack/spec": "workspace:*",
+    "@objectstack/types": "workspace:*",
     "fast-xml-parser": "^5.10.1",
     "zod": "^4.6.6"
   },

packages/drivers/driver-odata/src/odata-driver.ts

@@ -54,6 +54,7 @@ import type { DriverOptions } from '@objectstack/spec/data';
 import type { EdmEntitySet, EdmEntityType, EdmProperty, ODataDialect, ODataServiceModel } from './dialect.js';
 import { odataV4Dialect } from './dialect-v4.js';
 import { translateWhere, type FilterTarget } from './filter.js';
+import type { ConnectorCredentialHandle } from '@objectstack/types';
 import { ODataRequestError, ODataTransport, type FetchLike, type ODataResponse } from './transport.js';
 import type { ODataDriverConfig } from './config.js';
 
@@ -71,6 +72,8 @@ export interface ODataDriverRuntime {
   fetch?: FetchLike;
   /** The secret behind the datasource's `external.credentialsRef`; see `plugin.ts`. */
   credential?: () => Promise<string | undefined>;
+  /** The credential borrowed from the connector instance the datasource names in `external.connector`; see `plugin.ts`. */
+  borrowed?: () => Promise<ConnectorCredentialHandle | undefined>;
 }
 
 /** An introspected table, with the navigation properties the import offers as lookups. */
@@ -141,6 +144,7 @@ export class ODataDriver implements IDataDriver {
       pageSize: config.pageSize,
       ...(runtime.fetch ? { fetch: runtime.fetch } : {}),
       ...(runtime.credential ? { credential: runtime.credential } : {}),
+      ...(runtime.borrowed ? { borrowed: runtime.borrowed } : {}),
     });
   }
 

packages/drivers/driver-odata/src/plugin.test.ts

@@ -3,21 +3,35 @@
 /**
  * The plugin: what it publishes, and where the credential comes from — the
  * datasource's `external.credentialsRef`, dereferenced through the host's
- * secret store at the first request, failing closed every way.
+ * secret store at the first request, or the connector instance its
+ * `external.connector` borrows from (ADR-0015 D7) — failing closed every way.
  */
 
 import { describe, expect, it } from 'vitest';
 import type { PluginContext } from '@objectstack/core';
 import { needsExternalDriverFloor } from '@objectstack/objectql';
+import {
+  CONNECTOR_CREDENTIAL_SOURCE_SERVICE,
+  connectorCredentialRefusal,
+  type BorrowedConnectorCredential,
+  type IConnectorCredentialSource,
+} from '@objectstack/types';
 import { createODataDriverPlugin, type DatasourceDefinitionSource, type ODataDriverPluginOptions } from './plugin.js';
-import { METADATA_EXCHANGE, SERVICE_ROOT, page, recordedService } from './__fixtures__/recorded-service.js';
+import { METADATA_EXCHANGE, SERVICE_ROOT, page, recordedService, type Exchange } from './__fixtures__/recorded-service.js';
 
 type Def = ReturnType<DatasourceDefinitionSource['listDatasourceDefs']>[number];
 
-async function boot(options: Omit<ODataDriverPluginOptions, 'serviceRoot' | 'fetch'>, defs: Def[]) {
-  const service = recordedService([METADATA_EXCHANGE, { url: 'People', body: page([]) }]);
+async function boot(
+  options: Omit<ODataDriverPluginOptions, 'serviceRoot' | 'fetch'>,
+  defs: Def[],
+  extra: { exchanges?: Exchange[]; services?: Record<string, unknown> } = {},
+) {
+  const service = recordedService(extra.exchanges ?? [METADATA_EXCHANGE, { url: 'People', body: page([]) }]);
   const plugin = createODataDriverPlugin({ serviceRoot: SERVICE_ROOT, fetch: service.fetch, ...options });
-  const services = new Map<string, unknown>([['objectql', { listDatasourceDefs: () => defs } satisfies DatasourceDefinitionSource]]);
+  const services = new Map<string, unknown>([
+    ['objectql', { listDatasourceDefs: () => defs } satisfies DatasourceDefinitionSource],
+    ...Object.entries(extra.services ?? {}),
+  ]);
   const ctx = {
     registerService: (name: string, value: unknown) => { services.set(name, value); },
     getService: (name: string) => services.get(name),
@@ -80,3 +94,139 @@ describe('createODataDriverPlugin', () => {
     expect(requests).toHaveLength(0);
   });
 });
+
+/**
+ * A lender double standing in for `@objectstack/service-automation`'s source:
+ * `get` answers the current credential, `refresh` moves to the next one, the
+ * shape a grant-bearing (OAuth2) instance answers through.
+ */
+function lender(answers: BorrowedConnectorCredential[], refuse?: Error) {
+  const calls = { borrow: [] as string[], get: 0, refresh: 0 };
+  let current = 0;
+  const source: IConnectorCredentialSource = {
+    async borrow(connector) {
+      calls.borrow.push(connector);
+      if (refuse) throw refuse;
+      return {
+        connector,
+        async get() { calls.get += 1; return answers[current]!; },
+        async refresh() { calls.refresh += 1; current = Math.min(current + 1, answers.length - 1); return answers[current]!; },
+      };
+    },
+  };
+  return { source, calls, services: { [CONNECTOR_CREDENTIAL_SOURCE_SERVICE]: source } };
+}
+
+const BORROWING: Def[] = [{ name: 'trippin', external: { connector: 'trippin_api' } }];
+const bearer = (token: string, expiresAt?: number): BorrowedConnectorCredential =>
+  ({ auth: { type: 'bearer', token }, ...(expiresAt !== undefined ? { expiresAt } : {}) });
+
+describe('createODataDriverPlugin — a credential borrowed from external.connector (ADR-0015 D7)', () => {
+  it('sends the connector instance\'s credential, borrowed once, and never asks the secret store', async () => {
+    const { calls, services } = lender([bearer('tok-1')]);
+    const secrets = { resolve: async () => { throw new Error('the secret store was asked'); } };
+    const { plugin, requests } = await boot({ name: 'trippin', secrets }, BORROWING, { services });
+    await plugin.driver.find('trip_person', {});
+    expect(requests.map((r) => r.headers.Authorization)).toEqual(['Bearer tok-1', 'Bearer tok-1']);
+    expect(calls.borrow).toEqual(['trippin_api']);
+    expect(calls.refresh).toBe(0);
+  });
+
+  it('presents the instance\'s auth as it declares it: basic, an API key header, an API key query parameter', async () => {
+    const basic = lender([{ auth: { type: 'basic', username: 'svc', password: 'pw' } }]);
+    const one = await boot({ name: 'trippin' }, BORROWING, { services: basic.services });
+    await one.plugin.driver.find('trip_person', {});
+    expect(one.requests[0]!.headers.Authorization).toBe(`Basic ${Buffer.from('svc:pw').toString('base64')}`);
+
+    const header = lender([{ auth: { type: 'api-key', key: 'k-1', headerName: 'APIKey' } }]);
+    const two = await boot({ name: 'trippin' }, BORROWING, { services: header.services });
+    await two.plugin.driver.find('trip_person', {});
+    expect(two.requests[0]!.headers.APIKey).toBe('k-1');
+    expect(two.requests[0]!.headers.Authorization).toBeUndefined();
+
+    const query = lender([{ auth: { type: 'api-key', key: 'k-2', headerName: 'X-API-Key', paramName: 'apikey' } }]);
+    const three = await boot({ name: 'trippin' }, BORROWING, {
+      services: query.services,
+      exchanges: [{ ...METADATA_EXCHANGE, url: '$metadata?apikey=k-2' }, { url: 'People?apikey=k-2', body: page([]) }],
+    });
+    await three.plugin.driver.find('trip_person', {});
+    expect(three.requests.map((r) => r.url)).toEqual(['$metadata?apikey=k-2', 'People?apikey=k-2']);
+    expect(three.requests[0]!.headers['X-API-Key']).toBeUndefined();
+  });
+
+  it('refreshes once on a 401 and retries with the credential obtained afresh', async () => {
+    const { calls, services } = lender([bearer('tok-1'), bearer('tok-2')]);
+    const { plugin, requests } = await boot({ name: 'trippin' }, BORROWING, {
+      services,
+      exchanges: [{ url: '$metadata', status: 401, body: { error: { message: 'expired' } }, once: true }, METADATA_EXCHANGE, { url: 'People', body: page([]) }],
+    });
+    await plugin.driver.find('trip_person', {});
+    expect(requests.map((r) => r.headers.Authorization)).toEqual(['Bearer tok-1', 'Bearer tok-2', 'Bearer tok-2']);
+    expect(calls.refresh).toBe(1);
+  });
+
+  it('refreshes a credential at its expiry before sending it', async () => {
+    const { calls, services } = lender([bearer('tok-stale', Date.now() - 1), bearer('tok-fresh')]);
+    const { plugin, requests } = await boot({ name: 'trippin' }, BORROWING, { services });
+    await plugin.driver.find('trip_person', {});
+    expect(requests[0]!.headers.Authorization).toBe('Bearer tok-fresh');
+    expect(calls.refresh).toBe(1);
+  });
+
+  it('ends a second 401 as a 503 naming the instance, never the upstream\'s text', async () => {
+    const { services } = lender([bearer('tok-1'), bearer('tok-2')]);
+    const { plugin, requests } = await boot({ name: 'trippin' }, BORROWING, {
+      services,
+      exchanges: [{ url: '$metadata', status: 401, body: { error: { message: 'token tok-2 rejected' } } }],
+    });
+    const failure = (await plugin.driver.find('trip_person', {}).then(() => undefined, (err: unknown) => err)) as Error & { status?: number; code?: string };
+    expect(failure).toMatchObject({ status: 503, code: 'SERVICE_UNAVAILABLE' });
+    expect(failure.message).toContain("connector instance 'trippin_api'");
+    expect(failure.message).not.toContain('tok-2');
+    expect(requests).toHaveLength(2);
+  });
+
+  it('withholds a query-parameter API key from an upstream refusal that echoes the request URI', async () => {
+    const key = 'k-2/x+y==';
+    const { services } = lender([{ auth: { type: 'api-key', key, headerName: 'X-API-Key', paramName: 'apikey' } }]);
+    const echoed = new URL(`People?apikey=${encodeURIComponent(key)}`, SERVICE_ROOT).toString();
+    const { plugin } = await boot({ name: 'trippin' }, BORROWING, {
+      services,
+      exchanges: [
+        { ...METADATA_EXCHANGE, url: `$metadata?apikey=${key}` },
+        { url: `People?apikey=${key}`, status: 404, body: { error: { code: 'NotFound', message: `No resource at '${echoed}' (apikey=${key}, form ${new URLSearchParams({ apikey: key }).toString()})` } } },
+      ],
+    });
+    const failure = (await plugin.driver.find('trip_person', {}).then(() => undefined, (err: unknown) => err)) as Error & { status?: number };
+    expect(failure).toBeInstanceOf(Error);
+    expect(failure.status).toBe(404);
+    expect(failure.message).toContain('[credential withheld]');
+    for (const spelling of [key, encodeURIComponent(key), new URLSearchParams({ v: key }).toString().slice(2)]) {
+      expect(failure.message).not.toContain(spelling);
+    }
+  });
+
+  const refusals: Array<[string, Omit<ODataDriverPluginOptions, 'serviceRoot' | 'fetch'>, Def[], Record<string, unknown>, RegExp]> = [
+    ['no connector credential source registered', { name: 'trippin' }, BORROWING, {}, /no connector credential source is registered/],
+    ['both credential channels', { name: 'trippin', secrets: SECRETS }, [{ name: 'trippin', external: { connector: 'trippin_api', credentialsRef: 'sys_secret:sec_trippin' } }], lender([bearer('tok-1')]).services, /names both external\.connector and external\.credentialsRef/],
+    ['an auth scheme beside the connector', { name: 'trippin', auth: { scheme: 'bearer' } }, BORROWING, lender([bearer('tok-1')]).services, /set auth\.scheme to 'none'/],
+    ['an instance the source refuses to lend', { name: 'trippin' }, BORROWING, lender([], connectorCredentialRefusal("connector instance 'trippin_api' is disabled", 'VALIDATION_ERROR', 422)).services, /'trippin_api' is disabled/],
+    ['a credential the handle cannot answer', { name: 'trippin' }, BORROWING, { [CONNECTOR_CREDENTIAL_SOURCE_SERVICE]: { borrow: async (connector: string) => ({ connector, get: async () => { throw connectorCredentialRefusal("connector instance 'trippin_api': credential not set", 'SERVICE_UNAVAILABLE', 503); }, refresh: async () => { throw new Error('unreached'); } }) } }, /credential not set/],
+  ];
+
+  it.each(refusals)('fails closed, sending nothing, on %s', async (_label, options, defs, services, message) => {
+    const { plugin, requests } = await boot(options, defs, { services });
+    await expect(plugin.driver.find('trip_person', {})).rejects.toThrow(message);
+    expect(requests).toHaveLength(0);
+  });
+
+  it('answers a lend refusal without its 422, so the engine reads it as the connection\'s condition, and asks again next request', async () => {
+    const refused = lender([], connectorCredentialRefusal("connector instance 'trippin_api' is disabled", 'VALIDATION_ERROR', 422));
+    const { plugin } = await boot({ name: 'trippin' }, BORROWING, { services: refused.services });
+    const failure = (await plugin.driver.find('trip_person', {}).then(() => undefined, (err: unknown) => err)) as Error & { status?: number };
+    expect(failure).toBeInstanceOf(Error);
+    expect(failure.status).toBeUndefined();
+    await plugin.driver.find('trip_person', {}).catch(() => {});
+    expect(refused.calls.borrow).toEqual(['trippin_api', 'trippin_api']);
+  });
+});