diff --git a/.changeset/27028-data-engine-rpc-protocol-retired.md b/.changeset/27028-data-engine-rpc-protocol-retired.md new file mode 100644 index 00000000000..1a565ea4a8e --- /dev/null +++ b/.changeset/27028-data-engine-rpc-protocol-retired.md @@ -0,0 +1,28 @@ +--- +'@objectstack/spec': minor +'@objectstack/objectql': patch +--- + +feat(spec)!: retire the unserved Virtual ObjectQL request protocol from `data/data-engine.zod.ts` + +Clause-②: no (narrowing) +Metadata-protocol: narrowed + + + +**BREAKING** for TypeScript and JSON Schema consumers of one block that nothing ever served. `@objectstack/spec/data` published a serialized request protocol for a "Virtual Data Engine" in another process: `DataEngineRequestSchema` and its nine `method` arms, plus `DataEngineContractSchema`. No such engine existed and nothing parsed the protocol. A census found no importer or parser in this repo outside the defining module, and zero hits in hotcrm, objectui and docs.objectos.ai. As published it was also unsafe to implement: every arm admitted a caller-supplied `context` (`isSystem` included), and the `execute` arm carried a raw driver command. The whole block is retired (ADR-0049 enforce-or-remove, by the maintainer's ruling A). + +| FROM | TO | +|:--|:--| +| `DataEngineFindRequestSchema`, `DataEngineFindOneRequestSchema`, `DataEngineCountRequestSchema`, `DataEngineAggregateRequestSchema` | in process, `IDataEngine.find` / `findOne` / `count` / `aggregate` with `EngineQueryOptions` / `EngineCountOptions` / `EngineAggregateOptions`; over the wire, the REST data routes (`/data/:object`, `POST /data/:object/query`) | +| `DataEngineInsertRequestSchema`, `DataEngineUpdateRequestSchema`, `DataEngineDeleteRequestSchema` | `IDataEngine.insert` / `update` / `delete` with `DataEngineInsertOptions` / `EngineUpdateOptions` / `EngineDeleteOptions`; over the wire, the REST data routes, and `POST {basePath}/batch` for a batch | +| `DataEngineVectorFindRequestSchema`, `DataEngineExecuteRequestSchema` | `IDataEngine.vectorFind?` / `execute?`, in process only | +| `DataEngineRequestSchema` and the types `DataEngineRequest`, `DataEngineRequestParsed`, `DataEngineExecuteRequest`, `DataEngineInsertRequest`, `DataEngineVectorFindRequest` | none: there is no serialized engine-call protocol; call `IDataEngine` in process, or a hook or action body's `ctx.api.object(name)` | +| `DataEngineContractSchema` | the `IDataEngine` interface (`@objectstack/spec/contracts`) | + +The one-line fix: replace an import of any name above with the `IDataEngine` method or option type in the right-hand column. An import left behind is `TS2305` after upgrade. + +The retirement kit: + +- **`@objectstack/spec`**: the ten request schemas, `DataEngineContractSchema`, the five types and their private helpers (`RpcQueryOptionsSchema` and the `filter`-only fold) are deleted. The ten published defs `data/DataEngineRequest` and `data/DataEngine{Find,FindOne,Insert,Update,Delete,Count,Aggregate,Execute,VectorFind}Request` leave `json-schema/` and the reference docs, registered in `RETIRED_DEFS_BY_MAJOR[18]`, and the D3 entry `data-engine-rpc-protocol-retired` names the doors that exist. There is no `retiredKey()` tombstone and no conversion: no metadata source authors these shapes, and no parse remains for a prescription to reach. `IDataEngine`, every `*OptionsSchema`, `RPC_QUERY_ALIAS_SLOTS`, `foldQueryAliasSlots` and the query-transport schemas are unchanged. Runtime behaviour is unchanged. +- **`@objectstack/objectql`**: a comment that named `RpcQueryOptionsSchema` now names `QueryWithTransportSchema`. No behaviour changes. diff --git a/content/docs/kernel/contracts/data-engine.mdx b/content/docs/kernel/contracts/data-engine.mdx index b2db312eaca..d46b76d2878 100644 --- a/content/docs/kernel/contracts/data-engine.mdx +++ b/content/docs/kernel/contracts/data-engine.mdx @@ -27,7 +27,6 @@ import type { EngineDeleteOptions, EngineAggregateOptions, EngineCountOptions, - DataEngineRequest, } from '@objectstack/spec/data'; export interface IDataEngine { @@ -48,9 +47,6 @@ export interface IDataEngine { where?: any; limit?: number; fields?: string[]; threshold?: number; }): Promise; - // Batch Operations (optional, transactional) - batch?(requests: DataEngineRequest[], options?: { transaction?: boolean }): Promise; - // Raw Command Escape Hatch (optional) execute?(command: any, options?: Record): Promise; @@ -470,8 +466,8 @@ declares one; none carries a `readonly` policy of its own any more — see `WriteObservabilityOptions` is a **TS-contract-level, in-process-only** bag — all three members. It is deliberately not part of the serializable Zod options schemas: -a function is unrepresentable in JSON Schema and cannot cross the RPC (Virtual -Data Engine) boundary, so remote callers never receive these events; and putting +a function is unrepresentable in JSON Schema and cannot be serialized, so remote +callers (REST, the client SDK) never receive these events; and putting `strictReadonlyWrites` in the serializable bag would let any client toggle write-refusal on a security-adjacent path (#5126 ruling). A remote caller can set none of the three, and for the strips it gets NEITHER behaviour: its write is @@ -569,17 +565,12 @@ const results = await engine.vectorFind?.('document', embeddingVector, { }); ``` -### batch (Transactional) - -Execute multiple operations in a single transaction: +### Several writes, atomically -```typescript -const results = await engine.batch?.([ - { method: 'insert', object: 'project', data: { name: 'Website Redesign', status: 'active' } }, - { method: 'insert', object: 'task', data: { title: 'Design mockups', status: 'open' } }, - { method: 'insert', object: 'task', data: { title: 'Implement frontend', status: 'open' } }, -], { transaction: true }); -``` +There is no `batch` member: it was never implemented by any engine and was retired +(ADR-0119 D3). In process, wrap the writes in `IObjectQLEngine.transaction(cb)`; +for a batch over one object use the metadata protocol's `batchData` with +`options.atomic: true`; over the wire, `POST {basePath}/batch`. ### execute (Raw Command) @@ -624,7 +615,7 @@ detection locate drivers. **Deprecated Parameter Names** -The following legacy parameter names are accepted by the RPC layer for backward compatibility but should **not** be used in new code. The protocol normalizer resolves conflicts with canonical names taking precedence. +The following legacy parameter names are accepted on the wire (the protocol normalizer) for backward compatibility but should **not** be used in new code. The protocol normalizer resolves conflicts with canonical names taking precedence. | Legacy (Deprecated) | Canonical | Notes | |:---------------------|:----------|:------| diff --git a/content/docs/references/data/data-engine.mdx b/content/docs/references/data/data-engine.mdx index 88923ae3886..29b40bcf46a 100644 --- a/content/docs/references/data/data-engine.mdx +++ b/content/docs/references/data/data-engine.mdx @@ -22,8 +22,8 @@ The Data Engine acts as the "Driver" layer in the Hexagonal Architecture. ## TypeScript Usage ```typescript -import { BaseEngineOptionsSchema, DataEngineAggregateOptionsSchema, DataEngineAggregateRequestSchema, DataEngineCountOptionsSchema, DataEngineCountRequestSchema, DataEngineDeleteOptionsSchema, DataEngineDeleteRequestSchema, DataEngineExecuteRequestSchema, DataEngineFilterSchema, DataEngineFindOneRequestSchema, DataEngineFindRequestSchema, DataEngineInsertOptionsSchema, DataEngineInsertRequestSchema, DataEngineQueryOptionsSchema, DataEngineRequestSchema, DataEngineSortSchema, DataEngineUpdateOptionsSchema, DataEngineUpdateRequestSchema, DataEngineVectorFindRequestSchema, DroppedFieldsEventSchema, EngineAggregateOptionsSchema, EngineCountOptionsSchema, EngineDeleteOptionsSchema, EngineQueryOptionsSchema, EngineUpdateOptionsSchema, QueryPageWindowSchema, QueryPageWindowParamSchema, QueryTransportParamsSchema, QueryWithTransportSchema, ValidationAdvisoryEventSchema } from '@objectstack/spec/data'; -import type { BaseEngineOptions, DataEngineAggregateOptions, DataEngineCountOptions, DataEngineDeleteOptions, DataEngineExecuteRequest, DataEngineFilter, DataEngineInsertOptions, DataEngineInsertRequest, DataEngineQueryOptions, DataEngineRequest, DataEngineSort, DataEngineUpdateOptions, DataEngineVectorFindRequest, DroppedFieldsEvent, EngineAggregateOptions, EngineCountOptions, EngineDeleteOptions, EngineQueryOptions, EngineUpdateOptions, QueryPageWindow, QueryPageWindowParam, QueryTransportParams, QueryWithTransport, ValidationAdvisoryEvent } from '@objectstack/spec/data'; +import { BaseEngineOptionsSchema, DataEngineAggregateOptionsSchema, DataEngineCountOptionsSchema, DataEngineDeleteOptionsSchema, DataEngineFilterSchema, DataEngineInsertOptionsSchema, DataEngineQueryOptionsSchema, DataEngineSortSchema, DataEngineUpdateOptionsSchema, DroppedFieldsEventSchema, EngineAggregateOptionsSchema, EngineCountOptionsSchema, EngineDeleteOptionsSchema, EngineQueryOptionsSchema, EngineUpdateOptionsSchema, QueryPageWindowSchema, QueryPageWindowParamSchema, QueryTransportParamsSchema, QueryWithTransportSchema, ValidationAdvisoryEventSchema } from '@objectstack/spec/data'; +import type { BaseEngineOptions, DataEngineAggregateOptions, DataEngineCountOptions, DataEngineDeleteOptions, DataEngineFilter, DataEngineInsertOptions, DataEngineQueryOptions, DataEngineSort, DataEngineUpdateOptions, DroppedFieldsEvent, EngineAggregateOptions, EngineCountOptions, EngineDeleteOptions, EngineQueryOptions, EngineUpdateOptions, QueryPageWindow, QueryPageWindowParam, QueryTransportParams, QueryWithTransport, ValidationAdvisoryEvent } from '@objectstack/spec/data'; // Validate data const result = BaseEngineOptionsSchema.parse(data); @@ -132,33 +132,6 @@ Options for DataEngine.aggregate operations | **traceId** | `string` | optional | | ---- - -## DataEngineAggregateRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'aggregate'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; groupBy?: (string \| object)[]; aggregations?: object[]; … }` | ✅ | | - -### Nested Shape: `DataEngineAggregateRequest.query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **groupBy** | `(string \| { field: string; dateGranularity?: Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>; alias?: string })[]` | optional | GROUP BY targets (strings or `{field, dateGranularity?}` objects for date bucketing) | -| **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional | | -| **having** | `any` | optional | HAVING — filter over the aggregated rows (aggregation aliases + groupBy projections); applied engine-side after aggregation | -| **timezone** | `string` | optional | | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - - --- ## DataEngineCountOptions @@ -211,29 +184,6 @@ Options for DataEngine.count operations | **traceId** | `string` | optional | | ---- - -## DataEngineCountRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'count'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; search?: string \| object; searchFields?: string[]; … }` | optional | | - -### Nested Shape: `DataEngineCountRequest.query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - - --- ## DataEngineDeleteOptions @@ -287,42 +237,6 @@ Options for DataEngine.delete operations | **traceId** | `string` | optional | | ---- - -## DataEngineDeleteRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'delete'` | ✅ | | -| **object** | `string` | ✅ | | -| **id** | `string \| number` | optional | ID for single delete, or use where in options | -| **options** | `{ context?: object; where?: Record \| any; multi?: boolean; filter?: Record \| any }` | optional | | - -### Nested Shape: `DataEngineDeleteRequest.options` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **multi** | `boolean` | optional (default: `false`) | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - - ---- - -## DataEngineExecuteRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'execute'` | ✅ | | -| **command** | `any` | ✅ | | -| **options** | `Record` | optional | | - - --- ## DataEngineFilter @@ -346,76 +260,6 @@ Reference: any --- ---- - -## DataEngineFindOneRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'findOne'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; fields?: string[]; orderBy?: object[]; … }` | optional | | - -### Nested Shape: `DataEngineFindOneRequest.query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **fields** | `string[]` | optional | | -| **orderBy** | `{ field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | | -| **limit** | `number` | optional | | -| **offset** | `number` | optional | | -| **top** | `number` | optional | | -| **cursor** | `never` | optional | [REMOVED] `query.cursor` was removed in @objectstack/spec 17 (ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; `QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary `where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }` with the matching `orderBy` — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **expand** | `Record` | optional | | -| **distinct** | `never` | optional | [REMOVED] `query.distinct` was removed in @objectstack/spec 17 (ADR-0049 / ADR-0078) — no driver ever rendered SELECT DISTINCT; the flag's only observable effect was MIS-WIRED: the REST list path treated a distinct query as not countable and silently degraded `total`/`hasMore` to a page-local estimate while still returning duplicate rows. Delete the key; `QueryBuilder.distinct()` was removed with it, and the count suppression is gone (`total` is truthful again). For unique values of one column use the SQL/memory drivers' `distinct(object, field)` door; for unique combinations, `groupBy`; for a deduplicated count, the `count_distinct` aggregation. | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | -| **select** | `string[]` | optional | | -| **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | Sort order definition | -| **skip** | `integer` | optional | | -| **populate** | `string[]` | optional | | - - ---- - -## DataEngineFindRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'find'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; fields?: string[]; orderBy?: object[]; … }` | optional | | - -### Nested Shape: `DataEngineFindRequest.query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **fields** | `string[]` | optional | | -| **orderBy** | `{ field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | | -| **limit** | `number` | optional | | -| **offset** | `number` | optional | | -| **top** | `number` | optional | | -| **cursor** | `never` | optional | [REMOVED] `query.cursor` was removed in @objectstack/spec 17 (ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; `QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary `where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }` with the matching `orderBy` — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **expand** | `Record` | optional | | -| **distinct** | `never` | optional | [REMOVED] `query.distinct` was removed in @objectstack/spec 17 (ADR-0049 / ADR-0078) — no driver ever rendered SELECT DISTINCT; the flag's only observable effect was MIS-WIRED: the REST list path treated a distinct query as not countable and silently degraded `total`/`hasMore` to a page-local estimate while still returning duplicate rows. Delete the key; `QueryBuilder.distinct()` was removed with it, and the count suppression is gone (`total` is truthful again). For unique values of one column use the SQL/memory drivers' `distinct(object, field)` door; for unique combinations, `groupBy`; for a deduplicated count, the `count_distinct` aggregation. | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | -| **select** | `string[]` | optional | | -| **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | Sort order definition | -| **skip** | `integer` | optional | | -| **populate** | `string[]` | optional | | - - --- ## DataEngineInsertOptions @@ -468,20 +312,6 @@ Options for DataEngine.insert operations | **traceId** | `string` | optional | | ---- - -## DataEngineInsertRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'insert'` | ✅ | | -| **object** | `string` | ✅ | | -| **data** | `Record \| Record[]` | ✅ | | -| **options** | `{ context?: object; returning?: boolean }` | optional | Options for DataEngine.insert operations | - - --- ## DataEngineQueryOptions @@ -540,221 +370,6 @@ Query options for IDataEngine.find() operations | **traceId** | `string` | optional | | ---- - -## DataEngineRequest - -Virtual ObjectQL Request Protocol - -### Union Options - -This schema accepts one of the following structures: - -#### Option 1 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'find'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; fields?: string[]; orderBy?: object[]; … }` | optional | | - -### Nested Shape: `DataEngineRequest[method='find'].query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **fields** | `string[]` | optional | | -| **orderBy** | `{ field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | | -| **limit** | `number` | optional | | -| **offset** | `number` | optional | | -| **top** | `number` | optional | | -| **cursor** | `never` | optional | [REMOVED] `query.cursor` was removed in @objectstack/spec 17 (ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; `QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary `where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }` with the matching `orderBy` — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **expand** | `Record` | optional | | -| **distinct** | `never` | optional | [REMOVED] `query.distinct` was removed in @objectstack/spec 17 (ADR-0049 / ADR-0078) — no driver ever rendered SELECT DISTINCT; the flag's only observable effect was MIS-WIRED: the REST list path treated a distinct query as not countable and silently degraded `total`/`hasMore` to a page-local estimate while still returning duplicate rows. Delete the key; `QueryBuilder.distinct()` was removed with it, and the count suppression is gone (`total` is truthful again). For unique values of one column use the SQL/memory drivers' `distinct(object, field)` door; for unique combinations, `groupBy`; for a deduplicated count, the `count_distinct` aggregation. | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | -| **select** | `string[]` | optional | | -| **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | Sort order definition | -| **skip** | `integer` | optional | | -| **populate** | `string[]` | optional | | - ---- - -#### Option 2 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'findOne'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; fields?: string[]; orderBy?: object[]; … }` | optional | | - -### Nested Shape: `DataEngineRequest[method='findOne'].query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **fields** | `string[]` | optional | | -| **orderBy** | `{ field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | | -| **limit** | `number` | optional | | -| **offset** | `number` | optional | | -| **top** | `number` | optional | | -| **cursor** | `never` | optional | [REMOVED] `query.cursor` was removed in @objectstack/spec 17 (ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; `QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary `where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }` with the matching `orderBy` — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **expand** | `Record` | optional | | -| **distinct** | `never` | optional | [REMOVED] `query.distinct` was removed in @objectstack/spec 17 (ADR-0049 / ADR-0078) — no driver ever rendered SELECT DISTINCT; the flag's only observable effect was MIS-WIRED: the REST list path treated a distinct query as not countable and silently degraded `total`/`hasMore` to a page-local estimate while still returning duplicate rows. Delete the key; `QueryBuilder.distinct()` was removed with it, and the count suppression is gone (`total` is truthful again). For unique values of one column use the SQL/memory drivers' `distinct(object, field)` door; for unique combinations, `groupBy`; for a deduplicated count, the `count_distinct` aggregation. | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | -| **select** | `string[]` | optional | | -| **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | Sort order definition | -| **skip** | `integer` | optional | | -| **populate** | `string[]` | optional | | - ---- - -#### Option 3 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'insert'` | ✅ | | -| **object** | `string` | ✅ | | -| **data** | `Record \| Record[]` | ✅ | | -| **options** | `{ context?: object; returning?: boolean }` | optional | Options for DataEngine.insert operations | - ---- - -#### Option 4 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'update'` | ✅ | | -| **object** | `string` | ✅ | | -| **data** | `Record` | ✅ | | -| **id** | `string \| number` | optional | ID for single update, or use where in options | -| **options** | `{ context?: object; where?: Record \| any; multi?: boolean; returning?: boolean; … }` | optional | | - -### Nested Shape: `DataEngineRequest[method='update'].options` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **upsert** | `never` | optional | [REMOVED] `update.options.upsert` was removed in @objectstack/spec 17 (ADR-0049) — it was declared and allowlisted but never implemented: no engine or driver path ever read it, so `{ upsert: true }` was accepted and silently dropped and the update stayed a plain update. Delete the key. Express create-if-absent explicitly: a by-id update whose id names no row throws RECORD_NOT_FOUND (the by-id not-found gate) rather than inserting, so read the row first (`findOne`) and call `insert` or `update` on what you find. A first-class upsert, if ever built, must reconcile with that gate by design rather than through this silent flag. | -| **multi** | `boolean` | optional (default: `false`) | | -| **returning** | `boolean` | optional (default: `false`) | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - ---- - -#### Option 5 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'delete'` | ✅ | | -| **object** | `string` | ✅ | | -| **id** | `string \| number` | optional | ID for single delete, or use where in options | -| **options** | `{ context?: object; where?: Record \| any; multi?: boolean; filter?: Record \| any }` | optional | | - -### Nested Shape: `DataEngineRequest[method='delete'].options` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **multi** | `boolean` | optional (default: `false`) | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - ---- - -#### Option 6 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'count'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; search?: string \| object; searchFields?: string[]; … }` | optional | | - -### Nested Shape: `DataEngineRequest[method='count'].query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - ---- - -#### Option 7 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'aggregate'` | ✅ | | -| **object** | `string` | ✅ | | -| **query** | `{ context?: object; where?: Record \| any; groupBy?: (string \| object)[]; aggregations?: object[]; … }` | ✅ | | - -### Nested Shape: `DataEngineRequest[method='aggregate'].query` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **groupBy** | `(string \| { field: string; dateGranularity?: Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>; alias?: string })[]` | optional | GROUP BY targets (strings or `{field, dateGranularity?}` objects for date bucketing) | -| **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional | | -| **having** | `any` | optional | HAVING — filter over the aggregated rows (aggregation aliases + groupBy projections); applied engine-side after aggregation | -| **timezone** | `string` | optional | | -| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | | -| **searchFields** | `string[]` | optional | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - ---- - -#### Option 8 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'execute'` | ✅ | | -| **command** | `any` | ✅ | | -| **options** | `Record` | optional | | - ---- - -#### Option 9 - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'vectorFind'` | ✅ | | -| **object** | `string` | ✅ | | -| **vector** | `number[]` | ✅ | | -| **where** | `Record \| any` | optional | | -| **fields** | `string[]` | optional | | -| **limit** | `integer` | optional (default: `5`) | | -| **threshold** | `number` | optional | | - ---- - - --- ## DataEngineSort @@ -839,49 +454,6 @@ Options for DataEngine.update operations | **traceId** | `string` | optional | | ---- - -## DataEngineUpdateRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'update'` | ✅ | | -| **object** | `string` | ✅ | | -| **data** | `Record` | ✅ | | -| **id** | `string \| number` | optional | ID for single update, or use where in options | -| **options** | `{ context?: object; where?: Record \| any; multi?: boolean; returning?: boolean; … }` | optional | | - -### Nested Shape: `DataEngineUpdateRequest.options` - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **context** | `{ userId?: string; actor?: string; attributedUserId?: string; email?: string; … }` | optional | | -| **where** | `Record \| any` | optional | | -| **upsert** | `never` | optional | [REMOVED] `update.options.upsert` was removed in @objectstack/spec 17 (ADR-0049) — it was declared and allowlisted but never implemented: no engine or driver path ever read it, so `{ upsert: true }` was accepted and silently dropped and the update stayed a plain update. Delete the key. Express create-if-absent explicitly: a by-id update whose id names no row throws RECORD_NOT_FOUND (the by-id not-found gate) rather than inserting, so read the row first (`findOne`) and call `insert` or `update` on what you find. A first-class upsert, if ever built, must reconcile with that gate by design rather than through this silent flag. | -| **multi** | `boolean` | optional (default: `false`) | | -| **returning** | `boolean` | optional (default: `false`) | | -| **filter** | `Record \| any` | optional | Data Engine query filter conditions | - - ---- - -## DataEngineVectorFindRequest - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **method** | `'vectorFind'` | ✅ | | -| **object** | `string` | ✅ | | -| **vector** | `number[]` | ✅ | | -| **where** | `Record \| any` | optional | | -| **fields** | `string[]` | optional | | -| **limit** | `integer` | optional (default: `5`) | | -| **threshold** | `number` | optional | | - - --- ## DroppedFieldsEvent diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 17a7acbbad1..ad5714aef47 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,7 +1,7 @@ --- title: Protocol reference — every schema by module navTitle: Protocol Reference -description: Every schema published by @objectstack/spec — 1559 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1549 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -23,7 +23,7 @@ counts are sums of the rows they head. Regenerate with | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. | | [API Protocol](/docs/references/api) | 32 | 440 | REST contracts, endpoints, routing, realtime, batch, discovery. | | [Automation Protocol](/docs/references/automation) | 15 | 77 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. | -| [Data Protocol](/docs/references/data) | 31 | 184 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | +| [Data Protocol](/docs/references/data) | 31 | 174 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. | | [Integration Protocol](/docs/references/integration) | 1 | 12 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. | | [Kernel Protocol](/docs/references/kernel) | 29 | 151 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. | @@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | | [System Protocol](/docs/references/system) | 37 | 287 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 16 | 172 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **202** | **1559** | 14 protocol modules | +| **Total** | **202** | **1549** | 14 protocol modules | --- @@ -132,7 +132,7 @@ Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execu ## Data Protocol -**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **31 pages, 184 schemas** +**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **31 pages, 174 schemas** Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. @@ -141,7 +141,7 @@ Objects, fields, queries, filters, datasources and drivers — the ObjectQL laye | [`analytics.zod.ts`](/docs/references/data/analytics) | `AggregationMetricType`, `AnalyticsDateRange`, `AnalyticsDateRangePreset`, `AnalyticsQuery`, `Cube`, `CubeJoin`, `Dimension`, `DimensionType`, `Metric`, `TimeUpdateInterval` | | [`column-apply-on.zod.ts`](/docs/references/data/column-apply-on) | `ColumnApplyOn` | | [`context-tokens.zod.ts`](/docs/references/data/context-tokens) | `ContextToken`, `ContextTokenPlaceholder` | -| [`data-engine.zod.ts`](/docs/references/data/data-engine) | `BaseEngineOptions`, `DataEngineAggregateOptions`, `DataEngineAggregateRequest`, `DataEngineCountOptions`, `DataEngineCountRequest`, `DataEngineDeleteOptions`, `DataEngineDeleteRequest`, `DataEngineExecuteRequest`, `DataEngineFilter`, `DataEngineFindOneRequest`, `DataEngineFindRequest`, `DataEngineInsertOptions`, `DataEngineInsertRequest`, `DataEngineQueryOptions`, `DataEngineRequest`, `DataEngineSort`, `DataEngineUpdateOptions`, `DataEngineUpdateRequest`, `DataEngineVectorFindRequest`, `DroppedFieldsEvent`, `EngineAggregateOptions`, `EngineCountOptions`, `EngineDeleteOptions`, `EngineQueryOptions`, `EngineUpdateOptions`, `QueryPageWindow`, `QueryPageWindowParam`, `QueryTransportParams`, `QueryWithTransport`, `ValidationAdvisoryEvent` | +| [`data-engine.zod.ts`](/docs/references/data/data-engine) | `BaseEngineOptions`, `DataEngineAggregateOptions`, `DataEngineCountOptions`, `DataEngineDeleteOptions`, `DataEngineFilter`, `DataEngineInsertOptions`, `DataEngineQueryOptions`, `DataEngineSort`, `DataEngineUpdateOptions`, `DroppedFieldsEvent`, `EngineAggregateOptions`, `EngineCountOptions`, `EngineDeleteOptions`, `EngineQueryOptions`, `EngineUpdateOptions`, `QueryPageWindow`, `QueryPageWindowParam`, `QueryTransportParams`, `QueryWithTransport`, `ValidationAdvisoryEvent` | | [`datasource.zod.ts`](/docs/references/data/datasource) | `Datasource`, `DriverDefinition`, `DriverType`, `ExternalDatasourceSettings`, `SchemaMode` | | [`date-macros.zod.ts`](/docs/references/data/date-macros) | `DateMacroPlaceholder`, `DateMacroToken` | | [`driver.zod.ts`](/docs/references/data/driver) | `DriverCapabilities`, `DriverConfig`, `DriverOptions`, `PoolConfig` | diff --git a/docs/adr/0119-plugin-reachable-transactions-and-honest-atomic-batch.md b/docs/adr/0119-plugin-reachable-transactions-and-honest-atomic-batch.md index bb67ffbae5f..718c46619f8 100644 --- a/docs/adr/0119-plugin-reachable-transactions-and-honest-atomic-batch.md +++ b/docs/adr/0119-plugin-reachable-transactions-and-honest-atomic-batch.md @@ -58,7 +58,9 @@ The honest shape is therefore *both*: the transaction is the unit of atomicity f ### Rot found on the way -**`IDataEngine.batch?`** (`packages/spec/src/contracts/data-engine.ts#IDataEngine`) — the member #4612 opens with. Optional, so nothing must implement it; no engine does. Its entire specification is the three-word comment `Batch Operations (Transactional)` — nothing about partial failure, ordering, cross-object references, rollback scope, or what `transaction: false` means. `DataEngineRequest` is imported by exactly one non-spec file: the contract declaring it. Its request union (`packages/spec/src/data/data-engine.zod.ts#DataEngineRequest`) even nests batches recursively, a shape nobody designed against because nobody built it. The only test (`packages/spec/src/contracts/data-engine.test.ts`) constructs an ad-hoc literal with a `batch` property and asserts it is defined — it pins the type, not an implementation. +**`IDataEngine.batch?`** (`packages/spec/src/contracts/data-engine.ts#IDataEngine`) — the member #4612 opens with. Optional, so nothing must implement it; no engine does. Its entire specification is the three-word comment `Batch Operations (Transactional)` — nothing about partial failure, ordering, cross-object references, rollback scope, or what `transaction: false` means. `DataEngineRequest` is imported by exactly one non-spec file: the contract declaring it. Its request union (`DataEngineRequest`, in `packages/spec/src/data/data-engine.zod.ts`) even nests batches recursively, a shape nobody designed against because nobody built it. The only test (`packages/spec/src/contracts/data-engine.test.ts`) constructs an ad-hoc literal with a `batch` property and asserts it is defined — it pins the type, not an implementation. + +> **Editorial note (2026-10-28, #27028):** `DataEngineRequest` and the rest of that request protocol have since been retired (ADR-0049 enforce-or-remove; registered as `packages/spec/src/migrations/entries/semantic/18.data-engine-rpc-protocol-retired.ts`), so the symbol is named here without an anchor. The finding stands as written: nothing served the union. **`batchData`'s `atomic` flag** (`packages/metadata-protocol/src/protocol.ts#batchData`) — advertised as *"rollback entire batch on any failure (transaction mode)"*. It opens no transaction. It breaks the loop : diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md index 37c7e35c590..6d67334be3f 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md @@ -21,7 +21,7 @@ The `strict` column is the one the campaign schedules against; it counts both th | Dir | Sites | strict | passthrough | catchall | strip | |---|---|---|---|---|---| -| `data/` | 164 | 88 | 2 | 0 | 74 | +| `data/` | 154 | 88 | 2 | 0 | 64 | ## `data/` — sites @@ -32,7 +32,7 @@ classify and is not listed (it becomes reportable the day it grows its first sit | File | Sites | |---|---| | `analytics.zod.ts` | 6 | -| `data-engine.zod.ts` | 16 | +| `data-engine.zod.ts` | 6 | | `datasource.zod.ts` | 6 | | `driver-nosql.zod.ts` | 10 | | `driver-sql.zod.ts` | 2 | @@ -57,7 +57,7 @@ classify and is not listed (it becomes reportable the day it grows its first sit | `seed-loader.zod.ts` | 12 | | `seed.zod.ts` | 2 | | `validation.zod.ts` | 6 | -| **total** | **164** | +| **total** | **154** | ## `data/` — open @@ -65,11 +65,11 @@ Per file, how many of its sites still silently discard unknown keys. The `Class` column that decides the bucket split is hand-written in the ledger; the arithmetic over it is here. -**74 strip of 164**, in 10 file(s). +**64 strip of 154**, in 10 file(s). | File | Strip | Sites | |---|---|---| -| `data-engine.zod.ts` | 16 | 16 | +| `data-engine.zod.ts` | 6 | 6 | | `driver-nosql.zod.ts` | 10 | 10 | | `driver-sql.zod.ts` | 2 | 2 | | `driver.zod.ts` | 9 | 9 | @@ -79,13 +79,13 @@ over it is here. | `hook.zod.ts` | 5 | 8 | | `query.zod.ts` | 4 | 5 | | `seed-loader.zod.ts` | 12 | 12 | -| **total** | **74** | **164** | +| **total** | **64** | **154** | | Bucket | Sites | |---|---| | authorable — the ruling's forced scope | 0 | | unresolved — needs a per-schema verdict | 0 | -| wire / open — out of forced scope | 73 | +| wire / open — out of forced scope | 63 | | no door — no carrier, ADR-0049 territory | 1 | | no gate — carrier live, no parse | 0 | | covered — no carrier, no parse, guarded at every consumer | 0 | diff --git a/packages/objectql/src/engine.ts b/packages/objectql/src/engine.ts index 3dcaaed4af7..50a504332d7 100644 --- a/packages/objectql/src/engine.ts +++ b/packages/objectql/src/engine.ts @@ -597,8 +597,8 @@ export const HOOK_EVENT_DISPATCH_VOCABULARY = { * * `where` is the slot every method folds; `limit` additionally applies to the * find-shaped bags, which declare `top` (OData) as its alias. The other four - * pairs in the table are RPC/wire spellings folded at parse by - * `RpcQueryOptionsSchema` / the protocol normalizer — their values need shape + * pairs in the table are wire spellings folded at parse by + * `QueryWithTransportSchema` / the protocol normalizer — their values need shape * lowering (`sort` records, `populate` lists) that belongs to those layers, so * the engine deliberately does not fold them. */ diff --git a/packages/spec/DEVELOPMENT_PLAN.md b/packages/spec/DEVELOPMENT_PLAN.md index b5a2967cb2d..4d9f1cc0d86 100644 --- a/packages/spec/DEVELOPMENT_PLAN.md +++ b/packages/spec/DEVELOPMENT_PLAN.md @@ -310,8 +310,8 @@ Replace `id: z.any()` with `z.union([z.string(), z.number()])`: | File | Fields | |---|---| -| `data/data-engine.zod.ts` L200 | `DataEngineUpdateRequestSchema.id` | -| `data/data-engine.zod.ts` L207 | `DataEngineDeleteRequestSchema.id` | +| ~~`data/data-engine.zod.ts` L200~~ | ~~`DataEngineUpdateRequestSchema.id`~~ (retired with the Virtual ObjectQL request protocol, #27028) | +| ~~`data/data-engine.zod.ts` L207~~ | ~~`DataEngineDeleteRequestSchema.id`~~ (retired with the Virtual ObjectQL request protocol, #27028) | ### 3.3 Fix critical `z.any()` hotspots (per-file) diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 9031830606f..96b7e6023bd 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -169,38 +169,22 @@ "DRIVER_ID_ALIASES (const)", "DataEngineAggregateOptions (type)", "DataEngineAggregateOptionsSchema (const)", - "DataEngineAggregateRequestSchema (const)", - "DataEngineContractSchema (const)", "DataEngineCountOptions (type)", "DataEngineCountOptionsSchema (const)", - "DataEngineCountRequestSchema (const)", "DataEngineDeleteOptions (type)", "DataEngineDeleteOptionsSchema (const)", - "DataEngineDeleteRequestSchema (const)", - "DataEngineExecuteRequest (type)", - "DataEngineExecuteRequestSchema (const)", "DataEngineFilter (type)", "DataEngineFilterSchema (const)", - "DataEngineFindOneRequestSchema (const)", - "DataEngineFindRequestSchema (const)", "DataEngineInsertOptions (type)", "DataEngineInsertOptionsSchema (const)", - "DataEngineInsertRequest (type)", - "DataEngineInsertRequestSchema (const)", "DataEngineQueryOptions (type)", "DataEngineQueryOptionsParsed (type)", "DataEngineQueryOptionsSchema (const)", - "DataEngineRequest (type)", - "DataEngineRequestParsed (type)", - "DataEngineRequestSchema (const)", "DataEngineSort (type)", "DataEngineSortParsed (type)", "DataEngineSortSchema (const)", "DataEngineUpdateOptions (type)", "DataEngineUpdateOptionsSchema (const)", - "DataEngineUpdateRequestSchema (const)", - "DataEngineVectorFindRequest (type)", - "DataEngineVectorFindRequestSchema (const)", "DataTypeMapping (type)", "DataTypeMappingSchema (const)", "Datasource (type)", diff --git a/packages/spec/authorable-defaults/data.json b/packages/spec/authorable-defaults/data.json index cbf195cdae1..b15afc4aa00 100644 --- a/packages/spec/authorable-defaults/data.json +++ b/packages/spec/authorable-defaults/data.json @@ -16,7 +16,6 @@ "data/DataEngineInsertOptions:returning = true", "data/DataEngineUpdateOptions:multi = false", "data/DataEngineUpdateOptions:returning = false", - "data/DataEngineVectorFindRequest:limit = 5", "data/Datasource:active = true", "data/Datasource:autoConnect = false", "data/Datasource:origin = \"code\"", diff --git a/packages/spec/authorable-surface/data.json b/packages/spec/authorable-surface/data.json index 31a5e674f68..8307e8f4256 100644 --- a/packages/spec/authorable-surface/data.json +++ b/packages/spec/authorable-surface/data.json @@ -107,36 +107,13 @@ "data/DataEngineAggregateOptions:context", "data/DataEngineAggregateOptions:filter", "data/DataEngineAggregateOptions:groupBy", - "data/DataEngineAggregateRequest:method", - "data/DataEngineAggregateRequest:object", - "data/DataEngineAggregateRequest:query", "data/DataEngineCountOptions:context", "data/DataEngineCountOptions:filter", - "data/DataEngineCountRequest:method", - "data/DataEngineCountRequest:object", - "data/DataEngineCountRequest:query", "data/DataEngineDeleteOptions:context", "data/DataEngineDeleteOptions:filter", "data/DataEngineDeleteOptions:multi", - "data/DataEngineDeleteRequest:id", - "data/DataEngineDeleteRequest:method", - "data/DataEngineDeleteRequest:object", - "data/DataEngineDeleteRequest:options", - "data/DataEngineExecuteRequest:command", - "data/DataEngineExecuteRequest:method", - "data/DataEngineExecuteRequest:options", - "data/DataEngineFindOneRequest:method", - "data/DataEngineFindOneRequest:object", - "data/DataEngineFindOneRequest:query", - "data/DataEngineFindRequest:method", - "data/DataEngineFindRequest:object", - "data/DataEngineFindRequest:query", "data/DataEngineInsertOptions:context", "data/DataEngineInsertOptions:returning", - "data/DataEngineInsertRequest:data", - "data/DataEngineInsertRequest:method", - "data/DataEngineInsertRequest:object", - "data/DataEngineInsertRequest:options", "data/DataEngineQueryOptions:context", "data/DataEngineQueryOptions:filter", "data/DataEngineQueryOptions:limit", @@ -150,18 +127,6 @@ "data/DataEngineUpdateOptions:multi", "data/DataEngineUpdateOptions:returning", "data/DataEngineUpdateOptions:upsert [RETIRED]", - "data/DataEngineUpdateRequest:data", - "data/DataEngineUpdateRequest:id", - "data/DataEngineUpdateRequest:method", - "data/DataEngineUpdateRequest:object", - "data/DataEngineUpdateRequest:options", - "data/DataEngineVectorFindRequest:fields", - "data/DataEngineVectorFindRequest:limit", - "data/DataEngineVectorFindRequest:method", - "data/DataEngineVectorFindRequest:object", - "data/DataEngineVectorFindRequest:threshold", - "data/DataEngineVectorFindRequest:vector", - "data/DataEngineVectorFindRequest:where", "data/DataTypeMapping:binary", "data/DataTypeMapping:boolean", "data/DataTypeMapping:date", diff --git a/packages/spec/declaration-map/data.json b/packages/spec/declaration-map/data.json index 790a2205074..42d67397f28 100644 --- a/packages/spec/declaration-map/data.json +++ b/packages/spec/declaration-map/data.json @@ -53,34 +53,20 @@ "CurrencyConfigSchema": "data/CurrencyConfig", "DataEngineAggregateOptions": "data/DataEngineAggregateOptions", "DataEngineAggregateOptionsSchema": "data/DataEngineAggregateOptions", - "DataEngineAggregateRequestSchema": "data/DataEngineAggregateRequest", "DataEngineCountOptions": "data/DataEngineCountOptions", "DataEngineCountOptionsSchema": "data/DataEngineCountOptions", - "DataEngineCountRequestSchema": "data/DataEngineCountRequest", "DataEngineDeleteOptions": "data/DataEngineDeleteOptions", "DataEngineDeleteOptionsSchema": "data/DataEngineDeleteOptions", - "DataEngineDeleteRequestSchema": "data/DataEngineDeleteRequest", - "DataEngineExecuteRequest": "data/DataEngineExecuteRequest", - "DataEngineExecuteRequestSchema": "data/DataEngineExecuteRequest", "DataEngineFilter": "data/DataEngineFilter", "DataEngineFilterSchema": "data/DataEngineFilter", - "DataEngineFindOneRequestSchema": "data/DataEngineFindOneRequest", - "DataEngineFindRequestSchema": "data/DataEngineFindRequest", "DataEngineInsertOptions": "data/DataEngineInsertOptions", "DataEngineInsertOptionsSchema": "data/DataEngineInsertOptions", - "DataEngineInsertRequest": "data/DataEngineInsertRequest", - "DataEngineInsertRequestSchema": "data/DataEngineInsertRequest", "DataEngineQueryOptions": "data/DataEngineQueryOptions", "DataEngineQueryOptionsSchema": "data/DataEngineQueryOptions", - "DataEngineRequest": "data/DataEngineRequest", - "DataEngineRequestSchema": "data/DataEngineRequest", "DataEngineSort": "data/DataEngineSort", "DataEngineSortSchema": "data/DataEngineSort", "DataEngineUpdateOptions": "data/DataEngineUpdateOptions", "DataEngineUpdateOptionsSchema": "data/DataEngineUpdateOptions", - "DataEngineUpdateRequestSchema": "data/DataEngineUpdateRequest", - "DataEngineVectorFindRequest": "data/DataEngineVectorFindRequest", - "DataEngineVectorFindRequestSchema": "data/DataEngineVectorFindRequest", "DataTypeMapping": "data/DataTypeMapping", "DataTypeMappingSchema": "data/DataTypeMapping", "Datasource": "data/Datasource", diff --git a/packages/spec/docs-import-surface.baseline.json b/packages/spec/docs-import-surface.baseline.json index 89dd3292fcd..72926d1cef3 100644 --- a/packages/spec/docs-import-surface.baseline.json +++ b/packages/spec/docs-import-surface.baseline.json @@ -11,12 +11,6 @@ "api/MetadataQueryRequest — no type export", "automation/FlowVariable — no type export", "data/ComparisonOperator — no type export", - "data/DataEngineAggregateRequest — no type export", - "data/DataEngineCountRequest — no type export", - "data/DataEngineDeleteRequest — no type export", - "data/DataEngineFindOneRequest — no type export", - "data/DataEngineFindRequest — no type export", - "data/DataEngineUpdateRequest — no type export", "data/DateGranularity — no type export", "data/EqualityOperator — no type export", "data/HookEvent — no type export", diff --git a/packages/spec/dropped-refinements.baseline.json b/packages/spec/dropped-refinements.baseline.json index 8770d6cf062..60d3bd77748 100644 --- a/packages/spec/dropped-refinements.baseline.json +++ b/packages/spec/dropped-refinements.baseline.json @@ -2,8 +2,8 @@ "description": "Shrink-only ledger of every PUBLISHED JSON Schema that is STILL WIDER than the Zod type it was generated from, because a rule written as `.refine()` reaches the runtime and not the file (#18670). `z.toJSONSchema()` has no arm for a `custom` check: a plain record, the same record with a `.refine()`, and the same record with an ABORTING `.refine()` all project byte-identically (measured on zod 4.4.3, the version packages/spec resolves). So a document one of these files ACCEPTS can still be refused at parse time, and an author -- or an AI -- validating against packages/spec/json-schema/** finds out a release later. Each `sites` path is a position under that schema at which a refinement is dropped; the same paths are written onto the artifact itself as `x-dropped-refinements`. Item 2 closed the first patterns: a refinement DECLARED through the closed list in src/shared/refinement-projection.ts is emitted into the published file, reads `projected` rather than `dropped`, and its row LEAVES this ledger in the same PR -- which is why the ledger shrinks and never grows on a repair. Every refinement outside that closed list stays here, and adding an arm to the list is a public-contract decision, not a refactor. Hand-edited on purpose and with no `gen:` script: a generator would let a new gap be admitted by running a command instead of by a decision, which is the silence this ledger exists to end. Adding, removing or moving a site fails packages/spec/scripts/build-schemas.ts until the line moves with it, and the failure prints the corrected entry in full. ⛔ Do not delete or weaken a refinement to shorten this file -- the runtime rule is correct; it is the projection that is silent, and the remedy is to teach the closed list a NAMED pattern, never to drop the rule.", "measured": { "zod": "4.4.3", - "publishedSchemasWithDroppedRefinements": 252, - "droppedRefinementSites": 900, + "publishedSchemasWithDroppedRefinements": 244, + "droppedRefinementSites": 892, "refinementSitesThatDidProject": 369, "refinementSitesWithNoJsonFormToCompare": 0 }, @@ -771,71 +771,31 @@ "filter.options[1].lazy" ] }, - "data/DataEngineAggregateRequest": { - "sites": [ - "query.in.having.lazy" - ] - }, "data/DataEngineCountOptions": { "sites": [ "filter.options[1].lazy" ] }, - "data/DataEngineCountRequest": { - "sites": [ - "query.in.where.options[1].lazy" - ] - }, "data/DataEngineDeleteOptions": { "sites": [ "filter.options[1].lazy" ] }, - "data/DataEngineDeleteRequest": { - "sites": [ - "options.in.where.options[1].lazy" - ] - }, "data/DataEngineFilter": { "sites": [ "options[1].lazy" ] }, - "data/DataEngineFindOneRequest": { - "sites": [ - "query.in.where.options[1].lazy" - ] - }, - "data/DataEngineFindRequest": { - "sites": [ - "query.in.where.options[1].lazy" - ] - }, "data/DataEngineQueryOptions": { "sites": [ "filter.options[1].lazy" ] }, - "data/DataEngineRequest": { - "sites": [ - "options[8].where.options[1].lazy" - ] - }, "data/DataEngineUpdateOptions": { "sites": [ "filter.options[1].lazy" ] }, - "data/DataEngineUpdateRequest": { - "sites": [ - "options.in.where.options[1].lazy" - ] - }, - "data/DataEngineVectorFindRequest": { - "sites": [ - "where.options[1].lazy" - ] - }, "data/Datasource": { "sites": [ "" diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index c393808c02d..71e26d894d9 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -166,38 +166,22 @@ "DRIVER_ID_ALIASES": "src/data/driver/config-registry.zod.ts#DRIVER_ID_ALIASES (const)", "DataEngineAggregateOptions": "src/data/data-engine.zod.ts#DataEngineAggregateOptions (type)", "DataEngineAggregateOptionsSchema": "src/data/data-engine.zod.ts#DataEngineAggregateOptionsSchema (const)", - "DataEngineAggregateRequestSchema": "src/data/data-engine.zod.ts#DataEngineAggregateRequestSchema (const)", - "DataEngineContractSchema": "src/data/data-engine.zod.ts#DataEngineContractSchema (const)", "DataEngineCountOptions": "src/data/data-engine.zod.ts#DataEngineCountOptions (type)", "DataEngineCountOptionsSchema": "src/data/data-engine.zod.ts#DataEngineCountOptionsSchema (const)", - "DataEngineCountRequestSchema": "src/data/data-engine.zod.ts#DataEngineCountRequestSchema (const)", "DataEngineDeleteOptions": "src/data/data-engine.zod.ts#DataEngineDeleteOptions (type)", "DataEngineDeleteOptionsSchema": "src/data/data-engine.zod.ts#DataEngineDeleteOptionsSchema (const)", - "DataEngineDeleteRequestSchema": "src/data/data-engine.zod.ts#DataEngineDeleteRequestSchema (const)", - "DataEngineExecuteRequest": "src/data/data-engine.zod.ts#DataEngineExecuteRequest (type)", - "DataEngineExecuteRequestSchema": "src/data/data-engine.zod.ts#DataEngineExecuteRequestSchema (const)", "DataEngineFilter": "src/data/data-engine.zod.ts#DataEngineFilter (type)", "DataEngineFilterSchema": "src/data/data-engine.zod.ts#DataEngineFilterSchema (const)", - "DataEngineFindOneRequestSchema": "src/data/data-engine.zod.ts#DataEngineFindOneRequestSchema (const)", - "DataEngineFindRequestSchema": "src/data/data-engine.zod.ts#DataEngineFindRequestSchema (const)", "DataEngineInsertOptions": "src/data/data-engine.zod.ts#DataEngineInsertOptions (type)", "DataEngineInsertOptionsSchema": "src/data/data-engine.zod.ts#DataEngineInsertOptionsSchema (const)", - "DataEngineInsertRequest": "src/data/data-engine.zod.ts#DataEngineInsertRequest (type)", - "DataEngineInsertRequestSchema": "src/data/data-engine.zod.ts#DataEngineInsertRequestSchema (const)", "DataEngineQueryOptions": "src/data/data-engine.zod.ts#DataEngineQueryOptions (type)", "DataEngineQueryOptionsParsed": "src/data/data-engine.zod.ts#DataEngineQueryOptionsParsed (type)", "DataEngineQueryOptionsSchema": "src/data/data-engine.zod.ts#DataEngineQueryOptionsSchema (const)", - "DataEngineRequest": "src/data/data-engine.zod.ts#DataEngineRequest (type)", - "DataEngineRequestParsed": "src/data/data-engine.zod.ts#DataEngineRequestParsed (type)", - "DataEngineRequestSchema": "src/data/data-engine.zod.ts#DataEngineRequestSchema (const)", "DataEngineSort": "src/data/data-engine.zod.ts#DataEngineSort (type)", "DataEngineSortParsed": "src/data/data-engine.zod.ts#DataEngineSortParsed (type)", "DataEngineSortSchema": "src/data/data-engine.zod.ts#DataEngineSortSchema (const)", "DataEngineUpdateOptions": "src/data/data-engine.zod.ts#DataEngineUpdateOptions (type)", "DataEngineUpdateOptionsSchema": "src/data/data-engine.zod.ts#DataEngineUpdateOptionsSchema (const)", - "DataEngineUpdateRequestSchema": "src/data/data-engine.zod.ts#DataEngineUpdateRequestSchema (const)", - "DataEngineVectorFindRequest": "src/data/data-engine.zod.ts#DataEngineVectorFindRequest (type)", - "DataEngineVectorFindRequestSchema": "src/data/data-engine.zod.ts#DataEngineVectorFindRequestSchema (const)", "DataTypeMapping": "src/data/driver-sql.zod.ts#DataTypeMapping (type)", "DataTypeMappingSchema": "src/data/driver-sql.zod.ts#DataTypeMappingSchema (const)", "Datasource": "src/data/datasource.zod.ts#Datasource (type)", diff --git a/packages/spec/json-schema.manifest/data.json b/packages/spec/json-schema.manifest/data.json index 0ab42398d16..f0da49cd8df 100644 --- a/packages/spec/json-schema.manifest/data.json +++ b/packages/spec/json-schema.manifest/data.json @@ -29,23 +29,13 @@ "data/CubeJoin", "data/CurrencyConfig", "data/DataEngineAggregateOptions", - "data/DataEngineAggregateRequest", "data/DataEngineCountOptions", - "data/DataEngineCountRequest", "data/DataEngineDeleteOptions", - "data/DataEngineDeleteRequest", - "data/DataEngineExecuteRequest", "data/DataEngineFilter", - "data/DataEngineFindOneRequest", - "data/DataEngineFindRequest", "data/DataEngineInsertOptions", - "data/DataEngineInsertRequest", "data/DataEngineQueryOptions", - "data/DataEngineRequest", "data/DataEngineSort", "data/DataEngineUpdateOptions", - "data/DataEngineUpdateRequest", - "data/DataEngineVectorFindRequest", "data/DataTypeMapping", "data/Datasource", "data/DateGranularity", diff --git a/packages/spec/src/contracts/data-engine.ts b/packages/spec/src/contracts/data-engine.ts index acebfc49fbb..8c3c210504e 100644 --- a/packages/spec/src/contracts/data-engine.ts +++ b/packages/spec/src/contracts/data-engine.ts @@ -58,8 +58,8 @@ import type { IntrospectedSchema } from './schema-diff-service.js'; * * Lives on the TS contract — NOT in the serializable Zod options schemas * (`EngineUpdateOptionsSchema` etc.): a function is unrepresentable in JSON - * Schema and cannot cross the RPC (Virtual Data Engine) boundary, so remote - * callers simply never receive these events. The event payload itself is + * Schema and cannot be serialized, so remote callers (REST, the client SDK) + * simply never receive these events. The event payload itself is * Zod-first: `DroppedFieldsEventSchema` in `data/data-engine.zod.ts`. * * A listener that throws must never break the write — engines catch and log. diff --git a/packages/spec/src/data/data-engine-rpc-protocol-retirement.test.ts b/packages/spec/src/data/data-engine-rpc-protocol-retirement.test.ts new file mode 100644 index 00000000000..6fcad16101e --- /dev/null +++ b/packages/spec/src/data/data-engine-rpc-protocol-retirement.test.ts @@ -0,0 +1,216 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The Virtual ObjectQL request protocol — `data/data-engine.zod.ts`'s + * `DataEngineRequestSchema`, its nine arms and `DataEngineContractSchema` — + * RETIRED whole (#27028, ADR-0049 enforce-or-remove), by the maintainer's ruling + * A on its decision card (2026-10-28): retire it rather than keep it as a planned + * contract. No Virtual Data Engine existed and nothing parsed the protocol. + * + * Zero readers, measured before the removal (counts plus the tree each was + * taken against): objectstack `1287b2fa` — no importer or parser outside + * `packages/spec/src/data/` (generated ledgers, reference docs and changelogs + * aside; control: 22 files name `DataEngineInsertOptions`, which stays); + * hotcrm `353dace`, objectui `b2c0ade` and docs.objectos.ai `cab35fd` — 0 + * (controls: 311, 2095 and 74 files import `@objectstack/spec`). + * + * Bookkeeping shapes, pinned below: + * 1. No carrier key, so no `retiredKey()` tombstone and no D2 conversion: no + * metadata source authors these shapes, so a conversion would have no seam + * that runs. + * 2. `RETIRED_DEFS_BY_MAJOR[18]` carries the ten published defs, and the D3 + * entry `data-engine-rpc-protocol-retired` carries the judgement an + * upgrader owes. + * 3. The protocol's exports are gone from `@objectstack/spec/data` (and the + * root entry carries none of them), while the option bags `IDataEngine` is + * typed with and `RPC_QUERY_ALIAS_SLOTS` survive (the lit controls). + * 4. Tree-scoped absence: nothing inside the declared radius still imports a + * retired export from a spec specifier or reaches one through `Data.`. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { describe, expect, it } from 'vitest'; + +import * as data from './index'; +import * as root from '../index'; +import { MIGRATIONS_BY_MAJOR, RETIRED_DEFS_BY_MAJOR } from '../migrations/registry'; + +const ARMS = ['Find', 'FindOne', 'Insert', 'Update', 'Delete', 'Count', 'Aggregate', 'Execute', 'VectorFind']; +const RETIRED_DEFS = ['data/DataEngineRequest', ...ARMS.map((arm) => `data/DataEngine${arm}Request`)]; +const RETIRED_VALUE_EXPORTS = [ + 'DataEngineRequestSchema', + ...ARMS.map((arm) => `DataEngine${arm}RequestSchema`), + 'DataEngineContractSchema', +]; + +describe('the Virtual ObjectQL request protocol is retired whole, and registered', () => { + it('RETIRED_DEFS_BY_MAJOR[18] carries the ten published defs', () => { + expect(RETIRED_DEFS).toHaveLength(10); + for (const def of RETIRED_DEFS) expect(RETIRED_DEFS_BY_MAJOR[18], def).toContain(def); + }); + + it('the D3 entry `data-engine-rpc-protocol-retired` names the doors that exist', () => { + const entry = MIGRATIONS_BY_MAJOR[18]!.semantic.find((s) => s.id === 'data-engine-rpc-protocol-retired'); + expect(entry).toBeDefined(); + expect(entry!.replacement).toMatch(/`IDataEngine`/); + expect(entry!.replacement).toMatch(/POST \{basePath\}\/batch/); + expect(entry!.reason).toMatch(/ADR-0049/); + expect(entry!.acceptanceCriteria).toMatch(/TS2305/); + // No conversion judged: there is no seam a conversion could run on. + expect(entry!.conversionIds).toBeUndefined(); + }); +}); + +describe('the protocol\'s exports are gone from every entry that carried them', () => { + it.each(RETIRED_VALUE_EXPORTS)('`%s` is exported neither from `@objectstack/spec/data` nor from the root entry', (name) => { + expect(Object.keys(data)).not.toContain(name); + expect(Object.keys(root)).not.toContain(name); + }); + + it('the option bags and the alias table survive — the lit controls', () => { + for (const name of ['EngineQueryOptionsSchema', 'DataEngineInsertOptionsSchema', 'RPC_QUERY_ALIAS_SLOTS', 'QueryWithTransportSchema']) { + expect(Object.keys(data), name).toContain(name); + } + }); +}); + +// ─── Tree-scoped absence, with a DECLARED radius ───────────────────────────── +// +// `tsc` is the primary sweeper — the protocol's exports are gone, so every +// typed import fails to compile (TS2305). The residue is what `tsc` never +// judges: MD/MDX code fences, JSON, YAML and untyped `.js`. This walk covers +// that residue across the five repo roots `scripts/cross-package-test-inputs.mjs` +// declares for `@objectstack/spec#test` (mirrored in `turbo.json`), plus the +// example apps' own `src/` trees, declared there as `examples/*/src/**/*.ts`. +// +// Two matchers, each judging a USE, never a mention: +// - an `import` / `export … from` naming a retired export from an +// `@objectstack/spec` specifier, the type aliases included; +// - a `Data.` access through the documented +// `import * as Data from '@objectstack/spec/data'` namespace. +// Inline code is prose and is stripped before judging. The bound, stated: +// `docs/**`, `.claude/**`, `.github/**` and the repo-root files are outside what +// this walk sees. +describe('tree-scoped absence: nothing inside the declared radius still imports the Virtual ObjectQL protocol', () => { + const SPEC_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..'); + const REPO_ROOT = path.resolve(SPEC_ROOT, '../..'); + const THIS_FILE = path.relative(REPO_ROOT, fileURLToPath(import.meta.url)).split(path.sep).join('/'); + + /** The walked roots — declared in `scripts/cross-package-test-inputs.mjs` under `@objectstack/spec`. */ + const WALK_ROOTS = ['packages', 'examples', 'skills', 'content', 'scripts']; + const SCANNED_EXT = new Set(['.ts', '.mts', '.cts', '.tsx', '.js', '.mjs', '.cjs', '.json', '.md', '.mdx', '.yaml', '.yml']); + /** Under `examples/` the non-code extensions, plus `.ts` inside an app's own `src/` tree. */ + const EXAMPLES_EXT = new Set(['.json', '.md', '.mdx', '.yaml', '.yml']); + const EXAMPLE_APP_SRC_TS = /^examples\/[^/]+\/src\/.+\.ts$/; + const SKIPPED_DIRS = new Set(['node_modules', 'dist', '.git', '.turbo', '.cache', '.objectstack', 'coverage', '.next', '.source']); + + /** Every retired name: the ten request schemas and their types, and the contract schema. */ + const RETIRED_NAMES = + '(DataEngine(Find|FindOne|Insert|Update|Delete|Count|Aggregate|Execute|VectorFind)?Request(Schema|Parsed)?' + + '|DataEngineContractSchema)'; + const USES = [ + // An import or re-export naming a retired export from a spec specifier. + new RegExp(`\\b(import|export)\\s+(type\\s+)?\\{[^}]*\\b${RETIRED_NAMES}\\b[^}]*\\}\\s*from\\s*['"]@objectstack/spec`, 'm'), + // An access through the documented `import * as Data` namespace. + new RegExp(`\\bData\\.${RETIRED_NAMES}\\b`, 'm'), + ]; + + /** + * Inline code spans are prose — the house style `check:doc-authoring` enforces + * — so stripping single-backtick spans separates "the retirement kit naming + * what it removed" from "a source still using it". Newline-bounded: a fenced + * block's content is NOT stripped. + */ + const stripInlineCode = (text: string): string => text.replace(/`[^`\n]*`/g, ''); + const judge = (text: string): RegExpExecArray | null => { + const stripped = stripInlineCode(text); + for (const re of USES) { + const m = re.exec(stripped); + if (m) return m; + } + return null; + }; + + /** + * Structural exclusions — the retirement kit, each with its reason. ⛔ NOT an + * allowlist file (`spec-property-retirement` §4). + */ + const EXCLUDED = new Set([ + // This pin spells the retired names to assert their absence. + THIS_FILE, + ]); + const EXCLUDED_PREFIXES = [ + // Release-owned prose records the removal; never edited by a code PR. + 'content/docs/releases/', + // GITIGNORED build output (`packages/spec/json-schema/`), reached only + // because this is a FILESYSTEM walk. Its source is the Zod tree. + 'packages/spec/json-schema/', + ]; + /** tsup's own bundle of `tsup.config.ts`, written and deleted mid-build. */ + const TSUP_BUNDLED_CONFIG = /\.bundled_[^./]+\.mjs$/; + + /** Tolerates ONLY a path that vanished mid-walk; every other read fault is re-raised. */ + const readIfPresent = (full: string): string | undefined => { + try { + return fs.readFileSync(full, 'utf-8'); + } catch (err) { + if ((err as NodeJS.ErrnoException)?.code !== 'ENOENT') throw err; + return undefined; + } + }; + + it('the matcher recognises a use and ignores a prose mention and the neighbours (anti-vacuity)', () => { + // Uses — in each syntax the walk reads. + expect(judge("import { DataEngineRequestSchema } from '@objectstack/spec/data';")).not.toBeNull(); + expect(judge("import type {\n EngineQueryOptions,\n DataEngineRequest,\n} from '@objectstack/spec/data';")).not.toBeNull(); + expect(judge("Prose.\n\n```ts\nimport { DataEngineVectorFindRequestSchema } from '@objectstack/spec';\n```\n")).not.toBeNull(); + expect(judge("export { DataEngineContractSchema } from '@objectstack/spec/data';")).not.toBeNull(); + expect(judge('const r = Data.DataEngineFindOneRequestSchema.parse(x);')).not.toBeNull(); + // Neighbours that must NOT match. + expect(judge('the `DataEngineRequestSchema` export was retired')).toBeNull(); + expect(judge("import { DataEngineInsertOptionsSchema, DataEngineQueryOptions } from '@objectstack/spec/data';")).toBeNull(); + expect(judge("import { CrossObjectBatchRequestSchema } from '@objectstack/spec/api';")).toBeNull(); + expect(judge("import { DataEngineRequestSchema } from './local-protocol';")).toBeNull(); + }); + + it('no use of a retired export survives inside the declared radius outside the retirement kit', () => { + const offenders: string[] = []; + let visited = 0; + let exampleSources = 0; + const walk = (dir: string) => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + const rel = path.relative(REPO_ROOT, full).split(path.sep).join('/'); + if (entry.isDirectory()) { + if (SKIPPED_DIRS.has(entry.name) || entry.name.startsWith('.')) continue; + walk(full); + continue; + } + if (!entry.isFile()) continue; + const ext = path.extname(entry.name); + const scanned = rel.startsWith('examples/') + ? EXAMPLES_EXT.has(ext) || EXAMPLE_APP_SRC_TS.test(rel) + : SCANNED_EXT.has(ext); + if (!scanned) continue; + if (entry.name === 'CHANGELOG.md') continue; // release prose records the removal + if (EXCLUDED.has(rel) || EXCLUDED_PREFIXES.some((p) => rel.startsWith(p))) continue; + if (TSUP_BUNDLED_CONFIG.test(entry.name)) continue; + visited += 1; + if (EXAMPLE_APP_SRC_TS.test(rel)) exampleSources += 1; + const text = readIfPresent(full); + if (text === undefined) continue; + const m = judge(text); + if (m) offenders.push(`${rel} uses \`${m[0].trim().replace(/\s+/g, ' ')}\``); + } + }; + for (const root of WALK_ROOTS) walk(path.join(REPO_ROOT, root)); + // Anti-vacuity: the walk really covered the tree, and the example apps' + // sources were really read. + expect(visited).toBeGreaterThan(1000); + expect(exampleSources).toBeGreaterThan(50); + expect(offenders, 'a use of a retired Virtual ObjectQL export means the retirement is being undone').toEqual([]); + }); +}); diff --git a/packages/spec/src/data/data-engine.test.ts b/packages/spec/src/data/data-engine.test.ts index 6ad303fbb6e..acfe0db442a 100644 --- a/packages/spec/src/data/data-engine.test.ts +++ b/packages/spec/src/data/data-engine.test.ts @@ -14,16 +14,6 @@ import { EngineDeleteOptionsSchema, EngineAggregateOptionsSchema, EngineCountOptionsSchema, - DataEngineFindRequestSchema, - DataEngineFindOneRequestSchema, - DataEngineInsertRequestSchema, - DataEngineUpdateRequestSchema, - DataEngineDeleteRequestSchema, - DataEngineCountRequestSchema, - DataEngineAggregateRequestSchema, - DataEngineExecuteRequestSchema, - DataEngineVectorFindRequestSchema, - DataEngineRequestSchema, DroppedFieldsEventSchema, QueryTransportParamsSchema, } from './data-engine.zod'; @@ -615,436 +605,13 @@ describe('EngineCountOptionsSchema', () => { }); }); -describe('DataEngineFindRequestSchema', () => { - it('should accept minimal find request', () => { - const request = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - }); - - expect(request.method).toBe('find'); - expect(request.object).toBe('account'); - }); - - it('should accept find with query', () => { - const request = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - query: { - where: { status: 'active' }, - limit: 10, - }, - }); - - expect(request.query?.where).toBeDefined(); - }); - - it('folds legacy params into their canonical keys and drops the aliases', () => { - const request = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - query: { - filter: { status: 'active' }, - select: ['id', 'name'], - sort: [{ field: 'name', order: 'asc' }], - skip: 10, - populate: ['owner'], - top: 5, - }, - }); - - // Legacy input stays accepted (backward compat) — but the parsed output - // speaks canonical QueryAST only, so no consumer re-implements precedence. - expect(request.query?.where).toEqual({ status: 'active' }); - expect(request.query?.fields).toEqual(['id', 'name']); - expect(request.query?.orderBy).toEqual([{ field: 'name', order: 'asc' }]); - expect(request.query?.offset).toBe(10); - expect(request.query?.expand).toEqual({ owner: { object: 'owner' } }); - expect(request.query?.limit).toBe(5); - for (const alias of ['filter', 'select', 'sort', 'skip', 'populate', 'top']) { - expect(request.query && alias in request.query).toBe(false); - } - }); - - it('lowers the record sort spellings while folding', () => { - const named = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - query: { sort: { name: 'desc' } }, - }); - expect(named.query?.orderBy).toEqual([{ field: 'name', order: 'desc' }]); - - const numeric = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - query: { sort: { created_at: 1, name: -1 } }, - }); - expect(numeric.query?.orderBy).toEqual([ - { field: 'created_at', order: 'asc' }, - { field: 'name', order: 'desc' }, - ]); - }); - - it('tolerates a redundant identical spelling, refuses a conflicting one', () => { - const redundant = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - query: { fields: ['id'], select: ['id'] }, - }); - expect(redundant.query?.fields).toEqual(['id']); - expect(redundant.query && 'select' in redundant.query).toBe(false); - - const conflict = DataEngineFindRequestSchema.safeParse({ - method: 'find', - object: 'account', - query: { fields: ['id'], select: ['name'] }, - }); - expect(conflict.success).toBe(false); - if (!conflict.success) { - expect(conflict.error.issues[0].message).toMatch(/Send exactly one/); - } - }); - - it('refuses a conflicting value on every alias pair', () => { - const cases: Array> = [ - { filter: { a: 1 }, where: { b: 2 } }, - { select: ['a'], fields: ['b'] }, - { sort: [{ field: 'a', order: 'asc' }], orderBy: [{ field: 'b', order: 'asc' }] }, - { skip: 1, offset: 2 }, - { populate: ['a'], expand: { b: { object: 'b' } } }, - { top: 1, limit: 3 }, - ]; - for (const query of cases) { - const result = DataEngineFindRequestSchema.safeParse({ method: 'find', object: 'account', query }); - expect(result.success, `${Object.keys(query).join('+')} must be refused`).toBe(false); - } - }); -}); - -describe('DataEngineFindOneRequestSchema', () => { - it('should accept find one request', () => { - const request = DataEngineFindOneRequestSchema.parse({ - method: 'findOne', - object: 'account', - query: { - where: { id: '123' }, - }, - }); - - expect(request.method).toBe('findOne'); - }); - - it('folds legacy params into canonical keys (backward compat input)', () => { - const request = DataEngineFindOneRequestSchema.parse({ - method: 'findOne', - object: 'account', - query: { - filter: { id: '123' }, - select: ['id', 'name'], - }, - }); - - expect(request.query?.where).toEqual({ id: '123' }); - expect(request.query?.fields).toEqual(['id', 'name']); - expect(request.query && 'filter' in request.query).toBe(false); - }); -}); - -describe('DataEngineInsertRequestSchema', () => { - it('should accept single record insert', () => { - const request = DataEngineInsertRequestSchema.parse({ - method: 'insert', - object: 'account', - data: { name: 'Test Account' }, - }); - - expect(request.method).toBe('insert'); - expect(request.data.name).toBe('Test Account'); - }); - - it('should accept multiple records insert', () => { - const request = DataEngineInsertRequestSchema.parse({ - method: 'insert', - object: 'account', - data: [ - { name: 'Account 1' }, - { name: 'Account 2' }, - ], - }); - - expect(request.data).toHaveLength(2); - }); - - it('should accept insert with options', () => { - const request = DataEngineInsertRequestSchema.parse({ - method: 'insert', - object: 'account', - data: { name: 'Test' }, - options: { - returning: true, - }, - }); - - expect(request.options?.returning).toBe(true); - }); -}); - -describe('DataEngineUpdateRequestSchema', () => { - it('should accept update with id', () => { - const request = DataEngineUpdateRequestSchema.parse({ - method: 'update', - object: 'account', - id: '123', - data: { status: 'active' }, - }); - - expect(request.id).toBe('123'); - }); - - it('folds a legacy filter into where (backward compat input)', () => { - const request = DataEngineUpdateRequestSchema.parse({ - method: 'update', - object: 'account', - data: { status: 'active' }, - options: { - filter: { category: 'premium' }, - multi: true, - }, - }); - - expect(request.options?.where).toEqual({ category: 'premium' }); - expect(request.options && 'filter' in request.options).toBe(false); - expect(request.options?.multi).toBe(true); - }); - - it('should accept update with new where (preferred)', () => { - const request = DataEngineUpdateRequestSchema.parse({ - method: 'update', - object: 'account', - data: { status: 'active' }, - options: { - where: { category: 'premium' }, - multi: true, - }, - }); - - expect(request.options?.where).toEqual({ category: 'premium' }); - expect(request.options?.multi).toBe(true); - }); -}); - -describe('DataEngineDeleteRequestSchema', () => { - it('should accept delete with id', () => { - const request = DataEngineDeleteRequestSchema.parse({ - method: 'delete', - object: 'account', - id: '123', - }); - - expect(request.id).toBe('123'); - }); - - it('folds a legacy filter into where (backward compat input)', () => { - const request = DataEngineDeleteRequestSchema.parse({ - method: 'delete', - object: 'account', - options: { - filter: { status: 'archived' }, - multi: true, - }, - }); - - expect(request.options?.where).toEqual({ status: 'archived' }); - expect(request.options && 'filter' in request.options).toBe(false); - expect(request.options?.multi).toBe(true); - }); - - it('should accept delete with new where (preferred)', () => { - const request = DataEngineDeleteRequestSchema.parse({ - method: 'delete', - object: 'account', - options: { - where: { status: 'archived' }, - multi: true, - }, - }); - - expect(request.options?.where).toEqual({ status: 'archived' }); - expect(request.options?.multi).toBe(true); - }); -}); - -describe('DataEngineCountRequestSchema', () => { - it('should accept count request', () => { - const request = DataEngineCountRequestSchema.parse({ - method: 'count', - object: 'account', - }); - - expect(request.method).toBe('count'); - }); - - it('should accept count with where (preferred)', () => { - const request = DataEngineCountRequestSchema.parse({ - method: 'count', - object: 'account', - query: { - where: { status: 'active' }, - }, - }); - - expect(request.query?.where).toBeDefined(); - }); - - it('folds a legacy filter into where (backward compat input)', () => { - const request = DataEngineCountRequestSchema.parse({ - method: 'count', - object: 'account', - query: { - filter: { status: 'active' }, - }, - }); - - expect(request.query?.where).toEqual({ status: 'active' }); - expect(request.query && 'filter' in request.query).toBe(false); - }); - - it('refuses filter + where with different values', () => { - const result = DataEngineCountRequestSchema.safeParse({ - method: 'count', - object: 'account', - query: { filter: { a: 1 }, where: { b: 2 } }, - }); - expect(result.success).toBe(false); - }); -}); - -describe('DataEngineAggregateRequestSchema', () => { - it('should accept aggregate request', () => { - const request = DataEngineAggregateRequestSchema.parse({ - method: 'aggregate', - object: 'account', - query: { - groupBy: ['status'], - aggregations: [ - { function: 'sum', field: 'revenue', alias: 'total_revenue' }, - ], - }, - }); - - expect(request.method).toBe('aggregate'); - }); -}); - -describe('DataEngineExecuteRequestSchema', () => { - it('should accept execute with SQL command', () => { - const request = DataEngineExecuteRequestSchema.parse({ - method: 'execute', - command: 'SELECT * FROM accounts WHERE status = ?', - }); - - expect(request.method).toBe('execute'); - }); - - it('should accept execute with object command', () => { - const request = DataEngineExecuteRequestSchema.parse({ - method: 'execute', - command: { - operation: 'custom_query', - params: { limit: 10 }, - }, - }); - - expect(request.command.operation).toBe('custom_query'); - }); - - it('should accept execute with options', () => { - const request = DataEngineExecuteRequestSchema.parse({ - method: 'execute', - command: 'CUSTOM QUERY', - options: { - timeout: 5000, - }, - }); - - expect(request.options?.timeout).toBe(5000); - }); -}); - -describe('DataEngineVectorFindRequestSchema', () => { - it('should accept vector find request', () => { - const request = DataEngineVectorFindRequestSchema.parse({ - method: 'vectorFind', - object: 'documents', - vector: [0.1, 0.2, 0.3, 0.4], - }); - - expect(request.method).toBe('vectorFind'); - expect(request.vector).toHaveLength(4); - }); - - it('should accept custom limit', () => { - const request = DataEngineVectorFindRequestSchema.parse({ - method: 'vectorFind', - object: 'documents', - vector: [0.1, 0.2, 0.3], - limit: 10, - }); - - expect(request.limit).toBe(10); - }); - - it('should accept vector find with filter', () => { - const request = DataEngineVectorFindRequestSchema.parse({ - method: 'vectorFind', - object: 'documents', - vector: [0.1, 0.2, 0.3], - where: { category: 'tech' }, - limit: 10, - threshold: 0.8, - }); - - expect(request.where).toBeDefined(); - expect(request.limit).toBe(10); - expect(request.threshold).toBe(0.8); - }); - - it('should accept vector find with fields', () => { - const request = DataEngineVectorFindRequestSchema.parse({ - method: 'vectorFind', - object: 'documents', - vector: [0.1, 0.2], - fields: ['id', 'title', 'content'], - }); - - expect(request.fields).toHaveLength(3); - }); -}); - // The `DataEngineBatchRequestSchema` suite that stood here was removed with -// the schema itself (ADR-0119 D3, #4618). Every case parsed a literal through -// a schema no production code ever called, so it proved the schema matched -// itself and nothing more. - -describe('DataEngineRequestSchema', () => { - it('should accept all request types', () => { - const requests = [ - { method: 'find' as const, object: 'account' }, - { method: 'findOne' as const, object: 'account' }, - { method: 'insert' as const, object: 'account', data: {} }, - { method: 'update' as const, object: 'account', data: {} }, - { method: 'delete' as const, object: 'account' }, - { method: 'count' as const, object: 'account' }, - { method: 'aggregate' as const, object: 'account', query: {} }, - { method: 'execute' as const, command: 'SQL' }, - { method: 'vectorFind' as const, object: 'docs', vector: [0.1] }, - ]; - - requests.forEach(request => { - expect(() => DataEngineRequestSchema.parse(request)).not.toThrow(); - }); - }); -}); +// the schema itself (ADR-0119 D3, #4618), and the suites for the rest of the +// Virtual ObjectQL request protocol (`DataEngineRequestSchema` and its nine +// arms) went the same way when #27028 retired it. Every case parsed a literal +// through a schema no production code ever called, so it proved the schema +// matched itself and nothing more. The absence pin is +// `data-engine-rpc-protocol-retirement.test.ts`. describe('Integration Tests', () => { it('should support context in query options', () => { @@ -1122,79 +689,6 @@ describe('Integration Tests', () => { expect(countOpts.context).toBeUndefined(); expect(aggOpts.context).toBeUndefined(); }); - - it('should support complete CRUD workflow', () => { - // Create - const insertRequest = DataEngineInsertRequestSchema.parse({ - method: 'insert', - object: 'account', - data: { name: 'Test Account', status: 'active' }, - options: { returning: true }, - }); - - // Read - const findRequest = DataEngineFindRequestSchema.parse({ - method: 'find', - object: 'account', - query: { - where: { status: 'active' }, - fields: ['id', 'name', 'status'], - orderBy: [{ field: 'name', order: 'asc' }], - limit: 10, - }, - }); - - // Update - const updateRequest = DataEngineUpdateRequestSchema.parse({ - method: 'update', - object: 'account', - id: '123', - data: { status: 'inactive' }, - options: { returning: true }, - }); - - // Delete - const deleteRequest = DataEngineDeleteRequestSchema.parse({ - method: 'delete', - object: 'account', - id: '123', - }); - - expect(insertRequest.method).toBe('insert'); - expect(findRequest.method).toBe('find'); - expect(updateRequest.method).toBe('update'); - expect(deleteRequest.method).toBe('delete'); - }); - - it('should support analytics workflow', () => { - // Count - const countRequest = DataEngineCountRequestSchema.parse({ - method: 'count', - object: 'opportunity', - query: { - where: { stage: 'closed_won' }, - }, - }); - - // Aggregate - const aggregateRequest = DataEngineAggregateRequestSchema.parse({ - method: 'aggregate', - object: 'opportunity', - query: { - where: { status: 'closed' }, - groupBy: ['stage', 'owner_id'], - aggregations: [ - { function: 'sum', field: 'amount', alias: 'total_amount' }, - { function: 'avg', field: 'amount', alias: 'avg_amount' }, - { function: 'count', field: 'id', alias: 'deal_count' }, - ], - }, - }); - - expect(countRequest.method).toBe('count'); - expect(aggregateRequest.query.aggregations).toHaveLength(3); - }); - }); describe('DroppedFieldsEventSchema.reason — why a write dropped submitted fields, widened past the readonly pair', () => { @@ -1246,13 +740,13 @@ describe('DroppedFieldsEventSchema.reason — why a write dropped submitted fiel describe('the engine-side projection declarations admit exactly what their transport siblings admit', () => { /** - * [#26954] `EngineQueryOptionsSchema.fields` and the RPC request's - * deprecated `select` (`DataEngineFindRequestSchema.query`, and so - * `DataEngineFindOneRequestSchema` and the `DataEngineRequestSchema` union) - * take the projection's array arm from the one declaration their transport - * siblings use: at least one name, none blank. `fields` shares - * `FieldNodeSchema` with `$select`; `select` shares `z.string()` with the - * bare `select`. So each refusal reads exactly as its sibling's does. + * [#26954] `EngineQueryOptionsSchema.fields` and the legacy + * `DataEngineQueryOptionsSchema.select` take the projection's array arm from + * the one declaration their transport siblings use: at least one name, none + * blank. `fields` shares `FieldNodeSchema` with `$select`; `select` shares + * `z.string()` with the bare `select`. So each refusal reads exactly as its + * sibling's does. (The Virtual ObjectQL request arms' rows left with that + * protocol, #27028.) */ const REFUSED: ReadonlyArray]> = [ ['an empty list', [], []], @@ -1285,35 +779,8 @@ describe('the engine-side projection declarations admit exactly what their trans expect(EngineQueryOptionsSchema.parse({ fields: value }).fields).toEqual(value); }); - for (const [label, schema, method] of [ - ['DataEngineFindRequestSchema', DataEngineFindRequestSchema, 'find'], - ['DataEngineFindOneRequestSchema', DataEngineFindOneRequestSchema, 'findOne'], - ['DataEngineRequestSchema', DataEngineRequestSchema, 'find'], - ] as const) { - it.each(REFUSED)(`${label}: query.select refuses %s, naming the path, as the bare \`select\` does`, (_name, value, tail) => { - const parsed = schema.safeParse({ method, object: 'account', query: { select: value } }); - expect(parsed.success).toBe(false); - const issue = parsed.success ? undefined : parsed.error.issues[0]; - expect(issue?.path).toEqual(['query', 'select', ...tail]); - expect(issue?.message).toBe(sibling('select', value)?.message); - }); - - it.each(REFUSED)(`${label}: query.fields refuses %s, naming the path`, (_name, value, tail) => { - const parsed = schema.safeParse({ method, object: 'account', query: { fields: value } }); - expect(parsed.success).toBe(false); - const issue = parsed.success ? undefined : parsed.error.issues[0]; - expect(issue?.path).toEqual(['query', 'fields', ...tail]); - expect(issue?.message).toBe(sibling('$select', value)?.message); - }); - - it.each(KEPT)(`${label}: query.select keeps %s, folded onto \`fields\``, (_name, value) => { - const parsed = schema.parse({ method, object: 'account', query: { select: value } }) as { query?: Record }; - expect(parsed.query).toEqual({ fields: value }); - }); - } - - // The legacy options bag's `select` and the vector-find request's `fields` - // are `z.string()` lists, so their sibling is the bare `select`. + // The legacy options bag's `select` is a `z.string()` list, so its sibling + // is the bare `select`. it.each(REFUSED)('DataEngineQueryOptionsSchema.select refuses %s, naming the path, as the bare `select` does', (_name, value, tail) => { const parsed = DataEngineQueryOptionsSchema.safeParse({ select: value }); expect(parsed.success).toBe(false); @@ -1325,22 +792,4 @@ describe('the engine-side projection declarations admit exactly what their trans it.each(KEPT)('DataEngineQueryOptionsSchema.select keeps %s, unchanged', (_name, value) => { expect(DataEngineQueryOptionsSchema.parse({ select: value }).select).toEqual(value); }); - - for (const [label, schema] of [ - ['DataEngineVectorFindRequestSchema', DataEngineVectorFindRequestSchema], - ['DataEngineRequestSchema', DataEngineRequestSchema], - ] as const) { - it.each(REFUSED)(`${label}: vectorFind fields refuses %s, naming the path, as the bare \`select\` does`, (_name, value, tail) => { - const parsed = schema.safeParse({ method: 'vectorFind', object: 'doc', vector: [0.1, 0.2], fields: value }); - expect(parsed.success).toBe(false); - const issue = parsed.success ? undefined : parsed.error.issues[0]; - expect(issue?.path).toEqual(['fields', ...tail]); - expect(issue?.message).toBe(sibling('select', value)?.message); - }); - - it.each(KEPT)(`${label}: vectorFind fields keeps %s, unchanged`, (_name, value) => { - const parsed = schema.parse({ method: 'vectorFind', object: 'doc', vector: [0.1, 0.2], fields: value }) as { fields?: string[] }; - expect(parsed.fields).toEqual(value); - }); - } }); diff --git a/packages/spec/src/data/data-engine.zod.ts b/packages/spec/src/data/data-engine.zod.ts index 81651f409bf..cd3547b66cf 100644 --- a/packages/spec/src/data/data-engine.zod.ts +++ b/packages/spec/src/data/data-engine.zod.ts @@ -97,12 +97,10 @@ export const BaseEngineOptionsSchema = lazySchema(() => z.object({ * rule cannot drift apart. * * [#26954] The engine-side declarations of the same projection — - * `EngineQueryOptionsSchema.fields`, `RpcQueryOptionsSchema.select`, the - * legacy `DataEngineQueryOptionsSchema.select` and - * `DataEngineVectorFindRequestSchema.fields` — take the array arm from here - * too. It is declared ahead of both because - * `RpcQueryOptionsSchema` extends `EngineQueryOptionsSchema` at module load, - * which runs the lazy factory before any later `const` exists. + * `EngineQueryOptionsSchema.fields` and the legacy + * `DataEngineQueryOptionsSchema.select` — take the array arm from here too. + * (So did the Virtual ObjectQL request arms' `select` / `fields` until #27028 + * retired that block; see section 9.) * * [#27166] The array arm and its sentence live in * `./projection-names-a-field.ts`, because `QuerySchema.fields` (and so every @@ -309,8 +307,8 @@ export const EngineUpdateOptionsSchema = lazySchema(() => BaseEngineOptionsSchem * Delivered in-process via the `onFieldsDropped` listener on the write options * (see `WriteObservabilityOptions` in `contracts/data-engine.ts`). The * listener itself is deliberately NOT part of this serializable options - * schema: a function is unrepresentable in JSON Schema and cannot cross the - * RPC (Virtual Data Engine) boundary. + * schema: a function is unrepresentable in JSON Schema and cannot be + * serialized. */ export const DroppedFieldsEventSchema = lazySchema(() => z.object({ /** Object the write targeted (resolved object name). */ @@ -574,53 +572,21 @@ export const DataEngineCountOptionsSchema = lazySchema(() => BaseEngineOptionsSc }).describe('Options for DataEngine.count operations')); // ========================================================================== -// 8. Definition (Contract) +// 8. Definition (Contract) — retired // ========================================================================== - -export const DataEngineContractSchema = lazySchema(() => z.object({ - find: z.function() - .input(z.tuple([z.string(), EngineQueryOptionsSchema.optional()])) - .output(z.promise(z.array(z.unknown()))), - - findOne: z.function() - .input(z.tuple([z.string(), EngineQueryOptionsSchema.optional()])) - .output(z.promise(z.unknown())), - - insert: z.function() - .input(z.tuple([z.string(), z.union([z.record(z.string(), z.unknown()), z.array(z.record(z.string(), z.unknown()))]), DataEngineInsertOptionsSchema.optional()])) - .output(z.promise(z.unknown())), - - update: z.function() - .input(z.tuple([z.string(), z.record(z.string(), z.unknown()), EngineUpdateOptionsSchema.optional()])) - .output(z.promise(z.unknown())), - - delete: z.function() - .input(z.tuple([z.string(), EngineDeleteOptionsSchema.optional()])) - .output(z.promise(z.unknown())), - - count: z.function() - .input(z.tuple([z.string(), EngineCountOptionsSchema.optional()])) - .output(z.promise(z.number())), - - aggregate: z.function() - .input(z.tuple([z.string(), EngineAggregateOptionsSchema])) - .output(z.promise(z.array(z.unknown()))) -}).describe('Standard Data Engine Contract')); +// +// `DataEngineContractSchema` stood here until #27028 retired it with the +// Virtual ObjectQL request protocol (section 9). It restated the engine's +// methods as `z.function()` members, so no JSON Schema was ever emitted for it, +// nothing parsed it, and it had drifted from the contract it described: its +// reads took no trailing options argument and it declared neither `vectorFind` +// nor `execute`. The contract is `IDataEngine` in `../contracts/data-engine.ts`, +// a TypeScript interface the compiler holds every engine to. // ========================================================================== -// 9. Virtualization & RPC Protocol +// 9. Query alias slots // ========================================================================== -/** - * Data Engine RPC Request (Virtual ObjectQL) - * - * This schema defines the serialized format for executing Data Engine operations - * via HTTP, Message Queue, or Plugin boundaries. - * - * It enables "Virtual Data Engines" where the implementation resides in a - * separate microservice or plugin. - */ - /** * One RPC query-options slot: the canonical QueryAST key plus the deprecated * alias spellings that fold into it. @@ -645,12 +611,12 @@ export interface QueryAliasConflict { /** * The six alias pairs the RPC query surface accepts (#3795, #4346) — the ONE - * place the alias → canonical mapping is declared. The schema transform below - * folds parsed input by this table, the protocol normalizer - * (`metadata-protocol`) folds raw wire input by the same table, extended with - * the wire-only spellings `filters` / `$filter` / `$expand` that no schema - * declares, and the ObjectQL engine folds the slots its own option bags still - * admit (`where`, `limit`) on every entry point. Before this table existed the + * place the alias → canonical mapping is declared. The protocol normalizer + * (`metadata-protocol`) folds raw wire input by this table, and + * {@link QueryWithTransportSchema} folds parsed input by it, both extended with + * the transport-only spellings `filters` / `$filter` / `$expand` + * ({@link QUERY_TRANSPORT_ALIAS_SLOTS}); the ObjectQL engine folds the slots + * its own option bags still admit (`where`, `limit`) on every entry point. Before this table existed the * precedence lived in prose only, so every reader re-implemented it — the * #3713 condition — and the two readers disagreed on three of the five pairs, * four of them backwards. @@ -724,11 +690,6 @@ export function foldQueryAliasSlots( return arrivedAs; } -/** The `where` slot alone — for RPC options that accept only the `filter` alias. */ -const RPC_WHERE_SLOT: readonly QueryAliasSlot[] = RPC_QUERY_ALIAS_SLOTS.filter( - (slot) => slot.canonical === 'where', -); - function aliasConflictIssue(conflict: QueryAliasConflict): { code: 'custom'; path: string[]; @@ -744,84 +705,6 @@ function aliasConflictIssue(conflict: QueryAliasConflict): { }; } -/** - * RPC backward-compatibility mixin — shared `@deprecated filter` field. - * The parse transform folds `filter` into `where` and drops it; both spellings - * with different values fail the parse (see `RpcQueryOptionsSchema`). - */ -const RpcLegacyFilterMixin = { - /** @deprecated Use `where` */ - filter: DataEngineFilterSchema.optional(), -}; - -/** - * Parse-time fold for options that accept only the `filter` alias - * ({@link RpcLegacyFilterMixin}): `filter` lands on `where` and is dropped - * from the parsed output, so `filter` is absent from the inferred type and a - * TS consumer reading it fails to compile instead of silently reading - * `undefined` (the #3742 / #3764 shape, one layer down). - */ -function foldRpcLegacyFilter( - input: T, - ctx: z.core.$RefinementCtx, -): Omit { - const bag: Record = { ...input }; - foldQueryAliasSlots(bag, RPC_WHERE_SLOT, (conflict) => ctx.addIssue(aliasConflictIssue(conflict))); - return bag as Omit; -} - -/** - * Parse-time fold for the full RPC query options: each legacy alias lands on - * its canonical key (with the alias's legacy value shape lowered to the - * canonical one) and is dropped from the parsed output. - */ -function foldRpcQueryOptions(input: object, ctx: z.core.$RefinementCtx): EngineQueryOptions { - const bag = { ...(input as Record) }; - foldQueryAliasSlots(bag, RPC_QUERY_ALIAS_SLOTS, (conflict) => ctx.addIssue(aliasConflictIssue(conflict))); - // A folded `sort` may carry the record spellings `DataEngineSortSchema` - // allows; canonical `orderBy` declares `SortNode[]` only, so lower them. - if (bag.orderBy !== undefined && bag.orderBy !== null && !Array.isArray(bag.orderBy)) { - bag.orderBy = Object.entries(bag.orderBy as Record).map( - ([field, order]) => ({ field, order: order === 'asc' || order === 1 ? 'asc' : 'desc' }), - ); - } - // A folded `populate` is a relation-name list; canonical `expand` is a - // `{name: QueryAST}` record. - if (Array.isArray(bag.expand)) { - bag.expand = Object.fromEntries( - (bag.expand as string[]).map((rel) => [rel, { object: rel }]), - ); - } - return bag as EngineQueryOptions; -} - -/** - * RPC query options that accept BOTH new (where/fields/orderBy) and - * legacy (filter/select/sort/skip/populate) parameter names. - * - * **One slot, one value (#3795):** each legacy alias is folded into its - * canonical key at parse — `filter`→`where`, `select`→`fields`, - * `sort`→`orderBy`, `skip`→`offset`, `populate`→`expand`, `top`→`limit` - * ({@link RPC_QUERY_ALIAS_SLOTS}) — and the alias is dropped from the parsed - * output, so consumers only ever read canonical QueryAST keys. Sending both - * spellings with the SAME value is redundant and tolerated; sending DIFFERENT - * values for one slot is irreconcilable — picking either would silently drop - * the other (#4181) — and fails the parse. The protocol normalizer applies - * the same table to raw wire input, so mixed vocabularies resolve identically - * on every path. - */ -const RpcQueryOptionsSchema = EngineQueryOptionsSchema.extend({ - ...RpcLegacyFilterMixin, - /** @deprecated Use `fields` — the same projection rule (#26954) */ - select: transportProjectionArray(z.string()).optional(), - /** @deprecated Use `orderBy` */ - sort: DataEngineSortSchema.optional(), - /** @deprecated Use `offset` */ - skip: z.number().int().min(0).optional(), - /** @deprecated Use `expand` */ - populate: z.array(z.string()).optional(), -}).transform((options, ctx) => foldRpcQueryOptions(options, ctx)); - // ========================================================================== // 2c. The QUERY TRANSPORT dialect — the FLATTENED SPELLING of the QueryAST // ========================================================================== @@ -1112,9 +995,8 @@ export const QueryTransportParamsSchema = lazySchema(() => z.object({ * canonical slot via {@link foldQueryAliasSlots}. * * Deliberately a fold over the shared TABLES rather than a second copy of the - * normalizer: this is the parse-time application, exactly as - * `foldRpcQueryOptions` is the parse-time application of - * {@link RPC_QUERY_ALIAS_SLOTS}. + * normalizer: this is the parse-time application of the tables the + * normalizer applies at the wire. * * A CONFLICT — two spellings of one slot carrying different values — fails the * parse with {@link aliasConflictIssue} at the canonical path, quoting the @@ -1451,125 +1333,29 @@ export const QueryWithTransportSchema = lazySchema( */ export type QueryWithTransportParsed = z.infer; -export const DataEngineFindRequestSchema = lazySchema(() => z.object({ - method: z.literal('find'), - object: z.string(), - query: RpcQueryOptionsSchema.optional() -})); - -export const DataEngineFindOneRequestSchema = lazySchema(() => z.object({ - method: z.literal('findOne'), - object: z.string(), - query: RpcQueryOptionsSchema.optional() -})); - -export const DataEngineInsertRequestSchema = lazySchema(() => z.object({ - method: z.literal('insert'), - object: z.string(), - data: z.union([z.record(z.string(), z.unknown()), z.array(z.record(z.string(), z.unknown()))]), - options: DataEngineInsertOptionsSchema.optional() -})); - -export const DataEngineUpdateRequestSchema = lazySchema(() => z.object({ - method: z.literal('update'), - object: z.string(), - data: z.record(z.string(), z.unknown()), - id: z.union([z.string(), z.number()]).optional().describe('ID for single update, or use where in options'), - options: EngineUpdateOptionsSchema.extend(RpcLegacyFilterMixin) - .transform((options, ctx) => foldRpcLegacyFilter(options, ctx)).optional() -})); - -export const DataEngineDeleteRequestSchema = lazySchema(() => z.object({ - method: z.literal('delete'), - object: z.string(), - id: z.union([z.string(), z.number()]).optional().describe('ID for single delete, or use where in options'), - options: EngineDeleteOptionsSchema.extend(RpcLegacyFilterMixin) - .transform((options, ctx) => foldRpcLegacyFilter(options, ctx)).optional() -})); - -export const DataEngineCountRequestSchema = lazySchema(() => z.object({ - method: z.literal('count'), - object: z.string(), - query: EngineCountOptionsSchema.extend(RpcLegacyFilterMixin) - .transform((options, ctx) => foldRpcLegacyFilter(options, ctx)).optional() -})); - -export const DataEngineAggregateRequestSchema = lazySchema(() => z.object({ - method: z.literal('aggregate'), - object: z.string(), - query: EngineAggregateOptionsSchema.extend(RpcLegacyFilterMixin) - .transform((options, ctx) => foldRpcLegacyFilter(options, ctx)) -})); - -/** - * Data Engine Execute Request (Raw Command) - * Execute a raw command/query native to the driver (e.g. SQL, Shell, Remote API). - */ -export const DataEngineExecuteRequestSchema = lazySchema(() => z.object({ - method: z.literal('execute'), - /** The abstract command (string SQL, or JSON object) */ - command: z.unknown(), - /** Optional options */ - options: z.record(z.string(), z.unknown()).optional() -})); - -/** - * Data Engine Vector Find Request (AI/RAG) - * Perform a similarity search using vector embeddings. - */ -export const DataEngineVectorFindRequestSchema = lazySchema(() => z.object({ - method: z.literal('vectorFind'), - object: z.string(), - /** The vector embedding to search for */ - vector: z.array(z.number()), - /** Optional pre-filter (Metadata filtering) — standard QueryAST `where` */ - where: z.union([z.record(z.string(), z.unknown()), FilterConditionSchema]).optional(), - /** Fields to retrieve — standard QueryAST `fields`; the same projection rule (#26954) */ - fields: transportProjectionArray(z.string()).optional(), - /** Number of results */ - limit: z.number().int().default(5).optional(), - /** Minimum similarity score (0-1) or distance threshold */ - threshold: z.number().optional() -})); - -// `DataEngineBatchRequestSchema` stood here until ADR-0119 D3 (#4618) retired -// it with the `IDataEngine.batch?` member it existed to describe. Nothing ever -// parsed it: no engine implemented `batch`, no caller invoked it, and the -// wire-side batch route validates with `CrossObjectBatchRequestSchema` / -// `BatchUpdateRequestSchema` from `../api/batch.zod.ts` — a different schema -// entirely. Deleted outright rather than tombstoned with `retiredKey()`, -// because a tombstone's prescription reaches an author through a PARSE, and -// there was no parse to reach: a prescription nobody can receive is noise -// (the spec-property-retirement playbook's third route). Its three -// `authorable-surface.json` baseline lines and its `json-schema.manifest.json` -// entry go with it, deliberately. +// The Virtual ObjectQL request protocol stood here until #27028 retired it: +// `DataEngineRequestSchema`, the `method`-discriminated union, and its nine +// arms (`find`, `findOne`, `insert`, `update`, `delete`, `count`, `aggregate`, +// `execute`, `vectorFind`), the serialized form of an engine call for a +// "Virtual Data Engine" living in another process. No such engine existed and +// nothing parsed the protocol, in this repo or in its sister repos. The one +// real boundary engine calls cross, the QuickJS hook/action sandbox, speaks +// `ctx.api.object(name).find(...)` and parses no request schema; the wire batch +// validates with `CrossObjectBatchRequestSchema` / `BatchUpdateRequestSchema` +// from `../api/batch.zod.ts`. As published the protocol was also unsafe to +// implement: every arm admitted a caller-supplied `context` (`isSystem` +// included), and `execute` carried a raw driver command. Its private helpers +// (`RpcQueryOptionsSchema` and the `filter`-only fold) went with it; +// `RPC_QUERY_ALIAS_SLOTS` above stays, since the transport dialect and the +// engine fold by it. // -// The tell that nobody ever designed against it: its `requests` array nested -// the request union recursively, so a batch could contain batches, with no -// statement anywhere about what that meant for ordering or rollback. - -/** - * Unified Data Engine Request Union - * Use this to validate any incoming "Virtual ObjectQL" request. - * - * NOTE (#4618): every arm below now has zero readers in this repo — there is - * no Virtual Data Engine implementation, only this schema describing one. That - * makes the whole block an ADR-0049 enforce-or-remove candidate, deliberately - * left standing here because retiring a published wire protocol is a different - * decision from retiring `batch?`, and it is tracked separately rather than - * folded into a removal whose title promised something narrower. - */ -export const DataEngineRequestSchema = lazySchema(() => z.discriminatedUnion('method', [ - DataEngineFindRequestSchema, - DataEngineFindOneRequestSchema, - DataEngineInsertRequestSchema, - DataEngineUpdateRequestSchema, - DataEngineDeleteRequestSchema, - DataEngineCountRequestSchema, - DataEngineAggregateRequestSchema, - DataEngineExecuteRequestSchema, - DataEngineVectorFindRequestSchema -]).describe('Virtual ObjectQL Request Protocol')); +// Deleted outright rather than tombstoned with `retiredKey()`, the +// spec-property-retirement playbook's third route: a tombstone's prescription +// reaches an author through a PARSE, and there was no parse to reach. The +// ten JSON Schema defs are registered in `RETIRED_DEFS_BY_MAJOR` and the D3 +// entry `data-engine-rpc-protocol-retired` names the doors that exist. +// `DataEngineBatchRequestSchema` went the same way earlier (ADR-0119 D3, +// #4618), together with the `IDataEngine.batch?` member it described. // ========================================================================== // 10. Type Exports @@ -1634,9 +1420,3 @@ export type DataEngineDeleteOptions = z.input; /** @deprecated Use `EngineCountOptions` instead. */ export type DataEngineCountOptions = z.input; -export type DataEngineRequest = z.input; -/** Post-parse shape of {@link DataEngineRequest} — defaults applied, transforms run (ADR-0122). */ -export type DataEngineRequestParsed = z.infer; -export type DataEngineExecuteRequest = z.input; -export type DataEngineInsertRequest = z.input; -export type DataEngineVectorFindRequest = z.input; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineAggregateRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineAggregateRequest.ts new file mode 100644 index 00000000000..795c24de478 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineAggregateRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineAggregateRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineAggregateRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineCountRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineCountRequest.ts new file mode 100644 index 00000000000..a35dae3ae26 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineCountRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineCountRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineCountRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineDeleteRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineDeleteRequest.ts new file mode 100644 index 00000000000..8d1d5143ea3 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineDeleteRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineDeleteRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineDeleteRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineExecuteRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineExecuteRequest.ts new file mode 100644 index 00000000000..5b53f91523e --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineExecuteRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineExecuteRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineExecuteRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineFindOneRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineFindOneRequest.ts new file mode 100644 index 00000000000..ae839ff2196 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineFindOneRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineFindOneRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineFindOneRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineFindRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineFindRequest.ts new file mode 100644 index 00000000000..7eda21c145b --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineFindRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineFindRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineFindRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineInsertRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineInsertRequest.ts new file mode 100644 index 00000000000..67d21f8607e --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineInsertRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineInsertRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineInsertRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineRequest.ts new file mode 100644 index 00000000000..ffa5a4812f5 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineRequest` — the `method`-discriminated union of the nine request arms, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineUpdateRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineUpdateRequest.ts new file mode 100644 index 00000000000..c5aa5efeeaf --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineUpdateRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineUpdateRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineUpdateRequest'; diff --git a/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineVectorFindRequest.ts b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineVectorFindRequest.ts new file mode 100644 index 00000000000..e0afeddbde8 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/18.data__DataEngineVectorFindRequest.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #27028 — `data/DataEngineVectorFindRequest` — one arm of the `DataEngineRequestSchema` union, the +// Virtual ObjectQL request protocol (`data/data-engine.zod.ts`) — leaves whole +// with that protocol under ADR-0049 enforce-or-remove, by the maintainer's +// ruling A on its decision card (2026-10-28). No Virtual Data Engine existed and +// nothing parsed the protocol, in this repo or in hotcrm, objectui or +// docs.objectos.ai. No carrier key, so no `retiredKey()` tombstone and no D2 +// conversion: RETIRED_DEFS_BY_MAJOR plus the D3 semantic entry +// `data-engine-rpc-protocol-retired` ARE the declaration. +export const entry = 'data/DataEngineVectorFindRequest'; diff --git a/packages/spec/src/migrations/entries/semantic/18.data-engine-rpc-protocol-retired.ts b/packages/spec/src/migrations/entries/semantic/18.data-engine-rpc-protocol-retired.ts new file mode 100644 index 00000000000..8e565ff8511 --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.data-engine-rpc-protocol-retired.ts @@ -0,0 +1,60 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +export const entry: SemanticMigration = { + id: 'data-engine-rpc-protocol-retired', + // No backticks in `surface` — build-upgrade-guide.ts renders it inside a + // code span AND a table cell. + surface: + 'the Virtual ObjectQL request protocol in data/data-engine.zod.ts, retired whole: ' + + 'DataEngineRequestSchema and its nine arms (DataEngineFindRequestSchema, ' + + 'DataEngineFindOneRequestSchema, DataEngineInsertRequestSchema, DataEngineUpdateRequestSchema, ' + + 'DataEngineDeleteRequestSchema, DataEngineCountRequestSchema, DataEngineAggregateRequestSchema, ' + + 'DataEngineExecuteRequestSchema, DataEngineVectorFindRequestSchema), DataEngineContractSchema, ' + + 'the types DataEngineRequest, DataEngineRequestParsed, DataEngineExecuteRequest, ' + + 'DataEngineInsertRequest and DataEngineVectorFindRequest, and the ten published defs ' + + 'data/DataEngineRequest and data/DataEngine{Find,FindOne,Insert,Update,Delete,Count,Aggregate,' + + 'Execute,VectorFind}Request', + replacement: + 'call the engine through the doors that exist. In process, call `IDataEngine` ' + + '(`@objectstack/spec/contracts`) with the `Engine*OptionsSchema` option bags, which stay; a ' + + 'hook or action body uses `ctx.api.object(name).find(...)` and its siblings. Over the wire, ' + + 'use the REST data routes (`/data/:object`, `POST /data/:object/query`) or the metadata ' + + 'protocol behind them, and `POST {basePath}/batch` (`CrossObjectBatchRequestSchema` / ' + + '`BatchUpdateRequestSchema`) for a batch. `DataEngineContractSchema` has no replacement ' + + 'schema: the engine contract is the `IDataEngine` interface, which the compiler holds every ' + + 'engine to', + reason: + 'ADR-0049 enforce-or-remove, by the maintainer\'s ruling A on the decision card for this ' + + 'protocol (2026-10-28): retire it rather than keep it as a planned contract. The block ' + + 'described a serialized engine call for a "Virtual Data Engine" in another process (HTTP, ' + + 'a message queue or a plugin boundary). No such engine existed and nothing parsed the ' + + 'protocol: a census on objectstack `1287b2fa` found no importer or parser outside ' + + '`packages/spec/src/data/` (generated ledgers, reference docs and changelogs aside), and ' + + 'hotcrm `353dace`, objectui `b2c0ade` and docs.objectos.ai `cab35fd` returned zero hits ' + + 'against controls of 311, 2095 and 74 files importing `@objectstack/spec`. The one real ' + + 'boundary engine calls cross, the QuickJS hook and action sandbox, speaks ' + + '`ctx.api.object(name)` and parses no request schema. As published the protocol was also ' + + 'unsafe to implement: every arm admitted a caller-supplied `context` (`isSystem` included) ' + + 'and the `execute` arm carried a raw driver command, so enforcing it would first have ' + + 'meant narrowing it and building a second data door beside REST, batch, MCP and the ' + + 'sandbox. `DataEngineContractSchema` restated the engine methods as `z.function()` ' + + 'members, emitted no JSON Schema, and had drifted from `IDataEngine` (no trailing read ' + + 'options, no `vectorFind`, no `execute`). Why D3 semantic and not a D2 conversion: no ' + + 'metadata source authors these shapes, so a conversion has no seam that runs, and with ' + + 'no parse to reach there is no shape a `retiredKey()` tombstone could sit on (the ' + + '`data-engine-batch-retired` precedent, ADR-0119 D3). Third-party npm consumers are ' + + 'UNMEASURED.', + acceptanceCriteria: + 'No code imports DataEngineRequestSchema, any of its nine arm schemas, ' + + 'DataEngineContractSchema or the types DataEngineRequest, DataEngineRequestParsed, ' + + 'DataEngineExecuteRequest, DataEngineInsertRequest or DataEngineVectorFindRequest from ' + + '@objectstack/spec or @objectstack/spec/data: every such import is TS2305 after upgrade. ' + + 'No tool reads `json-schema/data/DataEngine*Request.json` from the package: the ten defs ' + + 'are absent from `json-schema.manifest/data.json`, the api-surface / declaration-map / ' + + 'export-origins shards and the generated reference docs. `IDataEngine`, every ' + + '`*OptionsSchema`, `RPC_QUERY_ALIAS_SLOTS` and the query-transport schemas are ' + + 'unaffected. ⚠️ Runtime behaviour is deliberately UNCHANGED and must be verified as such: ' + + 'nothing ever parsed these shapes, so removing them removes no behaviour.', +}; diff --git a/packages/spec/src/type-alias-convention.pin.test.ts b/packages/spec/src/type-alias-convention.pin.test.ts index 3dbec506362..10de7763a9a 100644 --- a/packages/spec/src/type-alias-convention.pin.test.ts +++ b/packages/spec/src/type-alias-convention.pin.test.ts @@ -285,7 +285,7 @@ import type * as M194 from './automation/assignment-rule.zod.js'; import type * as M195 from './security/queue.zod.js'; // --------------------------------------------------------------------------- -// 798 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. +// 795 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. // // That number is machine-checked, not hand-kept. The runtime companion at the // bottom of this file recomputes the pin count from the source and asserts that @@ -803,12 +803,9 @@ export type Iso_data_dataEngine__BaseEngineOptionsSchema = Assert, z.infer< typeof M56.DataEngineAggregateOptionsSchema > >>; export type Iso_data_dataEngine__DataEngineCountOptionsSchema = Assert, z.infer< typeof M56.DataEngineCountOptionsSchema > >>; export type Iso_data_dataEngine__DataEngineDeleteOptionsSchema = Assert, z.infer< typeof M56.DataEngineDeleteOptionsSchema > >>; -export type Iso_data_dataEngine__DataEngineExecuteRequestSchema = Assert, z.infer< typeof M56.DataEngineExecuteRequestSchema > >>; export type Iso_data_dataEngine__DataEngineFilterSchema = Assert, z.infer< typeof M56.DataEngineFilterSchema > >>; export type Iso_data_dataEngine__DataEngineInsertOptionsSchema = Assert, z.infer< typeof M56.DataEngineInsertOptionsSchema > >>; -export type Iso_data_dataEngine__DataEngineInsertRequestSchema = Assert, z.infer< typeof M56.DataEngineInsertRequestSchema > >>; export type Iso_data_dataEngine__DataEngineUpdateOptionsSchema = Assert, z.infer< typeof M56.DataEngineUpdateOptionsSchema > >>; -export type Iso_data_dataEngine__DataEngineVectorFindRequestSchema = Assert, z.infer< typeof M56.DataEngineVectorFindRequestSchema > >>; export type Iso_data_dataEngine__DroppedFieldsEventSchema = Assert, z.infer< typeof M56.DroppedFieldsEventSchema > >>; export type Iso_data_dataEngine__EngineDeleteOptionsSchema = Assert, z.infer< typeof M56.EngineDeleteOptionsSchema > >>; export type Iso_data_dataEngine__EngineUpdateOptionsSchema = Assert, z.infer< typeof M56.EngineUpdateOptionsSchema > >>; @@ -1732,7 +1729,7 @@ describe('ADR-0122 type-alias convention', () => { // this title and the section header above the pin list — are now asserted // against the recomputed count below, so neither can go stale without a red // test naming it. - it('still declares all 798 isomorphic pins', () => { + it('still declares all 795 isomorphic pins', () => { // The truth of each pin is proved by tsc, not here — an `Assert>` // that stops holding is a compile error with the alias named. What tsc // cannot notice is a pin that was DELETED: removing the assertion removes @@ -2667,7 +2664,12 @@ describe('ADR-0122 type-alias convention', () => { // never-consumed `{value, currency}` shape a currency value never had (it // is a bare number, ADR-0104 D1), is deleted, and its pin // (Iso_data_field__CurrencyValueSchema) goes with it. -1 removed. - expect(pins).toHaveLength(798); + // 798 -> 795 is #27028: the Virtual ObjectQL request protocol retired whole + // (`data/data-engine.zod.ts`), and with it the three isomorphic arms pinned + // here (Iso_data_dataEngine__DataEngineExecuteRequestSchema, + // Iso_data_dataEngine__DataEngineInsertRequestSchema, + // Iso_data_dataEngine__DataEngineVectorFindRequestSchema). -3 removed. + expect(pins).toHaveLength(795); // The count is stated in PROSE twice as well — this case's title and the // section header above the pin list — and until commit c6b05c76a nothing read either diff --git a/packages/spec/src/ui/view-grouping-query.ts b/packages/spec/src/ui/view-grouping-query.ts index 00137e57606..513d432e9d8 100644 --- a/packages/spec/src/ui/view-grouping-query.ts +++ b/packages/spec/src/ui/view-grouping-query.ts @@ -167,9 +167,8 @@ * (`packages/metadata-protocol/src/protocol.ts`, the `hasGroupBy || * hasAggregations` branch) and answers `{ object, records, total, hasMore }` * with the header rows as `records`; `client.data.query()` - * (`packages/client/src/index.ts`) posts there, and the RPC face declares - * `method: 'aggregate'` with an `EngineAggregateOptions` body - * (`data/data-engine.zod.ts`, `DataEngineAggregateRequestSchema`). The row + * (`packages/client/src/index.ts`) posts there; in process the same body is + * `IDataEngine.aggregate`'s `EngineAggregateOptions`. The row * page is the same door with the compiled `EngineQueryOptions`. No new * route, no new wire shape; the platform half of the card PINS that door on * the compiled queries (the 186-row fixture through the route, on driver-sql diff --git a/packages/spec/test-typecheck-debt.json b/packages/spec/test-typecheck-debt.json index 306f5a69174..4b879b91ffb 100644 --- a/packages/spec/test-typecheck-debt.json +++ b/packages/spec/test-typecheck-debt.json @@ -95,9 +95,8 @@ "TS6133: 'key' is declared but its value is never read.": 1 }, "src/data/data-engine.test.ts": { - "TS18046: 'request.command' is of type 'unknown'.": 1, "TS2339: Property 'created_at' does not exist on type '…'.": 2, - "TS2339: Property 'name' does not exist on type '…'.": 3 + "TS2339: Property 'name' does not exist on type '…'.": 2 }, "src/data/display-name.test.ts": { "TS2339: Property 'fields' does not exist on type '…'.": 1, diff --git a/packages/spec/unemitted-schemas.baseline.json b/packages/spec/unemitted-schemas.baseline.json index 8370b39756b..8daff12d1d7 100644 --- a/packages/spec/unemitted-schemas.baseline.json +++ b/packages/spec/unemitted-schemas.baseline.json @@ -45,10 +45,6 @@ "cause": "function", "reason": "The union of all four `functions` entry shapes. Its first member IS the bare `z.function()` above, and #16431 (a)'s per-branch projection does drop that member — measured: 1 branch pruned — but the union still does not publish, because the member behind it is FlowFunctionDeclarationSchema, whose required `handler` is itself a live callable. An unprojectable node in a PROPERTY position is not droppable: every document the declaration accepts has that key, so publishing the record without it would describe a shape no runtime value has. Both lowered members are plain JSON and would publish on their own; what this entry now records is a union that cannot be projected while one of its branches requires a callable." }, - "Data.DataEngineContractSchema": { - "cause": "function", - "reason": "The engine method contract — `find`/`findOne`/`insert`/`update`/`delete` are `z.function()` members. A CODE interface implemented by an engine, never a document anyone authors or transmits, so no reader loses a reference page." - }, "Data.DriverInterfaceSchema": { "cause": "function", "reason": "The driver method contract (`connect`, `disconnect`, `checkHealth`, `getPoolStats`, `execute` …), all `z.function()`. A code interface implemented by a driver package; nothing is authored against it." diff --git a/packages/spec/variant-docs.json b/packages/spec/variant-docs.json index a4e2a7060ea..762210a9ef7 100644 --- a/packages/spec/variant-docs.json +++ b/packages/spec/variant-docs.json @@ -127,12 +127,6 @@ "exempt": "not-authorable", "reason": "API response envelope, not authored metadata." }, - { - "key": "method:aggregate|count|delete|execute|find|findOne|insert|update|vectorFind", - "label": "data-engine request", - "exempt": "not-authorable", - "reason": "Engine RPC contract. Two entries collapsed into this one when #4618 retired `IDataEngine.batch?`: the outer union carried a `batch` variant, and a second entry described the inner union inside `DataEngineBatchRequestSchema.requests` — this same member set minus `batch`. That inner union is gone with the schema, and the outer one narrowed to exactly this set." - }, { "key": "type:delete|insert|retain", "label": "collaboration op component", diff --git a/packages/spec/vitest.repo-tests.json b/packages/spec/vitest.repo-tests.json index 5a6b43d58b4..c7c443eda16 100644 --- a/packages/spec/vitest.repo-tests.json +++ b/packages/spec/vitest.repo-tests.json @@ -43,6 +43,7 @@ "src/data/cube-refresh-key-retirement.test.ts", "src/data/currency-mode-family-closure.pin.test.ts", "src/data/currency-value-retirement.test.ts", + "src/data/data-engine-rpc-protocol-retirement.test.ts", "src/data/document-schemas-retirement.test.ts", "src/data/driver/turso-encryption-key-retirement.test.ts", "src/data/object-external-sharing-model-retirement.test.ts",