Skip to content

Commit 00a92e1

Browse files
fix(service-analytics)!: a cube or dataset dimension on a structured-JSON field is refused INVALID_FIELD / 400 at the analytics door, before any SQL is built (#20807) (#20886)
Fixes #20807 Clause-②: no (narrowing) ## What this changes A dimension that GROUPS an analytics query, and whose column is a declared **structured-JSON** field (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`), is now refused `INVALID_FIELD` / 400 at the analytics door, naming the member the caller wrote, before either strategy builds anything. "Groups" means a `dimensions` entry, or a `timeDimensions` entry that carries a `granularity`. It holds on `POST /api/v1/analytics/query`, its dry run `POST /api/v1/analytics/sql`, and `POST /api/v1/analytics/dataset/query`, on every driver. The words, as `POST /api/v1/analytics/query` returns them for a cube dimension: ```text Dimension 'meta' on cube 'json_dim_ledger' groups by field 'meta', which object 'analytics_json_dim_ledger' declares as json — a structured-JSON value, which analytics does not group by. The query was NOT run. Group by a field that stores one scalar value: store the part you group on in a field of its own and group by that field. A JSON document is no group key the SQL dialects share: one grouped each serialized document apart, another refused the statement. ``` For a dataset dimension over an included relationship, the same words say `groups by field 'account.hq', whose column 'hq' the joined object 'X' declares as json`. The thrown error is `invalidMemberError`'s envelope (`code: 'INVALID_FIELD'`, `status: 400`, `member`, `param`, `cube`) plus `field` and `object`. **Landing site: `packages/services/service-analytics` only.** - `src/structured-json-dimension-door.ts` (new): `assertNoStructuredJsonDimension`. The class is `@objectstack/spec/data`'s `STRUCTURED_JSON_TYPES`, called, never re-listed. It is the same predicate the engine's `groupBy` door (`packages/objectql/src/group-by-structured-json-door.ts`) reads, so there is one "is this field structured JSON" test for both doors. That file is read, not edited. - `src/analytics-service.ts`: one private method, `assertDimensionsGroupScalarColumns`, called in `ensureCube` right after the dimension source-field gate on each of its three paths (inferred cube, augmented cube, declared cube). It supplies the two answers only the service has: the dimension `sql` a member resolves to (`declaredMemberEntry`, the strategies' own `'dimension'` lookup, and the member itself when the cube declares none), and the column's declared type (`sourceFieldMeta`). - The column is read the way `NativeSQLStrategy` compiles it. A bare identifier is a column of the cube's object. A dotted identifier path (`account.hq`) is its last segment, on the object the cube's DECLARED join for that path names, which is the alias the dataset compiler registers and the strategy joins. A path with no declared join is a synthetic traversal and is not judged. ## Before, measured on `origin/main` `793fb839` Through the real `dispatcher-plugin` route over the service `AnalyticsServicePlugin` composes on a real `ObjectQL` engine, with both of its auto-bridges live (`executeRawSql` to `engine.execute`, `executeAggregate` to `engine.aggregate`). Three rows: `title` x, x, y, and a different `meta` document per row. Drivers: `SqlDriver` on SQLite (better-sqlite3), and on a private PostgreSQL 16.13 started for this run. | request | SQLite | PostgreSQL 16 | |:--|:--|:--| | `/analytics/query`, `dimensions: ['title']` (text, the control) | 200, `x` 2 · `y` 1 | same | | `/analytics/query`, cube dimension `meta` (json) | 200, one group per serialized document (3 groups, `count` 1 each) | 500 `DATABASE_ERROR` (42883, "could not identify an equality operator for type json") | | `/analytics/query`, dataset dimension `meta_doc` (over `meta`) | 200, 3 groups | 500 `DATABASE_ERROR` | | `/analytics/sql`, cube dimension `meta` | 200, the statement `SELECT meta AS "meta", COUNT(*) AS "count" FROM ... GROUP BY meta` | same | | `/analytics/dataset/query`, inline dataset dimension `meta_doc` | 200, 3 groups | 500 `DATABASE_ERROR` | Counted at the engine for the json dimension: raw SQL 1, `engine.aggregate` 0, on both dialects, so `NativeSQLStrategy` answered and the engine's `groupBy` door never saw the query. A dataset dimension over a JOINED object's json field (`include: ['account']`, `field: 'account.hq'`) answered the same two ways: SQLite 200 with one group per document (2 groups), PostgreSQL 500 (42883). This was measured through the service on the first fix commit `b1befe2a6`, which judged only bare columns, and through `/analytics/dataset/query` under the ablation below. The second fix commit closes it. ## After, on this branch (`075a46340`) Every refused row above answers `400 INVALID_FIELD` naming the member (`meta`, `meta_doc`, `acct_hq`), with zero raw-SQL statements and zero engine aggregates for the object. The `title`, `title_dim` and `acct_name` controls answer `x` 2 · `y` 1 (and `A` 2 · `B` 1) from the native strategy, unchanged. ## Mechanism assumptions (zone 2): which held - **B1: held.** It was reproduced on `793fb839` as the red pins. See the table above and the raw-SQL / aggregate counts. - **B2: held.** A dataset dimension compiles to a cube dimension whose key is the dataset dimension's name (`dataset-compiler.ts`: `dimensions[d.name] = { sql: d.field }`). `DatasetExecutor` passes `selection.dimensions` through as the cube query's `dimensions`, so the member reaching `ensureCube` IS the name the selection wrote. The compiler checks only the relationship path (`assertDeclared`). - **B3: held.** The one predicate class is `STRUCTURED_JSON_TYPES`, read at the engine door. The engine door's exported function takes `groupBy` entries and names `groupBy[i]`, and `@objectstack/objectql` is only a devDependency of this package, so the constant is what is called. No edit to `packages/objectql` or `packages/spec`. - **B4: held, measured.** `cube-registry.ts` stores cubes and knows no field type. `dataset-compiler.ts` knows a dataset dimension's name and the declared type, but it compiles the whole dataset, whether or not a dimension is selected (a refusal there would refuse every selection of the dataset), and it does not see cube queries. `analytics-service.ts`'s `ensureCube` is the first step every door passes through that knows both the member as written and the column's declared type, ahead of strategy selection. The GUARD and the per-face unit cases show one answer on both strategies. Before this, the ObjectQL face reached the engine door but was refused under `groupBy[0]`. - **B5: no CI harness runs this route live.** The `Temporal Conformance (live PG + MySQL)` job sets `OS_TEST_POSTGRES_URL` for three steps only: `driver-sql`, `metadata-protocol`'s `live-postgres` files, and `runtime`'s cascade-delete matrix. It runs `service-analytics` without a URL, and no step runs `@objectstack/rest`'s or `@objectstack/runtime`'s analytics pins. The PostgreSQL cells therefore sit beside each HTTP pin as a named skip without the URL, like `packages/rest/src/data-group-by-json-door.test.ts`. They are red-capable and un-run in CI. The local PostgreSQL 16.13 runs are quoted below. - **B6: measured `no (narrowing)`.** No new key reaches a published payload: the error envelope's members are the ones `invalidMemberError` and the dimension source-field gate already attach. The accept set narrows: SQLite answered 200, and it now refuses. Changeset `minor`, BREAKING banner, `Clause-②: no (narrowing)`. The ADR-0087 category measured is `not-required (no-migration-prescription)`: the package publishes, no ADR-0087 id covers a grouping target, and nothing authorable, exported or stored moves. - **B7: reproduced.** See the acceptance notes. Not fixed here. **Where the `/api/v1/analytics/query` pin lives, measured.** That route is served by `@objectstack/runtime`'s `dispatcher-plugin` (through `domains/analytics.ts`), not by `@objectstack/rest`. The REST package serves `/analytics/dataset/query`, and `runtime` depends on `rest`, so a REST-package test cannot reach the runtime route. The cube-face pin is therefore a new file beside the repo's other `/analytics/query` HTTP pins (`packages/runtime/src/analytics-*.test.ts`). The dataset-door pin is a new file in `packages/rest/src/`. Both are new pin files only. ## Tests All at `075a46340`, the final commit. - New `packages/services/service-analytics/src/__tests__/dimension-structured-json-door.test.ts`: **13 passed**. It covers every structured-JSON type on both strategy faces (native and ObjectQL), with the envelope (`code`, `status`, `member`, `param`, `field`, `object`) and zero raw SQL and zero aggregates. It also covers a cube key over another column, the cube-qualified spelling, a bucketed time dimension on both faces, a declared default granularity, a dataset dimension, a dataset dimension over an included relationship (judged on the joined object), an ad-hoc inferred cube, and the dry run. Controls: a text dimension is served, an unknown field keeps the existence gate's answer first, a synthetic dotted traversal is not judged, and a host without `sourceFieldMeta` stands down. GUARD: over every `FieldType`, the refused types are exactly `STRUCTURED_JSON_TYPES`. - New `packages/runtime/src/analytics-json-dimension-door.test.ts` (`POST /api/v1/analytics/query` and `/sql`): **8 passed**, SQLite 4 and live PostgreSQL 16.13 4. - New `packages/rest/src/analytics-dataset-json-dimension-door.test.ts` (`POST /api/v1/analytics/dataset/query`): **6 passed**, SQLite 3 and live PostgreSQL 16.13 3. - Red first: at `46ee85eda` (the pins on the base code) the unit file was 8 failed and 3 passed, the runtime file 6 failed and 2 passed, and the REST file 2 failed and 2 passed. The failures were the refusals; the controls were green. - `pnpm --filter @objectstack/service-analytics exec vitest run`: **145 files / 3325 passed**. - `typecheck`, exit 0: `@objectstack/service-analytics` (`tsc --listFilesOnly` includes the new door and the new test); `@objectstack/rest` (the new test is in its test program, and `check:test-typecheck` holds at 0 files / 0 errors); `@objectstack/runtime` (`check:test-typecheck` holds at 27 files / 190 errors / 68 signatures, unchanged). - Downstream consumers are declared to CI. No export, published type or `exports` entry of `@objectstack/service-analytics` changes, so only behaviour moves. A census of `examples/` at `793fb839` found no producer grouping by a structured-JSON field: 5 files declare a cube or dataset, 18 distinct dimension sources, none of them structured JSON. **Reverse verification (ablation), from the committed fix at `075a46340`.** It ran through `scripts/ablation-replace.mjs` in WRAP mode, trap-restored, under the verify lock. The door's own verdict line gained an always-true `continue` guard keyed on the marker `__ablated_20807__`. On disk: anchor 1 to 0, blob `47ce2b2acc11` to `a286ca8a75ae`. `service-analytics` was rebuilt, and `ablation-dist-preflight` found the marker in 2 built files (`dist/index.js`, `dist/index.cjs`). - Predicted direction: red. Observed: red. - Unit: **9 failed / 4 passed**. Every refusal case and the GUARD failed; the four controls stayed green. - `/analytics/query` pin: **6 failed / 2 passed**. On SQLite the cube and dataset dimensions answered 200 with 3 groups and the dry run served the statement. On PostgreSQL they answered 500 `DATABASE_ERROR`. The controls stayed green. - `/analytics/dataset/query` pin: **4 failed / 2 passed**. `meta_doc` and `acct_hq` answered SQLite 200 and PostgreSQL 500. The controls stayed green. - Restore leg: blob equals HEAD (`47ce2b2acc11`), `git diff HEAD` is empty, and the whole-tree `git status --porcelain` is empty. After a rebuild, `--absent` found the marker absent from all 6 built files and the tree clean. The re-run was green: 13, 8 and 6 passed. - The same ablation on the first fix commit `b1befe2a6`, before the joined-column extension, read 8/3, 6/2 and 2/2, the same direction. ## Gates `node scripts/pm/dispatch-gates.mjs --commands` (no paths) at `075a46340` derived **62** commands over the 6 changed paths. That is the same list the PM derived at `793fb839`. The four roster families the order names were run too: `node scripts/check-changeset-fixed.mjs`, `pnpm check:authz-resolver`, `pnpm check:error-code-casing` and `pnpm check:filter-alias-parity`. `--ran` reconciles: **62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN**. All 66 commands exit 0. `check:dual-build-cjs-loads` and `check:type-check-debt` first exited 3 (`PREREQUISITE NOT MET`) and were re-run to exit 0 after a full `turbo run build --filter='./packages/*' --filter='./packages/*/*'`. Among them: - `check:adr-0087-registration --base origin/main`: `[BREAKING+bang+clause-②-narrowing] not-required (no-migration-prescription)` accepted. - `check:changeset-no-major`, `check:empty-changeset`, `check:doc-authoring`, `check:nul-bytes`, `check:issue-citations`. - `check:cross-package-test-inputs`, `check:test-source-alias`, `check:driver-memory-census`, `check:rest-log-spy-declared`, `check:engine-double-contract`, `check:query-options-erasure`, `check:type-check-coverage`, `check:type-check-debt` (re-measure: none above its record). Note: `dispatch-gates` flagged the tree as 6 commits behind `origin/main` `2d5fe76f4`. None of those commits touches this diff's paths, `service-analytics`, the analytics routes, the engine door or `STRUCTURED_JSON_TYPES`, so `main` was not merged (the order merges only on a surface hit). The merge ref CI builds covers the joint tree. Lint, narrowed and proven: `pnpm exec eslint --no-inline-config --format json` over the 5 changed `.ts` files at `075a46340` found **5 files, 0 errors, 0 warnings**. Three facts make this narrowing a measurement: - The checked population comes from eslint's own config: `isPathIgnored` answers `false` for all 5. - The file count comes from the JSON output: 5 results. - Untouched files cannot change verdict: `parserOptions.project` and `projectService` are `null` for every file, so type-aware linting is not enabled. ## Changeset `.changeset/20807-analytics-json-dimension-refused.md`: `@objectstack/service-analytics` `minor`, BREAKING banner, `Clause-②: no (narrowing)`, and exactly one ADR-0087 marker, `not-required (no-migration-prescription)`, in the form the engine door's changeset uses. It states the refused shape, names the refusal's code, and says who is affected and what is unchanged. No export or published type changes. ## Acceptance notes - **Reported to the PM, not filed here:** - **The PostgreSQL count is a string on the native path (class a, B7).** Through `POST /api/v1/analytics/query` on PostgreSQL 16, the `text` control's rows came back `{"title":"x","count":"2"}` while `fields` said `{"name":"count","type":"number"}`. SQLite returned `2`. The registered dataset's `row_count` behaved the same. This is the result-typing class triage directed out of this card. - **No authoring leg for this refusal (class c).** `os validate` (the built CLI at `075a46340`) passes a stack whose dataset declares a dimension over a `json` field: exit 0, "Validation passed". The runtime now refuses that dimension at query time. The same stack with a measure `avg` over that field is refused by `measure-aggregate-field-type-refused`, so the command does judge dataset members against declared types. - **Boundaries of this door, not measured as defects:** - A `timeDimensions` entry with no `granularity` over a json field. It bounds a range and groups nothing, so it is a filter's question, not this door's. - A dotted dimension the cube declares no join for (a synthetic traversal). - A host that wires no `sourceFieldMeta`. - The draft preview. `queryDataset` with `previewDrafts` over a pending seed evaluates in memory (`evaluateAnalyticsQueryOverRows`) and does not pass `ensureCube`. - `MemoryAnalyticsService` (`driver-memory`'s cube face) is #20859's position and is not touched. - **Order: existence first, then type.** A member naming a column the object does not have keeps the existing `INVALID_FIELD` ("does not have") answer, because the dimension source-field gate runs first. - #20810 is not addressed here: it lowers filters, and this card refuses dimensions. --- _Generated by [Claude Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 26437ae commit 00a92e1

6 files changed

Lines changed: 1030 additions & 0 deletions

File tree

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
"@objectstack/service-analytics": minor
3+
---
4+
5+
fix(service-analytics)!: a cube or dataset dimension on a structured-JSON field is refused with `INVALID_FIELD` / 400 at the analytics door, before any SQL is built
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) a refusal of a grouping TARGET at the analytics door: a `dimensions` entry, or a bucketed `timeDimensions` entry, whose column is a declared json, composite, repeater, record, location, address or vector field. No authorable key, spelling, export or stored shape moves (the door module is internal; `@objectstack/service-analytics` exports nothing new and nothing less, and `CubeSchema`, `DatasetSchema` and the analytics query body keep parsing every member), and no stored row is read or rewritten. The grouping had no shared meaning to preserve (one group per serialized document on SQLite, a 500 on PostgreSQL), and which scalar part of the document a caller meant to group on is not something a ledger entry can rewrite. The other categories are closed on facts: the package publishes (not `unpublished`); no ADR-0087 id covers a grouping target (not `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
10+
11+
**BREAKING**: this narrows what the analytics query doors accept as a dimension. A cube dimension, or a dataset dimension, whose column is a declared field of the structured-JSON class (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) is refused before either strategy builds a statement, when it groups the result: a `dimensions` entry, or a `timeDimensions` entry with a `granularity`. The column is judged where it is declared: on the cube's object, or, for a dotted path such as a dataset dimension over `account.hq`, on the object the cube's declared join for that path names. It holds on `POST /api/v1/analytics/query`, on its dry run `POST /api/v1/analytics/sql`, and on `POST /api/v1/analytics/dataset/query`, on every driver. It ships as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes.
12+
13+
**What an author sees now.** `400 INVALID_FIELD`, naming the member as the request wrote it (the cube dimension, or the dataset dimension), the column it groups by, the object and the column's declared type, saying the query was not run, and naming the route: group by a field that stores one scalar value, storing the part of the document you group on in a field of its own. The thrown error carries `member`, `param` (`dimensions` or `timeDimensions`), `cube`, `field` and `object`.
14+
15+
**Why a refusal.** A JSON document is no group key the SQL dialects share. Measured through `POST /api/v1/analytics/query` over three rows with a different document each, on the service `AnalyticsServicePlugin` composes over a real engine: SQLite answered 200 with one group per serialized document, and PostgreSQL 16 answered 500 `DATABASE_ERROR`. A dataset dimension over a joined object's `json` field answered the same two ways through `POST /api/v1/analytics/dataset/query`. The native-SQL strategy compiled the `GROUP BY` itself, so the engine's own refusal of a structured-JSON `groupBy` never saw the query; the engine-aggregate strategy did reach that refusal, but named the engine's `groupBy[0]` position rather than the member the caller wrote. The class is `@objectstack/spec/data`'s `STRUCTURED_JSON_TYPES`, the one the engine's refusal reads. No producer groups by such a field: no cube or dataset dimension in the example apps names one.
16+
17+
**Who is affected.** A dashboard, report or caller that grouped an analytics query by a structured-JSON field on SQLite and read one group per serialized document as real groups. On PostgreSQL the same query was already a 500.
18+
19+
**Unchanged.** A dimension on any other type; a `timeDimensions` entry with no `granularity`, which bounds a range and groups nothing; measures (this door judges only the members that group); a dotted dimension path the cube declares no join for, whose object is not a declaration; a member naming a column the object does not have, which keeps its existing `INVALID_FIELD` answer first; and a host that wires no `sourceFieldMeta`, where the column's type cannot be read.
Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* A dataset dimension on a structured-JSON field is refused at the dataset
5+
* door — `POST /api/v1/analytics/dataset/query` answers `400 INVALID_FIELD`,
6+
* naming the dataset dimension the selection wrote, before any SQL is built —
7+
* over a real `SqlDriver`; and a `text` dimension (the control) is served
8+
* unchanged.
9+
*
10+
* The cube face of the same door (`POST /api/v1/analytics/query`, served by
11+
* `@objectstack/runtime`'s dispatcher, not by this package) is pinned in
12+
* `packages/runtime/src/analytics-json-dimension-door.test.ts`. This file is
13+
* the route this package serves: an INLINE dataset, compiled per request,
14+
* whose dimensions reach the same cube query through `DatasetExecutor`.
15+
*
16+
* ## Measured without the refusal, through this door
17+
*
18+
* On the base (the `meta_doc` row), and with the refusal ablated (the
19+
* `acct_hq` row). Three rows, `title` x, x, y, and a different `meta`
20+
* document per row; a dataset declaring `meta_doc` over the `json` field
21+
* `meta`, and `acct_hq` over the `json` field `hq` of the object the
22+
* `account` lookup references:
23+
*
24+
* | `selection.dimensions` | SQLite | PostgreSQL 16 |
25+
* |:--|:--|:--|
26+
* | `title_dim` (text, the control) | 200, `x` 2 · `y` 1 | same |
27+
* | `meta_doc` (json) | 200, one group per serialized document (3) | 500 |
28+
* | `acct_hq` (`account.hq`, a json field of the `include`d object) | 200, one group per document (2) | 500 |
29+
*
30+
* ## The composition, and the dialect axis of THIS file
31+
*
32+
* The analytics service is the one `AnalyticsServicePlugin` composes over a
33+
* real `ObjectQL` engine — both auto-bridges live, so `NativeSQLStrategy`
34+
* answers on a SQL driver. The SQLite cell always runs. The PostgreSQL cell
35+
* runs where `OS_TEST_POSTGRES_URL` is set and is a named skip otherwise; no
36+
* CI step provisions that variable for this package, so the live cell is
37+
* red-capable and un-run in CI, and the PR that landed this file carries its
38+
* local PostgreSQL 16 run. The live cell owns its table, dropped before and
39+
* after.
40+
*/
41+
42+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
43+
import { ObjectQL } from '@objectstack/objectql';
44+
import { SqlDriver } from '@objectstack/driver-sql';
45+
import { AnalyticsServicePlugin, type AnalyticsService } from '@objectstack/service-analytics';
46+
import { RestServer } from './rest-server';
47+
48+
const OBJECT = 'rest_dataset_json_dim_ledger';
49+
/** The object the ledger's `account` lookup references — joined through the dataset's `include`. */
50+
const ACCOUNT = 'rest_dataset_json_dim_account';
51+
52+
const ACCOUNT_OBJECT = {
53+
name: ACCOUNT,
54+
label: 'Dataset JSON dimension account',
55+
fields: {
56+
name: { name: 'name', type: 'text' as const },
57+
hq: { name: 'hq', type: 'json' as const },
58+
},
59+
};
60+
61+
const LEDGER = {
62+
name: OBJECT,
63+
label: 'Dataset JSON dimension ledger',
64+
fields: {
65+
title: { name: 'title', type: 'text' as const },
66+
meta: { name: 'meta', type: 'json' as const },
67+
account: { name: 'account', type: 'lookup' as const, reference: ACCOUNT },
68+
},
69+
};
70+
71+
const ACCOUNTS = [
72+
{ id: 'a1', name: 'A', hq: { city: 'Paris' } },
73+
{ id: 'a2', name: 'B', hq: { city: 'Rome' } },
74+
];
75+
76+
const ROWS = [
77+
{ id: 'r1', title: 'x', meta: { a: 1 }, account: 'a1' },
78+
{ id: 'r2', title: 'x', meta: { a: 2 }, account: 'a1' },
79+
{ id: 'r3', title: 'y', meta: { b: 1 }, account: 'a2' },
80+
];
81+
82+
/** The inline dataset the request carries — as a Studio preview or a widget posts it. */
83+
const DATASET = {
84+
name: 'json_dim_inline',
85+
label: 'JSON dimension inline',
86+
object: OBJECT,
87+
include: ['account'],
88+
dimensions: [
89+
{ name: 'title_dim', field: 'title', type: 'string' },
90+
{ name: 'meta_doc', field: 'meta', type: 'string' },
91+
{ name: 'acct_name', field: 'account.name', type: 'string' },
92+
{ name: 'acct_hq', field: 'account.hq', type: 'string' },
93+
],
94+
measures: [{ name: 'row_count', aggregate: 'count' }],
95+
};
96+
97+
/** The route the refusal prescribes — asserted on the wire body. */
98+
const ROUTE = 'Group by a field that stores one scalar value: store the part you group on in a field of its own and group by that field.';
99+
100+
interface Cell {
101+
id: 'sqlite' | 'pg';
102+
label: string;
103+
env: string | null;
104+
config: () => Record<string, unknown> | null;
105+
}
106+
107+
const CELLS: readonly Cell[] = [
108+
{ id: 'sqlite', label: 'sqlite', env: null, config: () => ({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }) },
109+
{
110+
id: 'pg',
111+
label: 'live postgres',
112+
env: 'OS_TEST_POSTGRES_URL',
113+
config: () => (process.env.OS_TEST_POSTGRES_URL ? { client: 'pg', connection: process.env.OS_TEST_POSTGRES_URL } : null),
114+
},
115+
];
116+
117+
const quiet = { debug() {}, info() {}, warn() {}, error() {}, child() { return quiet; } };
118+
119+
function createMockServer() {
120+
const noop = () => {};
121+
return { get: noop, post: noop, put: noop, delete: noop, patch: noop, use: noop, listen: async () => {}, close: async () => {} };
122+
}
123+
124+
function mockProtocol() {
125+
return {
126+
getDiscovery: async () => ({ version: 'v0', routes: { data: '', metadata: '' } }),
127+
getMetaTypes: async () => [],
128+
getMetaItems: async () => [],
129+
};
130+
}
131+
132+
function makeRes() {
133+
const res: any = {
134+
statusCode: 200,
135+
body: undefined as any,
136+
header: () => res,
137+
status: (code: number) => { res.statusCode = code; return res; },
138+
json: (body: unknown) => { res.body = body; return res; },
139+
end: () => res,
140+
};
141+
return res;
142+
}
143+
144+
for (const cell of CELLS) {
145+
const config = cell.config();
146+
describe.skipIf(!config)(
147+
`a dataset dimension on a json field at POST /api/v1/analytics/dataset/query — ${cell.label}${config ? '' : ` (skipped: set ${cell.env} to run this cell)`}`,
148+
() => {
149+
let driver: any;
150+
let engine: ObjectQL;
151+
/** Raw-SQL statements and engine aggregates that read THIS object. */
152+
const reads = { rawSql: 0, aggregate: 0 };
153+
let query: (selection: Record<string, unknown>) => Promise<{ status: number; body: any }>;
154+
155+
const dropTables = async () => {
156+
if (cell.id === 'sqlite') return;
157+
for (const table of [OBJECT, ACCOUNT]) await driver?.execute(`drop table if exists ${table}`).catch(() => {});
158+
};
159+
160+
beforeAll(async () => {
161+
driver = new SqlDriver(config as any);
162+
await dropTables();
163+
engine = new ObjectQL({ logger: quiet } as any);
164+
engine.registerDriver(driver, true);
165+
await engine.init();
166+
engine.registry.registerObject(ACCOUNT_OBJECT as any);
167+
engine.registry.registerObject(LEDGER as any);
168+
await engine.syncSchemas();
169+
for (const row of ACCOUNTS) await engine.insert(ACCOUNT, { ...row } as any);
170+
for (const row of ROWS) await engine.insert(OBJECT, { ...row } as any);
171+
172+
const realExecute = (engine as any).execute.bind(engine);
173+
(engine as any).execute = (sql: unknown, opts?: { object?: string }) => {
174+
if (opts?.object === OBJECT) reads.rawSql += 1;
175+
return realExecute(sql, opts);
176+
};
177+
const realAggregate = engine.aggregate.bind(engine);
178+
(engine as any).aggregate = (object: string, ...rest: unknown[]) => {
179+
if (object === OBJECT) reads.aggregate += 1;
180+
return (realAggregate as any)(object, ...rest);
181+
};
182+
183+
// The plugin's own composition over the real engine: both auto-bridges.
184+
const registered: Record<string, unknown> = {};
185+
await new AnalyticsServicePlugin().init({
186+
getService: (name: string) => (name === 'data' ? engine : registered[name]),
187+
registerService: (name: string, svc: unknown) => { registered[name] = svc; },
188+
replaceService: (name: string, svc: unknown) => { registered[name] = svc; },
189+
hook: () => {},
190+
logger: quiet,
191+
} as never);
192+
const analytics = registered.analytics as AnalyticsService;
193+
194+
const rest = new RestServer(
195+
createMockServer() as any, mockProtocol() as any, { api: { requireAuth: false } } as any,
196+
undefined, undefined, undefined, undefined, undefined, undefined, undefined,
197+
undefined, undefined, undefined, undefined,
198+
async () => analytics,
199+
);
200+
(rest as any).resolveExecCtx = async () => ({ userId: 'test-user' });
201+
rest.registerRoutes();
202+
const route = rest.getRoutes().find((r: any) => r.method === 'POST' && r.path === '/api/v1/analytics/dataset/query');
203+
expect(route).toBeDefined();
204+
query = async (selection) => {
205+
const res = makeRes();
206+
// What the wire carries: JSON.
207+
const body = JSON.parse(JSON.stringify({ dataset: DATASET, selection }));
208+
await route!.handler({ method: 'POST', params: {}, headers: {}, body, query: {} } as any, res);
209+
return { status: res.statusCode, body: res.body };
210+
};
211+
});
212+
213+
afterAll(async () => {
214+
await dropTables();
215+
try { await engine?.destroy(); } catch { /* noop */ }
216+
});
217+
218+
it('a dataset dimension on a json field answers 400 INVALID_FIELD naming the dataset dimension — no statement reaches the engine', async () => {
219+
const before = { ...reads };
220+
const res = await query({ measures: ['row_count'], dimensions: ['meta_doc'] });
221+
expect(res.status, JSON.stringify(res.body)).toBe(400);
222+
expect(res.body.code).toBe('INVALID_FIELD');
223+
expect(String(res.body.message)).toContain(`Dimension 'meta_doc' on cube '${DATASET.name}' groups by field 'meta'`);
224+
expect(String(res.body.message)).toContain(`'${OBJECT}' declares as json`);
225+
expect(String(res.body.message)).toContain(ROUTE);
226+
expect(reads, 'no raw SQL and no engine aggregate for the object').toEqual(before);
227+
});
228+
229+
it('a dataset dimension over an included relationship\'s json field answers the same 400, naming the joined object — no statement reaches the engine', async () => {
230+
const before = { ...reads };
231+
const res = await query({ measures: ['row_count'], dimensions: ['acct_hq'] });
232+
expect(res.status, JSON.stringify(res.body)).toBe(400);
233+
expect(res.body.code).toBe('INVALID_FIELD');
234+
expect(String(res.body.message)).toContain(
235+
`Dimension 'acct_hq' on cube '${DATASET.name}' groups by field 'account.hq', whose column 'hq' the joined object '${ACCOUNT}' declares as json`,
236+
);
237+
expect(String(res.body.message)).toContain(ROUTE);
238+
expect(reads, 'no raw SQL and no engine aggregate for the object').toEqual(before);
239+
});
240+
241+
it('CONTROL a text dataset dimension is served unchanged: one group per value, counted', async () => {
242+
const before = { ...reads };
243+
const res = await query({ measures: ['row_count'], dimensions: ['title_dim'] });
244+
expect(res.status, JSON.stringify(res.body)).toBe(200);
245+
const groups = (res.body.rows as Array<{ title_dim: string; row_count: number | string }>)
246+
.map((r) => [r.title_dim, Number(r.row_count)] as const)
247+
.sort(([a], [b]) => a.localeCompare(b));
248+
expect(groups).toEqual([['x', 2], ['y', 1]]);
249+
expect(reads.rawSql - before.rawSql, 'the native strategy answered').toBeGreaterThanOrEqual(1);
250+
251+
const joined = await query({ measures: ['row_count'], dimensions: ['acct_name'] });
252+
expect(joined.status, JSON.stringify(joined.body)).toBe(200);
253+
const joinedGroups = (joined.body.rows as Array<{ acct_name: string; row_count: number | string }>)
254+
.map((r) => [r.acct_name, Number(r.row_count)] as const)
255+
.sort(([a], [b]) => a.localeCompare(b));
256+
expect(joinedGroups).toEqual([['A', 2], ['B', 1]]);
257+
});
258+
},
259+
);
260+
}

0 commit comments

Comments
 (0)