fix(service-analytics): judge a dotted dimension on the object its declared join names; changeset

A dataset dimension over an included relationship (`account.hq`) reached
NativeSQLStrategy unjudged: measured, SQLite 200 with one group per document
and PostgreSQL 500, like a base-object one. The door now reads the column the
way the strategy compiles it: a bare identifier on the cube's object, a dotted
identifier path on the object the cube's declared join for that path names.
A path with no declared join is a synthetic traversal and stays unjudged.

Pins: the service door and the dataset door gain the joined case and its
text control. Changeset: service-analytics minor, BREAKING, Clause-② no
(narrowing), ADR-0087 not-required (no-migration-prescription).

Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H
Co-authored-by: Claude <noreply@anthropic.com>

Author: claude

.changeset/20807-analytics-json-dimension-refused.md

@@ -0,0 +1,19 @@
+---
+"@objectstack/service-analytics": minor
+---
+
+fix(service-analytics)!: a cube or dataset dimension on a structured-JSON field is refused with `INVALID_FIELD` / 400 at the analytics door, before any SQL is built
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a refusal of a grouping TARGET at the analytics door: a `dimensions` entry, or a bucketed `timeDimensions` entry, whose column is a declared json, composite, repeater, record, location, address or vector field. No authorable key, spelling, export or stored shape moves (the door module is internal; `@objectstack/service-analytics` exports nothing new and nothing less, and `CubeSchema`, `DatasetSchema` and the analytics query body keep parsing every member), and no stored row is read or rewritten. The grouping had no shared meaning to preserve (one group per serialized document on SQLite, a 500 on PostgreSQL), and which scalar part of the document a caller meant to group on is not something a ledger entry can rewrite. The other categories are closed on facts: the package publishes (not `unpublished`); no ADR-0087 id covers a grouping target (not `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
+
+**BREAKING**: this narrows what the analytics query doors accept as a dimension. A cube dimension, or a dataset dimension, whose column is a declared field of the structured-JSON class (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) is refused before either strategy builds a statement, when it groups the result: a `dimensions` entry, or a `timeDimensions` entry with a `granularity`. The column is judged where it is declared: on the cube's object, or, for a dotted path such as a dataset dimension over `account.hq`, on the object the cube's declared join for that path names. It holds on `POST /api/v1/analytics/query`, on its dry run `POST /api/v1/analytics/sql`, and on `POST /api/v1/analytics/dataset/query`, on every driver. It ships as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes.
+
+**What an author sees now.** `400 INVALID_FIELD`, naming the member as the request wrote it (the cube dimension, or the dataset dimension), the column it groups by, the object and the column's declared type, saying the query was not run, and naming the route: group by a field that stores one scalar value, storing the part of the document you group on in a field of its own. The thrown error carries `member`, `param` (`dimensions` or `timeDimensions`), `cube`, `field` and `object`.
+
+**Why a refusal.** A JSON document is no group key the SQL dialects share. Measured through `POST /api/v1/analytics/query` over three rows with a different document each, on the service `AnalyticsServicePlugin` composes over a real engine: SQLite answered 200 with one group per serialized document, and PostgreSQL 16 answered 500 `DATABASE_ERROR`. A dataset dimension over a joined object's `json` field answered the same two ways through `POST /api/v1/analytics/dataset/query`. The native-SQL strategy compiled the `GROUP BY` itself, so the engine's own refusal of a structured-JSON `groupBy` never saw the query; the engine-aggregate strategy did reach that refusal, but named the engine's `groupBy[0]` position rather than the member the caller wrote. The class is `@objectstack/spec/data`'s `STRUCTURED_JSON_TYPES`, the one the engine's refusal reads. No producer groups by such a field: no cube or dataset dimension in the example apps names one.
+
+**Who is affected.** A dashboard, report or caller that grouped an analytics query by a structured-JSON field on SQLite and read one group per serialized document as real groups. On PostgreSQL the same query was already a 500.
+
+**Unchanged.** A dimension on any other type; a `timeDimensions` entry with no `granularity`, which bounds a range and groups nothing; measures (this door judges only the members that group); a dotted dimension path the cube declares no join for, whose object is not a declaration; a member naming a column the object does not have, which keeps its existing `INVALID_FIELD` answer first; and a host that wires no `sourceFieldMeta`, where the column's type cannot be read.

packages/rest/src/analytics-dataset-json-dimension-door.test.ts

@@ -13,15 +13,19 @@
  * the route this package serves: an INLINE dataset, compiled per request,
  * whose dimensions reach the same cube query through `DatasetExecutor`.
  *
- * ## Measured on the base, through this door
+ * ## Measured without the refusal, through this door
  *
- * Three rows, `title` x, x, y, and a different `meta` document per row; a
- * dataset declaring `meta_doc` over the `json` field `meta`:
+ * On the base (the `meta_doc` row), and with the refusal ablated (the
+ * `acct_hq` row). Three rows, `title` x, x, y, and a different `meta`
+ * document per row; a dataset declaring `meta_doc` over the `json` field
+ * `meta`, and `acct_hq` over the `json` field `hq` of the object the
+ * `account` lookup references:
  *
  * | `selection.dimensions` | SQLite | PostgreSQL 16 |
  * |:--|:--|:--|
  * | `title_dim` (text, the control) | 200, `x` 2 · `y` 1 | same |
  * | `meta_doc` (json) | 200, one group per serialized document (3) | 500 |
+ * | `acct_hq` (`account.hq`, a json field of the `include`d object) | 200, one group per document (2) | 500 |
  *
  * ## The composition, and the dialect axis of THIS file
  *
@@ -42,30 +46,50 @@ import { AnalyticsServicePlugin, type AnalyticsService } from '@objectstack/serv
 import { RestServer } from './rest-server';
 
 const OBJECT = 'rest_dataset_json_dim_ledger';
+/** The object the ledger's `account` lookup references — joined through the dataset's `include`. */
+const ACCOUNT = 'rest_dataset_json_dim_account';
+
+const ACCOUNT_OBJECT = {
+  name: ACCOUNT,
+  label: 'Dataset JSON dimension account',
+  fields: {
+    name: { name: 'name', type: 'text' as const },
+    hq: { name: 'hq', type: 'json' as const },
+  },
+};
 
 const LEDGER = {
   name: OBJECT,
   label: 'Dataset JSON dimension ledger',
   fields: {
     title: { name: 'title', type: 'text' as const },
     meta: { name: 'meta', type: 'json' as const },
+    account: { name: 'account', type: 'lookup' as const, reference: ACCOUNT },
   },
 };
 
+const ACCOUNTS = [
+  { id: 'a1', name: 'A', hq: { city: 'Paris' } },
+  { id: 'a2', name: 'B', hq: { city: 'Rome' } },
+];
+
 const ROWS = [
-  { id: 'r1', title: 'x', meta: { a: 1 } },
-  { id: 'r2', title: 'x', meta: { a: 2 } },
-  { id: 'r3', title: 'y', meta: { b: 1 } },
+  { id: 'r1', title: 'x', meta: { a: 1 }, account: 'a1' },
+  { id: 'r2', title: 'x', meta: { a: 2 }, account: 'a1' },
+  { id: 'r3', title: 'y', meta: { b: 1 }, account: 'a2' },
 ];
 
 /** The inline dataset the request carries — as a Studio preview or a widget posts it. */
 const DATASET = {
   name: 'json_dim_inline',
   label: 'JSON dimension inline',
   object: OBJECT,
+  include: ['account'],
   dimensions: [
     { name: 'title_dim', field: 'title', type: 'string' },
     { name: 'meta_doc', field: 'meta', type: 'string' },
+    { name: 'acct_name', field: 'account.name', type: 'string' },
+    { name: 'acct_hq', field: 'account.hq', type: 'string' },
   ],
   measures: [{ name: 'row_count', aggregate: 'count' }],
 };
@@ -130,7 +154,7 @@ for (const cell of CELLS) {
 
       const dropTables = async () => {
         if (cell.id === 'sqlite') return;
-        await driver?.execute(`drop table if exists ${OBJECT}`).catch(() => {});
+        for (const table of [OBJECT, ACCOUNT]) await driver?.execute(`drop table if exists ${table}`).catch(() => {});
       };
 
       beforeAll(async () => {
@@ -139,8 +163,10 @@ for (const cell of CELLS) {
         engine = new ObjectQL({ logger: quiet } as any);
         engine.registerDriver(driver, true);
         await engine.init();
+        engine.registry.registerObject(ACCOUNT_OBJECT as any);
         engine.registry.registerObject(LEDGER as any);
         await engine.syncSchemas();
+        for (const row of ACCOUNTS) await engine.insert(ACCOUNT, { ...row } as any);
         for (const row of ROWS) await engine.insert(OBJECT, { ...row } as any);
 
         const realExecute = (engine as any).execute.bind(engine);
@@ -200,6 +226,18 @@ for (const cell of CELLS) {
         expect(reads, 'no raw SQL and no engine aggregate for the object').toEqual(before);
       });
 
+      it('a dataset dimension over an included relationship\'s json field answers the same 400, naming the joined object — no statement reaches the engine', async () => {
+        const before = { ...reads };
+        const res = await query({ measures: ['row_count'], dimensions: ['acct_hq'] });
+        expect(res.status, JSON.stringify(res.body)).toBe(400);
+        expect(res.body.code).toBe('INVALID_FIELD');
+        expect(String(res.body.message)).toContain(
+          `Dimension 'acct_hq' on cube '${DATASET.name}' groups by field 'account.hq', whose column 'hq' the joined object '${ACCOUNT}' declares as json`,
+        );
+        expect(String(res.body.message)).toContain(ROUTE);
+        expect(reads, 'no raw SQL and no engine aggregate for the object').toEqual(before);
+      });
+
       it('CONTROL a text dataset dimension is served unchanged: one group per value, counted', async () => {
         const before = { ...reads };
         const res = await query({ measures: ['row_count'], dimensions: ['title_dim'] });
@@ -209,6 +247,13 @@ for (const cell of CELLS) {
           .sort(([a], [b]) => a.localeCompare(b));
         expect(groups).toEqual([['x', 2], ['y', 1]]);
         expect(reads.rawSql - before.rawSql, 'the native strategy answered').toBeGreaterThanOrEqual(1);
+
+        const joined = await query({ measures: ['row_count'], dimensions: ['acct_name'] });
+        expect(joined.status, JSON.stringify(joined.body)).toBe(200);
+        const joinedGroups = (joined.body.rows as Array<{ acct_name: string; row_count: number | string }>)
+          .map((r) => [r.acct_name, Number(r.row_count)] as const)
+          .sort(([a], [b]) => a.localeCompare(b));
+        expect(joinedGroups).toEqual([['A', 2], ['B', 1]]);
       });
     },
   );

packages/services/service-analytics/src/__tests__/dimension-structured-json-door.test.ts

@@ -42,12 +42,20 @@ const silentLogger = {
 
 const OBJECT = 'ledger';
 
-/** One field per `FieldType`, named `f_<type>`, plus the text control `title`. */
+/** One field per `FieldType`, named `f_<type>`, plus the text control `title` and a lookup to `account`. */
 const FIELDS: Record<string, { type: string }> = {
   title: { type: 'text' },
+  account: { type: 'lookup' },
   ...Object.fromEntries(FieldType.options.map((type) => [`f_${type}`, { type }])),
 };
 
+/** The joined object: a text column and a json one. */
+const JOINED = 'account';
+const JOINED_FIELDS: Record<string, { type: string }> = {
+  name: { type: 'text' },
+  hq: { type: 'json' },
+};
+
 /** An authored cube: a text dimension, a json one filed under a key that is not its column, and a time one. */
 const LEDGER_CUBE: Cube = {
   name: 'ledger_cube',
@@ -69,9 +77,12 @@ const LEDGER_DATASET = {
   name: 'ledger_ds',
   label: 'Ledger dataset',
   object: OBJECT,
+  include: ['account'],
   dimensions: [
     { name: 'title_dim', field: 'title', type: 'string' },
     { name: 'meta_doc', field: 'f_json', type: 'string' },
+    { name: 'acct_name', field: 'account.name', type: 'string' },
+    { name: 'acct_hq', field: 'account.hq', type: 'string' },
   ],
   measures: [{ name: 'row_count', aggregate: 'count' }],
 } as unknown as Dataset;
@@ -106,7 +117,7 @@ function makeService(face: Face, opts: { sourceFieldMeta?: boolean } = {}) {
     getObjectFieldNames: (n: string) => (n === OBJECT ? Object.keys(FIELDS) : undefined),
     ...(opts.sourceFieldMeta === false
       ? {}
-      : { sourceFieldMeta: (o: string, f: string) => (o === OBJECT ? FIELDS[f] : undefined) }),
+      : { sourceFieldMeta: (o: string, f: string) => (o === OBJECT ? FIELDS[f] : o === JOINED ? JOINED_FIELDS[f] : undefined) }),
   });
   return { service, calls };
 }
@@ -182,6 +193,17 @@ describe('a dimension on a structured-JSON field is refused at the analytics doo
     expect(calls.raw).toEqual([]);
   });
 
+  it('a dataset dimension over an included relationship is judged on the JOINED object — naming the dataset dimension', async () => {
+    const { service, calls } = makeService('native');
+    const err = await rejection(service.queryDataset(LEDGER_DATASET, { measures: ['row_count'], dimensions: ['acct_hq'] }));
+    expect(envelopeOf(err)).toEqual({ code: 'INVALID_FIELD', status: 400, member: 'acct_hq', param: 'dimensions', field: 'account.hq', object: JOINED });
+    expect(err.message).toContain(`Dimension 'acct_hq' on cube 'ledger_ds' groups by field 'account.hq', whose column 'hq' the joined object '${JOINED}' declares as json`);
+    expect(calls.raw).toEqual([]);
+    // CONTROL the joined text column is served.
+    await service.queryDataset(LEDGER_DATASET, { measures: ['row_count'], dimensions: ['acct_name'] });
+    expect(calls.raw).toHaveLength(1);
+  });
+
   it('an ad-hoc query (no cube registered under the object name) is refused the same way', async () => {
     const { service, calls } = makeService('native');
     const err = await rejection(service.query({ cube: OBJECT, measures: ['count'], dimensions: ['f_json'] }));
@@ -211,6 +233,13 @@ describe('what the door does not judge', () => {
     expect(err.message).toContain("which object 'ledger' does not have");
   });
 
+  it('a dotted path the cube declares no join for is a synthetic traversal: its object is not a declaration, so it is not judged', async () => {
+    const { service, calls } = makeService('native');
+    await service.generateSql({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] });
+    await service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] });
+    expect(calls.raw).toHaveLength(1);
+  });
+
   it('a host that wires no sourceFieldMeta cannot name the type, so the door stands down', async () => {
     const { service, calls } = makeService('native', { sourceFieldMeta: false });
     await service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['doc'] });

packages/services/service-analytics/src/analytics-service.ts

@@ -2511,10 +2511,11 @@ export class AnalyticsService implements IAnalyticsService {
    * (`structured-json-dimension-door.ts`); this method supplies the two
    * answers only the service has.
    *
-   * - The column a member groups by is {@link resolveMemberSource}'s, the
-   *   resolver the dimension source-field gate above reads, with the same
-   *   `'dimension'` kind the strategies' `resolveDimensionSql` /
-   *   `resolveFieldName(…, 'dimension')` resolve by.
+   * - The dimension `sql` a member resolves to is {@link declaredMemberEntry}'s
+   *   (`cube.dimensions` only, the `'dimension'` kind the strategies'
+   *   `resolveDimensionSql` / `resolveFieldName(…, 'dimension')` resolve by),
+   *   and the member itself when the cube declares none — the column the
+   *   strategies group by in that case.
    * - Its declared type is {@link AnalyticsServiceConfig.sourceFieldMeta}'s.
    *
    * Runs after the dimension source-field gate on every `ensureCube` path, so a
@@ -2531,7 +2532,11 @@ export class AnalyticsService implements IAnalyticsService {
       query,
       cube,
       object,
-      (member) => resolveMemberSource(cube, member, 'dimension').source,
+      (member) => {
+        const entry = declaredMemberEntry(cube, member, 'dimension');
+        if (!entry) return member;
+        return typeof entry.sql === 'string' ? entry.sql : '';
+      },
       (o, field) => fieldMeta(o, field)?.type,
     );
   }