diff --git a/.changeset/21232-json-door-undeclared-join-hop.md b/.changeset/21232-json-door-undeclared-join-hop.md new file mode 100644 index 00000000000..7906552ab75 --- /dev/null +++ b/.changeset/21232-json-door-undeclared-join-hop.md @@ -0,0 +1,26 @@ +--- +"@objectstack/service-analytics": minor +--- + +fix(service-analytics)!: a grouped dimension or a `count_distinct` measure over a JSON-stored column reached through a relationship path the cube declares no join for is refused with `INVALID_FIELD` / 400 at the analytics door, as the same member over a declared join already was + +Clause-②: no (narrowing) + + + +**BREAKING**: this narrows what `POST /api/v1/analytics/query` and its dry run `POST /api/v1/analytics/sql` accept, on both strategies and every driver. It ships as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes. + +The column of a dotted path is now located by the one hop resolver both strategies join and read it through: the cube's declared join at that path, else the relationship field's declared `reference`, else the relationship's own name for a host that cannot answer. Before, the door read the cube's declared joins alone and stood down on a path the cube declares no join for. This reverses one clause of the earlier entries for this door in the same release, which listed such a path as unchanged. + +**Before and after**, measured on a configured cube over an object whose lookup the cube declares no join for — `owner`, declaring `reference` a person object — and a member over that lookup. An ad-hoc query's inferred cube declares no join at all, and a dotted dimension on it (`owner.prefs`) now gets the same refusal: + +- A `dimensions` entry, or a `timeDimensions` entry with a `granularity`, over a structured-JSON field (`owner.prefs`, `json`) or a multi-value field (`owner.labels`, `tags`; or a field declared `multiple: true`). Before, on the native-SQL strategy: `200` with one group per serialized value on SQLite and `500 DATABASE_ERROR` on PostgreSQL; the ObjectQL strategy answered `400 INVALID_FIELD` from the engine under its own position (`groupBy[1]`), a name the request never wrote. Now: `400 INVALID_FIELD` from this door on both strategies, before either reads anything. +- A `count_distinct` measure over the same columns. Before, on the native-SQL strategy: `200` with a count of serialized values on SQLite and `500` on PostgreSQL; the ObjectQL strategy refused it as a cross-object measure. Now: the same `400 INVALID_FIELD` from this door. + +**What an author sees now.** The refusal the same member over a declared join already got: `400 INVALID_FIELD`, naming the member as the request wrote it, the cube, the path, the object the lookup declares as its target and the column's declared type, saying the query was not run, and naming the route. The thrown error carries `member`, `param` (`dimensions`, `timeDimensions` or `measures`), `cube`, `field` (the path, `owner.prefs`) and `object` (the target object). + +**What to write instead.** Group by, or count distinct, a related field that stores one scalar value. For a multi-value field, run a record query on the target object filtered by one member with `$contains`, one query per member. + +**Who is affected.** A dashboard, report or caller that grouped or counted distinct such a related column through a lookup the cube declares no join for, on SQLite, and read the serialized values as real groups or a real count. On PostgreSQL the same queries were already a 500. No example app and no shipped cube or dataset authors such a member. + +**Unchanged.** Every member over a declared join; a scalar related column (`owner.email`), which is served on both strategies; a related column whose object the host's field metadata does not describe; an expression `sql`; a host that wires no `sourceFieldMeta`; and a dataset dimension over an `include`d relationship, whose join the dataset compiler declares. diff --git a/packages/services/service-analytics/src/__tests__/dimension-structured-json-door.test.ts b/packages/services/service-analytics/src/__tests__/dimension-structured-json-door.test.ts index 0ead1cec4be..47aa3caeb65 100644 --- a/packages/services/service-analytics/src/__tests__/dimension-structured-json-door.test.ts +++ b/packages/services/service-analytics/src/__tests__/dimension-structured-json-door.test.ts @@ -20,6 +20,10 @@ * column, a cube-qualified spelling, a dataset dimension, an ad-hoc * (inferred) cube, and a bucketed time dimension; * - no statement and no aggregate reaches the host bridges on a refusal; + * - [#21232] a relationship path the cube declares no join for is judged on + * the object the one hop resolver (`hop-object.ts`) names — the field's + * declared reference, else the alias — and stands down only where the host + * describes nothing there; * - GUARD: the judged types are exactly `@objectstack/spec/data`'s * `STRUCTURED_JSON_TYPES` and `isMultiValueField`, over every `FieldType` — * the predicates the engine's door reads, never a second list. @@ -99,7 +103,16 @@ interface Refusal extends Error { cube?: string; } -function makeService(face: Face, opts: { sourceFieldMeta?: boolean } = {}) { +/** + * `describes`: the object whose fields the host describes as {@link JOINED_FIELDS} + * (`null`: none). `reference`: the target a relationship resolver declares for + * `ledger.account` (absent: no resolver is wired). + */ +function makeService( + face: Face, + opts: { sourceFieldMeta?: boolean; describes?: string | null; reference?: string } = {}, +) { + const described = opts.describes === undefined ? JOINED : opts.describes; const calls = { raw: [] as string[], aggregate: [] as unknown[] }; const service = new AnalyticsService({ logger: silentLogger, @@ -117,7 +130,10 @@ function makeService(face: Face, opts: { sourceFieldMeta?: boolean } = {}) { getObjectFieldNames: (n: string) => (n === OBJECT ? Object.keys(FIELDS) : undefined), ...(opts.sourceFieldMeta === false ? {} - : { sourceFieldMeta: (o: string, f: string) => (o === OBJECT ? FIELDS[f] : o === JOINED ? JOINED_FIELDS[f] : undefined) }), + : { sourceFieldMeta: (o: string, f: string) => (o === OBJECT ? FIELDS[f] : o === described ? JOINED_FIELDS[f] : undefined) }), + ...(opts.reference === undefined + ? {} + : { relationshipResolver: (o: string, rel: string) => (o === OBJECT && rel === 'account' ? opts.reference : undefined) }), }); return { service, calls }; } @@ -218,6 +234,52 @@ describe('a dimension on a structured-JSON field is refused at the analytics doo }); }); +describe('[#21232] a dotted path the cube declares no join for is judged on the object the one hop resolver names', () => { + // `ledger_cube` declares no join; `account.hq` is an undeclared member, so + // its column is the path itself. The live-driver pins over the plugin's own + // composition are `json-stored-door-undeclared-join.test.ts`. + const REFERENCED = 'crm_account'; + + for (const face of ['native', 'objectql'] as const) { + it(`${face}: the relationship field's declared reference names the object — INVALID_FIELD / 400, nothing read`, async () => { + const { service, calls } = makeService(face, { describes: REFERENCED, reference: REFERENCED }); + const err = await rejection(service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] })); + expect(envelopeOf(err)).toEqual({ + code: 'INVALID_FIELD', status: 400, member: 'account.hq', param: 'dimensions', field: 'account.hq', object: REFERENCED, + }); + expect(calls.raw, 'no statement reached the raw-SQL bridge').toEqual([]); + expect(calls.aggregate, 'no aggregate reached the engine bridge').toEqual([]); + }); + } + + it('the reference, not the alias: an alias that names a described object is not what the door reads', async () => { + // The host describes `account` (the alias) with `hq` json, but the lookup + // declares `crm_account`, which it does not describe: the strategy joins + // `crm_account`, so the door has nothing to answer and stands down. + const { service, calls } = makeService('native', { describes: JOINED, reference: REFERENCED }); + await service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] }); + expect(calls.raw).toHaveLength(1); + expect(calls.raw[0]).toContain(`"${REFERENCED}"`); + }); + + it('a host that names no reference: the hop reads its alias, the table the strategy joins, and is judged there', async () => { + const { service, calls } = makeService('native'); + const err = await rejection(service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] })); + expect(envelopeOf(err)).toEqual({ + code: 'INVALID_FIELD', status: 400, member: 'account.hq', param: 'dimensions', field: 'account.hq', object: JOINED, + }); + const dry = await rejection(service.generateSql({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] })); + expect(envelopeOf(dry)).toEqual(envelopeOf(err)); + expect(calls.raw).toEqual([]); + }); + + it('CONTROL the referenced object\'s text column is served', async () => { + const { service, calls } = makeService('native', { describes: REFERENCED, reference: REFERENCED }); + await service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.name'] }); + expect(calls.raw).toHaveLength(1); + }); +}); + describe('what the door does not judge', () => { it('CONTROL a text dimension is served — the native face runs its one statement', async () => { const { service, calls } = makeService('native'); @@ -233,8 +295,8 @@ describe('what the door does not judge', () => { expect(err.message).toContain("which object 'ledger' does not have"); }); - it('a dotted path the cube declares no join for is a synthetic traversal: its object is not a declaration, so it is not judged', async () => { - const { service, calls } = makeService('native'); + it('a dotted path whose column the host does not describe cannot be answered, so the door stands down', async () => { + const { service, calls } = makeService('native', { describes: null }); await service.generateSql({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] }); await service.query({ cube: 'ledger_cube', measures: ['count'], dimensions: ['account.hq'] }); expect(calls.raw).toHaveLength(1); diff --git a/packages/services/service-analytics/src/__tests__/json-stored-door-undeclared-join.test.ts b/packages/services/service-analytics/src/__tests__/json-stored-door-undeclared-join.test.ts new file mode 100644 index 00000000000..a4357f3ae19 --- /dev/null +++ b/packages/services/service-analytics/src/__tests__/json-stored-door-undeclared-join.test.ts @@ -0,0 +1,332 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#21232] A grouped dimension and a `count_distinct` measure over a + * JSON-stored column reached through a relationship path the cube declares + * NO join for are refused by the structured-JSON door + * (`structured-json-dimension-door.ts`) — `INVALID_FIELD` / 400, the family's + * one refusal — on both strategy faces and both dialects, exactly as the same + * members over a declared join already were. + * + * ## The shape this closes + * + * Measured through `POST /api/v1/analytics/query` on the real dispatcher route + * at `3a7b6eb0`, SQLite and PostgreSQL 16.14, a configured cube over `deal` + * whose lookup `owner` (`reference` the person object) has no declared join, + * beside `account` (declared join): + * + * | member | face | SQLite | PostgreSQL | + * |:--|:--|:--|:--| + * | dimension over `owner.prefs` (json) / `owner.labels` (tags) | native | 200, one group per serialized value | 500 `DATABASE_ERROR` | + * | the same | ObjectQL | 400, the engine's `groupBy[1]` | same | + * | `count_distinct` over `owner.prefs` / `owner.labels` | native | 200 | 500 | + * | the same | ObjectQL | 400, its cross-object refusal | same | + * | control — the same members over `account.hq` | both | 400 `INVALID_FIELD`, the door | same | + * + * The door located a dotted path's object through `cube.joins` alone and + * stood down where the cube declares no join; the strategies joined the + * lookup's declared `reference` through the one hop resolver (`hop-object.ts`) + * and grouped by its column. + * + * ## What these pins hold + * + * - Every undeclared-join member is refused BEFORE either strategy reads + * anything, with the same envelope on both faces: `code`, `status`, the + * member, the request key, the cube, the path as the member spells it and + * the object the lookup's `reference` declares. Neither face's own refusal + * carries that envelope (the engine's names `groupBy[1]`, the cross-object + * one carries no `field` / `object`), so equal envelopes are the door's. + * - An ad-hoc query's inferred cube declares no join at all: a dotted + * dimension on it is refused the same way. + * - The declared-join members are the control: the same refusal, `object` the + * joined object. + * - The scalar column over the same undeclared-join path is served. + * + * ## The dialect axis of THIS file + * + * The SQLite cell always runs. The PostgreSQL cell runs where + * `OS_TEST_POSTGRES_URL` is set and is a named skip otherwise; no CI step + * provisions that variable for this package, so the live cell is red-capable + * and un-run in CI, and the PR that landed this file carries its local + * PostgreSQL 16 run. The live cell owns its tables, dropped before and after. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { ObjectQL } from '@objectstack/objectql'; +import { SqlDriver } from '@objectstack/driver-sql'; +import type { Cube } from '@objectstack/spec/data'; +import type { AnalyticsService } from '../analytics-service.js'; +import { AnalyticsServicePlugin } from '../plugin.js'; + +const PERSON = 'os21232_person'; +const ACCOUNT = 'os21232_account'; +const DEAL = 'os21232_deal'; + +const PERSON_OBJECT = { + name: PERSON, + label: 'Undeclared-join person', + fields: { + email: { name: 'email', type: 'text' as const }, + prefs: { name: 'prefs', type: 'json' as const }, + labels: { name: 'labels', type: 'tags' as const }, + }, +}; + +const ACCOUNT_OBJECT = { + name: ACCOUNT, + label: 'Undeclared-join account', + fields: { + name: { name: 'name', type: 'text' as const }, + hq: { name: 'hq', type: 'json' as const }, + }, +}; + +const DEAL_OBJECT = { + name: DEAL, + label: 'Undeclared-join deal', + fields: { + note: { name: 'note', type: 'text' as const }, + account: { name: 'account', type: 'lookup' as const, reference: ACCOUNT }, + // Named after neither its target nor anything the cube joins: only the + // field's declared `reference` reaches `os21232_person`. + owner: { name: 'owner', type: 'lookup' as const, reference: PERSON }, + }, +}; + +const PEOPLE = [ + { id: 'p1', email: 'p1@example.com', prefs: { a: 1 }, labels: ['a', 'b'] }, + { id: 'p2', email: 'p2@example.com', prefs: { b: 1 }, labels: ['b'] }, + { id: 'p3', email: 'p3@example.com', prefs: { a: 1 }, labels: ['a', 'b'] }, +] as const; +const ACCOUNTS = [ + { id: 'a1', name: 'alpha', hq: { x: 1 } }, + { id: 'a2', name: 'zeta', hq: { y: 1 } }, +] as const; +const DEALS = [ + { id: 'd1', note: 'x', account: 'a1', owner: 'p1' }, + { id: 'd2', note: 'y', account: 'a2', owner: 'p2' }, + { id: 'd3', note: 'z', account: 'a1', owner: 'p3' }, +] as const; + +const CUBE: Cube = { + name: 'os21232_cube', + title: 'Undeclared-join JSON-stored door cube', + sql: DEAL, + public: true, + // `account` is joined (tier 1); `owner` is not (tier 2, its `reference`). + joins: { account: { name: ACCOUNT } }, + measures: { + count: { type: 'count', sql: '*', label: 'Rows' }, + owner_prefs_distinct: { type: 'count_distinct', sql: 'owner.prefs', label: 'Distinct owner prefs (json)' }, + owner_labels_distinct: { type: 'count_distinct', sql: 'owner.labels', label: 'Distinct owner labels (tags)' }, + owner_email_distinct: { type: 'count_distinct', sql: 'owner.email', label: 'Distinct owner emails (text)' }, + acct_hq_distinct: { type: 'count_distinct', sql: 'account.hq', label: 'Distinct account HQs (json)' }, + }, + dimensions: { + owner_prefs: { type: 'string', sql: 'owner.prefs', label: 'Owner prefs (json)' }, + owner_labels: { type: 'string', sql: 'owner.labels', label: 'Owner labels (tags)' }, + owner_email: { type: 'string', sql: 'owner.email', label: 'Owner email (text)' }, + acct_hq: { type: 'string', sql: 'account.hq', label: 'Account HQ (json)' }, + }, +} as Cube; + +/** A refused member: the request key, the path its `sql` spells, and the object that declares the column. */ +type Refused = readonly [member: string, param: 'dimensions' | 'measures', path: string, object: string]; + +/** Over the lookup the cube declares NO join for — the card's rows. */ +const UNDECLARED: readonly Refused[] = [ + ['owner_prefs', 'dimensions', 'owner.prefs', PERSON], + ['owner_labels', 'dimensions', 'owner.labels', PERSON], + ['owner_prefs_distinct', 'measures', 'owner.prefs', PERSON], + ['owner_labels_distinct', 'measures', 'owner.labels', PERSON], +]; + +/** Over the declared join — the control: the family's existing refusal. */ +const DECLARED: readonly Refused[] = [ + ['acct_hq', 'dimensions', 'account.hq', ACCOUNT], + ['acct_hq_distinct', 'measures', 'account.hq', ACCOUNT], +]; + +interface Cell { + id: 'sqlite' | 'pg'; + label: string; + env: string | null; + config: () => Record | null; +} + +const CELLS: readonly Cell[] = [ + { id: 'sqlite', label: 'sqlite', env: null, config: () => ({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }) }, + { + id: 'pg', + label: 'live postgres', + env: 'OS_TEST_POSTGRES_URL', + config: () => (process.env.OS_TEST_POSTGRES_URL ? { client: 'pg', connection: process.env.OS_TEST_POSTGRES_URL } : null), + }, +]; + +const FACES = ['native', 'objectql'] as const; +type Face = (typeof FACES)[number]; + +const quiet = { debug() {}, info() {}, warn() {}, error() {}, child() { return quiet; } }; + +type Refusal = Error & { code?: string; status?: number; member?: string; param?: string; cube?: string; field?: string; object?: string }; + +/** The envelope a refusal carries, without its words. */ +const envelopeOf = (err: Refusal | undefined) => ({ + code: err?.code, + status: err?.status, + member: err?.member, + param: err?.param, + cube: err?.cube, + field: err?.field, + object: err?.object, +}); + +/** The query that asks for one member under its request key. */ +const queryFor = (member: string, param: 'dimensions' | 'measures') => + (param === 'dimensions' + ? { cube: CUBE.name, measures: ['count'], dimensions: [member] } + : { cube: CUBE.name, measures: [member] }) as any; + +for (const cell of CELLS) { + const config = cell.config(); + describe.skipIf(!config)( + `[#21232] a JSON-stored member over a relationship path the cube declares no join for (${cell.label})${config ? '' : ` (skipped: set ${cell.env} to run this cell)`}`, + () => { + let driver: any; + let engine: ObjectQL; + /** Raw-SQL statements and engine aggregates, on any object. */ + const reads = { rawSql: 0, aggregate: 0 }; + /** `native`: the plugin's own capabilities. `objectql`: narrowed to the engine-aggregate path. */ + const services: Partial> = {}; + + const dropTables = async () => { + if (cell.id !== 'pg') return; + for (const table of [DEAL, ACCOUNT, PERSON]) await driver?.execute(`drop table if exists ${table}`).catch(() => {}); + }; + + /** One query on one face, with the reads it caused counted. */ + const read = async (face: Face, query: unknown) => { + const before = { ...reads }; + const outcome = await services[face]!.query(query as any).then( + (res) => ({ res, err: undefined as Refusal | undefined }), + (err) => ({ res: undefined, err: err as Refusal }), + ); + return { ...outcome, rawSql: reads.rawSql - before.rawSql, aggregate: reads.aggregate - before.aggregate }; + }; + + beforeAll(async () => { + driver = new SqlDriver(config as any); + await dropTables(); + engine = new ObjectQL({ logger: quiet } as any); + engine.registerDriver(driver, true); + await engine.init(); + for (const object of [PERSON_OBJECT, ACCOUNT_OBJECT, DEAL_OBJECT]) engine.registry.registerObject(object as any); + await engine.syncSchemas(); + for (const row of PEOPLE) await engine.insert(PERSON, { ...row } as any); + for (const row of ACCOUNTS) await engine.insert(ACCOUNT, { ...row } as any); + for (const row of DEALS) await engine.insert(DEAL, { ...row } as any); + + const realExecute = (engine as any).execute.bind(engine); + (engine as any).execute = (sql: unknown, opts?: unknown) => { + reads.rawSql += 1; + return realExecute(sql, opts); + }; + const realAggregate = engine.aggregate.bind(engine); + (engine as any).aggregate = (...args: unknown[]) => { + reads.aggregate += 1; + return (realAggregate as any)(...args); + }; + + // The plugin's own composition over the real engine — `sourceFieldMeta` + // and the relationship resolver wired from the engine's registry. + for (const [face, caps] of [ + ['native', undefined], + ['objectql', () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false })], + ] as const) { + const registered: Record = {}; + await new AnalyticsServicePlugin({ cubes: [CUBE], ...(caps ? { queryCapabilities: caps } : {}) } as any).init({ + getService: (name: string) => (name === 'data' ? engine : registered[name]), + registerService: (name: string, svc: unknown) => { registered[name] = svc; }, + replaceService: (name: string, svc: unknown) => { registered[name] = svc; }, + hook: () => {}, + logger: quiet, + } as never); + services[face] = registered.analytics as AnalyticsService; + } + }); + + afterAll(async () => { + await dropTables(); + try { await engine?.destroy(); } catch { /* noop */ } + }); + + const refusedOnBothFaces = async (rows: readonly Refused[]) => { + for (const [member, param, path, object] of rows) { + const envelopes: Record = {}; + for (const face of FACES) { + const { res, err, rawSql, aggregate } = await read(face, queryFor(member, param)); + expect(res, `${face} ${member} must not be served`).toBeUndefined(); + expect(err?.code, `${face} ${member}: ${err?.message}`).toBe('INVALID_FIELD'); + expect(err?.status, `${face} ${member}`).toBe(400); + expect(err?.member, `${face} ${member}`).toBe(member); + expect(err?.param, `${face} ${member}`).toBe(param); + expect(err?.cube, `${face} ${member}`).toBe(CUBE.name); + expect(err?.field, `${face} ${member}: the path as the member spells it`).toBe(path); + expect(err?.object, `${face} ${member}: the object that declares the column`).toBe(object); + expect(rawSql, `${face} ${member}: no raw statement ran`).toBe(0); + expect(aggregate, `${face} ${member}: no engine aggregate ran`).toBe(0); + envelopes[face] = envelopeOf(err); + } + expect(envelopes.objectql, `${member}: one door, one envelope`).toEqual(envelopes.native); + } + }; + + it('a dimension and a count_distinct over the undeclared-join path answer INVALID_FIELD / 400 before anything is read, one envelope on both faces', async () => { + await refusedOnBothFaces(UNDECLARED); + }); + + it('the control — the same members over the declared join answer the same refusal', async () => { + await refusedOnBothFaces(DECLARED); + }); + + it('an ad-hoc query over the object — an inferred cube, which declares no join at all — is refused the same way', async () => { + for (const face of FACES) { + const { res, err, rawSql, aggregate } = await read(face, { cube: DEAL, measures: ['count'], dimensions: ['owner.prefs'] }); + expect(res, `${face} must not be served`).toBeUndefined(); + expect(envelopeOf(err), `${face}: ${err?.message}`).toEqual({ + code: 'INVALID_FIELD', status: 400, member: 'owner.prefs', param: 'dimensions', cube: DEAL, field: 'owner.prefs', object: PERSON, + }); + expect(rawSql + aggregate, `${face}: nothing was read`).toBe(0); + } + }); + + it('the dry-run door refuses what the query door refuses', async () => { + for (const [member, param, path, object] of UNDECLARED) { + const err = await services.native!.generateSql(queryFor(member, param)).then( + () => undefined, + (e) => e as Refusal, + ); + expect(err?.code, `${member}: ${err?.message}`).toBe('INVALID_FIELD'); + expect(err?.status, member).toBe(400); + expect(err?.field, member).toBe(path); + expect(err?.object, member).toBe(object); + } + }); + + it('the scalar column over the same undeclared-join path is served: one group per owner, and its distinct count', async () => { + for (const face of FACES) { + const { res, err } = await read(face, queryFor('owner_email', 'dimensions')); + expect(err, `${face}: ${err?.message}`).toBeUndefined(); + const groups = (res!.rows as Array>) + .map((r) => [String(r.owner_email), Number(r.count)] as const) + .sort(([a], [b]) => a.localeCompare(b)); + expect(groups, face).toEqual([['p1@example.com', 1], ['p2@example.com', 1], ['p3@example.com', 1]]); + } + const { res, err } = await read('native', queryFor('owner_email_distinct', 'measures')); + expect(err, err?.message).toBeUndefined(); + expect(Number(res!.rows[0]!.owner_email_distinct)).toBe(3); + }); + }, + ); +} diff --git a/packages/services/service-analytics/src/analytics-service.ts b/packages/services/service-analytics/src/analytics-service.ts index 34fb71b6048..16d63d721b6 100644 --- a/packages/services/service-analytics/src/analytics-service.ts +++ b/packages/services/service-analytics/src/analytics-service.ts @@ -3245,7 +3245,7 @@ export class AnalyticsService implements IAnalyticsService { * structured-JSON field — `INVALID_FIELD` / 400, naming the member the caller * wrote — before either strategy builds anything. The rule, the * measurements and the envelope are {@link assertNoStructuredJsonDimension}'s - * (`structured-json-dimension-door.ts`); this method supplies the two + * (`structured-json-dimension-door.ts`); this method supplies the three * answers only the service has. * * - The dimension `sql` a member resolves to is {@link declaredMemberEntry}'s @@ -3254,6 +3254,10 @@ export class AnalyticsService implements IAnalyticsService { * and the member itself when the cube declares none — the column the * strategies group by in that case. * - Its declared type is {@link AnalyticsServiceConfig.sourceFieldMeta}'s. + * - [#21232] A relationship-path column is located on the object its last + * hop reaches by the one hop resolver, with {@link hopReference}: the + * answer the field gate admitted the path with and the strategies join it + * by. * * Runs after the dimension source-field gate on every `ensureCube` path, so a * member naming a column the object does not have is answered as that first. @@ -3279,6 +3283,7 @@ export class AnalyticsService implements IAnalyticsService { if (typeof meta?.type !== 'string' || meta.type === '') return undefined; return { type: meta.type, multiple: meta.multiple === true }; }, + this.hopReference, ); } diff --git a/packages/services/service-analytics/src/structured-json-dimension-door.ts b/packages/services/service-analytics/src/structured-json-dimension-door.ts index 1915a5a4205..04bf4095e71 100644 --- a/packages/services/service-analytics/src/structured-json-dimension-door.ts +++ b/packages/services/service-analytics/src/structured-json-dimension-door.ts @@ -52,18 +52,45 @@ * dimension `sql` (the member itself when the cube declares none). A bare * identifier is a column of the cube's object. A dotted identifier path * (`account.hq`, a dataset dimension over an `include`d relationship) is the - * last segment, on the object the cube's own `joins` entry for that path - * names — the alias the dataset compiler registers and the strategy joins. + * last segment, on the object the path's last hop reaches — [#21232] asked + * of the one hop resolver (`hop-object.ts`'s `columnObjectOf`): the cube's + * declared join at that path, else the relationship field's declared + * `reference`, else the alias. That is the object both strategies join and + * read for the path, so the door judges the column the statement groups by. * Measured on the base the same way as the table above: a dataset dimension * over `account.hq` (a `json` field of the joined object) answered one group * per document on SQLite and 500 on PostgreSQL, like a base-object one. * * **Not judged** (the same "cannot answer, do not block" tiering as every * sibling gate in `ensureCube`): a host that wires no `sourceFieldMeta`, a cube - * whose `sql` is not a bare object name, an expression `sql`, a dotted path - * the cube declares no join for (a synthetic traversal: which object it lands - * on is the strategy's guess, not a declaration), and every other field type. - * The same tiers hold for the two #20912 judgements below. + * whose `sql` is not a bare object name, an expression `sql`, a column whose + * object the host does not describe (a hop the host cannot resolve reads its + * alias, and an alias that names no described object answers nothing), and + * every other field type. The same tiers hold for the two #20912 judgements + * below. + * + * ## [#21232] A relationship path the cube declares no join for + * + * Until #21232 the door located a dotted path's object through `cube.joins` + * alone and stood down on a path the cube declares no join for — while the + * strategies, through the one hop resolver, joined the lookup's declared + * target and grouped by its column. Measured on `origin/main` `3a7b6eb0` + * through `POST /api/v1/analytics/query` on the real dispatcher route, a + * configured cube over `deal` whose lookup `owner` (`reference` a person + * object holding `prefs` `json` and `labels` `tags`) has no declared join, + * beside `account` (declared join, `hq` `json`): + * + * | member | face | SQLite | PostgreSQL 16.14 | + * |:--|:--|:--|:--| + * | dimension over `owner.prefs` / `owner.labels` | native | 200, one group per serialized value | **500 `DATABASE_ERROR`** | + * | the same | ObjectQL | 400, the engine's `groupBy[1]` (a position the caller never wrote) | same | + * | `count_distinct` over `owner.prefs` / `owner.labels` | native | 200, 2 / 2 | **500** | + * | the same | ObjectQL | 400, its cross-object refusal (no `field` / `object`) | same | + * | control — the same two members over `account.hq` | both | 400 `INVALID_FIELD`, this door | same | + * + * Every row now answers this door's `INVALID_FIELD` / 400 on both faces and + * both dialects, naming the member, `field` the path and `object` the + * reference's target. * * ## [#20912] The engine's second class, and its second door * @@ -94,7 +121,7 @@ * `isAggregateCompatibleWithFieldType`), or `isMultiValueField` says the * declaration is a list. The member resolves the way the strategies resolve * a measure (cube-qualified, flattened), and its column the way a dimension's - * does (`columnOf`, a declared join included). An authored measure, a + * does (`columnOf`, through the one hop resolver). An authored measure, a * suffix-inferred one (`tags_count_distinct`) and a compiled dataset's are * one population here; a `count` measure compares nothing and is not judged. * @@ -124,6 +151,7 @@ * * @see https://github.com/objectstack-ai/objectstack/issues/20807 * @see https://github.com/objectstack-ai/objectstack/issues/20912 + * @see https://github.com/objectstack-ai/objectstack/issues/21232 */ import type { Cube, ValueShapeFieldDef } from '@objectstack/spec/data'; @@ -136,6 +164,7 @@ import { import type { AnalyticsQuery } from '@objectstack/spec/contracts'; import type { AnalyticsRequestKey } from './dataset-refusal.js'; import { invalidMemberError } from './dataset-refusal.js'; +import { columnObjectOf, type HopReference } from './hop-object.js'; /** The declared `FieldType` vocabulary — the only types the compatibility table can answer for. */ const DECLARED_FIELD_TYPES: ReadonlySet = new Set(FieldType.options); @@ -178,19 +207,28 @@ function groupedMembers(query: AnalyticsQuery): GroupedMember[] { } /** - * The column a dimension `sql` reads, or `null` when this door cannot pin one. - * A dotted path's object is the cube's DECLARED join at that path — the alias - * is the path with its dots as `__`, as the dataset compiler registers it and - * the strategy joins it — and a path with no declared join is not judged. + * The column a dimension `sql` reads, or `null` when this door cannot pin one + * (an expression). [#21232] A dotted path's column is its last segment, on the + * object the path's last hop reaches: {@link columnObjectOf}, the one hop + * resolver (`hop-object.ts`) — the cube's declared join, else the + * relationship field's declared `reference` through `referenceOf`, else the + * alias — which is the object both strategies join and read for that path. + * ⛔ No second walk. */ -function columnOf(cube: Cube, baseObject: string, sql: string): DimensionColumn | null { +function columnOf( + cube: Cube, + baseObject: string, + sql: string, + referenceOf: HopReference | undefined, +): DimensionColumn | null { const path = sql.trim(); if (BARE_IDENTIFIER.test(path)) return { object: baseObject, column: path, path }; if (!IDENTIFIER_PATH.test(path)) return null; - const segments = path.split('.'); - const column = segments.pop() as string; - const joined = (cube.joins as Record | undefined)?.[segments.join('__')]?.name; - return typeof joined === 'string' && joined !== '' ? { object: joined, column, path } : null; + return { + object: columnObjectOf(cube, baseObject, path, referenceOf), + column: path.slice(path.lastIndexOf('.') + 1), + path, + }; } /** @@ -280,6 +318,10 @@ function refusal( * `FieldType` and its `multiple` flag — or `undefined` when nothing * authoritative answers. The flag is half the verdict: a `select` with * `multiple: true` is a list stored as JSON, whatever its type name says. + * @param referenceOf - [#21232] The host's answer for a relationship field's + * declared target — tier 2 of the one hop resolver (`hop-object.ts`), the + * same function the field gate and both strategies resolve a hop with — or + * `undefined` for a host that cannot answer (a hop then reads its alias). * * The words put the verdict first, then that the query did not run, then the * route, then the reason: a door that bounds a 4xx message keeps the front of @@ -291,9 +333,10 @@ export function assertNoStructuredJsonDimension( baseObject: string, sqlOf: (member: string) => string, declaredValueShape: (object: string, field: string) => ValueShapeFieldDef | undefined, + referenceOf: HopReference | undefined, ): void { for (const { member, param } of groupedMembers(query)) { - const target = columnOf(cube, baseObject, sqlOf(member)); + const target = columnOf(cube, baseObject, sqlOf(member), referenceOf); if (!target) continue; const shape = declaredValueShape(target.object, target.column); if (typeof shape?.type !== 'string') continue; @@ -328,7 +371,7 @@ export function assertNoStructuredJsonDimension( for (const member of query.measures ?? []) { const entry = measureEntryOf(cube, member); if (entry?.type !== 'count_distinct' || typeof entry.sql !== 'string') continue; - const target = columnOf(cube, baseObject, entry.sql); + const target = columnOf(cube, baseObject, entry.sql, referenceOf); if (!target) continue; const shape = declaredValueShape(target.object, target.column); if (typeof shape?.type !== 'string') continue;