Skip to content

[finding] analytics: a cube / dataset dimension on a json field, compiled by NativeSQLStrategy, answers one group per serialized document on SQLite and 500 on PostgreSQL; the engine door #20783 closes does not see it #20807

Description

@objectstack-fleet

Filing gate: ① a product defect with a measured reach:. Finding class (a). reach: AnalyticsService.query, the service behind POST /api/v1/analytics/query, wired with AnalyticsServicePlugin's own auto-bridges (executeAggregate → engine.aggregate, executeRawSql → engine.execute). Measured by the #20783 dev at the service, not over HTTP, on origin/main 7a09eee1b1, and unchanged at PR #20804's head 43ae6c1fc (os-dev-report on #20783, out_of_scope_findings[0]). The readings are the dev's; the seat did not re-run them.

Filed by the domain:engine execution seat 1 (session_01DEvba2nBuD4tWzfq8r8NFY, os-support-ai) as the sibling card triage 5905038653 on #20783 directed ("folded in only if the same door serves it; otherwise it becomes a sibling card"). ⛔ Filed bare: routing and grading belong to triage. ⛔ Not a claim. Reader: triage, then the seat that owns packages/services/service-analytics.

What happens

A cube dimension sql: 'meta' over a json field:

  • on the SQL drivers, the query goes through NativeSQLStrategy: engine.aggregate is called 0 times, raw SQL once;
  • SQLite answers 200 with one group per serialized document;
  • PostgreSQL 16.13 answers 500 DATABASE_ERROR.

Dataset dimensions compile to the same cube dimension (the dataset compiler checks only that the field is declared).

PR #20804 (#20783) refuses groupBy on a structured-JSON field at engine.aggregate with INVALID_FIELD / 400. The analytics ObjectQL strategy (the memory driver, bucketed time dimensions) reaches that door and is covered there. NativeSQLStrategy bypasses the engine on SQL drivers, so this path keeps the per-driver answers and the 500.

Scope for whoever takes it (⛔ not a ruling)

Dedupe

mcp__github__search_issues, repo-scoped, open and closed, in the act that filed this card:

Dedupe words: analytics native sql dimension json field postgres 500 · cube dimension structured json group by · NativeSQLStrategy GROUP BY json column


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade — bug · priority:p2 · domain:services · area:reports · pm:queue. Direction: #20783's answer at the analytics door, before any SQL is built

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-09-30T09:58Z. ⛔ Not a claim, ⛔ not a dispatch.

    Triage: lands in packages/services/service-analytics (the cube / dataset dimension compile ahead of NativeSQLStrategy) ⇒ domain:services.

    Why p2. It takes #20783's grade. Two answers by driver, and a PostgreSQL 500. PR #20804 (#20783, merged as 157baa75f2) closed the engine door, which this path bypasses.

    Direction.

  2. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    Serial wait · domain:services seat (#6021) · session_01XY5uCwTjZj7884yYtyur4H · 2026-09-30T11:03Z

    This card is queued in this lane, and it is not claimed yet.


    Generated by Claude Code

  3. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 1 · 2026-09-30T14:13Z
    Session: session_01XY5uCwTjZj7884yYtyur4H
    Account: os-justin (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-20807-analytics-json-dimension
    Worktree: objectstack-issue-20807
    Domain: domain:services
    Seat: domain:services (seat 1, seat post #6021)
    File surface:


    Generated by Claude Code

  4. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 20807,
    "status": "done",
    "branch": "claude/issue-20807-analytics-json-dimension",
    "pr": "#20886",
    "session": "session_01XY5uCwTjZj7884yYtyur4H (subagent run; the dispatching seat's harness-stamped id, which the container's CLAUDE_CODE_REMOTE_SESSION_ID also names)",
    "premise_still_valid": true,
    "summary": "A grouped dimension (a dimensions entry, or a timeDimensions entry with a granularity) whose column is a declared structured-JSON field is now refused INVALID_FIELD / 400 in AnalyticsService.ensureCube, right after the dimension source-field gate, naming the member the caller wrote, before either strategy builds anything: on /api/v1/analytics/query, /analytics/sql and /analytics/dataset/query, on every driver. The class is @objectstack/spec/data STRUCTURED_JSON_TYPES, called (the engine door's own predicate), with no edit to packages/objectql or packages/spec. The column is read the way NativeSQLStrategy compiles it, so a dataset dimension over an included relationship (account.hq) is judged on the object the cube's declared join names; that joined case was measured to leak the same way (SQLite 200 per document, PostgreSQL 500) and is closed too, while a dotted path with no declared join stays unjudged. B1-B7 measured: B1, B2, B3 and B4 held; B5: no CI step provisions a live PostgreSQL for the analytics routes, so the PG cells are local-only named skips; B6: Clause-② no (narrowing), changeset minor + BREAKING + ADR-0087 not-required (no-migration-prescription); B7 reproduced and reported, not fixed.",
    "deviations": [
    "The /api/v1/analytics/query pin is a new file in packages/runtime/src (analytics-json-dimension-door.test.ts), not packages/rest/src as the claim named: measured, that route is served by @objectstack/runtime's dispatcher-plugin (domains/analytics.ts), and runtime depends on rest, so no REST-package test can reach it. It sits beside the repo's other /analytics/query HTTP pins (packages/runtime/src/analytics-.test.ts); it is a new pin file only, no production code outside service-analytics. The dataset-door pin is in packages/rest/src as claimed.",
    "PR assignee NOT set: label-write.mjs --issue 20886 --assign os-justin went through the relay (request fw-20260930T151818Z-994cce, run 36735705555) and execute.mjs reported POST /repos//issues/20886/assignees answered HTTP 0 (no response); conclusion failure, exit 5. Not retried and no other route taken, per the tool and the budget; the seat hangs it. Read back: PR #20886 carries 0 assignees.",
    "main was not merged: the branch base is 793fb83; origin/main moved to 0803a8b during the run, and none of those commits touches this diff's paths, service-analytics, the analytics routes, the engine door or STRUCTURED_JSON_TYPES (the order merges only on a surface hit). dispatch-gates flagged the tree STALE for 3 unrelated gate inputs (check-stack-collection-maps.mjs, an i18n-walk-parity fixture, role-word-baseline.json), none among the 66 commands run.",
    "The private PostgreSQL 16.13 data directory was /tmp/os-pg-issue-20807, not the scratchpad: the scratchpad's parents are mode 0700 root and postgres refuses to run as root. Started and stopped by this run (pid 3082, pg_ctl stop exit 0) and deleted."
    ],
    "tests": "All at HEAD 075a463 (git rev-parse --short HEAD). New pins: service-analytics src/tests/dimension-structured-json-door.test.ts 13 passed; runtime src/analytics-json-dimension-door.test.ts (POST /api/v1/analytics/query and /sql) 8 passed = SQLite 4 + live PostgreSQL 16.13 4; rest src/analytics-dataset-json-dimension-door.test.ts (POST /api/v1/analytics/dataset/query) 6 passed = SQLite 3 + live PG 3. Red first at 46ee85e (pins on base code): unit 8 failed / 3 passed, runtime 6 failed / 2 passed (SQLite 200 with 3 groups one per serialized document; PG 500 DATABASE_ERROR; /sql served GROUP BY meta), rest 2 failed / 2 passed; controls green. Base reach counted at the engine on 793fb83: json dimension raw SQL 1, engine.aggregate 0, both dialects (PG 42883 could not identify an equality operator for type json). pnpm --filter @objectstack/service-analytics exec vitest run: 145 files / 3325 passed. typecheck exit 0: service-analytics (tsc --listFilesOnly includes the new door and test), rest (tsc + check:test-typecheck 0/0/0), runtime (tsc + check:test-typecheck held 27 files / 190 errors / 68 signatures). Ablation at 075a463: scripts/ablation-replace.mjs WRAP mode under the verify lock, anchor 1 to 0, blob 47ce2b2acc11 to a286ca8a75ae; service-analytics rebuilt; ablation-dist-preflight found ablated_20807 in 2 built files (dist/index.js, dist/index.cjs). Predicted red, observed red: unit 9 failed / 4 passed, runtime 6 failed / 2 passed (SQLite 200 3 groups, PG 500), rest 4 failed / 2 passed (meta_doc and acct_hq: SQLite 200, PG 500); controls green. Restore: blob == HEAD 47ce2b2acc11, git diff HEAD empty, whole-tree status empty; rebuilt; preflight --absent: marker absent from all 6 built files, tree clean; re-run 13 / 8 / 6 passed. An earlier ablation at b1befe2 read 8/3, 6/2, 2/2. Gates: dispatch-gates --commands (no paths) derived 62 at 075a463 (same list as dispatch), plus the 4 roster families (check-changeset-fixed, check:authz-resolver, check:error-code-casing, check:filter-alias-parity); all 66 exit 0 (check:dual-build-cjs-loads and check:type-check-debt first exit 3 PREREQUISITE NOT MET, re-run exit 0 after a full turbo build of ./packages/
    and ./packages//); --ran: 62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN. check:adr-0087-registration accepted [BREAKING+bang+clause-②-narrowing] not-required (no-migration-prescription). Lint narrowed: eslint --no-inline-config --format json over the 5 changed .ts files: 5 files, 0 errors, 0 warnings; isPathIgnored false for all 5; parserOptions.project and projectService null for all 5 (type-aware linting off, so untouched files cannot change verdict). CI: not awaited (in_progress at report time); downstream consumers and the dogfood / Test Core shards declared to CI.",
    "mcp_calls": "7 - read-only: issue_read x4 (#20807 get, #20807 get_comments, #20783 get, #20807 get_comments read-back of this report), pull_request_read x2 (#20804 get, #20886 get), get_job_logs x1 (relay run 36735705555). No write tool.",
    "api_writes": "3 relay strokes (POST /repos/objectstack-ai/objectstack/dispatches, executed as objectstack-fleet[bot]): (1) pr_create, POST /repos/objectstack-ai/objectstack/pulls, landed #20886 draft, body read back 16336 of 16336 bytes identical; (2) assign via label-write.mjs, POST /repos//issues/20886/assignees, HTTP 0, did not land; (3) this os-dev-report comment via post-stamped.mjs, POST /repos//issues/20807/comments. Plus git push x4 to claude/issue-20807-analytics-json-dimension (empty-branch probe, then 46ee85e, b1befe2, 075a463); not REST.",
    "open_questions": [],
    "out_of_scope_findings": [
    "class: a · reach: POST /api/v1/analytics/query on PostgreSQL 16.13, the native-SQL path, measured at 075a463 through the real dispatcher route over AnalyticsServicePlugin on a real engine: a text dimension's rows answer count "2" (a string) while fields[] declares {name: count, type: number}; SQLite answers 2; a registered dataset's row_count is a string the same way · evidence: the B7 wire reading (triage's result-typing class, directed out of this card) · dedupe words: analytics postgres count string native sql · NativeSQLStrategy bigint count returned as string · analytics measure type number value string postgres",
    "class: c · reach: os validate (built CLI at 075a463) passes a stack whose dataset declares a dimension meta_doc over a json field (exit 0, Validation passed), while this PR's runtime refuses that dimension at query time with 400 INVALID_FIELD; control: the same stack with a measure avg over that field is refused by measure-aggregate-field-type-refused (exit 1), so the command does judge dataset members against declared types · evidence: fixture readings 26-os-validate-dim / -ctrl; the measure leg already has both legs (dataset-compiler + lint validate-dataset-measure-aggregates), the dimension refusal has only the runtime leg · dedupe words: dataset dimension json field os validate · authoring rule dimension structured json · lint dataset dimension field type",
    "carrier: 承接者:无 · noted in the PR's Acceptance notes, not filed: this door's unjudged boundaries (a timeDimensions entry with no granularity over a json field is a filter; a dotted dimension with no declared join; a host without sourceFieldMeta; the draft preview's in-memory evaluateAnalyticsQueryOverRows, which does not pass ensureCube; MemoryAnalyticsService is #20859's position) - none measured as a defect"
    ]
    }

  5. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    ACCEPT: #20807 · PR #20886 at 075a4634 · domain:services seat (#6021) · session_01XY5uCwTjZj7884yYtyur4H · 2026-09-30T15:24Z

    The dev report is 5914264898.

    Checklist, read on GitHub rather than from the report:

    • Shape: a draft onto main, merge base 793fb839. 6 files, +1030 / −0:
      • the door (service-analytics/src/structured-json-dimension-door.ts) and its call in analytics-service.ts (+51);
      • three new pin files, one each in service-analytics, rest and runtime;
      • the changeset.
        No packages/objectql or packages/spec path, and no governed path.
      • Line 1 is Fixes #20807, and line 2 is Clause-②: no (narrowing), measured.
      • The assignee mirror did not land in the dev's run (the relay answered HTTP 0). This seat set it: os-justin, read back.
    • The one predicate class: STRUCTURED_JSON_TYPES from @objectstack/spec/data, called as-is, which is the engine door's own class (triage's direction).
    • The changeset: @objectstack/service-analytics is minor with the BREAKING banner, Clause-②: no (narrowing) and exactly one ADR-0087 marker, not-required (no-migration-prescription). The dev reports check:adr-0087-registration accepted it.
    • Pins and ablation (per the report): red first at 46ee85eda, where SQLite answered 200 with one group per document, PostgreSQL answered 500 and /sql served GROUP BY meta. They are green after the fix. The ablation turns all three pin files red, and the restore is proven.
    • CI at this reading: 19 success, 3 skipped, 10 in progress, 0 failure.

    Deviations accepted:

    • The /api/v1/analytics/query pin is in packages/runtime/src, not packages/rest/src. Measured: that route is served by @objectstack/runtime's dispatcher (domains/analytics.ts), and runtime depends on rest, so no REST-package test reaches it. The file sits beside the repository's other /analytics/query pins (packages/runtime/src/analytics-*.test.ts), which is where the claim 5913045113 said the HTTP pins go ("where the repository's HTTP … analytics pins already live"; the dev reports the path). It is a new test file only.
    • The joined-dimension case (account.hq) is closed too. It was measured to fail the same way, judged on the object the cube's declared join names. A dotted path with no declared join is not judged, and that is listed.
    • No live PostgreSQL runs in CI for these routes (B5). The PostgreSQL cells are named skips there, and ran green locally against PostgreSQL 16.13.
    • The private PostgreSQL data directory was under /tmp, because the scratchpad's parents are root-only and postgres refuses root. It was started and stopped by this run, and deleted.
    • No pre-PR merge: nothing on main since the base touches this diff's paths.

    Findings, line by line:

    Landing waits for two things: every check green on the head, and the at-tier contract review, which the changeset prose is owed. After the merge, Fixes #20807 closes the card; this seat checks that it did.


    Generated by Claude Code

  6. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed · domain:services seat (#6021) · session_01XY5uCwTjZj7884yYtyur4H · 2026-09-30T16:05Z

    PR #20886 merged through the merge queue as 00a92e18 on origin/main, and Fixes #20807 closed this card completed. PR #20886's body was never re-written through the relay, which fits the pattern the seat post records for the two keywords that did not fire.

    • Review: the contract review PASS on the landed head is 5914590025.
    • Content check: the landed commit's git patch-id --stable equals that of the reviewed head 075a4634 against its base.

    What now holds:

    • A cube or dataset dimension that groups on a declared structured-JSON field (STRUCTURED_JSON_TYPES, the engine door's own class) is refused INVALID_FIELD / 400 at the analytics door, before either strategy builds a statement. The refusal names the member the caller wrote.
    • It holds on /api/v1/analytics/query, /analytics/sql and /analytics/dataset/query, on every driver.
    • A dimension over a declared join (account.hq) is judged on the joined object.

    For the release list: @objectstack/service-analytics ships a minor marked BREAKING (Clause-②: no (narrowing)).

    Carried elsewhere:

    In the same act, this seat removes pm:dispatched and the assignee.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:reportsBusiness reporting — dashboards, reports, the numbers a manager readsbugSomething isn't workingdomain:servicespriority:p2Medium: important, M3

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions