Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .changeset/20890-dataset-dimension-json-stored-refused.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
"@objectstack/lint": minor
---

fix(lint)!: `os validate`, `os build` and `os lint` refuse a dataset dimension over a JSON-stored field, the group key the analytics door already refuses at query time

Clause-②: yes (narrowing)

<!-- adr-0087: not-required (no-migration-prescription) a refusal of a grouping TARGET at authoring time: a dataset dimensions entry whose field resolves to a declared structured-JSON field (json, composite, repeater, record, location, address, vector) or multi-value field (multiselect, checkboxes, tags, or a select, radio, lookup, user, file or image declared multiple: true). It is the authoring leg of the analytics door that already refuses every query grouping by such a column with 400 INVALID_FIELD, and that door's own changeset declared this category for this surface. No authorable key, spelling, export or stored shape moves: DatasetSchema keeps parsing every dimension, no stored row is read or rewritten, and which scalar part of a document, or which member of a list, an author meant to group on is not something a ledger entry can rewrite. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a grouping target, and dataset-measure-aggregate-field-type-refused says in its own reason that a field used as a DIMENSION is untouched (not already-registered); and the change is a rule verdict, not a declaration (not runtime-interface-only or type-surface-only). -->

**BREAKING**: metadata that passed `os validate`, `os build` and `os lint` can now fail, and so can a runtime dataset save (Studio, REST `/meta`, MCP), which runs the same rule. A dataset dimension is a group key, and the analytics door refuses a query that groups by a JSON-stored column with `400 INVALID_FIELD` before any SQL is built, so such a dimension could be declared but never served. The new rule `dimension-json-stored-field-refused` (gating, `error`) refuses it where the author writes it. It ships as `minor` under the launch-window convention for accept-set narrowings. No export is removed. The package entry exports the new id as `DIMENSION_JSON_STORED_FIELD_REFUSED`, beside `MEASURE_AGGREGATE_FIELD_TYPE_REFUSED`, and the `rule` member of `DatasetMeasureAggregateFinding` gains it.

**What is refused.** A dimension whose `field` resolves, on the dataset's object or across its join chain, to a field declared with a structured-JSON type (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) or a multi-value declaration (`multiselect`, `checkboxes`, `tags`, or a `select`, `radio`, `lookup`, `user`, `file` or `image` declared `multiple: true`). The two classes are `@objectstack/spec/data`'s `STRUCTURED_JSON_TYPES` and `isMultiValueField`, the predicates the analytics door reads.

**What an author sees now.** The finding names the dataset, the dimension, the field, the object that declares it and its declaration, and says the analytics door refuses every query that groups by it. It names the route: group by a field that stores one scalar value, storing the part of the document you group on in a field of its own; for a multi-value field, filter by one member with `$contains` in a record query, one query per member. It is located at `datasets[N].dimensions[M].field`, name-keyed on the runtime wire.

**Unchanged.** A dimension over any other field, a single-value `select` or `lookup` included; a dimension whose field does not resolve (`dataset-field-unknown` reports that) or declares no type; a dataset over an object this stack does not define; measures, filters and every other position. A cube dimension (`analyticsCubes`) is not judged: no authoring rule reads cubes.
17 changes: 17 additions & 0 deletions .changeset/20890-dataset-distinct-multiple-refused.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
"@objectstack/lint": minor
---

fix(lint)!: a dataset `count_distinct` measure over a field declared `multiple: true` is refused by `measure-aggregate-field-type-refused`, as the compile leg and the engine already refuse it

Clause-②: yes (narrowing)

<!-- adr-0087: not-required (already-registered dataset-measure-aggregate-field-type-refused) the registered entry carries this family's hand-migration, an aggregate the field accepts with count as the one that stays for a JSON-stored field, and the count_distinct narrowing over the JSON-stored types already rides it. A select, radio, lookup, user, file or image field declared multiple: true is the one JSON-stored shape the per-type table cannot see; the dataset compile leg and the engine's count_distinct door refuse it beside the table's row, and this is the authoring leg of that same pair, so it adds no surface of its own. -->

**BREAKING**: metadata that passed `os validate`, `os build` and `os lint` can now fail, and so can a runtime dataset save, which runs the same rule. `measure-aggregate-field-type-refused` reads the field's declaration, not its type alone: `count_distinct` over a `select`, `radio`, `lookup`, `user`, `file` or `image` field declared `multiple: true` is refused, because that field is a list stored as JSON and no two backends compare such values for equality alike. The dataset compile leg already answers the pair `400 DATASET_INVALID`, and the engine's `count_distinct` door answers it `400 INVALID_FIELD`. It ships as `minor` under the launch-window convention for accept-set narrowings.

**What an author sees now.** The finding names the measure, the field, the object and the declaration with its flag (`select` with `multiple: true`), and says the aggregate accepts its row's types, none of them with `multiple: true`. The hint names the aggregates the declaration does accept, read from the same predicate: `count`.

**What to write instead.** `count` over the field, or `count_distinct` over a field that stores one scalar value. To count the records holding one member, filter by it with `$contains` in a record query.

**Unchanged.** `count_distinct` over the same types without the flag; `count` over any field; every other aggregate, whose rows accept no multi-capable type and whose verdicts therefore do not move; and the skips the rule already had.
18 changes: 16 additions & 2 deletions content/docs/deployment/validating-metadata.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -210,12 +210,26 @@ no such function and fails at query time. `min`/`max` over the same field are
accepted over every type, because it reads no value, and `count_distinct` over
every type but the JSON-stored ones (the structured-JSON types and
`multiselect` / `checkboxes` / `tags`), whose values no two backends compare
for equality alike. The analytics service refuses the same pair with
`400 DATASET_INVALID` when a query is built; this is the identical verdict,
for equality alike. The check reads the field's declaration as well as its
type: a `select`, `radio`, `lookup`, `user`, `file` or `image` field declared
`multiple: true` holds a list stored as JSON, so `count_distinct` over it is
refused too, although the table accepts the type. The analytics service
refuses the same pair with `400 DATASET_INVALID` when a query is built; this
is the identical verdict,
from the identical table, one door earlier. The rule stays silent wherever the
field's type cannot be resolved (an object this stack does not define, a
dangling field path, an untyped field) rather than guessing.

The same check judges the dataset's **dimensions**, under its own id. A
dimension is a group key, and the analytics service refuses a query grouped by
a JSON-stored column with `400 INVALID_FIELD` before any SQL is built. So a
dimension whose field is declared with a structured-JSON type, or as a
multi-value field (`multiselect` / `checkboxes` / `tags`, or one of the types
above declared `multiple: true`), is refused
(`dimension-json-stored-field-refused`) at `datasets[N].dimensions[M].field`.
Group by a field that stores one value instead. Like the measure check, it
stays silent when the field's type cannot be resolved.

### 7. Navigation exposing objects nobody can read

Navigation and permissions are separate metadata, each valid on its own — so an
Expand Down
1 change: 1 addition & 0 deletions packages/lint/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ export type { WidgetBindingFinding, WidgetBindingSeverity } from './validate-wid
export {
validateDatasetMeasureAggregates,
MEASURE_AGGREGATE_FIELD_TYPE_REFUSED,
DIMENSION_JSON_STORED_FIELD_REFUSED,
} from './validate-dataset-measure-aggregates.js';
export type { DatasetMeasureAggregateFinding } from './validate-dataset-measure-aggregates.js';

Expand Down
Loading
Loading