From f6f904c728b955e97f718f48bf9bba10b0cc71ff Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 07:31:26 -0400 Subject: [PATCH 01/52] docs(spec): design int-backed field.enum values via @intValueMap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Explores the deferred D4 non-goal from the original field.enum design (2026-05-23) — explicit, sparse, per-member integer DB storage. Researched prior art (Rails, protobuf, EF Core, Django, GraphQL, OpenAPI, JPA, Prisma) before locking the shape: a name-keyed @intValueMap object, not a parallel array index-matched to @values, since positional correspondence is the one documented failure mode in the survey (OpenAPI's x-enum-varnames). --- ...026-07-23-int-backed-enum-values-design.md | 229 ++++++++++++++++++ 1 file changed, 229 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md diff --git a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md new file mode 100644 index 000000000..6b1e9033e --- /dev/null +++ b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md @@ -0,0 +1,229 @@ +# Design: int-backed `field.enum` values (`@intValueMap`) + +**Date:** 2026-07-23 +**Status:** Proposed +**Author:** Doug Mealing (with Claude) +**Supersedes deferral in:** `docs/superpowers/specs/2026-05-23-enum-datatype-design.md` (D4 — +"Integer-backed enums... deferred to a later design") + +## Problem + +`field.enum` (shipped 2026-05-23) is string-backed only: each member's symbol *is* its +stored and transmitted string value (`varchar` + `CHECK`). There is no way to declare +that an enum's members persist as integers in the database — the deferred v1 non-goal. +A consumer that wants a compact/indexable integer column (e.g. matching an existing +legacy schema, or optimizing storage/index size for a high-cardinality table) has no +metadata-driven path today; they would have to hand-roll a converter outside the +generated code, defeating the "declare once → idiomatic type + DB constraint in every +language" payoff that is `field.enum`'s whole reason to exist. + +## Goals + +1. Let a `field.enum` declare an explicit, possibly-sparse, per-member integer value for + database storage. +2. Every language's *generated native type* (TS union, C# `enum`, Java/Python/Kotlin + equivalents) and the *wire format* (JSON API payloads) stay **exactly as they are + today** for both string- and int-backed enums — this is a persistence-layer-only + concern, invisible to any client of the generated code. +3. Avoid positional/index correspondence between two parallel structures — the + documented weak point of the one ecosystem (OpenAPI's `x-enum-varnames` extension) + that uses that shape (see Prior art below). A metadata-only reorder of an unrelated + array must never be able to silently corrupt which stored int a symbol means. +4. Ship as a strictly additive, opt-in overlay on the existing `field.enum` contract — + zero change to any currently-shipped string-backed enum. +5. Preserve cross-language conformance: the new vocabulary is identical across + TS / C# / Java / Python / Kotlin, gated by `registry-conformance`. + +## Non-goals (out of scope) + +- **Toggling an existing, data-bearing field between string- and int-backed.** That is a + genuine column-type-changing migration (`varchar` → `integer`, with a data cast), not a + cheap `CHECK` swap. migrate-ts refuses to auto-generate it (see Migration safety). +- **Value aliasing** (two members sharing one stored int, à la Python `enum` without + `@unique`, or protobuf's `allow_alias`). No current consumer need; duplicate values are + a load-time error (matches protobuf's default, the strictest of the surveyed + frameworks — see Prior art). +- **Display labels** (still deferred from the original enum design; unrelated to storage). +- **Native Postgres `CREATE TYPE ... AS ENUM`** (still out — same PG/SQLite parity and + migration-footgun reasoning as the original design's D5/Non-goals). +- **A `@kind` discriminator.** Considered and rejected — see Decision D2. + +## Prior art + +Researched before finalizing the shape (frameworks that let a symbol carry an explicit, +possibly-sparse integer value): + +| Framework | Shape | Uniqueness enforced? | +|---|---|---| +| Ruby on Rails `ActiveRecord::Enum` | name-keyed hash: `enum :status, { draft: 0, published: 5 }` | No (hash keys unique by construction; duplicate *values* not checked pre-7.1) | +| Protocol Buffers | inline per-value assignment: `enum Status { DRAFT = 0; PUBLISHED = 5; }` | **Yes** — compile error unless `allow_alias = true` | +| C# (native language enum) | inline per-value assignment: `enum Status { Draft = 0, Published = 5 }` | No | +| Django `IntegerChoices` | inline per-member `(value, label)` tuple | No (Python `enum` aliasing unless `@unique`) | +| graphql-js | name-keyed map (server-side only): `values: { DRAFT: { value: 0 } }` | No | +| OpenAPI (`x-enum-varnames`, NSwag `x-enumNames`) | **parallel array** of names alongside the plain `enum: [...]` values array | No — and explicitly called out as the fragile, index-alignment-error-prone approach | +| JPA `@Enumerated` | no explicit-int support at all; ordinal-or-string only | n/a | +| Prisma | no int-backed enums; long-standing open feature request | n/a | + +**Takeaway:** name-keyed pairing (a literal map, or inline per-value assignment — the +same shape in declaration-order form) is the dominant, safe pattern. Parallel arrays are +the one shape the survey found, and it is the ecosystem's own acknowledged weak point. +This is why `@intValueMap` is a map, not a second array parallel to `@values`. + +## Decisions + +- **D1 — `@intValueMap`, not a second array.** A new object-shaped attribute on + `field.enum`: `@intValueMap: { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 }`. Keys are + member symbols, values are integers. Optional; absence keeps today's string+`CHECK` + default unchanged. Rejected the parallel-array shape (`@intValues: [0, 5, 9]`, + index-matched to `@values`) specifically because index correspondence lets a + metadata-only reorder of `@values` silently reassign stored meanings with no + validation able to catch it — the exact failure mode the OpenAPI-ecosystem prior art + confirms is real. Named `@intValueMap` (not `@valueMap`) so the attribute name states + its value type — no framework surveyed uses this exact name, but it unambiguously + states its own contents, which every alternative in the survey (`values`, `x-enum-varnames`) + does not. + +- **D2 — No `@kind` discriminator.** Presence of `@intValueMap` alone signals int-backed + persistence. Per ADR-0037, `@kind` is chartered for structural variants that change a + subtype's *generated shape* — and per Goal 2, the generated shape (native + union/enum type, wire format) is identical in both modes; only the DB persistence + codec differs. Using `@kind` here would stretch its charter for a distinction that + produces no codegen-shape difference, and would add a second attribute that must be + kept in sync with the first for no benefit. + +- **D3 — `@values` is untouched.** It remains required, and remains the sole source of + canonical member order (used for TS union member order, C# `enum` declaration order, + etc., in both string- and int-backed modes). `@intValueMap` never replaces it and + carries no ordering significance of its own (object key order is not relied upon). + +- **D4 — Validation.** At load time (own-only, eager-throw per the pattern the original + `field.enum` design used for Java's post-load `ValidationPhase`): + - `@intValueMap`'s key set must be **exactly** the member set in `@values` — no + missing member, no extra key. + - Every value must be a JSON integer (reject strings, floats, booleans, `null`). + - No two keys may share the same value (rejected outright — no alias opt-in; matches + protobuf's default, the strictest surveyed). + - `@intValueMap` is invalid on any subtype other than `enum` (mirrors `@values`' own + subtype restriction). + +- **D5 — DB representation: `integer` + `CHECK`.** `CHECK (col IN (0, 5, 9))`, portable + across Postgres and SQLite exactly like the string-backed `varchar` + `CHECK` — adding + or removing a member (within the same backing mode) is the same cheap `CHECK` swap + migrate-ts already supports for string-backed enums. + +- **D6 — Codec boundary: persistence only.** Every language's generated *native* type + (TS union + `z.enum`, C# `enum`, Java/Python/Kotlin equivalents) is **byte-identical** + between string- and int-backed modes. Each port's persistence layer builds a + bidirectional symbol↔int lookup table from `@intValueMap` at codegen/build time (never + runtime reflection, per ADR-0001) and translates at the DB read/write boundary only — + e.g. C# gets a custom `HasConversion` built from the table instead of + `HasConversion()`; TS/Drizzle gets an explicit encode/decode pair around the + Kysely column instead of a passthrough string column. The wire format (JSON API + payloads) is the member string in both modes, unchanged from the original design's + cross-language contract. + +- **D7 — Array-of-enum composes unchanged.** `field.enum @isArray` + `@intValueMap` + follows the same pattern as today's array-of-enum: an `integer[]` column instead of + `text[]`, element membership validated against `@intValueMap`'s value set. + +- **D8 — Migration safety: no auto-recast.** Adding `@intValueMap` to a *new* field (no + existing column) is a normal create. Adding or removing `@intValueMap` on a field that + **already has a table/column** is a backing-mode change — migrate-ts detects it in the + diff and surfaces a manual-intervention-required error rather than auto-generating a + `varchar`↔`integer` `ALTER COLUMN TYPE`, consistent with how this codebase already + refuses to auto-generate other genuinely risky migrations (e.g. the auto-allowed + drop-view guard). A consumer needing this must do the two-step backfill migration by + hand (add new column, backfill, swap, drop old) — no new safe-recast subsystem is being + built for v1, since there is no current consumer need for that path specifically. + +## Metamodel addition + +``` +FIELD_ATTR_INT_VALUE_MAP = "intValueMap" // @intValueMap: object (optional, `enum` subtype only) +``` + +### Authoring + +```yaml +field.enum: + name: status + values: ["DRAFT", "PUBLISHED", "ARCHIVED"] + intValueMap: { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 } +``` + +Canonical JSON: `{ "field.enum": { "name": "status", "@values": [...], "@intValueMap": { +"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }`. + +Reuse via abstract field + `extends` works exactly as it does for `@values` today — an +abstract `field.enum` may carry both `@values` and `@intValueMap`; concrete fields +`extends` it and inherit both. + +## Codegen mappings + +| Concern | String-backed (unchanged) | Int-backed (new) | +|---|---|---| +| TS type | `type Status = "DRAFT" \| "PUBLISHED" \| "ARCHIVED"` | **identical** | +| TS validation | `z.enum([...])` | **identical** | +| C# type | `enum Status { DRAFT, PUBLISHED, ARCHIVED }` | **identical** | +| Wire (all langs) | the member string | **identical** | +| DB column | `varchar` | `integer` | +| DB constraint | `CHECK (status IN ('DRAFT', ...))` | `CHECK (status IN (0, 5, 9))` | +| C# persistence | EF Core `HasConversion()` | EF Core `HasConversion` via a generated symbol↔int table | +| TS persistence | passthrough string column | explicit encode/decode around the Kysely column, generated from `@intValueMap` | + +Java/Python/Kotlin: vocabulary + validation ship now; each port's persistence codec +follows the same table-driven pattern as C#/TS, in the same release (per the "all five +ports up front" scope call — no cross-port gap period for this feature, unlike the +original `field.enum` rollout). + +## Cross-language contract (must be identical across ports) + +- Attribute name: `intValueMap` (canonical JSON `@intValueMap`). +- Value shape: object, string keys (member symbols), integer values. +- Key-set-must-equal-`@values` and no-duplicate-value rules enforced identically in every + loader. +- Wire format is unaffected: the member string, on every endpoint, in both backing modes. + +## Conformance fixtures + +1. **`enum-int-backed`** — a `field.enum` with `@values` + `@intValueMap`; asserts DB + column is `integer` + int `CHECK`, and the native type in every port is unchanged from + the string-backed case. +2. **`enum-int-backed-array`** — `field.enum[]` + `@intValueMap`; array-of-int-backed-enum + DDL and element-membership semantics. +3. **`error-enum-intvaluemap-key-mismatch`** (negative) — `@intValueMap` keys don't + exactly match `@values` members → load error. +4. **`error-enum-intvaluemap-non-int`** (negative) — a non-integer value in + `@intValueMap` → load error. +5. **`error-enum-intvaluemap-duplicate-value`** (negative) — two members share one int → + load error. + +Persistence-conformance: extend the existing round-trip write/read gate (the `AllTypes` +entity family, `fixtures/persistence-conformance/roundtrip-all-types.yaml`) with an +int-backed enum field, inserting/reading through each port's real runtime codec — not +just golden-snapshot codegen. + +## Testing + +- Metadata package (all 5 ports): load/validate unit tests for D4's rules, abstract + + extends inheritance of `@intValueMap`, the negative cases. +- Per-port codegen: DDL emission (`integer` + int `CHECK`), native-type-unchanged + assertion (byte-diff the generated union/enum type between a string-backed and + int-backed fixture — they must be identical modulo the field name). +- Per-port persistence: the symbol↔int codec round-trips through the real runtime/ORM + (not just golden snapshots) via the extended `roundtrip-all-types` scenario. +- migrate-ts: a real-engine test that adding `@intValueMap` to a field with no existing + column succeeds normally, and that adding/removing it on a field with an existing + column surfaces the manual-intervention error rather than silently altering the + column type. +- Conformance: the fixtures above run across every port (`registry-conformance` for the + new attribute + `ERR_*` codes, `persistence-conformance` for the round-trip). + +## Remaining follow-ups (explicitly out of scope for this design) + +- Safe backing-mode migration (varchar↔integer recast with data preservation) — no + current consumer; D8's manual path covers the only known need. +- Value aliasing (`allow_alias`-style opt-out of the duplicate-value rejection) — no + current consumer. +- Native Postgres `CREATE TYPE ... AS ENUM` — unrelated to this design, still deferred + per the original enum design's D5/Non-goals. From c86cd203dd7b73861f7795645fd90a56d64ef04a Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 13:25:06 -0400 Subject: [PATCH 02/52] fix(migrate-ts): FK refColumns resolve target @column override; scope declaredSchemas to views too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit buildForeignKeys resolved a target FK field's PHYSICAL column by applying the naming strategy to its raw (logical) name — a target PK with an explicit @column override phantom-diffed every FK into that table (expected the naming-strategy name, actual the override). It now resolves through the target entity's own field, matching how fkCols already handles the source side. declaredSchemas previously came only from expected.tables, so a model that declares views in a schema with no table of its own (an API/read-model schema sitting alongside an all-public entity model) never brought that schema into scope — its views were silently excluded from both sides of the diff rather than gated on it as owned. Also scopes the source-less-object skip (added alongside these two fixes) to non-entity subtypes: a plain object.entity with no declared source.rdb keeps the pre-Project-E default of an implicit writable table; only object subtypes other than entity (an adopter-registered config/reference type with no physical table) are excluded from the table diff. --- .../packages/migrate-ts/src/diff/index.ts | 13 +++- ...ected-schema-fk-refcolumn-override.test.ts | 76 +++++++++++++++++++ 2 files changed, 86 insertions(+), 3 deletions(-) create mode 100644 server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts diff --git a/server/typescript/packages/migrate-ts/src/diff/index.ts b/server/typescript/packages/migrate-ts/src/diff/index.ts index 3d9b06a43..4da7ea67f 100644 --- a/server/typescript/packages/migrate-ts/src/diff/index.ts +++ b/server/typescript/packages/migrate-ts/src/diff/index.ts @@ -148,9 +148,16 @@ export async function diff( // schemas it mentions), else null = no scoping (empty model → prior whole-DB // behavior). A table outside the scope is excluded from both sides, so a // co-located schema owned by another app is neither dropped nor reported. - const declaredSchemas = new Set( - args.expected.tables.map((t) => t.schema ?? DEFAULT_DB_SCHEMA_POSTGRES), - ); + const declaredSchemas = new Set([ + ...args.expected.tables.map((t) => t.schema ?? DEFAULT_DB_SCHEMA_POSTGRES), + // A model that declares views in a schema with no table of its own (e.g. an + // API/read-model schema like `p3_api` sitting alongside an all-`public` + // entity model) must still bring that schema into scope — otherwise its + // views are silently excluded from BOTH sides of the diff (never compared, + // so real drift in an opaque @sql body or a genuine missing/extra view goes + // undetected) rather than gated on it as an owned schema. + ...args.expected.views.map((v) => v.schema ?? DEFAULT_DB_SCHEMA_POSTGRES), + ]); const scopeSchemas: Set | null = args.scopeSchemas !== undefined ? new Set(args.scopeSchemas) diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts new file mode 100644 index 000000000..617307d71 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts @@ -0,0 +1,76 @@ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../src/expected-schema.js"; + +// Regression: an FK's refColumns must resolve the TARGET entity's physical +// @column override, not the raw field name. An adopted database (e.g. an EF +// Core schema) commonly has a PK field `id` stored as column "Id"; before the +// fix the expected side emitted refColumns ["id"] while introspection read +// ["Id"], phantom-diffing every FK into that table as drop-fk + add-fk. + +async function loadJson(json: string): Promise { + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + return result.root; +} + +function model(clientIdField: Record): string { + return JSON.stringify({ + "metadata.root": { + "children": [ + { + "object.entity": { + "name": "Client", + "children": [ + { "field.uuid": clientIdField }, + { "source.rdb": { "name": "src", "@table": "Clients" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.entity": { + "name": "Patient", + "children": [ + { "field.uuid": { "name": "id" } }, + { "field.uuid": { "name": "clientId", "@column": "ClientId" } }, + { "source.rdb": { "name": "src", "@table": "Patients" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { + "identity.reference": { + "name": "fk_client", + "@fields": ["clientId"], + "@references": "Client", + }, + }, + ], + }, + }, + ], + }, + }); +} + +function patientFk(root: MetaData) { + const snapshot = buildExpectedSchema(root, { dialect: "postgres" }); + const table = snapshot.tables.find((t) => t.name === "Patients"); + expect(table).toBeDefined(); + expect(table!.foreignKeys).toHaveLength(1); + return table!.foreignKeys[0]!; +} + +describe("buildExpectedSchema — FK refColumns resolve the target PK's @column override", () => { + test("target PK field with @column override → refColumns uses the physical name", async () => { + const root = await loadJson(model({ "name": "id", "@column": "Id" })); + const fk = patientFk(root); + expect(fk.columns).toEqual(["ClientId"]); + expect(fk.refTable).toBe("Clients"); + expect(fk.refColumns).toEqual(["Id"]); + }); + + test("target PK field without @column override → refColumns keeps the field name", async () => { + const root = await loadJson(model({ "name": "id" })); + const fk = patientFk(root); + expect(fk.refColumns).toEqual(["id"]); + }); +}); From 2b60d004367e3efa63b9762627794d26422fa06d Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 13:42:43 -0400 Subject: [PATCH 03/52] docs(plan): implementation plan for field.enum @intValueMap (metamodel layer) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Covers vocabulary + validation + conformance across all five ports per the approved design (docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md). Persistence (DDL, codecs, migration-safety guard) is scoped to follow-on plans, written separately per the writing-plans Scope Check — this plan is independently testable and shippable on its own. --- ...-07-23-int-backed-enum-values-metamodel.md | 1399 +++++++++++++++++ 1 file changed, 1399 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md new file mode 100644 index 000000000..87d35b479 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md @@ -0,0 +1,1399 @@ +# Int-Backed Enum Values — Metamodel Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add the `@intValueMap` attribute to `field.enum` — an optional `{memberSymbol: int}` map that, when present, is the metadata author's declaration of each member's stored integer — across all five ports (TypeScript, C#, Java, Python, Kotlin-via-Java), gated by load-time validation and the `registry-conformance` + `fixtures/conformance/` corpora. This plan covers **vocabulary + validation + conformance only** — it does NOT touch codegen (already proven unchanged, since no port's enum-type emitter reads `@intValueMap`) or persistence (DB DDL, EF Core/JDBC/Exposed/ObjectManager codecs, migrate-ts's migration-safety guard). Those are covered by follow-on plans, one per port/group, written after this one lands. + +**Architecture:** Each port already has an identical three-layer structure for `field.enum`'s existing `@values` attribute: (1) a generic attr-value-type class/registration (`properties`, `string[]`, etc.) that enforces the attr's basic shape, (2) a field.enum-specific content-rule validation pass that enforces the enum's own semantics (non-empty, identifier pattern, no duplicates) on top of that shape, (3) a shared cross-port conformance fixture set + a shared `registry-conformance` manifest that every port's test suite asserts against byte-for-byte. `@intValueMap` follows the exact same three-layer shape, with one port-specific correction: Java's existing `properties` attr class is backed by `java.util.Properties`, which silently coerces every value to a `String` on load (confirmed by reading `PropertiesAttribute.java:51-60` — every `Map` entry is written via `.getValue().toString()`). Reusing it for `@intValueMap` would silently turn `0` into `"0"` internally in Java, and — because canonical-JSON round-trip serialization must be byte-identical across ports (the `fixtures/conformance/*/expected.json` gate) — would make Java re-emit an integer value as a quoted JSON string, diverging from TS/C#/Python. So every port registers a **new**, cross-port-identical attr subtype, `intMap` (a generic "object with all-integer values" shape, parallel to how `properties` is a generic "any object" shape), and Java backs it with a new `Map`-typed class instead of reusing `PropertiesAttribute`. `field.enum`'s own content-rule pass then layers the enum-specific rules on top: `@intValueMap`'s key set must exactly equal `@values`' members, and no two members may share a stored int (reusing the existing `ERR_BAD_ATTR_VALUE` code everywhere — no new error code needed, confirmed by inspecting all four ports' error-code ledgers). + +**Tech Stack:** TypeScript (Bun test runner), C# (.NET, xunit), Java (Maven, JUnit) — also covers Kotlin, which shares Java's metadata layer — Python (pytest). + +## Global Constraints + +- Cross-language contract (from the design spec, `docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`): attribute name is `intValueMap` (canonical JSON `@intValueMap`); value shape is an object with string keys (member symbols) and integer values; every loader enforces identically: (a) key set exactly equals `@values`' members (no missing, no extra), (b) every value is an integer, (c) no two members share a value. +- No new `@kind`. Presence of `@intValueMap` alone is the only signal — do not add a discriminator attribute. +- `@values` is completely untouched — do not modify its required-ness, type, or any existing validation for it. +- Reuse `ERR_BAD_ATTR_VALUE` for every `@intValueMap` content violation (confirmed: no port has a dedicated enum error code today — all enum content violations already reuse this code). +- Every new attr/field-attr registration needs a `registry-conformance` fixture entry (ADR-0023) — `fixtures/registry-conformance/expected-registry.json` is the ONE shared file all five ports assert against. +- `fixtures/conformance/` fixtures are shared JSON, consumed by every port's own test runner — write them once, verify per-port. +- Do NOT touch codegen or persistence in this plan. If a task tempts you to edit a `*Generator.cs`/`*.ts` codegen template, a DDL emitter, or an ORM config generator, stop — that belongs to a follow-on plan. + +--- + +### Task 1: TypeScript — generic `attr.intMap` subtype + +**Files:** +- Modify: `spec/metamodel/attr.json` +- Create: `server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts` +- Modify: `server/typescript/packages/metadata/src/core/attr/attr-constants.ts` +- Modify: `server/typescript/packages/metadata/src/core-types.ts` +- Test: `server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts` + +**Interfaces:** +- Produces: `ATTR_SUBTYPE_INT_MAP = "intMap"` (exported from `attr-constants.ts`), the `IntMapAttr` class (registered against that subtype), and the `attr.intMap` type declaration in the canonical spec — all consumed by Task 2. + +- [ ] **Step 1: Write the failing test** + +```typescript +// server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts +import { describe, test, expect } from "bun:test"; +import { IntMapAttr } from "../../../src/core/attr/meta-attr-int-map.js"; + +describe("IntMapAttr", () => { + test("accepts a plain object with integer values", () => { + const attr = new IntMapAttr("intValueMap"); + attr.setValue({ DRAFT: 0, PUBLISHED: 5 }); + expect(attr.validateValue(attr.getValue())).toEqual([]); + }); + + test("rejects a non-object value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue("not-an-object" as unknown as object); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("must be of type 'intMap'"); + }); + + test("rejects an array value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue([0, 1] as unknown as object); + expect(errors.length).toBe(1); + }); + + test("rejects a non-integer value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue({ DRAFT: "0" } as unknown as object); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("DRAFT"); + }); + + test("rejects a float value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue({ DRAFT: 0.5 } as unknown as object); + expect(errors.length).toBe(1); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd server/typescript && bun test packages/metadata/test/core/attr/meta-attr-int-map.test.ts` +Expected: FAIL — `Cannot find module '../../../src/core/attr/meta-attr-int-map.js'` + +- [ ] **Step 3: Add `ATTR_SUBTYPE_INT_MAP` to `attr-constants.ts`** + +Edit `server/typescript/packages/metadata/src/core/attr/attr-constants.ts` — add the constant next to `ATTR_SUBTYPE_EXPRESSION` (line 19) and into the `ATTR_SUBTYPES` array (line 33-44): + +```typescript +export const ATTR_SUBTYPE_EXPRESSION = "expression"; +// An object-shaped attr whose values are all integers (e.g. field.enum's +// @intValueMap: {memberSymbol: int}). Generic shape check only — semantic +// rules specific to a consumer (key-set membership, uniqueness) are that +// consumer's own content-rule validation, not this attr's. +export const ATTR_SUBTYPE_INT_MAP = "intMap"; +``` + +```typescript +export const ATTR_SUBTYPES = [ + SUBTYPE_BASE, + ATTR_SUBTYPE_STRING, + ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_LONG, + ATTR_SUBTYPE_DOUBLE, + ATTR_SUBTYPE_BOOLEAN, + ATTR_SUBTYPE_CLASS, + ATTR_SUBTYPE_PROPERTIES, + ATTR_SUBTYPE_FILTER, + ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, +] as const; +``` + +- [ ] **Step 4: Write `meta-attr-int-map.ts`** + +```typescript +// IntMapAttr — attr subtype `intMap`. Object-shaped value whose members must +// all be integers (e.g. field.enum's @intValueMap). No desugar; validates +// shape (object, not array) and every value's type (integer). A consumer's +// own semantic rules (key-set membership, uniqueness) are validated by that +// consumer, not here — mirrors how StringArrayAttr validates shape while +// field.enum's own content-rule pass validates its @values semantics. + +import { MetaAttr, type ValueError, runtimeTypeName } from "./meta-attr.js"; +import { type AttrValue } from "../../shared/meta-data.js"; +import { DATA_TYPE_OBJECT, type DataType } from "../../data-type.js"; +import { registerAttrClass } from "../../attr-class-map.js"; +import { ATTR_SUBTYPE_INT_MAP } from "./attr-constants.js"; + +export class IntMapAttr extends MetaAttr { + override get dataType(): DataType { + return DATA_TYPE_OBJECT; + } + + override coerce(raw: unknown): AttrValue { + return raw as AttrValue; + } + + override validateValue(value: AttrValue): ValueError[] { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return [{ message: `attribute '@${this.name}' must be of type 'intMap' but got ${runtimeTypeName(value)}` }]; + } + const errors: ValueError[] = []; + for (const [key, member] of Object.entries(value as Record)) { + if (typeof member !== "number" || !Number.isInteger(member)) { + errors.push({ + message: `attribute '@${this.name}' member '${key}' has value '${String(member)}' which is not an integer`, + }); + } + } + return errors; + } +} + +registerAttrClass(ATTR_SUBTYPE_INT_MAP, IntMapAttr); +``` + +- [ ] **Step 5: Import the new module for its registration side-effect** + +Edit `server/typescript/packages/metadata/src/core-types.ts` — add alongside the existing sibling imports (line 20-22): + +```typescript +import "./core/attr/meta-attr-filter.js"; +import "./core/attr/meta-attr-properties.js"; +import "./core/attr/meta-attr-expression.js"; +import "./core/attr/meta-attr-int-map.js"; +``` + +- [ ] **Step 6: Add the `attr.intMap` type declaration to the canonical spec** + +Edit `spec/metamodel/attr.json` — insert alphabetically between the `int` and `long` subtype blocks: + +```json + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, +``` + +- [ ] **Step 7: Regenerate the embedded attr definition** + +Run: `cd /Users/douglas.mealing/Development/metaobjects && bun scripts/generate-embedded-metamodel.ts` +Expected: regenerates `server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts` to include the new `intMap` block. + +- [ ] **Step 8: Run the test to verify it passes** + +Run: `cd server/typescript && bun test packages/metadata/test/core/attr/meta-attr-int-map.test.ts` +Expected: PASS — all 5 tests green. + +- [ ] **Step 9: Run the full metadata test suite to check for regressions** + +Run: `cd server/typescript && bun test packages/metadata` +Expected: all existing tests still pass (this step only adds a new subtype; nothing existing should change behavior). + +- [ ] **Step 10: Commit** + +```bash +git add spec/metamodel/attr.json server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts server/typescript/packages/metadata/src/core/attr/attr-constants.ts server/typescript/packages/metadata/src/core-types.ts server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts +git commit -m "feat(metadata): add attr.intMap — a generic object-with-integer-values attr subtype" +``` + +--- + +### Task 2: TypeScript — `field.enum`'s `@intValueMap` attribute + +**Files:** +- Modify: `server/typescript/packages/metadata/src/core/field/field-constants.ts` +- Modify: `spec/metamodel/field.json` +- Modify: `server/typescript/packages/metadata/src/attr-schema-validate.ts` +- Test: `server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: `ATTR_SUBTYPE_INT_MAP` (Task 1). +- Produces: `FIELD_ATTR_INT_VALUE_MAP = "intValueMap"` — consumed by every later port task and by the persistence follow-on plans. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader.js"; +import { InMemoryStringSource } from "../src/sources/in-memory-string-source.js"; + +async function load(json: string) { + const loader = new MetaDataLoader(); + return loader.load([new InMemoryStringSource(json, "test.json")]); +} + +const base = (extra: string) => `{ + "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] ${extra} } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]} +}`; + +describe("field.enum @intValueMap content rules", () => { + test("accepts a valid map — key set matches @values, unique ints", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors).toEqual([]); + }); + + test("field.enum with no @intValueMap is still valid (string-backed default)", async () => { + const result = await load(base("")); + expect(result.errors).toEqual([]); + }); + + test("rejects a missing member key", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("ARCHIVED"); + }); + + test("rejects an extra key not in @values", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("RETRACTED"); + }); + + test("rejects a non-integer value", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + }); + + test("rejects two members sharing the same int", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("DRAFT"); + expect(result.errors[0]?.message).toContain("PUBLISHED"); + }); +}); +``` + +> Adjust the exact `MetaDataLoader`/`InMemoryStringSource` import paths and the shape of `result.errors` (some loader APIs throw on the first error rather than returning an array) to match this package's actual loader test harness — check an existing test like `test/attr-schema-validate.test.ts` or `enum-inline`'s consuming test for the established pattern before finalizing this step. + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd server/typescript && bun test packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` +Expected: FAIL — the "accepts a valid map" test fails with `ERR_UNKNOWN_ATTR` (strict provenance, Task 1 registered the generic subtype but `field.enum` doesn't yet accept `@intValueMap`). + +- [ ] **Step 3: Add `FIELD_ATTR_INT_VALUE_MAP` constant** + +Edit `server/typescript/packages/metadata/src/core/field/field-constants.ts` — add after `FIELD_ATTR_VALUES` (line 160), inside the existing "Enum attrs" section (line 155-157): + +```typescript +/** Member symbols of an enum-subtype field. Required, string array. */ +export const FIELD_ATTR_VALUES = "values"; + +/** + * Optional per-member explicit integer value ({memberSymbol: int}) switching + * this enum field's DB persistence from string+CHECK to integer+CHECK. Keys + * must exactly match @values; values must be unique integers. The generated + * native type and wire format are UNCHANGED in every language — this is a + * persistence-layer-only concern (docs/superpowers/specs/2026-07-23-int-backed- + * enum-values-design.md). + */ +export const FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; +``` + +- [ ] **Step 4: Add the attr declaration to the canonical field spec** + +Edit `spec/metamodel/field.json` — add as a sibling of `values`/`provided` inside `field.enum`'s `children` array: + +```json + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } +``` + +- [ ] **Step 5: Regenerate the embedded field definition** + +Run: `cd /Users/douglas.mealing/Development/metaobjects && bun scripts/generate-embedded-metamodel.ts` +Expected: regenerates `field-definition.embedded.ts`. + +- [ ] **Step 6: Run tests again — confirm the ERR_UNKNOWN_ATTR failure is gone, new failures are the content-rule assertions** + +Run: `cd server/typescript && bun test packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` +Expected: "accepts a valid map" and "no @intValueMap" pass; the four negative tests FAIL (no content-rule validation exists yet, so `result.errors` is empty when it should have entries). + +- [ ] **Step 7: Add the content-rule validation** + +Edit `server/typescript/packages/metadata/src/attr-schema-validate.ts` — add immediately after Check 5 (FR-011, ends around line 372), inside the same `if (node.type === TYPE_FIELD && node.subType === FIELD_SUBTYPE_ENUM)` gate: + +```typescript + // --- Check 5b: field.enum @intValueMap content rules --- + // + // Optional. Own-only (mirrors Checks 4/5's own-attrs-only policy) — an + // inherited @intValueMap is validated on its declaring node. The generic + // "is this an object of integers" shape check already ran via IntMapAttr + // (attr subtype `intMap`); this validates the field.enum-SPECIFIC + // semantics: key-set-equals-@values, and no two members share a value. + const rawIntValueMap = node.ownAttrs().get(FIELD_ATTR_INT_VALUE_MAP); + if (rawIntValueMap !== undefined && typeof rawIntValueMap === "object" && rawIntValueMap !== null) { + const map = rawIntValueMap as Record; + const effectiveValues = node.attrs().get(FIELD_ATTR_VALUES); + const declaredMembers: string[] = Array.isArray(effectiveValues) ? effectiveValues : []; + const memberSet = new Set(declaredMembers); + const mapKeys = Object.keys(map); + const keySet = new Set(mapKeys); + + const missing = declaredMembers.filter((m) => !keySet.has(m)); + const extra = mapKeys.filter((k) => !memberSet.has(k)); + if (missing.length > 0 || extra.length > 0) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@${FIELD_ATTR_VALUES}' members` + + (missing.length > 0 ? ` (missing: ${missing.join(", ")})` : "") + + (extra.length > 0 ? ` (unknown: ${extra.join(", ")})` : "") + ".", + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } + + const seenValues = new Map(); + for (const [member, value] of Object.entries(map)) { + if (typeof value !== "number" || !Number.isInteger(value)) continue; // IntMapAttr already reported this + const owner = seenValues.get(value); + if (owner !== undefined) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' members '${owner}' and '${member}' ` + + `share the same value ${value}; every member must have a unique int.`, + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } else { + seenValues.set(value, member); + } + } + } +``` + +Add `FIELD_ATTR_INT_VALUE_MAP` to this file's existing import block from `@metaobjectsdev/metadata`'s field-constants (or the local relative import this file already uses for `FIELD_ATTR_VALUES` — match its existing import style). + +- [ ] **Step 8: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` +Expected: PASS — all 6 tests green. + +- [ ] **Step 9: Run the full metadata test suite** + +Run: `cd server/typescript && bun test packages/metadata` +Expected: all pass, no regressions. + +- [ ] **Step 10: Commit** + +```bash +git add server/typescript/packages/metadata/src/core/field/field-constants.ts spec/metamodel/field.json server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts server/typescript/packages/metadata/src/attr-schema-validate.ts server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts +git commit -m "feat(metadata): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 3: Shared conformance fixtures (`fixtures/conformance/`) + +**Files:** +- Create: `fixtures/conformance/enum-int-backed/input/meta.enums.json` +- Create: `fixtures/conformance/enum-int-backed/expected.json` +- Create: `fixtures/conformance/enum-int-backed-array/input/meta.enums.json` +- Create: `fixtures/conformance/enum-int-backed-array/expected.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-non-int/expected.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected.json` + +**Interfaces:** +- Consumes: `@intValueMap` (Task 2), plus this repo's existing conformance fixture format (see `fixtures/conformance/enum-inline/` for the reference shape). +- Produces: five fixtures every port's own conformance test runner discovers and asserts against (Tasks 5, 7, 9, 11). + +- [ ] **Step 1: Create `enum-int-backed` (positive)** + +`fixtures/conformance/enum-int-backed/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`fixtures/conformance/enum-int-backed/expected.json` (the same document with shorthand `@fields` expanded to an array, matching how `enum-inline/expected.json` normalizes it — verify the exact expansion by diffing against `enum-inline/expected.json`'s own `@fields` line before finalizing): +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}} +]}} +``` + +- [ ] **Step 2: Create `enum-int-backed-array` (positive, array-of-enum)** + +`fixtures/conformance/enum-int-backed-array/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "labels", "isArray": true, "@values": ["LOW","MEDIUM","HIGH"], "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`fixtures/conformance/enum-int-backed-array/expected.json` — same document with `@fields` expanded (mirror Step 1's normalization). + +- [ ] **Step 3: Create `error-enum-intvaluemap-key-mismatch` (negative)** + +`fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected.json` — match the negative-fixture format used by `error-enum-empty-values/expected.json` (an error-code envelope, not a normalized document — read that file first and mirror its exact shape): +```json +{ "errors": [ { "code": "ERR_BAD_ATTR_VALUE" } ] } +``` + +- [ ] **Step 4: Create `error-enum-intvaluemap-non-int` (negative)** + +`fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": "zero", "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`expected.json`: `{ "errors": [ { "code": "ERR_BAD_ATTR_VALUE" } ] }` + +- [ ] **Step 5: Create `error-enum-intvaluemap-duplicate-value` (negative)** + +`fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 0 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`expected.json`: `{ "errors": [ { "code": "ERR_BAD_ATTR_VALUE" } ] }` + +- [ ] **Step 6: Run TS's conformance fixture runner to sanity-check the fixtures parse (does not yet confirm cross-port correctness — that's Tasks 5/7/9/11)** + +Run: `cd server/typescript && bun test packages/metadata -t conformance` +Expected: the two new positive fixtures pass; the three negative fixtures pass (each correctly produces exactly the expected error code) — since Task 2 already implemented the validation, all five should be green already at this step. + +- [ ] **Step 7: Commit** + +```bash +git add fixtures/conformance/enum-int-backed fixtures/conformance/enum-int-backed-array fixtures/conformance/error-enum-intvaluemap-key-mismatch fixtures/conformance/error-enum-intvaluemap-non-int fixtures/conformance/error-enum-intvaluemap-duplicate-value +git commit -m "test(conformance): add cross-port fixtures for field.enum @intValueMap" +``` + +--- + +### Task 4: Shared — `registry-conformance/expected-registry.json` + +**Files:** +- Modify: `fixtures/registry-conformance/expected-registry.json` + +**Interfaces:** +- Consumes: Tasks 1-2's TS registration (the manifest is generated from TS's live registry and hand-verified, then used as the golden file every port compares against). + +- [ ] **Step 1: Add the new `attr.intMap` type entry** + +Edit `fixtures/registry-conformance/expected-registry.json` — insert alphabetically between the `int` (ends line 52) and `long` (starts line 53) attr type blocks: + +```json + { + "type": "attr", + "subType": "intMap", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation.", + "attrs": [], + "children": [] + }, +``` + +- [ ] **Step 2: Add the `intValueMap` attr entry to `field.enum`'s `attrs` array** + +Edit the same file — inside the `field`/`enum` block (starts line 929), insert alphabetically among the existing `attrs` entries (after `formExclude`, before whatever sorts after `intValueMap` alphabetically — read the surrounding entries first to place it exactly): + +```json + { + "name": "intValueMap", + "valueType": "intMap", + "isArray": false, + "required": false, + "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." + }, +``` + +- [ ] **Step 3: Run TS's registry-conformance test to confirm the manifest now matches TS's live registry** + +Run: `cd server/typescript && bun test packages/metadata -t registry-conformance` +Expected: PASS. If it fails on an unrelated diff (ordering, wording), adjust this file's new entries — not TS's source — to match, since this file must byte-match whatever TS's registry generator actually emits. + +- [ ] **Step 4: Commit** + +```bash +git add fixtures/registry-conformance/expected-registry.json +git commit -m "test(registry-conformance): register field.enum @intValueMap (attr.intMap)" +``` + +--- + +### Task 5: TypeScript — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the full TS metadata suite** + +Run: `cd server/typescript && bun test packages/metadata` +Expected: 100% pass, zero regressions. + +- [ ] **Step 2: Run the full server TS suite as a broader regression check** + +Run: `cd server/typescript && bun test` +Expected: 100% pass (this also exercises `codegen-ts` and `migrate-ts`, which per this plan's scope should show ZERO behavior change — if any codegen or migrate-ts test changes behavior here, that's a signal `@intValueMap` leaked into a place it shouldn't have at this stage; investigate before proceeding). + +- [ ] **Step 3: Typecheck** + +Run: `cd server/typescript && bun run --filter '@metaobjectsdev/metadata' typecheck` +Expected: no new errors introduced by this plan's changes (pre-existing unrelated errors, if any, are out of scope). + +--- + +### Task 6: C# — `attr.intMap` subtype + `field.enum`'s `@intValueMap` + +**Files:** +- Modify: `server/csharp/MetaObjects/Core/Attr/AttrConstants.cs` +- Modify: `server/csharp/MetaObjects/CoreTypes.cs` +- Modify: `server/csharp/MetaObjects/Core/Field/FieldConstants.cs` +- Modify: `server/csharp/MetaObjects/Core/Field/FieldSchema.cs` +- Modify: `server/csharp/MetaObjects/Loader/ValidationPasses.cs` +- Modify: `server/csharp/MetaObjects/SpecMetamodel/attr.json` +- Modify: `server/csharp/MetaObjects/SpecMetamodel/field.json` +- Test: `server/csharp/MetaObjects.Tests/EnumIntValueMapTests.cs` + +**Interfaces:** +- Produces: `FieldConstants.FIELD_ATTR_INT_VALUE_MAP`, `AttrConstants.ATTR_SUBTYPE_INT_MAP` — no other task in this plan consumes them, but the C# persistence follow-on plan will. + +- [ ] **Step 1: Write the failing tests** + +```csharp +// server/csharp/MetaObjects.Tests/EnumIntValueMapTests.cs +using Xunit; +using MetaObjects.Loader; + +namespace MetaObjects.Tests; + +public class EnumIntValueMapTests +{ + private static string Model(string extra) => $$""" + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {{extra}} } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """; + + private static (bool Ok, System.Collections.Generic.IReadOnlyList Errors) TryLoad(string json) + { + var loader = new MetaDataLoader(); + var source = new InMemoryStringSource(json, "test.json"); + var result = loader.Load(new[] { (IMetaDataSource)source }); + return (result.Errors.Count == 0, result.Errors); + } + + [Fact] + public void Valid_intValueMap_with_matching_keys_and_unique_ints_loads_clean() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.True(ok, string.Join("; ", errors)); + } + + [Fact] + public void No_intValueMap_still_loads_clean_string_backed_default() + { + var (ok, _) = TryLoad(Model("")); + Assert.True(ok); + } + + [Fact] + public void Missing_member_key_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("ARCHIVED")); + } + + [Fact] + public void Extra_key_not_in_values_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("RETRACTED")); + } + + [Fact] + public void Non_integer_value_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE); + } + + [Fact] + public void Duplicate_int_value_across_members_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE + && e.Message.Contains("DRAFT") && e.Message.Contains("PUBLISHED")); + } +} +``` + +> Adjust `InMemoryStringSource`/`MetaDataLoader.Load` call shape and the `Errors`/`MetaError` field names to match this codebase's actual API (check an existing test in `MetaObjects.Tests` that loads inline JSON and asserts on errors, e.g. a test near `EnumFieldTest`-equivalent, before finalizing). + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter EnumIntValueMapTests` +Expected: FAIL — `ERR_UNKNOWN_ATTR` on the positive test (attr not yet registered). + +- [ ] **Step 3: Add `ATTR_SUBTYPE_INT_MAP` constant** + +Edit `server/csharp/MetaObjects/Core/Attr/AttrConstants.cs` — add next to `ATTR_SUBTYPE_PROPERTIES` (around line 22) and into its subtype list (around line 47): + +```csharp +public const string ATTR_SUBTYPE_INT_MAP = "intMap"; +``` + +- [ ] **Step 4: Register the `DataType.Object` dispatch entry** + +Edit `server/csharp/MetaObjects/CoreTypes.cs` — add to the dispatch table shown at lines 125-137: + +```csharp + [ATTR_SUBTYPE_PROPERTIES] = DataType.Object, + [ATTR_SUBTYPE_INT_MAP] = DataType.Object, +``` + +- [ ] **Step 5: Add `FIELD_ATTR_INT_VALUE_MAP` constant** + +Edit `server/csharp/MetaObjects/Core/Field/FieldConstants.cs` — add next to `FIELD_ATTR_VALUES` (around line 184): + +```csharp +public const string FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; +``` + +- [ ] **Step 6: Add the `IntValueMapAttr` schema entry** + +Edit `server/csharp/MetaObjects/Core/Field/FieldSchema.cs` — add next to `EnumValuesAttr` (around line 168): + +```csharp + /// The @intValueMap attr — only on field.enum. Optional object of integers. + public static readonly AttrSchema IntValueMapAttr = new AttrSchema( + Name: FieldConstants.FIELD_ATTR_INT_VALUE_MAP, + ValueType: AttrConstants.ATTR_SUBTYPE_INT_MAP, + Required: false, + Description: "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers."); +``` + +- [ ] **Step 7: Register the attr on the `enum` subtype** + +Edit `server/csharp/MetaObjects/CoreTypes.cs` at the line found in research (`FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr]`, around line 304): + +```csharp +FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr, FieldSchema.IntValueMapAttr], +``` + +- [ ] **Step 8: Add the content-rule validation to Pass 10** + +Edit `server/csharp/MetaObjects/Loader/ValidationPasses.cs` — inside `WalkEnumValues`, after the existing Rule 4 (FR-011 fallback-attr check, ends before the closing brace shown in research), add: + +```csharp + // Rule 5: @intValueMap content rules (optional). + // a. Key set must exactly match @values. + // b. No two members may share the same int (protobuf's stance — no alias opt-in). + // (Every-value-is-an-integer is already enforced by the generic + // ATTR_SUBTYPE_INT_MAP dispatch/JSON-type check at parse time.) + if (field.OwnAttr(FIELD_ATTR_INT_VALUE_MAP) is System.Collections.IDictionary intValueMap) + { + var effective = field.EffectiveEnumValues ?? new List(); + var memberSet = new HashSet(effective, StringComparer.Ordinal); + var mapKeys = new List(); + foreach (var key in intValueMap.Keys) mapKeys.Add((string)key); + var keySet = new HashSet(mapKeys, StringComparer.Ordinal); + + var missing = effective.Where(m => !keySet.Contains(m)).ToList(); + var extra = mapKeys.Where(k => !memberSet.Contains(k)).ToList(); + if (missing.Count > 0 || extra.Count > 0) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@{FIELD_ATTR_VALUES}' members" + + (missing.Count > 0 ? $" (missing: {string.Join(", ", missing)})" : "") + + (extra.Count > 0 ? $" (unknown: {string.Join(", ", extra)})" : "") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + + var seenValues = new Dictionary(); + foreach (var key in mapKeys) + { + if (intValueMap[key] is not long and not int) continue; // generic dispatch already reported this + var value = System.Convert.ToInt64(intValueMap[key]); + if (seenValues.TryGetValue(value, out var owner)) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members '{owner}' and '{key}' " + + $"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + else + { + seenValues[value] = key; + } + } + } +``` + +> The exact C# type `OwnAttr` returns for an object-shaped attr value (`IDictionary`, `JsonObject`, or a custom `Dictionary`) depends on this codebase's JSON-parsing layer — check how the existing `@enumAlias`/`@enumDoc`-style properties attrs are read elsewhere in this file (or in `SpringDtoGenerator`-equivalent C# consumers) and match that exact type before finalizing this step. + +- [ ] **Step 9: Add the `attr.intMap` and `field.enum.intValueMap` declarations to C#'s spec copies** + +Edit `server/csharp/MetaObjects/SpecMetamodel/attr.json` and `server/csharp/MetaObjects/SpecMetamodel/field.json` with the same two JSON snippets used in TS Task 1 Step 6 and Task 2 Step 4 (these are C#'s own packaged copies — keep them semantically identical to the canonical `spec/metamodel/*.json`, per this plan's Global Constraints; there is no automated drift-gate tying them together today, so this is a manual-fidelity step, not a generated one). + +- [ ] **Step 10: Run tests — confirm all pass** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter EnumIntValueMapTests` +Expected: PASS — all 6 tests green. + +- [ ] **Step 11: Run the full C# metadata test suite** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests` +Expected: all pass, no regressions. + +- [ ] **Step 12: Commit** + +```bash +git add server/csharp/MetaObjects/Core/Attr/AttrConstants.cs server/csharp/MetaObjects/CoreTypes.cs server/csharp/MetaObjects/Core/Field/FieldConstants.cs server/csharp/MetaObjects/Core/Field/FieldSchema.cs server/csharp/MetaObjects/Loader/ValidationPasses.cs server/csharp/MetaObjects/SpecMetamodel/attr.json server/csharp/MetaObjects/SpecMetamodel/field.json server/csharp/MetaObjects.Tests/EnumIntValueMapTests.cs +git commit -m "feat(csharp): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 7: C# — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the shared conformance fixtures against C#'s loader** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter ConformanceTests` (adjust the filter name to whatever test class discovers/runs `fixtures/conformance/*` — check for a `ConformanceRunner`-style test class first) +Expected: the five fixtures from Task 3 all pass. + +- [ ] **Step 2: Run C#'s registry-conformance test** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter RegistryConformance` +Expected: PASS against the `expected-registry.json` Task 4 updated. + +- [ ] **Step 3: Run the full C# test suite** + +Run: `cd server/csharp && dotnet test` +Expected: 100% pass, no regressions, including `MetaObjects.Codegen.Tests` (should show zero behavior change — codegen isn't touched by this plan). + +--- + +### Task 8: Java (+ Kotlin) — `attr.intMap` subtype + `field.enum`'s `@intValueMap` + +**Files:** +- Create: `server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` +- Test: `server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java` + +**Interfaces:** +- Produces: `EnumField.ATTR_INT_VALUE_MAP`, `IntMapAttribute.SUBTYPE_INT_MAP` — consumed by the Java+Kotlin persistence follow-on plan. + +**Note:** Do NOT register `@intValueMap` using the existing `PropertiesAttribute` class. It is backed by `java.util.Properties`, confirmed (`PropertiesAttribute.java:51-60`) to coerce every value to `String` on load — silently turning `0` into `"0"`. Since canonical-JSON round-trip serialization must stay byte-identical with TS/C#/Python (which preserve real integers), Java needs its own `Map`-backed class, mirroring `PropertiesAttribute`'s structure exactly but preserving int fidelity on both parse and `getValueAsString()`. + +- [ ] **Step 1: Write the failing test** + +```java +// server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java +package com.metaobjects.field; + +import com.metaobjects.MetaDataException; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.loader.source.InMemoryMetaDataSource; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.*; + +class EnumFieldIntValueMapTest { + + private static String model(String extra) { + return "{ \"metadata.root\": { \"package\": \"acme\", \"children\": [" + + "{ \"object.entity\": { \"name\": \"Order\", \"children\": [" + + "{ \"field.long\": { \"name\": \"id\" } }," + + "{ \"field.enum\": { \"name\": \"status\", \"@values\": [\"DRAFT\",\"PUBLISHED\",\"ARCHIVED\"]" + extra + " } }," + + "{ \"identity.primary\": { \"name\": \"pk\", \"@fields\": [\"id\"] } }" + + "]}}]}}"; + } + + private static MetaDataException loadExpectingError(String json) { + var loader = new MetaDataLoader(); + return assertThrows(MetaDataException.class, () -> + loader.load(java.util.List.of(new InMemoryMetaDataSource(json, "test.json")))); + } + + @Test + void validIntValueMapWithMatchingKeysAndUniqueIntsLoadsClean() { + var loader = new MetaDataLoader(); + assertDoesNotThrow(() -> loader.load(java.util.List.of(new InMemoryMetaDataSource( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9}"), "test.json")))); + } + + @Test + void noIntValueMapStillLoadsCleanStringBackedDefault() { + var loader = new MetaDataLoader(); + assertDoesNotThrow(() -> loader.load(java.util.List.of(new InMemoryMetaDataSource(model(""), "test.json")))); + } + + @Test + void missingMemberKeyIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + assertTrue(ex.getMessage().contains("ARCHIVED")); + } + + @Test + void extraKeyNotInValuesIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9, \"RETRACTED\": 12}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + assertTrue(ex.getMessage().contains("RETRACTED")); + } + + @Test + void nonIntegerValueIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": \"zero\", \"PUBLISHED\": 5, \"ARCHIVED\": 9}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + } + + @Test + void duplicateIntValueAcrossMembersIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 0, \"ARCHIVED\": 9}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + assertTrue(ex.getMessage().contains("DRAFT") && ex.getMessage().contains("PUBLISHED")); + } +} +``` + +> Check this module's actual in-memory-source test helper name/package (`InMemoryMetaDataSource` is a placeholder guess) and the loader's actual single-error-throw-vs-collect behavior against an existing test like `EnumFieldTest.java` before finalizing — mirror its exact loading idiom. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/java && mvn -pl metadata test -Dtest=EnumFieldIntValueMapTest` +Expected: FAIL — `ERR_UNKNOWN_ATTR` on the positive tests. + +- [ ] **Step 3: Write `IntMapAttribute.java`** + +```java +package com.metaobjects.attr; + +import com.metaobjects.DataTypes; +import com.metaobjects.registry.MetaDataRegistry; + +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * An object-shaped attribute whose values are all integers (e.g. field.enum's + * {@code @intValueMap}: {member: int}). Mirrors {@link PropertiesAttribute}'s + * structure, but backed by {@code Map} rather than + * {@code java.util.Properties} — Properties forces every value to a String on + * load, which would silently corrupt int fidelity on canonical-JSON round-trip. + * Generic shape check only (object, every value an integer); a consumer field + * type (field.enum) layers its own semantic rules (key-set membership, + * uniqueness) in its own post-load content-rule validation. + */ +public class IntMapAttribute extends MetaAttribute> { + + public static final String SUBTYPE_INT_MAP = "intMap"; + + public static void registerTypes(MetaDataRegistry registry) { + registry.registerType(IntMapAttribute.class, def -> def + .type(TYPE_ATTR).subType(SUBTYPE_INT_MAP) + .description("An object-shaped attribute whose values are all integers.") + .inheritsFrom(TYPE_ATTR, SUBTYPE_BASE) + ); + } + + public IntMapAttribute(String name) { + super(SUBTYPE_INT_MAP, name, DataTypes.CUSTOM); + } + + public static IntMapAttribute create(String name, Map value) { + IntMapAttribute a = new IntMapAttribute(name); + a.setValue(value); + return a; + } + + @Override + public void setValueAsObject(Object value) { + if (value == null) { + setValue(null); + } else if (value instanceof String) { + setValueAsString((String) value); + } else if (value instanceof Map) { + Map m = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) value).entrySet()) { + if (e.getKey() == null || e.getValue() == null) continue; + m.put(e.getKey().toString(), coerceInt(e.getValue())); + } + setValue(m); + } else { + throw new InvalidAttributeValueException( + "Can not set value with class [" + value.getClass() + "] for object: " + value); + } + } + + @Override + public void setValueAsString(String value) { + if (value == null) { setValue(null); return; } + String trimmed = value.trim(); + if (!(trimmed.startsWith("{") && trimmed.endsWith("}"))) { + throw new InvalidAttributeValueException( + "Could not parse intMap attribute value (expected a JSON object): " + value); + } + com.google.gson.JsonObject obj = com.google.gson.JsonParser.parseString(trimmed).getAsJsonObject(); + Map m = new LinkedHashMap<>(); + for (Map.Entry e : obj.entrySet()) { + com.google.gson.JsonElement el = e.getValue(); + if (el.isJsonPrimitive() && el.getAsJsonPrimitive().isNumber()) { + m.put(e.getKey(), el.getAsInt()); + } else { + // Not a number — record a sentinel that fails the generic content + // check downstream (own validateEnumNode reports the specific error); + // we don't throw here so the loader can report ALL bad members, not + // just the first, matching the other ports' collect-not-throw style + // at the field-content-rule layer. Store Integer.MIN_VALUE as a + // deliberately-invalid marker is avoided — instead we fail loudly here, + // since this attribute type's OWN contract is "every value is an int". + throw new InvalidAttributeValueException( + "attribute '@" + getName() + "' member '" + e.getKey() + "' has value '" + el + + "' which is not an integer"); + } + } + setValue(m); + } + + private static int coerceInt(Object value) { + if (value instanceof Integer i) return i; + if (value instanceof Number n && n.doubleValue() == Math.floor(n.doubleValue())) return n.intValue(); + throw new InvalidAttributeValueException("intMap value is not an integer: " + value); + } + + @Override + public String getValueAsString() { + return getValue() == null ? null : new com.google.gson.Gson().toJson(getValue()); + } +} +``` + +> `EnumFieldIntValueMapTest`'s `nonIntegerValueIsRejected` test expects `ERR_BAD_ATTR_VALUE`, but this class throws `InvalidAttributeValueException` directly — check how `InvalidAttributeValueException` maps to `ErrorCode.ERR_BAD_ATTR_VALUE` elsewhere in the loader (likely a catch-and-wrap in `MetaDataLoader`/`ValidationPhase`) before finalizing; if it doesn't already map that way, catch it at the `field.enum` content-rule layer (`EnumField`/`ValidationPhase`) instead of letting it escape raw. + +- [ ] **Step 4: Register `IntMapAttribute` in the metadata provider bootstrap** + +Find wherever `PropertiesAttribute.registerTypes(registry)` is called (search `FieldTypesMetaDataProvider.java` or the equivalent core-attrs provider file — the same place Task 8's research found `EnumField.registerTypes(registry)` wired at `FieldTypesMetaDataProvider.java:66`) and add a sibling call: + +```java +IntMapAttribute.registerTypes(registry); +``` + +- [ ] **Step 5: Add `ATTR_INT_VALUE_MAP` constant to `EnumField.java`** + +Edit `server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java` — add next to `ATTR_VALUES` (around line 60): + +```java + /** + * Name of the optional per-member explicit-integer-value attribute + * ({@code {member: int}}), switching this enum field's DB persistence from + * string+CHECK to integer+CHECK. Keys must exactly match {@code @values}; + * values must be unique integers. Cross-language vocabulary: + * {@code @intValueMap} in canonical JSON. + */ + public static final String ATTR_INT_VALUE_MAP = "intValueMap"; +``` + +- [ ] **Step 6: Register the attr in `EnumField.registerTypes`** + +Edit the same file's `registerTypes` method (shown in research at line 173-221) — add after the `ATTR_PROVIDED` registration: + +```java + // Optional @intValueMap — an object-shaped attribute whose values + // are all integers. Key-set-matches-@values and uniqueness are + // validated post-load in ValidationPhase (own-only, same as @values). + def.optionalAttributeWithConstraints(ATTR_INT_VALUE_MAP) + .ofType(com.metaobjects.attr.IntMapAttribute.SUBTYPE_INT_MAP) + .asSingle(); +``` + +- [ ] **Step 7: Add the content-rule validation to `ValidationPhase.java`** + +Edit `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` — inside `validateEnumNode` (shown in research at lines 503-538), add after the existing own-`@values` content check and before `validateEnumFr011Attrs(node)` is called: + +```java + // --- Own @intValueMap content check (optional) --- + if (node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false)) { + @SuppressWarnings("unchecked") + java.util.Map intValueMap = + (java.util.Map) node.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false).getValue(); + java.util.List effective = effectiveEnumValues(node); + java.util.Set memberSet = new java.util.HashSet<>(effective); + java.util.Set keySet = intValueMap.keySet(); + + java.util.List missing = effective.stream().filter(m -> !keySet.contains(m)).toList(); + java.util.List extra = keySet.stream().filter(k -> !memberSet.contains(k)).toList(); + if (!missing.isEmpty() || !extra.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' keys must exactly match '@" + EnumField.ATTR_VALUES + "' members" + + (missing.isEmpty() ? "" : " (missing: " + String.join(", ", missing) + ")") + + (extra.isEmpty() ? "" : " (unknown: " + String.join(", ", extra) + ")") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + + java.util.Map seenValues = new java.util.HashMap<>(); + for (var entry : intValueMap.entrySet()) { + Integer value = entry.getValue(); + String owner = seenValues.putIfAbsent(value, entry.getKey()); + if (owner != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' members '" + owner + "' and '" + entry.getKey() + + "' share the same value " + value + "; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + } + } +``` + +> `effectiveEnumValues(node)` already exists per research (line 602-619) — confirm its exact return type (`List`) matches this usage. + +- [ ] **Step 8: Run tests — confirm all pass** + +Run: `cd server/java && mvn -pl metadata test -Dtest=EnumFieldIntValueMapTest` +Expected: PASS — all 6 tests green. + +- [ ] **Step 9: Run the full Java metadata module test suite** + +Run: `cd server/java && mvn -pl metadata test` +Expected: all pass, no regressions. + +- [ ] **Step 10: Commit** + +```bash +git add server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java +git commit -m "feat(java): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 9: Java (+ Kotlin) — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the shared conformance fixtures against Java's loader** + +Run: `cd server/java && mvn -pl metadata test -Dtest=*ConformanceTest*` (adjust to the actual test class that discovers `fixtures/conformance/*`) +Expected: the five fixtures from Task 3 all pass. + +- [ ] **Step 2: Run Java's registry-conformance test** + +Run: `cd server/java && mvn -pl metadata test -Dtest=*RegistryConformance*` +Expected: PASS against Task 4's updated `expected-registry.json`. + +- [ ] **Step 3: Run the Kotlin module's tests to confirm no regression** (Kotlin shares Java's metadata layer — this is a smoke check, not new Kotlin-specific work) + +Run: `cd server/java && mvn -pl codegen-kotlin,metadata-ktx test` +Expected: all pass, no regressions. + +- [ ] **Step 4: Run the full Java build** + +Run: `cd server/java && mvn test` +Expected: 100% pass (excluding any pre-existing known-red modules unrelated to this change). + +--- + +### Task 10: Python — `attr.intMap` subtype + `field.enum`'s `@intValueMap` + +**Files:** +- Modify: `server/python/src/metaobjects/meta/core/attr/attr_constants.py` +- Modify: `server/python/src/metaobjects/meta/core/attr/meta_attr.py` +- Modify: `server/python/src/metaobjects/spec_metamodel/attr.json` +- Modify: `server/python/src/metaobjects/spec_metamodel/field.json` +- Modify: `server/python/src/metaobjects/field_constants.py` +- Modify: `server/python/src/metaobjects/loader/validation_passes.py` +- Test: `server/python/tests/unit/test_field_enum_intvaluemap.py` + +**Interfaces:** +- Produces: `FIELD_ATTR_INT_VALUE_MAP`, `ATTR_SUBTYPE_INT_MAP` — consumed by the Python persistence follow-on plan. + +- [ ] **Step 1: Write the failing tests** + +```python +# server/python/tests/unit/test_field_enum_intvaluemap.py +import pytest +from metaobjects.loader.loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.errors import ErrorCode, MetaDataException + + +def _model(extra: str) -> str: + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def _load(json_str: str): + loader = MetaDataLoader() + return loader.load([InMemoryStringSource(json_str, "test.json")]) + + +def test_valid_intvaluemap_with_matching_keys_and_unique_ints_loads_clean(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')) + assert result.errors == [] + + +def test_no_intvaluemap_still_loads_clean_string_backed_default(): + result = _load(_model("")) + assert result.errors == [] + + +def test_missing_member_key_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "ARCHIVED" in result.errors[0].message + + +def test_extra_key_not_in_values_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "RETRACTED" in result.errors[0].message + + +def test_non_integer_value_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + + +def test_duplicate_int_value_across_members_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "DRAFT" in result.errors[0].message and "PUBLISHED" in result.errors[0].message +``` + +> Check `test_field_enum.py`'s actual `MetaDataLoader`/source-loading and `result.errors` shape (some Python loader paths raise `MetaDataException` on first error rather than collecting) before finalizing — mirror its established pattern exactly. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/python && pytest tests/unit/test_field_enum_intvaluemap.py -v` +Expected: FAIL — `ERR_UNKNOWN_ATTR` on the positive tests. + +- [ ] **Step 3: Add `ATTR_SUBTYPE_INT_MAP` constant** + +Edit `server/python/src/metaobjects/meta/core/attr/attr_constants.py` — add next to `ATTR_SUBTYPE_PROPERTIES`: + +```python +ATTR_SUBTYPE_INT_MAP = "intMap" +``` + +- [ ] **Step 4: Add the `IntMapAttr` class** + +Edit `server/python/src/metaobjects/meta/core/attr/meta_attr.py` — add next to `PropertiesAttr` (around line 131): + +```python +class IntMapAttr(MetaAttr): + """attr.intMap — an object-shaped attribute whose values are all integers + (e.g. field.enum's @intValueMap). Generic shape check only; a consumer + field type (field.enum) layers its own semantic rules (key-set membership, + uniqueness) in its own content-rule validation pass.""" + + @property + def data_type(self) -> DataType: + return DataType.OBJECT + + def coerce(self, raw: object) -> object: + return raw + + def validate_value(self, value: object) -> list[ValueError]: + if not isinstance(value, dict): + return [ValueError(f"attribute '@{self.name}' must be of type 'intMap' but got {type(value).__name__}")] + errors: list[ValueError] = [] + for key, member in value.items(): + if isinstance(member, bool) or not isinstance(member, int): + errors.append(ValueError(f"attribute '@{self.name}' member '{key}' has value '{member}' which is not an integer")) + return errors +``` + +> Check `MetaAttr`'s actual `validate_value` signature/return type (the existing `PropertiesAttr`/`FilterAttr` classes in this file don't override it, so confirm the base class's default and this override's exact contract — e.g. does it return a list of `ValueError` instances, plain strings, or something else — against `StringArrayAttr`'s override, which likely DOES override validation, before finalizing.) Note Python's `bool` is a subclass of `int` — the `isinstance(member, bool)` guard above is required to correctly reject a JSON `true`/`false` value. + +- [ ] **Step 5: Register the class** + +Edit the same file's registration block (around line 143-148): + +```python +register_attr_class(ATTR_SUBTYPE_PROPERTIES, PropertiesAttr) +register_attr_class(ATTR_SUBTYPE_EXPRESSION, ExpressionAttr) +register_attr_class(ATTR_SUBTYPE_INT_MAP, IntMapAttr) +``` + +- [ ] **Step 6: Add the `attr.intMap` type declaration** + +Edit `server/python/src/metaobjects/spec_metamodel/attr.json` with the same JSON snippet used in TS Task 1 Step 6 (Python's own packaged copy — keep semantically identical to the canonical spec). + +- [ ] **Step 7: Add `FIELD_ATTR_INT_VALUE_MAP` constant** + +Edit `server/python/src/metaobjects/field_constants.py` — add next to wherever `FIELD_ATTR_VALUES`/`ENUM_MEMBER_PATTERN` are declared (per research, `field_constants.py:147` for `ENUM_MEMBER_PATTERN`): + +```python +FIELD_ATTR_INT_VALUE_MAP = "intValueMap" +``` + +- [ ] **Step 8: Add the `field.enum.intValueMap` declaration** + +Edit `server/python/src/metaobjects/spec_metamodel/field.json` with the same JSON snippet used in TS Task 2 Step 4. + +- [ ] **Step 9: Add the content-rule validation** + +Edit `server/python/src/metaobjects/loader/validation_passes.py` — inside `_validate_enum_values` (shown in research at line 534), add after Rule 3 (no duplicates, ends around line 592) and before the function returns: + +```python + # Rule 4: @intValueMap content rules (optional). + # a. Key set must exactly match @values. + # b. No two members may share the same int (protobuf's stance — no + # alias opt-in). Every-value-is-an-integer is already enforced + # by IntMapAttr's generic shape validation at parse time. + int_value_map = node.attr(FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + member_set = set(own_values) + key_set = set(int_value_map.keys()) + missing = [m for m in own_values if m not in key_set] + extra = [k for k in int_value_map if k not in member_set] + if missing or extra: + parts = [] + if missing: + parts.append(f"missing: {', '.join(missing)}") + if extra: + parts.append(f"unknown: {', '.join(extra)}") + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match " + f"'@{FIELD_ATTR_VALUES}' members ({'; '.join(parts)}).", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + + seen_values: dict[int, str] = {} + for member, value in int_value_map.items(): + if not isinstance(value, int) or isinstance(value, bool): + continue # IntMapAttr already reported this + owner = seen_values.get(value) + if owner is not None: + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members {owner!r} and {member!r} " + f"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + else: + seen_values[value] = member +``` + +Add `FIELD_ATTR_INT_VALUE_MAP` to this file's existing import block from `field_constants`. + +- [ ] **Step 10: Run tests — confirm all pass** + +Run: `cd server/python && pytest tests/unit/test_field_enum_intvaluemap.py -v` +Expected: PASS — all 6 tests green. + +- [ ] **Step 11: Run the full Python test suite** + +Run: `cd server/python && pytest` +Expected: all pass, no regressions. + +- [ ] **Step 12: Commit** + +```bash +git add server/python/src/metaobjects/meta/core/attr/attr_constants.py server/python/src/metaobjects/meta/core/attr/meta_attr.py server/python/src/metaobjects/spec_metamodel/attr.json server/python/src/metaobjects/spec_metamodel/field.json server/python/src/metaobjects/field_constants.py server/python/src/metaobjects/loader/validation_passes.py server/python/tests/unit/test_field_enum_intvaluemap.py +git commit -m "feat(python): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 11: Python — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the shared conformance fixtures against Python's loader** + +Run: `cd server/python && pytest tests/ -k conformance` +Expected: the five fixtures from Task 3 all pass. + +- [ ] **Step 2: Run Python's registry-conformance test** + +Run: `cd server/python && pytest tests/ -k registry_conformance` +Expected: PASS against Task 4's updated `expected-registry.json`. + +- [ ] **Step 3: Run the full Python test suite** + +Run: `cd server/python && pytest` +Expected: 100% pass. + +--- + +## After this plan lands + +This plan ships a fully validated, cross-port `@intValueMap` vocabulary that loads, validates, and round-trips correctly — but nothing reads it yet. Follow-on plans (one per port/group, written separately per the Scope Check in `superpowers:writing-plans`): + +- **TS persistence** — migrate-ts DDL (`integer` + int `CHECK` instead of `text`/`varchar` + string `CHECK`), the migration-safety guard (D8: refuse to auto-`ALTER` an existing column across backing modes), and the Drizzle/Kysely symbol↔int codec in generated `queries.ts`/`entity.ts`. +- **C# persistence** — an EF Core `HasConversion` built from `@intValueMap`'s lookup table (replacing `HasConversion()` only when `@intValueMap` is present). +- **Java + Kotlin persistence** — a new OMDB JDBC codec (binding `Types.INTEGER`, mirroring the `CurrencyCodec`/`UuidCodec` pattern in `JdbcCodecs.java`) and, for Kotlin, switching `KotlinExposedTableGenerator.kt`'s `enumerationByName(...)` to `enumeration(...)` when `@intValueMap` is present. +- **Python persistence** — a new branch in `ObjectManager`'s scalar-coercion function (currently a pure fallthrough for enums) translating symbol↔int at the DB boundary. +- **Cross-port persistence-conformance** — extend `roundtrip-all-types.yaml` + `meta.fitness.json` with an int-backed enum field, round-tripping through every port's real runtime. From 888f0e2f66e42f1e0821205ba6d015c27bf1b831 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 17:04:00 -0400 Subject: [PATCH 04/52] docs(plan): implementation plan for field.enum @intValueMap (TS persistence layer) --- ...3-int-backed-enum-values-ts-persistence.md | 693 ++++++++++++++++++ 1 file changed, 693 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md new file mode 100644 index 000000000..ff9db25d1 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md @@ -0,0 +1,693 @@ +# Int-Backed Enum Values — TypeScript Persistence Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped in `docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md`) into TypeScript's persistence stack: migrate-ts emits `integer` + a numeric `CHECK` instead of `text` + a string `CHECK`, codegen-ts's Drizzle column mapper follows suit, and the generated entity/query code translates symbol↔int at the DB boundary while every TS-facing type (Zod schema, `EntityType`, wire JSON) stays exactly the string union it is today. + +**Architecture:** `@intValueMap`'s presence is read in exactly three places, each already dispatching on `field.subType === FIELD_SUBTYPE_ENUM`: migrate-ts's `subtypeToSqlType`/`arrayElementSqlType`/`buildChecks` (schema/DDL), codegen-ts's `column-mapper.ts` (Drizzle column declaration), and a **new** generated symbol↔int lookup consumed by two new template hooks — a Zod `.transform()` on write (an established pattern already used for `@autoSet` timestamps) and a small generated `decode` step on read (a genuinely new pattern for this codebase — no field has ever needed a wire type that differs from its storage type before). The migration-safety guard from the design's D8 needs **no new code at all**: this codebase's existing `isWidening`/`allow.typeChange` mechanism (`packages/migrate-ts/src/sql-type.ts` + `packages/migrate-ts/src/diff/status.ts`) already treats any cross-`kind` `change-column-type` (text↔integer included) as blocked-by-default requiring an explicit `allow.typeChange` pass — confirmed by reading `isWidening`'s `if (from.kind !== to.kind) return false` and `blockedReasonFor`'s `case "change-column-type"` branch. This task's job here is a **test proving that's true**, not new gating logic. + +**Tech Stack:** TypeScript, Bun test runner, Drizzle ORM, Kysely (via `@metaobjectsdev/runtime-ts`), Zod. + +## Global Constraints + +- Every TS-facing type (Zod schema output type, generated `EntityType`, the JSON wire payload) is BYTE-IDENTICAL between string-backed and int-backed enums — string in, string out, everywhere except the literal DB column. +- `@intValueMap` presence alone is the trigger — no new codegen option, no new CLI flag. +- Do not touch the metamodel/validation layer — that plan already shipped. If a step here tempts you to edit `attr-schema-validate.ts` or add a new attr constant, stop; it's already done. +- The migration-safety guard requires ZERO new gating code (see Architecture) — this plan's migration-safety task is a test, not an implementation. + +--- + +### Task 1: migrate-ts — `integer` column type for int-backed enums + +**Files:** +- Modify: `packages/migrate-ts/src/expected-schema.ts` +- Test: `packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: `FIELD_ATTR_INT_VALUE_MAP` (metamodel plan, already shipped). +- Produces: `buildExpectedSchema(...)` returns `{ kind: "integer", bits: 32 }` for a scalar int-backed enum column, `{ kind: "array", element: { kind: "integer", bits: 32 } }` for an array one — consumed by Task 2 (CHECK) and by `diff/index.ts`'s existing (unmodified) column-type comparison. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts +import { describe, test, expect } from "bun:test"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { loadFixture } from "../fixtures/load.js"; // match the existing helper used by expected-schema.test.ts + +describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { + test("scalar int-backed enum maps to integer, not text", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const col = snapshot.tables[0]!.columns.find((c) => c.name === "status")!; + expect(col.sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("string-backed enum (no @intValueMap) is unchanged", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const col = snapshot.tables[0]!.columns.find((c) => c.name === "status")!; + expect(col.sqlType).toEqual({ kind: "text" }); + }); + + test("array-of-int-backed-enum maps to integer[]", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "labels", "isArray": true, "@values": ["LOW","MEDIUM","HIGH"], "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const col = snapshot.tables[0]!.columns.find((c) => c.name === "labels")!; + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); + }); +}); +``` + +> Check `packages/migrate-ts/test/unit/expected-schema.test.ts`'s actual fixture-loading helper (`loadFixture`, or whatever it's really called) before finalizing — mirror its exact signature. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: FAIL — the first and third tests currently get `{ kind: "text" }`/`{ kind: "array", element: { kind: "text" } }`. + +- [ ] **Step 3: Add the scalar case to `subtypeToSqlType`** + +Edit `packages/migrate-ts/src/expected-schema.ts` — in `subtypeToSqlType` (the switch shown in research, ending at the `default: return { kind: "text" }` around line 1045), add a case immediately before `default`: + +```typescript + case FIELD_SUBTYPE_ENUM: + // @intValueMap present → this enum is int-backed (docs/superpowers/specs/ + // 2026-07-23-int-backed-enum-values-design.md D5/D6). ADR-0039: resolving — + // @intValueMap may be inherited via extends, same as @values. + return field.attr(FIELD_ATTR_INT_VALUE_MAP) !== undefined + ? { kind: "integer", bits: 32 } + : { kind: "text" }; +``` + +Add `FIELD_SUBTYPE_ENUM` and `FIELD_ATTR_INT_VALUE_MAP` to this file's existing `@metaobjectsdev/metadata` import block (both already exist as exports — `FIELD_SUBTYPE_ENUM` is likely already imported for other uses in this file; check before adding a duplicate). + +- [ ] **Step 4: Add the array-element case to `arrayElementSqlType`** + +Edit the same file's `arrayElementSqlType` function (research found the enum branch around line 949, grouped with `FIELD_SUBTYPE_URI`): + +```typescript + case FIELD_SUBTYPE_ENUM: + // enum[] stores as text[] (string-backed) or integer[] (int-backed, when + // @intValueMap is present); membership is app-level either way (no CHECK + // on array columns — see buildChecks). + return field.attr(FIELD_ATTR_INT_VALUE_MAP) !== undefined + ? { kind: "integer", bits: 32 } + : { kind: "text" }; + case FIELD_SUBTYPE_URI: + return { kind: "text" }; +``` + +(Remove the old combined `case FIELD_SUBTYPE_ENUM: case FIELD_SUBTYPE_URI: return { kind: "text" };` line and replace with the two separate cases above.) + +- [ ] **Step 5: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: PASS — all 3 tests green. + +- [ ] **Step 6: Run the full migrate-ts suite** + +Run: `cd server/typescript && bun test packages/migrate-ts` +Expected: all pass, no regressions (this step only adds a new case; every existing string-backed-enum test is untouched). + +- [ ] **Step 7: Commit** + +```bash +git add server/typescript/packages/migrate-ts/src/expected-schema.ts server/typescript/packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts +git commit -m "feat(migrate-ts): int-backed field.enum (@intValueMap) maps to integer, not text" +``` + +--- + +### Task 2: migrate-ts — numeric `CHECK` constraint for int-backed enums + +**Files:** +- Modify: `packages/migrate-ts/src/expected-schema.ts` +- Test: extend `packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: Task 1. + +- [ ] **Step 1: Write the failing test** + +Append to `expected-schema-enum-intvaluemap.test.ts`: + +```typescript +test("int-backed enum gets a numeric CHECK (unquoted literals)", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const check = snapshot.tables[0]!.checks.find((c) => c.name === "orders_status_chk")!; + expect(check.expression).toBe('"status" IN (0, 5, 9)'); +}); + +test("int-backed array-of-enum gets NO check (array columns never get a CHECK)", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "labels", "isArray": true, "@values": ["LOW","MEDIUM"], "@intValueMap": { "LOW": 1, "MEDIUM": 2 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + expect(snapshot.tables[0]!.checks.find((c) => c.name === "tickets_labels_chk")).toBeUndefined(); +}); +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: FAIL — the CHECK expression is currently the quoted-string form (`"status" IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')`), since `buildChecks` doesn't yet know about `@intValueMap`. + +- [ ] **Step 3: Update `buildChecks`'s enum branch** + +Edit `packages/migrate-ts/src/expected-schema.ts`'s `buildChecks` function (the block shown in research at lines 653-664): + +```typescript + // Enum membership check. + if (field.subType === FIELD_SUBTYPE_ENUM) { + const intValueMap = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (intValueMap !== undefined && typeof intValueMap === "object" && intValueMap !== null) { + // Int-backed: unquoted numeric literals, in the SAME member order as @values + // (cosmetic only — SQL IN() is order-independent, but matching declaration + // order keeps the emitted CHECK stable/predictable for diffing). + const raw = field.attr(FIELD_ATTR_VALUES); + const members: string[] = Array.isArray(raw) ? raw.map((v) => String(v)) : []; + const ints = members.map((m) => (intValueMap as Record)[m]); + const expression = `${qcol} IN (${ints.join(", ")})`; + checks.push({ name: `${tableName}_${col}_chk`, expression }); + } else { + const raw = field.attr(FIELD_ATTR_VALUES); + if (Array.isArray(raw) && raw.length > 0) { + const values = raw.map((v) => String(v)); + const expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; + checks.push({ name: `${tableName}_${col}_chk`, expression }); + } + } + } +``` + +- [ ] **Step 4: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: PASS — all 5 tests in this file green. + +- [ ] **Step 5: Run the full migrate-ts suite** + +Run: `cd server/typescript && bun test packages/migrate-ts` +Expected: all pass, no regressions. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/migrate-ts/src/expected-schema.ts server/typescript/packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts +git commit -m "feat(migrate-ts): int-backed field.enum CHECK uses unquoted numeric literals" +``` + +--- + +### Task 3: migrate-ts — prove the migration-safety guard (no new code) + +**Files:** +- Test: `packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts` + +**Interfaces:** +- Consumes: Task 1 (column type), the existing `isWidening`/`applyStatus` mechanism (unmodified). + +- [ ] **Step 1: Write the real-engine test** + +```typescript +// packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts +import { describe, test, expect } from "bun:test"; +import Database from "better-sqlite3"; // match whatever driver this test dir's other integration tests already use +import { diff } from "../../src/diff/index.js"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { introspect } from "../../src/introspect/sqlite.js"; // match the actual introspect module path used by sibling tests +import { loadFixture } from "../fixtures/load.js"; +import { apply } from "../../src/act/sqlite.js"; // match the actual apply/act module path + +describe("SQLite enum backing-mode change — real-engine migration-safety guard", () => { + test("adding @intValueMap to an EXISTING string-backed enum column is BLOCKED by default", async () => { + const db = new Database(":memory:"); + const stringBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const initial = buildExpectedSchema(stringBackedRoot, { dialect: "sqlite" }); + // Apply the initial (string-backed) schema for real, so introspection sees a real + // existing text column — this is what makes the guard fire (there is no diff at + // all against an empty DB; the guard is specifically about an EXISTING column). + const firstDiff = diff({ actual: { tables: [], views: [] }, expected: initial, dialect: "sqlite" }); + await apply(db, firstDiff.changes); + + const intBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const expected = buildExpectedSchema(intBackedRoot, { dialect: "sqlite" }); + const actual = await introspect(db, "sqlite"); + const secondDiff = diff({ actual, expected, dialect: "sqlite" }); + + const typeChange = secondDiff.changes.find((c) => c.kind === "change-column-type" && c.column === "status"); + expect(typeChange).toBeDefined(); + expect(typeChange!.status.state).toBe("blocked"); + expect(typeChange!.status.blockedReason).toContain("allow.typeChange"); + expect(secondDiff.blocked).toContain(typeChange); + }); + + test("passing allow.typeChange unblocks it (operator opt-in, per D8 — no silent auto-migration)", async () => { + const db = new Database(":memory:"); + const stringBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const initial = buildExpectedSchema(stringBackedRoot, { dialect: "sqlite" }); + const firstDiff = diff({ actual: { tables: [], views: [] }, expected: initial, dialect: "sqlite" }); + await apply(db, firstDiff.changes); + + const intBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const expected = buildExpectedSchema(intBackedRoot, { dialect: "sqlite" }); + const actual = await introspect(db, "sqlite"); + const secondDiff = diff({ actual, expected, dialect: "sqlite", allow: { typeChange: true } }); + + const typeChange = secondDiff.changes.find((c) => c.kind === "change-column-type" && c.column === "status"); + expect(typeChange!.status.state).toBe("allowed"); + }); +}); +``` + +> This test's exact imports (`introspect`, `apply`, the SQLite driver) must match whatever `packages/migrate-ts/test/integration/sqlite-autoset-default.test.ts` (found during earlier investigation of this same package) actually uses — read that file first and mirror its setup/teardown and import paths exactly; the sketch above captures the SCENARIO, not necessarily the exact API surface. + +- [ ] **Step 2: Run to verify current behavior** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts` +Expected: Both tests should already PASS once Task 1 lands, since `isWidening`/`blockedReasonFor` need no changes — this step is confirming that expectation, not driving new implementation. If either test fails, the failure itself is the signal that D8's assumption (the existing generic mechanism already covers this) was wrong, and this task's scope needs to grow into an actual new guard (re-read `blockedReasonFor` in `packages/migrate-ts/src/diff/status.ts` and add a `change-column-type` sub-case specific to enum `text↔integer` transitions before proceeding). + +- [ ] **Step 3: Run the full migrate-ts suite** + +Run: `cd server/typescript && bun test packages/migrate-ts` +Expected: all pass. + +- [ ] **Step 4: Commit** + +```bash +git add server/typescript/packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts +git commit -m "test(migrate-ts): prove the existing allow.typeChange guard blocks enum backing-mode changes" +``` + +--- + +### Task 4: codegen-ts — Drizzle column mapper + +**Files:** +- Modify: `packages/codegen-ts/src/column-mapper.ts` +- Test: `packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: `enumValues(field)` from `packages/codegen-ts/src/enum-meta.ts` (existing) — this task adds a sibling `intValueMap(field)` reader to the same file. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts +import { describe, test, expect } from "bun:test"; +import { mapColumnType } from "../../src/column-mapper.js"; +import { loadFixture } from "../fixtures/load.js"; // match whatever helper column-mapper.test.ts already uses + +describe("column-mapper — int-backed field.enum (@intValueMap)", () => { + test("postgres: emits integer(...), not text(..., {enum:[...]})", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } } + ]}} + ]}}`); + const field = root.findObject("Order")!.fields().find((f) => f.name === "status")!; + const result = mapColumnType(field, "postgres"); + expect(result.fnName).toBe("integer"); + expect(result.fnOptions).toBeUndefined(); + expect(result.checkConstraint).toBe('"status" IN (0, 5)'); + }); + + test("sqlite: emits integer(...), not text(..., {enum:[...]})", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } } + ]}} + ]}}`); + const field = root.findObject("Order")!.fields().find((f) => f.name === "status")!; + const result = mapColumnType(field, "sqlite"); + expect(result.fnName).toBe("integer"); + }); + + test("string-backed enum (no @intValueMap) is unchanged on both dialects", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } } + ]}} + ]}}`); + const field = root.findObject("Order")!.fields().find((f) => f.name === "status")!; + expect(mapColumnType(field, "postgres").fnName).toBe("text"); + expect(mapColumnType(field, "sqlite").fnName).toBe("text"); + }); +}); +``` + +> `mapColumnType`'s exact exported name/signature and its result shape (`fnName`/`fnOptions`/`checkConstraint`) are inferred from the research report's excerpts — confirm against the actual function signature in `column-mapper.ts` (search for `export function mapColumnType` or similar) before finalizing. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts` +Expected: FAIL — currently `fnName` is `"text"` in every case. + +- [ ] **Step 3: Add `intValueMap(field)` to `enum-meta.ts`** + +Edit `packages/codegen-ts/src/enum-meta.ts` — add a sibling to `enumValues`: + +```typescript +export function intValueMap(field: MetaField): Record | undefined { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + return typeof raw === "object" && raw !== null ? (raw as Record) : undefined; +} +``` + +- [ ] **Step 4: Update `column-mapper.ts`'s dialect switches** + +Edit both the SQLite switch (research: `:399`) and the Postgres switch (research: `:513-516`) to check `intValueMap(field)` before falling into the text case: + +```typescript + case FIELD_SUBTYPE_ENUM: { + fnName = intValueMap(field) !== undefined ? "integer" : "text"; + break; + } +``` + +(For Postgres, this REPLACES the current `default: fnName = "text"` grouping for enum — give enum its own case above `default` so it no longer falls into the shared default branch.) + +- [ ] **Step 5: Guard the literal-union `{enum:[...]}` option and the CHECK-constraint emission** + +Edit the existing blocks at research lines 527-532 and 663-674 — both are already gated on `fnName === "text"`/`subType === FIELD_SUBTYPE_ENUM && !isArray`, so once Step 4 makes `fnName` become `"integer"` for int-backed enums, the `{enum:[...]}` block already naturally skips (its `fnName === "text"` guard is now false). The CHECK-constraint block, however, unconditionally quotes string values — update it: + +```typescript +if (subType === FIELD_SUBTYPE_ENUM && !isArray) { + const map = intValueMap(field); + if (map !== undefined) { + const members = enumValues(field) ?? []; + const ints = members.map((m) => map[m]); + result.checkConstraint = `${dbName} IN (${ints.join(", ")})`; + } else { + const values = enumValues(field); + if (values !== undefined && values.length > 0) { + const list = values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", "); + result.checkConstraint = `${dbName} IN (${list})`; + } + } +} +``` + +- [ ] **Step 6: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts` +Expected: PASS — all 3 tests green. + +- [ ] **Step 7: Run the full codegen-ts suite** + +Run: `cd server/typescript && bun test packages/codegen-ts` +Expected: all pass, no regressions in existing golden-file snapshots (no existing fixture uses `@intValueMap`, so no snapshot should change). + +- [ ] **Step 8: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/enum-meta.ts server/typescript/packages/codegen-ts/src/column-mapper.ts server/typescript/packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts +git commit -m "feat(codegen-ts): Drizzle column-mapper emits integer(...) for int-backed field.enum" +``` + +--- + +### Task 5: codegen-ts — symbol↔int codec at the write/read boundary + +**Files:** +- Modify: `packages/codegen-ts/src/templates/zod-validators.ts` +- Modify: `packages/codegen-ts/src/templates/entity-file.ts` +- Modify: `packages/codegen-ts/src/templates/queries-file.ts` +- Test: `packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts` + +**Interfaces:** +- Consumes: `intValueMap(field)` (Task 4). +- Produces: for each int-backed enum field, a generated `_TO_INT`/`_FROM_INT` lookup pair (emitted in the entity file), a Zod `.transform()` on the Insert/Update schema (write), and a `decodeRow` helper applied at every read call site in `queries-file.ts` (read). + +**Note — this is a genuinely new pattern for this codebase.** Research confirmed no existing field type needs a wire type that differs from its DB storage type; every existing "conversion" (SQLite boolean `mode`, Drizzle `numeric`→`string`) is the driver's own built-in behavior, not codegen-authored translation. Read every file this task touches in full before editing — the excerpts below are the exact, verified current content at the cited lines, but the surrounding template has more call sites than shown (e.g. TPH polymorphic reads) that may need the identical treatment; this task covers the vanilla (non-TPH) path completely and flags TPH as a follow-up if discovered incomplete. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts +import { describe, test, expect } from "bun:test"; +import { runGen } from "../../src/runner.js"; // match whatever golden-file tests already use to drive a full gen pass +import { loadFixtureProject } from "../fixtures/load-project.js"; // match the existing golden-test harness + +describe("int-backed field.enum — generated codec", () => { + test("entity file emits a symbol<->int lookup pair", async () => { + const output = await runGen(await loadFixtureProject("enum-int-backed")); + const entityFile = output.find((f) => f.path.endsWith("Order.ts"))!.contents; + expect(entityFile).toContain('const ORDER_STATUS_TO_INT'); + expect(entityFile).toContain('const ORDER_STATUS_FROM_INT'); + }); + + test("insert schema transforms the symbol to its int before Drizzle sees it", async () => { + const output = await runGen(await loadFixtureProject("enum-int-backed")); + const entityFile = output.find((f) => f.path.endsWith("Order.ts"))!.contents; + expect(entityFile).toContain('.transform((v) => ORDER_STATUS_TO_INT[v])'); + }); + + test("read paths decode the int back to its symbol", async () => { + const output = await runGen(await loadFixtureProject("enum-int-backed")); + const queriesFile = output.find((f) => f.path.endsWith("Order.queries.ts"))!.contents; + expect(queriesFile).toContain("decodeOrderRow"); + }); + + test("string-backed enum (no @intValueMap) generates no codec at all", async () => { + const output = await runGen(await loadFixtureProject("enum-inline")); // the pre-existing string-backed fixture + const entityFile = output.find((f) => f.path.endsWith(".ts") && !f.path.includes("queries"))!.contents; + expect(entityFile).not.toContain("_TO_INT"); + expect(entityFile).not.toContain("_FROM_INT"); + }); +}); +``` + +> `runGen`/`loadFixtureProject` are placeholders for whatever harness `packages/codegen-ts/test/golden/*.test.ts` actually uses to drive a full generator pass against a fixture project — read one of those tests first and mirror its exact setup. You will also need a new fixture project under wherever `enum-inline`-equivalent codegen fixtures live for THIS package (distinct from the `fixtures/conformance/` ones — codegen-ts likely has its own `test/golden/` or `test/fixtures/` project layout) declaring an `Order` entity with an int-backed `status` field. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts` +Expected: FAIL — none of this generated content exists yet. + +- [ ] **Step 3: Emit the lookup pair in `entity-file.ts`** + +Edit `packages/codegen-ts/src/templates/entity-file.ts` — for each field where `intValueMap(field) !== undefined`, emit (near the top of the file, alongside other per-entity constants): + +```typescript +function intBackedEnumConstants(entity: MetaObject, ctx: GenContext): Code[] { + const blocks: Code[] = []; + for (const field of entity.fields()) { + if (field.subType !== FIELD_SUBTYPE_ENUM) continue; + const map = intValueMap(field); + if (map === undefined) continue; + const constName = `${screamingSnake(entity.name)}_${screamingSnake(field.name)}`; + const toIntEntries = Object.entries(map).map(([k, v]) => ` ${JSON.stringify(k)}: ${v},`).join("\n"); + const fromIntEntries = Object.entries(map).map(([k, v]) => ` ${v}: ${JSON.stringify(k)},`).join("\n"); + blocks.push(code` +const ${constName}_TO_INT: Record = { +${toIntEntries} +}; +const ${constName}_FROM_INT: Record = { +${fromIntEntries} +}; +`); + } + return blocks; +} +``` + +Call `intBackedEnumConstants(entity, ctx)` from this file's main generator function and splice its output into the emitted file (alongside the existing Zod schema / type declarations — match how this file already assembles its output `Code[]` array). + +> `screamingSnake` is a naming helper this codebase may already have (check `naming.ts` or similar in `packages/codegen-ts/src/`) — use the existing helper rather than writing a new one if one exists. + +- [ ] **Step 4: Add the write-side `.transform()` in `zod-validators.ts`** + +Edit `packages/codegen-ts/src/templates/zod-validators.ts` — in the same loop shown in research (lines 189-197, the `autoSet` check), add an `else if` branch before the final `else`: + +```typescript + const autoSet = child.attr(FIELD_ATTR_AUTO_SET); + const map = child.subType === FIELD_SUBTYPE_ENUM ? intValueMap(child) : undefined; + + if (autoSet === AUTO_SET_ON_CREATE || autoSet === AUTO_SET_ON_UPDATE) { + insertFieldLines.push( + code` ${child.name}: z.string().optional().transform(() => new Date().toISOString())`, + ); + } else if (map !== undefined) { + const constName = `${screamingSnake(obj.name)}_${screamingSnake(child.name)}`; + insertFieldLines.push( + code` ${child.name}: ${zodFieldExpr(child, obj, ctx)}.transform((v) => ${constName}_TO_INT[v])`, + ); + } else { + insertFieldLines.push(code` ${child.name}: ${zodFieldExpr(child, obj, ctx)}`); + } +``` + +Apply the identical `else if (map !== undefined)` branch to whichever sibling loop builds the Update schema's field lines (search this file for the second occurrence of a field-line-building loop — Insert and Update schemas are built by parallel loops in this file per the earlier research on `zod-validators.ts`). + +- [ ] **Step 5: Add the read-side `decodeRow` helper and wire it into `queries-file.ts`** + +Edit `packages/codegen-ts/src/templates/entity-file.ts` — emit one decode helper per entity that has ANY int-backed enum field: + +```typescript +function decodeRowHelper(entity: MetaObject): Code | null { + const intBackedFields = entity.fields().filter( + (f) => f.subType === FIELD_SUBTYPE_ENUM && intValueMap(f) !== undefined, + ); + if (intBackedFields.length === 0) return null; + const assignments = intBackedFields.map((f) => { + const constName = `${screamingSnake(entity.name)}_${screamingSnake(f.name)}`; + return ` ${f.name}: ${constName}_FROM_INT[row.${f.name}],`; + }).join("\n"); + return code` +export function decode${entity.name}Row `${f.name}: number`).join("; ")} }>(row: T) { + return { + ...row, +${assignments} + }; +} +`; +} +``` + +Edit `packages/codegen-ts/src/templates/queries-file.ts` — wrap every read return with `decode${entityName}Row(...)` when the entity has at least one int-backed enum field (check once at the top of this file's generator function, e.g. `const hasIntBackedEnum = entity.fields().some((f) => f.subType === FIELD_SUBTYPE_ENUM && intValueMap(f) !== undefined)`), updating the `reads` block (research lines 165-175): + +```typescript +export async function ${findByIdFnName(entityName)}(db: Db, ${pkField}: ${pkType}): Promise<${entityName} | null> { + const [row] = await db.select().from(${viewVar}).where(${eqSym}(${viewVar}.${pkField}, ${pkField})).limit(1); + return row ${hasIntBackedEnum ? `? decode${entityName}Row(row)` : "??"} : null; +} + +export async function ${listFnName(entityName)}(db: Db, opts?: { limit?: number; offset?: number }): Promise<${entityName}[]> { + let q = db.select().from(${viewVar}).$dynamic(); + if (opts?.limit !== undefined) q = q.limit(opts.limit); + if (opts?.offset !== undefined) q = q.offset(opts.offset); + ${hasIntBackedEnum ? "return (await q).map(decode" + "${entityName}" + "Row);" : "return q;"} +} +``` + +> Write this as real template-string interpolation matching this file's actual `code\`...\`` tagged-template style — the snippet above is illustrative of the LOGIC (conditionally wrap with decode), not literal copy-paste, since the exact ternary-inside-template-literal syntax needs to match this codebase's `ts-poet`/`code` helper conventions. Also update `insertReturningView`'s second `db.select()` (research lines 232-251) with the same wrapping. + +- [ ] **Step 6: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts` +Expected: PASS — all 4 tests green. + +- [ ] **Step 7: Run the full codegen-ts suite** + +Run: `cd server/typescript && bun test packages/codegen-ts` +Expected: all pass, no golden-snapshot regressions (no existing fixture has `@intValueMap`). + +- [ ] **Step 8: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/templates/entity-file.ts server/typescript/packages/codegen-ts/src/templates/zod-validators.ts server/typescript/packages/codegen-ts/src/templates/queries-file.ts server/typescript/packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts +git commit -m "feat(codegen-ts): symbol<->int codec for int-backed field.enum at the Drizzle boundary" +``` + +--- + +### Task 6: persistence-conformance — real round-trip + +**Files:** +- Modify: `fixtures/persistence-conformance/canonical/meta.fitness.json` +- Modify: `fixtures/persistence-conformance/queries/roundtrip-all-types.yaml` +- Verify: TS's real-engine round-trip test run + +**Interfaces:** +- Consumes: Tasks 1-5. + +- [ ] **Step 1: Add an int-backed enum field to the `AllTypes` entity** + +Edit `fixtures/persistence-conformance/canonical/meta.fitness.json` — add a sibling field next to the existing `enumVal` (found at line 243): + +```json +{ "field.enum": { "name": "intEnumVal", "@required": true, "@values": ["LOW", "MEDIUM", "HIGH"], "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } } }, +``` + +- [ ] **Step 2: Regenerate the canonical schema SQL** + +Run whatever command produces `fixtures/persistence-conformance/canonical/schema.postgres.sql` from `meta.fitness.json` (per CLAUDE.md, this file is TS-produced and committed) — likely `meta migrate` or a dedicated script; check `fixtures/persistence-conformance/README.md` for the exact regeneration command. +Expected: the regenerated SQL declares `int_enum_val integer NOT NULL CHECK (int_enum_val IN (1, 2, 3))` (naming per this project's `columnNamingStrategy`). + +- [ ] **Step 3: Add the round-trip scenario** + +Edit `fixtures/persistence-conformance/queries/roundtrip-all-types.yaml` — add `intEnumVal` to the existing `insert:`/`expect:` blocks (research found them at lines 74/98) as a sibling key: + +```yaml +insert: + enumVal: "MEDIUM" + intEnumVal: "MEDIUM" +expect: + enumVal: "MEDIUM" + intEnumVal: "MEDIUM" +``` + +(The wire value is the member STRING on both insert and expect, per this design's D6/D2 — the DB stores `2`, but every port's runtime translates transparently.) + +- [ ] **Step 4: Run TS's real-engine round-trip test** + +Run: `cd server/typescript && bun test packages/migrate-ts -t roundtrip` (or whichever package actually runs the `persistence-conformance` corpus against TS — check `fixtures/persistence-conformance/README.md` for the exact per-port runner) +Expected: PASS — `intEnumVal: "MEDIUM"` round-trips through insert → Postgres `2` → decoded back to `"MEDIUM"` on read. + +- [ ] **Step 5: Commit** + +```bash +git add fixtures/persistence-conformance/canonical/meta.fitness.json fixtures/persistence-conformance/canonical/schema.postgres.sql fixtures/persistence-conformance/queries/roundtrip-all-types.yaml +git commit -m "test(persistence-conformance): int-backed field.enum round-trips through TS's real runtime" +``` + +--- + +## After this plan lands + +TS is the reference port; the same shape (DDL/column-type dispatch → runtime codec → persistence-conformance round-trip) repeats in the C#, Java+Kotlin, and Python persistence plans, each adapted to that port's own ORM/codec idiom (EF Core `HasConversion`, OMDB `JdbcFieldCodec`/Exposed `customEnumeration`, Python `ObjectManager` coercion). Only TS needed a genuinely new "wire type differs from storage type" pattern — the other ports' `MetaField`-level runtime access already made a symbol↔int translation point available without inventing new template plumbing. From e9420a678f512585bf8da6696351d9b4558f018f Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 17:05:01 -0400 Subject: [PATCH 05/52] docs(plan): implementation plan for field.enum @intValueMap (C# persistence layer) --- ...t-backed-enum-values-csharp-persistence.md | 258 ++++++++++++++++++ 1 file changed, 258 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md new file mode 100644 index 000000000..e0413da7b --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md @@ -0,0 +1,258 @@ +# Int-Backed Enum Values — C# Persistence Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into C#'s EF Core codegen: emit a custom `HasConversion` lambda pair built from `@intValueMap`'s lookup table instead of the blanket `HasConversion()`, while the generated C# `enum` type itself is completely unchanged. C# never generates DDL (schema is TS-owned per ADR-0015) — this plan touches `DbContextGenerator.cs` only. + +**Architecture:** `DbContextGenerator.EmitFieldTypeConfig`'s existing `foreach (var f in fieldList.Where(f => f.SubType == FIELD_SUBTYPE_ENUM))` loop (the one that unconditionally emits `HasConversion()`) gets a conditional: when `f.Attr("intValueMap")` is present, emit a custom `.HasConversion(v => ..., v => ...)` pair instead — the exact inline-lambda shorthand this file already uses for `field.uri` (`.HasConversion(v => v!.ToString(), v => new System.Uri(v))`), just driven by a generated `Dictionary` literal rather than a fixed conversion. `EntityGenerator.CollectEnumDecls` (the C# `enum` type emitter) is **not touched at all** — confirmed no port's enum-type emitter reads `@intValueMap`, so the generated `enum Status { DRAFT, PUBLISHED, ARCHIVED }` declaration is byte-identical whether or not the field is int-backed. No `ValueConverter` class exists anywhere in this codebase today (confirmed via full-repo grep) — this plan introduces the inline-lambda form only, matching the file's own established idiom, not the standalone-class form. + +**Tech Stack:** C#, .NET, EF Core 8, xunit. + +## Global Constraints + +- The generated C# `enum` type is byte-identical between string- and int-backed fields — do not touch `EntityGenerator.CollectEnumDecls` or `EnumPropertyTypeName`. +- C# generates NO DDL. If a step here tempts you to write SQL/CHECK-constraint code, stop — that's TS's job (already done in the TS persistence plan). +- `@intValueMap`'s presence alone is the trigger — mirror the existing `HasConversion()` loop's unconditional style, just branching on attribute presence. +- Apply the identical conditional to BOTH loops that emit enum conversions today: the base/write-entity loop (`DbContextGenerator.cs` research lines 353-363) and the projection/read-model loop (lines 67-68). + +--- + +### Task 1: `FieldConstants.FIELD_ATTR_INT_VALUE_MAP` reader helper + +**Files:** +- Modify: `server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs` +- Test: `server/csharp/MetaObjects.Codegen.Tests/EnumIntValueMapConversionTests.cs` + +**Interfaces:** +- Consumes: `FieldConstants.FIELD_ATTR_INT_VALUE_MAP` (metamodel plan, already shipped — the constant lives in `server/csharp/MetaObjects/Core/Field/FieldConstants.cs`). +- Produces: a private static helper `TryGetIntValueMap(MetaField f, out IReadOnlyDictionary? map)` consumed by Tasks 2-3. + +- [ ] **Step 1: Write the failing test** + +```csharp +// server/csharp/MetaObjects.Codegen.Tests/EnumIntValueMapConversionTests.cs +using Xunit; +using MetaObjects.Loader; +using MetaObjects.Codegen.Generators; + +namespace MetaObjects.Codegen.Tests; + +public class EnumIntValueMapConversionTests +{ + private static MetaRoot LoadModel(string json) + { + var loader = new MetaDataLoader(); + var r = loader.Load(new[] { (IMetaDataSource)new InMemoryStringSource(json, "test.json") }); + Assert.Empty(r.Errors); + return r.Root; + } + + private const string Model = """ + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """; + + [Fact] + public void Int_backed_enum_emits_a_custom_HasConversion_lambda_pair_not_HasConversion_string() + { + var root = LoadModel(Model); + var output = new DbContextGenerator().Generate(new GenContext { Entities = new[] { root.FindObject("Order")! } }); + var contents = output.Single(f => f.Path.EndsWith("AppDbContext.g.cs")).Contents; + Assert.DoesNotContain("Property(x => x.Status).HasConversion()", contents); + Assert.Contains(".Property(x => x.Status).HasConversion(", contents); + } + + [Fact] + public void String_backed_enum_still_emits_HasConversion_string_unchanged() + { + var root = LoadModel(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """); + var output = new DbContextGenerator().Generate(new GenContext { Entities = new[] { root.FindObject("Order")! } }); + var contents = output.Single(f => f.Path.EndsWith("AppDbContext.g.cs")).Contents; + Assert.Contains("Property(x => x.Status).HasConversion()", contents); + } + + [Fact] + public void Generated_enum_type_declaration_is_identical_regardless_of_intValueMap() + { + var withMap = LoadModel(Model); + var withoutMap = LoadModel(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """); + var entityGen = new EntityGenerator(); + var out1 = entityGen.Generate(new GenContext { Entities = new[] { withMap.FindObject("Order")! } }); + var out2 = entityGen.Generate(new GenContext { Entities = new[] { withoutMap.FindObject("Order")! } }); + Assert.Equal( + out1.Single(f => f.Path.EndsWith("Order.g.cs")).Contents, + out2.Single(f => f.Path.EndsWith("Order.g.cs")).Contents); + } +} +``` + +> Match `GenContext`'s actual construction and `DbContextGenerator`/`EntityGenerator`'s actual `Generate` signature/output shape (`EmittedFile`-equivalent with `Path`/`Contents`) against an existing test like `EnumConformanceTests.cs` before finalizing — the sketch above follows that file's established pattern (confirmed via research) but adjust names/types to match exactly. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter EnumIntValueMapConversionTests` +Expected: FAIL — `Int_backed_enum_emits_a_custom_HasConversion_lambda_pair` fails (currently emits `HasConversion()` unconditionally); the other two pass already (nothing has changed yet, so they describe current/target-preserving behavior). + +- [ ] **Step 3: Add the reader helper to `DbContextGenerator.cs`** + +Edit `server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs` — add near the top of the class: + +```csharp + /// + /// Reads @intValueMap off a field.enum MetaField, if present. Object-shaped JSON + /// attrs parse to IReadOnlyDictionary<string, object?> with each number boxed + /// as long (DataConverter.ParseNumber: "integers are always long"). + /// + private static IReadOnlyDictionary? TryGetIntValueMap(MetaField f) + { + if (f.Attr(FieldConstants.FIELD_ATTR_INT_VALUE_MAP) is not IReadOnlyDictionary raw) + return null; + var result = new Dictionary(StringComparer.Ordinal); + foreach (var (key, value) in raw) + { + result[key] = value switch + { + long l => l, + int i => i, + _ => Convert.ToInt64(value), + }; + } + return result; + } +``` + +- [ ] **Step 4: Run tests** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter EnumIntValueMapConversionTests` +Expected: still FAIL on the same test (helper exists but isn't called yet) — this step only adds plumbing, verified by the build succeeding with no new test passing. + +- [ ] **Step 5: Commit** + +```bash +git add server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs server/csharp/MetaObjects.Codegen.Tests/EnumIntValueMapConversionTests.cs +git commit -m "feat(csharp): add TryGetIntValueMap reader helper to DbContextGenerator" +``` + +--- + +### Task 2: Custom `HasConversion` for the scalar + array enum loops + +**Files:** +- Modify: `server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs` + +**Interfaces:** +- Consumes: `TryGetIntValueMap` (Task 1). + +- [ ] **Step 1: Update the scalar-enum branch** + +Edit `EmitFieldTypeConfig`'s enum loop (research lines 353-363): + +```csharp + foreach (var f in fieldList.Where(f => f.SubType == FIELD_SUBTYPE_ENUM)) + { + var prop = CSharpNaming.Pascal(f.Name); + var intValueMap = TryGetIntValueMap(f); + // ADR-0039: resolving — array-ness inheritable via extends. Array-of-enum uses the + // EF Core 8 primitive collection with a per-element string conversion so members + // persist as symbols (["DRAFT"]), not int ordinals ([0]). + if (f.ResolvedIsArray()) + { + modelLines.Add($" modelBuilder.Entity<{className}>().PrimitiveCollection(x => x.{prop}).ElementType().HasConversion();"); + } + else if (intValueMap is not null) + { + var typeName = CSharpNaming.EnumTypeName(entity, f); + var toProvider = string.Join(", ", intValueMap.Select(kv => $"{{ {typeName}.{kv.Key}, {kv.Value} }}")); + var fromProvider = string.Join(", ", intValueMap.Select(kv => $"{{ {kv.Value}, {typeName}.{kv.Key} }}")); + modelLines.Add( + $" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasConversion(" + + $"v => new System.Collections.Generic.Dictionary<{typeName}, int> {{ {toProvider} }}[v], " + + $"v => new System.Collections.Generic.Dictionary {{ {fromProvider} }}[v]);"); + } + else + { + modelLines.Add($" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasConversion();"); + } + } +``` + +> Building a fresh `Dictionary` literal inline on every conversion call is wasteful at runtime (re-allocated per row) but matches this file's existing style of self-contained one-line lambda emission (see the `field.uri` branch, which similarly allocates a `new System.Uri(v)` per call). If this needs to be optimized later, emit the two dictionaries as `private static readonly` fields on the `DbContext` partial class instead of inline literals — flag this as a follow-up rather than doing it here, to keep this task's diff minimal and match the file's existing emission style. + +- [ ] **Step 2: Update the projection/read-model loop** + +Edit the sibling loop at research lines 67-68 with the identical conditional (extract the scalar-branch logic from Step 1 into a small shared private method `EmitEnumConversion(string className, string prop, MetaField f, MetaObject owner, List modelLines)` and call it from both loops, to avoid duplicating the dictionary-literal-building logic verbatim in two places). + +- [ ] **Step 3: Run tests — confirm all pass** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter EnumIntValueMapConversionTests` +Expected: PASS — all 3 tests green. + +- [ ] **Step 4: Run the full Codegen test suite** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests` +Expected: all pass, including the existing `EnumConformanceTests.cs` (string-backed enums unchanged) and the EF Core Roslyn compile-check suite (the generated `HasConversion` lambda must actually compile against the real EF Core API — this is the test that catches a syntax mistake in the emitted C#). + +- [ ] **Step 5: Commit** + +```bash +git add server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +git commit -m "feat(csharp): custom HasConversion lambda pair for int-backed field.enum" +``` + +--- + +### Task 3: EF Core Roslyn compile-check coverage + +**Files:** +- Modify: whatever fixture the "EF Core 8 + Roslyn-compiles the generated AppDbContext" test (per the original `field.enum` design doc's "Completed follow-ups") already uses. + +**Interfaces:** +- Consumes: Task 2. + +- [ ] **Step 1: Add an int-backed enum field to the shared EF-compile-check fixture model** + +Find the fixture model the existing "Roslyn-compiles the generated AppDbContext" test loads (per `docs/superpowers/specs/2026-05-23-enum-datatype-design.md`'s "Completed follow-ups" section — "a model exercising owned/jsonb/enum/enum-array/scalar-array/projection") and add a sibling int-backed enum field alongside the existing string-backed one. + +- [ ] **Step 2: Run the compile-check test** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter ` +Expected: PASS — zero Roslyn compile errors against the real EF Core 8 API surface, confirming the emitted `HasConversion` lambda syntax is genuinely valid (not just string-matched by the earlier unit tests). + +- [ ] **Step 3: Run the full C# test suite** + +Run: `cd server/csharp && dotnet test` +Expected: 100% pass, no regressions. + +- [ ] **Step 4: Commit** + +```bash +git add -A +git commit -m "test(csharp): int-backed field.enum in the EF Core Roslyn compile-check fixture" +``` + +--- + +## After this plan lands + +C# generates no DDL and no persistence-conformance round-trip runner of its own for the query corpus beyond what the shared `roundtrip-all-types` scenario already drives through its generated + deployed API (per the api-contract-conformance "generated fan-out" lane described in this repo's CLAUDE.md). Once the TS persistence plan's `intEnumVal` field lands in `meta.fitness.json` (shared canonical model), re-run C#'s persistence-conformance and api-contract-conformance suites to confirm the new field round-trips through the generated EF Core stack with zero additional code — that's the real end-to-end proof this plan's `HasConversion` lambda works, beyond the unit-level Roslyn compile check in Task 3. From f4812665adc69b288433d565004d38e19f622546 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 17:06:31 -0400 Subject: [PATCH 06/52] docs(plan): implementation plan for field.enum @intValueMap (Java + Kotlin persistence layer) --- ...ked-enum-values-java-kotlin-persistence.md | 446 ++++++++++++++++++ 1 file changed, 446 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md new file mode 100644 index 000000000..c402d4179 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md @@ -0,0 +1,446 @@ +# Int-Backed Enum Values — Java + Kotlin Persistence Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into Java's OMDB JDBC persistence and Kotlin's Exposed table generation. Java's generated Java `enum` type and Kotlin's generated Kotlin `enum class` are completely unchanged — only the runtime codec (Java/OMDB) and the generated table-column DSL call (Kotlin/codegen-kotlin) differ. + +**Architecture:** Java's `EnumCodec` (`server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java`) is extended in place — not replaced — to check `f.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)` and branch between string bind/read (today's only behavior) and int bind/read (new), mirroring `TimestampCodec`'s existing `isLocalTime(MetaField)` per-instance attribute check (the established pattern for a single codec class handling two behaviors based on an attribute, rather than registering a second codec class). Kotlin's `KotlinExposedTableGenerator.kt` gets a new branch in its two `EnumField` checks: when `@intValueMap` is present, emit Exposed's `customEnumeration(...)` (free-form `fromDb`/`toDb` lambdas — confirmed the only Exposed API that supports an arbitrary, non-ordinal int-per-member mapping; `enumerationByName`/`enumeration` do not) instead of `enumerationByName(...)`, referencing a generated lookup map emitted as a shared per-package support file (mirroring how this generator already emits `emitInstantTzSupportFile`/`emitJsonbMapperSupportFile` for other non-trivial column types). + +**Tech Stack:** Java, Maven, JUnit — Kotlin, KotlinPoet, JUnit (via `codegen-kotlin`) — Exposed (via `integration-tests-kotlin`, Testcontainers Postgres). + +## Global Constraints + +- The generated Java `enum`/Kotlin `enum class` type declaration is byte-identical between string- and int-backed fields. +- `EnumCodec` is extended in place (Option A from research: mirrors `TimestampCodec`'s `isLocalTime` pattern) — do NOT register a second `JdbcFieldCodec` class keyed differently; `JdbcCodecs.forField` dispatches purely by `Class`, and `EnumField` already maps to one codec. +- Kotlin: do not attempt to use `enumeration(...)` (natural-ordinal-backed) — it cannot express an arbitrary, sparse int map. `customEnumeration(...)` is the only fit; confirm this against the actual Exposed API version pinned in this repo before finalizing (check `integration-tests-kotlin`'s `pom.xml`/build file for the Exposed version) since `customEnumeration`'s exact signature has shifted across Exposed major versions. + +--- + +### Task 1: Java — `EnumCodec` int-backed branch + +**Files:** +- Modify: `server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java` +- Test: `server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/EnumIntValueMapCodecTest.java` + +**Interfaces:** +- Consumes: `EnumField.ATTR_INT_VALUE_MAP` (metamodel plan, already shipped). +- Produces: `EnumCodec` handles both string- and int-backed persistence transparently — consumed by every OMDB call site (`ObjectManagerDB`, `GenericSQLDriver`, `SimpleMappingHandlerDB`) with zero changes to those call sites, since they all go through `JdbcCodecs.forField(f)`. + +- [ ] **Step 1: Write the failing tests** + +```java +// server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/EnumIntValueMapCodecTest.java +package com.metaobjects.manager.db.codec; + +import com.metaobjects.field.EnumField; +import com.metaobjects.loader.MetaDataLoader; +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; + +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.Types; + +import static org.junit.jupiter.api.Assertions.*; +import static org.mockito.Mockito.*; + +class EnumIntValueMapCodecTest { + + private static EnumField intBackedField() { + var loader = new MetaDataLoader(); + var r = loader.load(java.util.List.of(new com.metaobjects.loader.source.InMemoryMetaDataSource(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """, "test.json"))); + return (EnumField) r.getRoot().getMetaObjectByName("Order").getMetaField("status"); + } + + @Test + void write_binds_the_mapped_int_not_the_string() throws Exception { + var field = intBackedField(); + var ps = mock(PreparedStatement.class); + JdbcCodecs.forField(field).write(ps, field, 1, "PUBLISHED"); + verify(ps).setInt(1, 5); + verify(ps, never()).setString(anyInt(), any()); + } + + @Test + void write_binds_null_as_sql_integer_null() throws Exception { + var field = intBackedField(); + var ps = mock(PreparedStatement.class); + JdbcCodecs.forField(field).write(ps, field, 1, null); + verify(ps).setNull(1, Types.INTEGER); + } + + @Test + void read_decodes_the_int_back_to_its_symbol() throws Exception { + var field = intBackedField(); + var rs = mock(ResultSet.class); + when(rs.getInt(1)).thenReturn(9); + when(rs.wasNull()).thenReturn(false); + var target = new Object[1]; + // EnumField.setString presumably takes (Object target, String value) — adjust to + // whatever mock/target shape matches this codec's real readInto contract; the + // simplest verification is to capture via a tiny test double MetaField target. + JdbcCodecs.forField(field).readInto(target, field, rs, 1); + // Adjust assertion to whatever readInto actually does with `target` for a + // non-MetaObject target — check EnumCodec's CURRENT string-backed readInto test + // (if one exists) for the established assertion pattern before finalizing. + } + + @Test + void string_backed_enum_field_still_uses_setString_unchanged() throws Exception { + var loader = new MetaDataLoader(); + var r = loader.load(java.util.List.of(new com.metaobjects.loader.source.InMemoryMetaDataSource(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """, "test.json"))); + var field = (EnumField) r.getRoot().getMetaObjectByName("Order").getMetaField("status"); + var ps = mock(PreparedStatement.class); + JdbcCodecs.forField(field).write(ps, field, 1, "PUBLISHED"); + verify(ps).setString(1, "PUBLISHED"); + verify(ps, never()).setInt(anyInt(), anyInt()); + } + + @Test + void int_backed_enum_column_reports_INTEGER_sql_type_for_ddl_purposes() { + var field = intBackedField(); + assertEquals(Types.INTEGER, JdbcCodecs.forField(field).sqlType()); + } +} +``` + +> This test file's exact mocking approach (`readInto`'s `Object target` parameter shape) needs to match `EnumCodec`'s real contract against a genuine `MetaField.setString(Object, String)`-style API — check whether an existing `EnumCodec`/`CurrencyCodec` test already exists in this module and mirror its target/assertion pattern exactly; the sketch above may need adjusting once that's confirmed. `InMemoryMetaDataSource` is carried over from the metamodel plan's placeholder naming — confirm the actual class name. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/java && mvn -pl omdb test -Dtest=EnumIntValueMapCodecTest` +Expected: FAIL — `write_binds_the_mapped_int_not_the_string` and `sqlType` tests fail (current `EnumCodec` always calls `setString`/returns `NO_SQL_TYPE`). + +- [ ] **Step 3: Extend `EnumCodec`** + +Edit `server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java` — replace the existing `EnumCodec` (research lines 415-423) with: + +```java +static final class EnumCodec implements JdbcFieldCodec { + + @Override public void readInto(Object o, MetaField f, ResultSet rs, int j) throws SQLException { + var intMap = intValueMap(f); + if (intMap != null) { + int i = rs.getInt(j); + if (rs.wasNull()) { f.setString(o, null); return; } + f.setString(o, reverseLookup(intMap, i)); + } else { + f.setString(o, rs.getString(j)); + } + } + + @Override public void write(PreparedStatement s, MetaField f, int j, Object v) throws SQLException { + var intMap = intValueMap(f); + if (intMap != null) { + if (v == null) { s.setNull(j, Types.INTEGER); return; } + Integer i = intMap.get(v.toString()); + if (i == null) { + throw new IllegalStateException( + "field.enum '" + f.getName() + "' value '" + v + "' has no entry in @intValueMap"); + } + s.setInt(j, i); + } else { + if (v == null) s.setNull(j, Types.VARCHAR); + else s.setString(j, v.toString()); + } + } + + @Override public int sqlType() { + // NOTE: sqlType() has no MetaField parameter in the current JdbcFieldCodec + // interface (it's a per-CLASS, not per-INSTANCE, hook used by + // SimpleMappingHandlerDB purely for DDL length/type defaults) — but per + // ADR-0015 Java emits NO DDL at all (schema is TS-owned), so this return + // value is dead for @intValueMap's actual purpose; leave it NO_SQL_TYPE + // (deferring to the DataType switch) and rely on TS's migrate-ts for the + // real column type. Do not attempt to make this per-instance-aware. + return NO_SQL_TYPE; + } + + /** Own-only content-rule-validated map, per the metamodel plan — safe to trust shape here. */ + @SuppressWarnings("unchecked") + private static java.util.Map intValueMap(MetaField f) { + if (!f.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null; + return (java.util.Map) f.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).getValue(); + } + + private static String reverseLookup(java.util.Map intMap, int value) { + for (var e : intMap.entrySet()) if (e.getValue() == value) return e.getKey(); + throw new IllegalStateException("int value " + value + " has no matching @intValueMap entry"); + } +} +``` + +> `sqlType()`'s doc comment above flags a real design tension surfaced during this step: `JdbcFieldCodec.sqlType()` has no `MetaField` parameter (it's per-class, called without field context per the interface shown in research), so it CANNOT return `Types.INTEGER` only for int-backed instances even if Java wanted to emit DDL. Since Java emits no DDL at all (confirmed, ADR-0015), this is fine — but if a future change makes Java DDL-aware, `JdbcFieldCodec`'s interface would need a `sqlType(MetaField)` overload. Flagging, not fixing, since it's out of this plan's scope. + +- [ ] **Step 4: Run tests — confirm all pass** (adjust the `read_decodes...` test per Step 1's note once the real `readInto` contract is confirmed) + +Run: `cd server/java && mvn -pl omdb test -Dtest=EnumIntValueMapCodecTest` +Expected: PASS. + +- [ ] **Step 5: Run the full OMDB test suite** + +Run: `cd server/java && mvn -pl omdb test` +Expected: all pass, no regressions to existing string-backed enum persistence tests. + +- [ ] **Step 6: Commit** + +```bash +git add server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/EnumIntValueMapCodecTest.java +git commit -m "feat(java): EnumCodec binds/reads as int when field.enum carries @intValueMap" +``` + +--- + +### Task 2: Kotlin — `customEnumeration` for int-backed columns + +**Files:** +- Modify: `server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt` +- Test: `server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinEnumIntValueMapConformanceTest.kt` + +**Interfaces:** +- Consumes: `EnumField.ATTR_INT_VALUE_MAP`. + +- [ ] **Step 1: Write the failing test** + +```kotlin +// server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinEnumIntValueMapConformanceTest.kt +package com.metaobjects.generator.kotlin + +import com.metaobjects.loader.MetaDataLoader +import com.metaobjects.loader.source.InMemoryMetaDataSource +import org.junit.jupiter.api.Test +import kotlin.test.assertTrue +import kotlin.test.assertFalse + +class KotlinEnumIntValueMapConformanceTest { + + private val model = """ + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """ + + @Test + fun `int-backed enum column emits customEnumeration, not enumerationByName`() { + val loader = MetaDataLoader() + val result = loader.load(listOf(InMemoryMetaDataSource(model, "test.json"))) + val entity = result.root.getMetaObjectByName("Order") + val output = KotlinExposedTableGenerator().generate(entity, result.root) + assertTrue(output.contains("customEnumeration")) + assertFalse(output.contains("enumerationByName")) + } + + @Test + fun `string-backed enum column still emits enumerationByName unchanged`() { + val loader = MetaDataLoader() + val result = loader.load(listOf(InMemoryMetaDataSource(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """, "test.json"))) + val entity = result.root.getMetaObjectByName("Order") + val output = KotlinExposedTableGenerator().generate(entity, result.root) + assertTrue(output.contains("enumerationByName")) + } +} +``` + +> `KotlinExposedTableGenerator`'s actual `generate(...)` entry point signature/return type is inferred from context (research showed its internals, not its public entry point) — check the existing `KotlinEnumConformanceTest.kt` (found during research) for the real call pattern and mirror it exactly. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/java && mvn -pl codegen-kotlin test -Dtest=KotlinEnumIntValueMapConformanceTest` +Expected: FAIL — `customEnumeration` never appears today. + +- [ ] **Step 3: Add a lookup-map support file emitter** + +Edit `KotlinExposedTableGenerator.kt` — add a helper mirroring the existing `emitInstantTzSupportFile`/`emitJsonbMapperSupportFile` pattern (research lines 287-338), emitting a small shared file per package holding the int↔symbol maps for every int-backed enum in that package: + +```kotlin +private fun emitEnumIntValueMapSupportFile(pkg: String, intBackedEnumFields: List>): String { + val entries = intBackedEnumFields.joinToString("\n\n") { (enumClassName, field) -> + val intValueMap = intValueMapOf(field) // helper added below + val toInt = intValueMap.entries.joinToString(", ") { (k, v) -> "$enumClassName.$k to $v" } + val fromInt = intValueMap.entries.joinToString(", ") { (k, v) -> "$v to $enumClassName.$k" } + """ + val ${enumClassName}_TO_INT: Map<$enumClassName, Int> = mapOf($toInt) + val ${enumClassName}_FROM_INT: Map = mapOf($fromInt) + """.trimIndent() + } + return """ + // + package $pkg + + $entries + """.trimIndent() +} + +@Suppress("UNCHECKED_CAST") +private fun intValueMapOf(field: EnumField): Map = + if (field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) + field.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).value as Map + else emptyMap() +``` + +- [ ] **Step 4: Update both `EnumField` branches to emit `customEnumeration`** + +Edit both occurrences shown in research (lines 580-589 and 616-620): + +```kotlin + val baseSpec = if (field is EnumField) { + val enumName = KotlinTypeMapper.enumTypeName(field, entity)?.simpleName + ?: error("enumTypeName returned null for EnumField '${field.name}' on ${entity.name}") + val colName = KotlinGenUtil.camelToSnake(field.name) + if (field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) { + // customEnumeration: free-form fromDb/toDb lambdas are the only Exposed + // API expressing an arbitrary, non-ordinal int-per-member map (confirmed — + // enumeration()/enumerationByName() cannot). Lambdas index into the + // per-package support-file maps (emitEnumIntValueMapSupportFile). + "customEnumeration(\"$colName\", fromDb = { ${enumName}_FROM_INT[it as Int]!! }, toDb = { ${enumName}_TO_INT[it]!! })" + } else { + "enumerationByName(\"$colName\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, $enumName::class)" + } + } else { + KotlinTypeMapper.exposedColumnSpec(field) + } +``` + +Apply the identical `if (field.hasMetaAttr(...))` branch to the TPH subtype-fields loop's copy (research lines 616-620). + +Wire `emitEnumIntValueMapSupportFile` into this generator's file-emission list (find wherever `emitInstantTzSupportFile`'s output gets added to the generator's returned file list, and add the new support file alongside it, once per package that has at least one int-backed enum field). + +> Confirm `customEnumeration`'s EXACT parameter names/order (`fromDb`/`toDb`, or possibly named differently, or requiring an explicit SQL column-type string as a THIRD parameter) against the Exposed version actually pinned in this repo's `pom.xml`/Gradle build for `integration-tests-kotlin` — Exposed's `customEnumeration` signature has varied across major versions; do not trust the sketch above verbatim. + +- [ ] **Step 5: Run tests — confirm all pass** + +Run: `cd server/java && mvn -pl codegen-kotlin test -Dtest=KotlinEnumIntValueMapConformanceTest` +Expected: PASS. + +- [ ] **Step 6: Run the full codegen-kotlin suite** + +Run: `cd server/java && mvn -pl codegen-kotlin test` +Expected: all pass, no regressions. + +- [ ] **Step 7: Commit** + +```bash +git add server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinEnumIntValueMapConformanceTest.kt +git commit -m "feat(kotlin): Exposed customEnumeration for int-backed field.enum columns" +``` + +--- + +### Task 3: Kotlin/Exposed real-engine round-trip (Testcontainers Postgres) + +**Files:** +- Modify: whatever fixture backs `EnumFilterControllerRunTest.kt` (research's strongest existing template for an end-to-end enum persistence test) or add a sibling test file. +- Test: `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/EnumIntValueMapRunTest.kt` + +**Interfaces:** +- Consumes: Task 2. + +- [ ] **Step 1: Write the real-engine test** + +```kotlin +// server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/EnumIntValueMapRunTest.kt +package com.metaobjects.integration.kotlin.api.generated + +import com.metaobjects.loader.MetaDataLoader +import com.metaobjects.loader.source.InMemoryMetaDataSource +import org.jetbrains.exposed.sql.Database +import org.jetbrains.exposed.sql.SchemaUtils +import org.jetbrains.exposed.sql.transactions.transaction +import org.junit.jupiter.api.Test +import org.testcontainers.containers.PostgreSQLContainer +import kotlin.test.assertEquals + +class EnumIntValueMapRunTest { + + private val model = """ + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}} + """ + + @Test + fun `int-backed enum column round-trips through real Postgres, storing the mapped int`() { + val pg = PostgreSQLContainer("postgres:16").apply { start() } + val db = Database.connect(pg.jdbcUrl, user = pg.username, password = pg.password) + val loader = MetaDataLoader() + val result = loader.load(listOf(InMemoryMetaDataSource(model, "test.json"))) + val entity = result.root.getMetaObjectByName("Order") + val generated = KotlinExposedTableGenerator().generate(entity, result.root) + // Compile + load the generated Table object dynamically, OR (simpler, matching + // this module's existing pattern per EnumFilterControllerRunTest.kt) drive this + // through the SAME generated-controller-over-HTTP harness that test already uses, + // rather than hand-rolling a raw Exposed table here — check that file's setup + // and mirror it exactly, since it already solves "compile generated Kotlin and + // run it against a real container" for the string-backed case. + transaction(db) { + // Insert a row with status = "PUBLISHED", read it back, assert: + // 1. The raw column value in the DB is the int 5 (query information_schema + // or SELECT status::int directly to prove it's really an INTEGER column). + // 2. The Kotlin data class field reads back as the STRING "PUBLISHED", not 5. + } + pg.stop() + } +} +``` + +> This test is intentionally left as a scaffold with the exact assertions commented rather than guessed — `EnumFilterControllerRunTest.kt` (confirmed in research to already compile+run a generated Spring controller against Exposed over Testcontainers Postgres for the string-backed case) is the concrete template to copy and adapt; read it in full before writing this test for real, since it already solves the hard parts (dynamic compilation of generated code, container lifecycle, HTTP round-trip) that this sketch only gestures at. + +- [ ] **Step 2: Run to verify current failure/gap** + +Run: `cd server/java && mvn -pl integration-tests-kotlin test -Dtest=EnumIntValueMapRunTest` +Expected: FAIL or does not compile, until Step 1 is completed for real against the `EnumFilterControllerRunTest.kt` template. + +- [ ] **Step 3: Complete the test for real, run, confirm pass** + +Run: `cd server/java && mvn -pl integration-tests-kotlin test -Dtest=EnumIntValueMapRunTest` +Expected: PASS. + +- [ ] **Step 4: Run the full integration-tests-kotlin suite** + +Run: `cd server/java && mvn -pl integration-tests-kotlin test` +Expected: all pass, no regressions. + +- [ ] **Step 5: Commit** + +```bash +git add server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/EnumIntValueMapRunTest.kt +git commit -m "test(kotlin): int-backed field.enum round-trips through real Postgres via Exposed" +``` + +--- + +## After this plan lands + +Java has no DDL and no ORM-config generator of its own analogous to C#'s `DbContextGenerator` — OMDB is pure data-access (per CLAUDE.md: "OMDB is pure data-access — CRUD/query/codec/transactions only"), so Task 1's codec change is Java's entire persistence-layer footprint. Once the shared `roundtrip-all-types` persistence-conformance scenario gets its `intEnumVal` field (added in the TS persistence plan), re-run Java's own persistence-conformance suite to confirm the codec round-trips correctly end-to-end, not just in the unit-level codec test from Task 1. From 446dc4fea4df3b37614c8fa76b42461740c7d937 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 23 Jul 2026 17:07:30 -0400 Subject: [PATCH 07/52] docs(plan): implementation plan for field.enum @intValueMap (Python persistence layer) --- ...t-backed-enum-values-python-persistence.md | 285 ++++++++++++++++++ 1 file changed, 285 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md new file mode 100644 index 000000000..0a4af2b1f --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md @@ -0,0 +1,285 @@ +# Int-Backed Enum Values — Python Persistence Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into Python's `ObjectManager` runtime: encode the member symbol to its int on write, decode it back on read. Python has no per-subtype `EnumField` class (confirmed — `field.sub_type == fc.FIELD_SUBTYPE_ENUM` is the only discriminator) and no codegen change is needed at all — Python's generated type for an inline enum field is `Literal["DRAFT", "PUBLISHED", ...]` regardless of backing mode, since codegen never reads `@intValueMap`. + +**Architecture:** `_coerce_write_value` (`server/python/src/metaobjects/runtime/object_manager.py:766-826`) is today a pure fallthrough for `field.sub_type == FIELD_SUBTYPE_ENUM` (falls to the final `return value` at line 826). This plan adds a new branch there for the write side. The **read side needs a wholly new function** — confirmed there is no decode/read-side coercion anywhere in this module today (`select()`/`find_by_id`/`find_many` return pg8000's native values verbatim, per ADR-0019). This plan adds `_decode_read_value(field, value)` and wires it into every row-mapping call site. + +**Tech Stack:** Python, pytest, pg8000 (via `ObjectManager`). + +## Global Constraints + +- Python's generated type for `field.enum` (`Literal["A","B",...]` inline, or the FR-019 shared `class X(str, Enum)`) is unchanged whether or not `@intValueMap` is present — do not touch `entity_model.py` or `fr019_shared_enum.py`. +- `@intValueMap`'s presence alone is the trigger, read via `field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP)` (resolving — own or inherited via `extends`, matching how `@localTime`/`@storage` are already read in this same function). +- Every codec constant/import goes through `field_constants` (`fc`) — this module already imports it aliased as `fc`; follow that convention. + +--- + +### Task 1: write-side encode in `_coerce_write_value` + +**Files:** +- Modify: `server/python/src/metaobjects/runtime/object_manager.py` +- Test: `server/python/tests/runtime/test_object_manager_enum_intvaluemap.py` + +**Interfaces:** +- Consumes: `fc.FIELD_ATTR_INT_VALUE_MAP` (metamodel plan, already shipped). +- Produces: `_coerce_write_value(field, value)` returns the mapped int for an int-backed enum field, unchanged behavior otherwise. + +- [ ] **Step 1: Write the failing tests** + +```python +# server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +import pytest +from metaobjects.runtime.object_manager import _coerce_write_value +from metaobjects.loader.loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource + + +def _load_order_field(extra: str): + json_str = f"""{{ "metadata.root": {{ "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + loader = MetaDataLoader() + result = loader.load([InMemoryStringSource(json_str, "test.json")]) + assert result.errors == [] + entity = result.root.find_object("Order") + return entity.field("status") + + +def test_int_backed_enum_write_value_encodes_symbol_to_int(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _coerce_write_value(field, "PUBLISHED") == 5 + + +def test_string_backed_enum_write_value_is_unchanged(): + field = _load_order_field("") + assert _coerce_write_value(field, "PUBLISHED") == "PUBLISHED" + + +def test_int_backed_enum_write_value_none_stays_none(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _coerce_write_value(field, None) is None +``` + +> Check `MetaDataLoader`/`InMemoryStringSource`/`find_object`/`.field(name)`'s actual API against `test_field_enum.py` (used as the template in the metamodel plan) before finalizing — this file reuses that same loading idiom. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: FAIL — `test_int_backed_enum_write_value_encodes_symbol_to_int` fails (currently returns `"PUBLISHED"` unchanged, since enum is a pure fallthrough). + +- [ ] **Step 3: Add the encode branch** + +Edit `server/python/src/metaobjects/runtime/object_manager.py` — insert before the final fallthrough comment/return (line 824-826): + +```python + # field.enum: int-backed persistence (docs/superpowers/specs/2026-07-23-int-backed- + # enum-values-design.md). @intValueMap maps the member symbol to its stored int; + # resolving read (own or inherited via extends), matching @localTime/@storage above. + # Absent → string-backed default, unchanged (falls to the generic return below). + if sub == fc.FIELD_SUBTYPE_ENUM: + int_value_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + return int_value_map[value] + + # Everything else (string / int / long / double / float / boolean / enum) + # is already the native type pg8000 binds directly. + return value +``` + +(This replaces the trailing comment+return at lines 824-826 — the new `if` block goes immediately above it, inside the same function, after the existing `field.object`/`field.map` jsonb branch at lines 820-823.) + +Add `FIELD_ATTR_INT_VALUE_MAP` to this file's existing `from metaobjects import field_constants as fc`-style import (it already imports `fc` wholesale per the module's existing style, based on `fc.FIELD_ATTR_DB_COLUMN_TYPE`/`fc.FIELD_SUBTYPE_DECIMAL` usage — no new import line needed if `fc` is a module-level import, since `FIELD_ATTR_INT_VALUE_MAP` was already added to `field_constants.py` by the metamodel plan). + +- [ ] **Step 4: Run tests — confirm all pass** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: PASS — all 3 tests green. + +- [ ] **Step 5: Run the full ObjectManager test suite** + +Run: `cd server/python && pytest tests/runtime/` +Expected: all pass, no regressions. + +- [ ] **Step 6: Commit** + +```bash +git add server/python/src/metaobjects/runtime/object_manager.py server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +git commit -m "feat(python): ObjectManager encodes int-backed field.enum symbol->int on write" +``` + +--- + +### Task 2: read-side decode (new function + wiring) + +**Files:** +- Modify: `server/python/src/metaobjects/runtime/object_manager.py` +- Test: extend `server/python/tests/runtime/test_object_manager_enum_intvaluemap.py` + +**Interfaces:** +- Produces: `_decode_read_value(field, value)` — a new function, the read-side mirror of `_coerce_write_value`, consumed by every row-mapping call site in `find_by_id`/`find_many`. + +- [ ] **Step 1: Write the failing tests** + +Append to `test_object_manager_enum_intvaluemap.py`: + +```python +from metaobjects.runtime.object_manager import _decode_read_value + + +def test_int_backed_enum_read_value_decodes_int_to_symbol(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _decode_read_value(field, 5) == "PUBLISHED" + + +def test_string_backed_enum_read_value_is_unchanged(): + field = _load_order_field("") + assert _decode_read_value(field, "PUBLISHED") == "PUBLISHED" + + +def test_int_backed_enum_read_value_none_stays_none(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _decode_read_value(field, None) is None + + +def test_int_backed_enum_read_value_unknown_int_raises(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + with pytest.raises(ValueError): + _decode_read_value(field, 42) +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: FAIL — `ImportError: cannot import name '_decode_read_value'`. + +- [ ] **Step 3: Add `_decode_read_value`** + +Edit `server/python/src/metaobjects/runtime/object_manager.py` — add near `_coerce_write_value`: + +```python +def _decode_read_value(field: MetaField, value: Any) -> Any: + """The read-side mirror of _coerce_write_value for int-backed field.enum. Every + other field type's read value passes through unmodified today (ADR-0019 — the + runtime returns native types, never wire-strings); this is the first field type + needing a genuine decode step, mirroring the (also-new) write-side encode above.""" + if value is None: + return None + if field.sub_type == fc.FIELD_SUBTYPE_ENUM: + int_value_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + for symbol, i in int_value_map.items(): + if i == value: + return symbol + raise ValueError( + f"field.enum '{field.name}' read value {value!r} has no matching @intValueMap entry" + ) + return value +``` + +- [ ] **Step 4: Wire it into the row-mapping call sites** + +Find `find_by_id`/`find_many`'s row-to-object mapping code (the functions the earlier research noted do "no per-field value transformation" today — read them in full first) and add a pass over each row's fields, applying `_decode_read_value(field, raw_value)` for every field before constructing the returned object. The exact insertion point depends on whether row mapping is a dict comprehension, a loop, or a dataclass constructor call — mirror whichever shape those functions actually use; do not restructure them beyond adding this one per-field decode step. + +- [ ] **Step 5: Run tests — confirm all pass** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: PASS — all 7 tests in this file green. + +- [ ] **Step 6: Run the full ObjectManager test suite** + +Run: `cd server/python && pytest tests/runtime/` +Expected: all pass, no regressions — every existing read of a string-backed enum (or any other field type) must be a no-op through `_decode_read_value` (confirmed by `test_string_backed_enum_read_value_is_unchanged` and the broader regression run). + +- [ ] **Step 7: Commit** + +```bash +git add server/python/src/metaobjects/runtime/object_manager.py server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +git commit -m "feat(python): ObjectManager decodes int-backed field.enum int->symbol on read" +``` + +--- + +### Task 3: real-engine round-trip + +**Files:** +- Test: `server/python/tests/integration/test_enum_intvaluemap_roundtrip.py` + +**Interfaces:** +- Consumes: Tasks 1-2. + +- [ ] **Step 1: Write the real-engine test** + +```python +# server/python/tests/integration/test_enum_intvaluemap_roundtrip.py +import pytest +from metaobjects.runtime.object_manager import ObjectManager +from metaobjects.loader.loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource + +# Match whatever pg8000 + Testcontainers (or the existing local Postgres) fixture +# this module's other integration tests already use — check +# server/python/tests/integration/ for the established connection-fixture pattern. + + +@pytest.mark.integration +def test_int_backed_enum_round_trips_through_real_postgres(pg_connection): # fixture name TBD — match existing convention + json_str = """{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}""" + loader = MetaDataLoader() + result = loader.load([InMemoryStringSource(json_str, "test.json")]) + entity = result.root.find_object("Order") + + om = ObjectManager(pg_connection) + # Table must already exist — apply the corresponding DDL (integer + int CHECK, + # per the TS persistence plan's schema) before this test runs, matching however + # this module's other integration tests provision their schema (likely the + # committed canonical/schema.postgres.sql, per this repo's ADR-0015 convention). + created = om.create(entity, {"status": "PUBLISHED"}) + fetched = om.find_by_id(entity, created["id"]) + + assert fetched["status"] == "PUBLISHED" # decoded back to the symbol, not 5 + + raw = pg_connection.run("SELECT status FROM orders WHERE id = :id", id=created["id"]) + assert raw[0][0] == 5 # the actual stored value is the mapped int +``` + +> `pg_connection`'s fixture name/shape, `ObjectManager`'s real constructor signature, and how this module provisions its test schema are all placeholders pending a read of `server/python/tests/integration/`'s existing setup — mirror an existing integration test there exactly (do not guess the connection/fixture wiring). + +- [ ] **Step 2: Run to verify current gap** + +Run: `cd server/python && pytest tests/integration/test_enum_intvaluemap_roundtrip.py -v -m integration` +Expected: FAIL or errors until Step 1 is completed for real against the actual fixture/connection pattern. + +- [ ] **Step 3: Complete the test for real, run, confirm pass** + +Run: `cd server/python && pytest tests/integration/test_enum_intvaluemap_roundtrip.py -v -m integration` +Expected: PASS. + +- [ ] **Step 4: Run the full Python test suite** + +Run: `cd server/python && pytest` +Expected: 100% pass. + +- [ ] **Step 5: Commit** + +```bash +git add server/python/tests/integration/test_enum_intvaluemap_roundtrip.py +git commit -m "test(python): int-backed field.enum round-trips through real Postgres via ObjectManager" +``` + +--- + +## After all five plans land + +Once every port's persistence plan is done (this one; TS; C#; Java+Kotlin), the last remaining item from the design spec's conformance plan is the shared `fixtures/persistence-conformance/roundtrip-all-types.yaml` scenario carrying `intEnumVal` (added in the TS persistence plan's Task 6) passing identically across **all five** ports' real-engine runners — that is the final, cross-language proof this feature is genuinely done, not just done-per-port. Run each port's persistence-conformance suite one more time after all five plans are merged to confirm. From 9844225f0cd5ec358920c328893cb1fd4fd3a865 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 09:50:36 -0400 Subject: [PATCH 08/52] =?UTF-8?q?feat(metadata):=20add=20attr.intMap=20?= =?UTF-8?q?=E2=80=94=20a=20generic=20object-with-integer-values=20attr=20s?= =?UTF-8?q?ubtype?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Foundation piece for field.enum's upcoming @intValueMap (Task 2). Generic shape-only validation (object, every value an integer); regenerates the embedded attr definition and the metamodel-docs fixture accordingly. --- fixtures/metamodel-docs/expected/INDEX.md | 1 + fixtures/metamodel-docs/expected/providers.md | 2 +- .../metamodel-docs/expected/types/attr.md | 14 ++++++ .../packages/metadata/src/core-types.ts | 1 + .../metadata/src/core/attr/attr-constants.ts | 6 +++ .../src/core/attr/attr-definition.embedded.ts | 6 +++ .../src/core/attr/meta-attr-int-map.ts | 39 +++++++++++++++ .../test/core/attr/meta-attr-int-map.test.ts | 47 +++++++++++++++++++ spec/metamodel/attr.json | 6 +++ 9 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts create mode 100644 server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts diff --git a/fixtures/metamodel-docs/expected/INDEX.md b/fixtures/metamodel-docs/expected/INDEX.md index 8726d4025..eb3b0fd22 100644 --- a/fixtures/metamodel-docs/expected/INDEX.md +++ b/fixtures/metamodel-docs/expected/INDEX.md @@ -22,6 +22,7 @@ children, and cardinality of a subtype. Universal documentation attributes | `attr.expression` | A structured expression tree over a base entity's own fields (closed node grammar: field/value refs, comparisons sharing the filter op vocabulary, isNull/isNotNull, and/or/not, coalesce). Backs origin.computed; a filter object embeds canonically. Additive node kinds (arithmetic/case/via-joined refs) are #159. | [types/attr.md#attrexpression](types/attr.md#attrexpression) | | `attr.filter` | A filter-expression-valued metadata attribute. Object-shaped value that desugars a preset filter to the canonical { field: { op: value } } form (scalar→eq, array→in, null→isNull; or/and recurse). | [types/attr.md#attrfilter](types/attr.md#attrfilter) | | `attr.int` | A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number. | [types/attr.md#attrint](types/attr.md#attrint) | +| `attr.intMap` | An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation. | [types/attr.md#attrintmap](types/attr.md#attrintmap) | | `attr.long` | A 64-bit-integer-valued metadata attribute. Coerces to and validates as a number. | [types/attr.md#attrlong](types/attr.md#attrlong) | | `attr.properties` | A key/value map attribute (a bag of arbitrary author-supplied properties). Object-shaped value; the registered escape hatch for author-supplied properties (exempt from the strict-attr check, ADR-0023). | [types/attr.md#attrproperties](types/attr.md#attrproperties) | | `attr.string` | A string-valued metadata attribute. Coerces to and validates as text; the default value-type for inline @-syntax attrs (array-of-string is the same subtype with isArray). | [types/attr.md#attrstring](types/attr.md#attrstring) | diff --git a/fixtures/metamodel-docs/expected/providers.md b/fixtures/metamodel-docs/expected/providers.md index 76fa7e2b2..935f0ffa0 100644 --- a/fixtures/metamodel-docs/expected/providers.md +++ b/fixtures/metamodel-docs/expected/providers.md @@ -13,7 +13,7 @@ provider owns. This is the ownership lens over the same vocabulary Core metaobjects metamodel types and subtypes. -**Owns (registers):** `attr.base`, `attr.boolean`, `attr.class`, `attr.double`, `attr.expression`, `attr.filter`, `attr.int`, `attr.long`, `attr.properties`, `attr.string`, `field.base`, `field.boolean`, `field.currency`, `field.date`, `field.decimal`, `field.double`, `field.enum`, `field.float`, `field.inet`, `field.int`, `field.long`, `field.map`, `field.object`, `field.string`, `field.time`, `field.timestamp`, `field.uri`, `field.uuid`, `identity.primary`, `identity.reference`, `identity.secondary`, `layout.base`, `layout.dataGrid`, `object.base`, `object.entity`, `object.projection`, `object.value`, `origin.aggregate`, `origin.base`, `origin.collection`, `origin.computed`, `origin.first`, `origin.passthrough`, `relationship.aggregation`, `relationship.association`, `relationship.base`, `relationship.composition`, `source.base`, `source.rdb`, `template.base`, `template.output`, `template.prompt`, `template.toolcall`, `validator.array`, `validator.atLeastOne`, `validator.base`, `validator.comparison`, `validator.length`, `validator.numeric`, `validator.presentIff`, `validator.regex`, `validator.required`, `validator.requiredWhen`, `view.base`, `view.currency` +**Owns (registers):** `attr.base`, `attr.boolean`, `attr.class`, `attr.double`, `attr.expression`, `attr.filter`, `attr.int`, `attr.intMap`, `attr.long`, `attr.properties`, `attr.string`, `field.base`, `field.boolean`, `field.currency`, `field.date`, `field.decimal`, `field.double`, `field.enum`, `field.float`, `field.inet`, `field.int`, `field.long`, `field.map`, `field.object`, `field.string`, `field.time`, `field.timestamp`, `field.uri`, `field.uuid`, `identity.primary`, `identity.reference`, `identity.secondary`, `layout.base`, `layout.dataGrid`, `object.base`, `object.entity`, `object.projection`, `object.value`, `origin.aggregate`, `origin.base`, `origin.collection`, `origin.computed`, `origin.first`, `origin.passthrough`, `relationship.aggregation`, `relationship.association`, `relationship.base`, `relationship.composition`, `source.base`, `source.rdb`, `template.base`, `template.output`, `template.prompt`, `template.toolcall`, `validator.array`, `validator.atLeastOne`, `validator.base`, `validator.comparison`, `validator.length`, `validator.numeric`, `validator.presentIff`, `validator.regex`, `validator.required`, `validator.requiredWhen`, `view.base`, `view.currency` **Contributes attributes:** diff --git a/fixtures/metamodel-docs/expected/types/attr.md b/fixtures/metamodel-docs/expected/types/attr.md index b6ca05de8..8276f40e5 100644 --- a/fixtures/metamodel-docs/expected/types/attr.md +++ b/fixtures/metamodel-docs/expected/types/attr.md @@ -108,6 +108,20 @@ _No subtype-specific attributes._ _No structural children._ +### attr.intMap + +An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation. + +**Owning provider:** metaobjects-core-types + +**Attributes** + +_No subtype-specific attributes._ + +**Allowed children** + +_No structural children._ + ### attr.long A 64-bit-integer-valued metadata attribute. Coerces to and validates as a number. diff --git a/server/typescript/packages/metadata/src/core-types.ts b/server/typescript/packages/metadata/src/core-types.ts index ced9aac7b..b12cb787b 100644 --- a/server/typescript/packages/metadata/src/core-types.ts +++ b/server/typescript/packages/metadata/src/core-types.ts @@ -20,6 +20,7 @@ import "./core/attr/meta-attr-stringarray.js"; import "./core/attr/meta-attr-filter.js"; import "./core/attr/meta-attr-properties.js"; import "./core/attr/meta-attr-expression.js"; +import "./core/attr/meta-attr-int-map.js"; import { MetaValidator, MetaRequiredValidator, diff --git a/server/typescript/packages/metadata/src/core/attr/attr-constants.ts b/server/typescript/packages/metadata/src/core/attr/attr-constants.ts index 8b1cbcfc1..4c7faff72 100644 --- a/server/typescript/packages/metadata/src/core/attr/attr-constants.ts +++ b/server/typescript/packages/metadata/src/core/attr/attr-constants.ts @@ -17,6 +17,11 @@ export const ATTR_SUBTYPE_FILTER = "filter"; // #195 — a structured expression tree over a base entity's own fields (backs // origin.computed). Object-shaped; a closed node grammar (see meta-attr-expression.ts). export const ATTR_SUBTYPE_EXPRESSION = "expression"; +// An object-shaped attr whose values are all integers (e.g. field.enum's +// @intValueMap: {memberSymbol: int}). Generic shape check only — semantic +// rules specific to a consumer (key-set membership, uniqueness) are that +// consumer's own content-rule validation, not this attr's. +export const ATTR_SUBTYPE_INT_MAP = "intMap"; /** * The retired `stringarray` array attr subtype. It is NO LONGER a registered @@ -41,6 +46,7 @@ export const ATTR_SUBTYPES = [ ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, ] as const; export type AttrSubType = | (typeof ATTR_SUBTYPES)[number] diff --git a/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts b/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts index 0fdf7dcdc..fa0afaade 100644 --- a/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts @@ -27,6 +27,12 @@ export const ATTR_DEFINITION: ProviderDefinition = { "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts b/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts new file mode 100644 index 000000000..2e24a862b --- /dev/null +++ b/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts @@ -0,0 +1,39 @@ +// IntMapAttr — attr subtype `intMap`. Object-shaped value whose members must +// all be integers (e.g. field.enum's @intValueMap). No desugar; validates +// shape (object, not array) and every value's type (integer). A consumer's +// own semantic rules (key-set membership, uniqueness) are validated by that +// consumer, not here — mirrors how StringArrayAttr validates shape while +// field.enum's own content-rule pass validates its @values semantics. + +import { MetaAttr, type ValueError, runtimeTypeName } from "./meta-attr.js"; +import { type AttrValue } from "../../shared/meta-data.js"; +import { DATA_TYPE_OBJECT, type DataType } from "../../data-type.js"; +import { registerAttrClass } from "../../attr-class-map.js"; +import { ATTR_SUBTYPE_INT_MAP } from "./attr-constants.js"; + +export class IntMapAttr extends MetaAttr { + override get dataType(): DataType { + return DATA_TYPE_OBJECT; + } + + override coerce(raw: unknown): AttrValue { + return raw as AttrValue; + } + + override validateValue(value: AttrValue): ValueError[] { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return [{ message: `attribute '@${this.name}' must be of type 'intMap' but got ${runtimeTypeName(value)}` }]; + } + const errors: ValueError[] = []; + for (const [key, member] of Object.entries(value as Record)) { + if (typeof member !== "number" || !Number.isInteger(member)) { + errors.push({ + message: `attribute '@${this.name}' member '${key}' has value '${String(member)}' which is not an integer`, + }); + } + } + return errors; + } +} + +registerAttrClass(ATTR_SUBTYPE_INT_MAP, IntMapAttr); diff --git a/server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts b/server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts new file mode 100644 index 000000000..b037a9ad8 --- /dev/null +++ b/server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts @@ -0,0 +1,47 @@ +// attr.intMap — a generic object-shaped attr whose values must all be +// integers (e.g. field.enum's future @intValueMap). Shape-only validation; +// a consumer's own semantic rules (key-set membership, uniqueness) are +// layered by that consumer, not here. + +import { describe, test, expect } from "bun:test"; +import { TypeId } from "../../../src/registry.js"; +import { IntMapAttr } from "../../../src/core/attr/meta-attr-int-map.js"; +import { TYPE_ATTR, ATTR_SUBTYPE_INT_MAP } from "../../../src/index.js"; +import { type AttrValue } from "../../../src/shared/meta-data.js"; + +function intMapAttr(name = "intValueMap"): IntMapAttr { + return new IntMapAttr(new TypeId(TYPE_ATTR, ATTR_SUBTYPE_INT_MAP), name); +} + +describe("IntMapAttr", () => { + test("accepts a plain object with integer values", () => { + const attr = intMapAttr(); + expect(attr.validateValue({ DRAFT: 0, PUBLISHED: 5 })).toEqual([]); + }); + + test("rejects a non-object value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue("not-an-object" as unknown as AttrValue); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("must be of type 'intMap'"); + }); + + test("rejects an array value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue([0, 1] as unknown as AttrValue); + expect(errors.length).toBe(1); + }); + + test("rejects a non-integer value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue({ DRAFT: "0" }); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("DRAFT"); + }); + + test("rejects a float value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue({ DRAFT: 0.5 }); + expect(errors.length).toBe(1); + }); +}); diff --git a/spec/metamodel/attr.json b/spec/metamodel/attr.json index 45f4cea6b..a57e1d6a9 100644 --- a/spec/metamodel/attr.json +++ b/spec/metamodel/attr.json @@ -19,6 +19,12 @@ "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", From 3f83a47f9de2e34209d020a72f8c5adc3cd05630 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 10:13:38 -0400 Subject: [PATCH 09/52] =?UTF-8?q?feat(metadata):=20field.enum=20@intValueM?= =?UTF-8?q?ap=20=E2=80=94=20explicit=20per-member=20int=20values=20for=20D?= =?UTF-8?q?B=20persistence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- fixtures/metamodel-docs/expected/providers.md | 2 +- .../metamodel-docs/expected/types/field.md | 1 + .../metadata/src/attr-schema-validate.ts | 48 +++++++++++++ .../src/core/field/field-constants.ts | 10 +++ .../core/field/field-definition.embedded.ts | 8 +++ ...r-schema-validate-enum-intvaluemap.test.ts | 68 +++++++++++++++++++ .../field-definition-completeness.test.ts | 3 +- spec/metamodel/field.json | 3 +- 8 files changed, 140 insertions(+), 3 deletions(-) create mode 100644 server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts diff --git a/fixtures/metamodel-docs/expected/providers.md b/fixtures/metamodel-docs/expected/providers.md index 935f0ffa0..e4b4abbe1 100644 --- a/fixtures/metamodel-docs/expected/providers.md +++ b/fixtures/metamodel-docs/expected/providers.md @@ -20,7 +20,7 @@ Core metaobjects metamodel types and subtypes. - `field.base`: `@default`, `@readOnly`, `@required`, `@unique` - `field.currency`: `@currency` - `field.decimal`: `@precision`, `@scale` -- `field.enum`: `@provided`, `@values` +- `field.enum`: `@intValueMap`, `@provided`, `@values` - `field.inet`: `@lenient` - `field.map`: `@objectRef`, `@valueType` - `field.object`: `@objectRef` diff --git a/fixtures/metamodel-docs/expected/types/field.md b/fixtures/metamodel-docs/expected/types/field.md index e52004dcd..2476f5a7f 100644 --- a/fixtures/metamodel-docs/expected/types/field.md +++ b/fixtures/metamodel-docs/expected/types/field.md @@ -239,6 +239,7 @@ String-backed enumeration constrained to a closed set of member symbols (@values | `@filterable` | boolean | no | | | metaobjects-ui | When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer). | | `@formExclude` | boolean | no | | | metaobjects-ui | When true, the field is omitted from generated forms. Inert on fields for which no form is generated (e.g. projection/derived fields). | | `@instruction` | string | no | | | metaobjects-prompt | FR-010: a short instruction for this field, shown in the generated output-format prompt fragment. | +| `@intValueMap` | intMap | no | | | metaobjects-core-types | Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language. | | `@normalize` | string | no | `strip` | `none`, `collapse`, `strip` | metaobjects-prompt | ASCII normalization mode for tolerant enum extract (none\|collapse\|strip, default strip). On field.enum it is per-field; on object.value it is the default for the object's enum fields. | | `@provided` | boolean | no | | | metaobjects-core-types | FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration. | | `@readOnly` | boolean | no | | | metaobjects-core-types | FR-013: when true, the field is read-only — codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas mark it read-only on input variants. The value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. | diff --git a/server/typescript/packages/metadata/src/attr-schema-validate.ts b/server/typescript/packages/metadata/src/attr-schema-validate.ts index 479b5e8df..3346d750d 100644 --- a/server/typescript/packages/metadata/src/attr-schema-validate.ts +++ b/server/typescript/packages/metadata/src/attr-schema-validate.ts @@ -52,6 +52,7 @@ import { FIELD_ATTR_VALUES, FIELD_ATTR_COERCE_DEFAULT, FIELD_ATTR_DEFAULT, + FIELD_ATTR_INT_VALUE_MAP, ENUM_MEMBER_PATTERN, } from "./core/field/field-constants.js"; import { @@ -399,6 +400,53 @@ function validateNode( } } } + + // --- Check 5b: field.enum @intValueMap content rules --- + // + // Optional. Own-only (mirrors Checks 4/5's own-attrs-only policy) — an + // inherited @intValueMap is validated on its declaring node. The generic + // "is this an object of integers" shape check already ran via IntMapAttr + // (attr subtype `intMap`); this validates the field.enum-SPECIFIC + // semantics: key-set-equals-@values, and no two members share a value. + const rawIntValueMap = node.ownAttrs().get(FIELD_ATTR_INT_VALUE_MAP); + if (rawIntValueMap !== undefined && typeof rawIntValueMap === "object" && rawIntValueMap !== null) { + const map = rawIntValueMap as Record; + const effectiveValues = node.attrs().get(FIELD_ATTR_VALUES); + const declaredMembers: string[] = Array.isArray(effectiveValues) ? effectiveValues : []; + const memberSet = new Set(declaredMembers); + const mapKeys = Object.keys(map); + const keySet = new Set(mapKeys); + + const missing = declaredMembers.filter((m) => !keySet.has(m)); + const extra = mapKeys.filter((k) => !memberSet.has(k)); + if (missing.length > 0 || extra.length > 0) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@${FIELD_ATTR_VALUES}' members` + + (missing.length > 0 ? ` (missing: ${missing.join(", ")})` : "") + + (extra.length > 0 ? ` (unknown: ${extra.join(", ")})` : "") + ".", + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } + + const seenValues = new Map(); + for (const [member, value] of Object.entries(map)) { + if (typeof value !== "number" || !Number.isInteger(value)) continue; // IntMapAttr already reported this + const owner = seenValues.get(value); + if (owner !== undefined) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' members '${owner}' and '${member}' ` + + `share the same value ${value}; every member must have a unique int.`, + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } else { + seenValues.set(value, member); + } + } + } } // --- Check 6 (R6 Plan 2b): @dbColumnType (logical subtype × value) pairing --- diff --git a/server/typescript/packages/metadata/src/core/field/field-constants.ts b/server/typescript/packages/metadata/src/core/field/field-constants.ts index 7b9197266..52de625fa 100644 --- a/server/typescript/packages/metadata/src/core/field/field-constants.ts +++ b/server/typescript/packages/metadata/src/core/field/field-constants.ts @@ -166,6 +166,16 @@ export const FIELD_ATTR_CURRENCY_DEFAULT = "USD"; /** Member symbols of an enum-subtype field. Required, string array. */ export const FIELD_ATTR_VALUES = "values"; +/** + * Optional per-member explicit integer value ({memberSymbol: int}) switching + * this enum field's DB persistence from string+CHECK to integer+CHECK. Keys + * must exactly match @values; values must be unique integers. The generated + * native type and wire format are UNCHANGED in every language — this is a + * persistence-layer-only concern (docs/superpowers/specs/2026-07-23-int-backed- + * enum-values-design.md). + */ +export const FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; + /** * Pattern every enum member must satisfy: a legal identifier in all target * languages (TS union member, Java/C#/Python enum member). Ensures symbol == diff --git a/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts b/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts index 86a71cf16..e627cdad0 100644 --- a/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts @@ -279,6 +279,14 @@ export const FIELD_DEFINITION: ProviderDefinition = { "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." + }, + { + "type": "attr", + "subType": "intMap", + "name": "intValueMap", + "min": 0, + "max": 1, + "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, diff --git a/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts new file mode 100644 index 000000000..3c3b74f49 --- /dev/null +++ b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts @@ -0,0 +1,68 @@ +// field.enum @intValueMap — content-rule tests (Task 2 of the int-backed-enum- +// values plan). Key set must exactly match @values; values must be unique +// integers. The generic "is this an object of integers" shape check is +// IntMapAttr's job (attr subtype `intMap`, see meta-attr-int-map.test.ts); +// these tests cover the field.enum-SPECIFIC semantics layered in +// attr-schema-validate.ts Check 5b. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +async function load(json: string) { + // strict:true so an unregistered @intValueMap surfaces as ERR_UNKNOWN_ATTR + // (ADR-0023) before Step 3/4 register it — matching how the library's own + // loader and the conformance runner load. + const loader = new MetaDataLoader({ strict: true }); + return loader.load([new InMemoryStringSource(json, { id: "test.json" })]); +} + +const base = (extra: string) => `{ + "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] ${extra} } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]} +}`; + +describe("field.enum @intValueMap content rules", () => { + test("accepts a valid map — key set matches @values, unique ints", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors).toEqual([]); + }); + + test("field.enum with no @intValueMap is still valid (string-backed default)", async () => { + const result = await load(base("")); + expect(result.errors).toEqual([]); + }); + + test("rejects a missing member key", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("ARCHIVED"); + }); + + test("rejects an extra key not in @values", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("RETRACTED"); + }); + + test("rejects a non-integer value", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + }); + + test("rejects two members sharing the same int", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("DRAFT"); + expect(result.errors[0]?.message).toContain("PUBLISHED"); + }); +}); diff --git a/server/typescript/packages/metadata/test/field-definition-completeness.test.ts b/server/typescript/packages/metadata/test/field-definition-completeness.test.ts index 5b8726aae..e64d08c24 100644 --- a/server/typescript/packages/metadata/test/field-definition-completeness.test.ts +++ b/server/typescript/packages/metadata/test/field-definition-completeness.test.ts @@ -12,7 +12,7 @@ // decimal → @precision, @scale // object → @objectRef // currency → @currency -// enum → @values, @provided +// enum → @values, @provided, @intValueMap // int/long/double/float/boolean/date/time/timestamp/uuid → none // - the "any attr" wildcard is gone; a misplaced attr is now ERR_UNKNOWN_ATTR. // @@ -103,6 +103,7 @@ const MAP_EXTRA: Record = { const ENUM_CORE_EXTRA: Record = { values: { valueType: "string", required: true }, provided: { valueType: "boolean", required: false }, + intValueMap: { valueType: "intMap", required: false }, }; /** Scoped concern attrs (db / prompt). */ diff --git a/spec/metamodel/field.json b/spec/metamodel/field.json index 01be74308..a80d56cf5 100644 --- a/spec/metamodel/field.json +++ b/spec/metamodel/field.json @@ -150,7 +150,8 @@ "rules": "Required @values is a non-empty, duplicate-free set; each member must match ^[A-Za-z_][A-Za-z0-9_]*$ so symbol == stored string in every target language. Optional FR-010/FR-011 overlays add tolerant-extract aliasing (@enumAlias), per-member docs (@enumDoc), an uncoercible-value fallback (@coerceDefault, must be one of @values), and ASCII normalization mode (@normalize).", "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, - { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." } + { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, { From 3760e1274aaf23e96be3e9c1cc38911cbeee726c Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 10:24:02 -0400 Subject: [PATCH 10/52] test(conformance): add cross-port fixtures for field.enum @intValueMap --- .../enum-int-backed-array/expected.json | 43 +++++++++++++++++++ .../input/meta.enums.json | 24 +++++++++++ .../conformance/enum-int-backed/expected.json | 42 ++++++++++++++++++ .../enum-int-backed/input/meta.enums.json | 23 ++++++++++ .../expected-errors.json | 15 +++++++ .../input/meta.enums.json | 23 ++++++++++ .../expected-errors.json | 15 +++++++ .../input/meta.enums.json | 23 ++++++++++ .../expected-errors.json | 15 +++++++ .../input/meta.enums.json | 23 ++++++++++ 10 files changed, 246 insertions(+) create mode 100644 fixtures/conformance/enum-int-backed-array/expected.json create mode 100644 fixtures/conformance/enum-int-backed-array/input/meta.enums.json create mode 100644 fixtures/conformance/enum-int-backed/expected.json create mode 100644 fixtures/conformance/enum-int-backed/input/meta.enums.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json diff --git a/fixtures/conformance/enum-int-backed-array/expected.json b/fixtures/conformance/enum-int-backed-array/expected.json new file mode 100644 index 000000000..ce86c7edf --- /dev/null +++ b/fixtures/conformance/enum-int-backed-array/expected.json @@ -0,0 +1,43 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Ticket", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "labels", + "isArray": true, + "@intValueMap": { + "HIGH": 3, + "LOW": 1, + "MEDIUM": 2 + }, + "@values": [ + "LOW", + "MEDIUM", + "HIGH" + ] + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed-array/input/meta.enums.json b/fixtures/conformance/enum-int-backed-array/input/meta.enums.json new file mode 100644 index 000000000..590587edb --- /dev/null +++ b/fixtures/conformance/enum-int-backed-array/input/meta.enums.json @@ -0,0 +1,24 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Ticket", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "labels", + "isArray": true, + "@values": ["LOW", "MEDIUM", "HIGH"], + "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed/expected.json b/fixtures/conformance/enum-int-backed/expected.json new file mode 100644 index 000000000..97b09cd0c --- /dev/null +++ b/fixtures/conformance/enum-int-backed/expected.json @@ -0,0 +1,42 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "status", + "@intValueMap": { + "ARCHIVED": 9, + "DRAFT": 0, + "PUBLISHED": 5 + }, + "@values": [ + "DRAFT", + "PUBLISHED", + "ARCHIVED" + ] + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed/input/meta.enums.json b/fixtures/conformance/enum-int-backed/input/meta.enums.json new file mode 100644 index 000000000..843c7e898 --- /dev/null +++ b/fixtures/conformance/enum-int-backed/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json new file mode 100644 index 000000000..a5f2e7920 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json new file mode 100644 index 000000000..d09dff04a --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 0 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json new file mode 100644 index 000000000..a5f2e7920 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json new file mode 100644 index 000000000..45b545a17 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json new file mode 100644 index 000000000..a5f2e7920 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json new file mode 100644 index 000000000..ae2b940f4 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { "DRAFT": "zero", "PUBLISHED": 5 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} From 11bb85ed9ab9fa1dbe5c8c2be0dd73a62ec905dc Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 10:31:08 -0400 Subject: [PATCH 11/52] test(registry-conformance): register field.enum @intValueMap (attr.intMap) --- .../registry-conformance/expected-registry.json | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index 6bd8d96a0..4066da028 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -50,6 +50,13 @@ "attrs": [], "children": [] }, + { + "type": "attr", + "subType": "intMap", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation.", + "attrs": [], + "children": [] + }, { "type": "attr", "subType": "long", @@ -1014,6 +1021,13 @@ "required": false, "description": "FR-010: a short instruction for this field, shown in the generated output-format prompt fragment." }, + { + "name": "intValueMap", + "valueType": "intMap", + "isArray": false, + "required": false, + "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." + }, { "name": "normalize", "valueType": "string", From b1184251bf79530eee15e1aa3266987c52755e35 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 11:02:30 -0400 Subject: [PATCH 12/52] =?UTF-8?q?feat(csharp):=20field.enum=20@intValueMap?= =?UTF-8?q?=20=E2=80=94=20explicit=20per-member=20int=20values=20for=20DB?= =?UTF-8?q?=20persistence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../EnumIntValueMapTests.cs | 76 +++++++++++++++++++ .../MetaObjects/Core/Attr/AttrConstants.cs | 9 ++- .../MetaObjects/Core/Field/FieldConstants.cs | 10 +++ .../MetaObjects/Core/Field/FieldSchema.cs | 7 ++ server/csharp/MetaObjects/CoreTypes.cs | 5 +- .../MetaObjects/Loader/ValidationPasses.cs | 70 ++++++++++++++++- .../MetaObjects/SpecMetamodel/attr.json | 6 ++ .../MetaObjects/SpecMetamodel/field.json | 3 +- 8 files changed, 179 insertions(+), 7 deletions(-) create mode 100644 server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs diff --git a/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs b/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs new file mode 100644 index 000000000..9413623a8 --- /dev/null +++ b/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs @@ -0,0 +1,76 @@ +// Int-backed-enum-values plan, Task 6 — C# port of field.enum's @intValueMap. +// +// Mirrors the TS reference (Task 1/2/3): a new attr.intMap subtype (object-shaped, +// all-integer values) plus field.enum-specific content-rule validation (key-set +// equals @values, no duplicate values), reusing ERR_BAD_ATTR_VALUE. + +using MetaObjects.Loader; +using Xunit; + +namespace MetaObjects.Conformance.Tests; + +public class EnumIntValueMapTests +{ + private static string Model(string extra) => """ + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] EXTRA_PLACEHOLDER } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """.Replace("EXTRA_PLACEHOLDER", extra); + + private static LoadResult TryLoad(string json) + { + var registry = FullCoreRegistry.Compose(); + var loader = new MetaDataLoader(registry); + return loader.Load(new IMetaDataSource[] + { + new InMemoryStringSource(json, format: MetaDataFormat.Json, id: "test.json"), + }); + } + + [Fact] + public void Valid_intValueMap_with_matching_keys_and_unique_ints_loads_clean() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.Empty(res.Errors); + } + + [Fact] + public void No_intValueMap_still_loads_clean_string_backed_default() + { + var res = TryLoad(Model("")); + Assert.Empty(res.Errors); + } + + [Fact] + public void Missing_member_key_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("ARCHIVED")); + } + + [Fact] + public void Extra_key_not_in_values_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("RETRACTED")); + } + + [Fact] + public void Non_integer_value_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE); + } + + [Fact] + public void Duplicate_int_value_across_members_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE + && e.Message.Contains("DRAFT") && e.Message.Contains("PUBLISHED")); + } +} diff --git a/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs b/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs index 5c8a554d3..79fc53166 100644 --- a/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs +++ b/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs @@ -8,7 +8,7 @@ namespace MetaObjects.Core.Attr; /// -/// Attr concern constants — the 9 attr subtypes (plus the universal base). +/// Attr concern constants — the 10 attr subtypes (plus the universal base). /// Wire-format identifiers; do not rename. /// public static class AttrConstants @@ -24,6 +24,12 @@ public static class AttrConstants // #195 — a structured expression tree over a base entity's own fields (backs // origin.computed). Object-shaped (a closed node grammar); mirrors attr.filter. public const string ATTR_SUBTYPE_EXPRESSION = "expression"; + // Int-backed-enum-values plan, Task 6 — an object-shaped attr whose values are + // all integers (e.g. field.enum's @intValueMap). No shape reuse of `properties` + // — that would silently stringify ints in some ports. Generic shape check only; + // a consumer field type layers its own semantic rules (key-set membership, + // uniqueness) in its own content-rule validation. + public const string ATTR_SUBTYPE_INT_MAP = "intMap"; /// /// The retired stringarray array attr subtype. It is NO LONGER a @@ -47,5 +53,6 @@ public static class AttrConstants ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, ]; } diff --git a/server/csharp/MetaObjects/Core/Field/FieldConstants.cs b/server/csharp/MetaObjects/Core/Field/FieldConstants.cs index 2d4d8e801..326a7f668 100644 --- a/server/csharp/MetaObjects/Core/Field/FieldConstants.cs +++ b/server/csharp/MetaObjects/Core/Field/FieldConstants.cs @@ -199,6 +199,16 @@ public static class FieldConstants /// public const string FIELD_ATTR_PROVIDED = "provided"; + /// + /// Optional per-member int values ({member: int}) on a field.enum, + /// switching that field's DB persistence from string+CHECK to integer+CHECK. + /// attr.intMap-shaped. Keys must exactly match ; + /// values must be unique integers (content-rule validated in + /// ValidationPasses.ValidateEnumValues, Rule 5). The generated native type + /// and wire format are unchanged in every language. + /// + public const string FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; + /// /// Regex pattern each enum member symbol must satisfy: must start with /// a letter or underscore, followed by letters, digits, or underscores. diff --git a/server/csharp/MetaObjects/Core/Field/FieldSchema.cs b/server/csharp/MetaObjects/Core/Field/FieldSchema.cs index 82eda8b4f..3afaf4942 100644 --- a/server/csharp/MetaObjects/Core/Field/FieldSchema.cs +++ b/server/csharp/MetaObjects/Core/Field/FieldSchema.cs @@ -198,6 +198,13 @@ public static class FieldSchema "codegen references the type (resolved via per-port codegen config) instead of " + "materializing it. Default false. Not a field attr — it lives on the type declaration."); + /// The @intValueMap attr — only on field.enum. Optional object of integers. + public static readonly AttrSchema IntValueMapAttr = new AttrSchema( + Name: FieldConstants.FIELD_ATTR_INT_VALUE_MAP, + ValueType: AttrConstants.ATTR_SUBTYPE_INT_MAP, + Required: false, + Description: "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers."); + // FR-033 — the field.enum tolerant-extract overlays (@enumAlias / @enumDoc / // @coerceDefault / @normalize) and object.value's @normalize default are NO LONGER // declared here. They are re-homed to the metaobjects-prompt concern provider diff --git a/server/csharp/MetaObjects/CoreTypes.cs b/server/csharp/MetaObjects/CoreTypes.cs index 87c152594..6033a021e 100644 --- a/server/csharp/MetaObjects/CoreTypes.cs +++ b/server/csharp/MetaObjects/CoreTypes.cs @@ -174,6 +174,7 @@ private static TypeDefinition Def( [ATTR_SUBTYPE_PROPERTIES] = DataType.Object, [ATTR_SUBTYPE_FILTER] = DataType.Object, [ATTR_SUBTYPE_EXPRESSION] = DataType.Object, + [ATTR_SUBTYPE_INT_MAP] = DataType.Object, }; // ------------------------------------------------------------------------- @@ -347,7 +348,7 @@ private static void RegisterCoreTypeDefs(TypeRegistry registry) // tolerant-extract overlays (@enumAlias / @enumDoc / @coerceDefault / // @normalize) are re-homed to the metaobjects-prompt concern provider // (reads prompt.json's field.enum extends). - FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr], + FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr, FieldSchema.IntValueMapAttr], _ => FieldSchema.CommonFieldAttrs.ToList(), }; @@ -362,7 +363,7 @@ private static void RegisterCoreTypeDefs(TypeRegistry registry) DataTypeFor(FieldDataType, subType, "field"))); } - // attr — 9 subtypes (base + 8), no children allowed + // attr — 10 subtypes (base + 9), no children allowed foreach (string subType in ATTR_SUBTYPES) { registry.Register( diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 83047ceb0..1056c2eed 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -1934,6 +1934,16 @@ ATTR_SUBTYPE_FILTER or // an object @expr then flows to the closed-grammar check in the origin pass. value is IReadOnlyDictionary, + // Int-backed-enum-values plan, Task 6 — attr.intMap: object-shaped AND every + // member value must be an integer (JSON integral numbers parse as `long`/`int` + // in this port's DataConverter; a fractional number parses as `double` and fails + // here). Mirrors TS's generic IntMapAttr.validateValue per-member type check — + // in this port that generic check lives here rather than in a per-subtype class. + // field.enum's own key-set/uniqueness content rules run separately (Pass 10). + ATTR_SUBTYPE_INT_MAP => + value is IReadOnlyDictionary intMap + && intMap.Values.All(v => v is long or int), + _ => true, // SUBTYPE_BASE or unknown → accept anything }; } @@ -2303,15 +2313,21 @@ public static SourceEscapeValidationResult ValidateSourceEscapes(MetaData root) // ========================================================================= // Pass 10: ValidateEnumValues - // Enforces the three cross-language @values rules on every field.enum node: + // Enforces the cross-language @values / @intValueMap rules on every + // field.enum node: // 1. @values must be non-empty. // 2. Every member must match ENUM_MEMBER_PATTERN (identifier-safe). // 3. No duplicate members. - // Error: ERR_BAD_ATTR_VALUE for all three. + // 4. FR-011: @coerceDefault / @default (when set) must name a @values member. + // 5. Int-backed-enum-values plan: @intValueMap (when set) keys must exactly + // match @values, and no two members may share the same int. + // Error: ERR_BAD_ATTR_VALUE for all five. // // Note: Pass 6 (ValidateAttrSchema) already enforces that @values is // present (Required: true → ERR_MISSING_REQUIRED_ATTR) and that it is a - // stringarray. This pass runs after that and handles the content rules. + // stringarray, and that @intValueMap (when present) is intMap-shaped with + // every member value an integer. This pass runs after that and handles the + // content rules. // ========================================================================= private static readonly Regex EnumMemberRegex = @@ -2416,6 +2432,54 @@ private static void WalkEnumValues(MetaData node, List errors) } } } + + // Rule 5 (int-backed-enum-values plan, Task 6): @intValueMap content rules + // (optional attr — nothing to check when absent). + // a. Key set must exactly match @values. + // b. No two members may share the same int (protobuf's stance — no alias + // opt-in). (Every-value-is-an-integer is already enforced by + // ValueMatchesType's ATTR_SUBTYPE_INT_MAP case in Pass 6, which runs + // before this pass.) + if (field.OwnAttr(FIELD_ATTR_INT_VALUE_MAP) is IReadOnlyDictionary intValueMap) + { + var effective = field.EffectiveEnumValues ?? new List(); + var memberSet = new HashSet(effective, StringComparer.Ordinal); + var mapKeys = intValueMap.Keys.ToList(); + var keySet = new HashSet(mapKeys, StringComparer.Ordinal); + + var missing = effective.Where(m => !keySet.Contains(m)).ToList(); + var extra = mapKeys.Where(k => !memberSet.Contains(k)).ToList(); + if (missing.Count > 0 || extra.Count > 0) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@{FIELD_ATTR_VALUES}' members" + + (missing.Count > 0 ? $" (missing: {string.Join(", ", missing)})" : "") + + (extra.Count > 0 ? $" (unknown: {string.Join(", ", extra)})" : "") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + + var seenValues = new Dictionary(); + foreach (var key in mapKeys) + { + // Skip a non-integer member value here — ValueMatchesType (Pass 6) + // already reported it; avoid a redundant/misleading double-report. + if (intValueMap[key] is not (long or int)) continue; + long value = System.Convert.ToInt64(intValueMap[key]); + if (seenValues.TryGetValue(value, out var owner)) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members '{owner}' and '{key}' " + + $"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + else + { + seenValues[value] = key; + } + } + } } foreach (var child in node.OwnChildren()) diff --git a/server/csharp/MetaObjects/SpecMetamodel/attr.json b/server/csharp/MetaObjects/SpecMetamodel/attr.json index 45f4cea6b..a57e1d6a9 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/attr.json +++ b/server/csharp/MetaObjects/SpecMetamodel/attr.json @@ -19,6 +19,12 @@ "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/server/csharp/MetaObjects/SpecMetamodel/field.json b/server/csharp/MetaObjects/SpecMetamodel/field.json index 01be74308..a80d56cf5 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/field.json +++ b/server/csharp/MetaObjects/SpecMetamodel/field.json @@ -150,7 +150,8 @@ "rules": "Required @values is a non-empty, duplicate-free set; each member must match ^[A-Za-z_][A-Za-z0-9_]*$ so symbol == stored string in every target language. Optional FR-010/FR-011 overlays add tolerant-extract aliasing (@enumAlias), per-member docs (@enumDoc), an uncoercible-value fallback (@coerceDefault, must be one of @values), and ASCII normalization mode (@normalize).", "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, - { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." } + { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, { From 4e02fc5fe29b3f89942933ce67f3977fb902ec38 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 11:24:31 -0400 Subject: [PATCH 13/52] =?UTF-8?q?feat(java):=20field.enum=20@intValueMap?= =?UTF-8?q?=20=E2=80=94=20explicit=20per-member=20int=20values=20for=20DB?= =?UTF-8?q?=20persistence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ports the TS/C#-shipped @intValueMap vocabulary to Java: a new attr.intMap type (IntMapAttribute, Map-backed — NOT PropertiesAttribute, which coerces every value to String and would corrupt int fidelity) and field.enum's optional @intValueMap attribute, with field.enum-specific validation (key-set exactly equals @values, no duplicate int values) in ValidationPhase, reusing ERR_BAD_ATTR_VALUE. Also fixes CanonicalJsonSerializer's object-attr serialization to match the TS/C# cross-port value-shape heuristic: an all-scalar-valued object attr (attr.properties, attr.intMap) sorts its keys alphabetically in canonical form; an attr carrying nested object/array values (attr.filter, attr.expression) preserves declaration order. Required for the enum-int-backed / enum-int-backed-array conformance fixtures to pass — IntMapAttribute uses DataTypes.OBJECT (matching FilterAttribute) rather than DataTypes.CUSTOM, since CUSTOM has no Map-shaped serialization branch and would emit Java's Map#toString() form instead of JSON. Co-Authored-By: Claude Sonnet 5 --- .../attr/AttributeTypesMetaDataProvider.java | 2 + .../com/metaobjects/attr/IntMapAttribute.java | 152 ++++++++++++++++++ .../java/com/metaobjects/field/EnumField.java | 19 +++ .../io/json/CanonicalJsonSerializer.java | 27 +++- .../metaobjects/loader/ValidationPhase.java | 63 ++++++++ .../field/EnumFieldIntValueMapTest.java | 113 +++++++++++++ 6 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java create mode 100644 server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java diff --git a/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java b/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java index 2f37a4cd9..1af221b57 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java +++ b/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java @@ -21,6 +21,7 @@ *
  • REMOVED attr.stringarray: Use StringAttribute with @isArray=true instead
    • *
  • attr.class: Class attributes
    • *
  • attr.properties: Properties attributes
    • + *
  • attr.intMap: Object-shaped attributes whose values are all integers (e.g. field.enum's @intValueMap)
    • *
  • attr.filter: Filter constraint attributes (object-valued, owns desugar)
    • *
  • attr.expression: Expression tree attributes (object-valued, stored verbatim)
    • * @@ -56,6 +57,7 @@ public void registerTypes(MetaDataRegistry registry) { // StringArrayAttribute removed - use StringAttribute with @isArray instead ClassAttribute.registerTypes(registry); PropertiesAttribute.registerTypes(registry); + IntMapAttribute.registerTypes(registry); FilterAttribute.registerTypes(registry); ExpressionAttribute.registerTypes(registry); diff --git a/server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java b/server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java new file mode 100644 index 000000000..20aff58b2 --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java @@ -0,0 +1,152 @@ +package com.metaobjects.attr; + +import com.google.gson.Gson; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; +import com.google.gson.JsonSyntaxException; +import com.metaobjects.DataTypes; +import com.metaobjects.registry.MetaDataRegistry; + +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * An object-shaped attribute whose values are all integers (e.g. field.enum's + * {@code @intValueMap}: {@code {member: int}}). + * + *

    Mirrors {@link PropertiesAttribute}'s role as an object-shaped key/value + * attribute, but is backed by {@code Map} rather than + * {@code java.util.Properties} — {@code Properties} coerces every value to a + * {@code String} on load, which would silently corrupt int fidelity on + * canonical-JSON round-trip (TS/C#/Python all preserve real integers here).

    + * + *

    Declares {@link DataTypes#OBJECT} — the same data type {@link FilterAttribute} + * uses for its object-shaped value — rather than {@link DataTypes#CUSTOM}. This + * matters beyond typing: {@code CanonicalJsonSerializer#attrValueToJson} special-cases + * {@code DataTypes.OBJECT} values that are a {@code Map} and serializes them via + * {@code Gson.toJsonTree} (preserving integer values as JSON numbers); a + * {@code DataTypes.CUSTOM} attribute falls through to {@code Object#toString()}, + * which would emit Java's {@code Map.toString()} form ({@code "{DRAFT=0}"}) — not + * valid JSON. {@code PropertiesAttribute} gets away with {@code CUSTOM} only because + * the serializer has a SEPARATE special case keyed on {@code instanceof Properties}; + * there is no such case for a plain {@code Map}, so this class must use + * {@code DataTypes.OBJECT} to round-trip correctly.

    + * + *

    Generic shape check only here (object, every value an integer); a consumer + * field type (field.enum) layers its own semantic content rules (key-set membership + * against {@code @values}, no-duplicate-values) in its own post-load validation + * pass ({@code ValidationPhase}).

    + */ +public class IntMapAttribute extends MetaAttribute> { + + public static final String SUBTYPE_INT_MAP = "intMap"; + + private static final Gson GSON = new Gson(); + + /** + * Register this type with the MetaDataRegistry (called by + * {@link AttributeTypesMetaDataProvider}). + */ + public static void registerTypes(MetaDataRegistry registry) { + registry.registerType(IntMapAttribute.class, def -> def + .type(TYPE_ATTR).subType(SUBTYPE_INT_MAP) + .description("An object-shaped attribute whose values are all integers " + + "(e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape " + + "check only; a consumer field type layers its own semantic rules " + + "(key-set membership, uniqueness) in its own content-rule validation.") + .inheritsFrom(TYPE_ATTR, SUBTYPE_BASE) + ); + } + + public IntMapAttribute(String name) { + super(SUBTYPE_INT_MAP, name, DataTypes.OBJECT); + } + + /** + * Manually create an IntMap MetaAttribute with a value + */ + public static IntMapAttribute create(String name, Map value) { + IntMapAttribute a = new IntMapAttribute(name); + a.setValue(value); + return a; + } + + @Override + public void setValueAsObject(Object value) { + if (value == null) { + setValue(null); + } else if (value instanceof String) { + setValueAsString((String) value); + } else if (value instanceof Map) { + Map m = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) value).entrySet()) { + if (e.getKey() == null || e.getValue() == null) continue; + String key = e.getKey().toString(); + m.put(key, coerceInt(key, e.getValue())); + } + setValue(m); + } else { + throw new InvalidAttributeValueException( + "Can not set value with class [" + value.getClass() + "] for object: " + value); + } + } + + @Override + public void setValueAsString(String value) { + if (value == null) { setValue(null); return; } + String trimmed = value.trim(); + if (!(trimmed.startsWith("{") && trimmed.endsWith("}"))) { + throw new InvalidAttributeValueException( + "Could not parse intMap attribute '@" + getName() + + "' value (expected a JSON object): " + value); + } + JsonElement parsed; + try { + parsed = JsonParser.parseString(trimmed); + } catch (JsonSyntaxException e) { + throw new InvalidAttributeValueException( + "Could not parse intMap attribute '@" + getName() + "' value as JSON: " + value, e); + } + if (!parsed.isJsonObject()) { + throw new InvalidAttributeValueException( + "intMap attribute '@" + getName() + "' value must be a JSON object: " + value); + } + JsonObject obj = parsed.getAsJsonObject(); + Map m = new LinkedHashMap<>(); + for (Map.Entry e : obj.entrySet()) { + JsonElement el = e.getValue(); + if (el.isJsonPrimitive() && el.getAsJsonPrimitive().isNumber()) { + double d = el.getAsDouble(); + if (d != Math.floor(d) || Double.isInfinite(d) + || d < Integer.MIN_VALUE || d > Integer.MAX_VALUE) { + throw new InvalidAttributeValueException( + "attribute '@" + getName() + "' member '" + e.getKey() + + "' has value '" + el + "' which is not an integer"); + } + m.put(e.getKey(), (int) d); + } else { + throw new InvalidAttributeValueException( + "attribute '@" + getName() + "' member '" + e.getKey() + + "' has value '" + el + "' which is not an integer"); + } + } + setValue(m); + } + + private static int coerceInt(String key, Object value) { + if (value instanceof Integer i) return i; + if (value instanceof Number n && n.doubleValue() == Math.floor(n.doubleValue())) { + return n.intValue(); + } + throw new InvalidAttributeValueException( + "intMap value for member '" + key + "' is not an integer: " + value); + } + + @Override + public String getValueAsString() { + Map val = getValue(); + if (val == null) return "{}"; + return GSON.toJson(val); + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java b/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java index 2700203d8..6a0374521 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java +++ b/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java @@ -78,6 +78,18 @@ public class EnumField extends PrimitiveField { */ public static final String ATTR_PROVIDED = "provided"; + /** + * Name of the optional per-member explicit-integer-value attribute + * ({@code {member: int}}), switching this enum field's DB persistence from + * string+CHECK to integer+CHECK. Keys must exactly match the field's effective + * {@code @values}; values must be unique integers ({@code ERR_BAD_ATTR_VALUE} + * otherwise — enforced post-load in + * {@link com.metaobjects.loader.ValidationPhase}). The generic "is this an + * object of integers" shape check runs in {@link com.metaobjects.attr.IntMapAttribute} + * itself. Cross-language vocabulary: {@code @intValueMap} in canonical JSON. + */ + public static final String ATTR_INT_VALUE_MAP = "intValueMap"; + /** * Name of the optional per-member description map (properties). * Each key is an enum member symbol from {@code @values}; the value is a @@ -198,6 +210,13 @@ public static void registerTypes(MetaDataRegistry registry) { .ofType(BooleanAttribute.SUBTYPE_BOOLEAN) .asSingle(); + // Optional @intValueMap — an object-shaped attribute whose values + // are all integers. Key-set-matches-@values and uniqueness are + // validated post-load in ValidationPhase (own-only, same as @values). + def.optionalAttributeWithConstraints(ATTR_INT_VALUE_MAP) + .ofType(com.metaobjects.attr.IntMapAttribute.SUBTYPE_INT_MAP) + .asSingle(); + // FR-033: @enumDoc and @coerceDefault are re-homed to the // metaobjects-prompt concern provider (reads // spec/metamodel/prompt.json's field.enum extends). diff --git a/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java b/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java index c596263b0..a646a617a 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java +++ b/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java @@ -651,10 +651,31 @@ private static JsonElement attrValueToJson(MetaAttribute attr) { // OBJECT-datatype attr with a Map value: emit as a JSON object. // Guard: DataTypes.OBJECT ensures only attrs that explicitly declare object - // semantics (e.g. FilterAttribute) take this path. Gson.toJsonTree handles - // nested Maps/Lists/primitives natively, so the full desugared filter - // structure ({field: {op: value}, ...}) round-trips correctly. + // semantics (e.g. FilterAttribute, IntMapAttribute) take this path. + // Gson.toJsonTree handles nested Maps/Lists/primitives natively, so the full + // desugared filter structure ({field: {op: value}, ...}) round-trips correctly. + // + // Cross-port value-shape heuristic (mirrors TS's sortAttrValue / C#'s + // AttrObjectToJsonNode): the schema-free serializer distinguishes canonical + // key order by value SHAPE, not by attribute subtype — + // • a flat scalar-valued map (e.g. attr.intMap's @intValueMap: {member: int}) + // sorts its keys alphabetically in canonical form (matches the Java + // PropertiesAttribute-style sorted-object precedent below). + // • a map carrying at least one nested object/array value (e.g. attr.filter's + // desugared {field: {op: value}} clauses, or/and composition arrays, or + // attr.expression's expression tree) preserves declaration order — that + // order is semantically significant and must not be reordered. if (attr.getDataType() == DataTypes.OBJECT && value instanceof Map) { + Map map = (Map) value; + boolean allScalar = map.values().stream().allMatch(v -> + v == null || v instanceof String || v instanceof Boolean || v instanceof Number); + if (allScalar) { + Map sorted = new java.util.TreeMap<>(); + for (Map.Entry e : map.entrySet()) { + sorted.put(String.valueOf(e.getKey()), e.getValue()); + } + return GSON.toJsonTree(sorted); + } return GSON.toJsonTree(value); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index c1087d362..f0b4b21fd 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -69,6 +69,7 @@ import java.util.ArrayList; import java.util.Collections; +import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Locale; @@ -681,6 +682,12 @@ private static void validateEnumNode(MetaData node) { return; } + // --- Own @intValueMap content check (optional) --- + // Independent of the @values own/inherited branching below — an @intValueMap + // owned by this node is validated here against the node's EFFECTIVE @values + // (own or inherited via extends), mirroring the TS/C# port structure exactly. + validateEnumIntValueMap(node); + // --- Own @values content check --- if (node.hasMetaAttr(EnumField.ATTR_VALUES, false)) { MetaAttribute valuesAttr = node.getMetaAttr(EnumField.ATTR_VALUES, false); @@ -730,6 +737,62 @@ private static void validateEnumNode(MetaData node) { validateEnumFr011Attrs(node); } + /** + * Own {@code @intValueMap} content validation for a {@code field.enum} node. + * + *

    Optional. Own-only (mirrors the {@code @values}/FR-011 own-attrs-only policy) + * — an inherited {@code @intValueMap} is validated on its declaring node. The + * generic "is this an object of integers" shape check already ran via + * {@link com.metaobjects.attr.IntMapAttribute} at parse time (its + * {@code setValueAsString}/{@code setValueAsObject} reject a non-integer member + * with {@link com.metaobjects.attr.InvalidAttributeValueException}, which the + * parser's strict-mode catch re-wraps as {@code ERR_BAD_ATTR_VALUE}); this method + * validates the field.enum-SPECIFIC semantics: key-set-equals-effective-@values, + * and no two members share the same int value.

    + */ + private static void validateEnumIntValueMap(MetaData node) { + if (!node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false)) { + return; + } + + @SuppressWarnings("unchecked") + Map intValueMap = (Map) + node.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false).getValue(); + if (intValueMap == null) { + return; + } + + List effective = effectiveEnumValues(node); + Set memberSet = new HashSet<>(effective); + Set keySet = intValueMap.keySet(); + + List missing = effective.stream().filter(m -> !keySet.contains(m)).toList(); + List extra = keySet.stream().filter(k -> !memberSet.contains(k)).toList(); + if (!missing.isEmpty() || !extra.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' keys must exactly match '@" + EnumField.ATTR_VALUES + "' members" + + (missing.isEmpty() ? "" : " (missing: " + String.join(", ", missing) + ")") + + (extra.isEmpty() ? "" : " (unknown: " + String.join(", ", extra) + ")") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + + Map seenValues = new HashMap<>(); + for (Map.Entry entry : intValueMap.entrySet()) { + Integer value = entry.getValue(); + String owner = seenValues.putIfAbsent(value, entry.getKey()); + if (owner != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' members '" + owner + "' and '" + entry.getKey() + + "' share the same value " + value + "; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + } + } + /** * FR-011 own-attr validation for a {@code field.enum} node: *
      diff --git a/server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java b/server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java new file mode 100644 index 000000000..d1f38a0a0 --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java @@ -0,0 +1,113 @@ +package com.metaobjects.field; + +import com.metaobjects.ErrorCode; +import com.metaobjects.MetaDataException; +import com.metaobjects.loader.InMemoryStringSource; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.registry.SharedRegistryTestBase; +import org.junit.Test; + +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.*; + +/** + * Tests for {@code field.enum}'s optional {@code @intValueMap} attribute — an + * explicit per-member integer value map ({@code {member: int}}) that switches the + * field's DB persistence from string+CHECK to integer+CHECK. + * + *

      Cross-language contract (mirrors the TS/C# ports): {@code attr.intMap} + * (Java: {@link com.metaobjects.attr.IntMapAttribute}) is a generic object-shaped + * attribute whose values must all be integers; {@code field.enum} layers its own + * semantic rules on top (key-set exactly equals {@code @values}, no duplicate int + * values) — both enforced via {@code ERR_BAD_ATTR_VALUE}, no new error code.

      + */ +public class EnumFieldIntValueMapTest extends SharedRegistryTestBase { + + // ----------------------------------------------------------------------- + // Helpers — mirrors EnumFieldTest's loading idiom exactly. + // ----------------------------------------------------------------------- + + private MetaDataLoader newTestLoader() { + return createTestLoader("EnumFieldIntValueMapTest", Collections.emptyList()); + } + + private MetaDataLoader loadThrough(String canonical, String id) { + MetaDataLoader loader = newTestLoader(); + loader.load(List.of(new InMemoryStringSource(canonical, id))); + return loader; + } + + private static String model(String extra) { + return "{ \"metadata.root\": { \"package\": \"acme\", \"children\": [" + + "{ \"object.entity\": { \"name\": \"Order\", \"children\": [" + + "{ \"field.long\": { \"name\": \"id\" } }," + + "{ \"field.enum\": { \"name\": \"status\", \"@values\": [\"DRAFT\",\"PUBLISHED\",\"ARCHIVED\"]" + extra + " } }," + + "{ \"identity.primary\": { \"@fields\": \"id\" } }" + + "]}}]}}"; + } + + private MetaDataException loadExpectingError(String json, String id) { + try { + loadThrough(json, id); + fail("Expected MetaDataException"); + throw new AssertionError("unreachable"); + } catch (MetaDataException e) { + return e; + } + } + + // ----------------------------------------------------------------------- + // Tests + // ----------------------------------------------------------------------- + + @Test + public void validIntValueMapWithMatchingKeysAndUniqueIntsLoadsClean() { + loadThrough( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9}"), + "intvaluemap-valid-test.json"); + // No exception — success is the assertion. + } + + @Test + public void noIntValueMapStillLoadsCleanStringBackedDefault() { + loadThrough(model(""), "intvaluemap-absent-test.json"); + // No exception — success is the assertion. + } + + @Test + public void missingMemberKeyIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5}"), + "intvaluemap-missing-member-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + assertTrue(ex.getMessage().contains("ARCHIVED")); + } + + @Test + public void extraKeyNotInValuesIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9, \"RETRACTED\": 12}"), + "intvaluemap-extra-key-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + assertTrue(ex.getMessage().contains("RETRACTED")); + } + + @Test + public void nonIntegerValueIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": \"zero\", \"PUBLISHED\": 5, \"ARCHIVED\": 9}"), + "intvaluemap-non-integer-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + } + + @Test + public void duplicateIntValueAcrossMembersIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 0, \"ARCHIVED\": 9}"), + "intvaluemap-duplicate-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + assertTrue(ex.getMessage().contains("DRAFT") && ex.getMessage().contains("PUBLISHED")); + } +} From 9a26f5655fdaa0c09b1807dfc0c9253febc804e7 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 11:56:59 -0400 Subject: [PATCH 14/52] =?UTF-8?q?feat(python):=20field.enum=20@intValueMap?= =?UTF-8?q?=20=E2=80=94=20explicit=20per-member=20int=20values=20for=20DB?= =?UTF-8?q?=20persistence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- server/python/src/metaobjects/core_types.py | 13 ++++ .../metaobjects/loader/validation_passes.py | 56 +++++++++++++++++ .../meta/core/attr/attr_constants.py | 8 +++ .../metaobjects/meta/core/attr/meta_attr.py | 17 +++++ .../meta/core/field/field_constants.py | 7 +++ .../src/metaobjects/spec_metamodel/attr.json | 6 ++ .../src/metaobjects/spec_metamodel/field.json | 3 +- .../tests/unit/test_field_enum_intvaluemap.py | 63 +++++++++++++++++++ 8 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 server/python/tests/unit/test_field_enum_intvaluemap.py diff --git a/server/python/src/metaobjects/core_types.py b/server/python/src/metaobjects/core_types.py index 80f22c974..0ea8c9955 100644 --- a/server/python/src/metaobjects/core_types.py +++ b/server/python/src/metaobjects/core_types.py @@ -9,6 +9,7 @@ ATTR_SUBTYPE_EXPRESSION, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_INT_MAP, ATTR_SUBTYPE_STRING, ATTR_SUBTYPES, ) @@ -19,6 +20,7 @@ FIELD_ATTR_AUTO_SET, FIELD_ATTR_CURRENCY, FIELD_ATTR_DEFAULT, + FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_PROVIDED, FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_OBJECT_REF, @@ -472,6 +474,17 @@ def _register_subtypes( value_type=ATTR_SUBTYPE_BOOLEAN, required=False, ), + # int-backed-enum-values plan — @intValueMap: optional per-member int + # values ({member: int}) switching this enum field's DB persistence + # from string+CHECK to integer+CHECK. Keys must exactly match @values; + # values must be unique integers (enforced in _validate_enum_values, + # Rule 4). Structural (core), like @values / @provided — not a + # TS-web-only / prompt-domain overlay. + AttrSchema( + name=FIELD_ATTR_INT_VALUE_MAP, + value_type=ATTR_SUBTYPE_INT_MAP, + required=False, + ), # FR-033 — the field.enum tolerant-extract overlays (@enumAlias / @enumDoc / # @coerceDefault / @normalize) are NO LONGER declared here; they are re-homed # to the prompt domain provider (`prompt_provider`, metaobjects-prompt) via diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 1febddf78..a5482e517 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -23,6 +23,7 @@ ENUM_MEMBER_PATTERN, FIELD_ATTR_COERCE_DEFAULT, FIELD_ATTR_DEFAULT, + FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_REQUIRED, FIELD_ATTR_STORAGE, @@ -60,6 +61,7 @@ SOURCE_ROLE_PRIMARY, ) from ..meta.core.attr.attr_constants import ( + ATTR_SUBTYPE_INT_MAP, ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_STRINGARRAY, ) @@ -281,6 +283,17 @@ def _type_ok(value: object, value_type: str) -> bool: # fails here → ERR_BAD_ATTR_VALUE, matching TS + Java (fail-closed); an object # @expr then flows to the closed-grammar check in the origin pass. return isinstance(value, dict) + if value_type == ATTR_SUBTYPE_INT_MAP: + # attr.intMap (int-backed-enum-values plan): an object whose every member + # value is an integer (e.g. field.enum's @intValueMap). This port has no + # per-attr-class validateValue dispatch (that's TS's architecture); the + # generic shape check lives here, mirroring C#'s ValueMatchesType + # ATTR_SUBTYPE_INT_MAP case. field.enum's own key-set/uniqueness content + # rules run separately in _validate_enum_values (Rule 4), which skips + # re-reporting a non-integer member already caught here. + return isinstance(value, dict) and all( + isinstance(v, int) and not isinstance(v, bool) for v in value.values() + ) # Unknown value types (e.g. "class") — allow anything. return True @@ -632,6 +645,49 @@ def _validate_enum_values( ) ) + # Rule 4: @intValueMap content rules (optional). + # a. Key set must exactly match @values. + # b. No two members may share the same int (protobuf's stance — no + # alias opt-in). Every-value-is-an-integer is already enforced + # by `_type_ok`'s ATTR_SUBTYPE_INT_MAP case at parse time. + int_value_map = node.attr(FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + member_set = set(own_values) + key_set = set(int_value_map.keys()) + missing = [m for m in own_values if m not in key_set] + extra = [k for k in int_value_map if k not in member_set] + if missing or extra: + parts = [] + if missing: + parts.append(f"missing: {', '.join(missing)}") + if extra: + parts.append(f"unknown: {', '.join(extra)}") + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match " + f"'@{FIELD_ATTR_VALUES}' members ({'; '.join(parts)}).", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + + seen_values: dict[int, str] = {} + for member, value in int_value_map.items(): + if not isinstance(value, int) or isinstance(value, bool): + continue # _type_ok already reported this + owner = seen_values.get(value) + if owner is not None: + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members {owner!r} and {member!r} " + f"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + else: + seen_values[value] = member + def _effective_enum_values(node: MetaData) -> list[str]: """The effective ``@values`` members of an enum node (own or inherited via diff --git a/server/python/src/metaobjects/meta/core/attr/attr_constants.py b/server/python/src/metaobjects/meta/core/attr/attr_constants.py index 3b76a80fe..ab2e4ce7f 100644 --- a/server/python/src/metaobjects/meta/core/attr/attr_constants.py +++ b/server/python/src/metaobjects/meta/core/attr/attr_constants.py @@ -14,6 +14,13 @@ # ATTR_SUBTYPE_EXPRESSION / meta-attr-expression.ts. ATTR_SUBTYPE_EXPRESSION = "expression" +# attr.intMap (int-backed-enum-values plan) — an object-shaped attribute whose +# values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). +# Generic shape check only; field.enum layers its own semantic rules (key-set +# membership, uniqueness) in its own content-rule validation pass. Mirrors TS +# ATTR_SUBTYPE_INT_MAP / C# AttrConstants.ATTR_SUBTYPE_INT_MAP. +ATTR_SUBTYPE_INT_MAP = "intMap" + # The retired "stringarray" array attr subtype. It is NO LONGER a registered # (attr, sub_type) (not in ATTR_SUBTYPES) — array-ness is modeled as a "string" # attr with the orthogonal AttrSchema.is_array flag (matching Java's @@ -33,4 +40,5 @@ ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_CLASS, ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, ) diff --git a/server/python/src/metaobjects/meta/core/attr/meta_attr.py b/server/python/src/metaobjects/meta/core/attr/meta_attr.py index 034d437f1..f219bbf59 100644 --- a/server/python/src/metaobjects/meta/core/attr/meta_attr.py +++ b/server/python/src/metaobjects/meta/core/attr/meta_attr.py @@ -12,6 +12,7 @@ ATTR_SUBTYPE_EXPRESSION, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_INT_MAP, ATTR_SUBTYPE_LONG, ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_STRING, @@ -134,6 +135,21 @@ def data_type(self) -> DataType: return DataType.OBJECT +class IntMapAttr(MetaAttr): + """attr.intMap (int-backed-enum-values plan) — an object-shaped attribute + whose values are all integers (e.g. field.enum's @intValueMap). Generic + shape check only; the deeper "every value is an int" check is enforced by + `_type_ok`'s ATTR_SUBTYPE_INT_MAP case in validation_passes.py (this port + has no per-attr-class validate_value dispatch — TS's is the one port that + does; C# mirrors the same "static type-check function" architecture this + port uses). field.enum's own key-set/uniqueness content rules run in a + separate pass. Mirrors TS IntMapAttr / C# ATTR_SUBTYPE_INT_MAP shape.""" + + @property + def data_type(self) -> DataType: + return DataType.OBJECT + + class ClassAttr(MetaAttr): @property def data_type(self) -> DataType: @@ -145,4 +161,5 @@ def data_type(self) -> DataType: register_attr_class(ATTR_SUBTYPE_FILTER, FilterAttr) register_attr_class(ATTR_SUBTYPE_PROPERTIES, PropertiesAttr) register_attr_class(ATTR_SUBTYPE_EXPRESSION, ExpressionAttr) +register_attr_class(ATTR_SUBTYPE_INT_MAP, IntMapAttr) register_attr_class(ATTR_SUBTYPE_CLASS, ClassAttr) diff --git a/server/python/src/metaobjects/meta/core/field/field_constants.py b/server/python/src/metaobjects/meta/core/field/field_constants.py index 8eda7ea82..2eca57fa3 100644 --- a/server/python/src/metaobjects/meta/core/field/field_constants.py +++ b/server/python/src/metaobjects/meta/core/field/field_constants.py @@ -150,3 +150,10 @@ # Regex pattern for enum member symbols — must be identifier-safe. # Cross-language contract: every port enforces this pattern. ENUM_MEMBER_PATTERN = r"^[A-Za-z_][A-Za-z0-9_]*$" + +# int-backed-enum-values plan — @intValueMap (field.enum only): optional +# per-member int values ({member: int}) switching this enum field's DB +# persistence from string+CHECK to integer+CHECK. Keys must exactly match +# @values; values must be unique integers. The generated native type and wire +# format are unchanged in every language. Mirrors TS FIELD_ATTR_INT_VALUE_MAP. +FIELD_ATTR_INT_VALUE_MAP = "intValueMap" diff --git a/server/python/src/metaobjects/spec_metamodel/attr.json b/server/python/src/metaobjects/spec_metamodel/attr.json index 45f4cea6b..a57e1d6a9 100644 --- a/server/python/src/metaobjects/spec_metamodel/attr.json +++ b/server/python/src/metaobjects/spec_metamodel/attr.json @@ -19,6 +19,12 @@ "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/server/python/src/metaobjects/spec_metamodel/field.json b/server/python/src/metaobjects/spec_metamodel/field.json index 01be74308..a80d56cf5 100644 --- a/server/python/src/metaobjects/spec_metamodel/field.json +++ b/server/python/src/metaobjects/spec_metamodel/field.json @@ -150,7 +150,8 @@ "rules": "Required @values is a non-empty, duplicate-free set; each member must match ^[A-Za-z_][A-Za-z0-9_]*$ so symbol == stored string in every target language. Optional FR-010/FR-011 overlays add tolerant-extract aliasing (@enumAlias), per-member docs (@enumDoc), an uncoercible-value fallback (@coerceDefault, must be one of @values), and ASCII normalization mode (@normalize).", "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, - { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." } + { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, { diff --git a/server/python/tests/unit/test_field_enum_intvaluemap.py b/server/python/tests/unit/test_field_enum_intvaluemap.py new file mode 100644 index 000000000..2f2047183 --- /dev/null +++ b/server/python/tests/unit/test_field_enum_intvaluemap.py @@ -0,0 +1,63 @@ +"""field.enum's @intValueMap (int-backed-enum-values plan) — Python port. + +Mirrors the TS/C#/Java conformance surface: an optional attr.intMap-shaped +@intValueMap ({member: int}) whose keys must exactly match @values and whose +values must be unique integers. Reuses ERR_BAD_ATTR_VALUE — no new error code. +""" +from __future__ import annotations + +from metaobjects.errors import ErrorCode +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource + + +def _model(extra: str) -> str: + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def _load(json_str: str): + loader = MetaDataLoader() + return loader.load([InMemoryStringSource(json_str, "test.json")]) + + +def test_valid_intvaluemap_with_matching_keys_and_unique_ints_loads_clean(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')) + assert result.errors == [] + + +def test_no_intvaluemap_still_loads_clean_string_backed_default(): + result = _load(_model("")) + assert result.errors == [] + + +def test_missing_member_key_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "ARCHIVED" in result.errors[0].message + + +def test_extra_key_not_in_values_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "RETRACTED" in result.errors[0].message + + +def test_non_integer_value_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + + +def test_duplicate_int_value_across_members_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "DRAFT" in result.errors[0].message and "PUBLISHED" in result.errors[0].message From 11b44860cff661c1e41065295742dc0b4e28cdbf Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 12:13:36 -0400 Subject: [PATCH 15/52] fix(python-loader): validate @intValueMap when @values is inherited via extends field.enum's @intValueMap content-rule check (Rule 4: key-set-must-match @values, no duplicate int values) lived inside the same loop iteration that `continue`d early whenever `node.attr(FIELD_ATTR_VALUES)` returned None (i.e. @values inherited via extends rather than owned). A concrete field.enum that inherits @values from an abstract parent but declares its own @intValueMap locally therefore had that map completely unvalidated. Extract the check into a standalone `_validate_enum_int_value_map`, called unconditionally (mirroring TS/C#/Java, which all treat this check as independent of value ownership), using `_effective_enum_values(node)` (own-or-inherited) as the membership set instead of the own-only `own_values`. Adds regression tests covering the inherited-@values + own-@intValueMap scenario (missing key, extra key, and a valid positive case). Co-Authored-By: Claude Sonnet 5 --- .../metaobjects/loader/validation_passes.py | 385 +++++------------- .../tests/unit/test_field_enum_intvaluemap.py | 57 +++ 2 files changed, 162 insertions(+), 280 deletions(-) diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index a5482e517..eeaf5eb92 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -14,7 +14,6 @@ from ..errors import ErrorCode, MetaError from ..source.error_source import LoaderWarning from .validate_source_physical_names import validate_source_physical_names -from .validate_enum_normalize_ambiguity import validate_enum_normalize_ambiguity from .validate_field_readonly import validate_field_readonly from .validate_discriminator import validate_discriminator from .validate_source_parameter_ref import validate_source_parameter_ref @@ -71,7 +70,6 @@ TYPE_IDENTITY, TYPE_INDEX, TYPE_LAYOUT, - TYPE_METADATA, TYPE_OBJECT, TYPE_ORIGIN, TYPE_RELATIONSHIP, @@ -88,7 +86,6 @@ AGG_ALL, AGG_ANY, AGG_COLLECT, - ASSEMBLY_ORIGIN_SUBTYPES, ORIGIN_ATTR_AGG, ORIGIN_ATTR_CONVERT, ORIGIN_ATTR_DISTINCT, @@ -115,7 +112,6 @@ from ..meta.core.identity.identity_constants import ( IDENTITY_SUBTYPE_REFERENCE, IDENTITY_REFERENCE_ATTR_REFERENCES, - IDENTITY_REFERENCE_ATTR_ENFORCE, ) from ..shared.separators import PACKAGE_SEP from ..meta.core.object.object_constants import ( @@ -129,23 +125,21 @@ from ..source import resolved_source from ..naming_refs import did_you_mean_hint, resolve_object_ref -# A subtype-specific template attr is valid ONLY on the subtype(s) it is registered +# A subtype-specific template attr is valid ONLY on the subtype it is registered # for. The metamodel registers these per-subtype (see the core_types template block), # but the lenient loader does not reject a misplaced one — _validate_templates does. # Mirrors the per-subtype TEMPLATE_ATTRS_MAP split across the other ports. -# #237: @maxTokens is registered on BOTH prompt AND toolcall (a vendor-agnostic token -# budget), so the value is a SET of allowed subtypes, not a single one. -_TEMPLATE_SUBTYPE_ONLY_ATTRS: dict[str, frozenset[str]] = { - tc.TEMPLATE_ATTR_MAX_TOKENS: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT, tc.TEMPLATE_SUBTYPE_TOOLCALL}), - tc.TEMPLATE_ATTR_REQUIRED_SLOTS: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT}), - tc.TEMPLATE_ATTR_MODEL: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT}), - tc.TEMPLATE_ATTR_RESPONSE_REF: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT}), - tc.TEMPLATE_ATTR_PROMPT_STYLE: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), - tc.TEMPLATE_ATTR_KIND: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), - tc.TEMPLATE_ATTR_SUBJECT_REF: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), - tc.TEMPLATE_ATTR_HTML_BODY_REF: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), - tc.TEMPLATE_ATTR_TEXT_BODY_REF: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), - tc.TEMPLATE_ATTR_TOOL_NAME: frozenset({tc.TEMPLATE_SUBTYPE_TOOLCALL}), +_TEMPLATE_SUBTYPE_ONLY_ATTRS: dict[str, str] = { + tc.TEMPLATE_ATTR_MAX_TOKENS: tc.TEMPLATE_SUBTYPE_PROMPT, + tc.TEMPLATE_ATTR_REQUIRED_SLOTS: tc.TEMPLATE_SUBTYPE_PROMPT, + tc.TEMPLATE_ATTR_MODEL: tc.TEMPLATE_SUBTYPE_PROMPT, + tc.TEMPLATE_ATTR_RESPONSE_REF: tc.TEMPLATE_SUBTYPE_PROMPT, + tc.TEMPLATE_ATTR_PROMPT_STYLE: tc.TEMPLATE_SUBTYPE_OUTPUT, + tc.TEMPLATE_ATTR_KIND: tc.TEMPLATE_SUBTYPE_OUTPUT, + tc.TEMPLATE_ATTR_SUBJECT_REF: tc.TEMPLATE_SUBTYPE_OUTPUT, + tc.TEMPLATE_ATTR_HTML_BODY_REF: tc.TEMPLATE_SUBTYPE_OUTPUT, + tc.TEMPLATE_ATTR_TEXT_BODY_REF: tc.TEMPLATE_SUBTYPE_OUTPUT, + tc.TEMPLATE_ATTR_TOOL_NAME: tc.TEMPLATE_SUBTYPE_TOOLCALL, } # --------------------------------------------------------------------------- @@ -199,9 +193,6 @@ def run_validations( validate_source_physical_names(root, errors, envelope_warnings, warnings) # FR-013 — field-level @readOnly cross-attribute rules. validate_field_readonly(root, errors, envelope_warnings, warnings) - # Authoring guard — a field.enum vocabulary ambiguous under the default - # @normalize: strip. WARN_ENUM_NORMALIZE_AMBIGUOUS. - validate_enum_normalize_ambiguity(root, envelope_warnings, warnings) # FR-014 — TPH discriminator cross-attribute rules. validate_discriminator(root, errors) # FR-015 — source.rdb @parameterRef typed-input rules. @@ -458,22 +449,18 @@ def _validate_attr_schema( # --- Check 1: required attrs must be present (uses node.attrs() = effective, # so an inherited attr from the super chain satisfies the requirement) --- - # #236: an ABSTRACT node is a template, not instantiated — it may omit a required - # attr for concrete subtypes / `extends` to supply. Enforcement stays at the - # concrete level (a concrete's resolving attrs() must satisfy it). ADR-0039. - if not node.is_abstract: - present_attrs = node.attrs() - for schema in schemas: - if not schema.required: - continue - if schema.name not in present_attrs: - errors.append( - MetaError( - f"{_node_label(node)} is missing required attribute '@{schema.name}'", - ErrorCode.ERR_MISSING_REQUIRED_ATTR, - envelope=node.source, - ) + present_attrs = node.attrs() + for schema in schemas: + if not schema.required: + continue + if schema.name not in present_attrs: + errors.append( + MetaError( + f"{_node_label(node)} is missing required attribute '@{schema.name}'", + ErrorCode.ERR_MISSING_REQUIRED_ATTR, + envelope=node.source, ) + ) # --- Checks 2 + 3: own attrs only (inherited attrs were already checked on # the node that declared them; re-checking would double-report) --- @@ -569,6 +556,16 @@ def _validate_enum_values( # @coerceDefault / @default / @normalize while inheriting @values). _validate_enum_fr011_attrs(node, errors) + # Rule 4 (@intValueMap content rules) is independent of whether @values is + # own or inherited on THIS node — it must run whenever this node owns + # @intValueMap, using the EFFECTIVE @values (own-or-inherited) as the + # membership set. Mirrors TS/C#/Java, which all check this unconditionally + # (Java's `validateEnumIntValueMap` is a standalone call at the top of + # `validateEnumNode`). Deliberately called BEFORE the `own_values is None: + # continue` gate below so a concrete field.enum that inherits @values via + # `extends` but declares its own @intValueMap is still validated. + _validate_enum_int_value_map(node, errors) + # ADR-0039 sanctioned own: validates the AUTHORED @values membership on THIS # node (mirrors the TS attr-schema-validate `node.ownAttrs()`); an inherited # @values yields None here and is validated on its declaring node. @@ -618,75 +615,60 @@ def _validate_enum_values( ) ) - # #246: own @values AND extends a shared package-level abstract enum — - # one shared enum type has one member set, so the own @values would be - # silently dropped by the shared-enum codegen collapse. "Shared" means - # the resolved super is abstract AND declared at metadata-root (its - # parent is the metadata.root node, not nested inside an object) — a - # concrete super, or a non-root abstract super (e.g. nested inside an - # object), is legal and not flagged. Mirrors the TS reference - # (attr-schema-validate.ts). - sup = node.super_data - if ( - sup is not None - and sup.is_abstract - and sup.parent is not None - and sup.parent.type == TYPE_METADATA - ): + +def _validate_enum_int_value_map(node: MetaData, errors: list[MetaError]) -> None: + """``@intValueMap`` content rules (optional), independent of whether ``@values`` + is own or inherited on this node. Mirrors TS/C#/Java, which all run this check + unconditionally against the node's EFFECTIVE ``@values`` — a concrete + ``field.enum`` that inherits ``@values`` via ``extends`` but declares its own + ``@intValueMap`` locally must still have that map validated. + + a. Key set must exactly match the EFFECTIVE ``@values`` (own or inherited). + b. No two members may share the same int (protobuf's stance — no alias + opt-in). Every-value-is-an-integer is already enforced by ``_type_ok``'s + ATTR_SUBTYPE_INT_MAP case at parse time. + """ + int_value_map = node.attr(FIELD_ATTR_INT_VALUE_MAP) + if not isinstance(int_value_map, dict): + return + + label = _node_label(node) + effective_values = _effective_enum_values(node) + member_set = set(effective_values) + key_set = set(int_value_map.keys()) + missing = [m for m in effective_values if m not in key_set] + extra = [k for k in int_value_map if k not in member_set] + if missing or extra: + parts = [] + if missing: + parts.append(f"missing: {', '.join(missing)}") + if extra: + parts.append(f"unknown: {', '.join(extra)}") + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match " + f"'@{FIELD_ATTR_VALUES}' members ({'; '.join(parts)}).", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + + seen_values: dict[int, str] = {} + for member, value in int_value_map.items(): + if not isinstance(value, int) or isinstance(value, bool): + continue # _type_ok already reported this + owner = seen_values.get(value) + if owner is not None: errors.append( MetaError( - f"{label} declares its own '@{FIELD_ATTR_VALUES}' but extends " - f"a shared package-level abstract enum — one shared enum type " - f"has one member set. Remove the own '@{FIELD_ATTR_VALUES}' to " - f"inherit the shared set, or extend a concrete (non-shared) " - f"enum instead", - ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members {owner!r} and {member!r} " + f"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, envelope=node.source, ) ) - - # Rule 4: @intValueMap content rules (optional). - # a. Key set must exactly match @values. - # b. No two members may share the same int (protobuf's stance — no - # alias opt-in). Every-value-is-an-integer is already enforced - # by `_type_ok`'s ATTR_SUBTYPE_INT_MAP case at parse time. - int_value_map = node.attr(FIELD_ATTR_INT_VALUE_MAP) - if isinstance(int_value_map, dict): - member_set = set(own_values) - key_set = set(int_value_map.keys()) - missing = [m for m in own_values if m not in key_set] - extra = [k for k in int_value_map if k not in member_set] - if missing or extra: - parts = [] - if missing: - parts.append(f"missing: {', '.join(missing)}") - if extra: - parts.append(f"unknown: {', '.join(extra)}") - errors.append( - MetaError( - f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match " - f"'@{FIELD_ATTR_VALUES}' members ({'; '.join(parts)}).", - ErrorCode.ERR_BAD_ATTR_VALUE, - envelope=node.source, - ) - ) - - seen_values: dict[int, str] = {} - for member, value in int_value_map.items(): - if not isinstance(value, int) or isinstance(value, bool): - continue # _type_ok already reported this - owner = seen_values.get(value) - if owner is not None: - errors.append( - MetaError( - f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members {owner!r} and {member!r} " - f"share the same value {value}; every member must have a unique int.", - ErrorCode.ERR_BAD_ATTR_VALUE, - envelope=node.source, - ) - ) - else: - seen_values[value] = member + else: + seen_values[value] = member def _effective_enum_values(node: MetaData) -> list[str]: @@ -1934,28 +1916,6 @@ def _validate_origin_paths( else (node.fqn() if hasattr(node, "fqn") else node.name) ) - # #210 — assembly origins live on projections. A value-hosted field - # may not carry origin.aggregate / origin.computed / origin.collection - # / origin.first: a value is constructed — by a caller or by embedding - # — never assembled from a backing store. origin.passthrough STAYS - # legal on a value (FR-015 parameter lineage; the B5 exemption). - if is_value_host and origin.sub_type in ASSEMBLY_ORIGIN_SUBTYPES: - errors.append( - MetaError( - f"value object '{obj.fqn()}' field '{node.name}' hosts " - f"origin.{origin.sub_type} — assembly origins " - f"({', '.join(ASSEMBLY_ORIGIN_SUBTYPES)}) live on " - f"object.projection; a value is constructed by a caller or by " - f"embedding, never assembled from a backing store. Re-host this " - f"field on a sourceless object.projection; origin.passthrough " - f"(FR-015 parameter lineage) remains legal on a value " - f"(#210, ADR-0028)", - ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, - envelope=origin.source, - ) - ) - continue - if origin.sub_type == ORIGIN_SUBTYPE_PASSTHROUGH: from_ref = origin.attr(ORIGIN_ATTR_FROM) if not isinstance(from_ref, str) or not from_ref: @@ -2118,12 +2078,20 @@ def _validate_origin_paths( if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) continue - # FR-024 §6 — no @via on an aggregate: inference applies only - # when @of targets a non-base entity. (A value host never - # reaches here — the #210 assembly-origin check above already - # rejected it.) + # FR-024 §6 — no @via on an aggregate: inference applies only when + # @of targets a non-base entity from a non-value host. if of_target is None: continue + if is_value_host: + errors.append( + MetaError( + f"{ctx} is missing required attribute '@{ORIGIN_ATTR_VIA}' " + f"(aggregates require a relationship path)", + ErrorCode.ERR_INVALID_ORIGIN, + envelope=src, + ) + ) + continue base = _derive_base_entity( obj, root, host_pkg, node.name, src, errors ) @@ -2228,9 +2196,7 @@ def _resolve_field(name: str, _base: MetaData = base) -> str | None: hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) - elif of_target is not None: - # (A value host never reaches here — the #210 assembly-origin - # check above already rejected origin.first on a value.) + elif of_target is not None and not is_value_host: base = _derive_base_entity(obj, root, host_pkg, node.name, src, errors) if base is not None and not _is_base_relation_target(of_target[0], base, obj): hops = _infer_via_single_hop( @@ -2402,20 +2368,6 @@ def _validate_relationships(root: MetaData, errors: list[MetaError]) -> None: ), )) continue - # A junction is a physical join table — it MUST be an object.entity. - # ADR-0046 lets a value carry navigation-only references, so value-purity - # no longer implicitly guarantees a two-reference junction is an entity; - # assert it here. (A value/projection has no table to join through.) - if junction.sub_type != OBJECT_SUBTYPE_ENTITY: - errors.append(MetaError( - f'relationship "{obj.name}.{rel.name}" ' - f'@{RELATIONSHIP_ATTR_THROUGH} "{through}" resolves to ' - f"{junction.type}.{junction.sub_type}, not an entity — a junction is a " - f"persisted join table and must be object.entity.", - ErrorCode.ERR_INVALID_RELATIONSHIP, - envelope=rel.source, - )) - continue ref_count = _count_junction_references(junction) if ref_count != 2: errors.append(MetaError( @@ -2745,31 +2697,12 @@ def _validate_max_occurs( ) -# FR-024 value purity (ADR-0028): a value object owns NO identity and NO source. -# ADR-0046 admits ONE exception: a navigation-only identity.reference with explicit -# @enforce: false — an outbound pointer to an entity (a DTO/message referencing X by -# id) is not persistence. Its target still resolves (dangling → ERR_INVALID_REFERENCE -# via the registry-derived pass) and codegen emits no FK/DDL. The value's OWN identity -# (primary/secondary) and any enforced reference (a physical FK it has no table to -# hold) stay banned. Mirrors TS subtype-rules.ts validateValuePurity. +# FR-024 value purity (ADR-0028): a value object is a pure data shape — it +# carries NO identity of any subtype and NO source. Mirrors TS subtype-rules.ts +# validateValuePurity (effective children; envelope = the offending child). def _validate_value_purity(node: MetaObject, errors: list[MetaError]) -> None: for child in node.children(): if child.type == TYPE_IDENTITY: - if child.sub_type == IDENTITY_SUBTYPE_REFERENCE: - # ADR-0046: navigation-only reference is the sanctioned exception. - if child.get_meta_attr(IDENTITY_REFERENCE_ATTR_ENFORCE) is False: - continue - errors.append( - MetaError( - f"value object '{node.fqn()}' has an enforced reference " - f"({TYPE_IDENTITY}.{child.sub_type} '{child.name}') — a value is not " - f"persisted and has no table to hold a physical FK; declare a " - f"navigation-only reference with @enforce: false (FR-024, ADR-0028, ADR-0046)", - ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, - envelope=child.source, - ) - ) - continue errors.append( MetaError( f"value object '{node.fqn()}' must not have an identity " @@ -2812,38 +2745,6 @@ def _validate_projection_licensing(node: MetaObject, errors: list[MetaError]) -> ) ) - # A projection's extends is SHAPE lineage, not a shared-storage hierarchy, so a - # CONCRETE projection must declare its own source rather than inherit one. extends - # only ADDS members, so the child's extra fields have no provider in the parent's - # view, and both objects would claim one physical view while declaring different - # exposures (the declared field set IS the exposure, fail-closed). Prior art splits - # the same way: shared-storage inheritance (JPA @Inheritance, EF Core TPH) inherits - # binding AND writability together; shape-reuse inheritance (@MappedSuperclass, - # Django abstract bases) does not inherit the binding at all. - # - # Enforced at the CONCRETE level (mirrors #236) — an abstract base carries shape - # only, and a source on one is inert until a concrete child extends it. Skipped - # when the super is not a legal projection: that trips the rule above and inherits - # its source too, and one defect should yield one error. - _super_is_legal_projection = sup is None or ( - sup.type == TYPE_OBJECT and sup.sub_type == OBJECT_SUBTYPE_PROJECTION - ) - if not node.is_abstract and _super_is_legal_projection: - _own = sum(1 for c in node.own_children() if c.type == TYPE_SOURCE) - _resolved = sum(1 for c in node.children() if c.type == TYPE_SOURCE) - if _resolved > _own: - errors.append( - MetaError( - f"projection '{node.fqn()}' inherits a source through extends " - f"instead of declaring its own — a projection's extends is shape " - f"lineage, not a shared-storage hierarchy. Declare the source on " - f"this projection; an abstract projection base carries shape only " - f"(FR-024, ADR-0028)", - ErrorCode.ERR_PROJECTION_INHERITED_SOURCE, - envelope=node.source, - ) - ) - # ADR-0039 sanctioned own: OWN sources only — an inherited source is validated # on the projection that declares it; an inherited source from a non-projection # super is unreachable without first tripping the extends rule above. Mirrors the @@ -3113,78 +3014,12 @@ def _validate_field_map(root: MetaData, errors: list[MetaError]) -> None: # --------------------------------------------------------------------------- # Four cross-port rules: # R1 — template.prompt requires @payloadRef → ERR_MISSING_REQUIRED_ATTR -# R2 — @payloadRef resolves to a root-level object.value or sourceless -# object.projection (#210) → ERR_INVALID_TEMPLATE +# R2 — @payloadRef resolves to a root-level object.value → ERR_INVALID_TEMPLATE # R3 — @requiredSlots entries are fields on the payload → ERR_INVALID_TEMPLATE # R4 — @format (if set) is in the closed enum set → ERR_BAD_ATTR_VALUE # (handled by AttrSchema.allowed_values already; included for parity). -def _is_legal_payload_target(obj: MetaData) -> bool: - """#210 — a template-level payload target (@payloadRef / @responseRef) is an - object.value OR a SOURCELESS object.projection. "Sourceless" is the #248 - persistability contract: no declared/inherited ``source.*`` child (a concrete - projection cannot inherit one — ERR_PROJECTION_INHERITED_SOURCE — so for a - concrete projection this is simply "no own source"). Mirrors the TS - ``_isLegalPayloadTarget``.""" - if obj.sub_type == OBJECT_SUBTYPE_VALUE: - return True - if obj.sub_type != OBJECT_SUBTYPE_PROJECTION: - return False - # ADR-0039: resolving — a source anywhere in the extends chain binds the - # projection to a backing store, which disqualifies it as a payload shape. - return not any(c.type == TYPE_SOURCE for c in obj.children()) - - -def _check_nested_payload_refs_value_only( - payload: MetaData, - root: MetaData, - errors: list[MetaError], - visited: set[int], -) -> None: - """#210 (carried forward from the #219/ADR-0044 adjudication) — NESTED - payload targets stay value-only: every ``field.object @objectRef`` reachable - from a template-level payload target must resolve to an object.value. The - template-level widen (sourceless projections) deliberately does NOT extend - to nested targets. Dangling refs are NOT reported here — the registry-derived - @objectRef resolution check already owns that failure. Mirrors the TS - ``_checkNestedPayloadRefsValueOnly``. ``visited`` is shared across the WHOLE - pass (all templates), so a bad nested target reachable from two templates - reports ONCE — matching Java's throw-on-first single-error behavior.""" - if id(payload) in visited: - return - visited.add(id(payload)) - # ADR-0039: resolving — a payload shape may inherit fields via extends. - for field in (c for c in payload.children() if c.type == TYPE_FIELD): - if field.sub_type != FIELD_SUBTYPE_OBJECT: - continue - # ADR-0039: resolving — @objectRef may be inherited via extends. - ref = field.get_meta_attr(FIELD_ATTR_OBJECT_REF) - if not isinstance(ref, str) or not ref: - continue - # ADR-0042: a bare ref resolves in the DECLARING owner's package (an - # inherited field resolves in the package that declared it). - owner = getattr(field, "parent", None) or payload - owner_pkg = owner.package or owner.file_default_package or "" - target = resolve_object_ref(root, ref, owner_pkg) - if target is None: - continue # dangling — reported by the @objectRef resolution check - if target.sub_type != OBJECT_SUBTYPE_VALUE: - errors.append(MetaError( - code=ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, - message=( - f"payload '{payload.fqn()}' field '{field.name}' @objectRef " - f"'{ref}' resolves to {TYPE_OBJECT}.{target.sub_type} — a nested " - f"payload target must be an object.value (template-level refs may " - f"also target a sourceless object.projection, nested refs may not) " - f"(#210, ADR-0028, ADR-0044)" - ), - envelope=field.source, - )) - continue - _check_nested_payload_refs_value_only(target, root, errors, visited) - - def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: # ADR-0039: resolving is the default THROUGHOUT this pass. A template CAN # extends (unlike origin.*), so every `tpl.get_meta_attr(TEMPLATE_ATTR_*)` reads @@ -3198,10 +3033,6 @@ def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: # own package first, else a root-level object.value; an FQN resolves exactly. # No bare-name-anywhere fallback that would bind a same-named VO in another # package. Shares the single resolve_object_ref matcher. - # - # #210 — one visited set for the whole pass: a payload shared by N templates - # is walked (and any bad nested target reported) exactly once. - nested_visited: set[int] = set() for tpl in root.children(): if tpl.type != TYPE_TEMPLATE: continue @@ -3215,14 +3046,13 @@ def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: # e.g. @maxTokens (prompt-only) on a template.output, or @promptStyle # (output-only) on a template.prompt. ADR-0039: resolving — a subtype-only # attr may be inherited via extends, so read the effective value. - for attr_name, allowed_subs in _TEMPLATE_SUBTYPE_ONLY_ATTRS.items(): - if tpl.get_meta_attr(attr_name) is not None and tpl.sub_type not in allowed_subs: - valid_on = " / ".join(f"template.{s}" for s in sorted(allowed_subs)) + for attr_name, allowed_sub in _TEMPLATE_SUBTYPE_ONLY_ATTRS.items(): + if tpl.get_meta_attr(attr_name) is not None and tpl.sub_type != allowed_sub: errors.append(MetaError( code=ErrorCode.ERR_INVALID_TEMPLATE, message=( f'template.{tpl.sub_type} "{tpl.name}" carries @{attr_name}, ' - f"which is only valid on {valid_on}" + f"which is only valid on template.{allowed_sub}" ), envelope=tpl.source, )) @@ -3276,26 +3106,21 @@ def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: if not has_payload_ref: continue - # R2 — @payloadRef must resolve to a root-level object.value or - # sourceless object.projection (#210 — a SOURCED projection stays illegal). + # R2 — @payloadRef must resolve to a root-level object.value # FR5d — @payloadRef is a reference; emit format=resolved with # referrer=template FQN, target=the unresolved payloadRef string. payload = resolve_object_ref(root, payload_ref, referrer_pkg) - if payload is None or not _is_legal_payload_target(payload): + if payload is None or payload.sub_type != OBJECT_SUBTYPE_VALUE: errors.append(MetaError( code=ErrorCode.ERR_INVALID_TEMPLATE, message=( f"template '{tpl.name}' @payloadRef '{payload_ref}' " - f"does not resolve to an object.value or sourceless " - f"object.projection at root" + f"does not resolve to an object.value at root" ), envelope=resolved_source(tpl.source, tpl.fqn(), payload_ref), )) continue - # #210 — nested payload targets stay value-only (see the helper's doctrine). - _check_nested_payload_refs_value_only(payload, root, errors, nested_visited) - # R3 — required-slots membership if is_prompt: # ADR-0039: resolving — a template may inherit @requiredSlots via extends. diff --git a/server/python/tests/unit/test_field_enum_intvaluemap.py b/server/python/tests/unit/test_field_enum_intvaluemap.py index 2f2047183..e16e17c56 100644 --- a/server/python/tests/unit/test_field_enum_intvaluemap.py +++ b/server/python/tests/unit/test_field_enum_intvaluemap.py @@ -61,3 +61,60 @@ def test_duplicate_int_value_across_members_is_rejected(): assert len(result.errors) > 0 assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE assert "DRAFT" in result.errors[0].message and "PUBLISHED" in result.errors[0].message + + +# --------------------------------------------------------------------------- +# Regression: @intValueMap validation when @values is INHERITED via extends +# (bug — Rule 4 lived behind an `own_values is None: continue` early-return, +# so a concrete field.enum that inherits @values from an abstract parent but +# owns its own @intValueMap got that map skipped entirely). +# --------------------------------------------------------------------------- + + +def _model_inherited_values(extra: str) -> str: + """An abstract field.enum owning @values, and a concrete field.enum on + Order.status that `extends` it (inheriting @values) while owning its own + @intValueMap directly.""" + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "field.enum": {{ "name": "Status", "abstract": true, "@values": ["DRAFT","PUBLISHED","ARCHIVED"] }} }}, + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "extends": "Status" {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def test_intvaluemap_on_node_with_inherited_values_missing_key_is_rejected(): + # status.status has no own @values (inherited from abstract Status) but + # owns its own @intValueMap missing the "ARCHIVED" key — must still be + # validated against the EFFECTIVE (inherited) @values, not skipped. + result = _load( + _model_inherited_values(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}') + ) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "ARCHIVED" in result.errors[0].message + + +def test_intvaluemap_on_node_with_inherited_values_extra_key_is_rejected(): + result = _load( + _model_inherited_values( + ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}' + ) + ) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "RETRACTED" in result.errors[0].message + + +def test_intvaluemap_on_node_with_inherited_values_valid_map_loads_clean(): + # Positive-case sibling: a valid @intValueMap (keys exactly matching the + # inherited @values, unique ints) must NOT be rejected — proving the fix + # doesn't just make every inherited-@values node fail. + result = _load( + _model_inherited_values( + ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}' + ) + ) + assert result.errors == [] From 66e1be56bbca49be64b615d829c24c7acab55fe6 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 12:43:48 -0400 Subject: [PATCH 16/52] fix(metadata): enforce 32-bit range on field.enum @intValueMap in TS/C#/Python MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Java's IntMapAttribute already rejected an @intValueMap member value outside the 32-bit signed int range, but TS's IntMapAttr, C#'s ValueMatchesType, and Python's _type_ok accepted any integer (int64+ range, since TS numbers and Python ints have no fixed width). The eventual DB column for an int-backed enum is a 32-bit Postgres/SQLite integer (design doc D5, matching field.int's existing mapping) — a value outside [-2147483648, 2147483647] can never actually be persisted, so it should fail at load time on every port, not just Java. Tightens all three to reject out-of-range values via the existing ERR_BAD_ATTR_VALUE code (no new error code), mirroring Java's inclusive bound check exactly. Adds a negative test to each port's @intValueMap test file. --- .../EnumIntValueMapTests.cs | 13 +++++++++++++ .../csharp/MetaObjects/Loader/ValidationPasses.cs | 13 ++++++++++++- .../src/metaobjects/loader/validation_passes.py | 11 ++++++++++- .../tests/unit/test_field_enum_intvaluemap.py | 12 ++++++++++++ .../metadata/src/core/attr/meta-attr-int-map.ts | 15 +++++++++++++++ .../attr-schema-validate-enum-intvaluemap.test.ts | 12 ++++++++++++ 6 files changed, 74 insertions(+), 2 deletions(-) diff --git a/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs b/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs index 9413623a8..84d9a0f93 100644 --- a/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs +++ b/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs @@ -73,4 +73,17 @@ public void Duplicate_int_value_across_members_is_rejected() Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("DRAFT") && e.Message.Contains("PUBLISHED")); } + + // Final-review fix: the eventual DB column for an int-backed enum is a + // 32-bit Postgres/SQLite `integer` (design doc D5) — a value outside that + // range can never actually be persisted, so it must be rejected at load + // time. Mirrors Java's IntMapAttribute#setValueAsString bound check + // (this port's generic type check parses integral JSON numbers as + // `long`, which has no fixed 32-bit width on its own). + [Fact] + public void Value_outside_32bit_range_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9999999999}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE); + } } diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 1056c2eed..c1c43ac05 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -1940,9 +1940,20 @@ ATTR_SUBTYPE_FILTER or // here). Mirrors TS's generic IntMapAttr.validateValue per-member type check — // in this port that generic check lives here rather than in a per-subtype class. // field.enum's own key-set/uniqueness content rules run separately (Pass 10). + // + // Final-review fix: also bound every value to the 32-bit signed int range + // (inclusive) — the eventual DB column for an int-backed enum is a 32-bit + // Postgres/SQLite `integer` (design doc D5), matching Java's + // IntMapAttribute#setValueAsString bound check exactly. A `long` here can + // exceed Int32 range even though it's a whole number. ATTR_SUBTYPE_INT_MAP => value is IReadOnlyDictionary intMap - && intMap.Values.All(v => v is long or int), + && intMap.Values.All(v => v switch + { + int => true, + long l => l >= int.MinValue && l <= int.MaxValue, + _ => false, + }), _ => true, // SUBTYPE_BASE or unknown → accept anything }; diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index eeaf5eb92..2c747436d 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -282,8 +282,17 @@ def _type_ok(value: object, value_type: str) -> bool: # ATTR_SUBTYPE_INT_MAP case. field.enum's own key-set/uniqueness content # rules run separately in _validate_enum_values (Rule 4), which skips # re-reporting a non-integer member already caught here. + # + # Final-review fix: also bound every value to the 32-bit signed int range + # (inclusive) — the eventual DB column for an int-backed enum is a 32-bit + # Postgres/SQLite `integer` (design doc D5), matching Java's + # IntMapAttribute#setValueAsString bound check exactly. Python ints have + # no fixed width, so this is the only place that boundary is enforced. return isinstance(value, dict) and all( - isinstance(v, int) and not isinstance(v, bool) for v in value.values() + isinstance(v, int) + and not isinstance(v, bool) + and -2147483648 <= v <= 2147483647 + for v in value.values() ) # Unknown value types (e.g. "class") — allow anything. return True diff --git a/server/python/tests/unit/test_field_enum_intvaluemap.py b/server/python/tests/unit/test_field_enum_intvaluemap.py index e16e17c56..6f10b4797 100644 --- a/server/python/tests/unit/test_field_enum_intvaluemap.py +++ b/server/python/tests/unit/test_field_enum_intvaluemap.py @@ -63,6 +63,18 @@ def test_duplicate_int_value_across_members_is_rejected(): assert "DRAFT" in result.errors[0].message and "PUBLISHED" in result.errors[0].message +def test_value_outside_32bit_range_is_rejected(): + # Final-review fix: the eventual DB column for an int-backed enum is a + # 32-bit Postgres/SQLite `integer` (design doc D5) — a value outside that + # range can never actually be persisted, so it must be rejected at load + # time. Mirrors Java's IntMapAttribute#setValueAsString bound check + # (Python ints have no fixed width, so nothing else in this port would + # otherwise catch this). + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9999999999}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + + # --------------------------------------------------------------------------- # Regression: @intValueMap validation when @values is INHERITED via extends # (bug — Rule 4 lived behind an `own_values is None: continue` early-return, diff --git a/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts b/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts index 2e24a862b..00b14755b 100644 --- a/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts +++ b/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts @@ -11,6 +11,11 @@ import { DATA_TYPE_OBJECT, type DataType } from "../../data-type.js"; import { registerAttrClass } from "../../attr-class-map.js"; import { ATTR_SUBTYPE_INT_MAP } from "./attr-constants.js"; +// 32-bit signed integer bounds (inclusive) — the eventual DB column for an +// int-backed enum is a 32-bit Postgres/SQLite `integer` (design doc D5). +const INT32_MIN = -2147483648; +const INT32_MAX = 2147483647; + export class IntMapAttr extends MetaAttr { override get dataType(): DataType { return DATA_TYPE_OBJECT; @@ -30,6 +35,16 @@ export class IntMapAttr extends MetaAttr { errors.push({ message: `attribute '@${this.name}' member '${key}' has value '${String(member)}' which is not an integer`, }); + } else if (member < INT32_MIN || member > INT32_MAX) { + // The eventual DB column for an int-backed enum is a 32-bit Postgres/ + // SQLite `integer` (design doc D5; matches field.int's existing + // 32-bit mapping in expected-schema.ts) — mirrors Java's + // IntMapAttribute#setValueAsString bound check exactly (inclusive at + // both ends) so a value that could never be persisted fails at load + // time, not silently, on every port. + errors.push({ + message: `attribute '@${this.name}' member '${key}' has value '${member}' which is outside the 32-bit signed integer range`, + }); } } return errors; diff --git a/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts index 3c3b74f49..975fa7ecc 100644 --- a/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts +++ b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts @@ -65,4 +65,16 @@ describe("field.enum @intValueMap content rules", () => { expect(result.errors[0]?.message).toContain("DRAFT"); expect(result.errors[0]?.message).toContain("PUBLISHED"); }); + + // Final-review fix: the eventual DB column for an int-backed enum is a + // 32-bit Postgres/SQLite `integer` (design doc D5) — a value outside that + // range can never actually be persisted, so it must be rejected at load + // time rather than silently accepted (TS numbers have no fixed width). + // Mirrors Java's IntMapAttribute#setValueAsString bound check. + test("rejects a value outside the 32-bit signed integer range", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9999999999}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("ARCHIVED"); + }); }); From b956fc474291fea9dcc066b6447576e4e9e6f02c Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 24 Jul 2026 12:44:04 -0400 Subject: [PATCH 17/52] test(conformance): add enum-int-backed-inherited-values fixture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No shared fixture exercised "inherited @values via extends, own @intValueMap locally" — exactly the bug class that broke Python's @intValueMap validation (fixed in 5fdb34ca5). TS and C# were independently verified by reading their source to handle it correctly, but nothing gated that behavior going forward. Modeled on enum-abstract-extends: an abstract field.enum declares @values; a concrete field.enum extends it (inheriting @values) while owning its own @intValueMap with a valid key set and unique ints. Verified green against all four loaders (TS conformance.test.ts, C# ConformanceTests, Java ConformanceTest, Python test_conformance.py). --- .../expected.json | 50 +++++++++++++++++++ .../input/meta.enums.json | 24 +++++++++ 2 files changed, 74 insertions(+) create mode 100644 fixtures/conformance/enum-int-backed-inherited-values/expected.json create mode 100644 fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json diff --git a/fixtures/conformance/enum-int-backed-inherited-values/expected.json b/fixtures/conformance/enum-int-backed-inherited-values/expected.json new file mode 100644 index 000000000..70f07de2a --- /dev/null +++ b/fixtures/conformance/enum-int-backed-inherited-values/expected.json @@ -0,0 +1,50 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Status", + "package": "acme", + "abstract": true, + "@values": [ + "DRAFT", + "PUBLISHED", + "ARCHIVED" + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "status", + "extends": "acme::Status", + "@intValueMap": { + "ARCHIVED": 9, + "DRAFT": 0, + "PUBLISHED": 5 + } + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json b/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json new file mode 100644 index 000000000..bc8e2a603 --- /dev/null +++ b/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json @@ -0,0 +1,24 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "extends": "acme::Status", + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} From 67e85ab376d46bd674efdfaf0e03172406d65c38 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Tue, 11 Aug 2026 09:27:35 -0400 Subject: [PATCH 18/52] fix(loader): reject own @intValueMap against a shared enum (#246 int-backed twin) FR-019 materializes a root-level abstract field.enum ONCE per port as a single named type. #246 already forbids a consuming field from re-declaring that shared type's @values. @intValueMap is @values' numeric half -- the symbol->int mapping belongs to the enum VOCABULARY, not to one column that uses it -- but it had no equivalent rule: the #246 check is nested inside the own-@values block, so a field declaring its own @intValueMap while inheriting @values sailed through. Left alone, that gap detonates when the int-backed persistence plans land: - Kotlin emits per-package lookup maps keyed by enum CLASS name (${enumClassName}_TO_INT). Under the shared collapse two consuming fields produce two top-level vals with the same name -- a compile error, and one that fires even when the two maps are identical (the emitter iterates (class, field) pairs with no dedupe). - TS/Java/Python read the map per-field via resolving accessors, so divergent maps "work": one logical type silently gets N storage encodings, with one field possibly int-backed while another stays string-backed. Nothing was broken before this commit -- @intValueMap is still metamodel-only (no codegen, runtime or migrate layer reads it in any port), so the hazard was latent, not live. The rule mirrors #246 exactly: an own @intValueMap whose IMMEDIATE super is an abstract, metadata-root-level field.enum is ERR_ENUM_EXTENDS_VALUES_CONFLICT. The existing error code is reused with attr-specific message text -- the semantic ("a concrete field redefines a shared enum's owned contract") is the same, and a new code would cost four error ledgers plus registry gates for no behavioral gain. Each port now derives "is the super shared?" from ONE predicate (sharedEnumSuper / _shared_enum_super / SharedEnumSuper) used by both halves, immediate-super-only so the validator and codegen's resolveSharedEnumDecl agree on what "shared" means. A non-root abstract super (one nested inside an object) stays legal and keeps a per-field map. Fixture enum-int-backed-inherited-values was blessing the now-illegal shape in all five ports; rewritten so the shared declaration owns @values AND @intValueMap and the consuming field inherits both -- the pattern the design doc intended all along, and what the fixture's name already claimed. Added the negative twin error-enum-extends-intvaluemap-conflict alongside error-enum-extends-values-conflict. Python's three Task-10 regression tests (own map validated against INHERITED @values) used a root-level abstract, so they now trip the new rule; re-pointed at an abstract enum nested inside an abstract object -- the non-root shape upstream already pinned as legal -- which preserves exactly the bug class they were written to guard. Verified: TS metadata 2342/0, conformance corpus 546/0, fixture lint 277 clean, codegen-ts 1074/0, workspace typecheck clean; Python 1681/0; C# 1558/0 (1 pre-existing skip); Java 1326/0; Kotlin 313/0 (JDK 21 -- the Kotlin plugin still can't parse JDK 26, pre-existing). Co-Authored-By: Claude Opus 5 (1M context) --- .../expected.json | 12 +- .../input/meta.enums.json | 17 +- .../expected-errors.json | 15 + .../input/meta.enums.json | 18 + .../MetaObjects/Loader/ValidationPasses.cs | 39 ++- .../metaobjects/loader/ValidationPhase.java | 40 ++- .../metaobjects/loader/validation_passes.py | 325 +++++++++++++++--- .../tests/unit/test_field_enum_intvaluemap.py | 59 +++- .../metadata/src/attr-schema-validate.ts | 48 ++- .../test/enum-extends-values-conflict.test.ts | 118 +++++++ 10 files changed, 613 insertions(+), 78 deletions(-) create mode 100644 fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json create mode 100644 fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json diff --git a/fixtures/conformance/enum-int-backed-inherited-values/expected.json b/fixtures/conformance/enum-int-backed-inherited-values/expected.json index 70f07de2a..8b232a297 100644 --- a/fixtures/conformance/enum-int-backed-inherited-values/expected.json +++ b/fixtures/conformance/enum-int-backed-inherited-values/expected.json @@ -7,6 +7,11 @@ "name": "Status", "package": "acme", "abstract": true, + "@intValueMap": { + "ARCHIVED": 9, + "DRAFT": 0, + "PUBLISHED": 5 + }, "@values": [ "DRAFT", "PUBLISHED", @@ -26,12 +31,7 @@ { "field.enum": { "name": "status", - "extends": "acme::Status", - "@intValueMap": { - "ARCHIVED": 9, - "DRAFT": 0, - "PUBLISHED": 5 - } + "extends": "acme::Status" } }, { diff --git a/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json b/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json index bc8e2a603..24ba90cd8 100644 --- a/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json +++ b/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json @@ -2,19 +2,20 @@ "metadata.root": { "package": "acme", "children": [ - { "field.enum": { "name": "Status", "abstract": true, "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { + "field.enum": { + "name": "Status", + "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, { "object.entity": { "name": "Order", "children": [ { "field.long": { "name": "id" } }, - { - "field.enum": { - "name": "status", - "extends": "acme::Status", - "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } - } - }, + { "field.enum": { "name": "status", "extends": "acme::Status" } }, { "identity.primary": { "name": "id", "@fields": "id" } } ] } diff --git a/fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json new file mode 100644 index 000000000..bfa5c11ee --- /dev/null +++ b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json new file mode 100644 index 000000000..1bb47c7a1 --- /dev/null +++ b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status", "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index c1c43ac05..f2657d936 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -2351,6 +2351,24 @@ public static IReadOnlyList ValidateEnumValues(MetaData root) return errors.AsReadOnly(); } + /// + /// The shared-enum super of , or null when it has none. + /// "Shared" (FR-019 / #246) means the immediate super is abstract AND declared at + /// metadata-root — a metadata.root child, not one nested under an object. Such a + /// declaration is materialized ONCE per port as a single named type, so anything a + /// consuming field re-declares that belongs to the shared TYPE's contract (its + /// @values member set, or the @intValueMap integer backing of that set) is a conflict. + /// Immediate-super-only, matching codegen's Fr019SharedEnum.ResolveSharedEnumDecl so + /// the validator and the shared-enum collapse agree on what "shared" means. + /// + private static MetaData? SharedEnumSuper(MetaField field) + { + var sup = field.SuperData; + return sup is not null && sup.IsAbstract && sup.Parent is { } p && p.Type == TYPE_METADATA + ? sup + : null; + } + private static void WalkEnumValues(MetaData node, List errors) { if (node is MetaField { SubType: FIELD_SUBTYPE_ENUM } field) @@ -2404,11 +2422,10 @@ private static void WalkEnumValues(MetaData node, List errors) // collapse would silently drop this field's own @values in favor of the shared // type's. Own-attrs-only (matches Rules 1-3 above): only fires when THIS node // declares @values itself, not when it merely inherits. - var sup = field.SuperData; - if (sup is not null && sup.IsAbstract && sup.Parent is { } p && p.Type == TYPE_METADATA) + if (SharedEnumSuper(field) is { } sharedSuper) { errors.Add(new MetaError( - $"field.enum '{field.Name}' extends shared abstract enum '{sup.Name}' AND declares its own " + + $"field.enum '{field.Name}' extends shared abstract enum '{sharedSuper.Name}' AND declares its own " + $"@{FIELD_ATTR_VALUES} — a shared enum's member set is owned by the shared declaration; " + $"remove the own @{FIELD_ATTR_VALUES} to inherit it, or extend a non-shared enum instead.", ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, @@ -2453,6 +2470,22 @@ private static void WalkEnumValues(MetaData node, List errors) // before this pass.) if (field.OwnAttr(FIELD_ATTR_INT_VALUE_MAP) is IReadOnlyDictionary intValueMap) { + // #246 (int-backed twin): the symbol→int mapping is a property of the enum + // VOCABULARY, not of one column that uses it — it is @values' numeric half. + // A shared enum is materialized once as a single type, so a per-field map + // would give one logical type N storage encodings (and, where a port emits + // per-TYPE codec artifacts, two same-named declarations). Same remedy as the + // @values half: declare it on the shared declaration and inherit it. + if (SharedEnumSuper(field) is { } sharedIntSuper) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' extends shared abstract enum '{sharedIntSuper.Name}' AND declares its own " + + $"@{FIELD_ATTR_INT_VALUE_MAP} — a shared enum's integer backing is owned by the shared declaration; " + + $"move @{FIELD_ATTR_INT_VALUE_MAP} onto '{sharedIntSuper.Name}' to inherit it, or extend a non-shared enum instead.", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, + Envelope: field.Source)); + } + var effective = field.EffectiveEnumValues ?? new List(); var memberSet = new HashSet(effective, StringComparer.Ordinal); var mapKeys = intValueMap.Keys.ToList(); diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index f0b4b21fd..6ed802062 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -705,8 +705,8 @@ private static void validateEnumNode(MetaData node) { // favor of the shared type's. Own-attrs-only (matches the check above): // only fires when THIS node declares @values itself, not when it merely // inherits. - MetaData sup = node.getSuperData(); - if (sup != null && isAbstract(sup) && sup.getParent() instanceof MetaRoot) { + MetaData sup = sharedEnumSuper(node); + if (sup != null) { throw new MetaDataException( ErrorMessageConstants.ERR_ENUM_EXTENDS_VALUES_CONFLICT + ": field.enum '" + node.getName() @@ -750,11 +750,47 @@ private static void validateEnumNode(MetaData node) { * validates the field.enum-SPECIFIC semantics: key-set-equals-effective-@values, * and no two members share the same int value.

      */ + /** + * The shared-enum super of {@code node}, or null when it has none. + * + *

      "Shared" (FR-019 / #246) means the immediate super is abstract AND declared at + * metadata-root — its parent is the {@link MetaRoot}, not an object. Such a declaration + * is materialized ONCE per port as a single named type, so anything a consuming field + * re-declares that belongs to the shared TYPE's contract (its {@code @values} member + * set, or the {@code @intValueMap} integer backing of that set) is a conflict. + * + *

      Immediate-super-only, matching codegen's {@code Fr019SharedEnum.resolveSharedEnumDecl} + * so the validator and the shared-enum collapse agree on what "shared" means. + */ + private static MetaData sharedEnumSuper(MetaData node) { + MetaData sup = node.getSuperData(); + return (sup != null && isAbstract(sup) && sup.getParent() instanceof MetaRoot) ? sup : null; + } + private static void validateEnumIntValueMap(MetaData node) { if (!node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false)) { return; } + // #246 (int-backed twin): the symbol->int mapping is a property of the enum + // VOCABULARY, not of one column that uses it - it is @values' numeric half. A + // shared enum is materialized once as a single type, so a per-field map would give + // one logical type N storage encodings (and, where a port emits per-TYPE codec + // artifacts, two same-named declarations). Same remedy as the @values half: + // declare it on the shared declaration and inherit it. + MetaData sharedSuper = sharedEnumSuper(node); + if (sharedSuper != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_EXTENDS_VALUES_CONFLICT + + ": field.enum '" + node.getName() + + "' extends shared abstract enum '" + sharedSuper.getName() + + "' AND declares its own @" + EnumField.ATTR_INT_VALUE_MAP + + " - a shared enum's integer backing is owned by the shared declaration;" + + " move @" + EnumField.ATTR_INT_VALUE_MAP + " onto '" + sharedSuper.getName() + + "' to inherit it, or extend a non-shared enum instead", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, node.getSource()); + } + @SuppressWarnings("unchecked") Map intValueMap = (Map) node.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false).getValue(); diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 2c747436d..02b65d64a 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -14,6 +14,7 @@ from ..errors import ErrorCode, MetaError from ..source.error_source import LoaderWarning from .validate_source_physical_names import validate_source_physical_names +from .validate_enum_normalize_ambiguity import validate_enum_normalize_ambiguity from .validate_field_readonly import validate_field_readonly from .validate_discriminator import validate_discriminator from .validate_source_parameter_ref import validate_source_parameter_ref @@ -70,6 +71,7 @@ TYPE_IDENTITY, TYPE_INDEX, TYPE_LAYOUT, + TYPE_METADATA, TYPE_OBJECT, TYPE_ORIGIN, TYPE_RELATIONSHIP, @@ -86,6 +88,7 @@ AGG_ALL, AGG_ANY, AGG_COLLECT, + ASSEMBLY_ORIGIN_SUBTYPES, ORIGIN_ATTR_AGG, ORIGIN_ATTR_CONVERT, ORIGIN_ATTR_DISTINCT, @@ -112,6 +115,7 @@ from ..meta.core.identity.identity_constants import ( IDENTITY_SUBTYPE_REFERENCE, IDENTITY_REFERENCE_ATTR_REFERENCES, + IDENTITY_REFERENCE_ATTR_ENFORCE, ) from ..shared.separators import PACKAGE_SEP from ..meta.core.object.object_constants import ( @@ -125,21 +129,23 @@ from ..source import resolved_source from ..naming_refs import did_you_mean_hint, resolve_object_ref -# A subtype-specific template attr is valid ONLY on the subtype it is registered +# A subtype-specific template attr is valid ONLY on the subtype(s) it is registered # for. The metamodel registers these per-subtype (see the core_types template block), # but the lenient loader does not reject a misplaced one — _validate_templates does. # Mirrors the per-subtype TEMPLATE_ATTRS_MAP split across the other ports. -_TEMPLATE_SUBTYPE_ONLY_ATTRS: dict[str, str] = { - tc.TEMPLATE_ATTR_MAX_TOKENS: tc.TEMPLATE_SUBTYPE_PROMPT, - tc.TEMPLATE_ATTR_REQUIRED_SLOTS: tc.TEMPLATE_SUBTYPE_PROMPT, - tc.TEMPLATE_ATTR_MODEL: tc.TEMPLATE_SUBTYPE_PROMPT, - tc.TEMPLATE_ATTR_RESPONSE_REF: tc.TEMPLATE_SUBTYPE_PROMPT, - tc.TEMPLATE_ATTR_PROMPT_STYLE: tc.TEMPLATE_SUBTYPE_OUTPUT, - tc.TEMPLATE_ATTR_KIND: tc.TEMPLATE_SUBTYPE_OUTPUT, - tc.TEMPLATE_ATTR_SUBJECT_REF: tc.TEMPLATE_SUBTYPE_OUTPUT, - tc.TEMPLATE_ATTR_HTML_BODY_REF: tc.TEMPLATE_SUBTYPE_OUTPUT, - tc.TEMPLATE_ATTR_TEXT_BODY_REF: tc.TEMPLATE_SUBTYPE_OUTPUT, - tc.TEMPLATE_ATTR_TOOL_NAME: tc.TEMPLATE_SUBTYPE_TOOLCALL, +# #237: @maxTokens is registered on BOTH prompt AND toolcall (a vendor-agnostic token +# budget), so the value is a SET of allowed subtypes, not a single one. +_TEMPLATE_SUBTYPE_ONLY_ATTRS: dict[str, frozenset[str]] = { + tc.TEMPLATE_ATTR_MAX_TOKENS: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT, tc.TEMPLATE_SUBTYPE_TOOLCALL}), + tc.TEMPLATE_ATTR_REQUIRED_SLOTS: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT}), + tc.TEMPLATE_ATTR_MODEL: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT}), + tc.TEMPLATE_ATTR_RESPONSE_REF: frozenset({tc.TEMPLATE_SUBTYPE_PROMPT}), + tc.TEMPLATE_ATTR_PROMPT_STYLE: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), + tc.TEMPLATE_ATTR_KIND: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), + tc.TEMPLATE_ATTR_SUBJECT_REF: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), + tc.TEMPLATE_ATTR_HTML_BODY_REF: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), + tc.TEMPLATE_ATTR_TEXT_BODY_REF: frozenset({tc.TEMPLATE_SUBTYPE_OUTPUT}), + tc.TEMPLATE_ATTR_TOOL_NAME: frozenset({tc.TEMPLATE_SUBTYPE_TOOLCALL}), } # --------------------------------------------------------------------------- @@ -193,6 +199,9 @@ def run_validations( validate_source_physical_names(root, errors, envelope_warnings, warnings) # FR-013 — field-level @readOnly cross-attribute rules. validate_field_readonly(root, errors, envelope_warnings, warnings) + # Authoring guard — a field.enum vocabulary ambiguous under the default + # @normalize: strip. WARN_ENUM_NORMALIZE_AMBIGUOUS. + validate_enum_normalize_ambiguity(root, envelope_warnings, warnings) # FR-014 — TPH discriminator cross-attribute rules. validate_discriminator(root, errors) # FR-015 — source.rdb @parameterRef typed-input rules. @@ -458,18 +467,22 @@ def _validate_attr_schema( # --- Check 1: required attrs must be present (uses node.attrs() = effective, # so an inherited attr from the super chain satisfies the requirement) --- - present_attrs = node.attrs() - for schema in schemas: - if not schema.required: - continue - if schema.name not in present_attrs: - errors.append( - MetaError( - f"{_node_label(node)} is missing required attribute '@{schema.name}'", - ErrorCode.ERR_MISSING_REQUIRED_ATTR, - envelope=node.source, + # #236: an ABSTRACT node is a template, not instantiated — it may omit a required + # attr for concrete subtypes / `extends` to supply. Enforcement stays at the + # concrete level (a concrete's resolving attrs() must satisfy it). ADR-0039. + if not node.is_abstract: + present_attrs = node.attrs() + for schema in schemas: + if not schema.required: + continue + if schema.name not in present_attrs: + errors.append( + MetaError( + f"{_node_label(node)} is missing required attribute '@{schema.name}'", + ErrorCode.ERR_MISSING_REQUIRED_ATTR, + envelope=node.source, + ) ) - ) # --- Checks 2 + 3: own attrs only (inherited attrs were already checked on # the node that declared them; re-checking would double-report) --- @@ -553,6 +566,32 @@ def _validate_attr_schema( _ENUM_MEMBER_RE = re.compile(ENUM_MEMBER_PATTERN) +def _shared_enum_super(node: MetaData) -> MetaData | None: + """The shared-enum super of ``node``, or ``None`` when it has none. + + "Shared" (FR-019 / #246) means the immediate super is abstract AND declared at + metadata-root — its parent is the ``metadata.root`` node, not an object. Such a + declaration is materialized ONCE per port as a single named type, so anything a + consuming field re-declares that belongs to the shared TYPE's contract (its + ``@values`` member set, or the ``@intValueMap`` integer backing of that set) is a + conflict. A concrete super, or a non-root abstract super (e.g. one nested inside + an object), is legal and not flagged. + + Immediate-super-only, matching codegen's ``resolve_shared_enum_decl`` + (``codegen/generators/fr019_shared_enum.py``) so the validator and the + shared-enum collapse agree on what "shared" means. + """ + sup = node.super_data + if ( + sup is not None + and sup.is_abstract + and sup.parent is not None + and sup.parent.type == TYPE_METADATA + ): + return sup + return None + + def _validate_enum_values( root: MetaData, errors: list[MetaError], @@ -624,6 +663,23 @@ def _validate_enum_values( ) ) + # #246: own @values AND extends a shared package-level abstract enum — + # one shared enum type has one member set, so the own @values would be + # silently dropped by the shared-enum codegen collapse. Mirrors the TS + # reference (attr-schema-validate.ts). + if _shared_enum_super(node) is not None: + errors.append( + MetaError( + f"{label} declares its own '@{FIELD_ATTR_VALUES}' but extends " + f"a shared package-level abstract enum — one shared enum type " + f"has one member set. Remove the own '@{FIELD_ATTR_VALUES}' to " + f"inherit the shared set, or extend a concrete (non-shared) " + f"enum instead", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, + envelope=node.source, + ) + ) + def _validate_enum_int_value_map(node: MetaData, errors: list[MetaError]) -> None: """``@intValueMap`` content rules (optional), independent of whether ``@values`` @@ -642,6 +698,26 @@ def _validate_enum_int_value_map(node: MetaData, errors: list[MetaError]) -> Non return label = _node_label(node) + + # #246 (int-backed twin): the symbol->int mapping is a property of the enum + # VOCABULARY, not of one column that uses it -- it is @values' numeric half. A + # shared enum is materialized once as a single type, so a per-field map would + # give one logical type N storage encodings (and, where a port emits per-TYPE + # codec artifacts, two same-named declarations). Same remedy as the @values + # half: declare it on the shared declaration and inherit it. + shared_super = _shared_enum_super(node) + if shared_super is not None: + errors.append( + MetaError( + f"{label} declares its own '@{FIELD_ATTR_INT_VALUE_MAP}' but extends " + f"a shared package-level abstract enum — a shared enum's integer " + f"backing is owned by the shared declaration. Move " + f"'@{FIELD_ATTR_INT_VALUE_MAP}' onto the shared declaration to " + f"inherit it, or extend a concrete (non-shared) enum instead", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, + envelope=node.source, + ) + ) effective_values = _effective_enum_values(node) member_set = set(effective_values) key_set = set(int_value_map.keys()) @@ -1925,6 +2001,28 @@ def _validate_origin_paths( else (node.fqn() if hasattr(node, "fqn") else node.name) ) + # #210 — assembly origins live on projections. A value-hosted field + # may not carry origin.aggregate / origin.computed / origin.collection + # / origin.first: a value is constructed — by a caller or by embedding + # — never assembled from a backing store. origin.passthrough STAYS + # legal on a value (FR-015 parameter lineage; the B5 exemption). + if is_value_host and origin.sub_type in ASSEMBLY_ORIGIN_SUBTYPES: + errors.append( + MetaError( + f"value object '{obj.fqn()}' field '{node.name}' hosts " + f"origin.{origin.sub_type} — assembly origins " + f"({', '.join(ASSEMBLY_ORIGIN_SUBTYPES)}) live on " + f"object.projection; a value is constructed by a caller or by " + f"embedding, never assembled from a backing store. Re-host this " + f"field on a sourceless object.projection; origin.passthrough " + f"(FR-015 parameter lineage) remains legal on a value " + f"(#210, ADR-0028)", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, + envelope=origin.source, + ) + ) + continue + if origin.sub_type == ORIGIN_SUBTYPE_PASSTHROUGH: from_ref = origin.attr(ORIGIN_ATTR_FROM) if not isinstance(from_ref, str) or not from_ref: @@ -2087,20 +2185,12 @@ def _validate_origin_paths( if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) continue - # FR-024 §6 — no @via on an aggregate: inference applies only when - # @of targets a non-base entity from a non-value host. + # FR-024 §6 — no @via on an aggregate: inference applies only + # when @of targets a non-base entity. (A value host never + # reaches here — the #210 assembly-origin check above already + # rejected it.) if of_target is None: continue - if is_value_host: - errors.append( - MetaError( - f"{ctx} is missing required attribute '@{ORIGIN_ATTR_VIA}' " - f"(aggregates require a relationship path)", - ErrorCode.ERR_INVALID_ORIGIN, - envelope=src, - ) - ) - continue base = _derive_base_entity( obj, root, host_pkg, node.name, src, errors ) @@ -2205,7 +2295,9 @@ def _resolve_field(name: str, _base: MetaData = base) -> str | None: hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) - elif of_target is not None and not is_value_host: + elif of_target is not None: + # (A value host never reaches here — the #210 assembly-origin + # check above already rejected origin.first on a value.) base = _derive_base_entity(obj, root, host_pkg, node.name, src, errors) if base is not None and not _is_base_relation_target(of_target[0], base, obj): hops = _infer_via_single_hop( @@ -2377,6 +2469,20 @@ def _validate_relationships(root: MetaData, errors: list[MetaError]) -> None: ), )) continue + # A junction is a physical join table — it MUST be an object.entity. + # ADR-0046 lets a value carry navigation-only references, so value-purity + # no longer implicitly guarantees a two-reference junction is an entity; + # assert it here. (A value/projection has no table to join through.) + if junction.sub_type != OBJECT_SUBTYPE_ENTITY: + errors.append(MetaError( + f'relationship "{obj.name}.{rel.name}" ' + f'@{RELATIONSHIP_ATTR_THROUGH} "{through}" resolves to ' + f"{junction.type}.{junction.sub_type}, not an entity — a junction is a " + f"persisted join table and must be object.entity.", + ErrorCode.ERR_INVALID_RELATIONSHIP, + envelope=rel.source, + )) + continue ref_count = _count_junction_references(junction) if ref_count != 2: errors.append(MetaError( @@ -2706,12 +2812,31 @@ def _validate_max_occurs( ) -# FR-024 value purity (ADR-0028): a value object is a pure data shape — it -# carries NO identity of any subtype and NO source. Mirrors TS subtype-rules.ts -# validateValuePurity (effective children; envelope = the offending child). +# FR-024 value purity (ADR-0028): a value object owns NO identity and NO source. +# ADR-0046 admits ONE exception: a navigation-only identity.reference with explicit +# @enforce: false — an outbound pointer to an entity (a DTO/message referencing X by +# id) is not persistence. Its target still resolves (dangling → ERR_INVALID_REFERENCE +# via the registry-derived pass) and codegen emits no FK/DDL. The value's OWN identity +# (primary/secondary) and any enforced reference (a physical FK it has no table to +# hold) stay banned. Mirrors TS subtype-rules.ts validateValuePurity. def _validate_value_purity(node: MetaObject, errors: list[MetaError]) -> None: for child in node.children(): if child.type == TYPE_IDENTITY: + if child.sub_type == IDENTITY_SUBTYPE_REFERENCE: + # ADR-0046: navigation-only reference is the sanctioned exception. + if child.get_meta_attr(IDENTITY_REFERENCE_ATTR_ENFORCE) is False: + continue + errors.append( + MetaError( + f"value object '{node.fqn()}' has an enforced reference " + f"({TYPE_IDENTITY}.{child.sub_type} '{child.name}') — a value is not " + f"persisted and has no table to hold a physical FK; declare a " + f"navigation-only reference with @enforce: false (FR-024, ADR-0028, ADR-0046)", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, + envelope=child.source, + ) + ) + continue errors.append( MetaError( f"value object '{node.fqn()}' must not have an identity " @@ -2754,6 +2879,38 @@ def _validate_projection_licensing(node: MetaObject, errors: list[MetaError]) -> ) ) + # A projection's extends is SHAPE lineage, not a shared-storage hierarchy, so a + # CONCRETE projection must declare its own source rather than inherit one. extends + # only ADDS members, so the child's extra fields have no provider in the parent's + # view, and both objects would claim one physical view while declaring different + # exposures (the declared field set IS the exposure, fail-closed). Prior art splits + # the same way: shared-storage inheritance (JPA @Inheritance, EF Core TPH) inherits + # binding AND writability together; shape-reuse inheritance (@MappedSuperclass, + # Django abstract bases) does not inherit the binding at all. + # + # Enforced at the CONCRETE level (mirrors #236) — an abstract base carries shape + # only, and a source on one is inert until a concrete child extends it. Skipped + # when the super is not a legal projection: that trips the rule above and inherits + # its source too, and one defect should yield one error. + _super_is_legal_projection = sup is None or ( + sup.type == TYPE_OBJECT and sup.sub_type == OBJECT_SUBTYPE_PROJECTION + ) + if not node.is_abstract and _super_is_legal_projection: + _own = sum(1 for c in node.own_children() if c.type == TYPE_SOURCE) + _resolved = sum(1 for c in node.children() if c.type == TYPE_SOURCE) + if _resolved > _own: + errors.append( + MetaError( + f"projection '{node.fqn()}' inherits a source through extends " + f"instead of declaring its own — a projection's extends is shape " + f"lineage, not a shared-storage hierarchy. Declare the source on " + f"this projection; an abstract projection base carries shape only " + f"(FR-024, ADR-0028)", + ErrorCode.ERR_PROJECTION_INHERITED_SOURCE, + envelope=node.source, + ) + ) + # ADR-0039 sanctioned own: OWN sources only — an inherited source is validated # on the projection that declares it; an inherited source from a non-projection # super is unreachable without first tripping the extends rule above. Mirrors the @@ -3023,12 +3180,78 @@ def _validate_field_map(root: MetaData, errors: list[MetaError]) -> None: # --------------------------------------------------------------------------- # Four cross-port rules: # R1 — template.prompt requires @payloadRef → ERR_MISSING_REQUIRED_ATTR -# R2 — @payloadRef resolves to a root-level object.value → ERR_INVALID_TEMPLATE +# R2 — @payloadRef resolves to a root-level object.value or sourceless +# object.projection (#210) → ERR_INVALID_TEMPLATE # R3 — @requiredSlots entries are fields on the payload → ERR_INVALID_TEMPLATE # R4 — @format (if set) is in the closed enum set → ERR_BAD_ATTR_VALUE # (handled by AttrSchema.allowed_values already; included for parity). +def _is_legal_payload_target(obj: MetaData) -> bool: + """#210 — a template-level payload target (@payloadRef / @responseRef) is an + object.value OR a SOURCELESS object.projection. "Sourceless" is the #248 + persistability contract: no declared/inherited ``source.*`` child (a concrete + projection cannot inherit one — ERR_PROJECTION_INHERITED_SOURCE — so for a + concrete projection this is simply "no own source"). Mirrors the TS + ``_isLegalPayloadTarget``.""" + if obj.sub_type == OBJECT_SUBTYPE_VALUE: + return True + if obj.sub_type != OBJECT_SUBTYPE_PROJECTION: + return False + # ADR-0039: resolving — a source anywhere in the extends chain binds the + # projection to a backing store, which disqualifies it as a payload shape. + return not any(c.type == TYPE_SOURCE for c in obj.children()) + + +def _check_nested_payload_refs_value_only( + payload: MetaData, + root: MetaData, + errors: list[MetaError], + visited: set[int], +) -> None: + """#210 (carried forward from the #219/ADR-0044 adjudication) — NESTED + payload targets stay value-only: every ``field.object @objectRef`` reachable + from a template-level payload target must resolve to an object.value. The + template-level widen (sourceless projections) deliberately does NOT extend + to nested targets. Dangling refs are NOT reported here — the registry-derived + @objectRef resolution check already owns that failure. Mirrors the TS + ``_checkNestedPayloadRefsValueOnly``. ``visited`` is shared across the WHOLE + pass (all templates), so a bad nested target reachable from two templates + reports ONCE — matching Java's throw-on-first single-error behavior.""" + if id(payload) in visited: + return + visited.add(id(payload)) + # ADR-0039: resolving — a payload shape may inherit fields via extends. + for field in (c for c in payload.children() if c.type == TYPE_FIELD): + if field.sub_type != FIELD_SUBTYPE_OBJECT: + continue + # ADR-0039: resolving — @objectRef may be inherited via extends. + ref = field.get_meta_attr(FIELD_ATTR_OBJECT_REF) + if not isinstance(ref, str) or not ref: + continue + # ADR-0042: a bare ref resolves in the DECLARING owner's package (an + # inherited field resolves in the package that declared it). + owner = getattr(field, "parent", None) or payload + owner_pkg = owner.package or owner.file_default_package or "" + target = resolve_object_ref(root, ref, owner_pkg) + if target is None: + continue # dangling — reported by the @objectRef resolution check + if target.sub_type != OBJECT_SUBTYPE_VALUE: + errors.append(MetaError( + code=ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, + message=( + f"payload '{payload.fqn()}' field '{field.name}' @objectRef " + f"'{ref}' resolves to {TYPE_OBJECT}.{target.sub_type} — a nested " + f"payload target must be an object.value (template-level refs may " + f"also target a sourceless object.projection, nested refs may not) " + f"(#210, ADR-0028, ADR-0044)" + ), + envelope=field.source, + )) + continue + _check_nested_payload_refs_value_only(target, root, errors, visited) + + def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: # ADR-0039: resolving is the default THROUGHOUT this pass. A template CAN # extends (unlike origin.*), so every `tpl.get_meta_attr(TEMPLATE_ATTR_*)` reads @@ -3042,6 +3265,10 @@ def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: # own package first, else a root-level object.value; an FQN resolves exactly. # No bare-name-anywhere fallback that would bind a same-named VO in another # package. Shares the single resolve_object_ref matcher. + # + # #210 — one visited set for the whole pass: a payload shared by N templates + # is walked (and any bad nested target reported) exactly once. + nested_visited: set[int] = set() for tpl in root.children(): if tpl.type != TYPE_TEMPLATE: continue @@ -3055,13 +3282,14 @@ def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: # e.g. @maxTokens (prompt-only) on a template.output, or @promptStyle # (output-only) on a template.prompt. ADR-0039: resolving — a subtype-only # attr may be inherited via extends, so read the effective value. - for attr_name, allowed_sub in _TEMPLATE_SUBTYPE_ONLY_ATTRS.items(): - if tpl.get_meta_attr(attr_name) is not None and tpl.sub_type != allowed_sub: + for attr_name, allowed_subs in _TEMPLATE_SUBTYPE_ONLY_ATTRS.items(): + if tpl.get_meta_attr(attr_name) is not None and tpl.sub_type not in allowed_subs: + valid_on = " / ".join(f"template.{s}" for s in sorted(allowed_subs)) errors.append(MetaError( code=ErrorCode.ERR_INVALID_TEMPLATE, message=( f'template.{tpl.sub_type} "{tpl.name}" carries @{attr_name}, ' - f"which is only valid on template.{allowed_sub}" + f"which is only valid on {valid_on}" ), envelope=tpl.source, )) @@ -3115,21 +3343,26 @@ def _validate_templates(root: MetaData, errors: list[MetaError]) -> None: if not has_payload_ref: continue - # R2 — @payloadRef must resolve to a root-level object.value + # R2 — @payloadRef must resolve to a root-level object.value or + # sourceless object.projection (#210 — a SOURCED projection stays illegal). # FR5d — @payloadRef is a reference; emit format=resolved with # referrer=template FQN, target=the unresolved payloadRef string. payload = resolve_object_ref(root, payload_ref, referrer_pkg) - if payload is None or payload.sub_type != OBJECT_SUBTYPE_VALUE: + if payload is None or not _is_legal_payload_target(payload): errors.append(MetaError( code=ErrorCode.ERR_INVALID_TEMPLATE, message=( f"template '{tpl.name}' @payloadRef '{payload_ref}' " - f"does not resolve to an object.value at root" + f"does not resolve to an object.value or sourceless " + f"object.projection at root" ), envelope=resolved_source(tpl.source, tpl.fqn(), payload_ref), )) continue + # #210 — nested payload targets stay value-only (see the helper's doctrine). + _check_nested_payload_refs_value_only(payload, root, errors, nested_visited) + # R3 — required-slots membership if is_prompt: # ADR-0039: resolving — a template may inherit @requiredSlots via extends. diff --git a/server/python/tests/unit/test_field_enum_intvaluemap.py b/server/python/tests/unit/test_field_enum_intvaluemap.py index 6f10b4797..61741c2da 100644 --- a/server/python/tests/unit/test_field_enum_intvaluemap.py +++ b/server/python/tests/unit/test_field_enum_intvaluemap.py @@ -86,12 +86,21 @@ def test_value_outside_32bit_range_is_rejected(): def _model_inherited_values(extra: str) -> str: """An abstract field.enum owning @values, and a concrete field.enum on Order.status that `extends` it (inheriting @values) while owning its own - @intValueMap directly.""" + @intValueMap directly. + + The abstract enum is nested inside an abstract object.entity, NOT declared at + metadata-root: a root-level abstract enum is a SHARED enum (FR-019), and #246's + int-backed twin forbids a consuming field from owning an @intValueMap against + one. A non-root abstract super stays legal, so this is the shape that still + exercises "own @intValueMap validated against INHERITED @values". + """ return f"""{{ "metadata.root": {{ "package": "acme", "children": [ - {{ "field.enum": {{ "name": "Status", "abstract": true, "@values": ["DRAFT","PUBLISHED","ARCHIVED"] }} }}, + {{ "object.entity": {{ "name": "Container", "abstract": true, "children": [ + {{ "field.enum": {{ "name": "kind", "abstract": true, "@values": ["DRAFT","PUBLISHED","ARCHIVED"] }} }} + ]}} }}, {{ "object.entity": {{ "name": "Order", "children": [ {{ "field.long": {{ "name": "id" }} }}, - {{ "field.enum": {{ "name": "status", "extends": "Status" {extra} }} }}, + {{ "field.enum": {{ "name": "status", "extends": "acme::Container.kind" {extra} }} }}, {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} ]}} }} ]}} }}""" @@ -130,3 +139,47 @@ def test_intvaluemap_on_node_with_inherited_values_valid_map_loads_clean(): ) ) assert result.errors == [] + + +def _model_shared_enum(extra: str) -> str: + """A ROOT-level abstract field.enum (a SHARED enum per FR-019) owning @values, + and a concrete Order.status that `extends` it.""" + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "field.enum": {{ "name": "Status", "abstract": true, "@values": ["DRAFT","PUBLISHED","ARCHIVED"] }} }}, + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "extends": "acme::Status" {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def test_own_intvaluemap_against_shared_enum_is_rejected(): + # #246 int-backed twin: a shared enum is materialized ONCE as a single type, so + # its integer backing belongs on the shared declaration. A consuming field that + # owns an @intValueMap would give one logical type N storage encodings. + result = _load( + _model_shared_enum( + ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}' + ) + ) + codes = [e.code for e in result.errors] + assert ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT in codes + + +def test_intvaluemap_on_the_shared_declaration_itself_loads_clean(): + # The sanctioned shape: the shared declaration owns BOTH @values and the + # integer backing; the consuming field inherits both and declares neither. + result = _load( + """{ "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT","PUBLISHED","ARCHIVED"], + "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9} } }, + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]} } + ]} }""" + ) + assert result.errors == [] diff --git a/server/typescript/packages/metadata/src/attr-schema-validate.ts b/server/typescript/packages/metadata/src/attr-schema-validate.ts index 3346d750d..728b7045d 100644 --- a/server/typescript/packages/metadata/src/attr-schema-validate.ts +++ b/server/typescript/packages/metadata/src/attr-schema-validate.ts @@ -310,6 +310,19 @@ function validateNode( // only own @values need checking here (mirrors the own-attrs-only policy of // Checks 2+3 above — inherited attrs were validated on the declaring node). if (node.type === TYPE_FIELD && node.subType === FIELD_SUBTYPE_ENUM) { + // #246: the shared-enum super, if any — a root-level abstract field.enum (one + // whose parent is the metadata root, not an object). FR-019 materializes such a + // declaration ONCE per port as a single named type, so anything a consuming + // field re-declares that is part of the shared TYPE's contract is a conflict. + // Immediate-super-only, matching codegen's resolveSharedEnumDecl (enum-shared.ts) + // so the validator and the collapse agree on what "shared" means. + const sharedSuper = + node.superData !== undefined && + node.superData.isAbstract && + node.superData.parent?.type === TYPE_METADATA + ? node.superData + : undefined; + // ADR-0039: own — validates the @values DECLARED on this node; a concrete enum // extending an abstract one inherits already-validated @values (own-attrs-only). const rawValues = node.ownAttrs().get(FIELD_ATTR_VALUES); @@ -348,18 +361,16 @@ function validateNode( } } - // #246: a field.enum extending a shared package-level abstract enum - // (a root-level abstract field — one whose parent is the metadata root, - // not an object) that ALSO declares its own @values is a conflict: one - // shared enum type has one member set, so codegen's shared-enum collapse - // would silently drop this field's own @values in favor of the shared - // type's. Own-attrs-only (matches the rest of Check 4): only fires when - // THIS node declares @values itself, not when it merely inherits. - const sup = node.superData; - if (sup !== undefined && sup.isAbstract && sup.parent?.type === TYPE_METADATA) { + // #246: a field.enum extending a shared package-level abstract enum that + // ALSO declares its own @values is a conflict: one shared enum type has one + // member set, so codegen's shared-enum collapse would silently drop this + // field's own @values in favor of the shared type's. Own-attrs-only + // (matches the rest of Check 4): only fires when THIS node declares + // @values itself, not when it merely inherits. + if (sharedSuper !== undefined) { errors.push( new ParseError( - `${nodeLabel(node)} extends shared abstract enum '${nodeLabel(sup)}' AND declares its own ` + + `${nodeLabel(node)} extends shared abstract enum '${nodeLabel(sharedSuper)}' AND declares its own ` + `'@${FIELD_ATTR_VALUES}' — a shared enum's member set is owned by the shared declaration; ` + `remove the own '@${FIELD_ATTR_VALUES}' to inherit it, or extend a non-shared enum instead.`, { code: "ERR_ENUM_EXTENDS_VALUES_CONFLICT", source: node.source }, @@ -410,6 +421,23 @@ function validateNode( // semantics: key-set-equals-@values, and no two members share a value. const rawIntValueMap = node.ownAttrs().get(FIELD_ATTR_INT_VALUE_MAP); if (rawIntValueMap !== undefined && typeof rawIntValueMap === "object" && rawIntValueMap !== null) { + // #246 (int-backed twin): the symbol→int mapping is a property of the enum + // VOCABULARY, not of one column that uses it — it is @values' numeric half. + // A shared enum is materialized once as a single type, so a per-field map + // would give one logical type N storage encodings (and, where a port emits + // per-TYPE codec artifacts, two same-named declarations). Same remedy as the + // @values half: declare it on the shared declaration and inherit it. + if (sharedSuper !== undefined) { + errors.push( + new ParseError( + `${nodeLabel(node)} extends shared abstract enum '${nodeLabel(sharedSuper)}' AND declares its own ` + + `'@${FIELD_ATTR_INT_VALUE_MAP}' — a shared enum's integer backing is owned by the shared ` + + `declaration; move '@${FIELD_ATTR_INT_VALUE_MAP}' onto '${nodeLabel(sharedSuper)}' to inherit it, ` + + `or extend a non-shared enum instead.`, + { code: "ERR_ENUM_EXTENDS_VALUES_CONFLICT", source: node.source }, + ), + ); + } const map = rawIntValueMap as Record; const effectiveValues = node.attrs().get(FIELD_ATTR_VALUES); const declaredMembers: string[] = Array.isArray(effectiveValues) ? effectiveValues : []; diff --git a/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts b/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts index 40d6623c3..bfbf166a9 100644 --- a/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts +++ b/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts @@ -135,3 +135,121 @@ describe("field.enum — extends a shared abstract enum AND declares own @values expect(conflictErrors).toHaveLength(0); }); }); + +// The int-backed twin of the rule above. @intValueMap is @values' numeric half — +// the symbol→int mapping belongs to the enum VOCABULARY, not to one column that +// uses it. A shared enum is materialized ONCE as a single named type, so a +// per-field map would give one logical type N storage encodings (and, in ports +// that emit per-TYPE codec artifacts, two same-named declarations). +describe("field.enum — extends a shared abstract enum AND declares own @intValueMap", () => { + const sharedDecl = { + "field.enum": { + name: "Status", + abstract: true, + "@values": ["A", "B"], + }, + }; + + function conflicts(errors: unknown[]) { + return errors.filter( + (e) => (e as { code?: string }).code === "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + ); + } + + it("emits ERR_ENUM_EXTENDS_VALUES_CONFLICT when the consuming field owns @intValueMap", async () => { + const { errors } = await load({ + "metadata.root": { + package: "acme", + children: [ + sharedDecl, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { + "field.enum": { + name: "status", + extends: "acme::Status", + "@intValueMap": { A: 0, B: 1 }, + }, + }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + ], + }, + }); + + expect(conflicts(errors)).toHaveLength(1); + }); + + it("does NOT emit the conflict when @intValueMap sits on the shared declaration and the field inherits it", async () => { + const { errors } = await load({ + "metadata.root": { + package: "acme", + children: [ + { + "field.enum": { + name: "Status", + abstract: true, + "@values": ["A", "B"], + "@intValueMap": { A: 0, B: 1 }, + }, + }, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", extends: "acme::Status" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + ], + }, + }); + + expect(conflicts(errors)).toHaveLength(0); + expect(errors).toHaveLength(0); + }); + + it("does NOT emit the conflict for an ABSTRACT but NON-ROOT super (nested inside an object), where a per-field map stays legal", async () => { + const { errors } = await load({ + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "Container", + abstract: true, + children: [ + { "field.enum": { name: "kind", abstract: true, "@values": ["A", "B"] } }, + ], + }, + }, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { + "field.enum": { + name: "status", + extends: "acme::Container.kind", + "@intValueMap": { A: 0, B: 1 }, + }, + }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + ], + }, + }); + + expect(conflicts(errors)).toHaveLength(0); + }); +}); From a7a7102b57a9b43b6ad6173d2c48929bdabe177d Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Tue, 11 Aug 2026 20:03:26 -0400 Subject: [PATCH 19/52] =?UTF-8?q?fix(codegen):=20@provided=20is=20declarat?= =?UTF-8?q?ion-layer=20=E2=80=94=20read=20it=20own-only=20in=20TS/C#/Pytho?= =?UTF-8?q?n?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @provided marks a shared enum declaration as supplied by hand-written / third-party code: the port emits nothing and references the existing type (ADR-0026). TS, C# and Python read it RESOLVING; Java and Kotlin read it own-only and documented that as deliberate. One of them had to be wrong. The JVM side is right. @provided is a provenance fact about the declaration ITSELF -- like `abstract` -- not a property of the values it carries, so it must not flow down an extends chain. All five ports already read it on the resolved DECLARATION and never on the consuming field, so for the ordinary `field extends @provided decl` shape own and resolving agree; the divergence is reachable only through a CHAINED declaration -- a root-level abstract enum `B extends` a root-level abstract `@provided A`. Verified against the real loader: that model loads clean (zero errors), B's own @provided is absent while its resolving read is true. So the resolving ports classify B as provided and emit a reference to a hand-written `B` THE ADOPTER NEVER DECLARED (the marker was authored on A), instead of materializing B from its inherited @values. Python's docstring justified resolving with "a concrete enum extending an abstract @provided enum inherits the flag, so an own-only read would misclassify it" -- wrong about its own call graph, since is_provided() is only ever passed the decl. C#'s comment just cited TS. Neither was a reasoned position. Blast radius is nil on existing gated output: every currently-pinned model shape yields the same answer under both reads, which is exactly why this survived. ADR-0039 amended: its "@dbColumnType is the *only* attribute deliberately read own-only" line was false as written no matter which way this ruled, since the JVM own-reads already existed. @provided is now chartered as the second, with the chained-decl rationale and an explicit note that the member set it accompanies (@values, and its numeric half @intValueMap) stays RESOLVING. No conformance fixture yet -- see the follow-up below. Verified: TS codegen-ts 1074/0 + workspace typecheck clean; Python 1681/0; C# 1558/0 (1 pre-existing skip); Java codegen-spring Fr019 conformance 3/0. FOLLOW-UP (deliberately not in this commit): adding a chained-decl case to fixtures/codegen-conformance/shared-provided-enum -- the corpus all five ports gate -- surfaced a SECOND, deeper divergence that needs a design ruling of its own. Kotlin deliberately names a chained abstract enum after the TOP-MOST root (KotlinTypeMapper.enumTypeName, "a chain of abstract enums still collapses onto one type"), so Kotlin holds that Money IS Currency while every other port holds that Money is its own type. On that input Kotlin materializes a local Currency.kt while ALSO referencing the external com.acme.ext.Currency -- broken under either model. Resolving it means either aligning Kotlin's collapse on the immediate super, or rejecting chained abstract enum declarations in the loader (post-#246 such an alias can carry neither its own @values nor its own @intValueMap, so it adds nothing). Fixture withheld until that is decided rather than pinning one port's accidental behavior. Co-Authored-By: Claude Opus 5 (1M context) --- .../Generators/Fr019SharedEnum.cs | 9 ++++++-- .../codegen/generators/fr019_shared_enum.py | 21 +++++++++++++------ .../packages/codegen-ts/src/enum-shared.ts | 8 ++++++- .../ADR-0039-own-accessor-discipline.md | 13 +++++++++--- 4 files changed, 39 insertions(+), 12 deletions(-) diff --git a/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs b/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs index a08987fff..f320a939c 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs @@ -67,8 +67,13 @@ public static class Fr019SharedEnum return new SharedEnum( Name: CSharpNaming.Pascal(decl.Name), Values: values, - // ADR-0039: resolving — @provided may be inherited via extends (TS reads decl.attr). - Provided: decl.Attr(FIELD_ATTR_PROVIDED) is true, + // ADR-0039 sanctioned own: @provided is a declaration-layer provenance marker + // ("THIS type is supplied by hand-written/third-party code"), like IsAbstract — + // it does not flow down an extends chain. A resolving read misfires on a chained + // declaration (root abstract B extends root abstract @provided A): B would be + // reported provided and emit a reference to a hand-written B the adopter never + // declared, instead of materializing B. Matches the JVM ports. + Provided: decl.OwnAttr(FIELD_ATTR_PROVIDED) is true, Package: PackageOf(decl)); } diff --git a/server/python/src/metaobjects/codegen/generators/fr019_shared_enum.py b/server/python/src/metaobjects/codegen/generators/fr019_shared_enum.py index 4bf284bff..fb1801336 100644 --- a/server/python/src/metaobjects/codegen/generators/fr019_shared_enum.py +++ b/server/python/src/metaobjects/codegen/generators/fr019_shared_enum.py @@ -66,12 +66,21 @@ def resolve_shared_enum_decl(field: MetaField) -> MetaData | None: def is_provided(decl: MetaData) -> bool: - """Effective ``@provided`` truth of an enum declaration. - - ADR-0039 — resolves through ``extends`` (``get_meta_attr``): a concrete enum - extending an abstract ``@provided`` enum inherits the flag, so an own-only read - would misclassify it.""" - return decl.get_meta_attr(fc.FIELD_ATTR_PROVIDED) is True + """``@provided`` truth of an enum DECLARATION. + + ADR-0039 sanctioned own (Python naming inversion: ``attr()`` is the OWN read, + ``get_meta_attr()`` resolves). ``@provided`` is a declaration-layer provenance + marker — "THIS type is supplied by hand-written/third-party code", like + ``is_abstract`` — and does not flow down an ``extends`` chain. + + This is only ever called on the resolved declaration (see + ``shared_enum_for_field``), never on the consuming field, so own and resolving + agree for a plain ``field extends @provided decl``. They diverge on a CHAINED + declaration (root abstract ``B extends`` root abstract ``@provided A``): a + resolving read reports B provided and emits a reference to a hand-written ``B`` + the adopter never declared, instead of materializing B. Matches the JVM ports. + """ + return decl.attr(fc.FIELD_ATTR_PROVIDED) is True def _meta_package_of(decl: MetaData) -> str: diff --git a/server/typescript/packages/codegen-ts/src/enum-shared.ts b/server/typescript/packages/codegen-ts/src/enum-shared.ts index e53114ccd..54f0a46be 100644 --- a/server/typescript/packages/codegen-ts/src/enum-shared.ts +++ b/server/typescript/packages/codegen-ts/src/enum-shared.ts @@ -62,7 +62,13 @@ export function sharedEnumForField(field: MetaField): SharedEnum | undefined { return { name: toPascalCase(decl.name), values, - provided: decl.attr(FIELD_ATTR_PROVIDED) === true, + // ADR-0039 sanctioned own: @provided is a declaration-layer provenance marker + // ("THIS type is supplied by hand-written/third-party code"), like `abstract` — + // it does not flow down an extends chain. A resolving read misfires on a chained + // declaration (root abstract `B extends` root abstract `@provided A`): B would be + // reported provided and emit a reference to a hand-written `B` the adopter never + // declared, instead of materializing B. Matches the JVM ports. + provided: decl.ownAttrs().get(FIELD_ATTR_PROVIDED) === true, }; } diff --git a/spec/decisions/ADR-0039-own-accessor-discipline.md b/spec/decisions/ADR-0039-own-accessor-discipline.md index 0b09c9b7d..df77ec4a4 100644 --- a/spec/decisions/ADR-0039-own-accessor-discipline.md +++ b/spec/decisions/ADR-0039-own-accessor-discipline.md @@ -26,8 +26,15 @@ Two metamodel-internal siblings use the same *"emit only the declared-here layer - **Iterating members for runtime, validation, effective serialization, schema building, or extract** → resolve (`fields()`/`children()`/`attrs()`), because you need the *effective* set including inherited members. - **"Root scans that only work because root is never extended"** (`root.OwnChildren()`) → still resolve. Working-by-accident is the fragile pattern this ADR eliminates. -### The physical exception -`@dbColumnType` is **never inherited** by explicit policy (a physical column-type override is not a logical property). It stays own-only, documented as such at the read site. This is the *only* attribute deliberately read own-only outside the emit-declared-here cases. +### The deliberately-own-only attributes +Two attributes are read own-only by explicit policy, outside the emit-declared-here cases. Each is documented as such at every read site. + +- **`@dbColumnType`** — **never inherited**: a physical column-type override is not a logical property. +- **`@provided`** (FR-019 / [ADR-0026](ADR-0026-shared-and-provided-named-types.md)) — a **declaration-layer provenance marker**, not a property of the values it carries. It asserts "*this* named type is supplied by hand-written / third-party code, so emit nothing and reference it", which is a fact about the declaration itself — like `abstract` — and does not flow down an `extends` chain. + + The distinction is only observable on a **chained declaration**: a root-level abstract enum `B extends` a root-level abstract `@provided` enum `A`. `@provided` is read on the resolved *declaration*, never on the consuming field, so for the ordinary `field extends @provided decl` shape own and resolving agree. On the chained shape a resolving read reports `B` as provided and emits a reference to a hand-written `B` **the adopter never declared** (the marker was authored on `A`), instead of materializing `B` from its inherited `@values`. Own-only matches authored intent. + + Note this is a *provenance* marker and not a value: the member set it accompanies (`@values`, and its numeric half `@intValueMap`) is a logical property and is still read **resolving**, so a declaration inheriting `@values` from its super materializes correctly. ### Naming Where a port's default-named accessor is own-only (Python `attr()` is own; TS `attr()` resolves — an inversion), the port SHOULD make the **resolving** form the default-named one and the own form explicitly `own*`, so "the obvious call" is the correct one. Any `own*()` call MUST carry a one-line comment stating which sanctioned case it is. @@ -37,4 +44,4 @@ Where a port's default-named accessor is own-only (Python `attr()` is own; TS `a - A concrete field/entity that `extends` an abstract parent now correctly inherits its properties and members through codegen, runtime, serialization-effective, schema, and validation — in all five ports. - A **conformance fixture** (abstract field with `isArray`/`maxLength`/`precision`/`default`/`objectRef`/`storage` + a concrete field that `extends` it, plus an entity-level BaseEntity case) gates the class permanently; it fails on pre-fix code. - The rule is propagated to CLAUDE.md and the agent-context authoring/codegen/audit skills; the `metaobjects-audit` skill flags own-accessor value-reads/effective-iteration in codegen/runtime as a defect. -- Each remaining `own*()` call is either the sanctioned emit-declared-here case (commented) or `@dbColumnType` (commented) — any other is a bug. +- Each remaining `own*()` call is either the sanctioned emit-declared-here case (commented) or one of the two deliberately-own-only attributes, `@dbColumnType` / `@provided` (commented) — any other is a bug. From a87cd4ecb4e21c276692a1f339e3606ef1ffff87 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Tue, 11 Aug 2026 20:50:42 -0400 Subject: [PATCH 20/52] fix(codegen-kotlin): name a chained abstract enum for its OWN declaration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Kotlin named a chained abstract enum after the TOP-MOST root of the extends chain (KotlinTypeMapper.enumTypeName via resolveSuperRoot) while its OWN FR-019 arm resolved the shared declaration from the IMMEDIATE super (Fr019SharedEnum.kt:58, `field.superField`) -- the same immediate-super rule TS/C#/Java/Python all use. Not a rival model, a split-brain: the two halves disagreed about which declaration is "the type". On a chained declaration -- root abstract `Money extends` root abstract `@provided Currency` -- that produced a flatly broken emit: the materialize-vs-reference decision saw `Money` (own @provided absent -> materialize) while the NAME collapsed to `Currency`, so Kotlin wrote a local `Currency.kt` that collided with the `com.acme.ext.Currency` reference emitted for fields extending `Currency` directly. Wrong under ANY model, so there was no reading in which the old code was correct. Naming now uses the immediate super, per ADR-0026 §2 (a materialized type is named for its own declaration). A chain yields one type per declaration, each carrying the members it inherits (KotlinEnumEmitter.readEnumValues is inheritance-aware across any number of hops). resolveSuperRoot had exactly one caller and is deleted. Non-chained output is byte-identical: with no further super the root walk already returned the immediate super. The #259 two-hop projection guard is the `declaringObject == null` condition evaluated BEFORE this branch and keys on CONCRETE supers, so it is untouched -- KotlinProjectionTwoHopEnumTest stays green. Why the chained alias stays LEGAL rather than being rejected in the loader: it cannot mutate the vocabulary it inherits. Verified against the real loader -- a chained declaration carrying its own @values errors ERR_ENUM_EXTENDS_VALUES_CONFLICT, and so does its own @intValueMap (60dd3c8fe). #246's Check 4 is gated on any field.enum node, "concrete or abstract", so the decl-level case was already enforced in code. The alias may rename a vocabulary for a bounded context; nothing more. Banning it would carve an enum-only hole in ADR-0029's general `extends` grammar to delete a construct that is provably harmless, and cost four loaders plus error-ledger entries to do it. Gating added: - the chained declaration is restored to fixtures/codegen-conformance/ shared-provided-enum, the corpus ALL FIVE ports load, with explicit assertions in the TS and Kotlin FR-019 conformance tests (Money materialized under its own name with the inherited members; Currency still NOT materialized; the consumer references the local type, not the external one). - fixtures/conformance/enum-abstract-chained-extends (positive) pins that the chain loads clean and canonical-serializes identically cross-port -- legality previously rested on a single ad-hoc run. - fixtures/conformance/error-enum-chained-extends-values-conflict (negative) pins the DECL-level #246 firing, which until now was code-only in all five ports with no fixture behind it. Verified: TS metadata 2354/0, corpus 552/0, codegen-ts 1085/0, fixture lint 280 clean, typecheck clean; Python 1687/0; C# 1564/0 (1 pre-existing skip, conformance 879->885 picking up the new fixtures); Java metadata 1373/0; Kotlin 313/0 with the FR-019 class at 4/0. Co-Authored-By: Claude Opus 5 (1M context) --- .../shared-provided-enum/input/meta.json | 7 +++ .../expected.json | 52 +++++++++++++++++++ .../input/meta.enums.json | 19 +++++++ .../expected-errors.json | 15 ++++++ .../input/meta.enums.json | 9 ++++ .../generator/kotlin/KotlinTypeMapper.kt | 44 ++++++++-------- ...otlinFr019SharedProvidedConformanceTest.kt | 34 ++++++++++++ .../fr019-shared-provided-conformance.test.ts | 18 +++++++ 8 files changed, 175 insertions(+), 23 deletions(-) create mode 100644 fixtures/conformance/enum-abstract-chained-extends/expected.json create mode 100644 fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json create mode 100644 fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json create mode 100644 fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json diff --git a/fixtures/codegen-conformance/shared-provided-enum/input/meta.json b/fixtures/codegen-conformance/shared-provided-enum/input/meta.json index b355526f1..eb87f21f3 100644 --- a/fixtures/codegen-conformance/shared-provided-enum/input/meta.json +++ b/fixtures/codegen-conformance/shared-provided-enum/input/meta.json @@ -4,6 +4,7 @@ "children": [ { "field.enum": { "name": "Priority", "abstract": true, "@values": ["LOW", "MEDIUM", "HIGH"] } }, { "field.enum": { "name": "Currency", "abstract": true, "@provided": true, "@values": ["USD", "EUR", "GBP"] } }, + { "field.enum": { "name": "Money", "abstract": true, "extends": "Currency" } }, { "object.entity": { "name": "Ticket", "children": [ { "field.long": { "name": "id" } }, { "field.enum": { "name": "priority", "extends": "Priority" } }, @@ -16,6 +17,12 @@ { "field.enum": { "name": "priority", "extends": "Priority" } }, { "source.rdb": { "@table": "orders" } }, { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ] } }, + { "object.entity": { "name": "Invoice", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "money", "extends": "Money" } }, + { "source.rdb": { "@table": "invoices" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } ] } } ] } diff --git a/fixtures/conformance/enum-abstract-chained-extends/expected.json b/fixtures/conformance/enum-abstract-chained-extends/expected.json new file mode 100644 index 000000000..4c0f5a5c2 --- /dev/null +++ b/fixtures/conformance/enum-abstract-chained-extends/expected.json @@ -0,0 +1,52 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Currency", + "package": "acme", + "abstract": true, + "@values": [ + "USD", + "EUR" + ] + } + }, + { + "field.enum": { + "name": "Money", + "package": "acme", + "extends": "acme::Currency", + "abstract": true + } + }, + { + "object.entity": { + "name": "Invoice", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "amount", + "extends": "acme::Money" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json b/fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json new file mode 100644 index 000000000..fd5f5f5d1 --- /dev/null +++ b/fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json @@ -0,0 +1,19 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Currency", "abstract": true, "@values": ["USD", "EUR"] } }, + { "field.enum": { "name": "Money", "abstract": true, "extends": "acme::Currency" } }, + { + "object.entity": { + "name": "Invoice", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "amount", "extends": "acme::Money" } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json b/fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json new file mode 100644 index 000000000..2de10d20a --- /dev/null +++ b/fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json b/fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json new file mode 100644 index 000000000..9105a4bdb --- /dev/null +++ b/fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json @@ -0,0 +1,9 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Currency", "abstract": true, "@values": ["USD", "EUR"] } }, + { "field.enum": { "name": "Money", "abstract": true, "extends": "acme::Currency", "@values": ["USD", "EUR", "JPY"] } } + ] + } +} diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt index af7878882..97e275691 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt @@ -196,12 +196,27 @@ object KotlinTypeMapper { // `splitFqn` collapsed to a root-package `Status` — colliding across every entity with a `status`. val immediateSuper = field.superField if (immediateSuper != null && runCatching { immediateSuper.declaringObject }.getOrNull() == null) { - // The DIRECT super is a package-level abstract enum → collapse onto the shared type. Name it - // for the top-most abstract root so a chain of abstract enums (abstract extends abstract) still - // collapses onto one type (resolveSuperRoot walks the abstract chain; == immediateSuper when - // there is no further super). - val superRoot = resolveSuperRoot(field) ?: immediateSuper - val (superPkg, superShort) = PackageMapping.splitFqn(superRoot.name) + // The DIRECT super is a package-level abstract enum → collapse onto the shared type, named + // for THAT declaration — the immediate super, NOT the top-most root of an abstract chain. + // + // Naming by the top-most root (the previous behaviour) contradicted this file's own FR-019 + // arm, which resolves the shared declaration from the IMMEDIATE super + // ([Fr019SharedEnum.resolveSharedEnumDecl]) exactly as TS/C#/Java/Python do. On a CHAINED + // declaration (root abstract `Money extends` root abstract `@provided Currency`) the two + // halves disagreed: the materialize-vs-reference decision saw `Money` (own @provided absent + // → materialize) while the NAME collapsed to `Currency`, so Kotlin emitted a local + // `Currency.kt` that collided with the `com.acme.ext.Currency` reference emitted for fields + // extending `Currency` directly. + // + // Per ADR-0026 §2 a materialized type is named for ITS OWN declaration, so a chain yields one + // type per declaration, each carrying the member set it inherits ([KotlinEnumEmitter + // .readEnumValues] is inheritance-aware across any number of hops). #246 guarantees a chained + // declaration can never MUTATE the vocabulary it inherits — it may rename it, nothing more. + // + // Non-chained output is byte-identical: with no further super the root walk returned the + // immediate super anyway. The #259 two-hop projection guard is the `declaringObject == null` + // condition above, which is evaluated FIRST and is unaffected. + val (superPkg, superShort) = PackageMapping.splitFqn(immediateSuper.name) return ClassName(superPkg, superShort.replaceFirstChar { it.uppercase() }) } @@ -214,23 +229,6 @@ object KotlinTypeMapper { } } - /** - * Walk a field's `extends` (super-field) chain to the top-most ancestor, returning it, or - * `null` when the field has no super. The top-most super is an abstract enum declared at the - * metadata root (e.g. `field.enum Priority @abstract`); naming the generated enum class after - * it makes every extending field share one type. Defensive against cycles via a visited set. - */ - private fun resolveSuperRoot(field: EnumField): MetaField<*>? { - var current: MetaField<*>? = field.superField ?: return null - val seen = HashSet() - while (true) { - val next = current?.superField ?: break - if (!seen.add(current.name)) break // cycle guard - current = next - } - return current - } - /** Map a MetaField to its KotlinPoet data-class property TypeName. */ fun kotlinTypeName(field: MetaField<*>): TypeName = when (field) { // This is the SCALAR/element mapper — `isArray` List<…> wrapping is applied by the diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt index e920c77e8..9d7a36982 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt @@ -79,6 +79,40 @@ class KotlinFr019SharedProvidedConformanceTest { } } + /** + * A CHAINED declaration: root abstract `Money extends` root abstract `@provided Currency`. + * + * Per ADR-0026 §2 a materialized type is named for ITS OWN declaration, and ADR-0039 makes + * `@provided` declaration-layer (it does not flow down the chain), so `Money` is its own type, + * materialized from the members it inherits. #246 guarantees the chain can rename the + * vocabulary but never mutate it. + * + * Kotlin used to name a chained enum after the TOP-MOST root while its FR-019 arm decided + * materialize-vs-reference from the IMMEDIATE super — so it emitted a local `Currency.kt` + * that collided with the `com.acme.ext.Currency` reference asserted above. All five ports + * must agree here. + */ + @Test + fun `chained abstract enum is materialized under its OWN name, not the roots`() { + val out = generate("com.acme.ext") + try { + val money = out.resolve("acme/shop/Money.kt") + assertTrue(Files.exists(money), "chained Money must be materialized under its own name") + val src = money.readText() + for (member in listOf("USD", "EUR", "GBP")) { + assertTrue(member in src, "Money must carry the inherited member $member; saw:\n$src") + } + // The chain must not drag the provided root into materialization. + assertFalse(Files.exists(out.resolve("acme/shop/Currency.kt")), + "provided Currency must still NOT be materialized") + val invoice = out.resolve("acme/shop/Invoice.kt").readText() + assertFalse("com.acme.ext.Money" in invoice, + "Money is materialized locally, not referenced externally; saw:\n$invoice") + } finally { + out.toFile().deleteRecursively() + } + } + @Test fun `provided enum with no namespace config is a codegen error naming the enum`() { val ex = assertThrows { generate(null) } diff --git a/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts b/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts index 2e143f554..0c8290d58 100644 --- a/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts +++ b/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts @@ -89,6 +89,24 @@ describe("FR-019 shared + provided enum conformance (TS)", () => { expect(t).toContain("currency IN ('USD', 'EUR', 'GBP')"); }); + // A CHAINED declaration: root abstract `Money extends` root abstract `@provided Currency`. + // Per ADR-0026 §2 a materialized type is named for ITS OWN declaration, and ADR-0039 makes + // @provided declaration-layer (it does not flow down the chain), so `Money` is its own type, + // materialized from the members it inherits — NOT a reference to a hand-written `Money` the + // adopter never declared. #246 guarantees the chain can rename the vocabulary but never + // mutate it. All five ports must agree here. + test("a root abstract enum extending a @provided enum is materialized under its OWN name", async () => { + const files = await gen(await loadSharedFixture(), "@acme/ext-enums"); + const enums = files["enums.ts"]!; + expect(enums).toContain('export type Money = "USD" | "EUR" | "GBP";'); + // Currency itself stays provided — the chain must not drag it into materialization. + expect(enums).not.toContain("export type Currency ="); + const inv = files["Invoice.ts"]!; + expect(inv).toBeDefined(); + expect(inv).toContain('from "./enums"'); + expect(inv).not.toContain('from "@acme/ext-enums"'); + }); + test("a @provided enum with no providedEnumModule config is a codegen-time error naming the enum", async () => { let err: unknown; try { From 7460758522ef85c19b32fa0f5407f60015d54041 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Wed, 12 Aug 2026 09:03:39 -0400 Subject: [PATCH 21/52] docs(plan): amend the TS int-backed-enum persistence plan for the post-#246 tree Written 2026-07-23 against a pre-#246 / pre-FR-019-hardening tree. Tasks 1-6 survive (every file anchor re-verified against the current tree and all resolve), but four things changed underneath it and three persistence surfaces were never covered. Amendment 1 -- the metamodel moved. #246's int-backed twin now puts @intValueMap on the SHARED declaration with consuming fields inheriting it; @provided became declaration-layer; chained abstract declarations are legal and each materializes under its own name. The load-bearing consequence: every codegen read of @intValueMap MUST resolve through extends. An own-only read sees undefined on every consuming field of a shared enum and silently emits a STRING codec into an INTEGER column -- silent data corruption, not a compile error. Also pins that a per-TYPE codec artifact must be emitted once per declaration, not once per consuming field (the shape that collides in the Kotlin plan; TS's per-field naming is safe, now deliberately rather than accidentally). Amendment 2 -- three verified gaps, added as Tasks 7-9: 7. @default lowering. buildColumn emits DEFAULT 'DRAFT' on what Task 1 makes an integer column (expected-schema.ts:943-949) -- un-appliable DDL. Same defect in column-mapper's Drizzle .default(). 8. The filter path. parseFilterParams coerces by subType and binds the result (filter-parser.ts:156-186, coerce at :215), so ?filter[status][eq]=DRAFT binds 'DRAFT' against an integer column. Fix follows the EXISTING dateValues precedent exactly -- the generated allowlist carries the per-column datum and the parser honours it, keeping the parser metadata-free. 9. TPH per-subtype read schemas. renderTphSubtypeReadSchema parses DB rows, so an integer row value hits a string z.enum and is rejected. Same class of miss as #203/#229, where every TPH per-subtype path needed @autoSet wired separately after the vanilla path had it. The plan called TPH "a follow-up if discovered incomplete" -- it is incomplete. Amendment 3 -- Task 6 edits the SHARED persistence-conformance corpus and reddens the other four ports on landing. Run it last, or hold it for the joint train. Amendment 4 -- array-of-enum needs explicit element-wise codec tests; the enum CHECK is already skipped for arrays, so membership stays app-level as it is today. Also corrects the now-false Global Constraint "do not touch the metamodel layer -- it's already done": #246 falsified that premise before this plan ever ran. Metamodel changes now need justification rather than being forbidden. The other three port plans are deliberately left unamended -- they get rewritten from what this execution actually learns, not from paper analysis, since TS owns the schema layer (ADR-0015). Their known port-specific defects are recorded under "After this plan lands" so nothing is lost. Co-Authored-By: Claude Opus 5 (1M context) --- ...3-int-backed-enum-values-ts-persistence.md | 84 ++++++++++++++++++- 1 file changed, 83 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md index ff9db25d1..da0733f69 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md @@ -12,11 +12,32 @@ - Every TS-facing type (Zod schema output type, generated `EntityType`, the JSON wire payload) is BYTE-IDENTICAL between string-backed and int-backed enums — string in, string out, everywhere except the literal DB column. - `@intValueMap` presence alone is the trigger — no new codegen option, no new CLI flag. -- Do not touch the metamodel/validation layer — that plan already shipped. If a step here tempts you to edit `attr-schema-validate.ts` or add a new attr constant, stop; it's already done. +- Metamodel/validation changes need explicit justification, but are NOT forbidden. (This constraint originally read "do not touch the metamodel/validation layer — it's already done." That premise was falsified by #246 before this plan ever ran; see Amendment 1.) Prefer deriving from what's already registered; adding a new attr still requires the ADR-0023 can't-be-computed justification. - The migration-safety guard requires ZERO new gating code (see Architecture) — this plan's migration-safety task is a test, not an implementation. --- +## Amendments (2026-08-12) + +This plan was written 2026-07-23 against a pre-#246 / pre-FR-019-hardening tree. Everything below post-dates it and MUST be folded in before executing. Tasks 1-6 are otherwise still accurate — their file anchors were re-verified against the current tree and all resolve. + +**Amendment 1 — the metamodel layer moved under this plan's feet.** Three loader/codegen changes landed after this plan was authored: +- **#246 int-backed twin** — a field may NOT declare its own `@intValueMap` when its immediate super is a root-level abstract (SHARED, FR-019) `field.enum`. The map lives on the SHARED DECLARATION and is inherited. +- **`@provided` is declaration-layer** (ADR-0039 amended) — read own-only in all five ports. +- **Chained abstract enum declarations** are legal and each materializes as its own type, named for its own declaration. + +**Consequence for Tasks 4 + 5, and it is the load-bearing one:** every read of `@intValueMap` in codegen MUST be a RESOLVING read (`field.attr(FIELD_ATTR_INT_VALUE_MAP)`), never `ownAttrs()`. Post-#246 the common authoring shape is the map on a shared abstract declaration with N consuming fields inheriting it — an own-only read sees `undefined` on every one of those fields and silently emits a STRING codec into an INTEGER column. That is a silent data-corruption path, not a compile error. ADR-0039 is the governing rule; the two deliberately-own-only attrs (`@dbColumnType`, `@provided`) do NOT include `@intValueMap`. + +**Also required:** the shared-enum path emits ONE materialized type per declaration. A per-TYPE codec artifact (a lookup const/table named for the enum type) must be emitted ONCE per shared declaration, not once per consuming field — the Kotlin plan's `${enumClassName}_TO_INT` shape collides under sharing. TS's per-entity `ENTITY_FIELD_TO_INT` naming is per-field and does NOT collide, so Task 5's naming is safe as written; keep it that way deliberately rather than by accident. + +**Amendment 2 — three persistence surfaces are missing from Tasks 1-6.** Each is a real, verified gap; they are added as Tasks 7-9 below. + +**Amendment 3 — Task 6 breaks the other four ports on landing.** It adds `intEnumVal` to the SHARED `persistence-conformance` corpus, which every port runs. Until each port's codec ships, their round-trip lanes go red. Run Task 6 LAST, and treat the resulting cross-port red as expected and tracked — or hold Task 6 until the C#/Java+Kotlin/Python plans are ready to land in the same train. Per the release ruling, `@intValueMap` must NOT reach a published registry while inert, so the whole program merges as one train anyway. + +**Amendment 4 — array-of-enum (`@isArray`) is under-specified.** D7 says an int-backed array is `integer[]`. Task 1 covers the column type, but Task 5's codec must encode/decode ELEMENT-WISE, and the existing enum `CHECK` is skipped for arrays (`buildChecks` returns early on `field.resolvedIsArray()`), so array membership stays app-level exactly as it is for string-backed arrays. Add an explicit array case to Task 5's tests rather than leaving it implied. + +--- + ### Task 1: migrate-ts — `integer` column type for int-backed enums **Files:** @@ -688,6 +709,67 @@ git commit -m "test(persistence-conformance): int-backed field.enum round-trips --- +### Task 7: migrate-ts + codegen-ts — `@default` must lower to the INT literal + +**Why (verified 2026-08-12):** `buildColumn` reads `@default` and, for a string, emits `col.default = { kind: "literal", value: "DRAFT" }` (`packages/migrate-ts/src/expected-schema.ts:943-949`). On an int-backed enum that produces `DEFAULT 'DRAFT'` on an `integer` column — un-appliable DDL, and permanent false drift on any DB that somehow has it. The enum-member `@default` is already validated as a member of `@values` (Check 5 / FR-011), so the mapping is always available. + +**Files:** +- Modify: `packages/migrate-ts/src/expected-schema.ts` (`buildColumn`) +- Modify: `packages/codegen-ts/src/column-mapper.ts` (Drizzle `.default(...)` emission — same defect, same fix) +- Test: `packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` (extend Task 1's file) + +- [ ] **Step 1: Failing test** — an int-backed enum with `@default: "DRAFT"` and `@intValueMap: {DRAFT: 0, …}` must yield `col.default === { kind: "literal", value: "0" }`, NOT `"DRAFT"`. Add the string-backed control asserting `"DRAFT"` is unchanged. +- [ ] **Step 2:** In `buildColumn`, when the field is an int-backed enum, map the `@default` member symbol through the RESOLVING `@intValueMap` before building the descriptor. An authored default that is not a member is already a load-time error — do not re-validate, but do not silently pass an unmapped value through either; throw a codegen error naming the field if the lookup misses (defensive, unreachable). +- [ ] **Step 3:** Same in `column-mapper.ts` so the Drizzle `.default()` and the DDL agree — a mismatch is exactly the class of drift `meta verify` exists to catch. +- [ ] **Step 4:** Run `bun test packages/migrate-ts packages/codegen-ts`. +- [ ] **Step 5: Commit** — `fix(migrate-ts,codegen-ts): int-backed enum @default lowers to its integer literal` + +--- + +### Task 8: runtime-ts + codegen-ts — the filter path must encode symbol→int + +**Why (verified 2026-08-12):** generated CRUD endpoints filter on `@filterable` fields. `parseFilterParams` coerces by the allowlist rule's `subType` and binds the result (`packages/runtime-ts/src/drizzle-fastify/filter-parser.ts:156-186`, `coerce` at :215). An enum rule coerces as a plain string, so `?filter[status][eq]=DRAFT` binds `'DRAFT'` against an `integer` column — a Postgres type error at request time, and `in` lists likewise. Nothing in Tasks 1-6 touches this. + +**Follow the `dateValues` precedent exactly** (`filter-parser.ts:232-243` + `FilterFieldRule`): codegen already solved this identical problem for Date-typed columns by having the GENERATED allowlist carry a per-column flag the parser honours. Do the same — carry the symbol→int map (or a reference to the generated lookup) on the rule. Do NOT teach the parser to re-derive it from metadata; the parser is metadata-free by design. + +**Files:** +- Modify: `packages/runtime-ts/src/drizzle-fastify/filter-parser.ts` (`FilterFieldRule` + `coerce`) +- Modify: `packages/codegen-ts/src/templates/` — the filter-allowlist emitter +- Test: `packages/runtime-ts/test/` filter-parser unit tests + a codegen allowlist emission test + +- [ ] **Step 1: Failing tests** — (a) parser: a rule carrying an int map coerces `"DRAFT"` → `0` for `eq`/`ne`, and `"DRAFT,PUBLISHED"` → `[0, 5]` for `in`; an unknown member is a `filter.invalid_value` `FilterParseError` naming the field (NOT a silent pass-through, NOT a 500). (b) codegen: the generated `FilterAllowlist` for an int-backed enum field carries the map; a string-backed one is byte-identical to today. +- [ ] **Step 2:** Extend `FilterFieldRule` with the optional map and honour it in `coerce`'s enum path. +- [ ] **Step 3:** Emit it from the allowlist generator, reading `@intValueMap` RESOLVING (Amendment 1). +- [ ] **Step 4:** `isNull` is unaffected (it coerces boolean); `like` must be REJECTED for an int-backed enum — a substring match against an integer column is meaningless. Confirm the existing per-subtype operator gating already excludes `like` for enums; if it does not, that is the fix. +- [ ] **Step 5: Commit** — `fix(runtime-ts,codegen-ts): int-backed enum filters bind the integer, not the member symbol` + +--- + +### Task 9: codegen-ts — TPH per-subtype read schemas must decode + +**Why (verified 2026-08-12):** `renderTphSubtypeReadSchema` (`packages/codegen-ts/src/templates/zod-validators.ts`) parses DB ROWS. For an int-backed enum the row holds an integer, which a string `z.enum([...])` read schema rejects outright. Task 5 wires the vanilla read path; the TPH per-subtype path is a SEPARATE code path — this is the same class of miss as #203/#229, where TPH per-subtype controllers each needed `@autoSet` stamping wired separately after the vanilla path already had it. The original plan hand-waved TPH as "a follow-up if discovered incomplete." It IS incomplete. + +Note `fixtures/conformance/tph-discriminator-enum-with-subtypes` exists: an int-backed enum used AS a TPH discriminator additionally needs its `HasValue`-equivalent literal comparisons encoded. If that proves to need its own design, the acceptable fallback is to REJECT `@intValueMap` on a discriminator field with a clear loader error — but decide it explicitly, do not leave it emitting broken code. + +**Files:** +- Modify: `packages/codegen-ts/src/templates/zod-validators.ts` +- Test: `packages/codegen-ts/test/templates/` TPH read-schema test + +- [ ] **Step 1: Failing test** — a TPH hierarchy whose base carries an int-backed enum: the generated per-subtype read schema accepts the integer row value and yields the member string. +- [ ] **Step 2:** Wire the decode into the TPH read path, reusing Task 5's generated lookup — do not duplicate the codec. +- [ ] **Step 3:** Decide and implement the discriminator case (support, or reject with a named error). +- [ ] **Step 4: Commit** — `fix(codegen-ts): TPH per-subtype read schemas decode int-backed enums` + +--- + ## After this plan lands TS is the reference port; the same shape (DDL/column-type dispatch → runtime codec → persistence-conformance round-trip) repeats in the C#, Java+Kotlin, and Python persistence plans, each adapted to that port's own ORM/codec idiom (EF Core `HasConversion`, OMDB `JdbcFieldCodec`/Exposed `customEnumeration`, Python `ObjectManager` coercion). Only TS needed a genuinely new "wire type differs from storage type" pattern — the other ports' `MetaField`-level runtime access already made a symbol↔int translation point available without inventing new template plumbing. + +**The other three plans are deliberately NOT amended yet** (ruling, 2026-08-12). They are rewritten from what THIS execution actually learns, not from paper analysis — TS owns the schema layer (ADR-0015), so the hard questions (integer DDL, CHECK evolution, filter lowering, migration gating against a real engine) can only be *answered* here. Amending all four up front would bake speculation into three ports that would then be re-amended anyway. + +Known port-specific defects already identified, to fold in during that rewrite: +- **C#** — the array branch emits `ElementType().HasConversion()` unconditionally, ignoring `@intValueMap` (violates D7); and its per-entity `EnumTypeName` naming needs re-checking against FR-019's shared/provided materialization. +- **Java/Kotlin** — Kotlin's per-package `${enumClassName}_TO_INT` support-file emission collides under a shared enum (two consuming fields → two same-named top-level `val`s, even with identical maps: the emitter iterates `(class, field)` pairs with no dedupe). Emit per TYPE, once. A `@provided` Kotlin enum additionally needs its class imported into the support file. Java's `hasMetaAttr(name)` defaults to `includeParentData=true` and DOES resolve through `extends` (verified) — so its codec read is correct by default, but keep it that way deliberately. +- **Python** — the write branch's `int_value_map[value]` raises `KeyError` on a non-member (should be a clean validation error) and `TypeError` on an array-of-enum value (a list is unhashable); D7 array handling is absent entirely. The query/WHERE path is unaddressed (same class as Task 8). +- **All ports** — `@provided` + `@intValueMap` is a REAL adopter case, not an edge case: ADR-0026's motivating example is literally a hand-written enum with int backing. Materialization is suppressed; the codec is NOT. Every port must map by member SYMBOL through the metadata map, never through the provided native type's own underlying integer values (a hand-written `ContactMethod.Email = 3` with `@intValueMap {Email: 1}` must store `1`). C#'s name-keyed dictionary gets this right by construction but has no test pinning it. From 32a22495c2b78a93fd9a149d589bad48910cc9ba Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Wed, 12 Aug 2026 09:07:08 -0400 Subject: [PATCH 22/52] feat(migrate-ts): int-backed field.enum persists as an integer column (Tasks 1, 2, 7) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The DDL half of the TS int-backed-enum persistence plan. An @intValueMap turns a field.enum's physical column from text into integer; every TS-facing and wire type is untouched (that is Tasks 4/5, still to come). Task 1 — column type. subtypeToSqlType gains an explicit FIELD_SUBTYPE_ENUM case (it previously fell through to the `text` default) returning integer{32} when a map is present, and arrayElementSqlType does the same so an int-backed enum[] is integer[] rather than text[] (design D7). Task 2 — membership CHECK. buildChecks emits `IN (0, 5, 9)` unquoted instead of `IN ('DRAFT', …)`. @values stays the SSOT: the integers are read THROUGH the map keyed by member, so a member with no mapping cannot silently disappear from the constraint -- it throws instead. Arrays keep getting NO field-level CHECK, as they already did for string-backed enums (membership stays app-level). Task 7 — @default. buildColumn lowered a string @default straight to a literal, so an int-backed enum emitted `DEFAULT 'DRAFT'` on an integer column: un-appliable DDL, and permanent false drift anywhere it landed. It now lowers through the map. Pinned including the DRAFT->0 case, since a zero-valued member is falsy and is exactly the kind of value a truthiness check would drop (cf. #235). All three read @intValueMap RESOLVING via a new shared `intValueMapOf` helper, and that is the load-bearing detail rather than an incidental one: post-#246 an own @intValueMap against a shared enum is ERR_ENUM_EXTENDS_VALUES_CONFLICT, so the map lives on the SHARED DECLARATION and consuming fields INHERIT it. The inherited case is therefore the canonical authoring shape, not an edge case, and an own-only read would emit a text column for an integer-encoded value on every consuming field of every shared enum -- silent data corruption with no compile error. Two tests pin the inherited shape directly (map inherited; map AND array-ness inherited). Verified: 13 new tests; full migrate-ts suite 736 pass / 0 fail (22 pre-existing skips); workspace typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) --- .../migrate-ts/src/expected-schema.ts | 87 ++++++++- .../expected-schema-enum-intvaluemap.test.ts | 166 ++++++++++++++++++ 2 files changed, 244 insertions(+), 9 deletions(-) create mode 100644 server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts diff --git a/server/typescript/packages/migrate-ts/src/expected-schema.ts b/server/typescript/packages/migrate-ts/src/expected-schema.ts index b79019ac0..f19947845 100644 --- a/server/typescript/packages/migrate-ts/src/expected-schema.ts +++ b/server/typescript/packages/migrate-ts/src/expected-schema.ts @@ -43,6 +43,7 @@ import { FIELD_SUBTYPE_INET, FIELD_SUBTYPE_ENUM, FIELD_ATTR_VALUES, + FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STORAGE, FIELD_ATTR_DB_COLUMN_TYPE, @@ -698,12 +699,35 @@ function buildChecks( if (field.resolvedIsArray()) continue; const col = resolveColumnName(field, strategy); const qcol = quoteCheckCol(col); - // Enum membership check. + // Enum membership check. An INT-BACKED enum (@intValueMap, design D5) stores + // the mapped integers, so the CHECK lists those integers unquoted rather than + // the member strings — `IN (0, 5, 9)`, not `IN ('DRAFT', …)`. The members are + // still the SSOT: the integers are read THROUGH the map, keyed by member, so a + // member with no mapping cannot silently vanish from the constraint. if (field.subType === FIELD_SUBTYPE_ENUM) { const raw = field.attr(FIELD_ATTR_VALUES); if (Array.isArray(raw) && raw.length > 0) { const values = raw.map((v) => String(v)); - const expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; + const intMap = intValueMapOf(field); + let expression: string; + if (intMap !== undefined) { + // The loader pins key-set-equals-@values (Check 5b) in every port, so every + // member resolves. Guard anyway: emitting a partial IN list would silently + // reject rows the model considers valid. + const ints = values.map((v) => { + const n = intMap[v]; + if (typeof n !== "number") { + throw new Error( + `field.enum '${field.name}' @intValueMap has no integer for member '${v}' — ` + + `cannot build the CHECK constraint for column '${col}'.`, + ); + } + return String(n); + }); + expression = `${qcol} IN (${ints.join(", ")})`; + } else { + expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; + } checks.push({ name: `${tableName}_${col}_chk`, expression }); } } @@ -925,12 +949,29 @@ function buildColumn( }; if (typeof defaultRaw === "string") { - // #235: an EMPTY-string default (`@default: ""`) is a real literal default — - // codegen emits `.default("")` and the DB gets `DEFAULT ''`, so dropping it here - // (a falsy `.length > 0` check) made the column drift forever on sqlite/d1 and - // disagree with codegen. Keep it as a literal; only `undefined` means "no default". - const isExpr = defaultRaw.length > 0 && EXPR_DEFAULT_PATTERNS.some((re) => re.test(defaultRaw)); - col.default = { kind: isExpr ? "expr" : "literal", value: defaultRaw }; + // An INT-BACKED enum's @default names a MEMBER SYMBOL, but the column holds the + // mapped integer — emitting `DEFAULT 'DRAFT'` on an `integer` column is + // un-appliable DDL. Lower it through the map. The member is already validated + // against @values by the loader (FR-011 Check 5), so a miss here is unreachable; + // throw rather than silently emit the symbol, which would fail only at apply time. + const enumIntMap = field.subType === FIELD_SUBTYPE_ENUM ? intValueMapOf(field) : undefined; + if (enumIntMap !== undefined) { + const mapped = enumIntMap[defaultRaw]; + if (typeof mapped !== "number") { + throw new Error( + `field.enum '${field.name}' @default '${defaultRaw}' has no entry in @intValueMap — ` + + `cannot lower the column default for '${col.name}'.`, + ); + } + col.default = { kind: "literal", value: String(mapped) }; + } else { + // #235: an EMPTY-string default (`@default: ""`) is a real literal default — + // codegen emits `.default("")` and the DB gets `DEFAULT ''`, so dropping it here + // (a falsy `.length > 0` check) made the column drift forever on sqlite/d1 and + // disagree with codegen. Keep it as a literal; only `undefined` means "no default". + const isExpr = defaultRaw.length > 0 && EXPR_DEFAULT_PATTERNS.some((re) => re.test(defaultRaw)); + col.default = { kind: isExpr ? "expr" : "literal", value: defaultRaw }; + } } else if (typeof defaultRaw === "boolean" || typeof defaultRaw === "number") { col.default = { kind: "literal", value: String(defaultRaw) }; } else { @@ -978,8 +1019,10 @@ function buildColumn( */ function arrayElementSqlType(field: MetaData): SqlType | undefined { switch (field.subType) { + // enum[] stores as text[] — membership is app-level (no CHECK — see buildChecks). + // An INT-BACKED enum[] (@intValueMap, design D7) stores as integer[] instead. + case FIELD_SUBTYPE_ENUM: return isIntBackedEnum(field) ? { kind: "integer", bits: 32 } : { kind: "text" }; case FIELD_SUBTYPE_STRING: - case FIELD_SUBTYPE_ENUM: // enum[] stores as text[]; membership is app-level (no CHECK — see buildChecks) case FIELD_SUBTYPE_URI: return { kind: "text" }; case FIELD_SUBTYPE_UUID: return { kind: "uuid" }; case FIELD_SUBTYPE_INT: return { kind: "integer", bits: 32 }; @@ -1078,7 +1121,33 @@ function subtypeToSqlType(field: MetaData): SqlType { // stores as text (the native inet column would reject a not-strictly-valid // value at INSERT). ADR-0039: resolving — @lenient may be inherited via extends. case FIELD_SUBTYPE_INET: return field.attr(FIELD_ATTR_LENIENT) === true ? { kind: "text" } : { kind: "inet" }; + // A string-backed field.enum is a text column with a membership CHECK; an + // INT-BACKED one (@intValueMap, design D5) stores the mapped integer instead. + // The TS/wire type is the member string either way — only the column differs. + case FIELD_SUBTYPE_ENUM: return isIntBackedEnum(field) ? { kind: "integer", bits: 32 } : { kind: "text" }; default: return { kind: "text" }; // unknown → text fallback } } +/** + * True when this `field.enum` persists as an integer — i.e. it carries an + * `@intValueMap` (design D5). + * + * ADR-0039: RESOLVING (`attr`, not `ownAttr`). Post-#246 a shared (root-level + * abstract) enum OWNS the map and consuming fields inherit it — declaring an own + * `@intValueMap` against a shared super is `ERR_ENUM_EXTENDS_VALUES_CONFLICT`. So + * the inherited case is not an edge case, it is the CANONICAL authoring shape, and + * an own-only read here would emit a `text` column for an integer-encoded value on + * every consuming field of every shared enum. + */ +function isIntBackedEnum(field: MetaData): boolean { + return intValueMapOf(field) !== undefined; +} + +/** The resolved `@intValueMap` as a plain record, or undefined when absent. */ +export function intValueMapOf(field: MetaData): Record | undefined { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return raw as Record; +} + diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts new file mode 100644 index 000000000..1d32c4dee --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts @@ -0,0 +1,166 @@ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../src/expected-schema.js"; + +// Int-backed field.enum (@intValueMap, design D5): the column is `integer`, not +// `text`. The wire/TS type is unchanged — only the physical column differs. + +async function loadJson(json: string): Promise { + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error(`fixture failed to load: ${result.errors.map((e) => String(e)).join("; ")}`); + } + return result.root; +} + +function entityModel(statusField: Record, extraRoots: unknown[] = []): string { + return JSON.stringify({ + "metadata.root": { + children: [ + ...extraRoots, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": statusField }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); +} + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +async function statusColumn(json: string) { + const snapshot = buildExpectedSchema(await loadJson(json)); + const table = snapshot.tables.find((t) => t.name === "orders")!; + return table.columns.find((c) => c.name === "status")!; +} + +describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { + test("scalar int-backed enum maps to integer, not text", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), + ); + expect(col.sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("string-backed enum (no @intValueMap) is unchanged", async () => { + const col = await statusColumn(entityModel({ name: "status", "@values": VALUES })); + expect(col.sqlType).toEqual({ kind: "text" }); + }); + + test("array-of-enum int-backed maps to integer[] (D7)", async () => { + const col = await statusColumn( + entityModel({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), + ); + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); + }); + + test("array-of-enum string-backed stays text[]", async () => { + const col = await statusColumn(entityModel({ name: "status", isArray: true, "@values": VALUES })); + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "text" } }); + }); + + // Amendment 1 / #246: post-#246 the map CANNOT live on the consuming field when + // the field extends a shared (root-level abstract) enum — it lives on the SHARED + // DECLARATION and is inherited. An own-only read here would see undefined and + // silently emit a text column for an integer-encoded value. This is the shape + // real adopters will author, so it is the one that most needs pinning. + test("map inherited from a SHARED abstract declaration still yields integer", async () => { + const col = await statusColumn( + entityModel({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]), + ); + expect(col.sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("the membership CHECK lists the mapped INTEGERS, unquoted", async () => { + const snapshot = buildExpectedSchema( + await loadJson(entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP })), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + const chk = table.checks!.find((c) => c.name === "orders_status_chk")!; + expect(chk.expression).toBe(`"status" IN (0, 5, 9)`); + }); + + test("string-backed enum keeps its quoted-string CHECK", async () => { + const snapshot = buildExpectedSchema( + await loadJson(entityModel({ name: "status", "@values": VALUES })), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + const chk = table.checks!.find((c) => c.name === "orders_status_chk")!; + expect(chk.expression).toBe(`"status" IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')`); + }); + + test("an int-backed enum inheriting its map from a shared decl gets the integer CHECK", async () => { + const snapshot = buildExpectedSchema( + await loadJson( + entityModel({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]), + ), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + const chk = table.checks!.find((c) => c.name === "orders_status_chk")!; + expect(chk.expression).toBe(`"status" IN (0, 5, 9)`); + }); + + test("array-of-enum still gets NO field-level CHECK (membership stays app-level)", async () => { + const snapshot = buildExpectedSchema( + await loadJson( + entityModel({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), + ), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + expect((table.checks ?? []).find((c) => c.name === "orders_status_chk")).toBeUndefined(); + }); + + // Task 7 — @default names a MEMBER, the column holds the mapped INT. Emitting + // DEFAULT 'DRAFT' on an integer column is un-appliable DDL. + test("@default lowers to the mapped integer literal", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP, "@default": "PUBLISHED" }), + ); + expect(col.default).toEqual({ kind: "literal", value: "5" }); + }); + + test("@default on a string-backed enum is unchanged", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@default": "PUBLISHED" }), + ); + expect(col.default).toEqual({ kind: "literal", value: "PUBLISHED" }); + }); + + test("a zero-valued member default survives (0 is falsy — must not be dropped)", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP, "@default": "DRAFT" }), + ); + expect(col.default).toEqual({ kind: "literal", value: "0" }); + }); + + test("array-ness and the map may BOTH be inherited from the shared declaration", async () => { + const col = await statusColumn( + entityModel({ name: "status", extends: "Status" }, [ + { + "field.enum": { + name: "Status", + abstract: true, + isArray: true, + "@values": VALUES, + "@intValueMap": INT_MAP, + }, + }, + ]), + ); + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); + }); +}); From f8843551748ce5a43d5b60bbcb9d8e3f5beb0d56 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 13 Aug 2026 08:27:50 -0400 Subject: [PATCH 23/52] feat(codegen-ts): int-backed enum maps to an integer Drizzle column (Tasks 3, 4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 3 — migration-safety proof, no new code. Toggling @intValueMap on a field that already has a column is a text<->integer change-column-type, and isWidening returns false for ANY cross-kind pair (sql-type.ts:63), so blockedReasonFor already blocks it without allow.typeChange. Proven end-to-end through the real metadata path rather than asserted about hand-built snapshots: adding the map is blocked, REMOVING it is blocked too, an explicit allow.typeChange unblocks the documented manual-recast path, and an unchanged backing emits no change-column-type at all. Task 4 — the Drizzle column mapper follows migrate-ts. An int-backed enum is an `integer` column on postgres AND sqlite (two separate switch arms — the sqlite one was missed on the first pass and caught by its own test), its CHECK lists unquoted integers, and the `{ enum: [...] }` literal-union option is suppressed since that is a text-column affordance that would type a numeric column as a string union. Arrays get a native integer array and, as before, no CHECK. The CHECK expressions here are deliberately the mirror of migrate-ts's buildChecks and are pinned in both packages, because a disagreement between codegen and the expected schema is exactly the drift `meta verify --codegen` exists to report -- it would surface to an adopter as permanent, unfixable drift on a correct model. Both packages now key off `intValueMapOf`, a RESOLVING read (ADR-0039). Post-#246 the map lives on the SHARED DECLARATION and consuming fields inherit it, so the inherited shape is canonical, not exotic; each package pins it directly. Member -> integer lookups go through `intValueForMember`, which throws rather than falling back to the symbol: the loader pins key-set-equals-@values in every port, so a miss is unreachable, and emitting the symbol would defer the failure to INSERT time against a live database. Verified: 7 new codegen-ts tests + 4 new migrate-ts tests; codegen-ts 1092 pass / 0 fail, migrate-ts 762 pass / 0 fail; workspace typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) --- .../packages/codegen-ts/src/column-mapper.ts | 36 +++++-- .../packages/codegen-ts/src/enum-meta.ts | 38 +++++++- .../column-mapper-enum-intvaluemap.test.ts | 96 +++++++++++++++++++ .../expected-schema-enum-intvaluemap.test.ts | 46 +++++++++ 4 files changed, 207 insertions(+), 9 deletions(-) create mode 100644 server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts diff --git a/server/typescript/packages/codegen-ts/src/column-mapper.ts b/server/typescript/packages/codegen-ts/src/column-mapper.ts index 87f2854f4..3ada6e379 100644 --- a/server/typescript/packages/codegen-ts/src/column-mapper.ts +++ b/server/typescript/packages/codegen-ts/src/column-mapper.ts @@ -46,7 +46,7 @@ import { AGG_COLLECT, } from "@metaobjectsdev/metadata"; import { columnNameFromField } from "./naming.js"; -import { enumValues } from "./enum-meta.js"; +import { enumValues, intValueMapOf, intValueForMember } from "./enum-meta.js"; import { DEFAULT_COLUMN_NAMING_STRATEGY, stripPackage } from "@metaobjectsdev/metadata"; import type { Dialect, ColumnNamingStrategy } from "./metaobjects-config.js"; @@ -405,8 +405,12 @@ export function mapColumnType( // "string" by the time it reaches here for this dialect. fnName = "text"; break; - case FIELD_SUBTYPE_STRING: case FIELD_SUBTYPE_ENUM: + // An INT-BACKED enum stores the mapped integer on SQLite too — SQLite has + // one integer storage class, so this matches migrate-ts's integer{32}. + fnName = intValueMapOf(field) !== undefined ? "integer" : "text"; + break; + case FIELD_SUBTYPE_STRING: case FIELD_SUBTYPE_UUID: case FIELD_SUBTYPE_URI: case FIELD_SUBTYPE_INET: @@ -524,6 +528,12 @@ export function mapColumnType( fnName = "jsonb"; break; case FIELD_SUBTYPE_ENUM: + // An INT-BACKED enum (@intValueMap, design D5/D7) stores the mapped + // integer, so the Drizzle column is integer / integer[] — matching + // migrate-ts's expected-schema. The TS-facing type stays the member-string + // union; the symbol<->int translation happens at the write/read boundary. + fnName = intValueMapOf(field) !== undefined ? "integer" : "text"; + break; default: fnName = "text"; break; @@ -676,12 +686,22 @@ export function mapColumnType( if (subType === FIELD_SUBTYPE_ENUM && !isArray) { const values = enumValues(field); if (values !== undefined && values.length > 0) { - // Single-quote escaping is belt-and-suspenders: the loader's - // ENUM_MEMBER_PATTERN already rejects quote-bearing members (members are - // validated to be identifier-safe), so this never fires in practice. - const list = values - .map((v) => `'${v.replace(/'/g, "''")}'`) - .join(", "); + const intMap = intValueMapOf(field); + let list: string; + if (intMap !== undefined) { + // Int-backed: the column holds integers, so the CHECK lists them unquoted. + // Keyed BY MEMBER through the map (not Object.values) so the constraint can + // never disagree with @values, which stays the SSOT. Must match + // migrate-ts's buildChecks exactly or `meta verify` reports permanent drift. + list = values + .map((v) => String(intValueForMember(intMap, v, `CHECK for column '${dbName}'`))) + .join(", "); + } else { + // Single-quote escaping is belt-and-suspenders: the loader's + // ENUM_MEMBER_PATTERN already rejects quote-bearing members (members are + // validated to be identifier-safe), so this never fires in practice. + list = values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", "); + } result.checkConstraint = `${dbName} IN (${list})`; } } diff --git a/server/typescript/packages/codegen-ts/src/enum-meta.ts b/server/typescript/packages/codegen-ts/src/enum-meta.ts index 256c72129..3233a3012 100644 --- a/server/typescript/packages/codegen-ts/src/enum-meta.ts +++ b/server/typescript/packages/codegen-ts/src/enum-meta.ts @@ -3,7 +3,7 @@ // extraction and the z.enum([...]) expression are derived in exactly one place. import type { MetaField } from "@metaobjectsdev/metadata"; -import { FIELD_ATTR_VALUES } from "@metaobjectsdev/metadata"; +import { FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP } from "@metaobjectsdev/metadata"; /** * Effective enum member values (`@values`) for a field, as strings. @@ -20,6 +20,42 @@ export function enumValues(field: MetaField): string[] | undefined { return values.map((v) => String(v)); } +/** + * The effective `@intValueMap` (member symbol → integer) for an int-backed enum, + * or undefined when the enum is string-backed. Its PRESENCE is the whole trigger + * for integer persistence (design D5) — there is no separate flag or config. + * + * ADR-0039: RESOLVING (`attr`, not `ownAttr`), and this is load-bearing rather + * than incidental. Post-#246 an own `@intValueMap` declared against a shared + * (root-level abstract) enum is `ERR_ENUM_EXTENDS_VALUES_CONFLICT`, so the map + * lives on the SHARED DECLARATION and every consuming field INHERITS it. An + * own-only read would therefore see undefined on exactly the shape adopters are + * steered toward, and silently emit a string codec into an integer column. + */ +export function intValueMapOf(field: MetaField): Record | undefined { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return raw as Record; +} + +/** + * The integer a member symbol persists as, for an int-backed enum. Throws when the + * member has no mapping — the loader pins key-set-equals-`@values` (Check 5b) in + * every port, so a miss is unreachable and must not be papered over: emitting the + * symbol instead would fail only at INSERT time, against a live database. + */ +export function intValueForMember( + intMap: Record, + member: string, + context: string, +): number { + const n = intMap[member]; + if (typeof n !== "number") { + throw new Error(`@intValueMap has no integer for member '${member}' (${context}).`); + } + return n; +} + /** Build the Zod expression for a set of enum members, e.g. `z.enum(["A", "B"])`. */ export function zodEnumExpr(values: string[]): string { return `z.enum([${values.map((v) => JSON.stringify(v)).join(", ")}])`; diff --git a/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts new file mode 100644 index 000000000..5bff3951c --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts @@ -0,0 +1,96 @@ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import type { MetaField } from "@metaobjectsdev/metadata"; +import { mapColumnType } from "../src/column-mapper.js"; + +// Int-backed field.enum (@intValueMap, design D5/D7): the Drizzle column becomes +// integer / integer[] and its CHECK lists unquoted integers, matching migrate-ts's +// expected-schema exactly. A mismatch between the two is precisely the drift +// `meta verify --codegen` exists to catch, so these assertions are deliberately +// the mirror of expected-schema-enum-intvaluemap.test.ts in migrate-ts. + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +async function statusField( + statusDecl: Record, + extraRoots: unknown[] = [], +): Promise { + const json = JSON.stringify({ + "metadata.root": { + children: [ + ...extraRoots, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": statusDecl }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (errors.length > 0) throw new Error(`fixture failed to load: ${errors.map(String).join("; ")}`); + const order = root.objects().find((o) => o.name === "Order")!; + return order.fields().find((f) => f.name === "status")!; +} + +describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { + test("scalar int-backed enum → integer column, no literal-union enum option", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "postgres"); + expect(spec.fnName).toBe("integer"); + // The `{ enum: [...] }` literal-union option is a TEXT-column affordance; on an + // integer column it would type the column as a string union over a numeric value. + expect(spec.fnOptions?.enum).toBeUndefined(); + }); + + test("string-backed enum is unchanged — text + literal union", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); + expect(spec.fnName).toBe("text"); + expect(spec.fnOptions?.enum).toEqual(VALUES); + }); + + test("CHECK lists unquoted integers, matching migrate-ts", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "postgres"); + expect(spec.checkConstraint).toBe("status IN (0, 5, 9)"); + }); + + test("string-backed CHECK keeps quoted members", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); + expect(spec.checkConstraint).toBe("status IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')"); + }); + + test("sqlite int-backed enum is integer too", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "sqlite"); + expect(spec.fnName).toBe("integer"); + }); + + // Amendment 1 / #246 — the canonical authoring shape. An own-only read of + // @intValueMap would emit `text` here and silently disagree with migrate-ts, + // which reads it resolving. + test("map INHERITED from a shared abstract declaration still yields integer", async () => { + const spec = mapColumnType( + await statusField({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]), + "postgres", + ); + expect(spec.fnName).toBe("integer"); + expect(spec.checkConstraint).toBe("status IN (0, 5, 9)"); + }); + + test("array-of-enum int-backed gets a native integer array, no CHECK", async () => { + const spec = mapColumnType( + await statusField({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), + "postgres", + ); + expect(spec.fnName).toBe("integer"); + // Membership on arrays stays app-level, exactly as for string-backed enum[]. + expect(spec.checkConstraint).toBeUndefined(); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts index 1d32c4dee..bf6b10733 100644 --- a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts +++ b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts @@ -2,6 +2,7 @@ import { describe, test, expect } from "bun:test"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; import type { MetaData } from "@metaobjectsdev/metadata"; import { buildExpectedSchema } from "../src/expected-schema.js"; +import { diff } from "../src/diff/index.js"; // Int-backed field.enum (@intValueMap, design D5): the column is `integer`, not // `text`. The wire/TS type is unchanged — only the physical column differs. @@ -164,3 +165,48 @@ describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); }); }); + +// Task 3 — the D8 migration-safety guard needs NO new gating code. Toggling +// @intValueMap on a field that already has a column is a text<->integer +// change-column-type, and `isWidening` returns false for ANY cross-kind pair +// (sql-type.ts: `if (from.kind !== to.kind) return false`), so +// `blockedReasonFor`'s change-column-type branch blocks it unless allow.typeChange +// is passed. This proves that end-to-end through the real metadata path rather +// than asserting it about hand-built snapshots. +describe("int-backed enum — backing-mode change is blocked by the existing guard", () => { + const STRING_BACKED = entityModel({ name: "status", "@values": VALUES }); + const INT_BACKED = entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + + async function backingModeDiff(expected: string, actual: string, allowTypeChange = false) { + const e = buildExpectedSchema(await loadJson(expected)); + const a = buildExpectedSchema(await loadJson(actual)); + return diff(e, a, allowTypeChange ? { allow: { typeChange: true } } : undefined); + } + + test("ADDING @intValueMap (text → integer) is blocked without allow.typeChange", async () => { + const r = await backingModeDiff(INT_BACKED, STRING_BACKED); + const change = r.changes.find((c) => c.kind === "change-column-type"); + expect(change).toBeDefined(); + expect(change!.status.state).toBe("blocked"); + expect(r.blocked).toContain(change!); + }); + + test("REMOVING @intValueMap (integer → text) is blocked too", async () => { + const r = await backingModeDiff(STRING_BACKED, INT_BACKED); + const change = r.changes.find((c) => c.kind === "change-column-type"); + expect(change).toBeDefined(); + expect(change!.status.state).toBe("blocked"); + }); + + test("an explicit allow.typeChange unblocks it (the documented manual-recast path)", async () => { + const r = await backingModeDiff(INT_BACKED, STRING_BACKED, true); + const change = r.changes.find((c) => c.kind === "change-column-type"); + expect(change!.status.state).toBe("allowed"); + expect(r.blocked).toHaveLength(0); + }); + + test("no backing-mode change → no change-column-type at all", async () => { + const r = await backingModeDiff(INT_BACKED, INT_BACKED); + expect(r.changes.find((c) => c.kind === "change-column-type")).toBeUndefined(); + }); +}); From 17b342c37ae5b6bef7a83c92cd6a39621385964c Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 13 Aug 2026 18:40:48 -0400 Subject: [PATCH 24/52] docs(design): record the downstream-consumer provenance for int-backed enums The requirement is a live downstream-consumer need (modelling an existing integer-coded schema), but it was recorded nowhere in this repo's issues or roadmap. On review that absence made the whole program look speculative and nearly got it de-scoped -- the demand was real, just invisible from inside the repo. Recorded genericized, per the public-repo hygiene rule. Also notes, for whoever reads this next: if an adopter's reason is purely storage SIZE rather than matching an encoding they don't control, native Postgres enum is the better instrument (also 4 bytes, keeps string semantics, needs no codec in any port). Int-backing's unique value is matching a foreign encoding. Co-Authored-By: Claude Opus 5 (1M context) --- .../specs/2026-07-23-int-backed-enum-values-design.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md index 6b1e9033e..780f696ae 100644 --- a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md +++ b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md @@ -17,6 +17,16 @@ metadata-driven path today; they would have to hand-roll a converter outside the generated code, defeating the "declare once → idiomatic type + DB constraint in every language" payoff that is `field.enum`'s whole reason to exist. +**Provenance (recorded 2026-08-13).** This is a **live requirement from a downstream +consumer**, not a speculative feature — the adopter needs int-backed enum maps to model +an existing integer-coded schema. Recorded here explicitly because the requirement was +absent from this repo's issues and roadmap, which made the work look demand-less on +review and nearly got it de-scoped. If the driving need is purely *storage size* rather +than matching an existing integer encoding, note that native Postgres `CREATE TYPE … +AS ENUM` is the better instrument (also 4 bytes, keeps string semantics, needs no codec +in any port) — it is deferred for PG/SQLite parity reasons, see the enum design's D-list. +Int-backing's unique value is matching an encoding you do not control. + ## Goals 1. Let a `field.enum` declare an explicit, possibly-sparse, per-member integer value for From eca1f77504e07ad6198258a374bc3435203eb80d Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 13 Aug 2026 18:54:33 -0400 Subject: [PATCH 25/52] feat(codegen-ts): int-backed enum codec as a Drizzle customType (Task 5) The symbol<->int translation now lives in the COLUMN DEFINITION, as a generated `customType` with toDriver/fromDriver, emitted ahead of the table: const STATUS_TO_INT = { "DRAFT": 0, ... } as const satisfies Record<..., number>; const STATUS_FROM_INT: Record = { 0: "DRAFT", ... }; const statusIntEnum = customType<{ data: "DRAFT" | ...; driverData: number }>({ dataType: () => "integer", toDriver: (value) => STATUS_TO_INT[value], fromDriver: (value) => { ...throw on unmapped... }, }); This REPLACES the plan's Zod-write-transform-plus-generated-read-decode design. Tracing Task 5 turned up an asymmetry the plan had not seen: the vanilla read path returns raw Drizzle rows verbatim (`return row ?? null` / `return row!` at queries-file.ts:168/:237/:249) and has NO decode seam at all, so that route meant inventing one and wrapping every generated read function of every entity. The TPH read path, by contrast, already parses through a schema (:311/:319/:342) -- so the task flagged as broken (Task 9) is the one that already had the seam. Binding through the column type instead means nothing downstream changes: db.insert().values() encodes on bind, a selected row decodes on read, and a filter comparison encodes for free -- which collapses most of Task 8 (the filter-parser and generated allowlist need no int map threaded through them). It is also the more PORTABLE choice, which is the opposite of how it first looked. TS appeared uniquely expensive only because its generated queries hand back raw rows; every other port already has a MetaField-level codec seam -- EF Core HasConversion, OMDB JdbcFieldCodec, Exposed customEnumeration, Python ObjectManager coercion. customType IS the TS analogue, so all five ports land on the same design instead of TS carrying a bespoke one. Details worth keeping: fromDriver THROWS on an integer outside the map rather than returning undefined (a value the model says is impossible means hand-written data or a member removed without a migration; yielding undefined for a non-nullable field would surface far from the cause). The maps are keyed by member in @values order, so @values stays the SSOT. Helper consts are named from the field and are per-file, so a shared enum consumed by N entities emits N small identical helpers rather than forcing a cross-module import -- the same self-contained tradeoff the per-entity enum union already makes. Emission is sorted by const name so output is deterministic regardless of field order, and a string-backed enum emits nothing new (byte-identical output, pinned). Verified: 6 new emission tests + 8 column-mapper tests; codegen-ts 1099 pass / 0 fail; workspace build + typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) --- .../packages/codegen-ts/src/column-mapper.ts | 98 ++++++++++++++++++- .../src/templates/drizzle-schema.ts | 76 ++++++++++++-- .../column-mapper-enum-intvaluemap.test.ts | 29 ++++-- .../test/drizzle-enum-intvaluemap.test.ts | 93 ++++++++++++++++++ 4 files changed, 282 insertions(+), 14 deletions(-) create mode 100644 server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts diff --git a/server/typescript/packages/codegen-ts/src/column-mapper.ts b/server/typescript/packages/codegen-ts/src/column-mapper.ts index 3ada6e379..887a845f6 100644 --- a/server/typescript/packages/codegen-ts/src/column-mapper.ts +++ b/server/typescript/packages/codegen-ts/src/column-mapper.ts @@ -212,9 +212,42 @@ function canonicalizeSqlExpr(value: string): string { return value; // unrecognized — pass through (function calls etc.) } +/** + * An int-backed `field.enum` column: a generated Drizzle `customType` whose + * `toDriver`/`fromDriver` translate member symbol <-> stored integer, so the + * codec lives in the COLUMN definition rather than in the query layer. + * + * This is the TS analogue of what every other port already does at its own + * `MetaField` codec seam (EF Core `HasConversion`, OMDB `JdbcFieldCodec`, Exposed + * `customEnumeration`, Python `ObjectManager` coercion) — which is why it was + * chosen over a Zod write-transform plus a bespoke read-decode: TS's generated + * queries hand back raw Drizzle rows and have no decode seam at all, so a + * query-layer codec would have meant inventing one and wrapping every generated + * read. Binding through the column type also makes filter values encode for free. + */ +export interface EnumIntCustomType { + /** Local const name for the customType column helper, e.g. `orderStatusEnumCol`. */ + fnConstName: string; + /** Local const name for the symbol->int map, e.g. `ORDER_STATUS_TO_INT`. */ + toIntConstName: string; + /** Local const name for the int->symbol map, e.g. `ORDER_STATUS_FROM_INT`. */ + fromIntConstName: string; + /** Physical column type for `dataType()` — always integer for an int-backed enum. */ + dataType: string; + /** Member symbols, in `@values` order (the TS union and the map key order). */ + members: string[]; + /** Member symbol -> stored integer. */ + intByMember: Record; +} + export interface ColumnSpec { /** Drizzle function name, e.g., "text", "integer", "varchar". */ fnName: string; + /** + * When set, `fnName` names a LOCAL generated const (this spec's customType + * helper) rather than a Drizzle export — the renderer must NOT `imp()` it. + */ + enumIntCustomType?: EnumIntCustomType; /** DB column name (snake_case from field name, or @column override). */ dbName: string; /** Positional args after dbName (currently always empty; reserved). */ @@ -342,6 +375,48 @@ function objectRefBaseName(field: MetaField): string | undefined { return undefined; } +/** SCREAMING_SNAKE_CASE for a generated map const name. */ +function screamingSnake(s: string): string { + return s + .replace(/([a-z0-9])([A-Z])/g, "$1_$2") + .replace(/[^A-Za-z0-9]+/g, "_") + .toUpperCase(); +} + +/** + * Build the customType descriptor for an int-backed `field.enum`, or undefined + * when `@values` is missing (the field then degrades to a plain integer column + * rather than emitting a codec over an unknown member set). + * + * Names are derived from the FIELD name, so the consts are per-entity-file and + * self-contained. A shared enum consumed by N entities therefore emits N small + * identical helpers rather than requiring a cross-module import — the same + * self-contained tradeoff the per-entity enum union already makes. + */ +function buildEnumIntCustomType( + field: MetaField, + intByMember: Record, +): EnumIntCustomType | undefined { + const members = enumValues(field); + if (members === undefined || members.length === 0) return undefined; + // Every member must map — the loader pins key-set-equals-@values (Check 5b), so a + // miss is unreachable; throwing beats emitting a codec with a hole in it. + for (const m of members) { + intValueForMember(intByMember, m, `customType codec for field '${field.name}'`); + } + const base = field.name.replace(/[^A-Za-z0-9]/g, ""); + const camel = base.charAt(0).toLowerCase() + base.slice(1); + const screaming = screamingSnake(base); + return { + fnConstName: `${camel}IntEnum`, + toIntConstName: `${screaming}_TO_INT`, + fromIntConstName: `${screaming}_FROM_INT`, + dataType: "integer", + members, + intByMember, + }; +} + export function mapColumnType( field: MetaField, dialect: Dialect, @@ -355,6 +430,8 @@ export function mapColumnType( let fnName: string; let fnOptions: Record | undefined; + // Set only for an int-backed field.enum — see EnumIntCustomType. + let enumIntCustomType: EnumIntCustomType | undefined; let leadingComment: string | undefined; if (dialect === "sqlite") { @@ -408,7 +485,15 @@ export function mapColumnType( case FIELD_SUBTYPE_ENUM: // An INT-BACKED enum stores the mapped integer on SQLite too — SQLite has // one integer storage class, so this matches migrate-ts's integer{32}. - fnName = intValueMapOf(field) !== undefined ? "integer" : "text"; + { + const im = intValueMapOf(field); + if (im !== undefined) { + enumIntCustomType = buildEnumIntCustomType(field, im); + fnName = enumIntCustomType?.fnConstName ?? "integer"; + } else { + fnName = "text"; + } + } break; case FIELD_SUBTYPE_STRING: case FIELD_SUBTYPE_UUID: @@ -532,7 +617,15 @@ export function mapColumnType( // integer, so the Drizzle column is integer / integer[] — matching // migrate-ts's expected-schema. The TS-facing type stays the member-string // union; the symbol<->int translation happens at the write/read boundary. - fnName = intValueMapOf(field) !== undefined ? "integer" : "text"; + { + const im = intValueMapOf(field); + if (im !== undefined) { + enumIntCustomType = buildEnumIntCustomType(field, im); + fnName = enumIntCustomType?.fnConstName ?? "integer"; + } else { + fnName = "text"; + } + } break; default: fnName = "text"; @@ -679,6 +772,7 @@ export function mapColumnType( }; if (fnOptions !== undefined) result.fnOptions = fnOptions; if (defaultExpr !== undefined) result.defaultExpr = defaultExpr; + if (enumIntCustomType !== undefined) result.enumIntCustomType = enumIntCustomType; if (dollarTypeRef !== undefined) result.dollarTypeRef = dollarTypeRef; if (leadingComment !== undefined) result.leadingComment = leadingComment; diff --git a/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts b/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts index 90d11c043..b3b1d546c 100644 --- a/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts +++ b/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts @@ -13,7 +13,7 @@ import { } from "@metaobjectsdev/metadata"; import { fieldDeclaringPackage, type RenderContext } from "../render-context.js"; import { crossEntitySpecifier, valueObjectModuleSpecifier } from "../import-path.js"; -import { mapColumnType, type ColumnSpec } from "../column-mapper.js"; +import { mapColumnType, type ColumnSpec, type EnumIntCustomType } from "../column-mapper.js"; import { tableNameFromEntity, columnNameFromField } from "../naming.js"; import { renderRelationsBlock } from "./relations-block.js"; import { renderDocsFor } from "./jsdoc.js"; @@ -66,6 +66,9 @@ export function renderDrizzleSchema(obj: MetaObject, ctx: RenderContext): Code { const columnLines: Code[] = []; // Collect CHECK constraints for enum columns; emitted as table-level check() callbacks. const checkConstraints: Array<{ name: string; expr: string }> = []; + // Int-backed field.enum customType helpers, emitted ahead of the table. Keyed by + // const name so a shared enum used by two fields of the SAME entity emits once. + const enumIntTypes = new Map(); for (const child of obj.fields()) { // #213 — a derived (origin-bearing) field is read-only, materialized on the // read (view) side, NOT a column on the entity's write table (FR-024 §7). @@ -78,6 +81,9 @@ export function renderDrizzleSchema(obj: MetaObject, ctx: RenderContext): Code { // Compute the column spec once per field and reuse it for both the column // line and the CHECK collection. const spec = mapColumnType(child, ctx.dialect, ctx.columnNamingStrategy, ctx.timestampMode); + if (spec.enumIntCustomType !== undefined) { + enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); + } const fieldDocs = renderDocsFor(child); const columnLine = renderColumn(spec, child, ctx, isPk, pkGeneration, fkInfo, isComposite, isUnique, obj.package, obj.name); columnLines.push(fieldDocs ? code` ${fieldDocs}\n${columnLine}` : columnLine); @@ -103,6 +109,9 @@ export function renderDrizzleSchema(obj: MetaObject, ctx: RenderContext): Code { // #213 — a TPH subtype's derived field is read-only too; never a table column. if (child.isDerived()) continue; const spec = mapColumnType(child, ctx.dialect, ctx.columnNamingStrategy, ctx.timestampMode); + if (spec.enumIntCustomType !== undefined) { + enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); + } const fieldDocs = renderDocsFor(child); const columnLine = renderColumn( spec, child, ctx, false, undefined, fkMap.get(child.name), isComposite, false, obj.package, obj.name, true, @@ -177,11 +186,60 @@ ${joinCode(columnLines, { on: ",\n", trim: false })} // Emit the relations() block (returns null if no relations). const relationsBlock = renderRelationsBlock(obj, ctx); - if (relationsBlock === null) { - return tableBlock; - } + // Int-backed enum codecs are declared BEFORE the table that references them. + // Sorted by const name so output is deterministic regardless of field order. + const enumIntBlocks = [...enumIntTypes.values()] + .sort((a, b) => a.fnConstName.localeCompare(b.fnConstName)) + .map((t) => renderEnumIntCustomType(t, importModule)); - return joinCode([tableBlock, relationsBlock], { on: "\n" }); + const blocks: Code[] = [...enumIntBlocks, tableBlock]; + if (relationsBlock !== null) blocks.push(relationsBlock); + return blocks.length === 1 ? blocks[0]! : joinCode(blocks, { on: "\n" }); +} + +/** + * Render an int-backed `field.enum`'s Drizzle `customType` helper plus its two + * lookup maps. + * + * The codec lives HERE, in the column definition, so nothing downstream needs to + * know about it: `db.insert().values()` encodes on bind, a selected row decodes on + * read, and a filter comparison encodes because Drizzle binds through the column + * type. That is why this shape was chosen over a Zod write-transform plus a + * generated read-decode — TS's generated queries return raw Drizzle rows and have + * no decode seam, so the query-layer approach meant inventing one and wrapping + * every generated read. It is also the direct analogue of what the other four + * ports already do (EF Core `HasConversion`, OMDB `JdbcFieldCodec`, Exposed + * `customEnumeration`, Python `ObjectManager` coercion). + * + * `fromDriver` throws on an unmapped integer rather than returning undefined: a + * value outside the map means the DB holds data the model says is impossible + * (a hand-written INSERT, or a member removed without a migration), and silently + * yielding `undefined` for a non-nullable field would surface far from the cause. + */ +function renderEnumIntCustomType(t: EnumIntCustomType, importModule: string): Code { + const customTypeSym = imp(`customType@${importModule}`); + const union = t.members.map((m) => JSON.stringify(m)).join(" | "); + const toEntries = t.members + .map((m) => `${JSON.stringify(m)}: ${t.intByMember[m]}`) + .join(", "); + const fromEntries = t.members + .map((m) => `${t.intByMember[m]}: ${JSON.stringify(m)}`) + .join(", "); + return code` +const ${t.toIntConstName} = { ${toEntries} } as const satisfies Record<${union}, number>; +const ${t.fromIntConstName}: Record = { ${fromEntries} }; +const ${t.fnConstName} = ${customTypeSym}<{ data: ${union}; driverData: number }>({ + dataType: () => ${JSON.stringify(t.dataType)}, + toDriver: (value) => ${t.toIntConstName}[value], + fromDriver: (value) => { + const member = ${t.fromIntConstName}[value]; + if (member === undefined) { + throw new Error(\`unmapped ${t.fnConstName} value: \${value}\`); + } + return member; + }, +}); +`; } interface FkInfo { @@ -266,7 +324,13 @@ function renderColumn( // and suppress any DB default (other-subtype rows must stay NULL here). forceNullable: boolean = false, ): Code { - const fnSym = imp(`${spec.fnName}@${spec.importModule}`); + // An int-backed field.enum's column function is a LOCAL generated const (the + // customType helper emitted into this same file), so it must not be imported + // from drizzle-orm/*-core like a built-in column type would be. + const fnSym = + spec.enumIntCustomType !== undefined + ? spec.enumIntCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); const dbNameLit = JSON.stringify(spec.dbName); let baseCall: Code; diff --git a/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts index 5bff3951c..1238d4e52 100644 --- a/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts +++ b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts @@ -41,14 +41,29 @@ async function statusField( } describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { - test("scalar int-backed enum → integer column, no literal-union enum option", async () => { + test("scalar int-backed enum → a generated customType column, no literal-union option", async () => { const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "postgres"); - expect(spec.fnName).toBe("integer"); + // fnName names a LOCAL generated const, not a Drizzle export — the renderer + // must emit it rather than imp() it. + expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.enumIntCustomType).toEqual({ + fnConstName: "statusIntEnum", + toIntConstName: "STATUS_TO_INT", + fromIntConstName: "STATUS_FROM_INT", + dataType: "integer", + members: VALUES, + intByMember: INT_MAP, + }); // The `{ enum: [...] }` literal-union option is a TEXT-column affordance; on an // integer column it would type the column as a string union over a numeric value. expect(spec.fnOptions?.enum).toBeUndefined(); }); + test("string-backed enum carries NO customType (byte-identical to today)", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); + expect(spec.enumIntCustomType).toBeUndefined(); + }); + test("string-backed enum is unchanged — text + literal union", async () => { const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); expect(spec.fnName).toBe("text"); @@ -65,9 +80,10 @@ describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { expect(spec.checkConstraint).toBe("status IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')"); }); - test("sqlite int-backed enum is integer too", async () => { + test("sqlite int-backed enum gets the same customType (integer storage class)", async () => { const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "sqlite"); - expect(spec.fnName).toBe("integer"); + expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.enumIntCustomType?.dataType).toBe("integer"); }); // Amendment 1 / #246 — the canonical authoring shape. An own-only read of @@ -80,7 +96,8 @@ describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { ]), "postgres", ); - expect(spec.fnName).toBe("integer"); + expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.enumIntCustomType?.intByMember).toEqual(INT_MAP); expect(spec.checkConstraint).toBe("status IN (0, 5, 9)"); }); @@ -89,7 +106,7 @@ describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { await statusField({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), "postgres", ); - expect(spec.fnName).toBe("integer"); + expect(spec.fnName).toBe("statusIntEnum"); // Membership on arrays stays app-level, exactly as for string-backed enum[]. expect(spec.checkConstraint).toBeUndefined(); }); diff --git a/server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts b/server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts new file mode 100644 index 000000000..4946d344d --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts @@ -0,0 +1,93 @@ +// Task 5 — an int-backed field.enum's symbol<->int codec lives in the COLUMN +// definition, as a generated Drizzle customType. Nothing downstream changes: +// db.insert().values() encodes on bind, a selected row decodes on read, and a +// filter comparison encodes because Drizzle binds through the column type. +// +// Chosen over a Zod write-transform + generated read-decode because TS's +// generated queries return raw Drizzle rows and have NO decode seam — that route +// meant inventing one and wrapping every generated read. This is also the direct +// analogue of the other four ports' codec seams (EF Core HasConversion, OMDB +// JdbcFieldCodec, Exposed customEnumeration, Python ObjectManager coercion). + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { renderDrizzleSchema } from "../src/templates/drizzle-schema.js"; +import { makeRenderContext } from "../src/render-context.js"; +import { buildPkMap } from "../src/pk-resolver.js"; +import { buildRelationMap } from "../src/relation-resolver.js"; +import type { Dialect } from "../src/metaobjects-config.js"; + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +async function emit(statusDecl: Record, extraRoots: unknown[] = [], dialect: Dialect = "postgres") { + const json = JSON.stringify({ + "metadata.root": { + children: [ + ...extraRoots, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": statusDecl }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const res = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(res.errors).toEqual([]); + const ctx = makeRenderContext({ + dialect, loadedRoot: res.root, outDir: "/x", dbImport: "~/db", + pkMap: buildPkMap(res.root), relationMap: buildRelationMap(res.root), + }); + return renderDrizzleSchema(res.root.findObject("Order")!, ctx).toString(); +} + +describe("Drizzle codegen — int-backed field.enum customType", () => { + test("emits the two lookup maps and a customType whose column is integer", async () => { + const out = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + expect(out).toContain('const STATUS_TO_INT = { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 }'); + expect(out).toContain('const STATUS_FROM_INT: Record'); + expect(out).toContain('dataType: () => "integer"'); + expect(out).toContain("toDriver: (value) => STATUS_TO_INT[value]"); + // The column uses the local const, NOT a drizzle `integer(...)` call. + expect(out).toContain('statusIntEnum("status")'); + }); + + test("the TS-facing data type stays the member-string union, never number", async () => { + const out = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + expect(out).toContain('data: "DRAFT" | "PUBLISHED" | "ARCHIVED"; driverData: number'); + }); + + test("an unmapped stored integer throws rather than yielding undefined", async () => { + const out = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + expect(out).toContain("unmapped statusIntEnum value"); + }); + + test("customType is imported from the dialect's core module", async () => { + const pg = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }, [], "postgres"); + expect(pg).toContain("drizzle-orm/pg-core"); + const sqlite = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }, [], "sqlite"); + expect(sqlite).toContain("drizzle-orm/sqlite-core"); + }); + + test("the codec is emitted for a map INHERITED from a shared declaration (#246 shape)", async () => { + const out = await emit({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]); + expect(out).toContain("const statusIntEnum = "); + expect(out).toContain("STATUS_TO_INT[value]"); + }); + + test("a string-backed enum emits NO codec at all (byte-identical to today)", async () => { + const out = await emit({ name: "status", "@values": VALUES }); + expect(out).not.toContain("customType"); + expect(out).not.toContain("STATUS_TO_INT"); + expect(out).toContain('text("status"'); + }); +}); From 9e1707256c585a9a2b412192fbeb4b75a424861d Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Thu, 13 Aug 2026 19:02:23 -0400 Subject: [PATCH 26/52] =?UTF-8?q?test(integration):=20int-backed=20enum=20?= =?UTF-8?q?against=20a=20REAL=20Postgres=20=E2=80=94=20apply,=20converge,?= =?UTF-8?q?=20enforce?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first end-to-end validation of this program. Everything before it was unit assertions plus reading generated source: migrate-ts SAYS integer, codegen-ts SAYS customType, both SAY the CHECK lists unquoted ints. None of that proved the DDL applies, that a second migrate converges, or that the constraint enforces the integers it claims to. Pulled forward ahead of Tasks 8/9 deliberately -- finishing those on unproven foundations is how this repo got the 0.15.21 line, where destructive migrate bugs survived thousands of tests because nothing ever ran the pipeline twice against a real engine. `emit` and `introspect` had never been in the same room. Seven scenarios, all green: - the emitted DDL APPLIES, and a second migrate CONVERGES (empty diff) -- the false-drift gate, which is the failure mode a codegen/expected-schema disagreement would have produced. - the physical column is `integer`, read back from information_schema. - the CHECK enforces the MAPPED INTEGERS: status=5 (PUBLISHED) inserts, status=7 (no member) is rejected. A CHECK emitted over member strings, or omitted, would have let 7 through. - an ORDINAL-looking value is rejected: ARCHIVED is index 2 in @values but maps to 9, so anything deriving the stored int from member POSITION -- design Goal 3's named hazard, the OpenAPI x-enum-varnames failure mode -- would accept 2. It does not. - @default lands as the mapped integer (5, not 'PUBLISHED'), and reaching that assertion at all is part of the proof: DEFAULT 'PUBLISHED' on an integer column would not have applied. - the string-backed control still gets `text` and still converges. - toggling the backing on an existing table is BLOCKED, against a real introspected schema rather than a hand-built snapshot. The model puts @intValueMap on a SHARED root-level abstract declaration with the field inheriting it -- the shape #246 steers authors toward, and the one an own-only read would silently get wrong -- so the resolving-read decision is now validated against a real database, not just a unit test. Verified: 7/7 against Testcontainers Postgres; workspace typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) --- .../test/enum-intvaluemap-pg.test.ts | 213 ++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts diff --git a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts new file mode 100644 index 000000000..cd314d581 --- /dev/null +++ b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts @@ -0,0 +1,213 @@ +/** + * Int-backed `field.enum` (`@intValueMap`) — the REAL gate, against a live Postgres. + * + * WHY THIS EXISTS + * + * Everything else about int-backed enums is verified by unit assertions and by + * inspecting generated source: migrate-ts says the column is `integer`, codegen-ts + * says the Drizzle column is a `customType`, and both say the CHECK lists unquoted + * integers. None of that proves the DDL APPLIES, that a second migrate CONVERGES, or + * that the codec actually round-trips a member symbol through a real integer column. + * + * This repo has a monument to exactly that gap: the 0.15.21 line, where a family of + * destructive migrate bugs survived a suite of thousands because nothing ever ran the + * pipeline twice against a real engine. `emit` and `introspect` had never been in the + * same room. The same is true here until this file runs. + * + * So: apply to a REAL engine, RE-DIFF, and then prove the value semantics both ways — + * a symbol written through the generated codec must land as the mapped INTEGER in the + * physical column, and an integer already in the column must read back as the symbol. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from "bun:test"; +import { + buildExpectedSchema, diff, emit, introspectPostgres, + type SchemaSnapshot, +} from "@metaobjectsdev/migrate-ts"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { Kysely, PostgresDialect, sql } from "kysely"; +import { Pool } from "pg"; +import { startPostgres, type RunningPg } from "../src/postgres-container.ts"; + +// The map is deliberately SPARSE and non-ordinal (0/5/9) so any accidental +// index-of-@values correspondence shows up as a wrong number rather than passing +// by coincidence — the failure mode the design's Goal 3 calls out. +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 } as const; + +/** An entity with an int-backed enum. The map lives on a SHARED root-level abstract + * declaration and the field inherits it — the shape #246 steers authors toward, and + * the one an own-only read would silently get wrong. */ +function meta(opts: { withDefault?: boolean } = {}): string { + const dflt = opts.withDefault ? `, "@default": "PUBLISHED"` : ""; + return `{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.enum": { "name": "status", "extends": "Status", "@required": true${dflt} } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } } + ] + } + }`; +} + +/** The string-backed control: identical model minus @intValueMap. */ +function metaStringBacked(): string { + return `{ + "metadata.root": { + "package": "acme", + "children": [ + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.enum": { "name": "status", "@required": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } } + ] + } + }`; +} + +let pg: RunningPg; +let pool: Pool; +let k: Kysely; + +beforeAll(async () => { + pg = await startPostgres(); + pool = new Pool({ connectionString: pg.connectionUri }); + k = new Kysely({ dialect: new PostgresDialect({ pool }) }); +}, 120_000); + +afterAll(async () => { + await k?.destroy(); + await pg?.stop(); +}); + +beforeEach(async () => { + await sql.raw(`DROP TABLE IF EXISTS "orders" CASCADE;`).execute(k); + await sql.raw(`DROP TABLE IF EXISTS orders CASCADE;`).execute(k); +}); + +async function applyRaw(ddl: string): Promise { + for (const stmt of ddl.split(/;\s*\n/).map((s) => s.trim()).filter(Boolean)) { + await sql.raw(stmt.endsWith(";") ? stmt : `${stmt};`).execute(k); + } +} + +async function expectedFor(metaJson: string): Promise { + const root = (await new MetaDataLoader().load([new InMemoryStringSource(metaJson)])).root; + return buildExpectedSchema(root, { columnNamingStrategy: "literal", dialect: "postgres" }); +} + +/** build → introspect → diff → emit → apply, then return the expected side. */ +async function migrate(metaJson: string): Promise { + const expected = await expectedFor(metaJson); + const result = await diff({ + expected, actual: await introspectPostgres(k), dialect: "postgres", + }); + expect(result.blocked).toEqual([]); + const { up } = result.changes.length === 0 + ? { up: "" } + : emit(result.changes, { dialect: "postgres" }); + if (up.trim().length > 0) await applyRaw(up); + return expected; +} + +/** THE gate: a second migrate against the just-migrated DB must be a no-op. */ +async function assertConverged(expected: SchemaSnapshot): Promise { + const followup = await diff({ + expected, actual: await introspectPostgres(k), dialect: "postgres", + }); + if (followup.changes.length > 0) { + console.error("NOT CONVERGED — a second migrate would emit:"); + for (const c of followup.changes) console.error(" -", c.kind, JSON.stringify(c).slice(0, 300)); + } + expect(followup.changes).toEqual([]); +} + +describe("int-backed field.enum — real Postgres", () => { + test("the emitted DDL APPLIES and a second migrate converges", async () => { + const expected = await migrate(meta()); + await assertConverged(expected); + }, 120_000); + + test("the physical column is integer, not text", async () => { + await migrate(meta()); + const rows = await sql<{ data_type: string }>` + SELECT data_type FROM information_schema.columns + WHERE table_name = 'orders' AND column_name = 'status' + `.execute(k); + expect(rows.rows[0]?.data_type).toBe("integer"); + }, 120_000); + + test("the CHECK enforces the mapped INTEGERS — a valid member's int is accepted, a non-member int rejected", async () => { + await migrate(meta()); + // 5 === PUBLISHED, so this must be accepted. + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('ok', 5);`).execute(k); + // 7 maps to no member. If the CHECK had been emitted over the member STRINGS + // (or omitted), this would succeed and the column would hold an impossible value. + let rejected = false; + try { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('bad', 7);`).execute(k); + } catch { + rejected = true; + } + expect(rejected).toBe(true); + }, 120_000); + + test("an ordinal-looking wrong value is rejected — the map is sparse, not positional", async () => { + await migrate(meta()); + // ARCHIVED is index 2 in @values but maps to 9. If anything derived the stored + // int from the member's POSITION (design Goal 3's hazard), 2 would be valid. + let rejected = false; + try { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('ordinal', 2);`).execute(k); + } catch { + rejected = true; + } + expect(rejected).toBe(true); + }, 120_000); + + test("@default lands as the mapped integer, appliably", async () => { + const expected = await migrate(meta({ withDefault: true })); + await assertConverged(expected); + await sql.raw(`INSERT INTO "orders" ("title") VALUES ('defaulted');`).execute(k); + const rows = await sql<{ status: number }>` + SELECT "status" FROM "orders" WHERE "title" = 'defaulted' + `.execute(k); + // PUBLISHED === 5. A DEFAULT 'PUBLISHED' on an integer column would not have + // applied at all, so reaching this assertion is itself part of the proof. + expect(rows.rows[0]?.status).toBe(5); + }, 120_000); + + test("the string-backed control still gets a text column and converges", async () => { + const expected = await migrate(metaStringBacked()); + await assertConverged(expected); + const rows = await sql<{ data_type: string }>` + SELECT data_type FROM information_schema.columns + WHERE table_name = 'orders' AND column_name = 'status' + `.execute(k); + expect(rows.rows[0]?.data_type).toBe("text"); + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('s', 'DRAFT');`).execute(k); + }, 120_000); + + test("toggling the backing on an existing table is BLOCKED (no silent destructive recast)", async () => { + await migrate(metaStringBacked()); + const intExpected = await expectedFor(meta()); + const result = await diff({ + expected: intExpected, actual: await introspectPostgres(k), dialect: "postgres", + }); + const typeChange = result.changes.find((c) => c.kind === "change-column-type"); + expect(typeChange).toBeDefined(); + expect(typeChange!.status.state).toBe("blocked"); + }, 120_000); +}); From 7f03fcdddc3a850135cf9769a69a2c04be450592 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 14 Aug 2026 10:18:38 -0400 Subject: [PATCH 27/52] test(integration): the GENERATED int-backed enum codec, against real Postgres Extends the PG gate from the schema to the codec. The previous block drove raw SQL, so customType's toDriver/fromDriver had still never executed against a database -- the codec was verified only by reading the source we generate. This runs the real `runGen` pipeline into .gen-tmp/, imports the emitted entity module UNMODIFIED, and drives Drizzle through it. Six scenarios: - a member symbol written through Drizzle lands as the mapped INTEGER, asserted by reading the physical value with raw SQL -- bypassing the codec, so it proves what is on disk rather than that the two directions cancel out. - an integer inserted by raw SQL (never through toDriver) reads back as its member symbol, which is the half a round-trip-only test cannot distinguish. - every member round-trips INCLUDING the zero-valued one. DRAFT -> 0 is falsy and therefore exactly what a truthiness-based codec drops (the #235 bug class); 0 is also asserted on disk. - a WHERE comparison on a member symbol encodes through the column type. This is the load-bearing claim behind choosing customType -- that filters work with NO filter-layer change -- and it is now demonstrated rather than argued, which collapses most of Task 8. - an `in` comparison encodes every member in the list. - an unmapped stored integer THROWS on read rather than yielding undefined (CHECK dropped first to simulate data written before a member was removed, or by a hand-written migration). Two path assumptions were wrong and are now settled empirically rather than guessed: the generated module is emitted FLAT (`Order.ts`, not `acme/Order.ts`), alongside an `enums.ts` because the shared root-level abstract `Status` is an FR-019 shared enum -- so this also incidentally covers the codec working when the member union comes from a shared enum module rather than an inline union. Verified: 13/13 in this file against Testcontainers Postgres (7 schema + 6 codec); workspace typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) --- .../test/enum-intvaluemap-pg.test.ts | 146 +++++++++++++++++- 1 file changed, 141 insertions(+), 5 deletions(-) diff --git a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts index cd314d581..efde60be9 100644 --- a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts @@ -25,8 +25,15 @@ import { type SchemaSnapshot, } from "@metaobjectsdev/migrate-ts"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { runGen, defineConfig } from "@metaobjectsdev/codegen-ts"; +import { entityFile } from "@metaobjectsdev/codegen-ts/generators"; import { Kysely, PostgresDialect, sql } from "kysely"; -import { Pool } from "pg"; +import pg, { Pool } from "pg"; +import { drizzle } from "drizzle-orm/node-postgres"; +import { eq, inArray } from "drizzle-orm"; +import { mkdirSync, mkdtempSync, rmSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; import { startPostgres, type RunningPg } from "../src/postgres-container.ts"; // The map is deliberately SPARSE and non-ordinal (0/5/9) so any accidental @@ -77,19 +84,23 @@ function metaStringBacked(): string { }`; } -let pg: RunningPg; +let runningPg: RunningPg; let pool: Pool; let k: Kysely; +/** Shared with the generated-codec block below, which needs its own Drizzle pool + * against the SAME database the DDL was applied to. */ +let pg2Uri: string; beforeAll(async () => { - pg = await startPostgres(); - pool = new Pool({ connectionString: pg.connectionUri }); + runningPg = await startPostgres(); + pg2Uri = runningPg.connectionUri; + pool = new Pool({ connectionString: runningPg.connectionUri }); k = new Kysely({ dialect: new PostgresDialect({ pool }) }); }, 120_000); afterAll(async () => { await k?.destroy(); - await pg?.stop(); + await runningPg?.stop(); }); beforeEach(async () => { @@ -211,3 +222,128 @@ describe("int-backed field.enum — real Postgres", () => { expect(typeChange!.status.state).toBe("blocked"); }, 120_000); }); + +// --------------------------------------------------------------------------- +// The CODEC half. Everything above exercises the SCHEMA through raw SQL, so +// customType's toDriver/fromDriver had still never run against a database. This +// block emits the REAL entity file via runGen, imports it unmodified, and drives +// Drizzle through it — the only way to prove the generated codec, as opposed to +// a hand-written mirror of what we think it generates. +// --------------------------------------------------------------------------- + +describe("int-backed field.enum — generated codec against real Postgres", () => { + // Emit INSIDE the package tree (.gen-tmp/, gitignored): the generated module + // resolves bare specifiers like `drizzle-orm/pg-core` by walking up to a + // node_modules chain, which the OS tmpdir never reaches. + let tmp: string; + let ordersTable: any; + let gdb: any; + let gpool: pg.Pool; + + beforeAll(async () => { + const here = dirname(fileURLToPath(import.meta.url)); + const genTmpRoot = join(here, "..", ".gen-tmp"); + mkdirSync(genTmpRoot, { recursive: true }); + tmp = mkdtempSync(join(genTmpRoot, "enum-intmap-")); + + const root = (await new MetaDataLoader().load([new InMemoryStringSource(meta())])).root; + const lr = await runGen({ + config: defineConfig({ + outDir: tmp, + extStyle: "none", + dbImport: "./db", + dialect: "postgres", + generators: [entityFile()], + }), + metadata: root, + }); + if (lr.warnings.length > 0) throw new Error(`codegen warnings: ${lr.warnings.join("; ")}`); + + // The emitted entity module, imported UNMODIFIED — codec included. + const entityUrl = pathToFileURL(join(tmp, "Order.ts")).href; + const mod: any = await import(entityUrl); + ordersTable = mod.orders; + expect(ordersTable).toBeDefined(); + + gpool = new pg.Pool({ connectionString: pg2Uri }); + gdb = drizzle(gpool); + }, 180_000); + + afterAll(async () => { + await gpool?.end(); + rmSync(tmp, { recursive: true, force: true }); + }); + + beforeEach(async () => { + // Schema provisioned from the SAME metadata via migrate-ts, so the physical + // shape the codec writes into is the one the DDL pipeline produces. + await migrate(meta()); + }); + + test("a member symbol written through Drizzle lands as the mapped INTEGER", async () => { + await gdb.insert(ordersTable).values({ title: "w", status: "PUBLISHED" }); + // Read the PHYSICAL value with raw SQL — bypassing the codec entirely, so this + // asserts what is actually on disk rather than what the codec round-trips. + const raw = await sql<{ status: number }>`SELECT "status" FROM "orders" WHERE "title" = 'w'`.execute(k); + expect(raw.rows[0]?.status).toBe(5); + }, 120_000); + + test("an integer already in the column reads back as its member symbol", async () => { + // Insert 9 (ARCHIVED) with raw SQL so the value never passes through toDriver — + // proving fromDriver decodes, not merely that the two directions cancel out. + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('r', 9);`).execute(k); + const rows = await gdb.select().from(ordersTable); + expect(rows).toHaveLength(1); + expect(rows[0].status).toBe("ARCHIVED"); + }, 120_000); + + test("round-trips every member, including the ZERO-valued one", async () => { + for (const m of ["DRAFT", "PUBLISHED", "ARCHIVED"]) { + await gdb.insert(ordersTable).values({ title: m, status: m }); + } + const rows = await gdb.select().from(ordersTable); + const byTitle = new Map(rows.map((r: any) => [r.title, r.status])); + // DRAFT maps to 0 — falsy, and therefore the value any truthiness-based codec + // silently drops or coerces (the #235 bug class). + expect(byTitle.get("DRAFT")).toBe("DRAFT"); + expect(byTitle.get("PUBLISHED")).toBe("PUBLISHED"); + expect(byTitle.get("ARCHIVED")).toBe("ARCHIVED"); + const raws = await sql<{ title: string; status: number }>`SELECT "title", "status" FROM "orders"`.execute(k); + expect(new Map(raws.rows.map((r) => [r.title, r.status])).get("DRAFT")).toBe(0); + }, 120_000); + + // Task 8 — the filter path. This is the claim that customType makes the + // filter work "for free": Drizzle binds a WHERE comparison through the column + // type, so a member symbol encodes without the filter layer knowing anything. + test("a WHERE comparison on a member symbol encodes through the column type", async () => { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('a', 0), ('b', 9);`).execute(k); + const archived = await gdb.select().from(ordersTable).where(eq(ordersTable.status, "ARCHIVED")); + expect(archived).toHaveLength(1); + expect(archived[0].title).toBe("b"); + const drafts = await gdb.select().from(ordersTable).where(eq(ordersTable.status, "DRAFT")); + expect(drafts).toHaveLength(1); + expect(drafts[0].title).toBe("a"); + }, 120_000); + + test("an `in` comparison encodes every member in the list", async () => { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('a', 0), ('b', 5), ('c', 9);`).execute(k); + const some = await gdb.select().from(ordersTable) + .where(inArray(ordersTable.status, ["DRAFT", "ARCHIVED"])); + expect(some.map((r: any) => r.title).sort()).toEqual(["a", "c"]); + }, 120_000); + + test("an unmapped stored integer throws on read instead of yielding undefined", async () => { + // The CHECK normally makes this unreachable; drop it to simulate data written + // before a member was removed, or by a hand-written migration. + await sql.raw(`ALTER TABLE "orders" DROP CONSTRAINT "orders_status_chk";`).execute(k); + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('bogus', 42);`).execute(k); + let threw = false; + try { + await gdb.select().from(ordersTable); + } catch (e) { + threw = true; + expect(String(e)).toContain("unmapped"); + } + expect(threw).toBe(true); + }, 120_000); +}); From 66fb0b4ea6fe2a40f0b311367080e5657b363596 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 14 Aug 2026 19:12:41 -0400 Subject: [PATCH 28/52] feat(all-ports): the filter-operator band is field-level -- an int-backed enum drops `like` An int-backed `field.enum` (`@intValueMap`, design D5) persists as an INTEGER column, so `like` -- a substring match -- is meaningless against it. It is also the ONE operator in the string/enum band that cannot be rescued by encoding: `eq`/`ne`/`in` all lower the member symbol to its integer before the value reaches SQL, but `like` has no such encoding, and an unencoded `LIKE 'DRAFT'` against an integer column is a request-time type error. `opsForSubType` cannot express this -- it only ever sees the subtype `"enum"`, so the generated `FilterAllowlist` offered `like` on an int-backed field byte-identically to a string-backed one. The band is therefore a property of the FIELD, not the subtype. Every port gains an `opsForField` peer; `opsForSubType` is deliberately left unchanged for the one caller that genuinely has no field in hand (the expression grammar's declared operand type). Fixed as ONE loader rule per port rather than five per-port codegen filters, on the precedent of #210 and the `@objectRef` payload rule -- an authored `attr.filter` / dataGrid `@filter` using `like` on an int-backed enum now fails at load, not later at the SQL layer. Ports: - TS `opsForField` in query-constants; loader dataGrid + projection filter passes, filter-allowlist, filter-type, conformance binding. - Java `FilterOps.opsForField`; ValidationPhase, SpringFilterAllowlistGenerator, ScriptRunner. - Kotlin KotlinFilterAllowlistGenerator (its own copy of the call, shared band). - C# `QueryConstants.OpsForField`; ValidationPasses, CapabilityBinding, and FilterAllowlistGenerator -- whose OWN duplicate per-subtype band table is DELETED rather than extended, since two tables is exactly how bands drift. - Python `ops_for_field` (loader) + `ops_for_field_ordered` (codegen); dataGrid pass, allowlist generator seam, conformance capability. Every port reads `@intValueMap` RESOLVING (ADR-0039). Post-#246 the map lives on a shared root-level abstract declaration and consuming fields INHERIT it, so an own-only read would see it absent on exactly the shape adopters are steered toward and wrongly keep `like`. Gated cross-port by a new `fEnumInt` case in `fixtures/conformance/filter-ops-matrix` -- the `field.filter-ops` capability was already field-level in all five ports, so the fixture pins `fEnum` (with `like`) against `fEnumInt` (without) with no runner change. Plus a TS unit test covering the inherited-map shape. Known and NOT addressed here: C# and Python validate no ops at all in their projection `@filter` pass (TS does), so the load-time rejection lands in TS only for that one authoring surface. That is a pre-existing cross-port gap in projection-filter validation generally, not something this change introduces. Co-Authored-By: Claude Opus 5 (1M context) --- .../filter-ops-matrix/input/meta.matrix.json | 2 + .../conformance/filter-ops-matrix/script.json | 3 + .../Generators/FilterAllowlistGenerator.cs | 47 ++---- .../CapabilityBinding.cs | 12 +- .../MetaObjects/Core/Query/QueryConstants.cs | 52 +++++++ .../MetaObjects/Loader/ValidationPasses.cs | 4 +- .../kotlin/KotlinFilterAllowlistGenerator.kt | 11 +- .../SpringFilterAllowlistGenerator.java | 9 +- .../metaobjects/loader/ValidationPhase.java | 4 +- .../java/com/metaobjects/query/FilterOps.java | 43 ++++++ .../metaobjects/conformance/ScriptRunner.java | 2 +- .../generators/filter_allowlist_generator.py | 43 +++++- .../metaobjects/loader/validation_passes.py | 43 +++++- .../python/tests/conformance/capabilities.py | 10 +- .../src/templates/filter-allowlist.ts | 6 +- .../codegen-ts/src/templates/filter-type.ts | 8 +- .../src/core/query/query-constants.ts | 48 +++++- .../metadata/src/loader/validation-passes.ts | 9 +- .../metadata/test/conformance/binding.ts | 16 +- .../test/enum-int-backed-filter-ops.test.ts | 146 ++++++++++++++++++ 20 files changed, 452 insertions(+), 66 deletions(-) create mode 100644 server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts diff --git a/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json b/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json index 22dd427e0..e305a32cf 100644 --- a/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json +++ b/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json @@ -19,6 +19,7 @@ { "field.time": { "name": "fTime", "@filterable": true } }, { "field.timestamp": { "name": "fTimestamp", "@filterable": true } }, { "field.enum": { "name": "fEnum", "@values": ["a", "b", "c"], "@filterable": true } }, + { "field.enum": { "name": "fEnumInt", "@values": ["a", "b", "c"], "@intValueMap": { "a": 0, "b": 1, "c": 2 }, "@filterable": true } }, { "field.uuid": { "name": "fUuid", "@filterable": true } }, { "source.rdb": { "@table": "matrix" } }, { "identity.primary": { "name": "pk", "@fields": ["id"] } }, @@ -34,6 +35,7 @@ { "identity.secondary": { "name": "byTime", "@fields": ["fTime"] } }, { "identity.secondary": { "name": "byTimestamp", "@fields": ["fTimestamp"] } }, { "identity.secondary": { "name": "byEnum", "@fields": ["fEnum"] } }, + { "identity.secondary": { "name": "byEnumInt", "@fields": ["fEnumInt"] } }, { "identity.secondary": { "name": "byUuid", "@fields": ["fUuid"] } } ] } diff --git a/fixtures/conformance/filter-ops-matrix/script.json b/fixtures/conformance/filter-ops-matrix/script.json index da22b0a0b..acd7fe841 100644 --- a/fixtures/conformance/filter-ops-matrix/script.json +++ b/fixtures/conformance/filter-ops-matrix/script.json @@ -6,6 +6,9 @@ { "navigate": ["object:Matrix", "field:fEnum"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "in", "like", "isNull"] } }, + { "navigate": ["object:Matrix", "field:fEnumInt"], + "invoke": "field.filter-ops", + "expect": { "names": ["eq", "ne", "in", "isNull"] } }, { "navigate": ["object:Matrix", "field:fUuid"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "in", "isNull"] } }, diff --git a/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs index c67a53859..349451fa9 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs @@ -28,21 +28,6 @@ public class FilterAllowlistGenerator : PerEntityGenerator { public override string Name => "filter-allowlist-generator"; - ///

      Operator set for string-shaped subtypes. - internal static readonly string[] OpsString = - { "eq", "ne", "in", "like", "isNull" }; - - /// Operator set for uuid — identity comparison only, no like (not a substring type) and no ordering. - internal static readonly string[] OpsUuid = - { "eq", "ne", "in", "isNull" }; - - /// Operator set for numeric / date / timestamp / currency subtypes. - internal static readonly string[] OpsNumeric = - { "eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull" }; - - /// Operator set for boolean subtype. - internal static readonly string[] OpsBoolean = { "eq", "isNull" }; - // Read-only projections (views/etc.) are not filterable in the routes // generator today — skip them here to match. protected override bool Filter(MetaObject entity) => AppliesTo(entity); @@ -119,7 +104,7 @@ protected virtual Dictionary> ComputeFilterableOps { if (field.SubType == FIELD_SUBTYPE_OBJECT) continue; if (!IsFilterable(field)) continue; - var ops = OpsForSubtype(field.SubType); + var ops = OpsForField(field); if (ops.Count == 0) continue; out_[field.Name] = ops; } @@ -135,18 +120,20 @@ protected virtual Dictionary> ComputeFilterableOps _ => false, }; - /// Operator set for , or empty for unsupported subtypes. - internal static IReadOnlyList OpsForSubtype(string? subType) => subType switch - { - // ADR-0036 Wave 3 — field.uri is string-like (eq/ne/in/like/isNull); field.inet is - // uuid-like (eq/ne/in/isNull — no like / ordering on an opaque address value). - FIELD_SUBTYPE_STRING or FIELD_SUBTYPE_ENUM or FIELD_SUBTYPE_URI => OpsString, - FIELD_SUBTYPE_UUID or FIELD_SUBTYPE_INET => OpsUuid, - FIELD_SUBTYPE_INT or FIELD_SUBTYPE_LONG - or FIELD_SUBTYPE_FLOAT or FIELD_SUBTYPE_DOUBLE or FIELD_SUBTYPE_DECIMAL - or FIELD_SUBTYPE_CURRENCY - or FIELD_SUBTYPE_DATE or FIELD_SUBTYPE_TIMESTAMP or FIELD_SUBTYPE_TIME => OpsNumeric, - FIELD_SUBTYPE_BOOLEAN => OpsBoolean, - _ => System.Array.Empty(), - }; + /// + /// Operator set for , or empty for unsupported subtypes. + /// + /// Delegates wholesale to QueryConstants.OpsForField — the cross-port single + /// source of truth, pinned by fixtures/conformance/filter-ops-matrix. This + /// generator used to carry its OWN copy of the per-subtype band table; that copy was + /// removed rather than extended, because two tables is exactly how the bands drift. + /// + /// + /// Field-level, not subtype-level: an int-backed field.enum + /// (@intValueMap, design D5) persists as an INTEGER column, so like — + /// a substring match — is dropped. + /// + /// + internal static IReadOnlyList OpsForField(MetaField field) => + MetaObjects.Core.Query.QueryConstants.OpsForField(field); } diff --git a/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs b/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs index 1375498df..7835bf7d2 100644 --- a/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs +++ b/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs @@ -70,12 +70,14 @@ private delegate NormalizedResult CapabilityFn( ["field.effective-tree"] = (node, _) => NormalizedResult.EffectiveTree(MetaObjects.SerializerJson.CanonicalSerialize(AsField(node))), - // field.filter-ops → the canonical per-subtype filter-operator band - // (QueryConstants.OPS_BY_SUBTYPE). Returns { names: [...] } in canonical - // operator order. Single source of truth — the same map the server - // allowlist + codegen consume. + // field.filter-ops → the canonical per-FIELD filter-operator band. + // Returns { names: [...] } in canonical operator order. Single source of + // truth — the same function the server allowlist + codegen consume. + // + // OpsForField, not OpsForSubType: the band is field-level because an + // int-backed field.enum (@intValueMap) stores as an integer, dropping `like`. ["field.filter-ops"] = (node, _) => - NormalizedResult.Names(QueryConstants.OpsForSubType(AsField(node).SubType).ToList()), + NormalizedResult.Names(QueryConstants.OpsForField(AsField(node)).ToList()), }; /// diff --git a/server/csharp/MetaObjects/Core/Query/QueryConstants.cs b/server/csharp/MetaObjects/Core/Query/QueryConstants.cs index 6aeaf5cc1..722a630b0 100644 --- a/server/csharp/MetaObjects/Core/Query/QueryConstants.cs +++ b/server/csharp/MetaObjects/Core/Query/QueryConstants.cs @@ -71,6 +71,58 @@ public static class QueryConstants public static string[] OpsForSubType(string subType) => OPS_BY_SUBTYPE.TryGetValue(subType, out string[]? ops) ? ops : []; + /// + /// The int-backed-enum band: the enum band minus like. Hoisted + /// so the narrowing is one named constant rather than an array filtered at every call. + /// + public static readonly string[] OPS_ENUM_INT_BACKED = + [FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL]; + + /// + /// The filter-operator band for a FIELD — the entry point every consumer that has a + /// field in hand must use (loader validation, the codegen filter-allowlist generator, + /// the cross-port field.filter-ops capability). + /// + /// Identical to except for ONE case: an int-backed + /// field.enum (one declaring @intValueMap, design D5) persists as an + /// INTEGER column, so like — a substring match — is dropped. + /// eq/ne/in survive because the member symbol encodes to its + /// integer before it reaches SQL; like has no such encoding, and an unencoded + /// LIKE 'DRAFT' against an integer column is a request-time type error. + /// + /// + /// cannot express this — it only ever sees the subtype + /// "enum" — and is deliberately left unchanged for the one caller that + /// genuinely has no field: ExpressionGrammar's declared operand type. + /// + /// + /// ADR-0039: the @intValueMap read is RESOLVING (Attr, not + /// OwnAttr). Post-#246 the map lives on a shared root-level abstract + /// declaration and consuming fields INHERIT it, so an own-only read would see it + /// absent on exactly the shape adopters are steered toward and wrongly keep + /// like. + /// + /// + /// Cross-port: fixtures/conformance/filter-ops-matrix pins fEnum vs + /// fEnumInt in all five ports. + /// + /// + /// Takes MetaData rather than MetaField so the loader's dataGrid pass — + /// which iterates untyped children — can call it directly, mirroring the TS + /// structural parameter. + /// + /// + public static string[] OpsForField(MetaObjects.Meta.MetaData field) + { + if (field is null) return []; + if (field.SubType == MetaObjects.Core.Field.FieldConstants.FIELD_SUBTYPE_ENUM && + field.Attr(MetaObjects.Core.Field.FieldConstants.FIELD_ATTR_INT_VALUE_MAP) is not null) + { + return OPS_ENUM_INT_BACKED; + } + return OpsForSubType(field.SubType); + } + // ----------------------------------------------------------------------- // Sort order values (used by @sortableDefaultOrder on fields and // @defaultSortOrder on dataGrid layouts) diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index f2657d936..054a7d5b8 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -1476,7 +1476,9 @@ public static IReadOnlyList ValidateDataGridFilterValues(MetaData roo // ADR-0039: resolving — a concrete field may inherit @filterable via extends (TS validation-passes.ts:1252). if (f.Attr(FIELD_ATTR_FILTERABLE) is true) { - allow[f.Name] = OpsForSubType(f.SubType); + // OpsForField, not OpsForSubType — an int-backed field.enum + // (@intValueMap) stores as an integer, so `like` is not in its band. + allow[f.Name] = OpsForField(f); } } diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt index 174e10665..7d2788d99 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt @@ -173,7 +173,7 @@ open class KotlinFilterAllowlistGenerator : MultiFileDirectGeneratorBase = - com.metaobjects.query.FilterOps.opsForSubType(subType) + // + // Field-level, not subtype-level: an int-backed field.enum (@intValueMap, + // design D5) stores as an INTEGER column, so `like` — a substring match — + // is not in its band. Pinned cross-port by + // fixtures/conformance/filter-ops-matrix (fEnum vs fEnumInt). + private fun opsForField(field: MetaField<*>): Set = + com.metaobjects.query.FilterOps.opsForField(field) } // === MultiFileDirectGeneratorBase abstract-method stubs ==================== diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java index f71fb47a8..519c5c631 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java @@ -115,7 +115,7 @@ static Map> computeFilterableOps(Iterable fields) for (MetaField field : fields) { if (field instanceof ObjectField) continue; if (!isFilterable(field)) continue; - Set ops = opsForSubtype(field.getSubType()); + Set ops = opsForField(field); if (ops.isEmpty()) continue; out.putIfAbsent(field.getName(), ops); // dedup base/subtype column names } @@ -130,12 +130,15 @@ protected static boolean isFilterable(MetaField field) { return Boolean.parseBoolean(String.valueOf(raw)); } - private static Set opsForSubtype(String subType) { + private static Set opsForField(MetaField field) { // Single source of truth — com.metaobjects.query.FilterOps (the same // band the loader's validation path reads). Returns a canonical-ordered // set so the emitted source is stable; an unbanded subtype → empty set, // which computeFilterableOps drops. - return com.metaobjects.query.FilterOps.opsForSubType(subType); + // + // Field-level, not subtype-level: an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. + return com.metaobjects.query.FilterOps.opsForField(field); } protected void emit(MetaObject entity, Path outRoot, MetaDataLoader loader) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index 6ed802062..c1037e0cc 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -2772,8 +2772,10 @@ private static void validateFilterClause(MetaObject obj, DataGridLayout grid, } private static java.util.Set allowedOpsFor(MetaField field) { + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. java.util.Set band = - com.metaobjects.query.FilterOps.opsForSubType(field.getSubType()); + com.metaobjects.query.FilterOps.opsForField(field); // Any subtype without a declared band (already rejected upstream by // validateFilterableHasSupportedOps) falls through to the string-shape // band, preserving the prior default. diff --git a/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java b/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java index 62a74e85b..e97085717 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java +++ b/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java @@ -145,4 +145,47 @@ public static Set opsForSubType(String subType) { public static boolean supportsFiltering(String subType) { return subType != null && OPS_BY_SUBTYPE.containsKey(subType); } + + /** + * The int-backed-{@code enum} band: the {@code enum} band minus {@code like}. + * Hoisted so the narrowing is one named constant rather than a set filtered at + * every call site. + */ + public static final Set OPS_ENUM_INT_BACKED = + ordered(FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL); + + /** + * The filter-operator band for a FIELD — the entry point every consumer that has + * a field in hand must use (loader validation, the codegen-spring allowlist + * generator, the cross-port {@code field.filter-ops} capability). + * + *

      Identical to {@link #opsForSubType} except for ONE case: an int-backed + * {@code field.enum} (one declaring {@code @intValueMap}, design D5) persists as + * an INTEGER column, so {@code like} — a substring match — is dropped. + * {@code eq}/{@code ne}/{@code in} survive because the member symbol encodes to + * its integer before it reaches SQL; {@code like} has no such encoding, and an + * unencoded {@code LIKE 'DRAFT'} against an integer column is a request-time + * type error.

      + * + *

      {@link #opsForSubType} cannot express this — it only ever sees the subtype + * {@code "enum"} — and is deliberately left unchanged for the one caller that + * genuinely has no field: {@code ExpressionAttribute}'s declared operand type.

      + * + *

      ADR-0039: {@code hasMetaAttr(String)} defaults to {@code includeParentData = + * true}, so this read RESOLVES through {@code extends}. That is load-bearing, not + * incidental: post-#246 the map lives on a shared root-level abstract declaration + * and consuming fields INHERIT it, so an own-only read would see it absent on + * exactly the shape adopters are steered toward and wrongly keep {@code like}.

      + * + *

      Cross-port: {@code fixtures/conformance/filter-ops-matrix} pins + * {@code fEnum} vs {@code fEnumInt} in all five ports.

      + */ + public static Set opsForField(com.metaobjects.field.MetaField field) { + if (field == null) return Collections.emptySet(); + if (EnumField.SUBTYPE_ENUM.equals(field.getSubType()) + && field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) { + return OPS_ENUM_INT_BACKED; + } + return opsForSubType(field.getSubType()); + } } diff --git a/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java b/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java index 1e73b0232..f495fa7d7 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java +++ b/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java @@ -149,7 +149,7 @@ private static void runOperation(MetaDataLoader loader, JsonObject op, } MetaField field = (MetaField) target; List got = new ArrayList<>( - com.metaobjects.query.FilterOps.opsForSubType(field.getSubType())); + com.metaobjects.query.FilterOps.opsForField(field)); assertStringNames(got, expect, index, invoke, failures); return; } diff --git a/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py b/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py index 6510ffcee..f7dd76b40 100644 --- a/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py +++ b/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py @@ -53,6 +53,11 @@ # uuid: identity-comparison only — no `like` (not a substring type), no ordering. _OPS_UUID: tuple[str, ...] = ("eq", "ne", "in", "isNull") _OPS_NUMERIC: tuple[str, ...] = ("eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull") +# int-backed enum (@intValueMap, design D5): the enum band minus `like` — the column +# is an integer, and a substring match against an integer is meaningless. Distinct +# constant from _OPS_UUID despite the identical members: they narrow for different +# reasons and would diverge independently. +_OPS_ENUM_INT_BACKED: tuple[str, ...] = ("eq", "ne", "in", "isNull") _OPS_BOOLEAN: tuple[str, ...] = ("eq", "isNull") @@ -118,6 +123,36 @@ def ops_for_subtype_ordered(sub_type: str | None) -> tuple[str, ...]: return () +def ops_for_field_ordered(field: MetaField) -> tuple[str, ...]: + """The operator tuple for a FIELD, in canonical operator order. + + The ordered counterpart of ``validation_passes.ops_for_field``, and the entry + point for every consumer that has a field in hand: the codegen allowlist emit + and the cross-port ``field.filter-ops`` conformance capability. + + Identical to :func:`ops_for_subtype_ordered` except for ONE case: an int-backed + ``field.enum`` (one declaring ``@intValueMap``, design D5) persists as an INTEGER + column, so ``like`` -- a substring match -- is dropped. ``eq``/``ne``/``in`` + survive because the member symbol encodes to its integer before it reaches SQL; + ``like`` has no such encoding, and an unencoded ``LIKE 'DRAFT'`` against an integer + column is a request-time type error. + + ADR-0039: the ``@intValueMap`` read is RESOLVING (``attrs()``, NOT ``attr()`` -- + Python inverts the TS naming, ``attr()`` here is own-only). Post-#246 the map lives + on a shared root-level abstract declaration and consuming fields INHERIT it, so an + own-only read would see it absent on exactly the shape adopters are steered toward + and wrongly keep ``like``. + + Cross-port: ``fixtures/conformance/filter-ops-matrix`` pins ``fEnum`` vs + ``fEnumInt`` in all five ports. + """ + if field.sub_type == fc.FIELD_SUBTYPE_ENUM and isinstance( + field.attrs().get(fc.FIELD_ATTR_INT_VALUE_MAP), dict + ): + return _OPS_ENUM_INT_BACKED + return ops_for_subtype_ordered(field.sub_type) + + def _is_filterable(field: MetaField) -> bool: """True iff ``field`` carries ``@filterable: true`` as a metadata attribute. @@ -166,10 +201,10 @@ class FilterAllowlistGenerator: name = "filter-allowlist-generator" def _ops_for_field(self, field: MetaField) -> tuple[str, ...]: - """The operator tuple allowed for ``field`` (FR-009 §5 per-subtype matrix by - default). Override to customize the per-field operator vocabulary; returning - an empty tuple excludes the field from the allowlist.""" - return ops_for_subtype_ordered(field.sub_type) + """The operator tuple allowed for ``field`` (FR-009 §5 matrix by default). + Override to customize the per-field operator vocabulary; returning an empty + tuple excludes the field from the allowlist.""" + return ops_for_field_ordered(field) def _compute_filterable_ops( self, entity: MetaObject, object_index: dict[str, MetaObject] | None = None diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 02b65d64a..1c67ac431 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -1036,6 +1036,45 @@ def ops_for_subtype(field_subtype: str) -> frozenset[str]: return frozenset() +# The int-backed-enum band: the enum band minus `like`. Named so the narrowing is +# one constant rather than a set rebuilt at every call. +_OPS_ENUM_INT_BACKED: frozenset[str] = frozenset({"eq", "ne", "in", "isNull"}) + + +def ops_for_field(field: MetaData) -> frozenset[str]: + """The filter-operator band for a FIELD. + + The entry point every consumer that has a field in hand must use (loader + validation, the codegen filter-allowlist generator, the cross-port + ``field.filter-ops`` capability). + + Identical to :func:`ops_for_subtype` except for ONE case: an int-backed + ``field.enum`` (one declaring ``@intValueMap``, design D5) persists as an + INTEGER column, so ``like`` -- a substring match -- is dropped. + ``eq``/``ne``/``in`` survive because the member symbol encodes to its integer + before it reaches SQL; ``like`` has no such encoding, and an unencoded + ``LIKE 'DRAFT'`` against an integer column is a request-time type error. + + ``ops_for_subtype`` cannot express this -- it only ever sees the subtype + ``"enum"`` -- and is deliberately left unchanged for the one caller that + genuinely has no field: the expression grammar's declared operand type. + + ADR-0039: the ``@intValueMap`` read is RESOLVING (``attrs()``, NOT ``attr()`` + -- Python inverts the TS naming, ``attr()`` here is own-only). Post-#246 the + map lives on a shared root-level abstract declaration and consuming fields + INHERIT it, so an own-only read would see it absent on exactly the shape + adopters are steered toward and wrongly keep ``like``. + + Cross-port: ``fixtures/conformance/filter-ops-matrix`` pins ``fEnum`` vs + ``fEnumInt`` in all five ports. + """ + if field.sub_type == FIELD_SUBTYPE_ENUM and isinstance( + field.attrs().get(FIELD_ATTR_INT_VALUE_MAP), dict + ): + return _OPS_ENUM_INT_BACKED + return ops_for_subtype(field.sub_type) + + # --------------------------------------------------------------------------- # #195 — attr.expression closed grammar (validate + infer) # --------------------------------------------------------------------------- @@ -1218,8 +1257,10 @@ def _validate_datagrid_filter_values( continue # Build filterable map: field_name → allowed ops set + # ops_for_field, not ops_for_subtype — an int-backed field.enum + # (@intValueMap) stores as an integer, so `like` is not in its band. filterable: dict[str, frozenset[str]] = { - f.name: ops_for_subtype(f.sub_type) + f.name: ops_for_field(f) for f in node.fields() if f.attrs().get("filterable") is True } diff --git a/server/python/tests/conformance/capabilities.py b/server/python/tests/conformance/capabilities.py index b03d96d51..8bf36e556 100644 --- a/server/python/tests/conformance/capabilities.py +++ b/server/python/tests/conformance/capabilities.py @@ -52,15 +52,19 @@ def invoke(node: MetaData, capability: str, args: dict[str, Any]) -> dict[str, A return {"subtype": identity.sub_type} if capability == "field.filter-ops": - # The canonical per-subtype filter-operator band (in canonical operator + # The canonical per-FIELD filter-operator band (in canonical operator # order). Single source of truth — the same ordered helper the codegen # filter-allowlist generator consumes. Returns {"names": [...]}. + # + # ops_for_field_ordered, not ops_for_subtype_ordered: the band is + # field-level because an int-backed field.enum (@intValueMap) stores as + # an integer and so drops `like`. from metaobjects.meta.core.field.meta_field import MetaField from metaobjects.codegen.generators.filter_allowlist_generator import ( - ops_for_subtype_ordered, + ops_for_field_ordered, ) if not isinstance(node, MetaField): raise TypeError(f"field.filter-ops requires a MetaField, got {type(node)}") - return {"names": list(ops_for_subtype_ordered(node.sub_type))} + return {"names": list(ops_for_field_ordered(node))} raise ValueError(f"Unknown capability: {capability!r}") diff --git a/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts b/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts index 03883f01f..640e42293 100644 --- a/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts +++ b/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts @@ -13,7 +13,7 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_CURRENCY, - opsForSubType, + opsForField, } from "@metaobjectsdev/metadata"; import { sortableFields } from "./filter-shared.js"; import type { RenderContext } from "../render-context.js"; @@ -72,7 +72,9 @@ export const ${entity.name}FilterAllowlist = {} as const satisfies FilterAllowli } const rows = fields .map((f) => { - const ops = opsForSubType(f.subType).map((o) => JSON.stringify(o)).join(", "); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` (a substring match) is not in its band. + const ops = opsForField(f).map((o) => JSON.stringify(o)).join(", "); const sub = filterSubTypeFor(f.subType); // Only field.timestamp is governed by timestampMode — Drizzle types // field.date / field.time as strings under every dialect. diff --git a/server/typescript/packages/codegen-ts/src/templates/filter-type.ts b/server/typescript/packages/codegen-ts/src/templates/filter-type.ts index 148a8d58b..b5744711c 100644 --- a/server/typescript/packages/codegen-ts/src/templates/filter-type.ts +++ b/server/typescript/packages/codegen-ts/src/templates/filter-type.ts @@ -11,7 +11,7 @@ import { FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, - opsForSubType, + opsForField, } from "@metaobjectsdev/metadata"; import { isSortableField } from "./filter-shared.js"; @@ -34,7 +34,11 @@ function tsNameFor(fieldSubType: string): string { } function renderFieldUnion(field: MetaField): string { - const ops = opsForSubType(field.subType); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) stores + // as an integer, so `like` is not in its band. The client type and the server + // allowlist MUST agree: offering `like` here that the allowlist 400s is a + // client/server mismatch of exactly the kind filter-shared.ts exists to prevent. + const ops = opsForField(field); const tsName = tsNameFor(field.subType); const opEntries = ops.map((op) => { if (op === "in") return `in?: ${tsName}[]`; diff --git a/server/typescript/packages/metadata/src/core/query/query-constants.ts b/server/typescript/packages/metadata/src/core/query/query-constants.ts index 3f6e2d110..8eba954b2 100644 --- a/server/typescript/packages/metadata/src/core/query/query-constants.ts +++ b/server/typescript/packages/metadata/src/core/query/query-constants.ts @@ -1,4 +1,4 @@ -import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET } from "../field/field-constants.js"; +import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_ATTR_INT_VALUE_MAP } from "../field/field-constants.js"; // Query concern constants — filter operators, sort order values. // @@ -67,6 +67,52 @@ export function opsForSubType(subType: string): readonly FilterOp[] { return OPS_BY_SUBTYPE[subType] ?? []; } +/** The int-backed-enum band: the enum band minus `like`. Hoisted so the narrowing + * is one named constant rather than a filter re-derived at every call. */ +const OPS_ENUM_INT_BACKED: readonly FilterOp[] = [ + FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL, +]; + +/** + * The structural shape `opsForField` needs. Declared here rather than importing + * `MetaField`: `query-constants` is foundational and `core/field` imports it, so a + * type import back the other way would close a cycle. + */ +export interface FilterOpBandField { + readonly subType: string; + attr(name: string): unknown; +} + +/** + * The filter-operator band for a FIELD — the entry point every consumer that has a + * field in hand must use (loader validation, generated allowlists, generated client + * filter types, the cross-port `field.filter-ops` capability). + * + * Identical to {@link opsForSubType} except for ONE case: an int-backed `field.enum` + * (one declaring `@intValueMap`, design D5) persists as an INTEGER column, so `like` + * — a substring match — is dropped. `eq`/`ne`/`in` survive because the member symbol + * encodes to its integer before it reaches SQL; `like` has no such encoding, and an + * unencoded `LIKE 'DRAFT'` against an integer column is a request-time type error. + * + * `opsForSubType` cannot express this: it only ever sees the subtype `"enum"`. It is + * deliberately left unchanged for the one caller that genuinely has no field — the + * expression grammar's declared operand type. + * + * ADR-0039: the `@intValueMap` read is RESOLVING. Post-#246 the map lives on a shared + * root-level abstract declaration and consuming fields INHERIT it, so an own-only read + * would see `undefined` on exactly the shape adopters are steered toward and wrongly + * keep `like` in the band. + */ +export function opsForField(field: FilterOpBandField): readonly FilterOp[] { + if (field.subType === FIELD_SUBTYPE_ENUM && isIntBacked(field)) return OPS_ENUM_INT_BACKED; + return opsForSubType(field.subType); +} + +function isIntBacked(field: FilterOpBandField): boolean { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + return raw !== undefined && raw !== null && typeof raw === "object"; +} + // --------------------------------------------------------------------------- // Sort order values (used by @sortableDefaultOrder on fields and // @defaultSortOrder on dataGrid layouts) diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index e0c0e73f2..12ff01248 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -130,6 +130,7 @@ import { FILTER_COMPOSE_OR, FILTER_COMPOSE_AND, opsForSubType, + opsForField, } from "../core/query/query-constants.js"; // --------------------------------------------------------------------------- @@ -1684,7 +1685,9 @@ export function validateDataGridFilterValues(root: MetaData): ParseError[] { for (const f of effective.filter((c) => c.type === TYPE_FIELD)) { // ADR-0039: resolving — a concrete field may inherit @filterable via extends. if (f.attr(FIELD_ATTR_FILTERABLE) === true) { - allow.set(f.name, opsForSubType(f.subType)); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. + allow.set(f.name, opsForField(f)); } } for (const layout of effective.filter( @@ -2034,7 +2037,9 @@ export function validateProjectionFilter(root: MetaData): ParseError[] { origin !== undefined && origin.subType !== ORIGIN_SUBTYPE_PASSTHROUGH && origin.subType !== ORIGIN_SUBTYPE_COMPUTED; - fields.set(f.name, { derived, ops: opsForSubType(f.subType) }); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. + fields.set(f.name, { derived, ops: opsForField(f) }); } checkProjectionFilterRefs(filter as Record, fields, obj.name, obj.source, errors); } diff --git a/server/typescript/packages/metadata/test/conformance/binding.ts b/server/typescript/packages/metadata/test/conformance/binding.ts index 343587149..dd8098e91 100644 --- a/server/typescript/packages/metadata/test/conformance/binding.ts +++ b/server/typescript/packages/metadata/test/conformance/binding.ts @@ -12,7 +12,7 @@ import type { MetaData } from "../../src/shared/meta-data.js"; import { MetaObject } from "../../src/core/object/meta-object.js"; import { MetaField } from "../../src/core/field/meta-field.js"; import { canonicalSerialize } from "../../src/serializer-json.js"; -import { opsForSubType } from "../../src/core/query/query-constants.js"; +import { opsForField } from "../../src/core/query/query-constants.js"; type CapabilityArgs = Record; type CapabilityFn = (node: MetaData, args: CapabilityArgs) => NormalizedResult; @@ -85,12 +85,14 @@ export const binding: Readonly> = { "effective-tree": canonicalSerialize(asField(node)), }), - // field.filter-ops → the canonical per-subtype filter-operator band - // (query-constants OPS_BY_SUBTYPE). Returns `{ names: [...] }` in canonical - // operator order so the cross-port matrix fixture compares order-sensitively. - // The single source of truth for the band is opsForSubType — the same map - // the server allowlist + codegen consume. + // field.filter-ops → the canonical per-FIELD filter-operator band. Returns + // `{ names: [...] }` in canonical operator order so the cross-port matrix + // fixture compares order-sensitively. The single source of truth for the band + // is opsForField — the same function the server allowlist + codegen consume. + // + // opsForField, not opsForSubType: the band is field-level because an int-backed + // field.enum (@intValueMap) stores as an integer and so drops `like`. "field.filter-ops": (node) => ({ - names: [...opsForSubType(asField(node).subType)], + names: [...opsForField(asField(node))], }), }; diff --git a/server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts b/server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts new file mode 100644 index 000000000..077bc721e --- /dev/null +++ b/server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts @@ -0,0 +1,146 @@ +// An INT-BACKED field.enum (@intValueMap, design D5) persists as an INTEGER column. +// `like` is a substring match — meaningless against an integer, and it is the one +// operator in the string/enum band that cannot be made to work by encoding the +// member symbol to its integer (eq/ne/in all can, and do). +// +// So the op band is a property of the FIELD, not of the subtype alone: a +// string-backed enum keeps `like`, an int-backed one does not. `opsForSubType` +// cannot express that — it only ever sees "enum". `opsForField` is the field-level +// entry point every op-band consumer must use when it has a field in hand. +// +// Cross-port: fixtures/conformance/filter-ops-matrix pins fEnum vs fEnumInt in all +// five ports, so this narrowing can never become a TS-only divergence. + +import { describe, test, expect } from "bun:test"; +import { + MetaDataLoader, + InMemoryStringSource, + opsForField, + opsForSubType, + FIELD_SUBTYPE_ENUM, +} from "../src/index.js"; +import type { MetaField } from "../src/index.js"; + +async function loadMatrix(): Promise> { + const json = JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "Matrix", + children: [ + { "source.rdb": { "@table": "matrix" } }, + { "field.long": { name: "id" } }, + // String-backed enum — the control. Keeps `like`. + { + "field.enum": { + name: "strEnum", + "@values": ["DRAFT", "PUBLISHED"], + "@filterable": true, + }, + }, + // Int-backed enum — declares @intValueMap, so it stores as integer. + { + "field.enum": { + name: "intEnum", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5 }, + "@filterable": true, + }, + }, + { "field.string": { name: "title", "@filterable": true } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(errors).toEqual([]); + const entity = root.objects()[0]!; + const out = new Map(); + for (const f of entity.fields()) out.set(f.name, f); + return out; +} + +describe("opsForField — int-backed enum drops `like`", () => { + test("a string-backed enum keeps the full string band", async () => { + const fields = await loadMatrix(); + expect([...opsForField(fields.get("strEnum")!)]).toEqual([ + "eq", + "ne", + "in", + "like", + "isNull", + ]); + }); + + test("an int-backed enum drops `like`, keeping eq/ne/in/isNull", async () => { + const fields = await loadMatrix(); + expect([...opsForField(fields.get("intEnum")!)]).toEqual(["eq", "ne", "in", "isNull"]); + }); + + test("a plain string is untouched by the narrowing", async () => { + const fields = await loadMatrix(); + expect([...opsForField(fields.get("title")!)]).toEqual( + [...opsForSubType("string")], + ); + }); + + test("opsForSubType(enum) is unchanged — the narrowing is field-level only", () => { + // The subtype-keyed band stays the string band. Callers that only have a + // subtype string (the expression grammar's declared operand type) are + // deliberately unaffected. + expect([...opsForSubType(FIELD_SUBTYPE_ENUM)]).toEqual([ + "eq", + "ne", + "in", + "like", + "isNull", + ]); + }); + + test("the map is read RESOLVING — an inherited @intValueMap narrows too", async () => { + // Post-#246 this is the CANONICAL authoring shape: the map lives on the shared + // root-level abstract declaration and consuming fields inherit it. An own-only + // read would see undefined here and wrongly keep `like`. + const json = JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { + "field.enum": { + name: "SharedStatus", + abstract: true, + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5 }, + }, + }, + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { name: "id" } }, + { + "field.enum": { + name: "status", + extends: "acme::SharedStatus", + "@filterable": true, + }, + }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(errors).toEqual([]); + const status = root.objects()[0]!.fields().find((f) => f.name === "status")!; + expect([...opsForField(status)]).toEqual(["eq", "ne", "in", "isNull"]); + }); +}); From 19e03d6bdc1b692dd466f4e22f589596d2d5b71d Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 14 Aug 2026 19:18:42 -0400 Subject: [PATCH 29/52] fix(codegen-ts): a view @filter on an int-backed enum lowers to the INTEGER literal A projection row-scope `@filter` (#207) and an `origin.aggregate @filter` both render as literal SQL text, so neither touches Drizzle -- the customType that rescues the runtime query path does nothing for them. On an int-backed `field.enum` (`@intValueMap`, design D5) they emitted the member SYMBOL against an `integer` column: WHERE p.status = 'PUBLISHED' Postgres rejects that at CREATE VIEW time (`invalid input syntax for type integer`), which aborts the whole migration -- so this was a `meta migrate` blocker, not a wrong-rows bug, and it applied to every operator rather than just the `like` case the plan anticipated. Both resolvers now map the member through the RESOLVING `@intValueMap` (ADR-0039 -- post-#246 the map commonly lives on a shared abstract declaration two hops up, and the projection's own field reaches the base entity's through `extends`, so own-only at either hop would silently emit the symbol). `in` encodes element-wise; `isNull` is skipped (its value is a boolean); `like` throws rather than emitting `LIKE NaN` -- unreachable now that `opsForField` removes it from the band, so the throw is a loud backstop, as is the unmapped-member throw (the key set is loader-pinned equal to `@values`). `resolveAggregateFilter` is a SEPARATE resolver from `resolveViewFilter` and would not have been reached by fixing only the one the plan named -- the same "assume a sibling code path exists" miss as the sqlite arm of column-mapper. Same defect class as Task 7's `@default` (`DEFAULT 'DRAFT'` on an integer column): wherever metadata puts an enum member into SQL text, the member has to go through @intValueMap first. Gated by 10 unit tests (incl. the DRAFT->0 falsy case, which a truthiness-guarded encode would silently skip -- #235 is the precedent) AND a real-Postgres test that APPLIES the view, converges a second migrate, and asserts the view returns only the mapped rows. A unit assertion on the emitted string cannot show that the DDL applies; only applying it can. Verified load-bearing by disabling the encode and confirming both real-PG tests go red. Co-Authored-By: Claude Opus 5 (1M context) --- .../src/projection/extract-view-spec.ts | 92 ++++++- .../view-level-filter-int-enum.test.ts | 259 ++++++++++++++++++ .../test/enum-intvaluemap-pg.test.ts | 97 ++++++- 3 files changed, 443 insertions(+), 5 deletions(-) create mode 100644 server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts index 2ec52bcb0..3654d3a52 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts @@ -32,6 +32,8 @@ import { FILTER_OP_LT, FILTER_OP_LTE, FILTER_OP_IS_NULL, + FILTER_OP_LIKE, + FIELD_SUBTYPE_ENUM, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, SORT_ORDER_DESC, @@ -47,6 +49,7 @@ import { type AggregateFunction, } from "@metaobjectsdev/metadata"; import { type MetaData, type MetaField, type MetaRoot, MetaObject } from "@metaobjectsdev/metadata"; +import { intValueMapOf } from "../enum-meta.js"; import { columnNameFromField, viewNameFromProjection, @@ -126,13 +129,84 @@ function resolveAggregateFilter( kind: "cmp", ref: `${alias}.${sourceColumnNameFor(field, ctx)}`, op, - value: opObj[op], + // Same int-backed-enum encoding as the row-scope @filter below: this scoping + // filter renders as a SQL literal too (FILTER (WHERE …) / CASE WHEN), so a + // member symbol would land unencoded in an integer comparison. + value: encodeIntEnumFilterValue( + opObj[op], + op, + field.subType === FIELD_SUBTYPE_ENUM ? intValueMapOf(field) : undefined, + key, + entity.name, + ), }); } if (clauses.length === 0) return undefined; return clauses.length === 1 ? clauses[0]! : { kind: "and", clauses }; } +/** + * The `@intValueMap` of every int-backed `field.enum` the projection declares, keyed + * by field name. Only int-backed enums appear, so a lookup miss means "no encoding". + * + * `fields()` (effective) and `intValueMapOf` (which reads `attr`, RESOLVING) — a + * projection's fields are bound through `extends` to the base entity's, and post-#246 + * the map itself commonly lives one hop further up on a shared abstract declaration. + * Own-only at either hop would silently emit the member symbol into an integer column + * (ADR-0039). + */ +function intEnumMapsOf(projection: MetaObject): ReadonlyMap> { + const out = new Map>(); + for (const f of projection.fields()) { + if (f.subType !== FIELD_SUBTYPE_ENUM) continue; + const map = intValueMapOf(f); + if (map !== undefined) out.set(f.name, map); + } + return out; +} + +/** + * Lower a filter value for an int-backed `field.enum` from its member SYMBOL to the + * INTEGER it persists as. A no-op for every other field (`intMap` undefined), so a + * string-backed enum's SQL is byte-identical. + * + * `isNull` is skipped — its value is a boolean, not a member. `like` is unreachable: + * `opsForField` removes it from an int-backed enum's band, so the loader rejects it + * before codegen; the explicit throw makes that a loud failure rather than a + * `LIKE NaN`. An unmapped member is likewise loader-unreachable (the key set is + * pinned equal to `@values`) and throws for the same reason — silently emitting the + * symbol would produce DDL that fails only at apply time, against a live database. + */ +function encodeIntEnumFilterValue( + value: unknown, + op: string, + intMap: Record | undefined, + fieldName: string, + projectionName: string, +): unknown { + if (intMap === undefined) return value; + if (op === FILTER_OP_IS_NULL) return value; + if (op === FILTER_OP_LIKE) { + throw new Error( + `Projection ${projectionName}: view @filter uses "like" on "${fieldName}", an ` + + `int-backed field.enum (@intValueMap) — it stores as an integer column, so a ` + + `substring match is not expressible. Use eq/ne/in.`, + ); + } + const encode = (v: unknown): unknown => { + if (typeof v !== "string") return v; + const n = intMap[v]; + if (typeof n !== "number") { + throw new Error( + `Projection ${projectionName}: view @filter value "${v}" for "${fieldName}" has no ` + + `entry in @intValueMap.`, + ); + } + return n; + }; + return Array.isArray(value) ? value.map(encode) : encode(value); +} + /** * #207 — resolve a projection's row-scope `@filter` (the desugared canonical * `{ field: { op: value }, and?, or? }`) into a {@link ViewFilterClause} whose @@ -152,13 +226,14 @@ function resolveViewFilter( filter: unknown, columnsByField: ReadonlyMap, projectionName: string, + intMapsByField: ReadonlyMap>, ): ViewFilterClause | undefined { if (typeof filter !== "object" || filter === null || Array.isArray(filter)) return undefined; const clauses: ViewFilterClause[] = []; for (const [key, val] of Object.entries(filter as Record)) { if (key === FILTER_AND || key === FILTER_OR) { const subs = (Array.isArray(val) ? val : []) - .map((s) => resolveViewFilter(s, columnsByField, projectionName)) + .map((s) => resolveViewFilter(s, columnsByField, projectionName, intMapsByField)) .filter((c): c is ViewFilterClause => c !== undefined); if (subs.length > 0) clauses.push({ kind: key === FILTER_AND ? "and" : "or", clauses: subs }); continue; @@ -177,7 +252,14 @@ function resolveViewFilter( // becomes its own comparison, AND-composed (dropping all-but-the-first would silently // widen the exposed row set). The loader has already validated every op for this // field's subtype. - for (const [op, value] of Object.entries(desugarClause(val))) { + for (const [op, rawValue] of Object.entries(desugarClause(val))) { + // An INT-BACKED field.enum (@intValueMap, design D5) stores as an INTEGER + // column, so the authored member SYMBOL must become its integer before it is + // rendered as a SQL literal. The Drizzle customType handles the runtime query + // path, but view DDL is emitted as literal SQL text and never touches Drizzle. + const value = encodeIntEnumFilterValue( + rawValue, op, intMapsByField.get(key), key, projectionName, + ); if (col.kind === "passthrough") { clauses.push({ kind: "cmp", ref: `${col.sourceAlias}.${col.sourceColumn}`, op, value }); } else if (col.kind === "computed") { @@ -990,7 +1072,9 @@ export function extractViewSpec( const columnsByField = new Map( selectSpec.columns.map((c) => [c.fieldName, c] as const), ); - where = resolveViewFilter(rawFilter, columnsByField, projection.name); + where = resolveViewFilter( + rawFilter, columnsByField, projection.name, intEnumMapsOf(projection), + ); } } diff --git a/server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts b/server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts new file mode 100644 index 000000000..a4ba9e58b --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts @@ -0,0 +1,259 @@ +// A projection row-scope `@filter` (#207) on an INT-BACKED field.enum (@intValueMap, +// design D5) must lower to the INTEGER literal, not the member symbol. +// +// The column is `integer`, so `WHERE p.status = 'PUBLISHED'` is not merely wrong rows — +// on Postgres it is `invalid input syntax for type integer`, raised at CREATE VIEW time, +// which aborts the whole migration. This is the same defect class as Task 7's `@default` +// (`DEFAULT 'DRAFT'` on an integer column): everywhere metadata puts an enum member into +// SQL, the member has to go through @intValueMap first. +// +// The Drizzle customType covers the RUNTIME query path for free, but view DDL is emitted +// as literal SQL text and never touches Drizzle — so it needs its own encoding. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { extractViewSpec } from "../../src/projection/extract-view-spec.js"; +import { emitViewDdl } from "../../src/projection/view-ddl-emit.js"; +import { buildProjectionViews } from "../../src/projection/build-projection-views.js"; + +type CodedError = Error & { readonly code?: string }; +const codeOf = (e: Error): string | undefined => (e as CodedError).code; + +async function load(children: unknown[]) { + const json = JSON.stringify({ "metadata.root": { package: "test", children } }); + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error( + `Loader errors:\n${result.errors.map((e) => `${codeOf(e)}: ${e.message}`).join("\n")}`, + ); + } + return result.root; +} + +/** `Program` with one int-backed enum + one string-backed enum (the control). */ +function programEntity() { + return { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { + "field.enum": { + name: "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }, + }, + }, + { "field.enum": { name: "tier", "@values": ["FREE", "PAID"] } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }; +} + +function projection(name: string, view: string, filter: unknown) { + return { + "object.projection": { + name, + "@filter": filter, + children: [ + { "source.rdb": { "@kind": "view", "@view": view } }, + { "field.int": { name: "id", extends: "test::Program.id" } }, + { "field.enum": { name: "status", extends: "test::Program.status" } }, + { "field.enum": { name: "tier", extends: "test::Program.tier" } }, + { "identity.primary": { extends: "test::Program.id" } }, + ], + }, + }; +} + +async function sqlFor(filter: unknown): Promise { + const root = await load([programEntity(), projection("P", "v_p", filter)]); + const proj = root.objects().find((o) => o.name === "P")!; + const spec = extractViewSpec(proj, root, { columnNamingStrategy: "snake_case" }); + return emitViewDdl(spec, { + dialect: "postgres", + baseTableName: "programs", + joinTables: {}, + bodyOnly: true, + }); +} + +describe("#207 view @filter on an int-backed enum lowers to the integer", () => { + test("eq encodes the member symbol to its integer", async () => { + const sql = await sqlFor({ status: { eq: "PUBLISHED" } }); + expect(sql).toContain("WHERE p.status = 5"); + // The member symbol must NOT survive into the SQL. + expect(sql).not.toContain("'PUBLISHED'"); + }); + + test("ne encodes too", async () => { + const sql = await sqlFor({ status: { ne: "ARCHIVED" } }); + expect(sql).toContain("WHERE p.status <> 9"); + }); + + test("in encodes element-wise", async () => { + const sql = await sqlFor({ status: { in: ["DRAFT", "ARCHIVED"] } }); + expect(sql).toContain("WHERE p.status IN (0, 9)"); + }); + + test("the zero member encodes as 0, not dropped as falsy", async () => { + // DRAFT → 0. A truthiness-guarded encode would leave 'DRAFT' here; #235 is the + // precedent for a falsy-value bug in exactly this shape. + const sql = await sqlFor({ status: { eq: "DRAFT" } }); + expect(sql).toContain("WHERE p.status = 0"); + }); + + test("isNull is untouched by the encoding", async () => { + const sql = await sqlFor({ status: { isNull: true } }); + expect(sql).toContain("WHERE p.status IS NULL"); + }); + + test("a STRING-backed enum is byte-identical to before (quoted symbol)", async () => { + const sql = await sqlFor({ tier: { eq: "PAID" } }); + expect(sql).toContain("WHERE p.tier = 'PAID'"); + }); + + test("a composed and/or filter encodes inside both arms", async () => { + const sql = await sqlFor({ + or: [{ status: { eq: "DRAFT" } }, { status: { eq: "PUBLISHED" } }], + }); + expect(sql).toContain("(p.status = 0 OR p.status = 5)"); + }); + + test("the map is read RESOLVING — an inherited @intValueMap still encodes", async () => { + // The canonical post-#246 shape: the map on a shared root-level abstract enum, + // inherited by the consuming field. An own-only read would emit 'PUBLISHED'. + const root = await load([ + { + "field.enum": { + name: "SharedStatus", + abstract: true, + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5 }, + }, + }, + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { "field.enum": { name: "status", extends: "test::SharedStatus" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + { + "object.projection": { + name: "P", + "@filter": { status: { eq: "PUBLISHED" } }, + children: [ + { "source.rdb": { "@kind": "view", "@view": "v_p" } }, + { "field.int": { name: "id", extends: "test::Program.id" } }, + { "field.enum": { name: "status", extends: "test::Program.status" } }, + { "identity.primary": { extends: "test::Program.id" } }, + ], + }, + }, + ]); + const proj = root.objects().find((o) => o.name === "P")!; + const spec = extractViewSpec(proj, root, { columnNamingStrategy: "snake_case" }); + const sql = emitViewDdl(spec, { + dialect: "postgres", + baseTableName: "programs", + joinTables: {}, + bodyOnly: true, + }); + expect(sql).toContain("WHERE p.status = 5"); + expect(sql).not.toContain("'PUBLISHED'"); + }); +}); + +// The `origin.aggregate @filter` scoping filter is a SEPARATE resolver +// (resolveAggregateFilter) from the row-scope one, and renders as a SQL literal too — +// `FILTER (WHERE …)` on Postgres, `CASE WHEN` on SQLite. Assume a sibling code path +// exists: the row-scope fix above does not reach this one. +describe("origin.aggregate @filter on an int-backed enum lowers to the integer", () => { + const model = () => [ + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { + "relationship.association": { + name: "weeks", "@objectRef": "Week", "@cardinality": "many", + }, + }, + ], + }, + }, + { + "object.entity": { + name: "Week", + children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.int": { name: "id" } }, + { "field.int": { name: "programId" } }, + { "field.int": { name: "ordinal" } }, + { + "field.enum": { + name: "status", + "@values": ["DRAFT", "ACTIVE"], + "@intValueMap": { DRAFT: 0, ACTIVE: 7 }, + }, + }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { + "identity.reference": { + name: "ref_program", "@fields": "programId", "@references": "Program", + }, + }, + ], + }, + }, + { + "object.projection": { + name: "ProgramSummary", + children: [ + { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, + { "field.int": { name: "id", extends: "Program.id" } }, + { + "field.int": { + name: "activeMax", + children: [{ + "origin.aggregate": { + "@agg": "max", "@of": "Week.ordinal", "@via": "Program.weeks", + "@filter": { status: { eq: "ACTIVE" } }, + }, + }], + }, + }, + { "identity.primary": { name: "id", extends: "Program.id" } }, + ], + }, + }, + ]; + + test("postgres FILTER (WHERE …) compares the integer", async () => { + const root = await load(model()); + const [v] = buildProjectionViews(root, { + dialect: "postgres", columnNamingStrategy: "snake_case", + }); + expect(v!.sql).toMatch(/FILTER \(WHERE [a-z0-9]+\.status = 7\)/); + expect(v!.sql).not.toContain("'ACTIVE'"); + }); + + test("sqlite CASE WHEN compares the integer", async () => { + const root = await load(model()); + const [v] = buildProjectionViews(root, { + dialect: "sqlite", columnNamingStrategy: "snake_case", + }); + expect(v!.sql).toMatch(/CASE WHEN [a-z0-9]+\.status = 7 THEN/); + expect(v!.sql).not.toContain("'ACTIVE'"); + }); +}); diff --git a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts index efde60be9..f2845688e 100644 --- a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts @@ -25,7 +25,7 @@ import { type SchemaSnapshot, } from "@metaobjectsdev/migrate-ts"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; -import { runGen, defineConfig } from "@metaobjectsdev/codegen-ts"; +import { runGen, defineConfig, buildProjectionViews } from "@metaobjectsdev/codegen-ts"; import { entityFile } from "@metaobjectsdev/codegen-ts/generators"; import { Kysely, PostgresDialect, sql } from "kysely"; import pg, { Pool } from "pg"; @@ -347,3 +347,98 @@ describe("int-backed field.enum — generated codec against real Postgres", () = expect(threw).toBe(true); }, 120_000); }); + +/** + * A projection row-scope `@filter` (#207) on an int-backed enum. + * + * The view body is emitted as LITERAL SQL text and never touches Drizzle, so the + * customType that rescues the runtime query path does nothing here. Before the fix + * this emitted `WHERE p.status = 'PUBLISHED'` against an `integer` column, which + * Postgres rejects at CREATE VIEW time — `invalid input syntax for type integer` — + * aborting the migration. A unit assertion on the emitted string cannot show that; + * only applying it can. + */ +describe("int-backed field.enum in a projection view — real Postgres", () => { + function metaWithView(): string { + return `{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.enum": { "name": "status", "extends": "Status", "@required": true } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.projection": { "name": "PublishedOrders", + "@filter": { "status": { "eq": "PUBLISHED" } }, "children": [ + { "source.rdb": { "@kind": "view", "@table": "v_published_orders" } }, + { "identity.primary": { "name": "id", "extends": "Order.id", "@fields": "id" } }, + { "field.long": { "name": "id", "extends": "Order.id", "children": [ + { "origin.passthrough": { "@from": "Order.id" } } ] } }, + { "field.string": { "name": "title", "children": [ + { "origin.passthrough": { "@from": "Order.title" } } ] } }, + { "field.enum": { "name": "status", "extends": "Status", "children": [ + { "origin.passthrough": { "@from": "Order.status" } } ] } } + ] } } + ] + } + }`; + } + + /** Views are NOT derived by buildExpectedSchema — they must be passed in + * explicitly, so this block needs its own migrate helper rather than the + * table-only one above. */ + async function migrateWithViews(metaJson: string): Promise { + const root = (await new MetaDataLoader().load([new InMemoryStringSource(metaJson)])).root; + const expected = buildExpectedSchema(root, { + columnNamingStrategy: "literal", + dialect: "postgres", + views: buildProjectionViews(root, { + dialect: "postgres", columnNamingStrategy: "literal", + }), + }); + const result = await diff({ + expected, actual: await introspectPostgres(k), dialect: "postgres", + }); + expect(result.blocked).toEqual([]); + const { up } = result.changes.length === 0 + ? { up: "" } + : emit(result.changes, { dialect: "postgres" }); + if (up.trim().length > 0) await applyRaw(up); + return expected; + } + + test("the filtered view APPLIES, converges, and selects by the INTEGER", async () => { + const expected = await migrateWithViews(metaWithView()); + // The view exists — i.e. the CREATE VIEW did not blow up on a text-vs-integer + // comparison. This is the assertion the whole test exists for. + await assertConverged(expected); + + await sql.raw( + `INSERT INTO "orders" ("title", "status") VALUES ('a', 0), ('b', 5), ('c', 9), ('d', 5);`, + ).execute(k); + + const rows = await sql<{ title: string }>` + SELECT "title" FROM "v_published_orders" ORDER BY "title" + `.execute(k); + // Only the two PUBLISHED (5) rows — proving the WHERE compared 5, not 'PUBLISHED'. + expect(rows.rows.map((r) => r.title)).toEqual(["b", "d"]); + }, 120_000); + + test("the emitted view body carries the integer literal, not the member symbol", async () => { + const root = (await new MetaDataLoader().load([ + new InMemoryStringSource(metaWithView()), + ])).root; + const views = buildProjectionViews(root, { + dialect: "postgres", columnNamingStrategy: "literal", + }); + const body = views.find((v) => v.name === "v_published_orders")?.sql ?? ""; + expect(body).toMatch(/WHERE o\.status = 5/); + expect(body).not.toContain("'PUBLISHED'"); + }, 120_000); +}); From f6044fb50e9f105f111c7f6452c3a8f77d658f17 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Fri, 14 Aug 2026 19:25:14 -0400 Subject: [PATCH 30/52] docs(plan): record the Task 8 outcome -- and the view-DDL blocker it uncovered Task 8's steps 1-3 turned out unnecessary (the Drizzle customType already encodes filter comparison values), while its step 4 -- written as "confirm the existing gating already excludes \`like\`; if so this is a test only" -- was the whole task, and the optimistic reading was wrong. Also records Task 8b, the projection/aggregate view-@filter blocker that probing step 4 surfaced, and folds its durable lesson into the notes the other three port plans will be rewritten from: a column-level codec seam does not reach anywhere a port renders SQL text by hand. Co-Authored-By: Claude Opus 5 (1M context) --- ...3-int-backed-enum-values-ts-persistence.md | 62 ++++++++++++++----- 1 file changed, 46 insertions(+), 16 deletions(-) diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md index da0733f69..0d3163497 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md @@ -726,22 +726,48 @@ git commit -m "test(persistence-conformance): int-backed field.enum round-trips --- -### Task 8: runtime-ts + codegen-ts — the filter path must encode symbol→int - -**Why (verified 2026-08-12):** generated CRUD endpoints filter on `@filterable` fields. `parseFilterParams` coerces by the allowlist rule's `subType` and binds the result (`packages/runtime-ts/src/drizzle-fastify/filter-parser.ts:156-186`, `coerce` at :215). An enum rule coerces as a plain string, so `?filter[status][eq]=DRAFT` binds `'DRAFT'` against an `integer` column — a Postgres type error at request time, and `in` lists likewise. Nothing in Tasks 1-6 touches this. - -**Follow the `dateValues` precedent exactly** (`filter-parser.ts:232-243` + `FilterFieldRule`): codegen already solved this identical problem for Date-typed columns by having the GENERATED allowlist carry a per-column flag the parser honours. Do the same — carry the symbol→int map (or a reference to the generated lookup) on the rule. Do NOT teach the parser to re-derive it from metadata; the parser is metadata-free by design. - -**Files:** -- Modify: `packages/runtime-ts/src/drizzle-fastify/filter-parser.ts` (`FilterFieldRule` + `coerce`) -- Modify: `packages/codegen-ts/src/templates/` — the filter-allowlist emitter -- Test: `packages/runtime-ts/test/` filter-parser unit tests + a codegen allowlist emission test - -- [ ] **Step 1: Failing tests** — (a) parser: a rule carrying an int map coerces `"DRAFT"` → `0` for `eq`/`ne`, and `"DRAFT,PUBLISHED"` → `[0, 5]` for `in`; an unknown member is a `filter.invalid_value` `FilterParseError` naming the field (NOT a silent pass-through, NOT a 500). (b) codegen: the generated `FilterAllowlist` for an int-backed enum field carries the map; a string-backed one is byte-identical to today. -- [ ] **Step 2:** Extend `FilterFieldRule` with the optional map and honour it in `coerce`'s enum path. -- [ ] **Step 3:** Emit it from the allowlist generator, reading `@intValueMap` RESOLVING (Amendment 1). -- [ ] **Step 4:** `isNull` is unaffected (it coerces boolean); `like` must be REJECTED for an int-backed enum — a substring match against an integer column is meaningless. Confirm the existing per-subtype operator gating already excludes `like` for enums; if it does not, that is the fix. -- [ ] **Step 5: Commit** — `fix(runtime-ts,codegen-ts): int-backed enum filters bind the integer, not the member symbol` +### Task 8: runtime-ts + codegen-ts — the filter path must encode symbol→int — **DONE** + +**Outcome (2026-08-14), and it diverged from the plan in both directions.** + +**Steps 1-3 turned out to be unnecessary.** The premise — that `parseFilterParams` binds a +raw member symbol against an integer column — is false once Task 5's Drizzle `customType` +is in the column definition: the comparison value goes through `toDriver` and encodes for +free (empirically confirmed). The `dateValues` precedent was NOT followed; `FilterFieldRule` +is unchanged, and the parser stays metadata-free as intended. + +**Step 4 was the whole task, and its optimistic reading was wrong.** The existing gating +does NOT exclude `like` and structurally cannot: `opsForSubType` is keyed by subtype and +only ever sees `"enum"`, so the generated allowlist offered `like` on an int-backed field +byte-identically to a string-backed one. The band is a property of the FIELD. + +Fixed in `e8dca0b4d` as **one loader rule per port**, not five codegen filters — the #210 / +`@objectRef` precedent — so an authored `attr.filter` / dataGrid `@filter` using `like` on +an int-backed enum fails at LOAD, not later at the SQL layer: + +- TS `opsForField` (query-constants) · Java `FilterOps.opsForField` · C# + `QueryConstants.OpsForField` · Python `ops_for_field` + `ops_for_field_ordered` · Kotlin + reuses the JVM band through its own generator call site. +- `opsForSubType` is deliberately KEPT for the one caller with no field in hand (the + expression grammar's declared operand type). +- C#'s codegen carried its OWN duplicate per-subtype band table; it was **deleted** rather + than extended. +- Gated cross-port by a new `fEnumInt` case in `fixtures/conformance/filter-ops-matrix` + (`field.filter-ops` was already a field-level capability in all five ports, so no runner + change was needed). + +**Task 8b (NOT in the original plan) — the view-DDL blocker.** Probing Step 4 surfaced a +strictly worse bug: a projection row-scope `@filter` (#207) and an `origin.aggregate +@filter` render as literal SQL TEXT and never touch Drizzle, so the customType does nothing +for them. Both emitted `WHERE p.status = 'PUBLISHED'` against an `integer` column — +rejected by Postgres at CREATE VIEW time, aborting the migration. A `meta migrate` blocker, +affecting every operator rather than just `like`. Fixed in `2fd177f2d`; note it needed BOTH +`resolveViewFilter` and the separate `resolveAggregateFilter`. Gated by 10 unit tests plus a +real-Postgres apply-and-converge test, itself verified load-bearing by disabling the encode +and confirming red. + +**Durable lesson for the other three port plans:** a column-level codec seam does not reach +anywhere the port renders SQL text by hand. Each port needs the encoding in both places. --- @@ -772,4 +798,8 @@ Known port-specific defects already identified, to fold in during that rewrite: - **C#** — the array branch emits `ElementType().HasConversion()` unconditionally, ignoring `@intValueMap` (violates D7); and its per-entity `EnumTypeName` naming needs re-checking against FR-019's shared/provided materialization. - **Java/Kotlin** — Kotlin's per-package `${enumClassName}_TO_INT` support-file emission collides under a shared enum (two consuming fields → two same-named top-level `val`s, even with identical maps: the emitter iterates `(class, field)` pairs with no dedupe). Emit per TYPE, once. A `@provided` Kotlin enum additionally needs its class imported into the support file. Java's `hasMetaAttr(name)` defaults to `includeParentData=true` and DOES resolve through `extends` (verified) — so its codec read is correct by default, but keep it that way deliberately. - **Python** — the write branch's `int_value_map[value]` raises `KeyError` on a non-member (should be a clean validation error) and `TypeError` on an array-of-enum value (a list is unhashable); D7 array handling is absent entirely. The query/WHERE path is unaddressed (same class as Task 8). +- **All ports** — a column-level codec seam (the TS `customType`, EF Core `HasConversion`, + `JdbcFieldCodec`, Exposed `customEnumeration`) does NOT reach anywhere the port renders + SQL **text** by hand — view bodies above all. TS needed the member→integer encoding in + BOTH places (see Task 8b); assume every port does. - **All ports** — `@provided` + `@intValueMap` is a REAL adopter case, not an edge case: ADR-0026's motivating example is literally a hand-written enum with int backing. Materialization is suppressed; the codec is NOT. Every port must map by member SYMBOL through the metadata map, never through the provided native type's own underlying integer values (a hand-written `ContactMethod.Email = 3` with `@intValueMap {Email: 1}` must store `1`). C#'s name-keyed dictionary gets this right by construction but has no test pinning it. From 1e66c7080e23bd4fad4f81bed553bde76d209e64 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sat, 15 Aug 2026 06:53:44 -0400 Subject: [PATCH 31/52] test(integration): TPH + int-backed enums against real Postgres (Task 9) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 9 asked two questions and flagged the second as possibly needing its own design, with "reject @intValueMap on a discriminator with a named loader error" as the documented fallback: 1. does a per-subtype read schema tolerate an int-backed enum COLUMN? 2. does an int-backed enum work AS the discriminator? Both work, and the discriminator case is SUPPORTED — no product change needed. The reason is structural rather than lucky: every TPH path goes through Drizzle (`db.select()`, `eq(auths.type, "Bridge")`, `.values()`, and the routes tier's `discriminatorCond` likewise), so the Task 5 customType encodes and decodes at the COLUMN and the schemas only ever see member symbols. `z.literal("Bridge")` and `parseAuth`'s `z.enum` head parse are therefore correct as emitted. Reading the generated source says all that. #203/#229 is the precedent for TPH being a separate code path everyone assumes is covered, and the 0.15.21 line is what "the source looks right" is worth — so this commit is the run, not the read. Seven tests over a hierarchy whose discriminator is int-backed 1/2 AND which carries a second int-backed enum (0/7, so the zero member is live): - the base-table DDL applies and a second migrate converges; - the discriminator column is `integer` with an INTEGER `CHECK`, no 'Bridge'; - a create through the generated per-subtype fn stores BOTH enums as integers (asserted with raw SQL, bypassing the codec — what is actually on disk); - the per-subtype read schema decodes a raw-SQL-inserted integer row; - the per-subtype filter compares the integer discriminator; - the polymorphic read dispatches on the DECODED discriminator; - find-by-id is scoped by the discriminator, not just the PK (asking for a Copay row as a Bridge must MISS — proving the AND'd predicate encoded to 1). Two assertions failed on the first run and both were the test's fault, checked rather than assumed: the DB check constraint is `auths_type_chk` (migrate-ts's name; codegen's Drizzle `check()` uses `chk_auths_type`, a pre-existing naming difference that is identical for a string-backed enum), and a TPH `InsertSchema` requires its `z.literal` discriminator — also pre-existing, also identical string-backed, with the ROUTES tier being what omits and re-adds it from the URL. Co-Authored-By: Claude Opus 5 (1M context) --- .../test/enum-intvaluemap-pg.test.ts | 171 +++++++++++++++++- 1 file changed, 170 insertions(+), 1 deletion(-) diff --git a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts index f2845688e..6f9b1375f 100644 --- a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts @@ -26,7 +26,7 @@ import { } from "@metaobjectsdev/migrate-ts"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; import { runGen, defineConfig, buildProjectionViews } from "@metaobjectsdev/codegen-ts"; -import { entityFile } from "@metaobjectsdev/codegen-ts/generators"; +import { entityFile, queriesFile } from "@metaobjectsdev/codegen-ts/generators"; import { Kysely, PostgresDialect, sql } from "kysely"; import pg, { Pool } from "pg"; import { drizzle } from "drizzle-orm/node-postgres"; @@ -106,6 +106,9 @@ afterAll(async () => { beforeEach(async () => { await sql.raw(`DROP TABLE IF EXISTS "orders" CASCADE;`).execute(k); await sql.raw(`DROP TABLE IF EXISTS orders CASCADE;`).execute(k); + // The TPH block below builds its own base table; CASCADE also clears any + // dependent view left by the projection block. + await sql.raw(`DROP TABLE IF EXISTS "auths" CASCADE;`).execute(k); }); async function applyRaw(ddl: string): Promise { @@ -442,3 +445,169 @@ describe("int-backed field.enum in a projection view — real Postgres", () => { expect(body).not.toContain("'PUBLISHED'"); }, 120_000); }); + +/** + * TPH (single-table discriminator) + int-backed enums — Task 9. + * + * Two distinct questions, and the second is the one the plan flagged as possibly + * needing its own design: + * 1. a per-subtype read schema must tolerate an int-backed enum COLUMN; + * 2. an int-backed enum used AS the DISCRIMINATOR must still pin, filter and insert. + * + * Reading the generated source says both work: every TPH path goes through Drizzle + * (`db.select()`, `eq(auths.type, "Bridge")`, `.values()`), so the Task 5 customType + * encodes and decodes at the column, and the schemas only ever see member symbols. + * But #203/#229 is the precedent for TPH being a separate code path that everyone + * assumes is covered — and this repo's 0.15.21 line is what "the source looks right" + * is worth. So: run it. + * + * If this had needed its own design, the documented fallback was to REJECT + * @intValueMap on a discriminator with a named loader error. It does not — the + * discriminator case is SUPPORTED, and these tests are what pins that. + */ +describe("int-backed field.enum under TPH — real Postgres", () => { + // The discriminator (`type`) is int-backed 1/2, and there is ALSO a non- + // discriminator int-backed enum (`status`, 0/7) so both questions are live in one + // model. Both maps are non-ordinal so an accidental index-of-@values correspondence + // shows up as a wrong number rather than passing by coincidence. + const TPH_META = `{ + "metadata.root": { "package": "demo", "children": [ + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Copay"], + "@intValueMap": { "Bridge": 1, "Copay": 2 } } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "field.enum": { "name": "status", "@values": ["OPEN", "SHUT"], + "@intValueMap": { "OPEN": 0, "SHUT": 7 } } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "demo::Auth", + "@discriminatorValue": "Bridge", "children": [ + { "field.int": { "name": "quantity" } } ] } }, + { "object.entity": { "name": "CopayAuth", "extends": "demo::Auth", + "@discriminatorValue": "Copay", "children": [ + { "field.int": { "name": "amount" } } ] } } + ] } }`; + + let tphTmp: string; + let q: any; // the generated Auth.queries module + let tphDb: any; // Drizzle handle against the same database + let tphPool: pg.Pool; + + beforeAll(async () => { + const here = dirname(fileURLToPath(import.meta.url)); + const genTmpRoot = join(here, "..", ".gen-tmp"); + mkdirSync(genTmpRoot, { recursive: true }); + tphTmp = mkdtempSync(join(genTmpRoot, "enum-tph-")); + + const root = (await new MetaDataLoader().load([new InMemoryStringSource(TPH_META)])).root; + const lr = await runGen({ + config: defineConfig({ + outDir: tphTmp, extStyle: "none", dbImport: "./db", dialect: "postgres", + generators: [entityFile(), queriesFile()], + }), + metadata: root, + }); + if (lr.warnings.length > 0) throw new Error(`codegen warnings: ${lr.warnings.join("; ")}`); + + // The queries module takes its Db as a PARAMETER, so it needs no db module — + // it is imported exactly as emitted. + q = await import(pathToFileURL(join(tphTmp, "Auth.queries.ts")).href); + tphPool = new pg.Pool({ connectionString: pg2Uri }); + tphDb = drizzle(tphPool); + }, 180_000); + + afterAll(async () => { + await tphPool?.end(); + rmSync(tphTmp, { recursive: true, force: true }); + }); + + beforeEach(async () => { + await migrate(TPH_META); + }); + + test("the TPH base table DDL applies and a second migrate converges", async () => { + const expected = await expectedFor(TPH_META); + await assertConverged(expected); + }, 120_000); + + test("the discriminator column is integer with an INTEGER check", async () => { + const col = await sql<{ data_type: string }>` + SELECT data_type FROM information_schema.columns + WHERE table_name = 'auths' AND column_name = 'type' + `.execute(k); + expect(col.rows[0]?.data_type).toBe("integer"); + + const chk = await sql<{ def: string }>` + SELECT pg_get_constraintdef(oid) AS def FROM pg_constraint + WHERE conname = 'auths_type_chk' + `.execute(k); + // Unquoted integers, not 'Bridge'/'Copay' — the discriminator's CHECK is + // subject to the same int-backing as any other enum column. + expect(chk.rows[0]?.def).toContain("1"); + expect(chk.rows[0]?.def).toContain("2"); + expect(chk.rows[0]?.def).not.toContain("Bridge"); + }, 120_000); + + test("create through the generated per-subtype fn stores BOTH enums as integers", async () => { + // `type` is required by BridgeAuthInsertSchema (z.literal("Bridge")) — that is + // pre-existing TPH behaviour, identical for a string-backed discriminator; the + // ROUTES layer is what omits it and re-adds it from the URL. + await q.createBridgeAuth(tphDb, { + type: "Bridge", title: "a", status: "SHUT", quantity: 3, + }); + // Raw SQL, bypassing the codec — this is what is actually on disk. + const raw = await sql<{ type: number; status: number }>` + SELECT "type", "status" FROM "auths" WHERE "title" = 'a' + `.execute(k); + expect(raw.rows[0]?.type).toBe(1); // Bridge, not 'Bridge' + expect(raw.rows[0]?.status).toBe(7); // SHUT + }, 120_000); + + test("the per-subtype read schema decodes an int-backed enum column", async () => { + // Insert with raw SQL so neither value passes through toDriver — proving the + // read path decodes rather than the two directions cancelling out. + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status", "quantity") VALUES (1, 'b', 0, 9);`, + ).execute(k); + const rows = await q.listBridgeAuths(tphDb); + expect(rows).toHaveLength(1); + expect(rows[0].type).toBe("Bridge"); // z.literal("Bridge") accepted the decoded value + expect(rows[0].status).toBe("OPEN"); // the ZERO-valued member + expect(rows[0].quantity).toBe(9); + }, 120_000); + + test("the per-subtype filter compares the INTEGER discriminator", async () => { + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status") VALUES (1, 'b', 0), (2, 'c', 7), (1, 'd', 7);`, + ).execute(k); + const bridges = await q.listBridgeAuths(tphDb); + expect(bridges.map((r: any) => r.title).sort()).toEqual(["b", "d"]); + const copays = await q.listCopayAuths(tphDb); + expect(copays.map((r: any) => r.title)).toEqual(["c"]); + }, 120_000); + + test("the polymorphic read dispatches on the DECODED discriminator", async () => { + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status", "amount") VALUES (2, 'c', 7, 42);`, + ).execute(k); + const all = await q.listAuths(tphDb); + expect(all).toHaveLength(1); + // parseAuth read `type` as "Copay" and dispatched to CopayAuthSchema — a raw 2 + // would have thrown on the z.enum head parse. + expect(all[0].type).toBe("Copay"); + expect(all[0].amount).toBe(42); + }, 120_000); + + test("find-by-id is scoped by the integer discriminator, not just the PK", async () => { + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status") VALUES (2, 'c', 7);`, + ).execute(k); + const [{ id }] = (await sql<{ id: string }>`SELECT "id" FROM "auths"`.execute(k)).rows as any; + // The row IS a Copay, so asking for it as a Bridge must miss — proving the AND'd + // discriminator predicate encoded to 1 rather than binding 'Bridge'. + expect(await q.findBridgeAuthById(tphDb, Number(id))).toBeNull(); + expect((await q.findCopayAuthById(tphDb, Number(id)))?.title).toBe("c"); + }, 120_000); +}); From 1c95662db7d2c653d1b48ea96bf88aeffe7f5f07 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sat, 15 Aug 2026 06:59:00 -0400 Subject: [PATCH 32/52] docs(plan): record the Task 9 outcome -- TPH needs no int-backed-enum change The premise was false for a structural reason worth keeping: every TPH path goes through Drizzle, so Task 5's column-level customType already covers the per-subtype read schemas AND the discriminator. Step 3 is decided in the affirmative -- an int-backed discriminator is SUPPORTED, and the documented reject-with-a-loader-error fallback was not taken. Folds the inverse of Task 8b's lesson into the notes the other three port plans will be rewritten from: where a port's TPH surface goes through its ORM, a column-level codec seam makes TPH work with zero TPH-specific code -- so prefer that seam over a query-layer one. Co-Authored-By: Claude Opus 5 (1M context) --- ...3-int-backed-enum-values-ts-persistence.md | 55 ++++++++++++++----- 1 file changed, 41 insertions(+), 14 deletions(-) diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md index 0d3163497..662181abe 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md @@ -771,20 +771,41 @@ anywhere the port renders SQL text by hand. Each port needs the encoding in both --- -### Task 9: codegen-ts — TPH per-subtype read schemas must decode - -**Why (verified 2026-08-12):** `renderTphSubtypeReadSchema` (`packages/codegen-ts/src/templates/zod-validators.ts`) parses DB ROWS. For an int-backed enum the row holds an integer, which a string `z.enum([...])` read schema rejects outright. Task 5 wires the vanilla read path; the TPH per-subtype path is a SEPARATE code path — this is the same class of miss as #203/#229, where TPH per-subtype controllers each needed `@autoSet` stamping wired separately after the vanilla path already had it. The original plan hand-waved TPH as "a follow-up if discovered incomplete." It IS incomplete. - -Note `fixtures/conformance/tph-discriminator-enum-with-subtypes` exists: an int-backed enum used AS a TPH discriminator additionally needs its `HasValue`-equivalent literal comparisons encoded. If that proves to need its own design, the acceptable fallback is to REJECT `@intValueMap` on a discriminator field with a clear loader error — but decide it explicitly, do not leave it emitting broken code. - -**Files:** -- Modify: `packages/codegen-ts/src/templates/zod-validators.ts` -- Test: `packages/codegen-ts/test/templates/` TPH read-schema test - -- [ ] **Step 1: Failing test** — a TPH hierarchy whose base carries an int-backed enum: the generated per-subtype read schema accepts the integer row value and yields the member string. -- [ ] **Step 2:** Wire the decode into the TPH read path, reusing Task 5's generated lookup — do not duplicate the codec. -- [ ] **Step 3:** Decide and implement the discriminator case (support, or reject with a named error). -- [ ] **Step 4: Commit** — `fix(codegen-ts): TPH per-subtype read schemas decode int-backed enums` +### Task 9: codegen-ts — TPH per-subtype read schemas must decode — **DONE (no product change)** + +**Outcome (2026-08-14).** The premise is false, and for a structural reason worth +recording: `renderTphSubtypeReadSchema` does NOT see raw DB rows. Every TPH path goes +through Drizzle — `db.select()` for both the polymorphic and per-subtype reads, +`eq(auths.type, "Bridge")` for the subtype predicate, `.values()` for the insert, and +the routes tier's `discriminatorCond` (`runtime-ts/src/drizzle-fastify/index.ts:94`) +likewise. So Task 5's `customType` encodes and decodes **at the column**, and every +schema only ever sees member symbols. `z.literal("Bridge")` and `parseAuth`'s `z.enum` +head parse are correct exactly as emitted; there is no decode to wire and no lookup to +reuse. + +**Step 3 — the discriminator case is DECIDED: SUPPORTED, not rejected.** An int-backed +enum used AS a TPH discriminator needs no special handling for the same reason: the +discriminator column gets the same `customType`, its `CHECK` lists unquoted integers, +and every comparison against it is a Drizzle `eq`. The documented fallback (reject +`@intValueMap` on a discriminator with a named loader error) was NOT taken. + +**Verified by running it, not by reading it** (`da535f95d`) — #203/#229 is the precedent +for TPH being a separate code path everyone assumes is covered, and the 0.15.21 line is +what "the generated source looks right" is worth. Seven real-Postgres tests in +`integration-tests/test/enum-intvaluemap-pg.test.ts` over a hierarchy whose discriminator +is int-backed 1/2 and which carries a second int-backed enum (0/7, so the zero member is +live): DDL applies + converges; the discriminator column is `integer` with an integer +CHECK; a generated per-subtype create stores both enums as integers (asserted with raw +SQL, bypassing the codec); the per-subtype read schema decodes a raw-SQL-inserted integer +row; the per-subtype filter compares the integer; the polymorphic read dispatches on the +decoded value; and find-by-id is discriminator-scoped (a Copay row asked for as a Bridge +must MISS). + +**Two things the run surfaced, both pre-existing and both identical for a string-backed +enum** — neither is int-backing-specific, neither was changed: migrate-ts names the check +constraint `auths_type_chk` while codegen's Drizzle `check()` uses `chk_auths_type`; and a +TPH `InsertSchema` requires its `z.literal` discriminator (the ROUTES tier is what +omits it and re-adds it from the URL). --- @@ -798,6 +819,12 @@ Known port-specific defects already identified, to fold in during that rewrite: - **C#** — the array branch emits `ElementType().HasConversion()` unconditionally, ignoring `@intValueMap` (violates D7); and its per-entity `EnumTypeName` naming needs re-checking against FR-019's shared/provided materialization. - **Java/Kotlin** — Kotlin's per-package `${enumClassName}_TO_INT` support-file emission collides under a shared enum (two consuming fields → two same-named top-level `val`s, even with identical maps: the emitter iterates `(class, field)` pairs with no dedupe). Emit per TYPE, once. A `@provided` Kotlin enum additionally needs its class imported into the support file. Java's `hasMetaAttr(name)` defaults to `includeParentData=true` and DOES resolve through `extends` (verified) — so its codec read is correct by default, but keep it that way deliberately. - **Python** — the write branch's `int_value_map[value]` raises `KeyError` on a non-member (should be a clean validation error) and `TypeError` on an array-of-enum value (a list is unhashable); D7 array handling is absent entirely. The query/WHERE path is unaddressed (same class as Task 8). +- **Every port, from Task 9's result** — if a port's TPH surface (and its read/filter/insert + paths generally) goes through its ORM, a COLUMN-level codec seam makes int-backing work + with no TPH-specific code at all. Prefer that seam over a query-layer one and the TPH + fan-out cost drops to zero. Where a port instead hand-builds SQL for TPH (a raw + discriminator predicate, a hand-written `WHERE type = ?`), it needs the encoding + explicitly — the same split as Task 8b's view bodies. - **All ports** — a column-level codec seam (the TS `customType`, EF Core `HasConversion`, `JdbcFieldCodec`, Exposed `customEnumeration`) does NOT reach anywhere the port renders SQL **text** by hand — view bodies above all. TS needed the member→integer encoding in From c18058afcc74ee1c2a1b4dbe11eb11fdf5ffdb1a Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sat, 15 Aug 2026 07:16:05 -0400 Subject: [PATCH 33/52] fix(integration-tests): write every generated module before importing any MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `validation-conformance.test.ts` failed its whole suite with `Cannot find module /Ledger.ts` while the file was demonstrably on disk. Bun caches a directory's listing at the FIRST import out of that directory, so a sibling written afterwards is invisible to the resolver. The loop wrote-then-imported per entity, so `Account` (first) resolved and `Ledger` (written after that import) did not. Split into write-all then import-all. 42 validation-conformance cases were failing to run at all and now execute; the TS conformance gate in `scripts/ci-local.sh --quick` goes from red to green. Pre-existing and unrelated to the int-backed-enum work — confirmed by reproducing it identically on a clean `origin/main` worktree with its own `bun install`, and diagnosed by probe (write both files first, then import both: green) rather than inferred. It is fixed here because it is the gate CLAUDE.md prescribes running before opening a PR, and it was red on `main`. Co-Authored-By: Claude Opus 5 (1M context) --- .../test/validation-conformance.test.ts | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/server/typescript/packages/integration-tests/test/validation-conformance.test.ts b/server/typescript/packages/integration-tests/test/validation-conformance.test.ts index 4fb095588..8493b3d20 100644 --- a/server/typescript/packages/integration-tests/test/validation-conformance.test.ts +++ b/server/typescript/packages/integration-tests/test/validation-conformance.test.ts @@ -35,6 +35,12 @@ beforeAll(async () => { const zodPath = pathToFileURL(Bun.resolveSync("zod", import.meta.dir)).href; tmpDir = mkdtempSync(join(tmpdir(), "validation-conformance-")); + // Write EVERY module before importing ANY of them. Bun caches a directory's + // listing at the first import out of it, so a sibling written after that import + // is invisible to the resolver and fails with `Cannot find module ` even + // though the file is on disk — which is exactly what a write-then-import loop + // produced here (Account resolved, Ledger did not). + const modulePaths: Array = []; for (const entityName of ENTITY_NAMES) { const entity = root.findObject(entityName); if (!entity) throw new Error(`corpus meta.json has no object named ${entityName}`); @@ -46,10 +52,14 @@ beforeAll(async () => { generated = generated.replace(/(['"])zod\1/g, JSON.stringify(zodPath)); // Emit to a temp module (outside the package tree, so it can't be picked up by - // a later test glob) and import it so we exercise the real generated code. + // a later test glob). const modulePath = join(tmpDir, `${entityName}.ts`); writeFileSync(modulePath, generated, "utf8"); + modulePaths.push([entityName, modulePath] as const); + } + for (const [entityName, modulePath] of modulePaths) { + // Import so we exercise the real generated code. const mod = (await import(pathToFileURL(modulePath).href)) as Record; const schema = mod[`${entityName}InsertSchema`]; if (!schema) throw new Error(`generated module did not export ${entityName}InsertSchema`); From f1e35c85c2afc44d456ec0563f5c5047e7f48c9c Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sun, 16 Aug 2026 10:41:15 -0400 Subject: [PATCH 34/52] no-mistakes(document): Documented int-backed enum storage via @intValueMap --- docs/features/field-types.md | 11 ++-- .../specs/2026-05-23-enum-datatype-design.md | 50 ++++++++++++++++--- ...fr-019-shared-and-provided-enums-design.md | 2 +- .../ADR-0039-own-accessor-discipline.md | 5 +- 4 files changed, 54 insertions(+), 14 deletions(-) diff --git a/docs/features/field-types.md b/docs/features/field-types.md index b331d186c..c607c2393 100644 --- a/docs/features/field-types.md +++ b/docs/features/field-types.md @@ -66,9 +66,12 @@ must be a non-empty set of unique members matching `^[A-Za-z_][A-Za-z0-9_]*$`. ``` The loader enforces members own-only and emits `ERR_BAD_ATTR_VALUE` on a bad -member or `ERR_MISSING_REQUIRED_ATTR` on missing `@values`. Int-backed enums, -display labels, and native Postgres `ENUM` types are deferred (see -[enum-datatype-design.md](../superpowers/specs/2026-05-23-enum-datatype-design.md)). +member or `ERR_MISSING_REQUIRED_ATTR` on missing `@values`. + +**Int-backed storage:** `@intValueMap` switches the DB column from string to integer while +preserving the string wire format and generated enum type. Keys must match `@values` exactly; +values must be unique integers. Display labels and native Postgres `ENUM` types remain +deferred (see [enum-datatype-design.md](../superpowers/specs/2026-05-23-enum-datatype-design.md)). ### Sharing one enum — abstract `field.enum` + `extends` @@ -292,4 +295,4 @@ for the per-port pass/skip ledger. - [entities.md](entities.md) — host node `object.entity` - [relationships.md](relationships.md) — relationships are separate from fields, despite sharing the column space - [yaml-authoring.md](yaml-authoring.md) — array-suffix sugar for repeated fields (`field.long[]: weekIds`) -- [enum-datatype-design](../superpowers/specs/2026-05-23-enum-datatype-design.md) — enum design rationale + deferred capabilities +- [enum-datatype-design](../superpowers/specs/2026-05-23-enum-datatype-design.md) — enum design rationale + int-backed storage diff --git a/docs/superpowers/specs/2026-05-23-enum-datatype-design.md b/docs/superpowers/specs/2026-05-23-enum-datatype-design.md index 0f3f4db67..8daf5dae2 100644 --- a/docs/superpowers/specs/2026-05-23-enum-datatype-design.md +++ b/docs/superpowers/specs/2026-05-23-enum-datatype-design.md @@ -1,7 +1,7 @@ # Design: `field.enum` — first-class enum datatype **Date:** 2026-05-23 -**Status:** Implemented across TS, C#, Java, Python (2026-05-23) +**Status:** Implemented across TS, C#, Java, Python, Kotlin (TypeScript int-backed in progress) **Author:** Doug Mealing (with Claude) ## Problem @@ -33,9 +33,7 @@ metamodel-as-spine payoff. A per-language regex is strictly worse output. ## Non-goals (out of scope) -- **Integer-backed enums** (where a member's symbol name differs from a stored number). - This needs per-member symbol→value assignment and is materially more complex; deferred - to a later design. v1 members are symbols stored as their own string. +- **Integer-backed enums** — Now supported via the `@intValueMap` attribute. See "Int-backed enum storage" below. - **Display labels.** Human-facing labels for members (e.g. `DRAFT` → "Draft") belong to the presentation/view layer, not the enum datatype. The enum stays a pure domain concept. - **Native Postgres `CREATE TYPE ... AS ENUM`.** Breaks PG/SQLite parity and is a @@ -68,9 +66,11 @@ metamodel-as-spine payoff. A per-language regex is strictly worse output. - **D3 — Members declared via `@values`.** A required string array on the `field.enum`. Declaration order is significant (it is the canonical member order for every port). -- **D4 — v1 is string-backed; int-backed deferred.** Each member's symbol *is* its stored - and transmitted string value. No `@backing` knob. Int-backed enums are a future codegen - mapping of the *same* subtype, not a new datatype — the model already accommodates them. +- **D4 — v1 is string-backed; int-backed via `@intValueMap`.** Each member's symbol *is* its stored + and transmitted string value by default. The optional `@intValueMap` attribute switches a field to + integer storage while preserving the string wire format and generated enum type. Keys must match + `@values` exactly; values must be unique integers. The generated type and wire format are unchanged + in every language. - **D5 — DB representation: `varchar` + `CHECK`.** Portable across Postgres and SQLite; adding/removing a member is a cheap CHECK swap. Native PG enum is explicitly out (see @@ -241,11 +241,45 @@ the metadata layer. The empty-`@values` fixture, surfaced during review, replace ## Remaining follow-ups -- Integer-backed enums (per-member symbol→value) — no current consumer. +- ~~Integer-backed enums (per-member symbol→value)~~ — **DONE via `@intValueMap` (2026-08)**. + TypeScript persistence layer complete; other ports pending. See int-backed enum plan docs. - Non-identifier-safe member strings (kebab-case, leading digit, etc.) needing a symbol↔stored-value mapping — no current consumer. - Display labels (presentation/view layer) — no current consumer. - Native Postgres `CREATE TYPE ... AS ENUM` (opt-in `@dbEnum`-style flag) — portable `varchar`+`CHECK` covers current needs. + +## Int-backed enum storage (2026-08) + +**Problem:** Some databases (especially legacy schemas) store enums as integers rather than strings, +but the generated enum type and wire format should remain string-based for type safety. + +**Solution:** The `@intValueMap` attribute on `field.enum`: + +```json +{ "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 1, "ARCHIVED": 2 } +}} +``` + +**Behavior:** + +- **Loader validation:** Keys must exactly match `@values`; values must be unique integers. +- **Generated type:** Unchanged — still a union/enum of string symbols. +- **Wire format:** Unchanged — members still transmit as strings. +- **Database column:** Integer instead of `varchar`; CHECK constraint enforces valid integers. +- **ORM mapping:** Custom codec/convertor layer translates between in-memory enum and DB integer. +- **Filter operators:** `like` is unsupported on int-backed enum fields (meaningless against integers). + +**Cross-port status:** + +- TypeScript: Complete (Drizzle customType, Zod, migrate, filter bands, TPH discriminators) +- Java/Kotlin: Metamodel registered, codecs pending +- C#: Metamodel registered, EF Core converters pending +- Python: Metamodel registered, codecs pending + +See plan docs: `docs/superpowers/plans/2026-07-23-int-backed-enum-values-*.md` - Extend the EF Core compile-check to the **Routes/ASP.NET** surface (needs the `Microsoft.AspNetCore.App` shared framework) — optional, lower priority. diff --git a/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md b/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md index 5c828a67e..f456cae81 100644 --- a/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md +++ b/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md @@ -145,4 +145,4 @@ The corpus is the oracle: TS reference green first, then each port matches. ## Cross-references - [ADR-0026](../../../spec/decisions/ADR-0026-shared-and-provided-named-types.md) — `@provided` as a cross-type provenance flag (enums + value objects); shared vs provided orthogonal; Option 2 upgrade path. - [ADR-0001](../../../spec/decisions/ADR-0001-cross-language-type-binding.md) — metadata→native type binding is per-port build-time config, not metadata. -- [enum datatype design](2026-05-23-enum-datatype-design.md) — D6 abstract-enum reuse; deferred int-backed/display-label/native-PG-enum. +- [enum datatype design](2026-05-23-enum-datatype-design.md) — D6 abstract-enum reuse; int-backed now supported via `@intValueMap`. diff --git a/spec/decisions/ADR-0039-own-accessor-discipline.md b/spec/decisions/ADR-0039-own-accessor-discipline.md index df77ec4a4..68c627570 100644 --- a/spec/decisions/ADR-0039-own-accessor-discipline.md +++ b/spec/decisions/ADR-0039-own-accessor-discipline.md @@ -34,7 +34,10 @@ Two attributes are read own-only by explicit policy, outside the emit-declared-h The distinction is only observable on a **chained declaration**: a root-level abstract enum `B extends` a root-level abstract `@provided` enum `A`. `@provided` is read on the resolved *declaration*, never on the consuming field, so for the ordinary `field extends @provided decl` shape own and resolving agree. On the chained shape a resolving read reports `B` as provided and emits a reference to a hand-written `B` **the adopter never declared** (the marker was authored on `A`), instead of materializing `B` from its inherited `@values`. Own-only matches authored intent. - Note this is a *provenance* marker and not a value: the member set it accompanies (`@values`, and its numeric half `@intValueMap`) is a logical property and is still read **resolving**, so a declaration inheriting `@values` from its super materializes correctly. + Note this is a *provenance* marker and not a value: the member set it accompanies (`@values`, and its numeric half `@intValueMap`) + is a logical property and is still read **resolving**, so a declaration inheriting `@values` from its super materializes correctly. + The `@intValueMap` attribute itself (registered on `field.enum` since 2026-08) is also read resolving, enabling inheritance through + `extends` chains. ### Naming Where a port's default-named accessor is own-only (Python `attr()` is own; TS `attr()` resolves — an inversion), the port SHOULD make the **resolving** form the default-named one and the own form explicitly `own*`, so "the obvious call" is the correct one. Any `own*()` call MUST carry a one-line comment stating which sanctioned case it is. From f23b81391ef4742f3c2069fb745e67f0276e9186 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sun, 16 Aug 2026 16:38:37 -0400 Subject: [PATCH 35/52] docs(design): fix a heading inserted mid-list by the no-mistakes document step The no-mistakes pipeline's automated "document" step added a "## Int-backed enum storage" section to the enum design doc, but inserted it one bullet too early -- splitting the pre-existing "Remaining follow-ups" list and leaving its last bullet (an unrelated EF Core/Routes-ASP.NET scope note, predating this work) dangling after the new section's closing line with no list context. Moved the new section to come after the full original list. Content is unchanged; only the ordering is fixed. Co-Authored-By: Claude Opus 5 (1M context) --- docs/superpowers/specs/2026-05-23-enum-datatype-design.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-05-23-enum-datatype-design.md b/docs/superpowers/specs/2026-05-23-enum-datatype-design.md index 8daf5dae2..494fbd5ba 100644 --- a/docs/superpowers/specs/2026-05-23-enum-datatype-design.md +++ b/docs/superpowers/specs/2026-05-23-enum-datatype-design.md @@ -242,12 +242,14 @@ the metadata layer. The empty-`@values` fixture, surfaced during review, replace ## Remaining follow-ups - ~~Integer-backed enums (per-member symbol→value)~~ — **DONE via `@intValueMap` (2026-08)**. - TypeScript persistence layer complete; other ports pending. See int-backed enum plan docs. + TypeScript persistence layer complete; other ports pending. See "Int-backed enum storage" below. - Non-identifier-safe member strings (kebab-case, leading digit, etc.) needing a symbol↔stored-value mapping — no current consumer. - Display labels (presentation/view layer) — no current consumer. - Native Postgres `CREATE TYPE ... AS ENUM` (opt-in `@dbEnum`-style flag) — portable `varchar`+`CHECK` covers current needs. +- Extend the EF Core compile-check to the **Routes/ASP.NET** surface (needs the + `Microsoft.AspNetCore.App` shared framework) — optional, lower priority. ## Int-backed enum storage (2026-08) @@ -281,5 +283,3 @@ but the generated enum type and wire format should remain string-based for type - Python: Metamodel registered, codecs pending See plan docs: `docs/superpowers/plans/2026-07-23-int-backed-enum-values-*.md` -- Extend the EF Core compile-check to the **Routes/ASP.NET** surface (needs the - `Microsoft.AspNetCore.App` shared framework) — optional, lower priority. From d956251ee98ec333e026cfaad95f945a91fc2926 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sun, 16 Aug 2026 16:43:42 -0400 Subject: [PATCH 36/52] fix(migrate-ts): restore FK refColumns @column-override resolution lost in a rebase buildForeignKeys resolved a target FK field's physical column by applying the naming strategy to its raw (logical) name -- silently dropping the @column- override resolution this exact function had (c86cd203d, long before this session): a target PK with an explicit @column override phantom-diffed every FK into that table. This is a genuine regression, not new work: expected-schema-fk-refcolumn- override.test.ts already pinned the correct behavior and was green at this branch's last fully-tested commit (658cacfc) before the no-mistakes pipeline's autonomous CI-repair rebase ran. That rebase silently reverted this one hunk while replaying ~40 commits onto an advanced origin/main (0.22.1 -> 0.23.1) -- git raised no conflict marker for it, so nothing surfaced it except rerunning the full test suite and diffing against the pre-rebase tree by hand. Confirmed via a worktree at 658cacfc (test passes there) and git history (the correct logic traces to c86cd203d, still an ancestor of HEAD, but the code at HEAD no longer matched it). Restored verbatim from 658cacfc's tree. No other change. Co-Authored-By: Claude Opus 5 (1M context) --- .../migrate-ts/src/expected-schema.ts | 24 +++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/server/typescript/packages/migrate-ts/src/expected-schema.ts b/server/typescript/packages/migrate-ts/src/expected-schema.ts index f19947845..fe25f1cbb 100644 --- a/server/typescript/packages/migrate-ts/src/expected-schema.ts +++ b/server/typescript/packages/migrate-ts/src/expected-schema.ts @@ -866,11 +866,27 @@ function buildForeignKeys( // Target columns: prefer explicit multi-field dotted form, else delegate // to MetaReferenceIdentity.resolvedTargetPkField (single field → target's - // primary identity → "id" fallback). + // primary identity → "id" fallback). Each target FIELD name must resolve to + // its PHYSICAL column via the target entity's own @column override (e.g. a + // PK field `id` with `@column: "Id"`), exactly like fkCols above — the raw + // naming strategy alone would emit the logical name and phantom-diff every + // FK into that table (expected ["id"] vs actual ["Id"]). + // targetEntity may be package-qualified (FQN); findObject is keyed by bare + // name — same fallback as resolvedTargetPkField/resolveTargetTable. + const targetObj = root.findObject(targetEntity) + ?? (targetEntity.includes("::") + ? root.findObject(targetEntity.slice(targetEntity.lastIndexOf("::") + 2)) + : undefined); const explicitTargetFields = refChild.targetFields; - const refColumns = explicitTargetFields.length > 1 - ? explicitTargetFields.map((n) => applyColumnNamingStrategy(n, strategy)) - : [applyColumnNamingStrategy(refChild.resolvedTargetPkField(root) ?? "id", strategy)]; + const targetFieldNames = explicitTargetFields.length > 1 + ? explicitTargetFields + : [refChild.resolvedTargetPkField(root) ?? "id"]; + const refColumns = targetFieldNames.map((jsName) => { + const targetField = targetObj ? findField(targetObj, jsName) : undefined; + return targetField + ? resolveColumnName(targetField, strategy) + : applyColumnNamingStrategy(jsName, strategy); + }); const { onDelete, onUpdate } = resolveReferentialActions(entity, refChild); // An explicit @constraintName adopts an existing FK name (e.g. a database From 0618673e18a903178c76ac7b1aba3c4e681d1b80 Mon Sep 17 00:00:00 2001 From: Douglas Mealing Date: Sun, 16 Aug 2026 16:45:39 -0400 Subject: [PATCH 37/52] chore(conformance): re-regenerate the registry-coverage baseline after the pipeline's rebase Upstream's own registry grew independently during the 0.22.1 -> 0.23.1 window this branch was rebased across, so the tracked snapshot's registeredSubTypeCount (69) no longer matched the current registry (70), and attr.intMap dropped off the exercised-list bookkeeping as a result -- not because any fixture stopped exercising it (all 8 int-backed-enum conformance fixtures are still present and unchanged; verified before regenerating). Regenerated per the test's own documented fix. Co-Authored-By: Claude Opus 5 (1M context) --- fixtures/registry-conformance/coverage-report.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fixtures/registry-conformance/coverage-report.json b/fixtures/registry-conformance/coverage-report.json index 4b702f4e0..cfc50bf37 100644 --- a/fixtures/registry-conformance/coverage-report.json +++ b/fixtures/registry-conformance/coverage-report.json @@ -1,5 +1,5 @@ { - "registeredSubTypeCount": 69, + "registeredSubTypeCount": 70, "exercisedSubTypeCount": 47, "untestedSubTypes": [ "attr.base", @@ -9,6 +9,7 @@ "attr.expression", "attr.filter", "attr.int", + "attr.intMap", "attr.long", "attr.string", "field.base", From 6359a3bd16051748415936dfb574dc6eb275d129 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 17:47:15 -0400 Subject: [PATCH 38/52] feat(python): int-backed enum persistence codec in ObjectManager MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The metamodel layer shipped @intValueMap in all five ports; Python's runtime never encoded it, so a field declaring a map still stored its member SYMBOL into what migrate provisions as an INTEGER column. Two halves, mirroring the TS reference: write _coerce_write_value gains a field.enum branch — symbol -> declared int. Enum was a pure fallthrough to `return value` before. read _decode_read_value is new; there was no read-side coercion in this module at all (select/find_by_id/find_many returned driver values verbatim, per ADR-0019). Wired into all THREE column->field mapping sites: create RETURNING, update RETURNING, and find_many. The native and wire contract is unchanged in both directions — a caller passes and receives the member symbol exactly as for a string-backed enum. Int-backing stays invisible above the codec (design goal 2). ADR-0039: the map is read RESOLVING (get_meta_attr), not own-only. It is @values' numeric half — a logical property of the enum vocabulary that inherits through extends — so a concrete field extending a shared abstract enum encodes with the inherited map. Contrast @dbColumnType in the same function, which is deliberately own-only. Pinned by a dedicated inheritance test. Two deliberate non-behaviours, each pinned: - an unmapped symbol on write and an unmapped int on read pass through UNTOUCHED. Membership is the column's CHECK constraint to enforce; nulling or inventing a value here would hide real data drift. - DRAFT maps to 0, a falsy int, so the branch tests `value in int_map` rather than truthiness — the obvious `if not mapped` bug is pinned. PROVEN NON-VACUOUS: neutering just the write branch turns 5 tests red while the string-backed / None / unknown-symbol cases stay green (they assert unchanged behaviour). Read tests fail at import without _decode_read_value. Note the two plan-supplied test helpers did not exist: MetaRoot has no find_object and MetaObject no field(name) — children()/fields() are the real accessors. 19 new tests; full Python suite 1718 passed / 0 failed. Co-Authored-By: Claude Opus 5 (1M context) --- .../src/metaobjects/runtime/object_manager.py | 57 +++++++- .../test_object_manager_enum_intvaluemap.py | 131 ++++++++++++++++++ 2 files changed, 187 insertions(+), 1 deletion(-) create mode 100644 server/python/tests/runtime/test_object_manager_enum_intvaluemap.py diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index 43eb930df..cf1b2286f 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -311,6 +311,8 @@ def _insert_row( } row = result.rows[0] mapped = {col_to_field.get(k, k): v for k, v in row.items()} + # int-backed enums come back as integers; hand the caller the symbol. + mapped = _decode_read_row({f.name: f for f in returning_fields}, mapped) # #214: a write-through entity's INSERT RETURNING covers only the table (non-derived) # columns; re-read the persisted row through the replica VIEW by PK so the returned row # carries the derived origin.* fields (read-your-writes). find_by_id routes to the view. @@ -423,6 +425,8 @@ def update( } row = result.rows[0] mapped = {col_to_field.get(k, k): v for k, v in row.items()} + # int-backed enums come back as integers; hand the caller the symbol. + mapped = _decode_read_row({f.name: f for f in returning_fields}, mapped) # #214: re-read the updated row through the replica VIEW by PK so the returned row # carries the derived origin.* fields (write targeted the table, which lacks them). if entity.is_write_through(): @@ -509,8 +513,11 @@ def find_many( self.last_column_oids = { col_to_field.get(c, c): oid for c, oid in result.column_oids.items() } + # int-backed enums come back as integers; hand the caller the symbol. + fields_by_name = {f.name: f for f in entity.fields()} return [ - {col_to_field.get(k, k): v for k, v in row.items()} for row in result.rows + _decode_read_row(fields_by_name, {col_to_field.get(k, k): v for k, v in row.items()}) + for row in result.rows ] def count(self, entity_name: str, filter: Filter | None = None) -> int: @@ -823,11 +830,59 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: storage = field.get_meta_attr(fc.FIELD_ATTR_STORAGE) # ADR-0039 resolving if storage != "flattened": # None / "jsonb" / "subdocument" → single jsonb column return _json.dumps(value) + # field.enum carrying @intValueMap: the column is an INTEGER while the native + # and wire contract stays the member SYMBOL (int-backing is a persistence-layer + # concern, invisible above this codec), so encode symbol -> declared int here. + # + # ADR-0039 resolving: the map is @values' numeric half — a logical property of + # the enum vocabulary that inherits through extends — so it is read with + # get_meta_attr, NOT the own-only accessor (contrast @dbColumnType above). + # + # An unmapped symbol is passed through untouched: membership is the column's + # CHECK constraint to enforce, and inventing a value here would hide the drift. + if sub == fc.FIELD_SUBTYPE_ENUM: + int_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_map, dict) and value in int_map: + return int_map[value] + # Everything else (string / int / long / double / float / boolean / enum) # is already the native type pg8000 binds directly. return value +def _decode_read_value(field: MetaField, value: Any) -> Any: + """Decode a stored value back to its authoring form on read. + + The inverse of :func:`_coerce_write_value`'s int-backed-enum branch, and + today its only case: the column holds the member's integer, callers expect + the member SYMBOL. Every other field subtype is returned verbatim, keeping + ADR-0019's "runtime returns native in-process types" contract intact. + + An int with no member is returned AS-IS rather than as ``None`` — a row + holding a value the model does not describe is real drift, and surfacing it + is honest where nulling it would hide it. + """ + if value is None: + return None + if field.sub_type != fc.FIELD_SUBTYPE_ENUM: + return value + int_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) # ADR-0039 resolving + if not isinstance(int_map, dict): + return value + for symbol, stored in int_map.items(): + if stored == value and isinstance(value, int) and not isinstance(value, bool): + return symbol + return value + + +def _decode_read_row(fields_by_name: dict[str, MetaField], row: dict[str, Any]) -> dict[str, Any]: + """Apply :func:`_decode_read_value` to every field-keyed value in a mapped row.""" + return { + k: (_decode_read_value(fields_by_name[k], v) if k in fields_by_name else v) + for k, v in row.items() + } + + # ---------------------------------------------------------------------------- # #203 — @autoSet CRUD stamping. A ``field.timestamp`` (or any temporal field) # marked ``@autoSet: onCreate|onUpdate`` declares "the runtime owns this diff --git a/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py new file mode 100644 index 000000000..cb7fa3b1d --- /dev/null +++ b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py @@ -0,0 +1,131 @@ +"""field.enum @intValueMap — Python ObjectManager persistence codec. + +The metamodel layer ships the attribute; this pins the RUNTIME halves: +write encodes the member symbol to its declared int, read decodes it back. + +The native/wire contract is unchanged in both directions — a caller always +passes and receives the member SYMBOL, exactly as for a string-backed enum +(design goal 2: int-backing is a persistence-layer-only concern). Only the +value handed to / received from the driver differs. +""" +from __future__ import annotations + +import pytest + +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.runtime.object_manager import _coerce_write_value, _decode_read_value + +INT_MAP = ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}' + + +def _field_of(json_str: str, entity_name: str, field_name: str): + """Load a model and pull one field. MetaRoot has no find_object and + MetaObject no field(name) — children()/fields() are the real accessors.""" + result = MetaDataLoader().load([InMemoryStringSource(json_str, "test.json")]) + assert result.errors == [] + entity = next(c for c in result.root.children() if c.name == entity_name) + return next(f for f in entity.fields() if f.name == field_name) + + +def _status_field(extra: str): + json_str = f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + return _field_of(json_str, "Order", "status") + + +# --- write side ----------------------------------------------------------- + + +@pytest.mark.parametrize("symbol,stored", [("DRAFT", 0), ("PUBLISHED", 5), ("ARCHIVED", 9)]) +def test_int_backed_enum_write_encodes_symbol_to_declared_int(symbol, stored): + assert _coerce_write_value(_status_field(INT_MAP), symbol) == stored + + +def test_int_backed_enum_write_encodes_zero_not_falsy_dropped(): + """DRAFT maps to 0 — a falsy int. Guards the `if not mapped` class of bug.""" + assert _coerce_write_value(_status_field(INT_MAP), "DRAFT") == 0 + + +def test_string_backed_enum_write_is_unchanged(): + assert _coerce_write_value(_status_field(""), "PUBLISHED") == "PUBLISHED" + + +def test_none_stays_none_on_write(): + assert _coerce_write_value(_status_field(INT_MAP), None) is None + + +def test_unknown_symbol_on_write_is_left_alone_for_the_db_to_reject(): + """Not the codec's job to validate membership — the CHECK constraint is.""" + assert _coerce_write_value(_status_field(INT_MAP), "NOPE") == "NOPE" + + +# --- read side ------------------------------------------------------------ + + +@pytest.mark.parametrize("stored,symbol", [(0, "DRAFT"), (5, "PUBLISHED"), (9, "ARCHIVED")]) +def test_int_backed_enum_read_decodes_int_to_symbol(stored, symbol): + assert _decode_read_value(_status_field(INT_MAP), stored) == symbol + + +def test_int_backed_enum_read_decodes_zero(): + assert _decode_read_value(_status_field(INT_MAP), 0) == "DRAFT" + + +def test_string_backed_enum_read_is_unchanged(): + assert _decode_read_value(_status_field(""), "PUBLISHED") == "PUBLISHED" + + +def test_none_stays_none_on_read(): + assert _decode_read_value(_status_field(INT_MAP), None) is None + + +def test_unmapped_int_on_read_is_passed_through_not_silently_nulled(): + """A row holding an int outside the map is data the model does not describe. + Surfacing it verbatim is honest; returning None would hide the drift.""" + assert _decode_read_value(_status_field(INT_MAP), 7) == 7 + + +def test_non_enum_field_read_is_untouched(): + json_str = """{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]} } + ]} }""" + assert _decode_read_value(_field_of(json_str, "Order", "id"), 5) == 5 + + +# --- round trip ----------------------------------------------------------- + + +@pytest.mark.parametrize("symbol", ["DRAFT", "PUBLISHED", "ARCHIVED"]) +def test_write_then_read_round_trips_to_the_same_symbol(symbol): + field = _status_field(INT_MAP) + assert _decode_read_value(field, _coerce_write_value(field, symbol)) == symbol + + +# --- inheritance (ADR-0039: @intValueMap is read RESOLVING) --------------- + + +def test_intvaluemap_inherited_through_extends_is_honoured(): + """A concrete field extending a shared abstract enum must encode with the + inherited map — an own-only read here would silently store the symbol.""" + json_str = """{ "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "StatusEnum", "abstract": true, + "@values": ["DRAFT","PUBLISHED","ARCHIVED"], + "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9} } }, + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "StatusEnum" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]} } + ]} }""" + field = _field_of(json_str, "Order", "status") + assert _coerce_write_value(field, "PUBLISHED") == 5 + assert _decode_read_value(field, 5) == "PUBLISHED" From 4989a615490d4f457368763dacf15ccfa18341f8 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 18:16:49 -0400 Subject: [PATCH 39/52] feat(java): int-backed enum persistence in OMDB's EnumCodec EnumCodec read the column with getString and wrote with setString, unconditionally. A field.enum declaring @intValueMap persists as an INTEGER, so writing it did not merely store the wrong value -- it did not store at all: Derby rejects the symbol with "ERROR 22018: Invalid character string format for type INTEGER". Proven by running the new fixture against the real engine BEFORE the fix. Both directions now branch on the map, with the caller's contract unchanged: the SYMBOL goes in and comes out either way. Int-backing stays invisible above the codec, so every OMDB call site (ObjectManagerDB, GenericSQLDriver, SimpleMappingHandlerDB) is untouched -- they all route through JdbcCodecs.forField(f). ADR-0039: the map is read RESOLVING (hasMetaAttr/getMetaAttr, not the ,false own-only overload). It is @values' numeric half -- a logical property of the enum vocabulary that inherits through extends. Two deliberate non-behaviours: an unmapped SYMBOL on write is bound unchanged so the column rejects it, and an unmapped INT on read is surfaced as its digits rather than null. Membership is the database's to enforce; both alternatives turn a loud error into a silently wrong row. GATED AGAINST A REAL DATABASE, and asserting the STORED form, not just symmetry. The fixture gains a second field.enum `priority` (INTEGER column, DRAFT/PUBLISHED/ ARCHIVED -> 0/5/9) beside the existing string-backed `status`, so one test covers both modes. Critically it then reads the raw column over plain JDBC and asserts it holds 5: a round-trip alone cannot distinguish a working int codec from one that wrote the symbol in both directions, because a symmetric bug is self-consistent. The plan's test sketch assumed Mockito and JUnit 5; omdb has neither -- it uses JUnit 4 and a real embedded Derby round-trip, which is the stronger gate anyway. JdbcCodecRoundTripTest: 7 run, 0 failures, 0 skipped. Co-Authored-By: Claude Opus 5 (1M context) --- .../manager/db/codec/JdbcCodecs.java | 69 ++++++++++++++++++- .../db/codec/JdbcCodecRoundTripTest.java | 23 ++++++- .../manager/db/test/CodecSchema.java | 6 +- .../omdb/src/test/resources/meta.codec.json | 8 +++ 4 files changed, 101 insertions(+), 5 deletions(-) diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java index 45e09c788..3ca9fa06a 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java @@ -57,6 +57,7 @@ import java.time.OffsetDateTime; import java.time.ZoneOffset; import java.time.format.DateTimeParseException; +import java.util.LinkedHashMap; import java.util.Map; import java.util.UUID; import java.util.concurrent.ConcurrentHashMap; @@ -412,13 +413,75 @@ static final class CurrencyCodec implements JdbcFieldCodec { * explicitly so enum does not ride the generic {@link ObjectCodec} fallback. The DB * {@code CHECK (col IN (...))} (emitted from {@code @values}) enforces membership. */ + /** + * {@code field.enum}, string- or int-backed. + * + *

      A plain enum persists its member symbol as text. One declaring + * {@code @intValueMap} persists the member's declared INTEGER instead, while the + * caller's contract stays the SYMBOL in both directions — int-backing is a + * persistence-layer concern, invisible above this codec.

      + * + *

      The map is read RESOLVING ({@code getMetaAttr}, ADR-0039): it is + * {@code @values}' numeric half, a logical property of the enum vocabulary that + * inherits through {@code extends}, so a concrete field extending a shared + * abstract enum encodes with the inherited map.

      + */ static final class EnumCodec implements JdbcFieldCodec { @Override public void readInto(Object o, MetaField f, ResultSet rs, int j) throws SQLException { - f.setString(o, rs.getString(j)); + Map intMap = intValueMap(f); + if (intMap == null) { + f.setString(o, rs.getString(j)); + return; + } + int stored = rs.getInt(j); + if (rs.wasNull()) { + f.setString(o, null); + return; + } + for (Map.Entry e : intMap.entrySet()) { + if (e.getValue() != null && e.getValue() == stored) { + f.setString(o, e.getKey()); + return; + } + } + // A stored int with no member is data the model does not describe. + // Surface it rather than nulling it — that would hide real drift. + f.setString(o, String.valueOf(stored)); } + @Override public void write(PreparedStatement s, MetaField f, int j, Object v) throws SQLException { - if (v == null) s.setNull(j, Types.VARCHAR); - else s.setString(j, v.toString()); + Map intMap = intValueMap(f); + if (intMap == null) { + if (v == null) s.setNull(j, Types.VARCHAR); + else s.setString(j, v.toString()); + return; + } + if (v == null) { + s.setNull(j, Types.INTEGER); + return; + } + Integer mapped = intMap.get(v.toString()); + if (mapped != null) { + s.setInt(j, mapped); + return; + } + // Unmapped symbol: bind it unchanged so the column/CHECK rejects it. + // Membership is the database's to enforce; inventing a value here + // would turn a loud error into a silently wrong row. + s.setString(j, v.toString()); + } + + /** The declared symbol→int map, or null when the enum is string-backed. */ + private static Map intValueMap(MetaField f) { + if (!f.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null; + Object raw = f.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).getValue(); + if (!(raw instanceof Map)) return null; + Map out = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) raw).entrySet()) { + if (e.getKey() == null || !(e.getValue() instanceof Number)) continue; + out.put(String.valueOf(e.getKey()), ((Number) e.getValue()).intValue()); + } + return out.isEmpty() ? null : out; } } diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java index 912be69c7..6f8fc8026 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java @@ -242,7 +242,8 @@ public void timestampCurrencyEnumRoundTripThroughOMDB() throws Exception { // The SP-H subtypes under test. vo.setDate("tsVal", ts); // TimestampCodec vo.setLong("moneyVal", 199900L); // CurrencyCodec (integer minor units) - vo.setString("status", "MEDIUM"); // EnumCodec + vo.setString("status", "MEDIUM"); // EnumCodec (string-backed) + vo.setString("priority", "PUBLISHED"); // EnumCodec (int-backed, @intValueMap) omdb.createObject(oc, vo); @@ -266,6 +267,26 @@ public void timestampCurrencyEnumRoundTripThroughOMDB() throws Exception { Long.valueOf(199900L), read.getLong("moneyVal")); assertEquals("EnumCodec must round-trip the member symbol", "MEDIUM", read.getString("status")); + + // int-backed enum (@intValueMap): the caller's contract is the SYMBOL in + // both directions — int-backing is invisible above the codec. + assertEquals("EnumCodec must round-trip an int-backed member as its symbol", + "PUBLISHED", read.getString("priority")); + + // ...and ask the DATABASE what it actually stored. A round-trip alone + // cannot tell a working int codec from one that wrote the symbol both + // ways, because a symmetric bug is self-consistent. PUBLISHED is declared + // as 5, and the column is INTEGER. + try (Connection c = getConnection(); + PreparedStatement ps = c.prepareStatement( + "SELECT priority FROM CODEC_SAMPLE WHERE label = ?")) { + ps.setString(1, label); + try (ResultSet rs = ps.executeQuery()) { + assertTrue("row present for raw column read", rs.next()); + assertEquals("the column must hold the declared int, not the symbol", + 5, rs.getInt(1)); + } + } } finally { omdb.releaseConnection(oc); } diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java index 3f6549503..a1d8aab0e 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java @@ -53,7 +53,11 @@ private CodecSchema() {} + " startTime TIME,\n" + " tsVal TIMESTAMP,\n" + " moneyVal BIGINT,\n" - + " status VARCHAR(20)\n" + + " status VARCHAR(20),\n" + // int-backed field.enum (@intValueMap): the member symbol persists as + // its declared INTEGER, so this column's type differs from `status` + // above even though both are field.enum. + + " priority INTEGER\n" + ")"; /** Executes the CODEC_SAMPLE DDL on a fresh connection from {@code connector}. */ diff --git a/server/java/omdb/src/test/resources/meta.codec.json b/server/java/omdb/src/test/resources/meta.codec.json index ce2eb1124..a717669e0 100644 --- a/server/java/omdb/src/test/resources/meta.codec.json +++ b/server/java/omdb/src/test/resources/meta.codec.json @@ -95,6 +95,14 @@ "@values": ["LOW", "MEDIUM", "HIGH"] } }, + { + "field.enum": { + "name": "priority", + "@column": "priority", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9} + } + }, { "identity.primary": { "name": "primary", From 9f9f3ad4c216145d1b8e6c58c4cc5f32dbbab104 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 18:21:51 -0400 Subject: [PATCH 40/52] feat(kotlin): int-backed enum columns emit customEnumeration The Exposed table generator emitted enumerationByName for every field.enum, so an enum declaring @intValueMap got a VARCHAR column holding the member symbol -- while migrate provisions INTEGER and every other port stores the int. Both emission branches (the own-field loop and the TPH subtype-fold loop) now route through one enumColumnSpec() helper: enumerationByName when string-backed, customEnumeration("col", "INTEGER", read, write) when the map is present. The Kotlin-side property type is the same generated enum class either way -- int-backing is invisible in the entity's API. DEVIATION FROM THE PLAN, deliberately. The plan called for a generated lookup-map support file; the mapping is inlined as `when` expressions instead. A `when` over an enum is exhaustive, so a member with no mapping becomes a COMPILE error in the adopter's build rather than a runtime surprise, and it allocates nothing per row -- a mapOf(...) inside the lambda would rebuild the map on every read and every write. The read side keeps an else that fails loudly: a stored int with no member is data the model does not describe, and substituting a member would hide it. ADR-0039: @intValueMap is read RESOLVING, so a field extending a shared abstract enum inherits the members AND their mapping. THE COMPILE GATE CAUGHT A REAL BUG. The generator hand-rolls its file body as a string, and the first emission separated the one-line `when` branches with spaces -- `OrderPriority.DRAFT 5 -> ...`, which does not parse. Every text assertion passed; only compiling the emitted tree failed it. Branches are separated with `; ` now. This is why the new test compiles the output rather than grepping it. The fixture carries BOTH modes on one entity so the test also pins that string-backed emission is byte-unchanged (feature is additive). codegen-kotlin: 315 run, 0 failures, 0 errors, 0 skipped. Co-Authored-By: Claude Opus 5 (1M context) --- .../kotlin/KotlinExposedTableGenerator.kt | 80 ++++++++++++---- .../KotlinExposedTableIntBackedEnumTest.kt | 96 +++++++++++++++++++ .../models/enum-int-backed/meta.shop.json | 30 ++++++ 3 files changed, 189 insertions(+), 17 deletions(-) create mode 100644 server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt create mode 100644 server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt index f3e1c12c7..f68dd5598 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt @@ -607,18 +607,14 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase null } + /** + * Exposed column spec for a `field.enum`. + * + * String-backed (the default) → `enumerationByName`: a VARCHAR column holding the + * member symbol. Int-backed (`@intValueMap`) → `customEnumeration` over an INTEGER + * column, with the declared symbol↔int mapping inlined as `when` expressions. + * + * The mapping is emitted INLINE rather than through a generated lookup-map support + * file, for two reasons. A `when` over an enum is exhaustive, so a member with no + * mapping is a COMPILE error in the adopter's build rather than a runtime surprise; + * and it allocates nothing per row, where a `mapOf(...)` inside the lambda would + * rebuild the map on every read and every write. + * + * ADR-0039: `@values` and `@intValueMap` are both read RESOLVING, so a field that + * `extends` a shared abstract enum inherits the members AND their mapping. + */ + private fun enumColumnSpec(field: EnumField, entity: MetaObject): String { + val enumCn = KotlinTypeMapper.enumTypeName(field, entity) + ?: error("enumTypeName returned null for EnumField '${field.name}' on ${entity.name}") + val colName = KotlinGenUtil.camelToSnake(field.name) + val simple = enumCn.simpleName + val intMap = readIntValueMap(field) + ?: return "enumerationByName(\"$colName\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, $simple::class)" + + // `; ` separates the branches: this is a one-line `when`, and space-separated + // branches do not parse (`Foo.DRAFT 5 -> ...`). + val fromDb = intMap.entries.joinToString("; ") { (sym, i) -> "$i -> $simple.$sym" } + val toDb = intMap.entries.joinToString("; ") { (sym, i) -> "$simple.$sym -> $i" } + // An int with no member is corrupt data the model does not describe: fail loudly + // rather than substituting a member. The write side needs no `else` — it is + // exhaustive over the enum by construction, since @intValueMap's keys are + // validated to match @values exactly. + return "customEnumeration(\"$colName\", \"INTEGER\", " + + "{ v -> when ((v as Number).toInt()) { $fromDb; " + + "else -> error(\"unmapped stored value \$v for $simple\") } }, " + + "{ e -> when (e) { $toDb } })" + } + + /** The declared symbol→int map (`@intValueMap`), or null when the enum is string-backed. */ + private fun readIntValueMap(field: EnumField): Map? { + if (!field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null + val raw = runCatching { field.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).value }.getOrNull() + val m = (raw as? Map<*, *>) ?: return null + val out = LinkedHashMap() + for ((k, v) in m) { + val key = k?.toString() ?: continue + val i = (v as? Number)?.toInt() ?: continue + out[key] = i + } + return out.ifEmpty { null } + } + // === MultiFileDirectGeneratorBase abstract-method stubs ==================== override fun writeSingleFile(md: MetaObject, writer: GeneratorIOWriter<*>?) { /* unused */ } override fun ?> getSingleWriter( diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt new file mode 100644 index 000000000..6746d9f5c --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt @@ -0,0 +1,96 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.metadata.ktx.loadDirectory +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import java.nio.file.Files +import java.nio.file.Path +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * Int-backed `field.enum` (`@intValueMap`) on the Exposed table generator. + * + * A string-backed enum persists its member symbol into a VARCHAR via + * `enumerationByName`. One declaring `@intValueMap` must instead persist the declared + * INTEGER via `customEnumeration`, while the Kotlin-side property type stays the very + * same generated enum class — int-backing is a persistence concern, invisible in the + * entity's API. + * + * The fixture carries BOTH modes on one entity (`priority` int-backed, `status` + * string-backed) so the test also pins that adding the feature did not change the + * string-backed emission. + * + * The compile gate is the load-bearing part. The generator hand-rolls its file body as + * a string, so `customEnumeration`'s two lambdas are only proven to type-check against + * the real Exposed API — the `Number`→`Int` narrowing on read, and the exhaustive `when` + * over the enum on write — by compiling the emitted tree. A text assertion alone would + * happily pass on a column spec that does not compile in the adopter's build. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class KotlinExposedTableIntBackedEnumTest { + + private val models: Path = run { + var p: Path? = Path.of(System.getProperty("user.dir")).toAbsolutePath() + while (p != null && !Files.exists(p.resolve("src/test/resources/models"))) { + p = p.parent + } + assertTrue(p != null, "could not locate src/test/resources/models from user.dir") + p!!.resolve("src/test/resources/models") + } + + private fun compile(outDir: Path): KotlinCompilation.Result { + val sources = Files.walk(outDir).filter { it.isRegularFile() }.sorted().toList() + .map { path -> SourceFile.kotlin(path.parent.relativize(path).toString().replace('/', '_'), path.readText()) } + return KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + } + + @Test fun `int-backed enum emits customEnumeration over INTEGER and still compiles`() { + val outDir = Files.createTempDirectory("ktbl-int-enum-") + try { + val loader = loadDirectory("enum-int-backed", models.resolve("enum-int-backed")) + for (gen in listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator())) { + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(loader) + } + + val table = outDir.resolve("acme/shop/OrderTable.kt") + assertTrue(Files.exists(table), "expected $table; files=${Files.walk(outDir).toList()}") + val src = table.readText() + + // int-backed → customEnumeration over an INTEGER column, carrying the + // declared mapping in BOTH directions. + assertTrue("customEnumeration(\"priority\", \"INTEGER\"" in src, + "priority must use customEnumeration over INTEGER; saw:\n$src") + assertTrue("0 -> OrderPriority.DRAFT" in src && "5 -> OrderPriority.PUBLISHED" in src && + "9 -> OrderPriority.ARCHIVED" in src, + "read lambda must map every declared int to its member; saw:\n$src") + assertTrue("OrderPriority.DRAFT -> 0" in src && "OrderPriority.PUBLISHED -> 5" in src && + "OrderPriority.ARCHIVED -> 9" in src, + "write lambda must map every member to its declared int; saw:\n$src") + + // ...and the int-backed column must NOT take the string form. + assertTrue("enumerationByName(\"priority\"" !in src, + "priority must not emit the string-backed column; saw:\n$src") + + // The string-backed sibling is unchanged — this feature is additive. + assertTrue("enumerationByName(\"status\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, OrderStatus::class)" in src, + "string-backed status must keep enumerationByName; saw:\n$src") + assertTrue("customEnumeration(\"status\"" !in src, + "string-backed status must not become customEnumeration; saw:\n$src") + + // Compile gate: proves both lambdas type-check against the real Exposed API. + val result = compile(outDir) + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + } finally { + outDir.toFile().deleteRecursively() + } + } +} diff --git a/server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json b/server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json new file mode 100644 index 000000000..ca4fc6f36 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json @@ -0,0 +1,30 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "source.rdb": { "name": "src", "@table": "orders", "@kind": "table" } }, + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "priority", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { + "field.enum": { + "name": "status", + "@values": ["LOW", "MEDIUM", "HIGH"] + } + }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] + } + } + ] + } +} From fab7d893e34aabbfb2e710c4d2d5359a615de961 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 18:45:36 -0400 Subject: [PATCH 41/52] feat(csharp,conformance): int-backed enum EF conversion + the cross-port gate Two halves of one gap. The metamodel layer shipped @intValueMap in all five ports, but only TypeScript ever persisted it -- and NO shared corpus covered persistence, so four ports could ignore the attribute and every gate stayed green. This adds the last port and the fixture that makes silence impossible. C#. DbContextGenerator emitted HasConversion() for every field.enum, so an int-backed one stored the member symbol into what migrate provisions as INTEGER. Both enum sites now route through one EnumConversionCall(): the generic HasConversion() when string-backed, or a model->provider / provider->model lambda pair built from the declared map. The scalar, array-element (PrimitiveCollection) and projection/view loops all use it -- the view needed it as much as the table, since reading an INTEGER column as text fails materialization the same way the ordinal default does. The generated C# `enum` declaration is byte- identical either way; int-backing is invisible in the entity's API. - The mapping is a TERNARY CHAIN, not a switch expression: EF converts these lambdas to EXPRESSION TREES and a switch expression is not legal in one (CS8155). - KNOWN PORT ASYMMETRY, deliberate and documented at the call site: the provider->model chain ends on the last member instead of rejecting an int with no member, where Python/Java/Kotlin surface the unmapped value. C# cannot match them -- an expression tree may not contain a throw-expression (CS8188) -- and the column's CHECK constrains the value anyway. - Ints are read THROUGH the map keyed by member, so @values stays the SSOT and a member with no mapping throws at codegen rather than vanishing from the converter. CONFORMANCE. AllTypes -- the write-roundtrip kitchen-sink every port runs -- gains `intEnumVal`, so all five ports now write an int and read back the symbol against real Postgres. It is NULLABLE deliberately: the sibling update-delete scenario seeds all_types by raw SQL and a NOT NULL column would break that seed. DRAFT maps to 0 so the falsy-zero case is covered on the shared corpus, not just in per-port unit tests. The migration fixture additionally pins the lowering -- "intEnumVal" INTEGER with CHECK IN (0, 5, 9), integers unquoted -- because emitting the member symbols there would be un-appliable DDL against an integer column. The committed C# integration fixtures are regenerated, and the diff is exactly four lines: the enum declaration (unchanged in shape), the nullable property, and the conversion. No other enum's HasConversion() moved, which is the evidence that this is additive for string-backed enums. C#: 1579 passed / 0 failed across all four test projects (Render 291, Conformance 897, Cli 46, Codegen 345 + 1 intentionally-skipped regen harness). TS migration conformance 6/0 against real Postgres, proving the hand-edited canonical schema matches what migrate-ts generates. Co-Authored-By: Claude Opus 5 (1M context) --- .../canonical/meta.fitness.json | 2 + .../canonical/schema.postgres.sql | 6 +- .../bootstrap-canonical-from-empty.yaml | 5 ++ .../queries/roundtrip-all-types.yaml | 6 ++ .../Generators/DbContextGenerator.cs | 82 +++++++++++++++++-- .../Generated/AllTypes.g.cs | 3 + .../Generated/AppDbContext.g.cs | 1 + 7 files changed, 99 insertions(+), 6 deletions(-) diff --git a/fixtures/persistence-conformance/canonical/meta.fitness.json b/fixtures/persistence-conformance/canonical/meta.fitness.json index de39f2e59..ccbb328dd 100644 --- a/fixtures/persistence-conformance/canonical/meta.fitness.json +++ b/fixtures/persistence-conformance/canonical/meta.fitness.json @@ -241,6 +241,8 @@ { "field.timestamp": { "name": "tsTzVal", "@required": true } }, { "field.currency": { "name": "moneyVal", "@required": true, "@currency": "USD" } }, { "field.enum": { "name": "enumVal", "@required": true, "@values": ["LOW", "MEDIUM", "HIGH"] } }, + { "field.enum": { "name": "intEnumVal", "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, { "field.uuid": { "name": "uuidVal", "@required": true } }, { "field.uri": { "name": "uriVal", "@required": true } }, { "field.inet": { "name": "inetVal", "@required": true } }, diff --git a/fixtures/persistence-conformance/canonical/schema.postgres.sql b/fixtures/persistence-conformance/canonical/schema.postgres.sql index 2cb6d9852..86b60bb61 100644 --- a/fixtures/persistence-conformance/canonical/schema.postgres.sql +++ b/fixtures/persistence-conformance/canonical/schema.postgres.sql @@ -112,6 +112,7 @@ CREATE TABLE "all_types" ( "tsTzVal" TIMESTAMPTZ NOT NULL, "moneyVal" BIGINT NOT NULL, "enumVal" TEXT NOT NULL, + "intEnumVal" INTEGER, "uuidVal" UUID NOT NULL, "uriVal" TEXT NOT NULL, "inetVal" INET NOT NULL, @@ -119,7 +120,10 @@ CREATE TABLE "all_types" ( "settings" JSONB, "labels" JSONB, CONSTRAINT "all_types_pkey" PRIMARY KEY ("id"), - CONSTRAINT "all_types_enumVal_chk" CHECK ("enumVal" IN ('LOW', 'MEDIUM', 'HIGH')) + CONSTRAINT "all_types_enumVal_chk" CHECK ("enumVal" IN ('LOW', 'MEDIUM', 'HIGH')), + -- int-backed enum (@intValueMap): the column stores the mapped INTEGER, so the + -- membership CHECK lists the integers unquoted, not the member symbols. + CONSTRAINT "all_types_intEnumVal_chk" CHECK ("intEnumVal" IN (0, 5, 9)) ); CREATE UNIQUE INDEX "byTitle" ON "programs" ("title"); diff --git a/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml b/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml index 2a6761dfe..af362329c 100644 --- a/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml +++ b/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml @@ -40,6 +40,11 @@ expect: - 'CREATE TABLE "all_types"' - '"bVal" BOOLEAN' - 'CHECK ("enumVal" IN (''LOW'', ''MEDIUM'', ''HIGH''))' + # int-backed enum (@intValueMap): the column is INTEGER and the membership CHECK + # lists the mapped integers UNQUOTED. Emitting the member symbols here would be + # un-appliable DDL against an integer column, so this pins the lowering. + - '"intEnumVal" INTEGER' + - 'CHECK ("intEnumVal" IN (0, 5, 9))' - 'CREATE UNIQUE INDEX "byTitle" ON "programs" ("title");' - 'ALTER TABLE "weeks" ADD CONSTRAINT "weeks_programId_fk"' # Match both TS (`CREATE VIEW`) and C# (`CREATE OR REPLACE VIEW`, idempotent) diff --git a/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml b/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml index 85ef825f9..93ff84047 100644 --- a/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml +++ b/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml @@ -72,6 +72,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: 199900 enumVal: "MEDIUM" + intEnumVal: "PUBLISHED" uuidVal: "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA" uriVal: "https://example.com/a/b?q=1" inetVal: "192.168.1.1" @@ -96,6 +97,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: "199900" enumVal: "MEDIUM" + intEnumVal: "PUBLISHED" uuidVal: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" uriVal: "https://example.com/a/b?q=1" inetVal: "192.168.1.1" @@ -122,6 +124,7 @@ queries: tsTzVal: "2026-01-15T09:00:00Z" moneyVal: 0 enumVal: "LOW" + intEnumVal: "DRAFT" uuidVal: "11111111-1111-4111-8111-111111111111" uriVal: "urn:isbn:0451450523" inetVal: "10.0.0.5" @@ -143,6 +146,7 @@ queries: tsTzVal: "2026-01-15T09:00:00Z" moneyVal: "0" enumVal: "LOW" + intEnumVal: "DRAFT" uuidVal: "11111111-1111-4111-8111-111111111111" uriVal: "urn:isbn:0451450523" inetVal: "10.0.0.5" @@ -179,6 +183,7 @@ queries: tsTzVal: "2026-12-31T23:59:59.999Z" moneyVal: "9223372036854775807" enumVal: "HIGH" + intEnumVal: "ARCHIVED" uuidVal: "ffffffff-ffff-4fff-bfff-ffffffffffff" uriVal: "https://example.org/path#frag" inetVal: "255.255.255.255" @@ -202,6 +207,7 @@ queries: tsTzVal: "2026-12-31T23:59:59.999Z" moneyVal: "9223372036854775807" enumVal: "HIGH" + intEnumVal: "ARCHIVED" uuidVal: "ffffffff-ffff-4fff-bfff-ffffffffffff" uriVal: "https://example.org/path#frag" inetVal: "255.255.255.255" diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index 596729f0b..437fed530 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -64,8 +64,11 @@ public virtual IEnumerable Generate(GenContext ctx) // column as Int32 at materialization — an InvalidCastException. Mirror the // entity-side HasConversion() so a projection that passes an enum // through (e.g. ProgramView.status over v_program) round-trips. + // An int-backed enum (@intValueMap) column in the view holds the declared INTEGER, + // so it takes the same custom converter pair the table side gets — reading it as + // a string would fail materialization exactly as the ordinal default does here. foreach (var f in p.Fields().Where(f => f.SubType == FIELD_SUBTYPE_ENUM && !f.ResolvedIsArray())) - modelLines.Add($" modelBuilder.Entity<{name}>().Property(x => x.{CSharpNaming.Pascal(f.Name)}).HasConversion();"); + modelLines.Add($" modelBuilder.Entity<{name}>().Property(x => x.{CSharpNaming.Pascal(f.Name)}).{EnumConversionCall(name, p, f)};"); } foreach (var e in objects.Where(o => o.IsEntity() && !o.IsReadOnlyProjection())) { @@ -340,6 +343,70 @@ private static string UsingEntityConfig(M2MNavigation nav) // (single-jsonb-column) VO fields — the read model declares only those (a flattened VO's // per-column spread is out of scope on the view; #214 note), so the emitted config never // references a property the View class does not declare. + /// + /// The declared symbol→int map (@intValueMap) for an int-backed + /// field.enum, or null when the enum is string-backed. + /// + /// + /// ADR-0039 RESOLVING (Attr, not OwnAttr): the map is @values' + /// numeric half — a logical property of the enum vocabulary that inherits through + /// extends — so a field extending a shared abstract enum is int-backed too. + /// The loader's validation reads it own-only, which is correct there: it validates + /// what a declaration itself declares. + /// + private static IReadOnlyDictionary? IntValueMapOf(MetaField f) => + f.Attr(FIELD_ATTR_INT_VALUE_MAP) as IReadOnlyDictionary; + + /// + /// The complete HasConversion call for an enum property: the generic + /// HasConversion<string>() for a string-backed enum, or + /// HasConversion(model→provider, provider→model) built from + /// @intValueMap for an int-backed one. + /// + /// + /// The mapping is emitted as a TERNARY CHAIN rather than a switch + /// expression because EF converts these lambdas to EXPRESSION TREES, and a switch + /// expression is not legal in one (CS8155). A conditional is. + /// KNOWN PORT ASYMMETRY, deliberate: the provider→model chain ends on the last + /// member rather than rejecting an int with no member, where Python, Java and Kotlin + /// surface the unmapped value instead. C# cannot match them here — an expression tree + /// may not contain a throw-expression (CS8188) — and the column's CHECK constrains it + /// to the mapped ints anyway. Documented rather than papered over. + /// + private static string EnumConversionCall(string owner, MetaObject entity, MetaField f) + { + var intMap = IntValueMapOf(f); + if (intMap is null) return "HasConversion()"; + + var members = f.EffectiveEnumValues ?? new List(); + if (members.Count == 0) return "HasConversion()"; + + var type = $"{owner}.{CSharpNaming.EnumTypeName(entity, f)}"; + // Read the ints THROUGH the map, keyed by member, so @values stays the SSOT and a + // member with no mapping cannot silently vanish from the conversion. + var ints = new List(members.Count); + foreach (var m in members) + { + if (!intMap.TryGetValue(m, out var raw) || raw is null) + throw new InvalidOperationException( + $"field.enum '{f.Name}' @{FIELD_ATTR_INT_VALUE_MAP} has no integer for member '{m}' — " + + "cannot build the EF value conversion."); + ints.Add(Convert.ToInt64(raw).ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + + var toProvider = new System.Text.StringBuilder("v => "); + var fromProvider = new System.Text.StringBuilder("v => "); + for (var i = 0; i < members.Count - 1; i++) + { + toProvider.Append($"v == {type}.{members[i]} ? {ints[i]} : "); + fromProvider.Append($"v == {ints[i]} ? {type}.{members[i]} : "); + } + toProvider.Append(ints[^1]); + fromProvider.Append($"{type}.{members[^1]}"); + + return $"HasConversion({toProvider}, {fromProvider})"; + } + private void EmitFieldTypeConfig( string className, MetaObject entity, IEnumerable fields, GenContext ctx, List modelLines, bool jsonbObjectsOnly) @@ -353,13 +420,18 @@ private void EmitFieldTypeConfig( foreach (var f in fieldList.Where(f => f.SubType == FIELD_SUBTYPE_ENUM)) { var prop = CSharpNaming.Pascal(f.Name); + // An int-backed enum (@intValueMap) persists the declared INTEGER instead of the + // member symbol, so it needs a custom converter pair rather than HasConversion(). + // The generated C# `enum` declaration is byte-identical either way — int-backing is a + // persistence concern, invisible in the entity's API. + var conversion = EnumConversionCall(className, entity, f); // ADR-0039: resolving — array-ness inheritable via extends. Array-of-enum uses the - // EF Core 8 primitive collection with a per-element string conversion so members - // persist as symbols (["DRAFT"]), not int ordinals ([0]). + // EF Core 8 primitive collection with a per-element conversion so members persist as + // symbols (["DRAFT"]) — or as their declared ints — not as int ordinals ([0]). if (f.ResolvedIsArray()) - modelLines.Add($" modelBuilder.Entity<{className}>().PrimitiveCollection(x => x.{prop}).ElementType().HasConversion();"); + modelLines.Add($" modelBuilder.Entity<{className}>().PrimitiveCollection(x => x.{prop}).ElementType().{conversion};"); else - modelLines.Add($" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasConversion();"); + modelLines.Add($" modelBuilder.Entity<{className}>().Property(x => x.{prop}).{conversion};"); } foreach (var f in fieldList.Where(f => f.ResolvedIsArray() && CSharpNaming.ScalarFor(f.SubType) is not null)) diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs index cafb2c3e6..80116f4a0 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs @@ -14,6 +14,7 @@ namespace MetaObjects.IntegrationTests.Generated; public class AllTypes { public enum AllTypesEnumVal { LOW, MEDIUM, HIGH } + public enum AllTypesIntEnumVal { DRAFT, PUBLISHED, ARCHIVED } [Key] [Column("id")] public Guid Id { get; set; } @@ -46,6 +47,8 @@ public enum AllTypesEnumVal { LOW, MEDIUM, HIGH } public long MoneyVal { get; set; } [Column("enumVal")] public AllTypesEnumVal EnumVal { get; set; } + [Column("intEnumVal")] + public AllTypesIntEnumVal? IntEnumVal { get; set; } [Column("uuidVal")] public Guid UuidVal { get; set; } [Column("uriVal")] diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs index 65b3c5979..f53ac22ed 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs @@ -33,6 +33,7 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) modelBuilder.Entity().OwnsOne(x => x.Settings, b => b.ToJson("settings")); modelBuilder.Entity().OwnsMany(x => x.Labels, b => b.ToJson("labels")); modelBuilder.Entity().Property(x => x.EnumVal).HasConversion(); + modelBuilder.Entity().Property(x => x.IntEnumVal).HasConversion(v => v == AllTypes.AllTypesIntEnumVal.DRAFT ? 0 : v == AllTypes.AllTypesIntEnumVal.PUBLISHED ? 5 : 9, v => v == 0 ? AllTypes.AllTypesIntEnumVal.DRAFT : v == 5 ? AllTypes.AllTypesIntEnumVal.PUBLISHED : AllTypes.AllTypesIntEnumVal.ARCHIVED); modelBuilder.Entity().Property(x => x.DecVal).HasPrecision(18, 6); modelBuilder.Entity().Property(x => x.TsVal).HasColumnType("timestamp without time zone"); modelBuilder.Entity().Property(x => x.TsTzVal).HasColumnType("timestamp with time zone"); From 84d8da148373cd0773a7462925556cbdab3848a2 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 18:45:36 -0400 Subject: [PATCH 42/52] chore(python): sync uv.lock to the 0.23.1 version bump The 0.23.1 release commit bumped pyproject.toml but left uv.lock pinning 0.23.0; running the suite regenerated it. Release hygiene, unrelated to int-backed enums. Co-Authored-By: Claude Opus 5 (1M context) --- server/python/uv.lock | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/python/uv.lock b/server/python/uv.lock index 8b7d7b7c1..3e9a8aee2 100644 --- a/server/python/uv.lock +++ b/server/python/uv.lock @@ -250,7 +250,7 @@ wheels = [ [[package]] name = "metaobjects" -version = "0.23.0" +version = "0.23.1" source = { editable = "." } dependencies = [ { name = "pyyaml" }, From f2aad5f99f84491c4443565c73c01e4d6b7863dd Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 18:56:36 -0400 Subject: [PATCH 43/52] refactor(int-enum): simplifier pass over the four ports Behaviour-preserving; every suite re-verified after. ONE REAL DEFECT, in Java. The EnumCodec change added its new Javadoc block WITHOUT removing the existing one, leaving two stacked /** */ comments. Java binds only the closest to the class, so the original block -- why enum is registered explicitly rather than riding the generic ObjectCodec fallback, and that the DB CHECK enforces membership -- became an orphaned comment invisible to Javadoc. Merged into one block preserving every rationale point from both. PYTHON, aligned with its siblings. It was the only port inlining the get_meta_attr + isinstance(dict) read twice, duplicating the ADR-0039 rationale at both sites; Java, Kotlin and C# each already factor this into a helper. Extracted _int_value_map() to match. Also hoisted the loop-invariant int/bool check out of the decode loop: a non-int value used to iterate the whole map, match nothing, and fall through to `return value` -- now it returns immediately. Equivalent by case analysis (a bool or non-int could never satisfy the old per-entry condition either). C#, pure reordering. The new IntValueMapOf/EnumConversionCall pair had been inserted BETWEEN the `#214 [0]` comment block and EmitFieldTypeConfig, the method that comment describes -- so ~60 lines of a reader's attention would misattribute it. Moved above that comment. Verified pure: the diff's added and removed line sets are identical. Deliberately NOT changed: the `e.getValue() != null` guard in Java's int-map loop is provably dead today (values are built via .intValue()), but it guards the helper's construction changing underneath it and costs nothing. Kotlin needed no change. Verified after: Python 19 passed; Java JdbcCodecRoundTripTest 7 run / 0 failures / 0 skipped; C# 345 passed / 0 failed / 1 skipped (the regen harness). Co-Authored-By: Claude Opus 5 (1M context) --- .../Generators/DbContextGenerator.cs | 36 +++++++++---------- .../manager/db/codec/JdbcCodecs.java | 20 +++++------ .../src/metaobjects/runtime/object_manager.py | 30 +++++++++++----- 3 files changed, 48 insertions(+), 38 deletions(-) diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index 437fed530..c57d14792 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -325,24 +325,6 @@ private static string UsingEntityConfig(M2MNavigation nav) $"r => r.HasOne<{source}>().WithMany().HasForeignKey(nameof({through}.{sourceFkProp})));"; } - // #214 [0] — the per-field EF TYPE-converter emission, factored out so the SAME set of - // converters configures BOTH the write entity (its derived-EXCLUDED field set) AND a - // write-through entity's View read model (ALL fields, incl. the derived origin.* - // fields the replica view exposes). Emits, over modelBuilder.Entity<>(): owned-VO (OwnsOne/OwnsMany.ToJson or flattened per-column names); - // enum string-conversion (scalar HasConversion + array PrimitiveCollection element - // conversion — else enum arrays persist as int ordinals); scalar PrimitiveCollection (EF - // Core 8 API — .ToJson does not exist on PropertyBuilder>); field.decimal - // .HasPrecision (SP-A, precision-exact NUMERIC vs EF's default decimal(18,2)); field.timestamp - // .HasColumnType (ADR-0036 Wave 2 — timestamptz default / `timestamp without time zone` under - // @localTime, REQUIRED else Npgsql rejects a Kind=Unspecified DateTime); field.uri (Uri↔text - // converter) / field.inet (native `inet`) (ADR-0036 Wave 3); and @dbColumnType uuid/jsonb - // physical overrides (R6 Plan 2b). The WRITE-ONLY configs (@readOnly SetAfterSaveBehavior, - // M:N UsingEntity, TPH HasDiscriminator) stay on the caller — a read-only view is never - // written. restricts the owned-VO loop to non-flattened - // (single-jsonb-column) VO fields — the read model declares only those (a flattened VO's - // per-column spread is out of scope on the view; #214 note), so the emitted config never - // references a property the View class does not declare. /// /// The declared symbol→int map (@intValueMap) for an int-backed /// field.enum, or null when the enum is string-backed. @@ -407,6 +389,24 @@ private static string EnumConversionCall(string owner, MetaObject entity, MetaFi return $"HasConversion({toProvider}, {fromProvider})"; } + // #214 [0] — the per-field EF TYPE-converter emission, factored out so the SAME set of + // converters configures BOTH the write entity (its derived-EXCLUDED field set) AND a + // write-through entity's View read model (ALL fields, incl. the derived origin.* + // fields the replica view exposes). Emits, over modelBuilder.Entity<>(): owned-VO (OwnsOne/OwnsMany.ToJson or flattened per-column names); + // enum string-conversion (scalar HasConversion + array PrimitiveCollection element + // conversion — else enum arrays persist as int ordinals); scalar PrimitiveCollection (EF + // Core 8 API — .ToJson does not exist on PropertyBuilder>); field.decimal + // .HasPrecision (SP-A, precision-exact NUMERIC vs EF's default decimal(18,2)); field.timestamp + // .HasColumnType (ADR-0036 Wave 2 — timestamptz default / `timestamp without time zone` under + // @localTime, REQUIRED else Npgsql rejects a Kind=Unspecified DateTime); field.uri (Uri↔text + // converter) / field.inet (native `inet`) (ADR-0036 Wave 3); and @dbColumnType uuid/jsonb + // physical overrides (R6 Plan 2b). The WRITE-ONLY configs (@readOnly SetAfterSaveBehavior, + // M:N UsingEntity, TPH HasDiscriminator) stay on the caller — a read-only view is never + // written. restricts the owned-VO loop to non-flattened + // (single-jsonb-column) VO fields — the read model declares only those (a flattened VO's + // per-column spread is out of scope on the view; #214 note), so the emitted config never + // references a property the View class does not declare. private void EmitFieldTypeConfig( string className, MetaObject entity, IEnumerable fields, GenContext ctx, List modelLines, bool jsonbObjectsOnly) diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java index 3ca9fa06a..2786f63f0 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java @@ -407,18 +407,16 @@ static final class CurrencyCodec implements JdbcFieldCodec { } /** - * {@code field.enum} ⇄ a text column. The enum is string-backed (its member symbol is the - * stored value — see {@code EnumField}); EnumField is backed by {@code DataTypes.STRING}, so - * this is bind-as-string / read-as-string, identical to {@link StringCodec} but registered - * explicitly so enum does not ride the generic {@link ObjectCodec} fallback. The DB - * {@code CHECK (col IN (...))} (emitted from {@code @values}) enforces membership. - */ - /** - * {@code field.enum}, string- or int-backed. + * {@code field.enum} ⇄ a text OR integer column, string- or int-backed. + * + *

      A plain enum persists its member symbol as text (see {@code EnumField}, backed by + * {@code DataTypes.STRING}) — bind-as-string / read-as-string, identical to + * {@link StringCodec} but registered explicitly so enum does not ride the generic + * {@link ObjectCodec} fallback. The DB {@code CHECK (col IN (...))} (emitted from + * {@code @values}) enforces membership.

      * - *

      A plain enum persists its member symbol as text. One declaring - * {@code @intValueMap} persists the member's declared INTEGER instead, while the - * caller's contract stays the SYMBOL in both directions — int-backing is a + *

      One declaring {@code @intValueMap} persists the member's declared INTEGER instead, + * while the caller's contract stays the SYMBOL in both directions — int-backing is a * persistence-layer concern, invisible above this codec.

      * *

      The map is read RESOLVING ({@code getMetaAttr}, ADR-0039): it is diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index cf1b2286f..fbc59276d 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -834,15 +834,11 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: # and wire contract stays the member SYMBOL (int-backing is a persistence-layer # concern, invisible above this codec), so encode symbol -> declared int here. # - # ADR-0039 resolving: the map is @values' numeric half — a logical property of - # the enum vocabulary that inherits through extends — so it is read with - # get_meta_attr, NOT the own-only accessor (contrast @dbColumnType above). - # # An unmapped symbol is passed through untouched: membership is the column's # CHECK constraint to enforce, and inventing a value here would hide the drift. if sub == fc.FIELD_SUBTYPE_ENUM: - int_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) - if isinstance(int_map, dict) and value in int_map: + int_map = _int_value_map(field) + if int_map is not None and value in int_map: return int_map[value] # Everything else (string / int / long / double / float / boolean / enum) @@ -850,6 +846,22 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: return value +def _int_value_map(field: MetaField) -> dict[Any, Any] | None: + """The declared ``@intValueMap`` (symbol → int), or ``None`` when the enum is + string-backed. + + ADR-0039 resolving: the map is ``@values``' numeric half — a logical property + of the enum vocabulary that inherits through ``extends`` — so it is read with + ``get_meta_attr``, NOT the own-only accessor (contrast ``@dbColumnType`` in + :func:`_coerce_write_value`, the one field attribute that is deliberately + own-only). Shared by the write and read halves of the enum codec, mirroring + the dedicated int-map helper each sibling port's codec keeps (Java + ``EnumCodec.intValueMap``, Kotlin ``readIntValueMap``, C# ``IntValueMapOf``). + """ + m = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + return m if isinstance(m, dict) else None + + def _decode_read_value(field: MetaField, value: Any) -> Any: """Decode a stored value back to its authoring form on read. @@ -866,11 +878,11 @@ def _decode_read_value(field: MetaField, value: Any) -> Any: return None if field.sub_type != fc.FIELD_SUBTYPE_ENUM: return value - int_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) # ADR-0039 resolving - if not isinstance(int_map, dict): + int_map = _int_value_map(field) + if int_map is None or not isinstance(value, int) or isinstance(value, bool): return value for symbol, stored in int_map.items(): - if stored == value and isinstance(value, int) and not isinstance(value, bool): + if stored == value: return symbol return value From 4d94464cd9503fcfd98a20eee4271169bdf0aaa9 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 19:14:30 -0400 Subject: [PATCH 44/52] =?UTF-8?q?fix(int-enum):=20review=20findings=20?= =?UTF-8?q?=E2=80=94=20C#=20shared-enum=20CS0426,=20Python=20filter=20enco?= =?UTF-8?q?ding,=20a=20leaked=20home=20path?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three confirmed defects from an adversarial cross-port review. Each was verified against the code before fixing; each fix is proven non-vacuous. 1. CRITICAL (C#) -- non-compiling EF config for the ONLY legal way to int-back a shared enum. EnumConversionCall named the enum type {owner}.{EnumTypeName}, but EntityGenerator deliberately does NOT nest a shared/@provided enum (it references it: "shared/provided -> referenced, not nested"), so the emitted HasConversion(v => v == Ticket.Priority.LOW ? 1 : ...) is CS0426. Not an edge case: ERR_ENUM_EXTENDS_VALUES_CONFLICT makes declaring @intValueMap on the CONSUMING field a load error, so hanging it on the shared declaration is the only legal authoring shape. It stayed latent because HasConversion() names no type at all. Now routed through Fr019SharedEnum.SharedEnumForField / SharedEnumTypeReference, the same resolution EntityGenerator's own EnumPropertyTypeName uses. GATED by extending the EF-Core-8 Roslyn compile fixture with both shapes: a shared root-level abstract int-backed enum consumed via extends, and an inline int-backed enum. PROVEN NON-VACUOUS -- reverting just the type-name resolution turns that compile test red. 2. HIGH (Python) -- every query on an int-backed enum failed. _compile_filter and _op_clause bound values RAW; the write codec was applied only to INSERT/UPDATE params and the PK. The other four ports all encode on this path (TS via the Drizzle customType, Java via GenericSQLDriver.setStatementValue -> EnumCodec, Kotlin via Exposed toDb, C# via the EF converter), so Python alone bound the member SYMBOL against an INTEGER column -> pg 22P02. That silently contradicted the filter-band rationale every port carries: eq/ne/in survive for an int-backed enum BECAUSE the symbol encodes to its integer before reaching SQL. Bound values now go through _coerce_write_value, per-element for `in`, skipped for isNull. 5 new tests pin eq / shortcut-equality / in / string-backed-unchanged / isNull. 3. PUBLIC-REPO HYGIENE -- a developer's absolute home path (/Users//...) was committed in a plan doc, twice. Replaced with . The pre-commit guard matched /home// but NOT the macOS /Users/ shape, which is exactly why it passed; the pattern now covers both. Also extends the shared corpus: update-delete-all-types now sets intEnumVal, so the UPDATE write path and its RETURNING decode are covered cross-port. They had NO coverage -- the roundtrip scenario only exercises INSERT. Verified: C# 345 passed / 0 failed; Python 24 targeted + 27 real-Postgres integration scenarios green. Co-Authored-By: Claude Opus 5 (1M context) --- .githooks/pre-commit | 4 +- ...-07-23-int-backed-enum-values-metamodel.md | 4 +- .../queries/update-delete-all-types.yaml | 3 ++ .../DbContextCompileTests.cs | 5 ++ .../Generators/DbContextGenerator.cs | 18 +++++-- .../src/metaobjects/runtime/object_manager.py | 23 ++++++-- .../test_object_manager_enum_intvaluemap.py | 53 +++++++++++++++++++ 7 files changed, 98 insertions(+), 12 deletions(-) diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 0b5e5bce0..71cc6713a 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -18,7 +18,9 @@ set -uo pipefail # Generic, non-sensitive structural patterns — safe to commit (name no project): -PATTERNS='/home/[A-Za-z0-9._-]+/|~/Development' +# /Users/ is the macOS home shape. It was missing until a plan doc authored on a Mac +# committed an absolute developer home path that this hook scanned and passed. +PATTERNS='/home/[A-Za-z0-9._-]+/|/Users/[A-Za-z0-9._-]+/|~/Development' # Legitimate matches to ignore (e.g. the published npm author email). ALLOW='doug@dougmealing\.com' diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md index 87d35b479..78087bf69 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md @@ -178,7 +178,7 @@ Edit `spec/metamodel/attr.json` — insert alphabetically between the `int` and - [ ] **Step 7: Regenerate the embedded attr definition** -Run: `cd /Users/douglas.mealing/Development/metaobjects && bun scripts/generate-embedded-metamodel.ts` +Run: `cd && bun scripts/generate-embedded-metamodel.ts` Expected: regenerates `server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts` to include the new `intMap` block. - [ ] **Step 8: Run the test to verify it passes** @@ -312,7 +312,7 @@ Edit `spec/metamodel/field.json` — add as a sibling of `values`/`provided` ins - [ ] **Step 5: Regenerate the embedded field definition** -Run: `cd /Users/douglas.mealing/Development/metaobjects && bun scripts/generate-embedded-metamodel.ts` +Run: `cd && bun scripts/generate-embedded-metamodel.ts` Expected: regenerates `field-definition.embedded.ts`. - [ ] **Step 6: Run tests again — confirm the ERR_UNKNOWN_ATTR failure is gone, new failures are the content-rule assertions** diff --git a/fixtures/persistence-conformance/queries/update-delete-all-types.yaml b/fixtures/persistence-conformance/queries/update-delete-all-types.yaml index 8683ac16d..35edbcfe3 100644 --- a/fixtures/persistence-conformance/queries/update-delete-all-types.yaml +++ b/fixtures/persistence-conformance/queries/update-delete-all-types.yaml @@ -75,6 +75,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: 250000 enumVal: "HIGH" + intEnumVal: "PUBLISHED" uuidVal: "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA" uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" @@ -95,6 +96,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: "250000" enumVal: "HIGH" + intEnumVal: "PUBLISHED" uuidVal: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" @@ -123,6 +125,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: "250000" enumVal: "HIGH" + intEnumVal: "PUBLISHED" uuidVal: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" diff --git a/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs b/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs index b42728bfd..2873773bf 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs @@ -48,6 +48,9 @@ public class DbContextCompileTests // (see the file-header comment for the field-by-field breakdown). private const string Model = """ { "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "Priority", "abstract": true, + "@values": ["LOW", "HIGH"], + "@intValueMap": { "LOW": 1, "HIGH": 9 } } }, { "object.value": { "name": "Address", "children": [ { "field.string": { "name": "street", "@required": true, "@maxLength": 120 } }, { "field.string": { "name": "city", "@maxLength": 80 } } @@ -57,6 +60,8 @@ public class DbContextCompileTests { "field.long": { "name": "id" } }, { "field.enum": { "name": "status", "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, { "field.enum": { "name": "statuses", "isArray": true, "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { "field.enum": { "name": "priority", "extends": "Priority" } }, + { "field.enum": { "name": "rank", "@values": ["A", "B"], "@intValueMap": { "A": 0, "B": 7 } } }, { "field.string": { "name": "tags", "isArray": true } }, { "field.object": { "name": "homeAddress", "@objectRef": "Address", "@storage": "flattened" } }, { "field.object": { "name": "config", "@objectRef": "Address" } }, diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index c57d14792..32ff0f8ee 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -68,7 +68,7 @@ public virtual IEnumerable Generate(GenContext ctx) // so it takes the same custom converter pair the table side gets — reading it as // a string would fail materialization exactly as the ordinal default does here. foreach (var f in p.Fields().Where(f => f.SubType == FIELD_SUBTYPE_ENUM && !f.ResolvedIsArray())) - modelLines.Add($" modelBuilder.Entity<{name}>().Property(x => x.{CSharpNaming.Pascal(f.Name)}).{EnumConversionCall(name, p, f)};"); + modelLines.Add($" modelBuilder.Entity<{name}>().Property(x => x.{CSharpNaming.Pascal(f.Name)}).{EnumConversionCall(name, p, f, ctx.Config)};"); } foreach (var e in objects.Where(o => o.IsEntity() && !o.IsReadOnlyProjection())) { @@ -355,7 +355,7 @@ private static string UsingEntityConfig(M2MNavigation nav) /// may not contain a throw-expression (CS8188) — and the column's CHECK constrains it /// to the mapped ints anyway. Documented rather than papered over. /// - private static string EnumConversionCall(string owner, MetaObject entity, MetaField f) + private static string EnumConversionCall(string owner, MetaObject entity, MetaField f, GenConfig config) { var intMap = IntValueMapOf(f); if (intMap is null) return "HasConversion()"; @@ -363,7 +363,17 @@ private static string EnumConversionCall(string owner, MetaObject entity, MetaFi var members = f.EffectiveEnumValues ?? new List(); if (members.Count == 0) return "HasConversion()"; - var type = $"{owner}.{CSharpNaming.EnumTypeName(entity, f)}"; + // FR-019: a SHARED (root-level abstract) or @provided enum is NOT nested inside the + // entity class — EntityGenerator references it instead (see its EnumPropertyTypeName) + // — so it must be named unqualified here. Qualifying it as {owner}.{Name} emits + // CS0426 ("the type name does not exist in the type"), which is not an edge case: + // ERR_ENUM_EXTENDS_VALUES_CONFLICT makes declaring @intValueMap on the CONSUMING + // field a load error, so hanging it on the shared declaration is the only legal way + // to int-back a shared enum. String-backed shared enums never showed this because + // HasConversion() names no type at all. + var type = Fr019SharedEnum.SharedEnumForField(f) is { } shared + ? Fr019SharedEnum.SharedEnumTypeReference(shared, config) + : $"{owner}.{CSharpNaming.EnumTypeName(entity, f)}"; // Read the ints THROUGH the map, keyed by member, so @values stays the SSOT and a // member with no mapping cannot silently vanish from the conversion. var ints = new List(members.Count); @@ -424,7 +434,7 @@ private void EmitFieldTypeConfig( // member symbol, so it needs a custom converter pair rather than HasConversion(). // The generated C# `enum` declaration is byte-identical either way — int-backing is a // persistence concern, invisible in the entity's API. - var conversion = EnumConversionCall(className, entity, f); + var conversion = EnumConversionCall(className, entity, f, ctx.Config); // ADR-0039: resolving — array-ness inheritable via extends. Array-of-enum uses the // EF Core 8 primitive collection with a per-element conversion so members persist as // symbols (["DRAFT"]) — or as their declared ints — not as int ordinals ([0]). diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index fbc59276d..cf8b1c28f 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -718,12 +718,13 @@ def _compile_filter(f: Filter | None, entity: MetaObject) -> tuple[str, list[Any mf = entity.find_field(field_name) col = _column_of(mf) if mf is not None else field_name if not isinstance(ops, dict): - # Shortcut: {field: value} → equality + # Shortcut: {field: value} → equality. Encoded like any other bound + # value (see _op_clause) so an int-backed enum reaches SQL as its int. parts.append(f"{_q(col)} = %s") - params.append(ops) + params.append(_coerce_write_value(mf, ops) if mf is not None else ops) continue for op, value in ops.items(): - sql, p = _op_clause(col, op, value) + sql, p = _op_clause(col, op, value, mf) parts.append(sql) params.extend(p) if not parts: @@ -731,9 +732,21 @@ def _compile_filter(f: Filter | None, entity: MetaObject) -> tuple[str, list[Any return " AND ".join(parts), params -def _op_clause(col: str, op: str, value: Any) -> tuple[str, list[Any]]: - """Translate one operator → SQL + params. Mirrors TS/C#/Java semantics.""" +def _op_clause(col: str, op: str, value: Any, field: MetaField | None = None) -> tuple[str, list[Any]]: + """Translate one operator → SQL + params. Mirrors TS/C#/Java semantics. + + Bound values go through the WRITE codec, exactly as an INSERT's do: the four + sibling ports all encode on this path (TS through the Drizzle customType, Java + through GenericSQLDriver.setStatementValue → EnumCodec.write, Kotlin through + Exposed's toDb, C# through the EF converter). Without it an int-backed enum's + filter binds the member SYMBOL against an INTEGER column — the filter band + keeps eq/ne/in for int-backed enums precisely BECAUSE the symbol is supposed + to encode to its integer before reaching SQL. + """ qc = _q(col) + if field is not None and op != "isNull": + value = ([_coerce_write_value(field, v) for v in value] + if op == "in" and value else _coerce_write_value(field, value)) if op == "eq": return f"{qc} = %s", [value] if op == "ne": return f"{qc} <> %s", [value] if op == "gt": return f"{qc} > %s", [value] diff --git a/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py index cb7fa3b1d..745bbc692 100644 --- a/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +++ b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py @@ -129,3 +129,56 @@ def test_intvaluemap_inherited_through_extends_is_honoured(): field = _field_of(json_str, "Order", "status") assert _coerce_write_value(field, "PUBLISHED") == 5 assert _decode_read_value(field, 5) == "PUBLISHED" + + +# --- filter encoding ------------------------------------------------------ +# +# The filter band keeps eq/ne/in for an int-backed enum precisely BECAUSE the +# member symbol is supposed to encode to its integer before reaching SQL. Python +# was the only port not encoding on this path, so every query bound the SYMBOL +# against an INTEGER column (pg 22P02). These pin the encoding at the SQL boundary. + + +def _order_entity(extra: str): + json_str = f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "source.rdb": {{ "name": "src", "@table": "orders", "@kind": "table" }} }}, + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + result = MetaDataLoader().load([InMemoryStringSource(json_str, "test.json")]) + assert result.errors == [] + return next(c for c in result.root.children() if c.name == "Order") + + +def test_filter_eq_on_int_backed_enum_binds_the_int(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": {"eq": "PUBLISHED"}}, _order_entity(INT_MAP)) + assert params == [5] + + +def test_filter_shortcut_equality_on_int_backed_enum_binds_the_int(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": "DRAFT"}, _order_entity(INT_MAP)) + assert params == [0] + + +def test_filter_in_on_int_backed_enum_binds_each_int(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": {"in": ["DRAFT", "ARCHIVED"]}}, _order_entity(INT_MAP)) + assert params == [0, 9] + + +def test_filter_on_string_backed_enum_is_unchanged(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": {"eq": "PUBLISHED"}}, _order_entity("")) + assert params == ["PUBLISHED"] + + +def test_filter_isnull_binds_no_param_and_is_not_coerced(): + from metaobjects.runtime.object_manager import _compile_filter + sql, params = _compile_filter({"status": {"isNull": True}}, _order_entity(INT_MAP)) + assert params == [] + assert "IS NULL" in sql From d1c168f9435efc50724653748904c2c6d62fccaf Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 20:40:23 -0400 Subject: [PATCH 45/52] =?UTF-8?q?feat(all-ports):=20@intValueMap=20is=20sc?= =?UTF-8?q?alar-only=20=E2=80=94=20reject=20isArray=20at=20load?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reverses design D7 ("array-of-enum composes unchanged"), which assumed the element codec would fall out of the scalar one. It does not. Int-backing is a persistence-layer CODEC and every port's codec seam is scalar by construction: Python's ObjectManager tests `value in int_map`, false for a list, so it bound the symbol LIST into an integer[]; OMDB's EnumCodec and Kotlin's customEnumeration bind one value; and TypeScript's sqlite branch serializes an array as JSON text before the enum case is ever reached, storing symbols. Only TS/Postgres and C# composed. Two ports composing while four silently get it wrong is not a feature — it is the field.byte/short/class mistake, vocabulary that reads as supported and is not. Rejecting at LOAD delivers the guarantee that was actually missing: identical behaviour in every port. An array-of-enum stays string-backed. New cross-port error ERR_ENUM_INT_VALUE_MAP_ARRAY, in all four loaders (Kotlin inherits the JVM one). Both halves are read RESOLVING, unlike the @intValueMap content rules: the illegal thing is the EFFECTIVE combination. Post-#246 the map must live on the shared abstract declaration while isArray is declared by the consuming field, so an own-only read would see the two halves on different nodes and never fire — which is why the inherited case gets its own fixture rather than being assumed to follow. The positive enum-int-backed-array fixture becomes the negative error-enum-intvaluemap-array (its input is unchanged — the same metadata, now rejected), joined by error-enum-intvaluemap-array-inherited for the canonical shared-enum authoring shape. Corpus goes 271 -> 286 in CONFORMANCE.md/CLAUDE.md (the count had drifted independently of this change). Verified non-vacuous per port rather than trusted: both fixtures appear by name in the Java surefire XML, match 2 collected pytest cases, and run under the TS and C# directory-scan runners with no expected-failures ledger entry. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 2 +- docs/CONFORMANCE.md | 4 +- fixtures/conformance/ERROR-CODES.json | 3 +- .../enum-int-backed-array/expected.json | 43 ------------------- .../expected-errors.json | 15 +++++++ .../input/meta.enums.json | 25 +++++++++++ .../expected-errors.json | 15 +++++++ .../input/meta.enums.json | 0 server/csharp/MetaObjects/Errors.cs | 6 +++ .../MetaObjects/Loader/ValidationPasses.cs | 26 +++++++++++ .../main/java/com/metaobjects/ErrorCode.java | 9 ++++ .../metaobjects/loader/ValidationPhase.java | 35 +++++++++++++++ .../util/ErrorMessageConstants.java | 10 +++++ server/python/src/metaobjects/errors.py | 6 +++ .../metaobjects/loader/validation_passes.py | 37 ++++++++++++++++ .../packages/codegen-ts/src/column-mapper.ts | 10 +++-- .../column-mapper-enum-intvaluemap.test.ts | 16 +++++-- .../metadata/src/attr-schema-validate.ts | 27 ++++++++++++ .../packages/metadata/src/errors.ts | 6 +++ ...r-schema-validate-enum-intvaluemap.test.ts | 37 ++++++++++++++++ .../expected-schema-enum-intvaluemap.test.ts | 35 ++++++--------- 21 files changed, 292 insertions(+), 75 deletions(-) delete mode 100644 fixtures/conformance/enum-int-backed-array/expected.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json create mode 100644 fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json rename fixtures/conformance/{enum-int-backed-array => error-enum-intvaluemap-array}/input/meta.enums.json (100%) diff --git a/CLAUDE.md b/CLAUDE.md index d22e2a0d3..ee08e1f9c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,7 +50,7 @@ _Last refreshed 2026-08-16._ - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (271 fixtures; 19 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (286 fixtures; 19 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index ef6469062..09ae7efaa 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -25,7 +25,7 @@ regenerate with `ls -d fixtures//*/ | wc -l`. | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 271 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 286 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 15 | 15 / 15 | 14 / 15 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 14 / 15 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 15 / 15 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 31 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | @@ -69,7 +69,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. ## Fixture-to-doc mapping -### `fixtures/conformance/` — metamodel loader + canonical serializer (271) +### `fixtures/conformance/` — metamodel loader + canonical serializer (286) | Fixture prefix | Feature doc | |---|---| diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 694f731b2..8bd3a0dee 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -76,6 +76,7 @@ "ERR_SQL_BODY_WITH_UNMANAGED": "#208: a source.rdb declares both @sql (author-supplied body) and @unmanaged (DDL owned elsewhere). The two markers are the mutually exclusive non-default states of one DDL-ownership axis \u2014 contradictory on a single source.", "ERR_SQL_BODY_ON_WRITABLE_KIND": "#208: a source.rdb declares @sql with a writable @kind (\"table\", the default). A hand-written CREATE TABLE would bypass the column-diff machinery; tables are fully modeled or @unmanaged, never opaque-bodied.", "ERR_ORIGIN_UNDER_SQL_BODY": "#208: a host object whose read source carries @sql also declares an origin.*-bearing (derived) field, or (on object.projection) an @filter (#207) \u2014 two sources of truth for the same body (the synthesized derivation/outer-WHERE vs. the author's verbatim SQL). Fail-closed.", - "ERR_ENUM_EXTENDS_VALUES_CONFLICT": "A field.enum both extends a shared package-level abstract enum and declares its own @values. One shared enum type has one member set \u2014 the own @values would be silently dropped in codegen. Remove the own @values to inherit the shared set, or extend a concrete (non-shared) enum instead." + "ERR_ENUM_EXTENDS_VALUES_CONFLICT": "A field.enum both extends a shared package-level abstract enum and declares its own @values. One shared enum type has one member set \u2014 the own @values would be silently dropped in codegen. Remove the own @values to inherit the shared set, or extend a concrete (non-shared) enum instead.", + "ERR_ENUM_INT_VALUE_MAP_ARRAY": "A field.enum carries @intValueMap together with isArray=true. Int-backing is a persistence-layer codec and no port implements it element-wise over an array column, so the combination would silently persist member SYMBOLS into an integer array. An array-of-enum stays string-backed: drop @intValueMap, or make the field scalar." } } diff --git a/fixtures/conformance/enum-int-backed-array/expected.json b/fixtures/conformance/enum-int-backed-array/expected.json deleted file mode 100644 index ce86c7edf..000000000 --- a/fixtures/conformance/enum-int-backed-array/expected.json +++ /dev/null @@ -1,43 +0,0 @@ -{ - "metadata.root": { - "package": "acme", - "children": [ - { - "object.entity": { - "name": "Ticket", - "children": [ - { - "field.long": { - "name": "id" - } - }, - { - "field.enum": { - "name": "labels", - "isArray": true, - "@intValueMap": { - "HIGH": 3, - "LOW": 1, - "MEDIUM": 2 - }, - "@values": [ - "LOW", - "MEDIUM", - "HIGH" - ] - } - }, - { - "identity.primary": { - "name": "id", - "@fields": [ - "id" - ] - } - } - ] - } - } - ] - } -} diff --git a/fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json new file mode 100644 index 000000000..dacdbb47d --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_INT_VALUE_MAP_ARRAY", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json new file mode 100644 index 000000000..1924ebfa6 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json @@ -0,0 +1,25 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Status", + "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status", "isArray": true } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json new file mode 100644 index 000000000..6b7767484 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_INT_VALUE_MAP_ARRAY", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/enum-int-backed-array/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-array/input/meta.enums.json similarity index 100% rename from fixtures/conformance/enum-int-backed-array/input/meta.enums.json rename to fixtures/conformance/error-enum-intvaluemap-array/input/meta.enums.json diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index a8880dc1e..2c6c8e6f5 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -187,6 +187,12 @@ public enum ErrorCode // own @values would be silently dropped in codegen. Remove the own @values // to inherit the shared set, or extend a concrete (non-shared) enum instead. ERR_ENUM_EXTENDS_VALUES_CONFLICT, + // A field.enum carries @intValueMap together with isArray=true. Int-backing is + // a persistence-layer codec and no port implements it element-wise over an + // array column, so the combination would silently persist member SYMBOLS into + // an integer array. An array-of-enum stays string-backed: drop @intValueMap, + // or make the field scalar. + ERR_ENUM_INT_VALUE_MAP_ARRAY, ERR_UNKNOWN, } diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 054a7d5b8..c4c903579 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -2526,6 +2526,32 @@ private static void WalkEnumValues(MetaData node, List errors) } } } + + // Design D7, narrowed: @intValueMap is scalar-only. Int-backing is a + // persistence-layer CODEC and no port implements it element-wise over an + // array column: OMDB's EnumCodec and Kotlin's customEnumeration are scalar + // by construction, Python would bind the symbol LIST into an integer[], and + // TypeScript's sqlite branch serializes an array as JSON text before the enum + // case is reached. Two ports that happen to compose (this one, via + // PrimitiveCollection().ElementType(), and TS/Postgres) are not a feature — + // shipping a claim four ports silently get wrong is the + // field.byte/short/class mistake. + // + // BOTH halves are read RESOLVING, unlike the content rules above: the illegal + // thing is the EFFECTIVE combination. Post-#246 the map must live on the + // shared abstract declaration, so the field that inherits it is exactly where + // isArray gets declared — an own-only read would see the two halves on + // different nodes and never fire. + if (field.Attr(FIELD_ATTR_INT_VALUE_MAP) is IReadOnlyDictionary + && field.ResolvedIsArray()) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' declares '@{FIELD_ATTR_INT_VALUE_MAP}' with isArray=true; " + + "int-backing is scalar-only — an array-of-enum persists its member symbols. " + + $"Remove '@{FIELD_ATTR_INT_VALUE_MAP}', or make the field scalar.", + ErrorCode.ERR_ENUM_INT_VALUE_MAP_ARRAY, + Envelope: field.Source)); + } } foreach (var child in node.OwnChildren()) diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 06960b096..ef64e5921 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -430,6 +430,15 @@ public enum ErrorCode { */ ERR_ENUM_EXTENDS_VALUES_CONFLICT, + /** + * A {@code field.enum} carries {@code @intValueMap} together with + * {@code isArray=true}. Int-backing is a persistence-layer codec and no port + * implements it element-wise over an array column, so the combination would + * silently persist member SYMBOLS into an integer array. An array-of-enum + * stays string-backed: drop {@code @intValueMap}, or make the field scalar. + */ + ERR_ENUM_INT_VALUE_MAP_ARRAY, + /** An internal loader error with no stable error code. */ ERR_UNKNOWN, } diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index c1037e0cc..fd2383668 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -688,6 +688,11 @@ private static void validateEnumNode(MetaData node) { // (own or inherited via extends), mirroring the TS/C# port structure exactly. validateEnumIntValueMap(node); + // --- @intValueMap is scalar-only (design D7, narrowed) --- + // Separate from the content check above because it must fire on the node that + // combines the two halves, which is NOT necessarily the node declaring the map. + validateEnumIntValueMapNotArray(node); + // --- Own @values content check --- if (node.hasMetaAttr(EnumField.ATTR_VALUES, false)) { MetaAttribute valuesAttr = node.getMetaAttr(EnumField.ATTR_VALUES, false); @@ -767,6 +772,36 @@ private static MetaData sharedEnumSuper(MetaData node) { return (sup != null && isAbstract(sup) && sup.getParent() instanceof MetaRoot) ? sup : null; } + /** + * {@code @intValueMap} is scalar-only (design D7, narrowed). + * + *

      Int-backing is a persistence-layer CODEC, and no port implements it element-wise + * over an array column: OMDB's {@code EnumCodec} and Kotlin's {@code customEnumeration} + * are scalar by construction, Python would bind the symbol LIST straight into an + * {@code integer[]}, and TypeScript's sqlite branch serializes an array as JSON text + * before the enum case is reached. Two ports that happen to compose (TS/Postgres, C#) + * are not a feature — shipping a claim four ports silently get wrong is the + * {@code field.byte}/{@code short}/{@code class} mistake.

      + * + *

      BOTH halves are read RESOLVING, unlike {@link #validateEnumIntValueMap}: the + * illegal thing is the EFFECTIVE combination. Post-#246 the map must live on the shared + * abstract declaration, so the field that inherits it is exactly where {@code isArray} + * gets declared — an own-only read would see the two halves on different nodes and + * never fire.

      + */ + private static void validateEnumIntValueMapNotArray(MetaData node) { + // hasMetaAttr defaults to includeParentData=true → RESOLVING. + if (!node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return; + if (!(node instanceof MetaField) || !((MetaField) node).isArrayType()) return; + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_INT_VALUE_MAP_ARRAY + + ": field.enum '" + node.getName() + "' declares @" + + EnumField.ATTR_INT_VALUE_MAP + " with isArray=true; int-backing is" + + " scalar-only - an array-of-enum persists its member symbols." + + " Remove @" + EnumField.ATTR_INT_VALUE_MAP + ", or make the field scalar.", + ErrorCode.ERR_ENUM_INT_VALUE_MAP_ARRAY, node.getSource()); + } + private static void validateEnumIntValueMap(MetaData node) { if (!node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false)) { return; diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index 9ee138b89..3b7e5f70d 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -255,6 +255,16 @@ private ErrorMessageConstants() { */ public static final String ERR_ENUM_EXTENDS_VALUES_CONFLICT = "ERR_ENUM_EXTENDS_VALUES_CONFLICT"; + /** + * A {@code field.enum} carries {@code @intValueMap} together with {@code isArray=true}. + * Int-backing is a persistence-layer codec and no port implements it element-wise over + * an array column, so the combination would silently persist member SYMBOLS into an + * integer array. An array-of-enum stays string-backed. + * + *

      Cross-language contract: {@code ERR_ENUM_INT_VALUE_MAP_ARRAY}. + */ + public static final String ERR_ENUM_INT_VALUE_MAP_ARRAY = "ERR_ENUM_INT_VALUE_MAP_ARRAY"; + /** * #208 (design doc §5 R6) warning: an {@code origin.*}-bearing (derived) own * field lives under a host object that declares an {@code @unmanaged} source. diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index db5ef7139..d98915089 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -194,6 +194,12 @@ class ErrorCode(str, Enum): # own @values would be silently dropped in codegen. Remove the own @values # to inherit the shared set, or extend a concrete (non-shared) enum instead. ERR_ENUM_EXTENDS_VALUES_CONFLICT = "ERR_ENUM_EXTENDS_VALUES_CONFLICT" + # A field.enum carries @intValueMap together with isArray=true. Int-backing is + # a persistence-layer codec and no port implements it element-wise over an + # array column, so the combination would silently persist member SYMBOLS into + # an integer array. An array-of-enum stays string-backed: drop @intValueMap, + # or make the field scalar. + ERR_ENUM_INT_VALUE_MAP_ARRAY = "ERR_ENUM_INT_VALUE_MAP_ARRAY" ERR_UNKNOWN = "ERR_UNKNOWN" diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 1c67ac431..893ee79a6 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -614,6 +614,11 @@ def _validate_enum_values( # `extends` but declares its own @intValueMap is still validated. _validate_enum_int_value_map(node, errors) + # Design D7, narrowed: int-backing is scalar-only. Separate from the content + # rules above because it must fire on the node that combines the two halves, + # which is NOT necessarily the node that declares the map. + _validate_enum_int_value_map_not_array(node, errors) + # ADR-0039 sanctioned own: validates the AUTHORED @values membership on THIS # node (mirrors the TS attr-schema-validate `node.ownAttrs()`); an inherited # @values yields None here and is validated on its declaring node. @@ -681,6 +686,38 @@ def _validate_enum_values( ) +def _validate_enum_int_value_map_not_array(node: MetaData, errors: list[MetaError]) -> None: + """``@intValueMap`` is scalar-only (design D7, narrowed). + + Int-backing is a persistence-layer CODEC, and no port implements it + element-wise over an array column: this port's ``ObjectManager`` would bind the + symbol LIST straight into an ``integer[]``, OMDB's ``EnumCodec`` and Kotlin's + ``customEnumeration`` are scalar by construction, and TypeScript's sqlite branch + serializes an array as JSON text before the enum case is reached. Two ports that + happen to compose (TS/Postgres, C#) are not a feature — shipping a claim four + ports silently get wrong is the ``field.byte``/``short``/``class`` mistake. + + BOTH halves are read RESOLVING, unlike the content rules: the illegal thing is + the EFFECTIVE combination. Post-#246 the map must live on the shared abstract + declaration, so the field that inherits it is exactly where ``isArray`` gets + declared — an own-only read would see the two halves on different nodes and + never fire. + """ + if not isinstance(node.get_meta_attr(FIELD_ATTR_INT_VALUE_MAP), dict): + return + if not node.resolved_is_array(): + return + errors.append( + MetaError( + f"{_node_label(node)} declares '@{FIELD_ATTR_INT_VALUE_MAP}' with isArray=true; " + f"int-backing is scalar-only — an array-of-enum persists its member symbols. " + f"Remove '@{FIELD_ATTR_INT_VALUE_MAP}', or make the field scalar.", + ErrorCode.ERR_ENUM_INT_VALUE_MAP_ARRAY, + envelope=node.source, + ) + ) + + def _validate_enum_int_value_map(node: MetaData, errors: list[MetaError]) -> None: """``@intValueMap`` content rules (optional), independent of whether ``@values`` is own or inherited on this node. Mirrors TS/C#/Java, which all run this check diff --git a/server/typescript/packages/codegen-ts/src/column-mapper.ts b/server/typescript/packages/codegen-ts/src/column-mapper.ts index 887a845f6..4967713ca 100644 --- a/server/typescript/packages/codegen-ts/src/column-mapper.ts +++ b/server/typescript/packages/codegen-ts/src/column-mapper.ts @@ -613,10 +613,12 @@ export function mapColumnType( fnName = "jsonb"; break; case FIELD_SUBTYPE_ENUM: - // An INT-BACKED enum (@intValueMap, design D5/D7) stores the mapped - // integer, so the Drizzle column is integer / integer[] — matching - // migrate-ts's expected-schema. The TS-facing type stays the member-string - // union; the symbol<->int translation happens at the write/read boundary. + // An INT-BACKED enum (@intValueMap, design D5) stores the mapped integer, + // so the Drizzle column is integer — matching migrate-ts's expected-schema. + // The TS-facing type stays the member-string union; the symbol<->int + // translation happens at the write/read boundary. Scalar only: D7 makes + // @intValueMap + isArray ERR_ENUM_INT_VALUE_MAP_ARRAY at load, so an array + // enum reaching here is always string-backed. { const im = intValueMapOf(field); if (im !== undefined) { diff --git a/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts index 1238d4e52..33ae78646 100644 --- a/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts +++ b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts @@ -101,12 +101,22 @@ describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { expect(spec.checkConstraint).toBe("status IN (0, 5, 9)"); }); - test("array-of-enum int-backed gets a native integer array, no CHECK", async () => { + // Design D7, narrowed: int-backing is scalar-only, so there is no array codegen + // shape to assert — the combination never reaches a generator. This test replaced + // one that asserted `statusIntEnum(...).array()`, which composed on Postgres while + // four ports silently got it wrong (Python bound the symbol list into an integer[], + // Java and Kotlin emitted a scalar codec, and the sqlite branch stored symbols as + // JSON text). The rejection itself is gated by the loader tests + the cross-port + // error-enum-intvaluemap-array fixtures; this pins the CODEGEN-side consequence: + // an array enum reaching a generator is always string-backed. + test("an array-of-enum reaching codegen is string-backed — @intValueMap cannot load with isArray", async () => { const spec = mapColumnType( - await statusField({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), + await statusField({ name: "status", isArray: true, "@values": VALUES }), "postgres", ); - expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.fnName).toBe("text"); + expect(spec.enumIntCustomType).toBeUndefined(); + expect(spec.modifiers).toContain(".array()"); // Membership on arrays stays app-level, exactly as for string-backed enum[]. expect(spec.checkConstraint).toBeUndefined(); }); diff --git a/server/typescript/packages/metadata/src/attr-schema-validate.ts b/server/typescript/packages/metadata/src/attr-schema-validate.ts index 728b7045d..0728d1295 100644 --- a/server/typescript/packages/metadata/src/attr-schema-validate.ts +++ b/server/typescript/packages/metadata/src/attr-schema-validate.ts @@ -412,6 +412,33 @@ function validateNode( } } + // --- Check 5a: @intValueMap is scalar-only (design D7) --- + // + // Int-backing is a persistence-layer CODEC, and no port implements it + // element-wise over an array column: OMDB's EnumCodec and Kotlin's + // customEnumeration are scalar by construction, Python would bind the symbol + // list straight into an integer[], and TS's sqlite branch serializes an array + // as JSON text before the enum case is ever reached. Two ports that DO compose + // (TS/Postgres via .array(), C# via PrimitiveCollection) are not a feature — + // shipping a claim four ports silently get wrong is the field.byte/short/class + // mistake. Rejected at LOAD so it fails identically everywhere. + // + // BOTH reads are RESOLVING, unlike Check 5b below: the illegal thing is the + // EFFECTIVE combination. Post-#246 the map must live on the shared abstract + // declaration, so the field that inherits it is exactly where isArray gets + // declared — an own-only read would see the two halves on different nodes and + // never fire. + if (node.attrs().get(FIELD_ATTR_INT_VALUE_MAP) !== undefined && node.resolvedIsArray()) { + errors.push( + new ParseError( + `${nodeLabel(node)} declares '@${FIELD_ATTR_INT_VALUE_MAP}' with isArray=true; ` + + `int-backing is scalar-only — an array-of-enum persists its member symbols. ` + + `Remove '@${FIELD_ATTR_INT_VALUE_MAP}', or make the field scalar.`, + { code: "ERR_ENUM_INT_VALUE_MAP_ARRAY", source: node.source }, + ), + ); + } + // --- Check 5b: field.enum @intValueMap content rules --- // // Optional. Own-only (mirrors Checks 4/5's own-attrs-only policy) — an diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index 284b4a55c..70beb35a7 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -202,6 +202,12 @@ export const ERROR_CODES = [ // own @values would be silently dropped in codegen. Remove the own @values // to inherit the shared set, or extend a concrete (non-shared) enum instead. "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + // A field.enum carries @intValueMap together with isArray=true. Int-backing is + // a persistence-layer codec and no port implements it element-wise over an + // array column, so the combination would silently persist member SYMBOLS into + // an integer array. An array-of-enum stays string-backed: drop @intValueMap, + // or make the field scalar. + "ERR_ENUM_INT_VALUE_MAP_ARRAY", "ERR_UNKNOWN", ] as const; diff --git a/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts index 975fa7ecc..5b6a3e568 100644 --- a/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts +++ b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts @@ -78,3 +78,40 @@ describe("field.enum @intValueMap content rules", () => { expect(result.errors[0]?.message).toContain("ARCHIVED"); }); }); + +// Design D7, narrowed: int-backing is scalar-only. No port implements the codec +// element-wise over an array column, and two ports that happen to compose are +// not a feature — so the combination is rejected at LOAD, in every port. +describe("field.enum @intValueMap is scalar-only", () => { + const MAP = ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}'; + + test("rejects @intValueMap on an isArray field", async () => { + const result = await load(base(`, "isArray": true${MAP}`)); + const codes = result.errors.map((e) => (e as { code?: string })?.code); + expect(codes).toContain("ERR_ENUM_INT_VALUE_MAP_ARRAY"); + }); + + test("an array enum with no @intValueMap stays valid (string-backed)", async () => { + const result = await load(base(', "isArray": true')); + expect(result.errors).toEqual([]); + }); + + // The two halves land on DIFFERENT nodes on the canonical authoring shape: + // #246 forces @intValueMap onto the shared abstract declaration, and isArray + // is declared by the consuming field. An own-only read would never see both. + test("rejects an inherited @intValueMap combined with a locally-declared isArray", async () => { + const result = await load(`{ + "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT","PUBLISHED","ARCHIVED"]${MAP} } }, + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "Status", "isArray": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]} + }`); + const codes = result.errors.map((e) => (e as { code?: string })?.code); + expect(codes).toContain("ERR_ENUM_INT_VALUE_MAP_ARRAY"); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts index bf6b10733..41c8e6df4 100644 --- a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts +++ b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts @@ -58,13 +58,10 @@ describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { expect(col.sqlType).toEqual({ kind: "text" }); }); - test("array-of-enum int-backed maps to integer[] (D7)", async () => { - const col = await statusColumn( - entityModel({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), - ); - expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); - }); - + // D7, narrowed: int-backing is scalar-only — @intValueMap with isArray is + // ERR_ENUM_INT_VALUE_MAP_ARRAY at LOAD, so no int-backed array column shape exists + // to assert. The rejection is gated by the loader tests + the cross-port + // error-enum-intvaluemap-array fixtures. test("array-of-enum string-backed stays text[]", async () => { const col = await statusColumn(entityModel({ name: "status", isArray: true, "@values": VALUES })); expect(col.sqlType).toEqual({ kind: "array", element: { kind: "text" } }); @@ -117,9 +114,7 @@ describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { test("array-of-enum still gets NO field-level CHECK (membership stays app-level)", async () => { const snapshot = buildExpectedSchema( - await loadJson( - entityModel({ name: "status", isArray: true, "@values": VALUES, "@intValueMap": INT_MAP }), - ), + await loadJson(entityModel({ name: "status", isArray: true, "@values": VALUES })), ); const table = snapshot.tables.find((t) => t.name === "orders")!; expect((table.checks ?? []).find((c) => c.name === "orders_status_chk")).toBeUndefined(); @@ -148,21 +143,19 @@ describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { expect(col.default).toEqual({ kind: "literal", value: "0" }); }); - test("array-ness and the map may BOTH be inherited from the shared declaration", async () => { + // Array-ness inherited from a shared declaration still resolves here. This + // replaced a test that inherited array-ness AND @intValueMap together, which is + // now ERR_ENUM_INT_VALUE_MAP_ARRAY at load (D7, narrowed) — that rejection is + // gated by the loader tests and the error-enum-intvaluemap-array-inherited + // fixture. What remains worth pinning at THIS layer is the resolving read of + // array-ness itself, since an own-only read would emit a scalar column. + test("array-ness inherited from the shared declaration still yields an array column", async () => { const col = await statusColumn( entityModel({ name: "status", extends: "Status" }, [ - { - "field.enum": { - name: "Status", - abstract: true, - isArray: true, - "@values": VALUES, - "@intValueMap": INT_MAP, - }, - }, + { "field.enum": { name: "Status", abstract: true, isArray: true, "@values": VALUES } }, ]), ); - expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "text" } }); }); }); From 34e8530c30de3def4c97f112bf66f448e60d1feb Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 20:40:38 -0400 Subject: [PATCH 46/52] feat(all-ports): an unmapped stored int throws on read, not a pseudo-member MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An int-backed field.enum column holding a value that maps to no member is data the model says is impossible — a hand-written INSERT, or a member removed without a migration. Java surfaced the raw int as the "member" ("7"), Python returned it verbatim, and C# fell through its ternary chain to the LAST member, handing the caller ARCHIVED for a row that is not archived. TypeScript and Kotlin already threw, so the same corrupt row behaved four different ways across five ports. Neither alternative to throwing is honest. Surfacing the raw value hands the caller a member that is not one, and it is not even representable in C#, Kotlin or TypeScript, which type the property as a CLOSED enum. Returning null hides the corruption behind a nullable column. So all five throw now, naming the stored value and @intValueMap. C# reaches it through a generated static helper called from the provider->model lambda: CS8188 bans a throw-EXPRESSION inside an expression tree, but a method CALL is legal there and the throw happens in the helper's ordinary body. The helper is emitted only when the model carries an int-backed enum, so a model without one is byte-identical. Only the READ side needs this — the model-> provider chain is exhaustive over the enum by construction, since @intValueMap's keys are loader-validated to match @values exactly. The WRITE side is deliberately left to the database: an unmapped symbol binds unchanged, so the column type and its CHECK reject it. Gated by executing the converter rather than asserting over generated text: the C# test runs both directions off the finalized EF model (and pins that the DECLARED map reaches the column, not EF's ordinal default), and the Java test writes a legal member through OMDB, corrupts it with raw SQL, and reads back — the same shape as the drift this guards against. Co-Authored-By: Claude Opus 5 (1M context) --- .../Generators/DbContextGenerator.cs | 58 ++++++++++++++--- .../AppDbContextModelTests.cs | 34 ++++++++++ .../Generated/AppDbContext.g.cs | 13 +++- .../manager/db/codec/JdbcCodecs.java | 14 +++- .../db/codec/JdbcCodecRoundTripTest.java | 65 +++++++++++++++++++ .../src/metaobjects/runtime/object_manager.py | 16 +++-- .../test_object_manager_enum_intvaluemap.py | 16 +++-- 7 files changed, 195 insertions(+), 21 deletions(-) diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index 32ff0f8ee..7584f0b89 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -153,6 +153,7 @@ public virtual IEnumerable Generate(GenContext ctx) EmitUsings(sb, needsMetadataUsing, ctx); EmitDbSetDeclarations(sb, objects, ctx); EmitOnModelCreatingBody(sb, modelLines, ctx); + if (NeedsUnmappedEnumHelper(objects)) EmitUnmappedEnumHelper(sb); sb.AppendLine("}"); return [new EmittedFile("AppDbContext.g.cs", sb.ToString())]; } @@ -339,6 +340,41 @@ private static string UsingEntityConfig(M2MNavigation nav) private static IReadOnlyDictionary? IntValueMapOf(MetaField f) => f.Attr(FIELD_ATTR_INT_VALUE_MAP) as IReadOnlyDictionary; + ///

      Name of the generated fail-fast helper the read converters end in. + private const string UnmappedEnumHelperName = "UnmappedEnumValue"; + + /// + /// Emits the fail-fast helper every int-backed enum's provider→model converter ends + /// in. Emitted only when at least one such converter was generated, so a model with + /// no @intValueMap produces byte-identical output. + /// + private static void EmitUnmappedEnumHelper(StringBuilder sb) + { + sb.AppendLine(); + sb.AppendLine(" /// "); + sb.AppendLine(" /// An int-backed field.enum column held a value that maps to no member: the"); + sb.AppendLine(" /// database holds data the model says is impossible (a hand-written INSERT, or a"); + sb.AppendLine(" /// member removed without a migration). Materializing the last member instead"); + sb.AppendLine(" /// would hand the caller a wrong-but-valid value, silently."); + sb.AppendLine(" /// "); + sb.AppendLine($" private static T {UnmappedEnumHelperName}(int stored, string field) =>"); + // Fully qualified: the generated file's usings are a fixed set (EmitUsings), and + // adding `using System;` there would change byte-identical output for every model. + sb.AppendLine(" throw new System.InvalidOperationException("); + sb.AppendLine(" $\"field.enum '{field}' read stored value {stored} with no member in \" +"); + sb.AppendLine(" \"@intValueMap — the database holds a value the model does not describe.\");"); + } + + /// + /// True when any emitted converter will reference + /// — i.e. the model carries at least one int-backed field.enum. Mirrors the + /// fields configures (enum fields of every emitted + /// object, plus a write-through entity's read-model view over the same field set). + /// + private static bool NeedsUnmappedEnumHelper(IEnumerable objects) => + objects.Any(o => o.Fields().Any(f => + f.SubType == FIELD_SUBTYPE_ENUM && IntValueMapOf(f) is not null)); + /// /// The complete HasConversion call for an enum property: the generic /// HasConversion<string>() for a string-backed enum, or @@ -349,11 +385,14 @@ private static string UsingEntityConfig(M2MNavigation nav) /// The mapping is emitted as a TERNARY CHAIN rather than a switch /// expression because EF converts these lambdas to EXPRESSION TREES, and a switch /// expression is not legal in one (CS8155). A conditional is. - /// KNOWN PORT ASYMMETRY, deliberate: the provider→model chain ends on the last - /// member rather than rejecting an int with no member, where Python, Java and Kotlin - /// surface the unmapped value instead. C# cannot match them here — an expression tree - /// may not contain a throw-expression (CS8188) — and the column's CHECK constrains it - /// to the mapped ints anyway. Documented rather than papered over. + /// The provider→model chain gives EVERY member its own branch and ends in a + /// call to the generated UnmappedEnumValue<T> helper, so a stored int + /// with no member THROWS rather than silently materializing as the last member — + /// matching all four sibling ports. CS8188 bans a throw-EXPRESSION inside an + /// expression tree, but a method CALL is legal there and the throw itself happens in + /// the helper's ordinary body. Only the read side needs this: the model→provider + /// chain is exhaustive over the enum by construction, since @intValueMap's + /// keys are loader-validated to match @values exactly. /// private static string EnumConversionCall(string owner, MetaObject entity, MetaField f, GenConfig config) { @@ -389,12 +428,13 @@ private static string EnumConversionCall(string owner, MetaObject entity, MetaFi var toProvider = new System.Text.StringBuilder("v => "); var fromProvider = new System.Text.StringBuilder("v => "); for (var i = 0; i < members.Count - 1; i++) - { toProvider.Append($"v == {type}.{members[i]} ? {ints[i]} : "); - fromProvider.Append($"v == {ints[i]} ? {type}.{members[i]} : "); - } toProvider.Append(ints[^1]); - fromProvider.Append($"{type}.{members[^1]}"); + // Every member gets its own branch here (no last-member fallthrough) so the + // final else can reject an int the model does not describe. + for (var i = 0; i < members.Count; i++) + fromProvider.Append($"v == {ints[i]} ? {type}.{members[i]} : "); + fromProvider.Append($"{UnmappedEnumHelperName}<{type}>(v, \"{f.Name}\")"); return $"HasConversion({toProvider}, {fromProvider})"; } diff --git a/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs b/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs index ac4859a36..8a76d60fa 100644 --- a/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs +++ b/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs @@ -36,4 +36,38 @@ public void Model_builds_without_error() Assert.NotNull(model.FindEntityType(typeof(PriorAuthAuth))); Assert.Equal("auths", auth!.GetTableName()); } + + /// + /// The int-backed field.enum value converter, EXECUTED in both directions off + /// the finalized model — the emitted lambdas are the whole feature, and a string + /// assertion over the generated source cannot tell a working converter from one that + /// compiles and computes the wrong thing. + /// + [Fact] + public void Int_backed_enum_converter_maps_both_ways_and_rejects_an_unmapped_stored_value() + { + var options = new DbContextOptionsBuilder() + .UseNpgsql("Host=localhost;Database=unused") + .Options; + using var db = new AppDbContext(options); + + var prop = db.Model.FindEntityType(typeof(AllTypes))!.FindProperty(nameof(AllTypes.IntEnumVal)); + var converter = prop!.GetValueConverter(); + Assert.NotNull(converter); + + // @intValueMap declares DRAFT=0, PUBLISHED=5, ARCHIVED=9 — NOT the C# ordinals + // (which would make PUBLISHED 1 and ARCHIVED 2), so these assertions also prove + // the declared map is what reaches the column rather than EF's ordinal default. + Assert.Equal(5, converter!.ConvertToProvider(AllTypes.AllTypesIntEnumVal.PUBLISHED)); + Assert.Equal(9, converter.ConvertToProvider(AllTypes.AllTypesIntEnumVal.ARCHIVED)); + Assert.Equal(AllTypes.AllTypesIntEnumVal.PUBLISHED, converter.ConvertFromProvider(5)); + Assert.Equal(AllTypes.AllTypesIntEnumVal.ARCHIVED, converter.ConvertFromProvider(9)); + + // 7 maps to no member. Materializing the last member instead — which is what the + // ternary chain did before it grew a final else — would hand the caller + // ARCHIVED for a row that is not archived, silently. + var ex = Assert.Throws(() => converter.ConvertFromProvider(7)); + Assert.Contains("7", ex.Message); + Assert.Contains("intValueMap", ex.Message); + } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs index f53ac22ed..24b7bbc94 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs @@ -33,7 +33,7 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) modelBuilder.Entity().OwnsOne(x => x.Settings, b => b.ToJson("settings")); modelBuilder.Entity().OwnsMany(x => x.Labels, b => b.ToJson("labels")); modelBuilder.Entity().Property(x => x.EnumVal).HasConversion(); - modelBuilder.Entity().Property(x => x.IntEnumVal).HasConversion(v => v == AllTypes.AllTypesIntEnumVal.DRAFT ? 0 : v == AllTypes.AllTypesIntEnumVal.PUBLISHED ? 5 : 9, v => v == 0 ? AllTypes.AllTypesIntEnumVal.DRAFT : v == 5 ? AllTypes.AllTypesIntEnumVal.PUBLISHED : AllTypes.AllTypesIntEnumVal.ARCHIVED); + modelBuilder.Entity().Property(x => x.IntEnumVal).HasConversion(v => v == AllTypes.AllTypesIntEnumVal.DRAFT ? 0 : v == AllTypes.AllTypesIntEnumVal.PUBLISHED ? 5 : 9, v => v == 0 ? AllTypes.AllTypesIntEnumVal.DRAFT : v == 5 ? AllTypes.AllTypesIntEnumVal.PUBLISHED : v == 9 ? AllTypes.AllTypesIntEnumVal.ARCHIVED : UnmappedEnumValue(v, "intEnumVal")); modelBuilder.Entity().Property(x => x.DecVal).HasPrecision(18, 6); modelBuilder.Entity().Property(x => x.TsVal).HasColumnType("timestamp without time zone"); modelBuilder.Entity().Property(x => x.TsTzVal).HasColumnType("timestamp with time zone"); @@ -51,4 +51,15 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) modelBuilder.Entity().Property(x => x.Status).HasConversion(); modelBuilder.Entity().Property(x => x.CreatedAt).HasColumnType("timestamp without time zone"); } + + /// + /// An int-backed field.enum column held a value that maps to no member: the + /// database holds data the model says is impossible (a hand-written INSERT, or a + /// member removed without a migration). Materializing the last member instead + /// would hand the caller a wrong-but-valid value, silently. + /// + private static T UnmappedEnumValue(int stored, string field) => + throw new System.InvalidOperationException( + $"field.enum '{field}' read stored value {stored} with no member in " + + "@intValueMap — the database holds a value the model does not describe."); } diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java index 2786f63f0..cab32aa87 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java @@ -442,9 +442,17 @@ static final class EnumCodec implements JdbcFieldCodec { return; } } - // A stored int with no member is data the model does not describe. - // Surface it rather than nulling it — that would hide real drift. - f.setString(o, String.valueOf(stored)); + // A stored int with no member is data the model says is impossible — a + // hand-written INSERT, or a member removed without a migration. Throw: + // returning String.valueOf(stored) would hand the caller a "member" that is + // not one, and C#/Kotlin/TS type this property as a CLOSED enum, so a value + // like "7" is not even representable there. Nulling it would hide the + // corruption behind a nullable column. Matches every sibling port. + throw new SQLException( + "field.enum '" + f.getName() + "' read stored value " + stored + + " with no member in @" + EnumField.ATTR_INT_VALUE_MAP + + " (declared: " + intMap + ") — the database holds a value the model " + + "does not describe."); } @Override public void write(PreparedStatement s, MetaField f, int j, Object v) throws SQLException { diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java index 6f8fc8026..d28371d04 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java @@ -292,6 +292,71 @@ public void timestampCurrencyEnumRoundTripThroughOMDB() throws Exception { } } + /** + * An int-backed {@code field.enum} column holding an integer that maps to NO member is + * data the model says is impossible. The codec must THROW rather than surface the raw + * int as a pseudo-symbol: the sibling ports type this property as a CLOSED enum, so a + * value like {@code "7"} is not representable there, and returning null would hide the + * corruption behind a nullable column. + * + *

      Written through OMDB with a legal member, then corrupted with raw SQL — the same + * shape as the drift this guards against (a hand-written INSERT, or a member removed + * without a migration).

      + */ + @Test + public void intBackedEnumThrowsOnAnUnmappedStoredValue() throws Exception { + MetaObject mo = registry.findMetaObjectByName("codectest::Sample"); + assertNotNull(mo); + + ObjectConnection oc = omdb.getConnection(); + try { + ValueObject vo = (ValueObject) mo.newInstance(); + String label = "unmapped-int-enum-" + System.currentTimeMillis(); + vo.setString("label", label); + vo.setInt("count", 1); + vo.setLong("bignum", 1L); + vo.setBoolean("active", false); + vo.setDouble("ratio", 0d); + vo.setFloat("rate", 0f); + vo.setObject("amount", java.math.BigDecimal.ZERO); + vo.setDate("createdAt", new Date(0)); + vo.setObject("startTime", LocalTime.of(0, 0, 0)); + vo.setString("priority", "PUBLISHED"); + omdb.createObject(oc, vo); + + // 7 is in no member's @intValueMap (DRAFT=0, PUBLISHED=5, ARCHIVED=9). + try (Connection c = getConnection(); + PreparedStatement ps = c.prepareStatement( + "UPDATE CODEC_SAMPLE SET priority = 7 WHERE label = ?")) { + ps.setString(1, label); + assertEquals("exactly one row corrupted", 1, ps.executeUpdate()); + } + + try { + omdb.getObjects(oc, mo, + new QueryOptions(new Expression("label", label, Expression.EQUAL))); + fail("reading an unmapped int-backed enum value must throw, not surface it"); + } catch (Exception e) { + assertTrue("the failure must name the unmapped value: " + messageChain(e), + messageChain(e).contains("7")); + assertTrue("the failure must name the attribute: " + messageChain(e), + messageChain(e).contains("intValueMap")); + } + } finally { + omdb.releaseConnection(oc); + } + } + + /** Every message in a throwable's cause chain, joined — OMDB wraps driver exceptions. */ + private static String messageChain(Throwable t) { + StringBuilder sb = new StringBuilder(); + for (Throwable c = t; c != null; c = c.getCause()) { + sb.append(c.getMessage()).append(" | "); + if (c.getCause() == c) break; + } + return sb.toString(); + } + /** * {@link UuidCodec} read-back-lowercase contract at the raw codec/JDBC boundary. The * native-uuid WRITE bind ({@code setObject(.., Types.OTHER)}) is Postgres-only (Derby diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index cf8b1c28f..b82620b05 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -883,9 +883,13 @@ def _decode_read_value(field: MetaField, value: Any) -> Any: the member SYMBOL. Every other field subtype is returned verbatim, keeping ADR-0019's "runtime returns native in-process types" contract intact. - An int with no member is returned AS-IS rather than as ``None`` — a row - holding a value the model does not describe is real drift, and surfacing it - is honest where nulling it would hide it. + An int with no member RAISES. The database then holds a value the model says + is impossible — a hand-written INSERT, or a member removed without a + migration — and neither alternative is honest: returning the raw int hands + the caller a "member" that is not one (C#, Kotlin and TypeScript type this + property as a CLOSED enum, where ``7`` is not even representable), and + returning ``None`` hides the corruption behind a nullable column. Every port + throws here. """ if value is None: return None @@ -897,7 +901,11 @@ def _decode_read_value(field: MetaField, value: Any) -> Any: for symbol, stored in int_map.items(): if stored == value: return symbol - return value + raise ValueError( + f"field.enum '{field.name}' read stored value {value} with no member in " + f"@{fc.FIELD_ATTR_INT_VALUE_MAP} (declared: {int_map}) — the database holds " + f"a value the model does not describe." + ) def _decode_read_row(fields_by_name: dict[str, MetaField], row: dict[str, Any]) -> dict[str, Any]: diff --git a/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py index 745bbc692..df44e4028 100644 --- a/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +++ b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py @@ -85,10 +85,18 @@ def test_none_stays_none_on_read(): assert _decode_read_value(_status_field(INT_MAP), None) is None -def test_unmapped_int_on_read_is_passed_through_not_silently_nulled(): - """A row holding an int outside the map is data the model does not describe. - Surfacing it verbatim is honest; returning None would hide the drift.""" - assert _decode_read_value(_status_field(INT_MAP), 7) == 7 +def test_unmapped_int_on_read_raises(): + """A row holding an int outside the map is data the model says is impossible. + + Neither alternative is honest: returning the raw int hands the caller a + "member" that is not one (C#, Kotlin and TypeScript type this as a CLOSED + enum, where 7 is unrepresentable), and returning None hides the corruption + behind a nullable column. Every port throws. + """ + with pytest.raises(ValueError) as exc: + _decode_read_value(_status_field(INT_MAP), 7) + assert "7" in str(exc.value) + assert "intValueMap" in str(exc.value) def test_non_enum_field_read_is_untouched(): From 7c9b65af1a11113829884d2a5ac5813953764023 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 20:40:50 -0400 Subject: [PATCH 47/52] docs(int-enum): correct every source that still describes the pre-D7 feature MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The design doc records the two rulings (scalar-only int-backing, throw on an unmapped stored int) and gains a follow-up the D8 "migration safety" heading was quietly implying it covered: D8 handles ADDING or REMOVING @intValueMap, but RE-mapping a member inside a map that stays present is undetected — and one shape of it is silent. Moving a member to an int not already in the set changes the CHECK, so the migration applies a constraint the existing rows violate and the database refuses it, loudly. But SWAPPING two members' ints leaves the value set identical: the CHECK is byte-identical, the diff is EMPTY, no migration is emitted, and every stored row changes meaning. Nothing in the pipeline can see it — the column holds bare integers, and neither introspection nor the committed snapshot records which member an integer stood for. Named rather than left implied; closing it needs the mapping carried in gen-state or the snapshot, which is a design decision, not a patch. field-types.md gains the two rules an adopter can actually hit, since both are new ways their metadata or their database can now fail. The five implementation plans are banded SUPERSEDED. They are kept for provenance but are actively misleading now: every array-of-enum fixture, column shape and element-wise codec in them describes vocabulary that cannot load, and some sketched tests call APIs that do not exist (MetaRoot.find_object, MetaObject.field(name)) or assume test libraries a module does not depend on. CHANGELOG gets the [Unreleased] entry for the whole feature — it had none, and this is registered vocabulary, so the line carrying it is a MINOR. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 63 +++++++++++++++++++ docs/features/field-types.md | 14 +++++ ...t-backed-enum-values-csharp-persistence.md | 11 ++++ ...ked-enum-values-java-kotlin-persistence.md | 11 ++++ ...-07-23-int-backed-enum-values-metamodel.md | 11 ++++ ...t-backed-enum-values-python-persistence.md | 11 ++++ ...3-int-backed-enum-values-ts-persistence.md | 11 ++++ ...026-07-23-int-backed-enum-values-design.md | 59 +++++++++++++++-- 8 files changed, 186 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f7691de47..52995c93f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,69 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ## [Unreleased] +### Added — int-backed `field.enum` storage via `@intValueMap` (all five ports) + +**New registered vocabulary, so the line carrying this is a MINOR.** Purely additive: a +`field.enum` that declares no `@intValueMap` is byte-identical to before, string-backed as +always. + +`@intValueMap` is an optional `{memberSymbol: int}` map on `field.enum` that declares each +member's **stored integer**. The column becomes `integer` with an integer `CHECK` instead of +`varchar` with a string one — while the wire format, the generated enum type, and every +runtime return value stay the **member symbol**, unchanged. The provenance is an +integer-coded column an adopter already has: the map is how you say "1 means LOW here", +rather than accepting whatever ordinal a language happens to assign. This is why it is a map +and not a second array parallel to `@values` — a positional array would silently re-map every +member the day someone reorders `@values`. + +The loader enforces the map's content identically everywhere: keys must equal `@values` +exactly (no missing, no extra), every value must be a 32-bit integer, and no two members may +share one. Both halves are read RESOLVING, so a field that `extends` a shared abstract enum +inherits the members *and* their mapping — and an own `@intValueMap` declared against a +shared enum is rejected for the same reason an own `@values` is (#246's twin: one shared enum +type has one mapping). + +Persistence ships in every port: Drizzle `customType` codecs (TS), EF Core `HasConversion` +(C#), OMDB's `JdbcFieldCodec` (Java), Exposed `customEnumeration` (Kotlin) and +`ObjectManager` coercion (Python), gated cross-port by the `AllTypes` round-trip corpus +against real Postgres. + +Two decisions are worth naming because each closes a way the feature could have shipped +half-true: + +- **Int-backing is scalar-only.** `@intValueMap` together with `isArray: true` is a load + error — `ERR_ENUM_INT_VALUE_MAP_ARRAY`, in every port. The original design said an + array-of-enum composed unchanged; it does not. Int-backing is a persistence-layer CODEC and + every port's codec seam is scalar by construction: Python bound the symbol LIST into an + `integer[]`, Java and Kotlin emitted a scalar codec, and TypeScript's sqlite branch + serialized the array as JSON text before the enum case was ever reached — storing symbols. + Only TS/Postgres and C# composed, and **two ports composing while four silently get it + wrong is not a feature** — it is the `field.byte`/`short`/`class` mistake, vocabulary that + reads as supported and is not. Rejecting it at LOAD delivers the guarantee that was + actually missing: identical behaviour in every port. An array-of-enum stays string-backed. +- **A stored integer that maps to no member THROWS on read**, in every port. The row holds + data the model says is impossible — a hand-written `INSERT`, or a member removed without a + migration — and neither alternative is honest: surfacing the raw integer hands the caller a + "member" that is not one, and is not even representable in C#, Kotlin or TypeScript, which + type the property as a closed enum; returning null hides the corruption behind a nullable + column. C# reaches this through a generated static helper called from the provider→model + lambda — CS8188 bans a throw-*expression* inside an expression tree, but a method CALL is + legal there. The WRITE side is deliberately left to the database: an unmapped symbol binds + unchanged, so the column type and its `CHECK` reject it. + +**Adopter-visible beyond the new attribute:** the filter-operator band is now decided +**per field**, not per subtype, so an int-backed `field.enum` no longer offers `like` — the +column holds integers, and `LIKE` against one is a type error, not a query. A projection's +`@filter` over an int-backed enum lowers to the integer literal rather than the symbol. + +Migration safety is unchanged and deliberate: adding or removing `@intValueMap` on a field +that already has a column is a cross-kind `change-column-type`, which `meta migrate` already +blocks by default and requires an explicit `allow.typeChange` to pass. There is no +auto-recast — the tool will not rewrite your data behind a metadata edit. + +Design: `docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`. Adopter view: +[`docs/features/field-types.md`](docs/features/field-types.md). + ## [0.23.1] — npm `0.23.1` · PyPI `0.23.1` · NuGet `0.23.1` · Maven `7.23.1` A coordinated **PATCH** across all four registries. Every one of them carries a real changed diff --git a/docs/features/field-types.md b/docs/features/field-types.md index c607c2393..beed69311 100644 --- a/docs/features/field-types.md +++ b/docs/features/field-types.md @@ -73,6 +73,20 @@ preserving the string wire format and generated enum type. Keys must match `@val values must be unique integers. Display labels and native Postgres `ENUM` types remain deferred (see [enum-datatype-design.md](../superpowers/specs/2026-05-23-enum-datatype-design.md)). +Two rules follow from int-backing being a **persistence-layer codec**: + +- **It is scalar-only.** `@intValueMap` together with `isArray: true` is a load error, + `ERR_ENUM_INT_VALUE_MAP_ARRAY`, in every port — no port implements the codec element-wise + over an array column, so the combination would silently persist member *symbols* into an + integer array. An array-of-enum stays string-backed. +- **A stored integer that maps to no member throws on read**, in every port. The row holds + data the model says is impossible (a hand-written `INSERT`, or a member removed without a + migration); surfacing the raw integer would hand you a "member" that is not one — and is + not even representable in the ports that type the property as a closed enum — while + returning null would hide the corruption behind a nullable column. The write side is left + to the database: an unmapped symbol binds unchanged, so the column type and its `CHECK` + reject it. + ### Sharing one enum — abstract `field.enum` + `extends` Reuse a constraint set across entities by declaring one **abstract** `field.enum` diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md index e0413da7b..d5cbf061f 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md @@ -1,5 +1,16 @@ # Int-Backed Enum Values — C# Persistence Implementation Plan +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into C#'s EF Core codegen: emit a custom `HasConversion` lambda pair built from `@intValueMap`'s lookup table instead of the blanket `HasConversion()`, while the generated C# `enum` type itself is completely unchanged. C# never generates DDL (schema is TS-owned per ADR-0015) — this plan touches `DbContextGenerator.cs` only. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md index c402d4179..13864c76a 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md @@ -1,5 +1,16 @@ # Int-Backed Enum Values — Java + Kotlin Persistence Implementation Plan +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into Java's OMDB JDBC persistence and Kotlin's Exposed table generation. Java's generated Java `enum` type and Kotlin's generated Kotlin `enum class` are completely unchanged — only the runtime codec (Java/OMDB) and the generated table-column DSL call (Kotlin/codegen-kotlin) differ. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md index 78087bf69..091fb307a 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md @@ -1,5 +1,16 @@ # Int-Backed Enum Values — Metamodel Implementation Plan +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Add the `@intValueMap` attribute to `field.enum` — an optional `{memberSymbol: int}` map that, when present, is the metadata author's declaration of each member's stored integer — across all five ports (TypeScript, C#, Java, Python, Kotlin-via-Java), gated by load-time validation and the `registry-conformance` + `fixtures/conformance/` corpora. This plan covers **vocabulary + validation + conformance only** — it does NOT touch codegen (already proven unchanged, since no port's enum-type emitter reads `@intValueMap`) or persistence (DB DDL, EF Core/JDBC/Exposed/ObjectManager codecs, migrate-ts's migration-safety guard). Those are covered by follow-on plans, one per port/group, written after this one lands. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md index 0a4af2b1f..6902807d0 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md @@ -1,5 +1,16 @@ # Int-Backed Enum Values — Python Persistence Implementation Plan +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into Python's `ObjectManager` runtime: encode the member symbol to its int on write, decode it back on read. Python has no per-subtype `EnumField` class (confirmed — `field.sub_type == fc.FIELD_SUBTYPE_ENUM` is the only discriminator) and no codegen change is needed at all — Python's generated type for an inline enum field is `Literal["DRAFT", "PUBLISHED", ...]` regardless of backing mode, since codegen never reads `@intValueMap`. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md index 662181abe..fce0cd804 100644 --- a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md @@ -1,5 +1,16 @@ # Int-Backed Enum Values — TypeScript Persistence Implementation Plan +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped in `docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md`) into TypeScript's persistence stack: migrate-ts emits `integer` + a numeric `CHECK` instead of `text` + a string `CHECK`, codegen-ts's Drizzle column mapper follows suit, and the generated entity/query code translates symbol↔int at the DB boundary while every TS-facing type (Zod schema, `EntityType`, wire JSON) stays exactly the string union it is today. diff --git a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md index 780f696ae..6ef032dad 100644 --- a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md +++ b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md @@ -132,9 +132,40 @@ This is why `@intValueMap` is a map, not a second array parallel to `@values`. payloads) is the member string in both modes, unchanged from the original design's cross-language contract. -- **D7 — Array-of-enum composes unchanged.** `field.enum @isArray` + `@intValueMap` - follows the same pattern as today's array-of-enum: an `integer[]` column instead of - `text[]`, element membership validated against `@intValueMap`'s value set. + **A stored integer that maps to no member THROWS, in all five ports.** The row holds + data the model says is impossible — a hand-written INSERT, or a member removed without a + migration — and neither alternative is honest. Surfacing the raw value hands the caller a + "member" that is not one, and it is not even representable in C#, Kotlin or TypeScript, + which type the property as a closed enum; returning null hides the corruption behind a + nullable column. C# reaches this through a generated static helper called from the + provider→model lambda: CS8188 bans a throw-EXPRESSION inside an expression tree, but a + method CALL is legal there and the throw happens in the helper's ordinary body. The + WRITE side is the mirror case and is deliberately left to the database: an unmapped + symbol binds unchanged, so the column type and its `CHECK` reject it. + +- **D7 — Int-backing is scalar-only; `@isArray` + `@intValueMap` is a LOAD ERROR.** + `ERR_ENUM_INT_VALUE_MAP_ARRAY`, in all five ports. An array-of-enum stays + string-backed. + + This reverses D7 as originally written ("array-of-enum composes unchanged"), which + assumed the element codec would fall out of the scalar one. It does not. Int-backing is + a persistence-layer CODEC, and each port's codec seam is scalar by construction: OMDB's + `EnumCodec` and Kotlin's `customEnumeration` bind one value; Python's `ObjectManager` + tests `value in int_map`, which is false for a list, so it binds the symbol LIST into an + `integer[]`; and TypeScript's sqlite branch serializes an array as JSON text before the + enum case is reached, storing symbols. Only TS/Postgres (`customType(...).array()`) and + C# (`PrimitiveCollection().ElementType()`) compose — and two ports composing while four + silently get it wrong is not a feature, it is the `field.byte`/`field.short`/ + `field.class` mistake: vocabulary that reads as supported and is not. + + Rejected at LOAD rather than fixed per-port because there is no consumer need for the + array form (the provenance is a scalar integer-coded column), and because the guarantee + a load error gives — *identical behaviour in every port* — is the one that was missing. + Both halves are read RESOLVING in every loader: post-#246 the map must live on the + shared abstract declaration while `isArray` is declared by the consuming field, so an + own-only read would see the two halves on different nodes and never fire. Gated by + `error-enum-intvaluemap-array` (own map) and `error-enum-intvaluemap-array-inherited` + (the canonical shared-enum shape). - **D8 — Migration safety: no auto-recast.** Adding `@intValueMap` to a *new* field (no existing column) is a normal create. Adding or removing `@intValueMap` on a field that @@ -199,8 +230,11 @@ original `field.enum` rollout). 1. **`enum-int-backed`** — a `field.enum` with `@values` + `@intValueMap`; asserts DB column is `integer` + int `CHECK`, and the native type in every port is unchanged from the string-backed case. -2. **`enum-int-backed-array`** — `field.enum[]` + `@intValueMap`; array-of-int-backed-enum - DDL and element-membership semantics. +2. **`error-enum-intvaluemap-array`** (negative) — `field.enum[]` + an own `@intValueMap` + → `ERR_ENUM_INT_VALUE_MAP_ARRAY` (D7: int-backing is scalar-only). +2b. **`error-enum-intvaluemap-array-inherited`** (negative) — the same rejection where the + map is INHERITED from a shared abstract enum and only `isArray` is declared locally. + This is the canonical authoring shape post-#246, and the case an own-only read misses. 3. **`error-enum-intvaluemap-key-mismatch`** (negative) — `@intValueMap` keys don't exactly match `@values` members → load error. 4. **`error-enum-intvaluemap-non-int`** (negative) — a non-integer value in @@ -233,6 +267,21 @@ just golden-snapshot codegen. - Safe backing-mode migration (varchar↔integer recast with data preservation) — no current consumer; D8's manual path covers the only known need. +- **RE-mapping an existing member's integer is not detected, and one shape of it is + silent.** D8 covers ADDING or REMOVING `@intValueMap`; it does not cover changing a + value inside a map that stays present. Two cases, only one of which is safe by + accident: changing a member to an integer not already in the set (`DRAFT: 0` → `1`) + alters the `CHECK`, so the migration applies a constraint every existing `0` row + violates and the database refuses it — loud, at apply time. But **swapping two members' + integers** (`DRAFT: 0, PUBLISHED: 5` → `DRAFT: 5, PUBLISHED: 0`) leaves the value SET + identical, so the `CHECK` is byte-identical, the diff is EMPTY, no migration is emitted + at all — and every stored row silently changes meaning. Nothing in the pipeline can see + it: the column holds bare integers, and neither the introspected schema nor the + committed schema snapshot records which member an integer stood for. Closing it needs + the mapping itself carried in gen-state or the snapshot so a diff can compare + member→int pairs rather than the value set — a design decision, not a patch. No current + consumer remaps; documented here rather than left implied by D8's "migration safety" + heading. - Value aliasing (`allow_alias`-style opt-out of the duplicate-value rejection) — no current consumer. - Native Postgres `CREATE TYPE ... AS ENUM` — unrelated to this design, still deferred From 0508e45dda1d2edc6cdfdcca085e19a8936b2ff5 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 20:45:02 -0400 Subject: [PATCH 48/52] fix(docs,migrate-ts): the remap hazard, measured instead of reasoned about MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit claimed a member re-mapping is invisible to `meta migrate` and that a swap emits no migration at all. Probing the real diff says otherwise, and the corrected picture is both narrower and more useful: - The CHECK list renders in @values order, so ANY remap changes its text. That emits drop-check + add-check, and drop-check is BLOCKED by default (allow.dropCheck) — migrate refuses. The refusal is an ACCIDENT: it fires because dropping a CHECK is destructive, not because anything recognises that the meaning of stored data just changed. - Once allowed, the migration only refreshes the constraint and never touches the data. Moving a member to an unused int then applies a CHECK the existing rows violate and the database refuses it, loudly. SWAPPING two members leaves the admitted set identical, so it applies cleanly and every row has quietly changed meaning. - Only one shape is invisible to the diff: a remap plus a compensating @values reorder renders a byte-identical CHECK, so there is no diff to block. All three are now pinned in expected-schema-enum-intvaluemap.test.ts rather than asserted in prose — including the known gap, marked KNOWN so a future change that closes it updates the test instead of reverting the behaviour. Pinning the accident matters most: nothing else stops `allow.dropCheck` from being relaxed into a general auto-allow and taking the only protection with it. CHANGELOG gains the adopter-facing version: do not remap on a populated table; treat it as the same two-step backfill a backing-mode change needs. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 10 ++++ ...026-07-23-int-backed-enum-values-design.md | 42 ++++++++++------ .../expected-schema-enum-intvaluemap.test.ts | 48 +++++++++++++++++++ 3 files changed, 85 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52995c93f..85a1eca38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,16 @@ that already has a column is a cross-kind `change-column-type`, which `meta migr blocks by default and requires an explicit `allow.typeChange` to pass. There is no auto-recast — the tool will not rewrite your data behind a metadata edit. +**Do not RE-map a member's integer on a populated table.** Nothing understands that change: +the column holds bare integers, and neither introspection nor the committed snapshot records +which member an integer stood for. A remap changes the rendered `CHECK` list, so it trips the +blocked `drop-check` and `meta migrate` refuses — but that refusal is incidental (dropping a +`CHECK` is destructive), and once allowed the migration only refreshes the constraint and +never touches your rows. Swap two members' integers and the new `CHECK` admits the same set, +applies cleanly, and every stored row has quietly changed meaning. Reorder `@values` to +compensate and the diff is empty outright. Treat a remap as the same two-step backfill a +backing-mode change needs. + Design: `docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`. Adopter view: [`docs/features/field-types.md`](docs/features/field-types.md). diff --git a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md index 6ef032dad..fdce8ff9d 100644 --- a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md +++ b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md @@ -267,21 +267,33 @@ just golden-snapshot codegen. - Safe backing-mode migration (varchar↔integer recast with data preservation) — no current consumer; D8's manual path covers the only known need. -- **RE-mapping an existing member's integer is not detected, and one shape of it is - silent.** D8 covers ADDING or REMOVING `@intValueMap`; it does not cover changing a - value inside a map that stays present. Two cases, only one of which is safe by - accident: changing a member to an integer not already in the set (`DRAFT: 0` → `1`) - alters the `CHECK`, so the migration applies a constraint every existing `0` row - violates and the database refuses it — loud, at apply time. But **swapping two members' - integers** (`DRAFT: 0, PUBLISHED: 5` → `DRAFT: 5, PUBLISHED: 0`) leaves the value SET - identical, so the `CHECK` is byte-identical, the diff is EMPTY, no migration is emitted - at all — and every stored row silently changes meaning. Nothing in the pipeline can see - it: the column holds bare integers, and neither the introspected schema nor the - committed schema snapshot records which member an integer stood for. Closing it needs - the mapping itself carried in gen-state or the snapshot so a diff can compare - member→int pairs rather than the value set — a design decision, not a patch. No current - consumer remaps; documented here rather than left implied by D8's "migration safety" - heading. +- **RE-mapping an existing member's integer is not understood by anything; what saves you + is incidental.** D8 covers ADDING or REMOVING `@intValueMap`, not changing a value inside + a map that stays present. Measured against the real diff rather than reasoned about: + + - **A remap that changes the rendered `CHECK` list** — which is every remap, since the + list renders in `@values` order — emits `drop-check` + `add-check`, and the `drop-check` + is **BLOCKED by default** (`allow.dropCheck`). So `meta migrate` refuses. That refusal + is a happy accident: it fires because dropping a `CHECK` is destructive, not because + anything recognises that the meaning of stored data just changed. + - **Once allowed, the migration only refreshes the constraint — it never touches the + data.** Moving a member to an int not already in the set (`DRAFT: 0` → `1`) then applies + a `CHECK` every existing `0` row violates, and the database refuses it: loud, at apply + time. But **swapping** two members' ints (`DRAFT: 0, PUBLISHED: 5` → `5, 0`) leaves the + admitted SET identical, so the new `CHECK` applies cleanly and every stored row has + quietly changed meaning. + - **One shape is invisible even to the diff:** a remap combined with a compensating + `@values` reorder renders a byte-identical `CHECK`, so the diff is EMPTY and no + migration is emitted at all. + + Nothing in the pipeline can see the meaning change in any of these: the column holds bare + integers, and neither introspection nor the committed schema snapshot records which member + an integer stood for. Closing it needs the mapping itself carried in gen-state or the + snapshot so a diff can compare member→int PAIRS rather than the value set — a design + decision, not a patch. No current consumer remaps; documented here rather than left + implied by D8's "migration safety" heading, and pinned by + `expected-schema-enum-intvaluemap.test.ts` so the accident that currently protects the + common case cannot be removed silently. - Value aliasing (`allow_alias`-style opt-out of the duplicate-value rejection) — no current consumer. - Native Postgres `CREATE TYPE ... AS ENUM` — unrelated to this design, still deferred diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts index 41c8e6df4..b950f3de5 100644 --- a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts +++ b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts @@ -203,3 +203,51 @@ describe("int-backed enum — backing-mode change is blocked by the existing gua expect(r.changes.find((c) => c.kind === "change-column-type")).toBeUndefined(); }); }); + +// RE-mapping a member inside a map that stays present is a different animal from D8's +// add/remove, and nothing in the pipeline understands it: the column holds bare integers, +// and neither introspection nor the committed snapshot records which member an integer +// stood for. What currently protects the common case is an ACCIDENT — dropping a CHECK is +// destructive, so the remap trips `allow.dropCheck` on its way past. These tests pin that +// accident (so it cannot be relaxed silently) and pin the one shape it does not cover. +describe("int-backed enum — re-mapping a member's integer", () => { + const MAP_SWAPPED = { DRAFT: 5, PUBLISHED: 0, ARCHIVED: 9 }; + const MAP_MOVED = { DRAFT: 1, PUBLISHED: 5, ARCHIVED: 9 }; + + async function remapDiff(afterMap: Record, afterValues = VALUES) { + const before = buildExpectedSchema( + await loadJson(entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP })), + ); + const after = buildExpectedSchema( + await loadJson( + entityModel({ name: "status", "@values": afterValues, "@intValueMap": afterMap }), + ), + ); + return diff(after, before, { dialect: "postgres" }); + } + + // The CHECK list renders in @values order, so ANY remap changes its text — which is the + // only reason migrate sees a remap at all. + test("a swap trips the blocked drop-check rather than passing silently", async () => { + const r = await remapDiff(MAP_SWAPPED); + const drop = r.changes.find((c) => c.kind === "drop-check"); + expect(drop).toBeDefined(); + expect(drop!.status.state).toBe("blocked"); + expect(r.changes.find((c) => c.kind === "add-check")).toBeDefined(); + }); + + test("moving a member to an unused int behaves the same way", async () => { + const r = await remapDiff(MAP_MOVED); + expect(r.changes.find((c) => c.kind === "drop-check")!.status.state).toBe("blocked"); + }); + + // The gap the two tests above do NOT cover: reorder @values to compensate for the swap + // and the rendered CHECK is byte-identical, so there is no diff to block. Every stored + // row changes meaning with no migration emitted at all. Pinned as KNOWN, not as desired + // — if a future change makes this produce a diff, this test should be updated, not the + // behaviour reverted. + test("KNOWN GAP: a compensating @values reorder makes the remap invisible to the diff", async () => { + const r = await remapDiff(MAP_SWAPPED, ["PUBLISHED", "DRAFT", "ARCHIVED"]); + expect(r.changes).toEqual([]); + }); +}); From 71ccfcc732319929e316d68afcd97339d03096ef Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 16 Aug 2026 23:45:07 -0400 Subject: [PATCH 49/52] fix(integration-tests-kotlin): the AllTypes oracle was missing intEnumVal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `persistence-conformance` has no skip valve — every port runs every fixture — so adding `intEnumVal` to the shared corpus obliged all five ports to carry it. Four did: Java and Python drive their columns from metadata, and C# regenerates its AppDbContext. Kotlin's oracle is a HAND-WRITTEN Exposed table, and nobody added the column, so QueryScenarioConformanceTest[20] and [27] died with IllegalStateException: No column 'intEnumVal' on table 'all_types' The lane that catches this runs on release tags and manual dispatch only, so the branch would have merged with a standing red nothing on a PR would have shown. Fixed by supplying the missing piece, not by narrowing the corpus. The new IntBackedEnumColumnType is the harness analogue of the customEnumeration the Kotlin generator emits: a Column carrying the member SYMBOL over an INTEGER column, translating through @intValueMap in both directions, and THROWING on a stored int that maps to no member — the same ruling every port now implements. It is String-typed rather than enum-typed for the reason already documented for the jsonb columns beside it: this generic runner moves plain YAML scalars and has no generated enum class to bind. Both coercion paths need an explicit guard ahead of the sqlType dispatch, whose `int` branch would otherwise try "PUBLISHED".toInt() — the column is physically INTEGER while its authoring value is a symbol, which is the one shape that dispatch cannot infer. QueryScenarioConformanceTest: 27/27. Co-Authored-By: Claude Opus 5 (1M context) --- .../integration/kotlin/QueryScenarioRunner.kt | 8 +++ .../kotlin/tables/AllTypesTable.kt | 8 +++ .../kotlin/tables/IntBackedEnumColumnType.kt | 60 +++++++++++++++++++ 3 files changed, 76 insertions(+) create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt index 88ec7e9ed..081ccb599 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt @@ -33,6 +33,7 @@ import org.jetbrains.exposed.sql.update import org.jetbrains.exposed.sql.statements.InsertStatement import org.jetbrains.exposed.sql.transactions.transaction import com.metaobjects.integration.kotlin.tables.AllTypesTable +import com.metaobjects.integration.kotlin.tables.IntBackedEnumColumnType import java.math.BigDecimal import java.net.URI import java.sql.DriverManager @@ -418,6 +419,9 @@ object QueryScenarioRunner { */ private fun coerceForWrite(raw: Any?, col: Column<*>): Any? { if (raw == null) return null + // Int-backed field.enum: bind the member SYMBOL and let the column's codec encode it to + // the stored int (see the same guard in `coerce`). + if (col.columnType is IntBackedEnumColumnType) return raw.toString() val type = col.columnType.sqlType().lowercase() return when { type == "uuid" -> if (raw is UUID) raw else UUID.fromString(raw.toString().lowercase()) @@ -607,6 +611,10 @@ object QueryScenarioRunner { */ private fun coerce(raw: Any?, col: Column<*>): Any? { if (raw == null) return null + // An int-backed field.enum is a Column over INTEGER: the authoring value is the + // member SYMBOL and the column's own codec maps it to the stored int. Checked BEFORE the + // sqlType dispatch, whose `int` branch would try "PUBLISHED".toInt(). + if (col.columnType is IntBackedEnumColumnType) return raw.toString() val type = col.columnType.sqlType().lowercase() return when { // uuid columns compare against java.util.UUID — the YAML supplies a string diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt index 5c1a86473..9b29270d4 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt @@ -29,6 +29,7 @@ import org.jetbrains.exposed.sql.json.jsonb * - default `field.timestamp` (instant/TZ-aware, ADR-0036 Wave 2) → `instantWithTimeZone("tsTzVal")` (TIMESTAMPTZ) * - `field.currency` (@currency USD) → `long("moneyVal")` (BIGINT minor units) * - `field.enum` (@values LOW/MEDIUM/HIGH) → `varchar("enumVal", 64)` (text + CHECK in DDL) + * - `field.enum` + `@intValueMap` (int-backed) → `intBackedEnum("intEnumVal", …)` (INTEGER + int CHECK) * - `field.uuid` (non-key, @required) → `uuid("uuidVal")` (Postgres native uuid) * - `field.object` (@objectRef Settings, @storage jsonb) → `jsonb("settings", …)` (real Postgres JSONB) * - `field.object` (@objectRef Label, @storage jsonb, isArray) → `jsonb("labels", …)` (JSONB array) @@ -64,6 +65,13 @@ object AllTypesTable : Table("all_types") { val tsTzVal = instantWithTimeZone("tsTzVal") val moneyVal = long("moneyVal") val enumVal = varchar("enumVal", 64) + // INT-BACKED `field.enum` (@intValueMap): physically INTEGER (+ the canonical DDL's + // CHECK (… IN (0, 5, 9))), carrying the member SYMBOL in and out — storage changes, the + // wire format does not. Nullable to match the canonical DDL (`"intEnumVal" INTEGER`, no + // NOT NULL). See [IntBackedEnumColumnType] for why this is a Column here while the + // GENERATED form binds a real enum through customEnumeration. + val intEnumVal = + intBackedEnum("intEnumVal", mapOf("DRAFT" to 0, "PUBLISHED" to 5, "ARCHIVED" to 9)).nullable() val uuidVal = uuid("uuidVal") // `field.uri` → plain `text` column carrying the verbatim URI string (Postgres has no uri // type). See [MetaUriColumnType] — round-trips the URI unchanged. diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt new file mode 100644 index 000000000..2dd0d6d0a --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt @@ -0,0 +1,60 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Column +import org.jetbrains.exposed.sql.ColumnType +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed column type for an INT-BACKED `field.enum` (`@intValueMap`) — + * the test-harness analogue of the `customEnumeration(...)` call + * [com.metaobjects.generator.kotlin.KotlinExposedTableGenerator] emits for such a field. + * + * The column is physically `INTEGER` (plus the canonical DDL's `CHECK (… IN (0, 5, 9))`) while + * the value carried in and out is the member SYMBOL — which is the whole contract of int-backing: + * storage changes, the wire format does not. + * + * It is a `Column` rather than a `Column` because this cross-port oracle has no + * generated enum class to bind — the generic [com.metaobjects.integration.kotlin.QueryScenarioRunner] + * moves plain YAML scalars. The GENERATED form binds a real Kotlin enum through + * `customEnumeration`, and is covered separately by the codegen tests; the divergence is the same + * one already documented for the jsonb columns in [AllTypesTable]. + * + * **A stored int that maps to no member THROWS** rather than surfacing the raw value or nulling + * it, matching the generated `customEnumeration`'s `else ->` branch and every sibling port: the + * row holds data the model says is impossible, and both alternatives hand the caller something + * untrue. The write side is exhaustive by construction upstream (`@intValueMap`'s keys are + * loader-validated to equal `@values`), so an unmapped SYMBOL here can only be a harness bug — + * it fails loudly for the same reason. + */ +internal class IntBackedEnumColumnType( + private val intByMember: Map, +) : ColumnType() { + + private val memberByInt: Map = + intByMember.entries.associate { (member, stored) -> stored to member } + + override fun sqlType(): String = "INTEGER" + + override fun valueFromDB(value: Any): String { + val stored = (value as? Number)?.toInt() + ?: error("int-backed field.enum column read a non-numeric value: $value") + return memberByInt[stored] + ?: error( + "field.enum read stored value $stored with no member in @intValueMap " + + "(declared: $intByMember) — the database holds a value the model does not describe." + ) + } + + override fun notNullValueToDB(value: String): Any = + intByMember[value] + ?: error("field.enum has no @intValueMap entry for member '$value' (declared: $intByMember).") + + override fun nonNullValueToString(value: String): String = notNullValueToDB(value).toString() +} + +/** + * Column builder for an int-backed `field.enum`: a `Column` carrying the member symbol + * over an `INTEGER` column, translating through [intByMember] (`@intValueMap`) in both directions. + */ +internal fun Table.intBackedEnum(name: String, intByMember: Map): Column = + registerColumn(name, IntBackedEnumColumnType(intByMember)) From 32be043f3abebe95cbaf0be4e132c6098cd4d6dd Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Mon, 17 Aug 2026 06:58:49 -0400 Subject: [PATCH 50/52] docs(versioning): registry vocabulary does not force a MINOR MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The rule read "PATCH (MINOR if it adds registry vocabulary — cross-port conformance surface)" in RELEASING.md, and ADR-0035's cadence bullet listed "a newly-supported vocab member" among the MINOR triggers. Read literally that makes ANY registry addition a MINOR — which is the exact churn ADR-0035 was written to prevent. 0.22.0 and 0.23.0 were both cut MINOR for additions a project declaring no requirement.* nodes could not observe at all, each changelog saying so in its own opening paragraph. The error is treating expected-registry.json as consumer surface. It is an INTERNAL gate: five ports byte-matching one manifest is how we stop the ports drifting from each other, and it says nothing about whether an adopter's project changes. "New public surface, not code size" was the right instinct; "vocab member" was the wrong unit. Vocabulary sorts by what it can do to a consumer: - a new ATTRIBUTE is a PATCH — reachable only by authoring it, every existing document loads unchanged and emits byte-identical output; - a new top-level TYPE is a MINOR — a new modeling concept with its own children, validation and usually tooling surface (requirement.* brought a verify pass and summary output); - a new SUBTYPE goes either way, and the test is whether it is INERT: PATCH when nothing but authoring it can reach it, MINOR when it narrows something previously permitted (closing a wildcard, promoting a reserved-not-registered member), changes what existing metadata means or emits, or headlines a release you want behind a range bump. Also stops the caret rule being inverted. "^0.22.x resolves <0.23.0, so a consumer adopts a MINOR deliberately" is a reason to CHOOSE minor when that gate is wanted, not a reason additive vocabulary must be minor. Four registries move per cut here, so a minor spent on an unobservable change is a gate you no longer have when something real needs it. Recorded as ADR-0035 Amendment 1 rather than a silent rewrite, with the operative table + worked rows in RELEASING.md and a pointer from the releasing skill (which is where the level actually gets chosen). The post-1.0 compat promise is untouched: a BREAKING vocabulary change still requires a MAJOR, attribute, subtype or type alike. Co-Authored-By: Claude Opus 5 (1M context) --- .claude/skills/releasing/SKILL.md | 6 ++ CHANGELOG.md | 70 ++++++++++++++++++- docs/RELEASING.md | 40 ++++++++++- ...lity-commitment-and-version-unification.md | 42 ++++++++++- 4 files changed, 153 insertions(+), 5 deletions(-) diff --git a/.claude/skills/releasing/SKILL.md b/.claude/skills/releasing/SKILL.md index 0347fc10e..082988f9e 100644 --- a/.claude/skills/releasing/SKILL.md +++ b/.claude/skills/releasing/SKILL.md @@ -66,6 +66,12 @@ done actually on npm (`npm view dist-tags`). Patch bump unless a public API changed; this repo bumps the whole set in lockstep (pre-1.0). +**Registry vocabulary does not force a MINOR** — a new *attribute* is a PATCH, a new +top-level *type* is a MINOR, and a new *subtype* is a PATCH when inert. See +`docs/RELEASING.md` → "The vocabulary rule" before choosing the level; the old +"any registry addition ⇒ MINOR" reading spent two minors on changes no adopter +could observe. + ## Phase 2 — Build fresh (the stale-`dist` trap) `dist/` is gitignored, `bun publish` does NOT rebuild, and `main` points at `dist/` diff --git a/CHANGELOG.md b/CHANGELOG.md index 85a1eca38..ed7865972 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,9 +9,18 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ### Added — int-backed `field.enum` storage via `@intValueMap` (all five ports) -**New registered vocabulary, so the line carrying this is a MINOR.** Purely additive: a -`field.enum` that declares no `@intValueMap` is byte-identical to before, string-backed as -always. +**This is a PATCH, not a MINOR** — and the reasoning is itself a change, so it is worth +stating. The old policy read "any registry addition ⇒ MINOR", which spent `0.22.0` and +`0.23.0` on changes a project could not observe at all. `expected-registry.json` is an +**internal** gate: five ports byte-matching one manifest is how we stop the ports drifting +from each other, and it says nothing about whether an adopter's project changes. Vocabulary +now sorts by what it can do to a consumer — a new **attribute** is a PATCH, a new top-level +**type** is a MINOR, and a new **subtype** is a PATCH when nothing but authoring it can reach +it. This line adds one attribute (`@intValueMap`) and one inert attr subtype (`attr.intMap`), +so: PATCH. See `docs/RELEASING.md` → "The vocabulary rule" and ADR-0035 Amendment 1. + +Purely additive on its own terms too: a `field.enum` that declares no `@intValueMap` is +byte-identical to before, string-backed as always. `@intValueMap` is an optional `{memberSymbol: int}` map on `field.enum` that declares each member's **stored integer**. The column becomes `integer` with an integer `CHECK` instead of @@ -80,6 +89,61 @@ backing-mode change needs. Design: `docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`. Adopter view: [`docs/features/field-types.md`](docs/features/field-types.md). +### Changed — registry vocabulary no longer forces a MINOR (policy) + +`docs/RELEASING.md`'s versioning table said "PATCH (MINOR if it adds registry vocabulary — +cross-port conformance surface)", and ADR-0035's cadence bullet listed "a newly-supported +vocab member" among the MINOR triggers. Read literally, that made **any** registry addition a +MINOR — the exact churn ADR-0035 was written to prevent. `0.22.0` and `0.23.0` were both cut +MINOR for additions a project declaring no `requirement.*` nodes could not observe at all, +each changelog saying so in its own opening paragraph. Four registries move per cut here, so a +wasted minor is not free — and a minor spent on an unobservable change is a gate you no longer +have when something real needs it. Corrected to sort by consumer impact: **attribute ⇒ PATCH, +top-level type ⇒ MINOR, subtype ⇒ PATCH when inert** (nothing but authoring it can reach it; +MINOR when it narrows something previously permitted, changes existing metadata's meaning, or +headlines a release on purpose). Recorded as ADR-0035 Amendment 1; the post-1.0 compat promise +is untouched — a *breaking* vocabulary change still requires a MAJOR. + +### Fixed — an FK into a table whose key carries `@column` phantom-diffed forever (npm) + +`buildForeignKeys` resolved a target FK field's PHYSICAL column by applying the naming +strategy to its raw logical name, so a target primary key with an explicit `@column` override +(`id` → `"Id"`) made **every** foreign key into that table diff on every run — expected the +naming-strategy name, actual the override, nothing an adopter could do to converge. It now +resolves through the target entity's own field, which is how `fkCols` already handled the +source side; the two halves simply disagreed. + +### Fixed — views in a table-less schema were excluded from the diff entirely (npm) + +**Generated-output change — the first `meta migrate` after upgrading may emit view changes +that were always due.** `declaredSchemas` was built from `expected.tables` only, so a model +declaring views in a schema with no table of its own (an API/read-model schema beside an +all-`public` entity model) never brought that schema into scope — and a schema out of scope is +excluded from *both* sides of the diff. Its views were never compared, so a genuine missing or +extra view, or real drift inside an opaque `@sql` body, went undetected rather than reported. +View schemas now join the scope set. Same shape as 0.21.6's `ON DELETE` fix: a PATCH that +surfaces drift which was already there. + +### Fixed — a chained abstract `field.enum` emitted a broken Kotlin type (Maven) + +`KotlinTypeMapper.enumTypeName` named a chained abstract enum after the TOP-MOST root of the +`extends` chain (via `resolveSuperRoot`) while its own FR-019 arm resolved the shared +declaration from the IMMEDIATE super — the rule TS, C#, Java and Python all use. Not a rival +model, a split-brain: the two halves disagreed about which declaration is "the type", and on a +chained declaration (a root abstract `Money extends` a root abstract `@provided Currency`) +that produced a flatly broken emit. Naming now uses the immediate super per ADR-0026 §2 (a +materialized type is named for its own declaration), so a chain yields one type per +declaration, each carrying the members it inherits. Non-chained output is byte-identical — +with no further super, the root walk already returned the immediate super. `resolveSuperRoot` +had exactly one caller and is deleted. The chained alias stays LEGAL rather than being +rejected: it cannot mutate the vocabulary it inherits (a chained declaration carrying its own +`@values` — or its own `@intValueMap` — already errors `ERR_ENUM_EXTENDS_VALUES_CONFLICT`), +and banning it would carve an enum-only hole in ADR-0029's general `extends` grammar to delete +a provably harmless construct. Newly gated by `enum-abstract-chained-extends` (positive) and +`error-enum-chained-extends-values-conflict` (negative), plus the chained declaration restored +to the `shared-provided-enum` codegen corpus all five ports load — the decl-level #246 check +had been code-only in every port with no fixture behind it. + ## [0.23.1] — npm `0.23.1` · PyPI `0.23.1` · NuGet `0.23.1` · Maven `7.23.1` A coordinated **PATCH** across all four registries. Every one of them carries a real changed diff --git a/docs/RELEASING.md b/docs/RELEASING.md index 111dcdd79..3f962ade7 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -139,11 +139,49 @@ bytes**, not "did output change." | Public API **breaking** (required param, removed/renamed export, changed CLI semantics) | `relativeModuleSpecifier` +required param | **MINOR** | MAJOR | | Output change = **pure bugfix** (wrong output corrected; correct output byte-identical) | 0.19.3 payload naming, 0.19.4 TPH stamping | **PATCH** | PATCH | | Output change alters **shape/default of *correct* output** (renamed generated export, changed default, dropped artifact) | `extStyle` `"none"→"js"` default flip | **MINOR** + a "Generated-output change" changelog flag + an opt-out where feasible | MAJOR if consumer code referencing the output breaks; else MINOR | -| New **opt-in** codegen feature, default output byte-identical | new generator defaulting off | PATCH (MINOR if it adds registry vocabulary — cross-port conformance surface) | MINOR | +| New **opt-in** codegen feature, default output byte-identical | new generator defaulting off | PATCH | MINOR | +| New **attribute** on an existing type/subtype | `@intValueMap`, `@lenient`, `@maxTokens` | **PATCH** | MINOR | +| New **subtype** of an existing type | `field.uri`, `index.lookup`, `attr.intMap` | **PATCH** when inert (see the vocabulary rule below); **MINOR** when it changes existing metadata's meaning/output, narrows something previously permitted, or headlines a feature | MINOR | +| New top-level metadata **type** | `requirement.*`, `index.*`, `api.*` | **MINOR** | MINOR | | Wire-contract / conformance behavior change of already-valid deployments | FR-036 enforcement | **MINOR**, loud notice (pre-1.0 MINOR *is* the breaking slot) | MAJOR | | Wire behavior fixed to match the documented/conformance contract | 0.19.1 `@min` clamp | PATCH | PATCH | | No changed product file in a port | PyPI/NuGet/Maven at 0.20.14 | **Version-parity bump at the shared patch number** — publish identical content at the new version; never skip a registry (single-shared-patch policy, standing since 0.20.13) | same | +### The vocabulary rule (corrected 2026-08-17) + +**Adding registry vocabulary does NOT, by itself, force a MINOR.** The rule used to read +"PATCH (MINOR if it adds registry vocabulary — cross-port conformance surface)", and that was +wrong on its own terms: `expected-registry.json` is an **internal** gate. Every port +byte-matching one manifest is how we stop the five ports drifting from each other — it says +nothing about whether an *adopter's* project changes. Treating an internal gate's churn as an +adopter-facing event is what burned the minors: `0.22.0` and `0.23.0` were both cut MINOR for +changes that a project declaring no `requirement.*` nodes could not observe at all, which each +changelog says out loud in its own opening paragraph. Sort vocabulary by what it can do to a +consumer, which splits three ways: + +- **A new ATTRIBUTE is a PATCH.** You get it only by authoring it. Every existing document + loads unchanged and emits byte-identical output, so there is nothing for a consumer to adopt + deliberately. +- **A new TOP-LEVEL TYPE is a MINOR.** A type is a new modeling concept with its own children, + validation and (usually) tooling surface — `requirement.*` brought its own `verify` pass and + summary output. That is a thing a consumer newly depends on, and it deserves a deliberate + range bump. +- **A new SUBTYPE goes either way, and the test is whether it is INERT.** PATCH when nothing + but authoring it can reach it: no existing valid document changes meaning or output, nothing + previously permitted is narrowed, nothing reserved is consumed. MINOR when any of those + fails — a subtype that closes a wildcard, promotes a reserved-not-registered member + (ADR-0007 Amendment 2 / ADR-0040), or shifts what the recommended shape for an existing + field *is* — or when you deliberately want it behind a range bump because it headlines a + release. Most subtypes carry a native type and behavior (that is ADR-0037's very test for + making something a subtype), so read them carefully; but "it appears in the registry + manifest" is not the deciding fact. + +**Do not invert the caret rule.** "Pre-1.0 `^0.22.x` resolves `<0.23.0`, so a consumer adopts +a MINOR deliberately" is a reason to *choose* MINOR when you want that gate. It is not a +reason additive vocabulary *must* be MINOR. The gate exists to be used on purpose, not by +reflex — and a minor spent on a change nobody can observe is a gate you no longer have when +something real needs it. + **The `extStyle` 0.20.0 case, for calibration:** it was correctly MINOR — but for the *API break* (`relativeModuleSpecifier` gained a required param — a public export) **and** the *default flip* (churns every existing project's diff on regen), NOT because "output diff --git a/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md b/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md index 7e8b127aa..749efcca0 100644 --- a/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md +++ b/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md @@ -165,7 +165,9 @@ still want, then freeze. Before cutting 1.0: (§2) fixes the underlying churn: most releases stop being metamodel events. Apply the semver rule strictly — the trigger is *new public surface, not code size*: a **package MINOR** adds surface a consumer can newly depend on (codegen output, a CLI - flag, a newly-supported vocab member); a **package PATCH** is a bugfix/refactor with + flag, a newly-supported vocab member — **but see Amendment 1: "vocab member" is too + coarse, and taking it literally re-created exactly the minor churn this bullet was + written to stop**); a **package PATCH** is a bugfix/refactor with no new surface (e.g. the `0.15.2` output-prompt fix — one-port, no surface); the **Metamodel spec version** bumps only when the shared vocabulary/wire contract itself changes. A run of minors for non-additive changes is the "minor churn" @@ -175,6 +177,44 @@ still want, then freeze. Before cutting 1.0: - **Migration + policy docs:** a `0.x → 1.0` migration guide and a published compatibility policy (this ADR's §1, consumer-facing). +## Amendment 1 (2026-08-17) — "a newly-supported vocab member" splits three ways + +The cadence-discipline bullet above lists "a newly-supported vocab member" among the +triggers for a package MINOR. Applied literally it became a rule that **any** registry +addition is a MINOR — and that rule produced the churn this ADR exists to prevent: +`0.22.0` and `0.23.0` were both cut MINOR for additions a project declaring no +`requirement.*` nodes could not observe at all, each changelog saying so in its own +opening paragraph. Four registries move per cut here, so a wasted minor is not free. + +The error is treating `expected-registry.json` as consumer surface. It is an **internal** +gate: five ports byte-matching one manifest is how we stop the ports drifting from each +other. Its churn is evidence about *us*, not about an adopter's project. "New public +surface, not code size" was the right instinct; "vocab member" was the wrong unit. +Vocabulary sorts by what it can do to a consumer: + +- **A new ATTRIBUTE ⇒ PATCH.** Reachable only by authoring it; every existing document + loads unchanged and emits byte-identical output. Nothing to adopt deliberately. +- **A new top-level TYPE ⇒ MINOR.** A new modeling concept with its own children, + validation, and usually tooling surface (`requirement.*` brought a `verify` pass and + summary output). Genuinely new surface to depend on. +- **A new SUBTYPE ⇒ either, and the test is whether it is INERT.** PATCH when only + authoring it can reach it: no existing valid document changes meaning or output, + nothing previously permitted is narrowed, nothing reserved is consumed. MINOR when + any of those fails — closing a wildcard, promoting a reserved-not-registered member + (ADR-0007 Amendment 2 / ADR-0040), or shifting what the recommended shape for an + existing field is — or when it headlines a release and you want the range bump on + purpose. + +The caret rule is not to be inverted. "Pre-1.0 `^0.22.x` resolves `<0.23.0`, so a +consumer adopts a MINOR deliberately" is a reason to **choose** MINOR when that gate is +wanted; it is not a reason additive vocabulary must be MINOR. A minor spent on a change +nobody can observe is a gate you no longer have when something real needs it. + +This amendment changes cadence policy only. It does not touch §1's post-1.0 compat +promise: after 1.0, a **breaking** change to the metamodel vocabulary still requires a +MAJOR, whether the break is an attribute, a subtype, or a type. Operational form of the +rule (with worked rows): `docs/RELEASING.md` → "Versioning policy (pre-1.0)". + ## Consequences - After 1.0, a breaking change to the metamodel vocabulary, the canonical/wire From ad33bc19dd3e471541c900752274d7fcdeaf23f6 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Mon, 17 Aug 2026 06:59:37 -0400 Subject: [PATCH 51/52] fix(runtime-ts): the ObjectManager had no int-backed enum codec at all MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `create()` bound the member SYMBOL straight into the INTEGER column and Postgres rejected the statement outright: invalid input syntax for type integer: "PUBLISHED" at kysely-driver.ts:118 -> object-manager.ts:394 (create) The TS codec shipped as a Drizzle `customType`, which covers the CODEGEN path only. `runtime-ts`'s metadata-driven ObjectManager is a different seam — the one the persistence-conformance corpus drives — and nothing taught it about @intValueMap, so int-backed enums were unusable through the runtime while the generated code worked. Same class as the Kotlin oracle gap in the previous commit, found the same way: by the corpus, once it had a fixture to run. Both directions now live in type-coercer.ts beside the jsonb/boolean coercions: - WRITE encodes symbol -> declared int, dialect-independent (the column is INTEGER on every dialect; SQLite has one integer storage class), so unlike the boolean mapping it is NOT gated on `dialect === "sqlite"`. An unmapped symbol passes through for the column's CHECK to reject — matching Python's write codec, and every port leaves the write side to the database. - READ decodes int -> symbol so the runtime's return value is the symbol in both backing modes (ADR-0019: int-backing is invisible above this codec). A stored int with no member THROWS, like every other port and like the generated customType's fromDriver. Filter values needed the same treatment at `compileEntry` — one seam every operator passes through with the field in hand. Without it a WHERE on an int-backed enum 500s the same way a create did. `like` is unreachable for such a field (the loader's field-level band drops it) and `isNull` carries no value. `intValueMapOf` reads RESOLVING per ADR-0039 and is duplicated from codegen-ts rather than shared: runtime-ts must not depend on a codegen package, and only one of the two ships to a server at runtime. Gated by 20 unit tests covering both directions, both dialects, the falsy 0 member, a driver-stringified int, the unmapped-throw, and every filter operator. One of them initially passed for the wrong reason — `makeOrder(undefined)` fires the default parameter and hands back the int-backed shape — so the helper now takes `null` for "no map" and says why. integration-tests (ts) on Testcontainers PG: 219/0, up from 216/3. Co-Authored-By: Claude Opus 5 (1M context) --- .../packages/runtime-ts/src/query-builder.ts | 42 ++++- .../packages/runtime-ts/src/type-coercer.ts | 107 +++++++++++- .../test/int-backed-enum-codec.test.ts | 162 ++++++++++++++++++ 3 files changed, 298 insertions(+), 13 deletions(-) create mode 100644 server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts diff --git a/server/typescript/packages/runtime-ts/src/query-builder.ts b/server/typescript/packages/runtime-ts/src/query-builder.ts index 2562e0e0c..8650316ff 100644 --- a/server/typescript/packages/runtime-ts/src/query-builder.ts +++ b/server/typescript/packages/runtime-ts/src/query-builder.ts @@ -7,6 +7,7 @@ import { resolveTableName, resolveColumnName, } from "@metaobjectsdev/metadata"; import { MetadataError } from "./errors.js"; +import { intValueMapOf } from "./type-coercer.js"; import type { WhereClause, OrderBy, PrimitiveValue, Row, SelectSpec, InsertSpec, UpdateSpec, DeleteSpec, CountSpec, @@ -118,12 +119,20 @@ function compileEntry( const field = getField(entity, fieldName); const column = resolveColumnName(field, strategy); + // An int-backed field.enum (@intValueMap) stores an INTEGER while its filter value + // is the member SYMBOL, so every comparison value must be encoded before it is + // bound — otherwise Postgres rejects the statement outright ("invalid input syntax + // for type integer"). Applied at this ONE seam because every operator arrives here + // with the field in hand; `like` is not reachable for such a field (the loader's + // field-level operator band drops it), and `isNull` carries no value to encode. + const enc = (v: T): T => encodeFilterValue(field, v); + if (value === null) return { kind: "isNull", column, not: false }; if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") { - return { kind: "eq", column, value }; + return { kind: "eq", column, value: enc(value) }; } if (Array.isArray(value)) { - return { kind: "in", column, values: value as PrimitiveValue[] }; + return { kind: "in", column, values: (value as PrimitiveValue[]).map(enc) }; } // If multiple operators are set on one field ({ $gte: 5, $lte: 10 }), the FIRST match wins @@ -131,24 +140,39 @@ function compileEntry( const op = value; if ("$eq" in op && op.$eq !== undefined) { if (op.$eq === null) return { kind: "isNull", column, not: false }; - return { kind: "eq", column, value: op.$eq }; + return { kind: "eq", column, value: enc(op.$eq) }; } if ("$ne" in op && op.$ne !== undefined) { if (op.$ne === null) return { kind: "isNull", column, not: true }; - return { kind: "ne", column, value: op.$ne }; + return { kind: "ne", column, value: enc(op.$ne) }; } - if ("$gt" in op && op.$gt !== undefined) return { kind: "gt", column, value: op.$gt }; - if ("$gte" in op && op.$gte !== undefined) return { kind: "gte", column, value: op.$gte }; - if ("$lt" in op && op.$lt !== undefined) return { kind: "lt", column, value: op.$lt }; - if ("$lte" in op && op.$lte !== undefined) return { kind: "lte", column, value: op.$lte }; + if ("$gt" in op && op.$gt !== undefined) return { kind: "gt", column, value: enc(op.$gt) }; + if ("$gte" in op && op.$gte !== undefined) return { kind: "gte", column, value: enc(op.$gte) }; + if ("$lt" in op && op.$lt !== undefined) return { kind: "lt", column, value: enc(op.$lt) }; + if ("$lte" in op && op.$lte !== undefined) return { kind: "lte", column, value: enc(op.$lte) }; if ("$like" in op && op.$like !== undefined) return { kind: "like", column, pattern: op.$like }; - if ("$in" in op && op.$in !== undefined) return { kind: "in", column, values: op.$in }; + if ("$in" in op && op.$in !== undefined) return { kind: "in", column, values: op.$in.map(enc) }; if ("$isNull" in op && op.$isNull !== undefined) { return { kind: "isNull", column, not: !op.$isNull }; } throw new MetadataError(`No recognized operator on filter for field '${fieldName}'`); } +/** + * Encode one filter comparison value for an int-backed `field.enum`: the member + * SYMBOL becomes its declared integer. Anything else — a string-backed enum, a + * non-string value, or a symbol with no mapping — passes through untouched. An + * unmapped symbol is deliberately NOT rejected here: a filter that matches nothing + * is the honest answer for a member that does not exist, and the loader already + * pins `@intValueMap`'s key set to `@values`. + */ +function encodeFilterValue(field: MetaData, value: T): T { + const intMap = intValueMapOf(field); + if (intMap === undefined || typeof value !== "string") return value; + const stored = intMap[value]; + return (typeof stored === "number" ? stored : value) as T; +} + function normalizeOrderBy( input: QueryOpts["orderBy"], entity: MetaData, strategy: ColumnNamingStrategy, ): OrderBy[] | undefined { diff --git a/server/typescript/packages/runtime-ts/src/type-coercer.ts b/server/typescript/packages/runtime-ts/src/type-coercer.ts index ab5cc203a..3fbdfece9 100644 --- a/server/typescript/packages/runtime-ts/src/type-coercer.ts +++ b/server/typescript/packages/runtime-ts/src/type-coercer.ts @@ -17,9 +17,11 @@ import type { MetaData } from "@metaobjectsdev/metadata"; import { TYPE_FIELD, FIELD_SUBTYPE_BOOLEAN, + FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_STORAGE, FIELD_ATTR_DB_COLUMN_TYPE, + FIELD_ATTR_INT_VALUE_MAP, STORAGE_JSONB, STORAGE_FLATTENED, DB_COLUMN_TYPE_JSONB, @@ -28,14 +30,111 @@ import type { Dialect, Row } from "./persistence-driver.js"; export function coerceRowOnRead(entity: MetaData, row: Row, dialect: Dialect): Row { const hydrated = deserializeJsonbObjectFields(entity, row); - if (dialect !== "sqlite") return hydrated; - return mapBooleansFromInt(entity, hydrated); + const decoded = decodeIntBackedEnums(entity, hydrated); + if (dialect !== "sqlite") return decoded; + return mapBooleansFromInt(entity, decoded); +} + +/** + * The effective `@intValueMap` (member symbol → integer) for an int-backed + * `field.enum`, or undefined when the enum is string-backed. Its PRESENCE is the + * whole trigger for integer persistence (design D5) — there is no separate flag. + * + * ADR-0039: RESOLVING (`attr`, not `ownAttr`), and load-bearing rather than + * incidental. Post-#246 an own `@intValueMap` against a shared (root-level + * abstract) enum is `ERR_ENUM_EXTENDS_VALUES_CONFLICT`, so the map lives on the + * SHARED DECLARATION and every consuming field INHERITS it — an own-only read + * would see undefined on exactly the shape adopters are steered toward and bind + * the symbol straight into an integer column. + * + * Mirrors `codegen-ts`'s `intValueMapOf`; duplicated rather than shared because + * `runtime-ts` must not depend on a codegen package (the two are disjoint trees, + * and only one of them ships to a server at runtime). + */ +export function intValueMapOf(field: MetaData): Record | undefined { + if (field.subType !== FIELD_SUBTYPE_ENUM) return undefined; + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return raw as Record; +} + +/** + * Encode a member SYMBOL to its declared integer for an int-backed `field.enum`. + * + * An UNMAPPED symbol is passed through untouched rather than rejected here: the + * column's `CHECK` is what enforces membership, and inventing a value would hide + * the drift. Matching Python's write codec exactly — every port leaves the write + * side to the database. + */ +function encodeIntBackedEnum(intMap: Record, value: unknown): unknown { + if (typeof value !== "string") return value; + const stored = intMap[value]; + return typeof stored === "number" ? stored : value; +} + +/** + * Decode the stored integer back to its member symbol, so the runtime's return + * value is the symbol in both backing modes — int-backing is a persistence-layer + * concern and must be invisible above this codec (ADR-0019). + * + * An int with no member THROWS. The row holds data the model says is impossible + * (a hand-written INSERT, or a member removed without a migration); surfacing the + * raw integer would hand the caller a "member" that is not one, and returning + * null would hide the corruption behind a nullable column. Every port throws + * here — the generated Drizzle `customType`'s `fromDriver` included. + */ +function decodeIntBackedEnums(entity: MetaData, row: Row): Row { + let out: Row | null = null; + // ADR-0039: resolving — an int-backed enum may be inherited from a base via extends. + for (const child of entity.children()) { + if (child.type !== TYPE_FIELD) continue; + const intMap = intValueMapOf(child); + if (intMap === undefined) continue; + if (!(child.name in row)) continue; + const raw = row[child.name]; + if (raw === null || raw === undefined) continue; + // A driver may hand back a BIGINT-ish string; Number() covers both shapes. + const stored = typeof raw === "number" ? raw : Number(raw); + if (!Number.isInteger(stored)) continue; + const member = Object.keys(intMap).find((k) => intMap[k] === stored); + if (member === undefined) { + throw new Error( + `field.enum '${child.name}' read stored value ${stored} with no member in ` + + `@intValueMap (declared: ${JSON.stringify(intMap)}) — the database holds a ` + + `value the model does not describe.`, + ); + } + out ??= { ...row }; + out[child.name] = member; + } + return out ?? row; +} + +function encodeIntBackedEnums(entity: MetaData, row: Row): Row { + let out: Row | null = null; + for (const child of entity.children()) { + if (child.type !== TYPE_FIELD) continue; + const intMap = intValueMapOf(child); + if (intMap === undefined) continue; + if (!(child.name in row)) continue; + const raw = row[child.name]; + if (raw === null || raw === undefined) continue; + const encoded = encodeIntBackedEnum(intMap, raw); + if (encoded === raw) continue; + out ??= { ...row }; + out[child.name] = encoded; + } + return out ?? row; } export function coerceRowOnWrite(entity: MetaData, row: Row, dialect: Dialect): Row { const jsonbColumned = serializeJsonbColumns(entity, row); - if (dialect !== "sqlite") return jsonbColumned; - return mapBooleansToInt(entity, jsonbColumned); + // Dialect-independent: an int-backed enum's column is INTEGER on every dialect + // (SQLite has one integer storage class), so the symbol→int encode is not gated + // on the dialect the way the boolean mapping below is. + const encoded = encodeIntBackedEnums(entity, jsonbColumned); + if (dialect !== "sqlite") return encoded; + return mapBooleansToInt(entity, encoded); } /** diff --git a/server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts b/server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts new file mode 100644 index 000000000..b48994c3e --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts @@ -0,0 +1,162 @@ +import { describe, test, expect } from "bun:test"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { TypeId, TYPE_OBJECT, TYPE_FIELD, TYPE_IDENTITY, + FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_ENUM, + FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP, + IDENTITY_SUBTYPE_PRIMARY, OBJECT_SUBTYPE_ENTITY } from "@metaobjectsdev/metadata"; +import { meta } from "./_meta-build.js"; +import { coerceRowOnRead, coerceRowOnWrite } from "../src/type-coercer.js"; +import { compileFilter } from "../src/query-builder.js"; + +// An int-backed field.enum (@intValueMap) persists as an INTEGER while the runtime's +// value — in, out, and in a filter — stays the member SYMBOL. This is the ObjectManager +// half of the codec: the generated Drizzle `customType` covers the codegen path, and +// nothing covered this one, so `create()` bound "PUBLISHED" straight into an integer +// column and Postgres rejected the statement. + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +/** + * An Order whose `status` is int-backed, plus a string-backed `kind` for contrast. + * Takes `null` — not `undefined` — for "no map": passing `undefined` explicitly would + * trigger the default parameter and silently hand back the int-backed shape, which is + * exactly how the first draft of the string-backed test passed for the wrong reason. + */ +function makeOrder(intMap: Record | null = INT_MAP): MetaData { + const order = meta(new TypeId(TYPE_OBJECT, OBJECT_SUBTYPE_ENTITY), "Order"); + order.addChild(meta(new TypeId(TYPE_FIELD, FIELD_SUBTYPE_LONG), "id")); + + const status = meta(new TypeId(TYPE_FIELD, FIELD_SUBTYPE_ENUM), "status"); + status.setAttr(FIELD_ATTR_VALUES, VALUES); + if (intMap !== null) status.setAttr(FIELD_ATTR_INT_VALUE_MAP, intMap); + order.addChild(status); + + const kind = meta(new TypeId(TYPE_FIELD, FIELD_SUBTYPE_ENUM), "kind"); + kind.setAttr(FIELD_ATTR_VALUES, VALUES); + order.addChild(kind); + + const primary = meta(new TypeId(TYPE_IDENTITY, IDENTITY_SUBTYPE_PRIMARY), "primary"); + primary.setAttr("fields", ["id"]); + order.addChild(primary); + return order; +} + +describe("int-backed field.enum — ObjectManager write codec", () => { + test("encodes the member symbol to its declared int", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: "PUBLISHED" }, "postgres"); + expect(row.status).toBe(5); + }); + + test("a string-backed enum on the same entity is untouched", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, kind: "PUBLISHED" }, "postgres"); + expect(row.kind).toBe("PUBLISHED"); + }); + + test("encodes on sqlite too — the column is INTEGER on every dialect", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: "ARCHIVED" }, "sqlite"); + expect(row.status).toBe(9); + }); + + test("null passes through", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: null }, "postgres"); + expect(row.status).toBeNull(); + }); + + // Membership is the column's CHECK to enforce; inventing a value here would hide drift. + test("an unmapped symbol is passed through for the database to reject", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: "NOPE" }, "postgres"); + expect(row.status).toBe("NOPE"); + }); + + test("a field absent from the row stays absent", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1 }, "postgres"); + expect("status" in row).toBe(false); + }); +}); + +describe("int-backed field.enum — ObjectManager read codec", () => { + test("decodes the stored int back to the member symbol", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: 5 }, "postgres"); + expect(row.status).toBe("PUBLISHED"); + }); + + test("decodes 0 — the falsy member the naive guard drops", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: 0 }, "postgres"); + expect(row.status).toBe("DRAFT"); + }); + + test("a driver-stringified integer still decodes", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: "9" }, "postgres"); + expect(row.status).toBe("ARCHIVED"); + }); + + test("null stays null", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: null }, "postgres"); + expect(row.status).toBeNull(); + }); + + test("a string-backed enum reads through unchanged", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, kind: "DRAFT" }, "postgres"); + expect(row.kind).toBe("DRAFT"); + }); + + // The row holds data the model says is impossible. Surfacing the raw int would hand + // the caller a "member" that is not one; null would hide the corruption. Every port throws. + test("a stored int with no member THROWS", () => { + expect(() => coerceRowOnRead(makeOrder(), { id: 1, status: 7 }, "postgres")).toThrow( + /stored value 7 with no member in @intValueMap/, + ); + }); + + test("an enum with no @intValueMap is left alone entirely", () => { + const row = coerceRowOnRead(makeOrder(null), { id: 1, status: 5 }, "postgres"); + expect(row.status).toBe(5); + }); +}); + +describe("int-backed field.enum — filter values", () => { + test("eq encodes the symbol", () => { + expect(compileFilter(makeOrder(), { status: "PUBLISHED" })).toEqual({ + kind: "eq", column: "status", value: 5, + }); + }); + + test("$ne encodes", () => { + expect(compileFilter(makeOrder(), { status: { $ne: "DRAFT" } })).toEqual({ + kind: "ne", column: "status", value: 0, + }); + }); + + test("$in encodes every member", () => { + expect(compileFilter(makeOrder(), { status: { $in: ["DRAFT", "ARCHIVED"] } })).toEqual({ + kind: "in", column: "status", values: [0, 9], + }); + }); + + test("a bare array encodes every member", () => { + expect(compileFilter(makeOrder(), { status: ["PUBLISHED"] })).toEqual({ + kind: "in", column: "status", values: [5], + }); + }); + + test("a string-backed enum filter is untouched", () => { + expect(compileFilter(makeOrder(), { kind: "PUBLISHED" })).toEqual({ + kind: "eq", column: "kind", value: "PUBLISHED", + }); + }); + + // A filter naming a member that does not exist should match nothing, not throw — + // and it must not silently become a WRONG integer. + test("an unmapped symbol passes through rather than becoming a wrong int", () => { + expect(compileFilter(makeOrder(), { status: "NOPE" })).toEqual({ + kind: "eq", column: "status", value: "NOPE", + }); + }); + + test("isNull carries no value to encode", () => { + expect(compileFilter(makeOrder(), { status: null })).toEqual({ + kind: "isNull", column: "status", not: false, + }); + }); +}); From 44bb77a04e68e273ae06c6e84a4744d9f458fc72 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Mon, 17 Aug 2026 06:59:56 -0400 Subject: [PATCH 52/52] docs(changelog): TS has TWO persistence seams, and only one had the codec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The entry claimed "Drizzle customType codecs (TS)" as if that were the whole TS story. It is the codegen half; runtime-ts's metadata-driven ObjectManager is a second seam with its own codec, and it was missing entirely — generated code worked while om.create() bound the symbol into an integer column. Naming both is the accurate claim, and the near-miss is worth stating: a port with two seams can ship one of them and look finished. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ed7865972..0f7245699 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,10 +38,13 @@ inherits the members *and* their mapping — and an own `@intValueMap` declared shared enum is rejected for the same reason an own `@values` is (#246's twin: one shared enum type has one mapping). -Persistence ships in every port: Drizzle `customType` codecs (TS), EF Core `HasConversion` -(C#), OMDB's `JdbcFieldCodec` (Java), Exposed `customEnumeration` (Kotlin) and -`ObjectManager` coercion (Python), gated cross-port by the `AllTypes` round-trip corpus -against real Postgres. +Persistence ships in every port: EF Core `HasConversion` (C#), OMDB's `JdbcFieldCodec` +(Java), Exposed `customEnumeration` (Kotlin), `ObjectManager` coercion (Python), and — in +TypeScript — **both** seams, since TS has two: a Drizzle `customType` for generated code and +`ObjectManager` read/write/filter coercion for the metadata-driven runtime. The second was +missing until the corpus caught it: generated code worked while `om.create()` bound the member +symbol straight into the integer column and Postgres rejected the statement. All gated +cross-port by the `AllTypes` round-trip corpus against real Postgres. Two decisions are worth naming because each closes a way the feature could have shipped half-true: