diff --git a/.claude/skills/releasing/SKILL.md b/.claude/skills/releasing/SKILL.md index 0347fc10e..082988f9e 100644 --- a/.claude/skills/releasing/SKILL.md +++ b/.claude/skills/releasing/SKILL.md @@ -66,6 +66,12 @@ done actually on npm (`npm view dist-tags`). Patch bump unless a public API changed; this repo bumps the whole set in lockstep (pre-1.0). +**Registry vocabulary does not force a MINOR** — a new *attribute* is a PATCH, a new +top-level *type* is a MINOR, and a new *subtype* is a PATCH when inert. See +`docs/RELEASING.md` → "The vocabulary rule" before choosing the level; the old +"any registry addition ⇒ MINOR" reading spent two minors on changes no adopter +could observe. + ## Phase 2 — Build fresh (the stale-`dist` trap) `dist/` is gitignored, `bun publish` does NOT rebuild, and `main` points at `dist/` diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 0b5e5bce0..71cc6713a 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -18,7 +18,9 @@ set -uo pipefail # Generic, non-sensitive structural patterns — safe to commit (name no project): -PATTERNS='/home/[A-Za-z0-9._-]+/|~/Development' +# /Users/ is the macOS home shape. It was missing until a plan doc authored on a Mac +# committed an absolute developer home path that this hook scanned and passed. +PATTERNS='/home/[A-Za-z0-9._-]+/|/Users/[A-Za-z0-9._-]+/|~/Development' # Legitimate matches to ignore (e.g. the published npm author email). ALLOW='doug@dougmealing\.com' diff --git a/CHANGELOG.md b/CHANGELOG.md index f7691de47..0f7245699 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,146 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ## [Unreleased] +### Added — int-backed `field.enum` storage via `@intValueMap` (all five ports) + +**This is a PATCH, not a MINOR** — and the reasoning is itself a change, so it is worth +stating. The old policy read "any registry addition ⇒ MINOR", which spent `0.22.0` and +`0.23.0` on changes a project could not observe at all. `expected-registry.json` is an +**internal** gate: five ports byte-matching one manifest is how we stop the ports drifting +from each other, and it says nothing about whether an adopter's project changes. Vocabulary +now sorts by what it can do to a consumer — a new **attribute** is a PATCH, a new top-level +**type** is a MINOR, and a new **subtype** is a PATCH when nothing but authoring it can reach +it. This line adds one attribute (`@intValueMap`) and one inert attr subtype (`attr.intMap`), +so: PATCH. See `docs/RELEASING.md` → "The vocabulary rule" and ADR-0035 Amendment 1. + +Purely additive on its own terms too: a `field.enum` that declares no `@intValueMap` is +byte-identical to before, string-backed as always. + +`@intValueMap` is an optional `{memberSymbol: int}` map on `field.enum` that declares each +member's **stored integer**. The column becomes `integer` with an integer `CHECK` instead of +`varchar` with a string one — while the wire format, the generated enum type, and every +runtime return value stay the **member symbol**, unchanged. The provenance is an +integer-coded column an adopter already has: the map is how you say "1 means LOW here", +rather than accepting whatever ordinal a language happens to assign. This is why it is a map +and not a second array parallel to `@values` — a positional array would silently re-map every +member the day someone reorders `@values`. + +The loader enforces the map's content identically everywhere: keys must equal `@values` +exactly (no missing, no extra), every value must be a 32-bit integer, and no two members may +share one. Both halves are read RESOLVING, so a field that `extends` a shared abstract enum +inherits the members *and* their mapping — and an own `@intValueMap` declared against a +shared enum is rejected for the same reason an own `@values` is (#246's twin: one shared enum +type has one mapping). + +Persistence ships in every port: EF Core `HasConversion` (C#), OMDB's `JdbcFieldCodec` +(Java), Exposed `customEnumeration` (Kotlin), `ObjectManager` coercion (Python), and — in +TypeScript — **both** seams, since TS has two: a Drizzle `customType` for generated code and +`ObjectManager` read/write/filter coercion for the metadata-driven runtime. The second was +missing until the corpus caught it: generated code worked while `om.create()` bound the member +symbol straight into the integer column and Postgres rejected the statement. All gated +cross-port by the `AllTypes` round-trip corpus against real Postgres. + +Two decisions are worth naming because each closes a way the feature could have shipped +half-true: + +- **Int-backing is scalar-only.** `@intValueMap` together with `isArray: true` is a load + error — `ERR_ENUM_INT_VALUE_MAP_ARRAY`, in every port. The original design said an + array-of-enum composed unchanged; it does not. Int-backing is a persistence-layer CODEC and + every port's codec seam is scalar by construction: Python bound the symbol LIST into an + `integer[]`, Java and Kotlin emitted a scalar codec, and TypeScript's sqlite branch + serialized the array as JSON text before the enum case was ever reached — storing symbols. + Only TS/Postgres and C# composed, and **two ports composing while four silently get it + wrong is not a feature** — it is the `field.byte`/`short`/`class` mistake, vocabulary that + reads as supported and is not. Rejecting it at LOAD delivers the guarantee that was + actually missing: identical behaviour in every port. An array-of-enum stays string-backed. +- **A stored integer that maps to no member THROWS on read**, in every port. The row holds + data the model says is impossible — a hand-written `INSERT`, or a member removed without a + migration — and neither alternative is honest: surfacing the raw integer hands the caller a + "member" that is not one, and is not even representable in C#, Kotlin or TypeScript, which + type the property as a closed enum; returning null hides the corruption behind a nullable + column. C# reaches this through a generated static helper called from the provider→model + lambda — CS8188 bans a throw-*expression* inside an expression tree, but a method CALL is + legal there. The WRITE side is deliberately left to the database: an unmapped symbol binds + unchanged, so the column type and its `CHECK` reject it. + +**Adopter-visible beyond the new attribute:** the filter-operator band is now decided +**per field**, not per subtype, so an int-backed `field.enum` no longer offers `like` — the +column holds integers, and `LIKE` against one is a type error, not a query. A projection's +`@filter` over an int-backed enum lowers to the integer literal rather than the symbol. + +Migration safety is unchanged and deliberate: adding or removing `@intValueMap` on a field +that already has a column is a cross-kind `change-column-type`, which `meta migrate` already +blocks by default and requires an explicit `allow.typeChange` to pass. There is no +auto-recast — the tool will not rewrite your data behind a metadata edit. + +**Do not RE-map a member's integer on a populated table.** Nothing understands that change: +the column holds bare integers, and neither introspection nor the committed snapshot records +which member an integer stood for. A remap changes the rendered `CHECK` list, so it trips the +blocked `drop-check` and `meta migrate` refuses — but that refusal is incidental (dropping a +`CHECK` is destructive), and once allowed the migration only refreshes the constraint and +never touches your rows. Swap two members' integers and the new `CHECK` admits the same set, +applies cleanly, and every stored row has quietly changed meaning. Reorder `@values` to +compensate and the diff is empty outright. Treat a remap as the same two-step backfill a +backing-mode change needs. + +Design: `docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`. Adopter view: +[`docs/features/field-types.md`](docs/features/field-types.md). + +### Changed — registry vocabulary no longer forces a MINOR (policy) + +`docs/RELEASING.md`'s versioning table said "PATCH (MINOR if it adds registry vocabulary — +cross-port conformance surface)", and ADR-0035's cadence bullet listed "a newly-supported +vocab member" among the MINOR triggers. Read literally, that made **any** registry addition a +MINOR — the exact churn ADR-0035 was written to prevent. `0.22.0` and `0.23.0` were both cut +MINOR for additions a project declaring no `requirement.*` nodes could not observe at all, +each changelog saying so in its own opening paragraph. Four registries move per cut here, so a +wasted minor is not free — and a minor spent on an unobservable change is a gate you no longer +have when something real needs it. Corrected to sort by consumer impact: **attribute ⇒ PATCH, +top-level type ⇒ MINOR, subtype ⇒ PATCH when inert** (nothing but authoring it can reach it; +MINOR when it narrows something previously permitted, changes existing metadata's meaning, or +headlines a release on purpose). Recorded as ADR-0035 Amendment 1; the post-1.0 compat promise +is untouched — a *breaking* vocabulary change still requires a MAJOR. + +### Fixed — an FK into a table whose key carries `@column` phantom-diffed forever (npm) + +`buildForeignKeys` resolved a target FK field's PHYSICAL column by applying the naming +strategy to its raw logical name, so a target primary key with an explicit `@column` override +(`id` → `"Id"`) made **every** foreign key into that table diff on every run — expected the +naming-strategy name, actual the override, nothing an adopter could do to converge. It now +resolves through the target entity's own field, which is how `fkCols` already handled the +source side; the two halves simply disagreed. + +### Fixed — views in a table-less schema were excluded from the diff entirely (npm) + +**Generated-output change — the first `meta migrate` after upgrading may emit view changes +that were always due.** `declaredSchemas` was built from `expected.tables` only, so a model +declaring views in a schema with no table of its own (an API/read-model schema beside an +all-`public` entity model) never brought that schema into scope — and a schema out of scope is +excluded from *both* sides of the diff. Its views were never compared, so a genuine missing or +extra view, or real drift inside an opaque `@sql` body, went undetected rather than reported. +View schemas now join the scope set. Same shape as 0.21.6's `ON DELETE` fix: a PATCH that +surfaces drift which was already there. + +### Fixed — a chained abstract `field.enum` emitted a broken Kotlin type (Maven) + +`KotlinTypeMapper.enumTypeName` named a chained abstract enum after the TOP-MOST root of the +`extends` chain (via `resolveSuperRoot`) while its own FR-019 arm resolved the shared +declaration from the IMMEDIATE super — the rule TS, C#, Java and Python all use. Not a rival +model, a split-brain: the two halves disagreed about which declaration is "the type", and on a +chained declaration (a root abstract `Money extends` a root abstract `@provided Currency`) +that produced a flatly broken emit. Naming now uses the immediate super per ADR-0026 §2 (a +materialized type is named for its own declaration), so a chain yields one type per +declaration, each carrying the members it inherits. Non-chained output is byte-identical — +with no further super, the root walk already returned the immediate super. `resolveSuperRoot` +had exactly one caller and is deleted. The chained alias stays LEGAL rather than being +rejected: it cannot mutate the vocabulary it inherits (a chained declaration carrying its own +`@values` — or its own `@intValueMap` — already errors `ERR_ENUM_EXTENDS_VALUES_CONFLICT`), +and banning it would carve an enum-only hole in ADR-0029's general `extends` grammar to delete +a provably harmless construct. Newly gated by `enum-abstract-chained-extends` (positive) and +`error-enum-chained-extends-values-conflict` (negative), plus the chained declaration restored +to the `shared-provided-enum` codegen corpus all five ports load — the decl-level #246 check +had been code-only in every port with no fixture behind it. + ## [0.23.1] — npm `0.23.1` · PyPI `0.23.1` · NuGet `0.23.1` · Maven `7.23.1` A coordinated **PATCH** across all four registries. Every one of them carries a real changed diff --git a/CLAUDE.md b/CLAUDE.md index d22e2a0d3..ee08e1f9c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,7 +50,7 @@ _Last refreshed 2026-08-16._ - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (271 fixtures; 19 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (286 fixtures; 19 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index ef6469062..09ae7efaa 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -25,7 +25,7 @@ regenerate with `ls -d fixtures//*/ | wc -l`. | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 271 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 286 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 15 | 15 / 15 | 14 / 15 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 14 / 15 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 15 / 15 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 31 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | @@ -69,7 +69,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. ## Fixture-to-doc mapping -### `fixtures/conformance/` — metamodel loader + canonical serializer (271) +### `fixtures/conformance/` — metamodel loader + canonical serializer (286) | Fixture prefix | Feature doc | |---|---| diff --git a/docs/RELEASING.md b/docs/RELEASING.md index 111dcdd79..3f962ade7 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -139,11 +139,49 @@ bytes**, not "did output change." | Public API **breaking** (required param, removed/renamed export, changed CLI semantics) | `relativeModuleSpecifier` +required param | **MINOR** | MAJOR | | Output change = **pure bugfix** (wrong output corrected; correct output byte-identical) | 0.19.3 payload naming, 0.19.4 TPH stamping | **PATCH** | PATCH | | Output change alters **shape/default of *correct* output** (renamed generated export, changed default, dropped artifact) | `extStyle` `"none"→"js"` default flip | **MINOR** + a "Generated-output change" changelog flag + an opt-out where feasible | MAJOR if consumer code referencing the output breaks; else MINOR | -| New **opt-in** codegen feature, default output byte-identical | new generator defaulting off | PATCH (MINOR if it adds registry vocabulary — cross-port conformance surface) | MINOR | +| New **opt-in** codegen feature, default output byte-identical | new generator defaulting off | PATCH | MINOR | +| New **attribute** on an existing type/subtype | `@intValueMap`, `@lenient`, `@maxTokens` | **PATCH** | MINOR | +| New **subtype** of an existing type | `field.uri`, `index.lookup`, `attr.intMap` | **PATCH** when inert (see the vocabulary rule below); **MINOR** when it changes existing metadata's meaning/output, narrows something previously permitted, or headlines a feature | MINOR | +| New top-level metadata **type** | `requirement.*`, `index.*`, `api.*` | **MINOR** | MINOR | | Wire-contract / conformance behavior change of already-valid deployments | FR-036 enforcement | **MINOR**, loud notice (pre-1.0 MINOR *is* the breaking slot) | MAJOR | | Wire behavior fixed to match the documented/conformance contract | 0.19.1 `@min` clamp | PATCH | PATCH | | No changed product file in a port | PyPI/NuGet/Maven at 0.20.14 | **Version-parity bump at the shared patch number** — publish identical content at the new version; never skip a registry (single-shared-patch policy, standing since 0.20.13) | same | +### The vocabulary rule (corrected 2026-08-17) + +**Adding registry vocabulary does NOT, by itself, force a MINOR.** The rule used to read +"PATCH (MINOR if it adds registry vocabulary — cross-port conformance surface)", and that was +wrong on its own terms: `expected-registry.json` is an **internal** gate. Every port +byte-matching one manifest is how we stop the five ports drifting from each other — it says +nothing about whether an *adopter's* project changes. Treating an internal gate's churn as an +adopter-facing event is what burned the minors: `0.22.0` and `0.23.0` were both cut MINOR for +changes that a project declaring no `requirement.*` nodes could not observe at all, which each +changelog says out loud in its own opening paragraph. Sort vocabulary by what it can do to a +consumer, which splits three ways: + +- **A new ATTRIBUTE is a PATCH.** You get it only by authoring it. Every existing document + loads unchanged and emits byte-identical output, so there is nothing for a consumer to adopt + deliberately. +- **A new TOP-LEVEL TYPE is a MINOR.** A type is a new modeling concept with its own children, + validation and (usually) tooling surface — `requirement.*` brought its own `verify` pass and + summary output. That is a thing a consumer newly depends on, and it deserves a deliberate + range bump. +- **A new SUBTYPE goes either way, and the test is whether it is INERT.** PATCH when nothing + but authoring it can reach it: no existing valid document changes meaning or output, nothing + previously permitted is narrowed, nothing reserved is consumed. MINOR when any of those + fails — a subtype that closes a wildcard, promotes a reserved-not-registered member + (ADR-0007 Amendment 2 / ADR-0040), or shifts what the recommended shape for an existing + field *is* — or when you deliberately want it behind a range bump because it headlines a + release. Most subtypes carry a native type and behavior (that is ADR-0037's very test for + making something a subtype), so read them carefully; but "it appears in the registry + manifest" is not the deciding fact. + +**Do not invert the caret rule.** "Pre-1.0 `^0.22.x` resolves `<0.23.0`, so a consumer adopts +a MINOR deliberately" is a reason to *choose* MINOR when you want that gate. It is not a +reason additive vocabulary *must* be MINOR. The gate exists to be used on purpose, not by +reflex — and a minor spent on a change nobody can observe is a gate you no longer have when +something real needs it. + **The `extStyle` 0.20.0 case, for calibration:** it was correctly MINOR — but for the *API break* (`relativeModuleSpecifier` gained a required param — a public export) **and** the *default flip* (churns every existing project's diff on regen), NOT because "output diff --git a/docs/features/field-types.md b/docs/features/field-types.md index b331d186c..beed69311 100644 --- a/docs/features/field-types.md +++ b/docs/features/field-types.md @@ -66,9 +66,26 @@ must be a non-empty set of unique members matching `^[A-Za-z_][A-Za-z0-9_]*$`. ``` The loader enforces members own-only and emits `ERR_BAD_ATTR_VALUE` on a bad -member or `ERR_MISSING_REQUIRED_ATTR` on missing `@values`. Int-backed enums, -display labels, and native Postgres `ENUM` types are deferred (see -[enum-datatype-design.md](../superpowers/specs/2026-05-23-enum-datatype-design.md)). +member or `ERR_MISSING_REQUIRED_ATTR` on missing `@values`. + +**Int-backed storage:** `@intValueMap` switches the DB column from string to integer while +preserving the string wire format and generated enum type. Keys must match `@values` exactly; +values must be unique integers. Display labels and native Postgres `ENUM` types remain +deferred (see [enum-datatype-design.md](../superpowers/specs/2026-05-23-enum-datatype-design.md)). + +Two rules follow from int-backing being a **persistence-layer codec**: + +- **It is scalar-only.** `@intValueMap` together with `isArray: true` is a load error, + `ERR_ENUM_INT_VALUE_MAP_ARRAY`, in every port — no port implements the codec element-wise + over an array column, so the combination would silently persist member *symbols* into an + integer array. An array-of-enum stays string-backed. +- **A stored integer that maps to no member throws on read**, in every port. The row holds + data the model says is impossible (a hand-written `INSERT`, or a member removed without a + migration); surfacing the raw integer would hand you a "member" that is not one — and is + not even representable in the ports that type the property as a closed enum — while + returning null would hide the corruption behind a nullable column. The write side is left + to the database: an unmapped symbol binds unchanged, so the column type and its `CHECK` + reject it. ### Sharing one enum — abstract `field.enum` + `extends` @@ -292,4 +309,4 @@ for the per-port pass/skip ledger. - [entities.md](entities.md) — host node `object.entity` - [relationships.md](relationships.md) — relationships are separate from fields, despite sharing the column space - [yaml-authoring.md](yaml-authoring.md) — array-suffix sugar for repeated fields (`field.long[]: weekIds`) -- [enum-datatype-design](../superpowers/specs/2026-05-23-enum-datatype-design.md) — enum design rationale + deferred capabilities +- [enum-datatype-design](../superpowers/specs/2026-05-23-enum-datatype-design.md) — enum design rationale + int-backed storage diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md new file mode 100644 index 000000000..d5cbf061f --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-csharp-persistence.md @@ -0,0 +1,269 @@ +# Int-Backed Enum Values — C# Persistence Implementation Plan + +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into C#'s EF Core codegen: emit a custom `HasConversion` lambda pair built from `@intValueMap`'s lookup table instead of the blanket `HasConversion()`, while the generated C# `enum` type itself is completely unchanged. C# never generates DDL (schema is TS-owned per ADR-0015) — this plan touches `DbContextGenerator.cs` only. + +**Architecture:** `DbContextGenerator.EmitFieldTypeConfig`'s existing `foreach (var f in fieldList.Where(f => f.SubType == FIELD_SUBTYPE_ENUM))` loop (the one that unconditionally emits `HasConversion()`) gets a conditional: when `f.Attr("intValueMap")` is present, emit a custom `.HasConversion(v => ..., v => ...)` pair instead — the exact inline-lambda shorthand this file already uses for `field.uri` (`.HasConversion(v => v!.ToString(), v => new System.Uri(v))`), just driven by a generated `Dictionary` literal rather than a fixed conversion. `EntityGenerator.CollectEnumDecls` (the C# `enum` type emitter) is **not touched at all** — confirmed no port's enum-type emitter reads `@intValueMap`, so the generated `enum Status { DRAFT, PUBLISHED, ARCHIVED }` declaration is byte-identical whether or not the field is int-backed. No `ValueConverter` class exists anywhere in this codebase today (confirmed via full-repo grep) — this plan introduces the inline-lambda form only, matching the file's own established idiom, not the standalone-class form. + +**Tech Stack:** C#, .NET, EF Core 8, xunit. + +## Global Constraints + +- The generated C# `enum` type is byte-identical between string- and int-backed fields — do not touch `EntityGenerator.CollectEnumDecls` or `EnumPropertyTypeName`. +- C# generates NO DDL. If a step here tempts you to write SQL/CHECK-constraint code, stop — that's TS's job (already done in the TS persistence plan). +- `@intValueMap`'s presence alone is the trigger — mirror the existing `HasConversion()` loop's unconditional style, just branching on attribute presence. +- Apply the identical conditional to BOTH loops that emit enum conversions today: the base/write-entity loop (`DbContextGenerator.cs` research lines 353-363) and the projection/read-model loop (lines 67-68). + +--- + +### Task 1: `FieldConstants.FIELD_ATTR_INT_VALUE_MAP` reader helper + +**Files:** +- Modify: `server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs` +- Test: `server/csharp/MetaObjects.Codegen.Tests/EnumIntValueMapConversionTests.cs` + +**Interfaces:** +- Consumes: `FieldConstants.FIELD_ATTR_INT_VALUE_MAP` (metamodel plan, already shipped — the constant lives in `server/csharp/MetaObjects/Core/Field/FieldConstants.cs`). +- Produces: a private static helper `TryGetIntValueMap(MetaField f, out IReadOnlyDictionary? map)` consumed by Tasks 2-3. + +- [ ] **Step 1: Write the failing test** + +```csharp +// server/csharp/MetaObjects.Codegen.Tests/EnumIntValueMapConversionTests.cs +using Xunit; +using MetaObjects.Loader; +using MetaObjects.Codegen.Generators; + +namespace MetaObjects.Codegen.Tests; + +public class EnumIntValueMapConversionTests +{ + private static MetaRoot LoadModel(string json) + { + var loader = new MetaDataLoader(); + var r = loader.Load(new[] { (IMetaDataSource)new InMemoryStringSource(json, "test.json") }); + Assert.Empty(r.Errors); + return r.Root; + } + + private const string Model = """ + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """; + + [Fact] + public void Int_backed_enum_emits_a_custom_HasConversion_lambda_pair_not_HasConversion_string() + { + var root = LoadModel(Model); + var output = new DbContextGenerator().Generate(new GenContext { Entities = new[] { root.FindObject("Order")! } }); + var contents = output.Single(f => f.Path.EndsWith("AppDbContext.g.cs")).Contents; + Assert.DoesNotContain("Property(x => x.Status).HasConversion()", contents); + Assert.Contains(".Property(x => x.Status).HasConversion(", contents); + } + + [Fact] + public void String_backed_enum_still_emits_HasConversion_string_unchanged() + { + var root = LoadModel(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """); + var output = new DbContextGenerator().Generate(new GenContext { Entities = new[] { root.FindObject("Order")! } }); + var contents = output.Single(f => f.Path.EndsWith("AppDbContext.g.cs")).Contents; + Assert.Contains("Property(x => x.Status).HasConversion()", contents); + } + + [Fact] + public void Generated_enum_type_declaration_is_identical_regardless_of_intValueMap() + { + var withMap = LoadModel(Model); + var withoutMap = LoadModel(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """); + var entityGen = new EntityGenerator(); + var out1 = entityGen.Generate(new GenContext { Entities = new[] { withMap.FindObject("Order")! } }); + var out2 = entityGen.Generate(new GenContext { Entities = new[] { withoutMap.FindObject("Order")! } }); + Assert.Equal( + out1.Single(f => f.Path.EndsWith("Order.g.cs")).Contents, + out2.Single(f => f.Path.EndsWith("Order.g.cs")).Contents); + } +} +``` + +> Match `GenContext`'s actual construction and `DbContextGenerator`/`EntityGenerator`'s actual `Generate` signature/output shape (`EmittedFile`-equivalent with `Path`/`Contents`) against an existing test like `EnumConformanceTests.cs` before finalizing — the sketch above follows that file's established pattern (confirmed via research) but adjust names/types to match exactly. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter EnumIntValueMapConversionTests` +Expected: FAIL — `Int_backed_enum_emits_a_custom_HasConversion_lambda_pair` fails (currently emits `HasConversion()` unconditionally); the other two pass already (nothing has changed yet, so they describe current/target-preserving behavior). + +- [ ] **Step 3: Add the reader helper to `DbContextGenerator.cs`** + +Edit `server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs` — add near the top of the class: + +```csharp + /// + /// Reads @intValueMap off a field.enum MetaField, if present. Object-shaped JSON + /// attrs parse to IReadOnlyDictionary<string, object?> with each number boxed + /// as long (DataConverter.ParseNumber: "integers are always long"). + /// + private static IReadOnlyDictionary? TryGetIntValueMap(MetaField f) + { + if (f.Attr(FieldConstants.FIELD_ATTR_INT_VALUE_MAP) is not IReadOnlyDictionary raw) + return null; + var result = new Dictionary(StringComparer.Ordinal); + foreach (var (key, value) in raw) + { + result[key] = value switch + { + long l => l, + int i => i, + _ => Convert.ToInt64(value), + }; + } + return result; + } +``` + +- [ ] **Step 4: Run tests** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter EnumIntValueMapConversionTests` +Expected: still FAIL on the same test (helper exists but isn't called yet) — this step only adds plumbing, verified by the build succeeding with no new test passing. + +- [ ] **Step 5: Commit** + +```bash +git add server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs server/csharp/MetaObjects.Codegen.Tests/EnumIntValueMapConversionTests.cs +git commit -m "feat(csharp): add TryGetIntValueMap reader helper to DbContextGenerator" +``` + +--- + +### Task 2: Custom `HasConversion` for the scalar + array enum loops + +**Files:** +- Modify: `server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs` + +**Interfaces:** +- Consumes: `TryGetIntValueMap` (Task 1). + +- [ ] **Step 1: Update the scalar-enum branch** + +Edit `EmitFieldTypeConfig`'s enum loop (research lines 353-363): + +```csharp + foreach (var f in fieldList.Where(f => f.SubType == FIELD_SUBTYPE_ENUM)) + { + var prop = CSharpNaming.Pascal(f.Name); + var intValueMap = TryGetIntValueMap(f); + // ADR-0039: resolving — array-ness inheritable via extends. Array-of-enum uses the + // EF Core 8 primitive collection with a per-element string conversion so members + // persist as symbols (["DRAFT"]), not int ordinals ([0]). + if (f.ResolvedIsArray()) + { + modelLines.Add($" modelBuilder.Entity<{className}>().PrimitiveCollection(x => x.{prop}).ElementType().HasConversion();"); + } + else if (intValueMap is not null) + { + var typeName = CSharpNaming.EnumTypeName(entity, f); + var toProvider = string.Join(", ", intValueMap.Select(kv => $"{{ {typeName}.{kv.Key}, {kv.Value} }}")); + var fromProvider = string.Join(", ", intValueMap.Select(kv => $"{{ {kv.Value}, {typeName}.{kv.Key} }}")); + modelLines.Add( + $" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasConversion(" + + $"v => new System.Collections.Generic.Dictionary<{typeName}, int> {{ {toProvider} }}[v], " + + $"v => new System.Collections.Generic.Dictionary {{ {fromProvider} }}[v]);"); + } + else + { + modelLines.Add($" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasConversion();"); + } + } +``` + +> Building a fresh `Dictionary` literal inline on every conversion call is wasteful at runtime (re-allocated per row) but matches this file's existing style of self-contained one-line lambda emission (see the `field.uri` branch, which similarly allocates a `new System.Uri(v)` per call). If this needs to be optimized later, emit the two dictionaries as `private static readonly` fields on the `DbContext` partial class instead of inline literals — flag this as a follow-up rather than doing it here, to keep this task's diff minimal and match the file's existing emission style. + +- [ ] **Step 2: Update the projection/read-model loop** + +Edit the sibling loop at research lines 67-68 with the identical conditional (extract the scalar-branch logic from Step 1 into a small shared private method `EmitEnumConversion(string className, string prop, MetaField f, MetaObject owner, List modelLines)` and call it from both loops, to avoid duplicating the dictionary-literal-building logic verbatim in two places). + +- [ ] **Step 3: Run tests — confirm all pass** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter EnumIntValueMapConversionTests` +Expected: PASS — all 3 tests green. + +- [ ] **Step 4: Run the full Codegen test suite** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests` +Expected: all pass, including the existing `EnumConformanceTests.cs` (string-backed enums unchanged) and the EF Core Roslyn compile-check suite (the generated `HasConversion` lambda must actually compile against the real EF Core API — this is the test that catches a syntax mistake in the emitted C#). + +- [ ] **Step 5: Commit** + +```bash +git add server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +git commit -m "feat(csharp): custom HasConversion lambda pair for int-backed field.enum" +``` + +--- + +### Task 3: EF Core Roslyn compile-check coverage + +**Files:** +- Modify: whatever fixture the "EF Core 8 + Roslyn-compiles the generated AppDbContext" test (per the original `field.enum` design doc's "Completed follow-ups") already uses. + +**Interfaces:** +- Consumes: Task 2. + +- [ ] **Step 1: Add an int-backed enum field to the shared EF-compile-check fixture model** + +Find the fixture model the existing "Roslyn-compiles the generated AppDbContext" test loads (per `docs/superpowers/specs/2026-05-23-enum-datatype-design.md`'s "Completed follow-ups" section — "a model exercising owned/jsonb/enum/enum-array/scalar-array/projection") and add a sibling int-backed enum field alongside the existing string-backed one. + +- [ ] **Step 2: Run the compile-check test** + +Run: `cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter ` +Expected: PASS — zero Roslyn compile errors against the real EF Core 8 API surface, confirming the emitted `HasConversion` lambda syntax is genuinely valid (not just string-matched by the earlier unit tests). + +- [ ] **Step 3: Run the full C# test suite** + +Run: `cd server/csharp && dotnet test` +Expected: 100% pass, no regressions. + +- [ ] **Step 4: Commit** + +```bash +git add -A +git commit -m "test(csharp): int-backed field.enum in the EF Core Roslyn compile-check fixture" +``` + +--- + +## After this plan lands + +C# generates no DDL and no persistence-conformance round-trip runner of its own for the query corpus beyond what the shared `roundtrip-all-types` scenario already drives through its generated + deployed API (per the api-contract-conformance "generated fan-out" lane described in this repo's CLAUDE.md). Once the TS persistence plan's `intEnumVal` field lands in `meta.fitness.json` (shared canonical model), re-run C#'s persistence-conformance and api-contract-conformance suites to confirm the new field round-trips through the generated EF Core stack with zero additional code — that's the real end-to-end proof this plan's `HasConversion` lambda works, beyond the unit-level Roslyn compile check in Task 3. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md new file mode 100644 index 000000000..13864c76a --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-java-kotlin-persistence.md @@ -0,0 +1,457 @@ +# Int-Backed Enum Values — Java + Kotlin Persistence Implementation Plan + +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into Java's OMDB JDBC persistence and Kotlin's Exposed table generation. Java's generated Java `enum` type and Kotlin's generated Kotlin `enum class` are completely unchanged — only the runtime codec (Java/OMDB) and the generated table-column DSL call (Kotlin/codegen-kotlin) differ. + +**Architecture:** Java's `EnumCodec` (`server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java`) is extended in place — not replaced — to check `f.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)` and branch between string bind/read (today's only behavior) and int bind/read (new), mirroring `TimestampCodec`'s existing `isLocalTime(MetaField)` per-instance attribute check (the established pattern for a single codec class handling two behaviors based on an attribute, rather than registering a second codec class). Kotlin's `KotlinExposedTableGenerator.kt` gets a new branch in its two `EnumField` checks: when `@intValueMap` is present, emit Exposed's `customEnumeration(...)` (free-form `fromDb`/`toDb` lambdas — confirmed the only Exposed API that supports an arbitrary, non-ordinal int-per-member mapping; `enumerationByName`/`enumeration` do not) instead of `enumerationByName(...)`, referencing a generated lookup map emitted as a shared per-package support file (mirroring how this generator already emits `emitInstantTzSupportFile`/`emitJsonbMapperSupportFile` for other non-trivial column types). + +**Tech Stack:** Java, Maven, JUnit — Kotlin, KotlinPoet, JUnit (via `codegen-kotlin`) — Exposed (via `integration-tests-kotlin`, Testcontainers Postgres). + +## Global Constraints + +- The generated Java `enum`/Kotlin `enum class` type declaration is byte-identical between string- and int-backed fields. +- `EnumCodec` is extended in place (Option A from research: mirrors `TimestampCodec`'s `isLocalTime` pattern) — do NOT register a second `JdbcFieldCodec` class keyed differently; `JdbcCodecs.forField` dispatches purely by `Class`, and `EnumField` already maps to one codec. +- Kotlin: do not attempt to use `enumeration(...)` (natural-ordinal-backed) — it cannot express an arbitrary, sparse int map. `customEnumeration(...)` is the only fit; confirm this against the actual Exposed API version pinned in this repo before finalizing (check `integration-tests-kotlin`'s `pom.xml`/build file for the Exposed version) since `customEnumeration`'s exact signature has shifted across Exposed major versions. + +--- + +### Task 1: Java — `EnumCodec` int-backed branch + +**Files:** +- Modify: `server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java` +- Test: `server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/EnumIntValueMapCodecTest.java` + +**Interfaces:** +- Consumes: `EnumField.ATTR_INT_VALUE_MAP` (metamodel plan, already shipped). +- Produces: `EnumCodec` handles both string- and int-backed persistence transparently — consumed by every OMDB call site (`ObjectManagerDB`, `GenericSQLDriver`, `SimpleMappingHandlerDB`) with zero changes to those call sites, since they all go through `JdbcCodecs.forField(f)`. + +- [ ] **Step 1: Write the failing tests** + +```java +// server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/EnumIntValueMapCodecTest.java +package com.metaobjects.manager.db.codec; + +import com.metaobjects.field.EnumField; +import com.metaobjects.loader.MetaDataLoader; +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; + +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.Types; + +import static org.junit.jupiter.api.Assertions.*; +import static org.mockito.Mockito.*; + +class EnumIntValueMapCodecTest { + + private static EnumField intBackedField() { + var loader = new MetaDataLoader(); + var r = loader.load(java.util.List.of(new com.metaobjects.loader.source.InMemoryMetaDataSource(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """, "test.json"))); + return (EnumField) r.getRoot().getMetaObjectByName("Order").getMetaField("status"); + } + + @Test + void write_binds_the_mapped_int_not_the_string() throws Exception { + var field = intBackedField(); + var ps = mock(PreparedStatement.class); + JdbcCodecs.forField(field).write(ps, field, 1, "PUBLISHED"); + verify(ps).setInt(1, 5); + verify(ps, never()).setString(anyInt(), any()); + } + + @Test + void write_binds_null_as_sql_integer_null() throws Exception { + var field = intBackedField(); + var ps = mock(PreparedStatement.class); + JdbcCodecs.forField(field).write(ps, field, 1, null); + verify(ps).setNull(1, Types.INTEGER); + } + + @Test + void read_decodes_the_int_back_to_its_symbol() throws Exception { + var field = intBackedField(); + var rs = mock(ResultSet.class); + when(rs.getInt(1)).thenReturn(9); + when(rs.wasNull()).thenReturn(false); + var target = new Object[1]; + // EnumField.setString presumably takes (Object target, String value) — adjust to + // whatever mock/target shape matches this codec's real readInto contract; the + // simplest verification is to capture via a tiny test double MetaField target. + JdbcCodecs.forField(field).readInto(target, field, rs, 1); + // Adjust assertion to whatever readInto actually does with `target` for a + // non-MetaObject target — check EnumCodec's CURRENT string-backed readInto test + // (if one exists) for the established assertion pattern before finalizing. + } + + @Test + void string_backed_enum_field_still_uses_setString_unchanged() throws Exception { + var loader = new MetaDataLoader(); + var r = loader.load(java.util.List.of(new com.metaobjects.loader.source.InMemoryMetaDataSource(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """, "test.json"))); + var field = (EnumField) r.getRoot().getMetaObjectByName("Order").getMetaField("status"); + var ps = mock(PreparedStatement.class); + JdbcCodecs.forField(field).write(ps, field, 1, "PUBLISHED"); + verify(ps).setString(1, "PUBLISHED"); + verify(ps, never()).setInt(anyInt(), anyInt()); + } + + @Test + void int_backed_enum_column_reports_INTEGER_sql_type_for_ddl_purposes() { + var field = intBackedField(); + assertEquals(Types.INTEGER, JdbcCodecs.forField(field).sqlType()); + } +} +``` + +> This test file's exact mocking approach (`readInto`'s `Object target` parameter shape) needs to match `EnumCodec`'s real contract against a genuine `MetaField.setString(Object, String)`-style API — check whether an existing `EnumCodec`/`CurrencyCodec` test already exists in this module and mirror its target/assertion pattern exactly; the sketch above may need adjusting once that's confirmed. `InMemoryMetaDataSource` is carried over from the metamodel plan's placeholder naming — confirm the actual class name. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/java && mvn -pl omdb test -Dtest=EnumIntValueMapCodecTest` +Expected: FAIL — `write_binds_the_mapped_int_not_the_string` and `sqlType` tests fail (current `EnumCodec` always calls `setString`/returns `NO_SQL_TYPE`). + +- [ ] **Step 3: Extend `EnumCodec`** + +Edit `server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java` — replace the existing `EnumCodec` (research lines 415-423) with: + +```java +static final class EnumCodec implements JdbcFieldCodec { + + @Override public void readInto(Object o, MetaField f, ResultSet rs, int j) throws SQLException { + var intMap = intValueMap(f); + if (intMap != null) { + int i = rs.getInt(j); + if (rs.wasNull()) { f.setString(o, null); return; } + f.setString(o, reverseLookup(intMap, i)); + } else { + f.setString(o, rs.getString(j)); + } + } + + @Override public void write(PreparedStatement s, MetaField f, int j, Object v) throws SQLException { + var intMap = intValueMap(f); + if (intMap != null) { + if (v == null) { s.setNull(j, Types.INTEGER); return; } + Integer i = intMap.get(v.toString()); + if (i == null) { + throw new IllegalStateException( + "field.enum '" + f.getName() + "' value '" + v + "' has no entry in @intValueMap"); + } + s.setInt(j, i); + } else { + if (v == null) s.setNull(j, Types.VARCHAR); + else s.setString(j, v.toString()); + } + } + + @Override public int sqlType() { + // NOTE: sqlType() has no MetaField parameter in the current JdbcFieldCodec + // interface (it's a per-CLASS, not per-INSTANCE, hook used by + // SimpleMappingHandlerDB purely for DDL length/type defaults) — but per + // ADR-0015 Java emits NO DDL at all (schema is TS-owned), so this return + // value is dead for @intValueMap's actual purpose; leave it NO_SQL_TYPE + // (deferring to the DataType switch) and rely on TS's migrate-ts for the + // real column type. Do not attempt to make this per-instance-aware. + return NO_SQL_TYPE; + } + + /** Own-only content-rule-validated map, per the metamodel plan — safe to trust shape here. */ + @SuppressWarnings("unchecked") + private static java.util.Map intValueMap(MetaField f) { + if (!f.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null; + return (java.util.Map) f.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).getValue(); + } + + private static String reverseLookup(java.util.Map intMap, int value) { + for (var e : intMap.entrySet()) if (e.getValue() == value) return e.getKey(); + throw new IllegalStateException("int value " + value + " has no matching @intValueMap entry"); + } +} +``` + +> `sqlType()`'s doc comment above flags a real design tension surfaced during this step: `JdbcFieldCodec.sqlType()` has no `MetaField` parameter (it's per-class, called without field context per the interface shown in research), so it CANNOT return `Types.INTEGER` only for int-backed instances even if Java wanted to emit DDL. Since Java emits no DDL at all (confirmed, ADR-0015), this is fine — but if a future change makes Java DDL-aware, `JdbcFieldCodec`'s interface would need a `sqlType(MetaField)` overload. Flagging, not fixing, since it's out of this plan's scope. + +- [ ] **Step 4: Run tests — confirm all pass** (adjust the `read_decodes...` test per Step 1's note once the real `readInto` contract is confirmed) + +Run: `cd server/java && mvn -pl omdb test -Dtest=EnumIntValueMapCodecTest` +Expected: PASS. + +- [ ] **Step 5: Run the full OMDB test suite** + +Run: `cd server/java && mvn -pl omdb test` +Expected: all pass, no regressions to existing string-backed enum persistence tests. + +- [ ] **Step 6: Commit** + +```bash +git add server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/EnumIntValueMapCodecTest.java +git commit -m "feat(java): EnumCodec binds/reads as int when field.enum carries @intValueMap" +``` + +--- + +### Task 2: Kotlin — `customEnumeration` for int-backed columns + +**Files:** +- Modify: `server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt` +- Test: `server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinEnumIntValueMapConformanceTest.kt` + +**Interfaces:** +- Consumes: `EnumField.ATTR_INT_VALUE_MAP`. + +- [ ] **Step 1: Write the failing test** + +```kotlin +// server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinEnumIntValueMapConformanceTest.kt +package com.metaobjects.generator.kotlin + +import com.metaobjects.loader.MetaDataLoader +import com.metaobjects.loader.source.InMemoryMetaDataSource +import org.junit.jupiter.api.Test +import kotlin.test.assertTrue +import kotlin.test.assertFalse + +class KotlinEnumIntValueMapConformanceTest { + + private val model = """ + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """ + + @Test + fun `int-backed enum column emits customEnumeration, not enumerationByName`() { + val loader = MetaDataLoader() + val result = loader.load(listOf(InMemoryMetaDataSource(model, "test.json"))) + val entity = result.root.getMetaObjectByName("Order") + val output = KotlinExposedTableGenerator().generate(entity, result.root) + assertTrue(output.contains("customEnumeration")) + assertFalse(output.contains("enumerationByName")) + } + + @Test + fun `string-backed enum column still emits enumerationByName unchanged`() { + val loader = MetaDataLoader() + val result = loader.load(listOf(InMemoryMetaDataSource(""" + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """, "test.json"))) + val entity = result.root.getMetaObjectByName("Order") + val output = KotlinExposedTableGenerator().generate(entity, result.root) + assertTrue(output.contains("enumerationByName")) + } +} +``` + +> `KotlinExposedTableGenerator`'s actual `generate(...)` entry point signature/return type is inferred from context (research showed its internals, not its public entry point) — check the existing `KotlinEnumConformanceTest.kt` (found during research) for the real call pattern and mirror it exactly. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/java && mvn -pl codegen-kotlin test -Dtest=KotlinEnumIntValueMapConformanceTest` +Expected: FAIL — `customEnumeration` never appears today. + +- [ ] **Step 3: Add a lookup-map support file emitter** + +Edit `KotlinExposedTableGenerator.kt` — add a helper mirroring the existing `emitInstantTzSupportFile`/`emitJsonbMapperSupportFile` pattern (research lines 287-338), emitting a small shared file per package holding the int↔symbol maps for every int-backed enum in that package: + +```kotlin +private fun emitEnumIntValueMapSupportFile(pkg: String, intBackedEnumFields: List>): String { + val entries = intBackedEnumFields.joinToString("\n\n") { (enumClassName, field) -> + val intValueMap = intValueMapOf(field) // helper added below + val toInt = intValueMap.entries.joinToString(", ") { (k, v) -> "$enumClassName.$k to $v" } + val fromInt = intValueMap.entries.joinToString(", ") { (k, v) -> "$v to $enumClassName.$k" } + """ + val ${enumClassName}_TO_INT: Map<$enumClassName, Int> = mapOf($toInt) + val ${enumClassName}_FROM_INT: Map = mapOf($fromInt) + """.trimIndent() + } + return """ + // + package $pkg + + $entries + """.trimIndent() +} + +@Suppress("UNCHECKED_CAST") +private fun intValueMapOf(field: EnumField): Map = + if (field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) + field.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).value as Map + else emptyMap() +``` + +- [ ] **Step 4: Update both `EnumField` branches to emit `customEnumeration`** + +Edit both occurrences shown in research (lines 580-589 and 616-620): + +```kotlin + val baseSpec = if (field is EnumField) { + val enumName = KotlinTypeMapper.enumTypeName(field, entity)?.simpleName + ?: error("enumTypeName returned null for EnumField '${field.name}' on ${entity.name}") + val colName = KotlinGenUtil.camelToSnake(field.name) + if (field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) { + // customEnumeration: free-form fromDb/toDb lambdas are the only Exposed + // API expressing an arbitrary, non-ordinal int-per-member map (confirmed — + // enumeration()/enumerationByName() cannot). Lambdas index into the + // per-package support-file maps (emitEnumIntValueMapSupportFile). + "customEnumeration(\"$colName\", fromDb = { ${enumName}_FROM_INT[it as Int]!! }, toDb = { ${enumName}_TO_INT[it]!! })" + } else { + "enumerationByName(\"$colName\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, $enumName::class)" + } + } else { + KotlinTypeMapper.exposedColumnSpec(field) + } +``` + +Apply the identical `if (field.hasMetaAttr(...))` branch to the TPH subtype-fields loop's copy (research lines 616-620). + +Wire `emitEnumIntValueMapSupportFile` into this generator's file-emission list (find wherever `emitInstantTzSupportFile`'s output gets added to the generator's returned file list, and add the new support file alongside it, once per package that has at least one int-backed enum field). + +> Confirm `customEnumeration`'s EXACT parameter names/order (`fromDb`/`toDb`, or possibly named differently, or requiring an explicit SQL column-type string as a THIRD parameter) against the Exposed version actually pinned in this repo's `pom.xml`/Gradle build for `integration-tests-kotlin` — Exposed's `customEnumeration` signature has varied across major versions; do not trust the sketch above verbatim. + +- [ ] **Step 5: Run tests — confirm all pass** + +Run: `cd server/java && mvn -pl codegen-kotlin test -Dtest=KotlinEnumIntValueMapConformanceTest` +Expected: PASS. + +- [ ] **Step 6: Run the full codegen-kotlin suite** + +Run: `cd server/java && mvn -pl codegen-kotlin test` +Expected: all pass, no regressions. + +- [ ] **Step 7: Commit** + +```bash +git add server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinEnumIntValueMapConformanceTest.kt +git commit -m "feat(kotlin): Exposed customEnumeration for int-backed field.enum columns" +``` + +--- + +### Task 3: Kotlin/Exposed real-engine round-trip (Testcontainers Postgres) + +**Files:** +- Modify: whatever fixture backs `EnumFilterControllerRunTest.kt` (research's strongest existing template for an end-to-end enum persistence test) or add a sibling test file. +- Test: `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/EnumIntValueMapRunTest.kt` + +**Interfaces:** +- Consumes: Task 2. + +- [ ] **Step 1: Write the real-engine test** + +```kotlin +// server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/EnumIntValueMapRunTest.kt +package com.metaobjects.integration.kotlin.api.generated + +import com.metaobjects.loader.MetaDataLoader +import com.metaobjects.loader.source.InMemoryMetaDataSource +import org.jetbrains.exposed.sql.Database +import org.jetbrains.exposed.sql.SchemaUtils +import org.jetbrains.exposed.sql.transactions.transaction +import org.junit.jupiter.api.Test +import org.testcontainers.containers.PostgreSQLContainer +import kotlin.test.assertEquals + +class EnumIntValueMapRunTest { + + private val model = """ + { "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}} + """ + + @Test + fun `int-backed enum column round-trips through real Postgres, storing the mapped int`() { + val pg = PostgreSQLContainer("postgres:16").apply { start() } + val db = Database.connect(pg.jdbcUrl, user = pg.username, password = pg.password) + val loader = MetaDataLoader() + val result = loader.load(listOf(InMemoryMetaDataSource(model, "test.json"))) + val entity = result.root.getMetaObjectByName("Order") + val generated = KotlinExposedTableGenerator().generate(entity, result.root) + // Compile + load the generated Table object dynamically, OR (simpler, matching + // this module's existing pattern per EnumFilterControllerRunTest.kt) drive this + // through the SAME generated-controller-over-HTTP harness that test already uses, + // rather than hand-rolling a raw Exposed table here — check that file's setup + // and mirror it exactly, since it already solves "compile generated Kotlin and + // run it against a real container" for the string-backed case. + transaction(db) { + // Insert a row with status = "PUBLISHED", read it back, assert: + // 1. The raw column value in the DB is the int 5 (query information_schema + // or SELECT status::int directly to prove it's really an INTEGER column). + // 2. The Kotlin data class field reads back as the STRING "PUBLISHED", not 5. + } + pg.stop() + } +} +``` + +> This test is intentionally left as a scaffold with the exact assertions commented rather than guessed — `EnumFilterControllerRunTest.kt` (confirmed in research to already compile+run a generated Spring controller against Exposed over Testcontainers Postgres for the string-backed case) is the concrete template to copy and adapt; read it in full before writing this test for real, since it already solves the hard parts (dynamic compilation of generated code, container lifecycle, HTTP round-trip) that this sketch only gestures at. + +- [ ] **Step 2: Run to verify current failure/gap** + +Run: `cd server/java && mvn -pl integration-tests-kotlin test -Dtest=EnumIntValueMapRunTest` +Expected: FAIL or does not compile, until Step 1 is completed for real against the `EnumFilterControllerRunTest.kt` template. + +- [ ] **Step 3: Complete the test for real, run, confirm pass** + +Run: `cd server/java && mvn -pl integration-tests-kotlin test -Dtest=EnumIntValueMapRunTest` +Expected: PASS. + +- [ ] **Step 4: Run the full integration-tests-kotlin suite** + +Run: `cd server/java && mvn -pl integration-tests-kotlin test` +Expected: all pass, no regressions. + +- [ ] **Step 5: Commit** + +```bash +git add server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/generated/EnumIntValueMapRunTest.kt +git commit -m "test(kotlin): int-backed field.enum round-trips through real Postgres via Exposed" +``` + +--- + +## After this plan lands + +Java has no DDL and no ORM-config generator of its own analogous to C#'s `DbContextGenerator` — OMDB is pure data-access (per CLAUDE.md: "OMDB is pure data-access — CRUD/query/codec/transactions only"), so Task 1's codec change is Java's entire persistence-layer footprint. Once the shared `roundtrip-all-types` persistence-conformance scenario gets its `intEnumVal` field (added in the TS persistence plan), re-run Java's own persistence-conformance suite to confirm the codec round-trips correctly end-to-end, not just in the unit-level codec test from Task 1. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md new file mode 100644 index 000000000..091fb307a --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md @@ -0,0 +1,1410 @@ +# Int-Backed Enum Values — Metamodel Implementation Plan + +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add the `@intValueMap` attribute to `field.enum` — an optional `{memberSymbol: int}` map that, when present, is the metadata author's declaration of each member's stored integer — across all five ports (TypeScript, C#, Java, Python, Kotlin-via-Java), gated by load-time validation and the `registry-conformance` + `fixtures/conformance/` corpora. This plan covers **vocabulary + validation + conformance only** — it does NOT touch codegen (already proven unchanged, since no port's enum-type emitter reads `@intValueMap`) or persistence (DB DDL, EF Core/JDBC/Exposed/ObjectManager codecs, migrate-ts's migration-safety guard). Those are covered by follow-on plans, one per port/group, written after this one lands. + +**Architecture:** Each port already has an identical three-layer structure for `field.enum`'s existing `@values` attribute: (1) a generic attr-value-type class/registration (`properties`, `string[]`, etc.) that enforces the attr's basic shape, (2) a field.enum-specific content-rule validation pass that enforces the enum's own semantics (non-empty, identifier pattern, no duplicates) on top of that shape, (3) a shared cross-port conformance fixture set + a shared `registry-conformance` manifest that every port's test suite asserts against byte-for-byte. `@intValueMap` follows the exact same three-layer shape, with one port-specific correction: Java's existing `properties` attr class is backed by `java.util.Properties`, which silently coerces every value to a `String` on load (confirmed by reading `PropertiesAttribute.java:51-60` — every `Map` entry is written via `.getValue().toString()`). Reusing it for `@intValueMap` would silently turn `0` into `"0"` internally in Java, and — because canonical-JSON round-trip serialization must be byte-identical across ports (the `fixtures/conformance/*/expected.json` gate) — would make Java re-emit an integer value as a quoted JSON string, diverging from TS/C#/Python. So every port registers a **new**, cross-port-identical attr subtype, `intMap` (a generic "object with all-integer values" shape, parallel to how `properties` is a generic "any object" shape), and Java backs it with a new `Map`-typed class instead of reusing `PropertiesAttribute`. `field.enum`'s own content-rule pass then layers the enum-specific rules on top: `@intValueMap`'s key set must exactly equal `@values`' members, and no two members may share a stored int (reusing the existing `ERR_BAD_ATTR_VALUE` code everywhere — no new error code needed, confirmed by inspecting all four ports' error-code ledgers). + +**Tech Stack:** TypeScript (Bun test runner), C# (.NET, xunit), Java (Maven, JUnit) — also covers Kotlin, which shares Java's metadata layer — Python (pytest). + +## Global Constraints + +- Cross-language contract (from the design spec, `docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`): attribute name is `intValueMap` (canonical JSON `@intValueMap`); value shape is an object with string keys (member symbols) and integer values; every loader enforces identically: (a) key set exactly equals `@values`' members (no missing, no extra), (b) every value is an integer, (c) no two members share a value. +- No new `@kind`. Presence of `@intValueMap` alone is the only signal — do not add a discriminator attribute. +- `@values` is completely untouched — do not modify its required-ness, type, or any existing validation for it. +- Reuse `ERR_BAD_ATTR_VALUE` for every `@intValueMap` content violation (confirmed: no port has a dedicated enum error code today — all enum content violations already reuse this code). +- Every new attr/field-attr registration needs a `registry-conformance` fixture entry (ADR-0023) — `fixtures/registry-conformance/expected-registry.json` is the ONE shared file all five ports assert against. +- `fixtures/conformance/` fixtures are shared JSON, consumed by every port's own test runner — write them once, verify per-port. +- Do NOT touch codegen or persistence in this plan. If a task tempts you to edit a `*Generator.cs`/`*.ts` codegen template, a DDL emitter, or an ORM config generator, stop — that belongs to a follow-on plan. + +--- + +### Task 1: TypeScript — generic `attr.intMap` subtype + +**Files:** +- Modify: `spec/metamodel/attr.json` +- Create: `server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts` +- Modify: `server/typescript/packages/metadata/src/core/attr/attr-constants.ts` +- Modify: `server/typescript/packages/metadata/src/core-types.ts` +- Test: `server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts` + +**Interfaces:** +- Produces: `ATTR_SUBTYPE_INT_MAP = "intMap"` (exported from `attr-constants.ts`), the `IntMapAttr` class (registered against that subtype), and the `attr.intMap` type declaration in the canonical spec — all consumed by Task 2. + +- [ ] **Step 1: Write the failing test** + +```typescript +// server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts +import { describe, test, expect } from "bun:test"; +import { IntMapAttr } from "../../../src/core/attr/meta-attr-int-map.js"; + +describe("IntMapAttr", () => { + test("accepts a plain object with integer values", () => { + const attr = new IntMapAttr("intValueMap"); + attr.setValue({ DRAFT: 0, PUBLISHED: 5 }); + expect(attr.validateValue(attr.getValue())).toEqual([]); + }); + + test("rejects a non-object value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue("not-an-object" as unknown as object); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("must be of type 'intMap'"); + }); + + test("rejects an array value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue([0, 1] as unknown as object); + expect(errors.length).toBe(1); + }); + + test("rejects a non-integer value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue({ DRAFT: "0" } as unknown as object); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("DRAFT"); + }); + + test("rejects a float value", () => { + const attr = new IntMapAttr("intValueMap"); + const errors = attr.validateValue({ DRAFT: 0.5 } as unknown as object); + expect(errors.length).toBe(1); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd server/typescript && bun test packages/metadata/test/core/attr/meta-attr-int-map.test.ts` +Expected: FAIL — `Cannot find module '../../../src/core/attr/meta-attr-int-map.js'` + +- [ ] **Step 3: Add `ATTR_SUBTYPE_INT_MAP` to `attr-constants.ts`** + +Edit `server/typescript/packages/metadata/src/core/attr/attr-constants.ts` — add the constant next to `ATTR_SUBTYPE_EXPRESSION` (line 19) and into the `ATTR_SUBTYPES` array (line 33-44): + +```typescript +export const ATTR_SUBTYPE_EXPRESSION = "expression"; +// An object-shaped attr whose values are all integers (e.g. field.enum's +// @intValueMap: {memberSymbol: int}). Generic shape check only — semantic +// rules specific to a consumer (key-set membership, uniqueness) are that +// consumer's own content-rule validation, not this attr's. +export const ATTR_SUBTYPE_INT_MAP = "intMap"; +``` + +```typescript +export const ATTR_SUBTYPES = [ + SUBTYPE_BASE, + ATTR_SUBTYPE_STRING, + ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_LONG, + ATTR_SUBTYPE_DOUBLE, + ATTR_SUBTYPE_BOOLEAN, + ATTR_SUBTYPE_CLASS, + ATTR_SUBTYPE_PROPERTIES, + ATTR_SUBTYPE_FILTER, + ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, +] as const; +``` + +- [ ] **Step 4: Write `meta-attr-int-map.ts`** + +```typescript +// IntMapAttr — attr subtype `intMap`. Object-shaped value whose members must +// all be integers (e.g. field.enum's @intValueMap). No desugar; validates +// shape (object, not array) and every value's type (integer). A consumer's +// own semantic rules (key-set membership, uniqueness) are validated by that +// consumer, not here — mirrors how StringArrayAttr validates shape while +// field.enum's own content-rule pass validates its @values semantics. + +import { MetaAttr, type ValueError, runtimeTypeName } from "./meta-attr.js"; +import { type AttrValue } from "../../shared/meta-data.js"; +import { DATA_TYPE_OBJECT, type DataType } from "../../data-type.js"; +import { registerAttrClass } from "../../attr-class-map.js"; +import { ATTR_SUBTYPE_INT_MAP } from "./attr-constants.js"; + +export class IntMapAttr extends MetaAttr { + override get dataType(): DataType { + return DATA_TYPE_OBJECT; + } + + override coerce(raw: unknown): AttrValue { + return raw as AttrValue; + } + + override validateValue(value: AttrValue): ValueError[] { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return [{ message: `attribute '@${this.name}' must be of type 'intMap' but got ${runtimeTypeName(value)}` }]; + } + const errors: ValueError[] = []; + for (const [key, member] of Object.entries(value as Record)) { + if (typeof member !== "number" || !Number.isInteger(member)) { + errors.push({ + message: `attribute '@${this.name}' member '${key}' has value '${String(member)}' which is not an integer`, + }); + } + } + return errors; + } +} + +registerAttrClass(ATTR_SUBTYPE_INT_MAP, IntMapAttr); +``` + +- [ ] **Step 5: Import the new module for its registration side-effect** + +Edit `server/typescript/packages/metadata/src/core-types.ts` — add alongside the existing sibling imports (line 20-22): + +```typescript +import "./core/attr/meta-attr-filter.js"; +import "./core/attr/meta-attr-properties.js"; +import "./core/attr/meta-attr-expression.js"; +import "./core/attr/meta-attr-int-map.js"; +``` + +- [ ] **Step 6: Add the `attr.intMap` type declaration to the canonical spec** + +Edit `spec/metamodel/attr.json` — insert alphabetically between the `int` and `long` subtype blocks: + +```json + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, +``` + +- [ ] **Step 7: Regenerate the embedded attr definition** + +Run: `cd && bun scripts/generate-embedded-metamodel.ts` +Expected: regenerates `server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts` to include the new `intMap` block. + +- [ ] **Step 8: Run the test to verify it passes** + +Run: `cd server/typescript && bun test packages/metadata/test/core/attr/meta-attr-int-map.test.ts` +Expected: PASS — all 5 tests green. + +- [ ] **Step 9: Run the full metadata test suite to check for regressions** + +Run: `cd server/typescript && bun test packages/metadata` +Expected: all existing tests still pass (this step only adds a new subtype; nothing existing should change behavior). + +- [ ] **Step 10: Commit** + +```bash +git add spec/metamodel/attr.json server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts server/typescript/packages/metadata/src/core/attr/attr-constants.ts server/typescript/packages/metadata/src/core-types.ts server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts +git commit -m "feat(metadata): add attr.intMap — a generic object-with-integer-values attr subtype" +``` + +--- + +### Task 2: TypeScript — `field.enum`'s `@intValueMap` attribute + +**Files:** +- Modify: `server/typescript/packages/metadata/src/core/field/field-constants.ts` +- Modify: `spec/metamodel/field.json` +- Modify: `server/typescript/packages/metadata/src/attr-schema-validate.ts` +- Test: `server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: `ATTR_SUBTYPE_INT_MAP` (Task 1). +- Produces: `FIELD_ATTR_INT_VALUE_MAP = "intValueMap"` — consumed by every later port task and by the persistence follow-on plans. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader.js"; +import { InMemoryStringSource } from "../src/sources/in-memory-string-source.js"; + +async function load(json: string) { + const loader = new MetaDataLoader(); + return loader.load([new InMemoryStringSource(json, "test.json")]); +} + +const base = (extra: string) => `{ + "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] ${extra} } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]} +}`; + +describe("field.enum @intValueMap content rules", () => { + test("accepts a valid map — key set matches @values, unique ints", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors).toEqual([]); + }); + + test("field.enum with no @intValueMap is still valid (string-backed default)", async () => { + const result = await load(base("")); + expect(result.errors).toEqual([]); + }); + + test("rejects a missing member key", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("ARCHIVED"); + }); + + test("rejects an extra key not in @values", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("RETRACTED"); + }); + + test("rejects a non-integer value", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + }); + + test("rejects two members sharing the same int", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect(result.errors[0]?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("DRAFT"); + expect(result.errors[0]?.message).toContain("PUBLISHED"); + }); +}); +``` + +> Adjust the exact `MetaDataLoader`/`InMemoryStringSource` import paths and the shape of `result.errors` (some loader APIs throw on the first error rather than returning an array) to match this package's actual loader test harness — check an existing test like `test/attr-schema-validate.test.ts` or `enum-inline`'s consuming test for the established pattern before finalizing this step. + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd server/typescript && bun test packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` +Expected: FAIL — the "accepts a valid map" test fails with `ERR_UNKNOWN_ATTR` (strict provenance, Task 1 registered the generic subtype but `field.enum` doesn't yet accept `@intValueMap`). + +- [ ] **Step 3: Add `FIELD_ATTR_INT_VALUE_MAP` constant** + +Edit `server/typescript/packages/metadata/src/core/field/field-constants.ts` — add after `FIELD_ATTR_VALUES` (line 160), inside the existing "Enum attrs" section (line 155-157): + +```typescript +/** Member symbols of an enum-subtype field. Required, string array. */ +export const FIELD_ATTR_VALUES = "values"; + +/** + * Optional per-member explicit integer value ({memberSymbol: int}) switching + * this enum field's DB persistence from string+CHECK to integer+CHECK. Keys + * must exactly match @values; values must be unique integers. The generated + * native type and wire format are UNCHANGED in every language — this is a + * persistence-layer-only concern (docs/superpowers/specs/2026-07-23-int-backed- + * enum-values-design.md). + */ +export const FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; +``` + +- [ ] **Step 4: Add the attr declaration to the canonical field spec** + +Edit `spec/metamodel/field.json` — add as a sibling of `values`/`provided` inside `field.enum`'s `children` array: + +```json + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } +``` + +- [ ] **Step 5: Regenerate the embedded field definition** + +Run: `cd && bun scripts/generate-embedded-metamodel.ts` +Expected: regenerates `field-definition.embedded.ts`. + +- [ ] **Step 6: Run tests again — confirm the ERR_UNKNOWN_ATTR failure is gone, new failures are the content-rule assertions** + +Run: `cd server/typescript && bun test packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` +Expected: "accepts a valid map" and "no @intValueMap" pass; the four negative tests FAIL (no content-rule validation exists yet, so `result.errors` is empty when it should have entries). + +- [ ] **Step 7: Add the content-rule validation** + +Edit `server/typescript/packages/metadata/src/attr-schema-validate.ts` — add immediately after Check 5 (FR-011, ends around line 372), inside the same `if (node.type === TYPE_FIELD && node.subType === FIELD_SUBTYPE_ENUM)` gate: + +```typescript + // --- Check 5b: field.enum @intValueMap content rules --- + // + // Optional. Own-only (mirrors Checks 4/5's own-attrs-only policy) — an + // inherited @intValueMap is validated on its declaring node. The generic + // "is this an object of integers" shape check already ran via IntMapAttr + // (attr subtype `intMap`); this validates the field.enum-SPECIFIC + // semantics: key-set-equals-@values, and no two members share a value. + const rawIntValueMap = node.ownAttrs().get(FIELD_ATTR_INT_VALUE_MAP); + if (rawIntValueMap !== undefined && typeof rawIntValueMap === "object" && rawIntValueMap !== null) { + const map = rawIntValueMap as Record; + const effectiveValues = node.attrs().get(FIELD_ATTR_VALUES); + const declaredMembers: string[] = Array.isArray(effectiveValues) ? effectiveValues : []; + const memberSet = new Set(declaredMembers); + const mapKeys = Object.keys(map); + const keySet = new Set(mapKeys); + + const missing = declaredMembers.filter((m) => !keySet.has(m)); + const extra = mapKeys.filter((k) => !memberSet.has(k)); + if (missing.length > 0 || extra.length > 0) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@${FIELD_ATTR_VALUES}' members` + + (missing.length > 0 ? ` (missing: ${missing.join(", ")})` : "") + + (extra.length > 0 ? ` (unknown: ${extra.join(", ")})` : "") + ".", + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } + + const seenValues = new Map(); + for (const [member, value] of Object.entries(map)) { + if (typeof value !== "number" || !Number.isInteger(value)) continue; // IntMapAttr already reported this + const owner = seenValues.get(value); + if (owner !== undefined) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' members '${owner}' and '${member}' ` + + `share the same value ${value}; every member must have a unique int.`, + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } else { + seenValues.set(value, member); + } + } + } +``` + +Add `FIELD_ATTR_INT_VALUE_MAP` to this file's existing import block from `@metaobjectsdev/metadata`'s field-constants (or the local relative import this file already uses for `FIELD_ATTR_VALUES` — match its existing import style). + +- [ ] **Step 8: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts` +Expected: PASS — all 6 tests green. + +- [ ] **Step 9: Run the full metadata test suite** + +Run: `cd server/typescript && bun test packages/metadata` +Expected: all pass, no regressions. + +- [ ] **Step 10: Commit** + +```bash +git add server/typescript/packages/metadata/src/core/field/field-constants.ts spec/metamodel/field.json server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts server/typescript/packages/metadata/src/attr-schema-validate.ts server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts +git commit -m "feat(metadata): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 3: Shared conformance fixtures (`fixtures/conformance/`) + +**Files:** +- Create: `fixtures/conformance/enum-int-backed/input/meta.enums.json` +- Create: `fixtures/conformance/enum-int-backed/expected.json` +- Create: `fixtures/conformance/enum-int-backed-array/input/meta.enums.json` +- Create: `fixtures/conformance/enum-int-backed-array/expected.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-non-int/expected.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json` +- Create: `fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected.json` + +**Interfaces:** +- Consumes: `@intValueMap` (Task 2), plus this repo's existing conformance fixture format (see `fixtures/conformance/enum-inline/` for the reference shape). +- Produces: five fixtures every port's own conformance test runner discovers and asserts against (Tasks 5, 7, 9, 11). + +- [ ] **Step 1: Create `enum-int-backed` (positive)** + +`fixtures/conformance/enum-int-backed/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`fixtures/conformance/enum-int-backed/expected.json` (the same document with shorthand `@fields` expanded to an array, matching how `enum-inline/expected.json` normalizes it — verify the exact expansion by diffing against `enum-inline/expected.json`'s own `@fields` line before finalizing): +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}} +]}} +``` + +- [ ] **Step 2: Create `enum-int-backed-array` (positive, array-of-enum)** + +`fixtures/conformance/enum-int-backed-array/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "labels", "isArray": true, "@values": ["LOW","MEDIUM","HIGH"], "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`fixtures/conformance/enum-int-backed-array/expected.json` — same document with `@fields` expanded (mirror Step 1's normalization). + +- [ ] **Step 3: Create `error-enum-intvaluemap-key-mismatch` (negative)** + +`fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected.json` — match the negative-fixture format used by `error-enum-empty-values/expected.json` (an error-code envelope, not a normalized document — read that file first and mirror its exact shape): +```json +{ "errors": [ { "code": "ERR_BAD_ATTR_VALUE" } ] } +``` + +- [ ] **Step 4: Create `error-enum-intvaluemap-non-int` (negative)** + +`fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": "zero", "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`expected.json`: `{ "errors": [ { "code": "ERR_BAD_ATTR_VALUE" } ] }` + +- [ ] **Step 5: Create `error-enum-intvaluemap-duplicate-value` (negative)** + +`fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json`: +```json +{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 0 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ]}} +]}} +``` + +`expected.json`: `{ "errors": [ { "code": "ERR_BAD_ATTR_VALUE" } ] }` + +- [ ] **Step 6: Run TS's conformance fixture runner to sanity-check the fixtures parse (does not yet confirm cross-port correctness — that's Tasks 5/7/9/11)** + +Run: `cd server/typescript && bun test packages/metadata -t conformance` +Expected: the two new positive fixtures pass; the three negative fixtures pass (each correctly produces exactly the expected error code) — since Task 2 already implemented the validation, all five should be green already at this step. + +- [ ] **Step 7: Commit** + +```bash +git add fixtures/conformance/enum-int-backed fixtures/conformance/enum-int-backed-array fixtures/conformance/error-enum-intvaluemap-key-mismatch fixtures/conformance/error-enum-intvaluemap-non-int fixtures/conformance/error-enum-intvaluemap-duplicate-value +git commit -m "test(conformance): add cross-port fixtures for field.enum @intValueMap" +``` + +--- + +### Task 4: Shared — `registry-conformance/expected-registry.json` + +**Files:** +- Modify: `fixtures/registry-conformance/expected-registry.json` + +**Interfaces:** +- Consumes: Tasks 1-2's TS registration (the manifest is generated from TS's live registry and hand-verified, then used as the golden file every port compares against). + +- [ ] **Step 1: Add the new `attr.intMap` type entry** + +Edit `fixtures/registry-conformance/expected-registry.json` — insert alphabetically between the `int` (ends line 52) and `long` (starts line 53) attr type blocks: + +```json + { + "type": "attr", + "subType": "intMap", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation.", + "attrs": [], + "children": [] + }, +``` + +- [ ] **Step 2: Add the `intValueMap` attr entry to `field.enum`'s `attrs` array** + +Edit the same file — inside the `field`/`enum` block (starts line 929), insert alphabetically among the existing `attrs` entries (after `formExclude`, before whatever sorts after `intValueMap` alphabetically — read the surrounding entries first to place it exactly): + +```json + { + "name": "intValueMap", + "valueType": "intMap", + "isArray": false, + "required": false, + "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." + }, +``` + +- [ ] **Step 3: Run TS's registry-conformance test to confirm the manifest now matches TS's live registry** + +Run: `cd server/typescript && bun test packages/metadata -t registry-conformance` +Expected: PASS. If it fails on an unrelated diff (ordering, wording), adjust this file's new entries — not TS's source — to match, since this file must byte-match whatever TS's registry generator actually emits. + +- [ ] **Step 4: Commit** + +```bash +git add fixtures/registry-conformance/expected-registry.json +git commit -m "test(registry-conformance): register field.enum @intValueMap (attr.intMap)" +``` + +--- + +### Task 5: TypeScript — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the full TS metadata suite** + +Run: `cd server/typescript && bun test packages/metadata` +Expected: 100% pass, zero regressions. + +- [ ] **Step 2: Run the full server TS suite as a broader regression check** + +Run: `cd server/typescript && bun test` +Expected: 100% pass (this also exercises `codegen-ts` and `migrate-ts`, which per this plan's scope should show ZERO behavior change — if any codegen or migrate-ts test changes behavior here, that's a signal `@intValueMap` leaked into a place it shouldn't have at this stage; investigate before proceeding). + +- [ ] **Step 3: Typecheck** + +Run: `cd server/typescript && bun run --filter '@metaobjectsdev/metadata' typecheck` +Expected: no new errors introduced by this plan's changes (pre-existing unrelated errors, if any, are out of scope). + +--- + +### Task 6: C# — `attr.intMap` subtype + `field.enum`'s `@intValueMap` + +**Files:** +- Modify: `server/csharp/MetaObjects/Core/Attr/AttrConstants.cs` +- Modify: `server/csharp/MetaObjects/CoreTypes.cs` +- Modify: `server/csharp/MetaObjects/Core/Field/FieldConstants.cs` +- Modify: `server/csharp/MetaObjects/Core/Field/FieldSchema.cs` +- Modify: `server/csharp/MetaObjects/Loader/ValidationPasses.cs` +- Modify: `server/csharp/MetaObjects/SpecMetamodel/attr.json` +- Modify: `server/csharp/MetaObjects/SpecMetamodel/field.json` +- Test: `server/csharp/MetaObjects.Tests/EnumIntValueMapTests.cs` + +**Interfaces:** +- Produces: `FieldConstants.FIELD_ATTR_INT_VALUE_MAP`, `AttrConstants.ATTR_SUBTYPE_INT_MAP` — no other task in this plan consumes them, but the C# persistence follow-on plan will. + +- [ ] **Step 1: Write the failing tests** + +```csharp +// server/csharp/MetaObjects.Tests/EnumIntValueMapTests.cs +using Xunit; +using MetaObjects.Loader; + +namespace MetaObjects.Tests; + +public class EnumIntValueMapTests +{ + private static string Model(string extra) => $$""" + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {{extra}} } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """; + + private static (bool Ok, System.Collections.Generic.IReadOnlyList Errors) TryLoad(string json) + { + var loader = new MetaDataLoader(); + var source = new InMemoryStringSource(json, "test.json"); + var result = loader.Load(new[] { (IMetaDataSource)source }); + return (result.Errors.Count == 0, result.Errors); + } + + [Fact] + public void Valid_intValueMap_with_matching_keys_and_unique_ints_loads_clean() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.True(ok, string.Join("; ", errors)); + } + + [Fact] + public void No_intValueMap_still_loads_clean_string_backed_default() + { + var (ok, _) = TryLoad(Model("")); + Assert.True(ok); + } + + [Fact] + public void Missing_member_key_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("ARCHIVED")); + } + + [Fact] + public void Extra_key_not_in_values_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("RETRACTED")); + } + + [Fact] + public void Non_integer_value_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE); + } + + [Fact] + public void Duplicate_int_value_across_members_is_rejected() + { + var (ok, errors) = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}""")); + Assert.False(ok); + Assert.Contains(errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE + && e.Message.Contains("DRAFT") && e.Message.Contains("PUBLISHED")); + } +} +``` + +> Adjust `InMemoryStringSource`/`MetaDataLoader.Load` call shape and the `Errors`/`MetaError` field names to match this codebase's actual API (check an existing test in `MetaObjects.Tests` that loads inline JSON and asserts on errors, e.g. a test near `EnumFieldTest`-equivalent, before finalizing). + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter EnumIntValueMapTests` +Expected: FAIL — `ERR_UNKNOWN_ATTR` on the positive test (attr not yet registered). + +- [ ] **Step 3: Add `ATTR_SUBTYPE_INT_MAP` constant** + +Edit `server/csharp/MetaObjects/Core/Attr/AttrConstants.cs` — add next to `ATTR_SUBTYPE_PROPERTIES` (around line 22) and into its subtype list (around line 47): + +```csharp +public const string ATTR_SUBTYPE_INT_MAP = "intMap"; +``` + +- [ ] **Step 4: Register the `DataType.Object` dispatch entry** + +Edit `server/csharp/MetaObjects/CoreTypes.cs` — add to the dispatch table shown at lines 125-137: + +```csharp + [ATTR_SUBTYPE_PROPERTIES] = DataType.Object, + [ATTR_SUBTYPE_INT_MAP] = DataType.Object, +``` + +- [ ] **Step 5: Add `FIELD_ATTR_INT_VALUE_MAP` constant** + +Edit `server/csharp/MetaObjects/Core/Field/FieldConstants.cs` — add next to `FIELD_ATTR_VALUES` (around line 184): + +```csharp +public const string FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; +``` + +- [ ] **Step 6: Add the `IntValueMapAttr` schema entry** + +Edit `server/csharp/MetaObjects/Core/Field/FieldSchema.cs` — add next to `EnumValuesAttr` (around line 168): + +```csharp + /// The @intValueMap attr — only on field.enum. Optional object of integers. + public static readonly AttrSchema IntValueMapAttr = new AttrSchema( + Name: FieldConstants.FIELD_ATTR_INT_VALUE_MAP, + ValueType: AttrConstants.ATTR_SUBTYPE_INT_MAP, + Required: false, + Description: "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers."); +``` + +- [ ] **Step 7: Register the attr on the `enum` subtype** + +Edit `server/csharp/MetaObjects/CoreTypes.cs` at the line found in research (`FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr]`, around line 304): + +```csharp +FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr, FieldSchema.IntValueMapAttr], +``` + +- [ ] **Step 8: Add the content-rule validation to Pass 10** + +Edit `server/csharp/MetaObjects/Loader/ValidationPasses.cs` — inside `WalkEnumValues`, after the existing Rule 4 (FR-011 fallback-attr check, ends before the closing brace shown in research), add: + +```csharp + // Rule 5: @intValueMap content rules (optional). + // a. Key set must exactly match @values. + // b. No two members may share the same int (protobuf's stance — no alias opt-in). + // (Every-value-is-an-integer is already enforced by the generic + // ATTR_SUBTYPE_INT_MAP dispatch/JSON-type check at parse time.) + if (field.OwnAttr(FIELD_ATTR_INT_VALUE_MAP) is System.Collections.IDictionary intValueMap) + { + var effective = field.EffectiveEnumValues ?? new List(); + var memberSet = new HashSet(effective, StringComparer.Ordinal); + var mapKeys = new List(); + foreach (var key in intValueMap.Keys) mapKeys.Add((string)key); + var keySet = new HashSet(mapKeys, StringComparer.Ordinal); + + var missing = effective.Where(m => !keySet.Contains(m)).ToList(); + var extra = mapKeys.Where(k => !memberSet.Contains(k)).ToList(); + if (missing.Count > 0 || extra.Count > 0) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@{FIELD_ATTR_VALUES}' members" + + (missing.Count > 0 ? $" (missing: {string.Join(", ", missing)})" : "") + + (extra.Count > 0 ? $" (unknown: {string.Join(", ", extra)})" : "") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + + var seenValues = new Dictionary(); + foreach (var key in mapKeys) + { + if (intValueMap[key] is not long and not int) continue; // generic dispatch already reported this + var value = System.Convert.ToInt64(intValueMap[key]); + if (seenValues.TryGetValue(value, out var owner)) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members '{owner}' and '{key}' " + + $"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + else + { + seenValues[value] = key; + } + } + } +``` + +> The exact C# type `OwnAttr` returns for an object-shaped attr value (`IDictionary`, `JsonObject`, or a custom `Dictionary`) depends on this codebase's JSON-parsing layer — check how the existing `@enumAlias`/`@enumDoc`-style properties attrs are read elsewhere in this file (or in `SpringDtoGenerator`-equivalent C# consumers) and match that exact type before finalizing this step. + +- [ ] **Step 9: Add the `attr.intMap` and `field.enum.intValueMap` declarations to C#'s spec copies** + +Edit `server/csharp/MetaObjects/SpecMetamodel/attr.json` and `server/csharp/MetaObjects/SpecMetamodel/field.json` with the same two JSON snippets used in TS Task 1 Step 6 and Task 2 Step 4 (these are C#'s own packaged copies — keep them semantically identical to the canonical `spec/metamodel/*.json`, per this plan's Global Constraints; there is no automated drift-gate tying them together today, so this is a manual-fidelity step, not a generated one). + +- [ ] **Step 10: Run tests — confirm all pass** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter EnumIntValueMapTests` +Expected: PASS — all 6 tests green. + +- [ ] **Step 11: Run the full C# metadata test suite** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests` +Expected: all pass, no regressions. + +- [ ] **Step 12: Commit** + +```bash +git add server/csharp/MetaObjects/Core/Attr/AttrConstants.cs server/csharp/MetaObjects/CoreTypes.cs server/csharp/MetaObjects/Core/Field/FieldConstants.cs server/csharp/MetaObjects/Core/Field/FieldSchema.cs server/csharp/MetaObjects/Loader/ValidationPasses.cs server/csharp/MetaObjects/SpecMetamodel/attr.json server/csharp/MetaObjects/SpecMetamodel/field.json server/csharp/MetaObjects.Tests/EnumIntValueMapTests.cs +git commit -m "feat(csharp): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 7: C# — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the shared conformance fixtures against C#'s loader** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter ConformanceTests` (adjust the filter name to whatever test class discovers/runs `fixtures/conformance/*` — check for a `ConformanceRunner`-style test class first) +Expected: the five fixtures from Task 3 all pass. + +- [ ] **Step 2: Run C#'s registry-conformance test** + +Run: `cd server/csharp && dotnet test MetaObjects.Tests --filter RegistryConformance` +Expected: PASS against the `expected-registry.json` Task 4 updated. + +- [ ] **Step 3: Run the full C# test suite** + +Run: `cd server/csharp && dotnet test` +Expected: 100% pass, no regressions, including `MetaObjects.Codegen.Tests` (should show zero behavior change — codegen isn't touched by this plan). + +--- + +### Task 8: Java (+ Kotlin) — `attr.intMap` subtype + `field.enum`'s `@intValueMap` + +**Files:** +- Create: `server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` +- Test: `server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java` + +**Interfaces:** +- Produces: `EnumField.ATTR_INT_VALUE_MAP`, `IntMapAttribute.SUBTYPE_INT_MAP` — consumed by the Java+Kotlin persistence follow-on plan. + +**Note:** Do NOT register `@intValueMap` using the existing `PropertiesAttribute` class. It is backed by `java.util.Properties`, confirmed (`PropertiesAttribute.java:51-60`) to coerce every value to `String` on load — silently turning `0` into `"0"`. Since canonical-JSON round-trip serialization must stay byte-identical with TS/C#/Python (which preserve real integers), Java needs its own `Map`-backed class, mirroring `PropertiesAttribute`'s structure exactly but preserving int fidelity on both parse and `getValueAsString()`. + +- [ ] **Step 1: Write the failing test** + +```java +// server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java +package com.metaobjects.field; + +import com.metaobjects.MetaDataException; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.loader.source.InMemoryMetaDataSource; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.*; + +class EnumFieldIntValueMapTest { + + private static String model(String extra) { + return "{ \"metadata.root\": { \"package\": \"acme\", \"children\": [" + + "{ \"object.entity\": { \"name\": \"Order\", \"children\": [" + + "{ \"field.long\": { \"name\": \"id\" } }," + + "{ \"field.enum\": { \"name\": \"status\", \"@values\": [\"DRAFT\",\"PUBLISHED\",\"ARCHIVED\"]" + extra + " } }," + + "{ \"identity.primary\": { \"name\": \"pk\", \"@fields\": [\"id\"] } }" + + "]}}]}}"; + } + + private static MetaDataException loadExpectingError(String json) { + var loader = new MetaDataLoader(); + return assertThrows(MetaDataException.class, () -> + loader.load(java.util.List.of(new InMemoryMetaDataSource(json, "test.json")))); + } + + @Test + void validIntValueMapWithMatchingKeysAndUniqueIntsLoadsClean() { + var loader = new MetaDataLoader(); + assertDoesNotThrow(() -> loader.load(java.util.List.of(new InMemoryMetaDataSource( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9}"), "test.json")))); + } + + @Test + void noIntValueMapStillLoadsCleanStringBackedDefault() { + var loader = new MetaDataLoader(); + assertDoesNotThrow(() -> loader.load(java.util.List.of(new InMemoryMetaDataSource(model(""), "test.json")))); + } + + @Test + void missingMemberKeyIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + assertTrue(ex.getMessage().contains("ARCHIVED")); + } + + @Test + void extraKeyNotInValuesIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9, \"RETRACTED\": 12}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + assertTrue(ex.getMessage().contains("RETRACTED")); + } + + @Test + void nonIntegerValueIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": \"zero\", \"PUBLISHED\": 5, \"ARCHIVED\": 9}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + } + + @Test + void duplicateIntValueAcrossMembersIsRejected() { + var ex = loadExpectingError(model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 0, \"ARCHIVED\": 9}")); + assertEquals(com.metaobjects.ErrorCode.ERR_BAD_ATTR_VALUE, ex.getErrorCode()); + assertTrue(ex.getMessage().contains("DRAFT") && ex.getMessage().contains("PUBLISHED")); + } +} +``` + +> Check this module's actual in-memory-source test helper name/package (`InMemoryMetaDataSource` is a placeholder guess) and the loader's actual single-error-throw-vs-collect behavior against an existing test like `EnumFieldTest.java` before finalizing — mirror its exact loading idiom. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/java && mvn -pl metadata test -Dtest=EnumFieldIntValueMapTest` +Expected: FAIL — `ERR_UNKNOWN_ATTR` on the positive tests. + +- [ ] **Step 3: Write `IntMapAttribute.java`** + +```java +package com.metaobjects.attr; + +import com.metaobjects.DataTypes; +import com.metaobjects.registry.MetaDataRegistry; + +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * An object-shaped attribute whose values are all integers (e.g. field.enum's + * {@code @intValueMap}: {member: int}). Mirrors {@link PropertiesAttribute}'s + * structure, but backed by {@code Map} rather than + * {@code java.util.Properties} — Properties forces every value to a String on + * load, which would silently corrupt int fidelity on canonical-JSON round-trip. + * Generic shape check only (object, every value an integer); a consumer field + * type (field.enum) layers its own semantic rules (key-set membership, + * uniqueness) in its own post-load content-rule validation. + */ +public class IntMapAttribute extends MetaAttribute> { + + public static final String SUBTYPE_INT_MAP = "intMap"; + + public static void registerTypes(MetaDataRegistry registry) { + registry.registerType(IntMapAttribute.class, def -> def + .type(TYPE_ATTR).subType(SUBTYPE_INT_MAP) + .description("An object-shaped attribute whose values are all integers.") + .inheritsFrom(TYPE_ATTR, SUBTYPE_BASE) + ); + } + + public IntMapAttribute(String name) { + super(SUBTYPE_INT_MAP, name, DataTypes.CUSTOM); + } + + public static IntMapAttribute create(String name, Map value) { + IntMapAttribute a = new IntMapAttribute(name); + a.setValue(value); + return a; + } + + @Override + public void setValueAsObject(Object value) { + if (value == null) { + setValue(null); + } else if (value instanceof String) { + setValueAsString((String) value); + } else if (value instanceof Map) { + Map m = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) value).entrySet()) { + if (e.getKey() == null || e.getValue() == null) continue; + m.put(e.getKey().toString(), coerceInt(e.getValue())); + } + setValue(m); + } else { + throw new InvalidAttributeValueException( + "Can not set value with class [" + value.getClass() + "] for object: " + value); + } + } + + @Override + public void setValueAsString(String value) { + if (value == null) { setValue(null); return; } + String trimmed = value.trim(); + if (!(trimmed.startsWith("{") && trimmed.endsWith("}"))) { + throw new InvalidAttributeValueException( + "Could not parse intMap attribute value (expected a JSON object): " + value); + } + com.google.gson.JsonObject obj = com.google.gson.JsonParser.parseString(trimmed).getAsJsonObject(); + Map m = new LinkedHashMap<>(); + for (Map.Entry e : obj.entrySet()) { + com.google.gson.JsonElement el = e.getValue(); + if (el.isJsonPrimitive() && el.getAsJsonPrimitive().isNumber()) { + m.put(e.getKey(), el.getAsInt()); + } else { + // Not a number — record a sentinel that fails the generic content + // check downstream (own validateEnumNode reports the specific error); + // we don't throw here so the loader can report ALL bad members, not + // just the first, matching the other ports' collect-not-throw style + // at the field-content-rule layer. Store Integer.MIN_VALUE as a + // deliberately-invalid marker is avoided — instead we fail loudly here, + // since this attribute type's OWN contract is "every value is an int". + throw new InvalidAttributeValueException( + "attribute '@" + getName() + "' member '" + e.getKey() + "' has value '" + el + + "' which is not an integer"); + } + } + setValue(m); + } + + private static int coerceInt(Object value) { + if (value instanceof Integer i) return i; + if (value instanceof Number n && n.doubleValue() == Math.floor(n.doubleValue())) return n.intValue(); + throw new InvalidAttributeValueException("intMap value is not an integer: " + value); + } + + @Override + public String getValueAsString() { + return getValue() == null ? null : new com.google.gson.Gson().toJson(getValue()); + } +} +``` + +> `EnumFieldIntValueMapTest`'s `nonIntegerValueIsRejected` test expects `ERR_BAD_ATTR_VALUE`, but this class throws `InvalidAttributeValueException` directly — check how `InvalidAttributeValueException` maps to `ErrorCode.ERR_BAD_ATTR_VALUE` elsewhere in the loader (likely a catch-and-wrap in `MetaDataLoader`/`ValidationPhase`) before finalizing; if it doesn't already map that way, catch it at the `field.enum` content-rule layer (`EnumField`/`ValidationPhase`) instead of letting it escape raw. + +- [ ] **Step 4: Register `IntMapAttribute` in the metadata provider bootstrap** + +Find wherever `PropertiesAttribute.registerTypes(registry)` is called (search `FieldTypesMetaDataProvider.java` or the equivalent core-attrs provider file — the same place Task 8's research found `EnumField.registerTypes(registry)` wired at `FieldTypesMetaDataProvider.java:66`) and add a sibling call: + +```java +IntMapAttribute.registerTypes(registry); +``` + +- [ ] **Step 5: Add `ATTR_INT_VALUE_MAP` constant to `EnumField.java`** + +Edit `server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java` — add next to `ATTR_VALUES` (around line 60): + +```java + /** + * Name of the optional per-member explicit-integer-value attribute + * ({@code {member: int}}), switching this enum field's DB persistence from + * string+CHECK to integer+CHECK. Keys must exactly match {@code @values}; + * values must be unique integers. Cross-language vocabulary: + * {@code @intValueMap} in canonical JSON. + */ + public static final String ATTR_INT_VALUE_MAP = "intValueMap"; +``` + +- [ ] **Step 6: Register the attr in `EnumField.registerTypes`** + +Edit the same file's `registerTypes` method (shown in research at line 173-221) — add after the `ATTR_PROVIDED` registration: + +```java + // Optional @intValueMap — an object-shaped attribute whose values + // are all integers. Key-set-matches-@values and uniqueness are + // validated post-load in ValidationPhase (own-only, same as @values). + def.optionalAttributeWithConstraints(ATTR_INT_VALUE_MAP) + .ofType(com.metaobjects.attr.IntMapAttribute.SUBTYPE_INT_MAP) + .asSingle(); +``` + +- [ ] **Step 7: Add the content-rule validation to `ValidationPhase.java`** + +Edit `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` — inside `validateEnumNode` (shown in research at lines 503-538), add after the existing own-`@values` content check and before `validateEnumFr011Attrs(node)` is called: + +```java + // --- Own @intValueMap content check (optional) --- + if (node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false)) { + @SuppressWarnings("unchecked") + java.util.Map intValueMap = + (java.util.Map) node.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false).getValue(); + java.util.List effective = effectiveEnumValues(node); + java.util.Set memberSet = new java.util.HashSet<>(effective); + java.util.Set keySet = intValueMap.keySet(); + + java.util.List missing = effective.stream().filter(m -> !keySet.contains(m)).toList(); + java.util.List extra = keySet.stream().filter(k -> !memberSet.contains(k)).toList(); + if (!missing.isEmpty() || !extra.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' keys must exactly match '@" + EnumField.ATTR_VALUES + "' members" + + (missing.isEmpty() ? "" : " (missing: " + String.join(", ", missing) + ")") + + (extra.isEmpty() ? "" : " (unknown: " + String.join(", ", extra) + ")") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + + java.util.Map seenValues = new java.util.HashMap<>(); + for (var entry : intValueMap.entrySet()) { + Integer value = entry.getValue(); + String owner = seenValues.putIfAbsent(value, entry.getKey()); + if (owner != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' members '" + owner + "' and '" + entry.getKey() + + "' share the same value " + value + "; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + } + } +``` + +> `effectiveEnumValues(node)` already exists per research (line 602-619) — confirm its exact return type (`List`) matches this usage. + +- [ ] **Step 8: Run tests — confirm all pass** + +Run: `cd server/java && mvn -pl metadata test -Dtest=EnumFieldIntValueMapTest` +Expected: PASS — all 6 tests green. + +- [ ] **Step 9: Run the full Java metadata module test suite** + +Run: `cd server/java && mvn -pl metadata test` +Expected: all pass, no regressions. + +- [ ] **Step 10: Commit** + +```bash +git add server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java +git commit -m "feat(java): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 9: Java (+ Kotlin) — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the shared conformance fixtures against Java's loader** + +Run: `cd server/java && mvn -pl metadata test -Dtest=*ConformanceTest*` (adjust to the actual test class that discovers `fixtures/conformance/*`) +Expected: the five fixtures from Task 3 all pass. + +- [ ] **Step 2: Run Java's registry-conformance test** + +Run: `cd server/java && mvn -pl metadata test -Dtest=*RegistryConformance*` +Expected: PASS against Task 4's updated `expected-registry.json`. + +- [ ] **Step 3: Run the Kotlin module's tests to confirm no regression** (Kotlin shares Java's metadata layer — this is a smoke check, not new Kotlin-specific work) + +Run: `cd server/java && mvn -pl codegen-kotlin,metadata-ktx test` +Expected: all pass, no regressions. + +- [ ] **Step 4: Run the full Java build** + +Run: `cd server/java && mvn test` +Expected: 100% pass (excluding any pre-existing known-red modules unrelated to this change). + +--- + +### Task 10: Python — `attr.intMap` subtype + `field.enum`'s `@intValueMap` + +**Files:** +- Modify: `server/python/src/metaobjects/meta/core/attr/attr_constants.py` +- Modify: `server/python/src/metaobjects/meta/core/attr/meta_attr.py` +- Modify: `server/python/src/metaobjects/spec_metamodel/attr.json` +- Modify: `server/python/src/metaobjects/spec_metamodel/field.json` +- Modify: `server/python/src/metaobjects/field_constants.py` +- Modify: `server/python/src/metaobjects/loader/validation_passes.py` +- Test: `server/python/tests/unit/test_field_enum_intvaluemap.py` + +**Interfaces:** +- Produces: `FIELD_ATTR_INT_VALUE_MAP`, `ATTR_SUBTYPE_INT_MAP` — consumed by the Python persistence follow-on plan. + +- [ ] **Step 1: Write the failing tests** + +```python +# server/python/tests/unit/test_field_enum_intvaluemap.py +import pytest +from metaobjects.loader.loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.errors import ErrorCode, MetaDataException + + +def _model(extra: str) -> str: + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def _load(json_str: str): + loader = MetaDataLoader() + return loader.load([InMemoryStringSource(json_str, "test.json")]) + + +def test_valid_intvaluemap_with_matching_keys_and_unique_ints_loads_clean(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')) + assert result.errors == [] + + +def test_no_intvaluemap_still_loads_clean_string_backed_default(): + result = _load(_model("")) + assert result.errors == [] + + +def test_missing_member_key_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "ARCHIVED" in result.errors[0].message + + +def test_extra_key_not_in_values_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "RETRACTED" in result.errors[0].message + + +def test_non_integer_value_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + + +def test_duplicate_int_value_across_members_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "DRAFT" in result.errors[0].message and "PUBLISHED" in result.errors[0].message +``` + +> Check `test_field_enum.py`'s actual `MetaDataLoader`/source-loading and `result.errors` shape (some Python loader paths raise `MetaDataException` on first error rather than collecting) before finalizing — mirror its established pattern exactly. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/python && pytest tests/unit/test_field_enum_intvaluemap.py -v` +Expected: FAIL — `ERR_UNKNOWN_ATTR` on the positive tests. + +- [ ] **Step 3: Add `ATTR_SUBTYPE_INT_MAP` constant** + +Edit `server/python/src/metaobjects/meta/core/attr/attr_constants.py` — add next to `ATTR_SUBTYPE_PROPERTIES`: + +```python +ATTR_SUBTYPE_INT_MAP = "intMap" +``` + +- [ ] **Step 4: Add the `IntMapAttr` class** + +Edit `server/python/src/metaobjects/meta/core/attr/meta_attr.py` — add next to `PropertiesAttr` (around line 131): + +```python +class IntMapAttr(MetaAttr): + """attr.intMap — an object-shaped attribute whose values are all integers + (e.g. field.enum's @intValueMap). Generic shape check only; a consumer + field type (field.enum) layers its own semantic rules (key-set membership, + uniqueness) in its own content-rule validation pass.""" + + @property + def data_type(self) -> DataType: + return DataType.OBJECT + + def coerce(self, raw: object) -> object: + return raw + + def validate_value(self, value: object) -> list[ValueError]: + if not isinstance(value, dict): + return [ValueError(f"attribute '@{self.name}' must be of type 'intMap' but got {type(value).__name__}")] + errors: list[ValueError] = [] + for key, member in value.items(): + if isinstance(member, bool) or not isinstance(member, int): + errors.append(ValueError(f"attribute '@{self.name}' member '{key}' has value '{member}' which is not an integer")) + return errors +``` + +> Check `MetaAttr`'s actual `validate_value` signature/return type (the existing `PropertiesAttr`/`FilterAttr` classes in this file don't override it, so confirm the base class's default and this override's exact contract — e.g. does it return a list of `ValueError` instances, plain strings, or something else — against `StringArrayAttr`'s override, which likely DOES override validation, before finalizing.) Note Python's `bool` is a subclass of `int` — the `isinstance(member, bool)` guard above is required to correctly reject a JSON `true`/`false` value. + +- [ ] **Step 5: Register the class** + +Edit the same file's registration block (around line 143-148): + +```python +register_attr_class(ATTR_SUBTYPE_PROPERTIES, PropertiesAttr) +register_attr_class(ATTR_SUBTYPE_EXPRESSION, ExpressionAttr) +register_attr_class(ATTR_SUBTYPE_INT_MAP, IntMapAttr) +``` + +- [ ] **Step 6: Add the `attr.intMap` type declaration** + +Edit `server/python/src/metaobjects/spec_metamodel/attr.json` with the same JSON snippet used in TS Task 1 Step 6 (Python's own packaged copy — keep semantically identical to the canonical spec). + +- [ ] **Step 7: Add `FIELD_ATTR_INT_VALUE_MAP` constant** + +Edit `server/python/src/metaobjects/field_constants.py` — add next to wherever `FIELD_ATTR_VALUES`/`ENUM_MEMBER_PATTERN` are declared (per research, `field_constants.py:147` for `ENUM_MEMBER_PATTERN`): + +```python +FIELD_ATTR_INT_VALUE_MAP = "intValueMap" +``` + +- [ ] **Step 8: Add the `field.enum.intValueMap` declaration** + +Edit `server/python/src/metaobjects/spec_metamodel/field.json` with the same JSON snippet used in TS Task 2 Step 4. + +- [ ] **Step 9: Add the content-rule validation** + +Edit `server/python/src/metaobjects/loader/validation_passes.py` — inside `_validate_enum_values` (shown in research at line 534), add after Rule 3 (no duplicates, ends around line 592) and before the function returns: + +```python + # Rule 4: @intValueMap content rules (optional). + # a. Key set must exactly match @values. + # b. No two members may share the same int (protobuf's stance — no + # alias opt-in). Every-value-is-an-integer is already enforced + # by IntMapAttr's generic shape validation at parse time. + int_value_map = node.attr(FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + member_set = set(own_values) + key_set = set(int_value_map.keys()) + missing = [m for m in own_values if m not in key_set] + extra = [k for k in int_value_map if k not in member_set] + if missing or extra: + parts = [] + if missing: + parts.append(f"missing: {', '.join(missing)}") + if extra: + parts.append(f"unknown: {', '.join(extra)}") + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match " + f"'@{FIELD_ATTR_VALUES}' members ({'; '.join(parts)}).", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + + seen_values: dict[int, str] = {} + for member, value in int_value_map.items(): + if not isinstance(value, int) or isinstance(value, bool): + continue # IntMapAttr already reported this + owner = seen_values.get(value) + if owner is not None: + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members {owner!r} and {member!r} " + f"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + else: + seen_values[value] = member +``` + +Add `FIELD_ATTR_INT_VALUE_MAP` to this file's existing import block from `field_constants`. + +- [ ] **Step 10: Run tests — confirm all pass** + +Run: `cd server/python && pytest tests/unit/test_field_enum_intvaluemap.py -v` +Expected: PASS — all 6 tests green. + +- [ ] **Step 11: Run the full Python test suite** + +Run: `cd server/python && pytest` +Expected: all pass, no regressions. + +- [ ] **Step 12: Commit** + +```bash +git add server/python/src/metaobjects/meta/core/attr/attr_constants.py server/python/src/metaobjects/meta/core/attr/meta_attr.py server/python/src/metaobjects/spec_metamodel/attr.json server/python/src/metaobjects/spec_metamodel/field.json server/python/src/metaobjects/field_constants.py server/python/src/metaobjects/loader/validation_passes.py server/python/tests/unit/test_field_enum_intvaluemap.py +git commit -m "feat(python): field.enum @intValueMap — explicit per-member int values for DB persistence" +``` + +--- + +### Task 11: Python — full verification + +**Files:** none (verification only) + +- [ ] **Step 1: Run the shared conformance fixtures against Python's loader** + +Run: `cd server/python && pytest tests/ -k conformance` +Expected: the five fixtures from Task 3 all pass. + +- [ ] **Step 2: Run Python's registry-conformance test** + +Run: `cd server/python && pytest tests/ -k registry_conformance` +Expected: PASS against Task 4's updated `expected-registry.json`. + +- [ ] **Step 3: Run the full Python test suite** + +Run: `cd server/python && pytest` +Expected: 100% pass. + +--- + +## After this plan lands + +This plan ships a fully validated, cross-port `@intValueMap` vocabulary that loads, validates, and round-trips correctly — but nothing reads it yet. Follow-on plans (one per port/group, written separately per the Scope Check in `superpowers:writing-plans`): + +- **TS persistence** — migrate-ts DDL (`integer` + int `CHECK` instead of `text`/`varchar` + string `CHECK`), the migration-safety guard (D8: refuse to auto-`ALTER` an existing column across backing modes), and the Drizzle/Kysely symbol↔int codec in generated `queries.ts`/`entity.ts`. +- **C# persistence** — an EF Core `HasConversion` built from `@intValueMap`'s lookup table (replacing `HasConversion()` only when `@intValueMap` is present). +- **Java + Kotlin persistence** — a new OMDB JDBC codec (binding `Types.INTEGER`, mirroring the `CurrencyCodec`/`UuidCodec` pattern in `JdbcCodecs.java`) and, for Kotlin, switching `KotlinExposedTableGenerator.kt`'s `enumerationByName(...)` to `enumeration(...)` when `@intValueMap` is present. +- **Python persistence** — a new branch in `ObjectManager`'s scalar-coercion function (currently a pure fallthrough for enums) translating symbol↔int at the DB boundary. +- **Cross-port persistence-conformance** — extend `roundtrip-all-types.yaml` + `meta.fitness.json` with an int-backed enum field, round-tripping through every port's real runtime. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md new file mode 100644 index 000000000..6902807d0 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-python-persistence.md @@ -0,0 +1,296 @@ +# Int-Backed Enum Values — Python Persistence Implementation Plan + +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped) into Python's `ObjectManager` runtime: encode the member symbol to its int on write, decode it back on read. Python has no per-subtype `EnumField` class (confirmed — `field.sub_type == fc.FIELD_SUBTYPE_ENUM` is the only discriminator) and no codegen change is needed at all — Python's generated type for an inline enum field is `Literal["DRAFT", "PUBLISHED", ...]` regardless of backing mode, since codegen never reads `@intValueMap`. + +**Architecture:** `_coerce_write_value` (`server/python/src/metaobjects/runtime/object_manager.py:766-826`) is today a pure fallthrough for `field.sub_type == FIELD_SUBTYPE_ENUM` (falls to the final `return value` at line 826). This plan adds a new branch there for the write side. The **read side needs a wholly new function** — confirmed there is no decode/read-side coercion anywhere in this module today (`select()`/`find_by_id`/`find_many` return pg8000's native values verbatim, per ADR-0019). This plan adds `_decode_read_value(field, value)` and wires it into every row-mapping call site. + +**Tech Stack:** Python, pytest, pg8000 (via `ObjectManager`). + +## Global Constraints + +- Python's generated type for `field.enum` (`Literal["A","B",...]` inline, or the FR-019 shared `class X(str, Enum)`) is unchanged whether or not `@intValueMap` is present — do not touch `entity_model.py` or `fr019_shared_enum.py`. +- `@intValueMap`'s presence alone is the trigger, read via `field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP)` (resolving — own or inherited via `extends`, matching how `@localTime`/`@storage` are already read in this same function). +- Every codec constant/import goes through `field_constants` (`fc`) — this module already imports it aliased as `fc`; follow that convention. + +--- + +### Task 1: write-side encode in `_coerce_write_value` + +**Files:** +- Modify: `server/python/src/metaobjects/runtime/object_manager.py` +- Test: `server/python/tests/runtime/test_object_manager_enum_intvaluemap.py` + +**Interfaces:** +- Consumes: `fc.FIELD_ATTR_INT_VALUE_MAP` (metamodel plan, already shipped). +- Produces: `_coerce_write_value(field, value)` returns the mapped int for an int-backed enum field, unchanged behavior otherwise. + +- [ ] **Step 1: Write the failing tests** + +```python +# server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +import pytest +from metaobjects.runtime.object_manager import _coerce_write_value +from metaobjects.loader.loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource + + +def _load_order_field(extra: str): + json_str = f"""{{ "metadata.root": {{ "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + loader = MetaDataLoader() + result = loader.load([InMemoryStringSource(json_str, "test.json")]) + assert result.errors == [] + entity = result.root.find_object("Order") + return entity.field("status") + + +def test_int_backed_enum_write_value_encodes_symbol_to_int(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _coerce_write_value(field, "PUBLISHED") == 5 + + +def test_string_backed_enum_write_value_is_unchanged(): + field = _load_order_field("") + assert _coerce_write_value(field, "PUBLISHED") == "PUBLISHED" + + +def test_int_backed_enum_write_value_none_stays_none(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _coerce_write_value(field, None) is None +``` + +> Check `MetaDataLoader`/`InMemoryStringSource`/`find_object`/`.field(name)`'s actual API against `test_field_enum.py` (used as the template in the metamodel plan) before finalizing — this file reuses that same loading idiom. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: FAIL — `test_int_backed_enum_write_value_encodes_symbol_to_int` fails (currently returns `"PUBLISHED"` unchanged, since enum is a pure fallthrough). + +- [ ] **Step 3: Add the encode branch** + +Edit `server/python/src/metaobjects/runtime/object_manager.py` — insert before the final fallthrough comment/return (line 824-826): + +```python + # field.enum: int-backed persistence (docs/superpowers/specs/2026-07-23-int-backed- + # enum-values-design.md). @intValueMap maps the member symbol to its stored int; + # resolving read (own or inherited via extends), matching @localTime/@storage above. + # Absent → string-backed default, unchanged (falls to the generic return below). + if sub == fc.FIELD_SUBTYPE_ENUM: + int_value_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + return int_value_map[value] + + # Everything else (string / int / long / double / float / boolean / enum) + # is already the native type pg8000 binds directly. + return value +``` + +(This replaces the trailing comment+return at lines 824-826 — the new `if` block goes immediately above it, inside the same function, after the existing `field.object`/`field.map` jsonb branch at lines 820-823.) + +Add `FIELD_ATTR_INT_VALUE_MAP` to this file's existing `from metaobjects import field_constants as fc`-style import (it already imports `fc` wholesale per the module's existing style, based on `fc.FIELD_ATTR_DB_COLUMN_TYPE`/`fc.FIELD_SUBTYPE_DECIMAL` usage — no new import line needed if `fc` is a module-level import, since `FIELD_ATTR_INT_VALUE_MAP` was already added to `field_constants.py` by the metamodel plan). + +- [ ] **Step 4: Run tests — confirm all pass** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: PASS — all 3 tests green. + +- [ ] **Step 5: Run the full ObjectManager test suite** + +Run: `cd server/python && pytest tests/runtime/` +Expected: all pass, no regressions. + +- [ ] **Step 6: Commit** + +```bash +git add server/python/src/metaobjects/runtime/object_manager.py server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +git commit -m "feat(python): ObjectManager encodes int-backed field.enum symbol->int on write" +``` + +--- + +### Task 2: read-side decode (new function + wiring) + +**Files:** +- Modify: `server/python/src/metaobjects/runtime/object_manager.py` +- Test: extend `server/python/tests/runtime/test_object_manager_enum_intvaluemap.py` + +**Interfaces:** +- Produces: `_decode_read_value(field, value)` — a new function, the read-side mirror of `_coerce_write_value`, consumed by every row-mapping call site in `find_by_id`/`find_many`. + +- [ ] **Step 1: Write the failing tests** + +Append to `test_object_manager_enum_intvaluemap.py`: + +```python +from metaobjects.runtime.object_manager import _decode_read_value + + +def test_int_backed_enum_read_value_decodes_int_to_symbol(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _decode_read_value(field, 5) == "PUBLISHED" + + +def test_string_backed_enum_read_value_is_unchanged(): + field = _load_order_field("") + assert _decode_read_value(field, "PUBLISHED") == "PUBLISHED" + + +def test_int_backed_enum_read_value_none_stays_none(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + assert _decode_read_value(field, None) is None + + +def test_int_backed_enum_read_value_unknown_int_raises(): + field = _load_order_field(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}') + with pytest.raises(ValueError): + _decode_read_value(field, 42) +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: FAIL — `ImportError: cannot import name '_decode_read_value'`. + +- [ ] **Step 3: Add `_decode_read_value`** + +Edit `server/python/src/metaobjects/runtime/object_manager.py` — add near `_coerce_write_value`: + +```python +def _decode_read_value(field: MetaField, value: Any) -> Any: + """The read-side mirror of _coerce_write_value for int-backed field.enum. Every + other field type's read value passes through unmodified today (ADR-0019 — the + runtime returns native types, never wire-strings); this is the first field type + needing a genuine decode step, mirroring the (also-new) write-side encode above.""" + if value is None: + return None + if field.sub_type == fc.FIELD_SUBTYPE_ENUM: + int_value_map = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + if isinstance(int_value_map, dict): + for symbol, i in int_value_map.items(): + if i == value: + return symbol + raise ValueError( + f"field.enum '{field.name}' read value {value!r} has no matching @intValueMap entry" + ) + return value +``` + +- [ ] **Step 4: Wire it into the row-mapping call sites** + +Find `find_by_id`/`find_many`'s row-to-object mapping code (the functions the earlier research noted do "no per-field value transformation" today — read them in full first) and add a pass over each row's fields, applying `_decode_read_value(field, raw_value)` for every field before constructing the returned object. The exact insertion point depends on whether row mapping is a dict comprehension, a loop, or a dataclass constructor call — mirror whichever shape those functions actually use; do not restructure them beyond adding this one per-field decode step. + +- [ ] **Step 5: Run tests — confirm all pass** + +Run: `cd server/python && pytest tests/runtime/test_object_manager_enum_intvaluemap.py -v` +Expected: PASS — all 7 tests in this file green. + +- [ ] **Step 6: Run the full ObjectManager test suite** + +Run: `cd server/python && pytest tests/runtime/` +Expected: all pass, no regressions — every existing read of a string-backed enum (or any other field type) must be a no-op through `_decode_read_value` (confirmed by `test_string_backed_enum_read_value_is_unchanged` and the broader regression run). + +- [ ] **Step 7: Commit** + +```bash +git add server/python/src/metaobjects/runtime/object_manager.py server/python/tests/runtime/test_object_manager_enum_intvaluemap.py +git commit -m "feat(python): ObjectManager decodes int-backed field.enum int->symbol on read" +``` + +--- + +### Task 3: real-engine round-trip + +**Files:** +- Test: `server/python/tests/integration/test_enum_intvaluemap_roundtrip.py` + +**Interfaces:** +- Consumes: Tasks 1-2. + +- [ ] **Step 1: Write the real-engine test** + +```python +# server/python/tests/integration/test_enum_intvaluemap_roundtrip.py +import pytest +from metaobjects.runtime.object_manager import ObjectManager +from metaobjects.loader.loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource + +# Match whatever pg8000 + Testcontainers (or the existing local Postgres) fixture +# this module's other integration tests already use — check +# server/python/tests/integration/ for the established connection-fixture pattern. + + +@pytest.mark.integration +def test_int_backed_enum_round_trips_through_real_postgres(pg_connection): # fixture name TBD — match existing convention + json_str = """{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}""" + loader = MetaDataLoader() + result = loader.load([InMemoryStringSource(json_str, "test.json")]) + entity = result.root.find_object("Order") + + om = ObjectManager(pg_connection) + # Table must already exist — apply the corresponding DDL (integer + int CHECK, + # per the TS persistence plan's schema) before this test runs, matching however + # this module's other integration tests provision their schema (likely the + # committed canonical/schema.postgres.sql, per this repo's ADR-0015 convention). + created = om.create(entity, {"status": "PUBLISHED"}) + fetched = om.find_by_id(entity, created["id"]) + + assert fetched["status"] == "PUBLISHED" # decoded back to the symbol, not 5 + + raw = pg_connection.run("SELECT status FROM orders WHERE id = :id", id=created["id"]) + assert raw[0][0] == 5 # the actual stored value is the mapped int +``` + +> `pg_connection`'s fixture name/shape, `ObjectManager`'s real constructor signature, and how this module provisions its test schema are all placeholders pending a read of `server/python/tests/integration/`'s existing setup — mirror an existing integration test there exactly (do not guess the connection/fixture wiring). + +- [ ] **Step 2: Run to verify current gap** + +Run: `cd server/python && pytest tests/integration/test_enum_intvaluemap_roundtrip.py -v -m integration` +Expected: FAIL or errors until Step 1 is completed for real against the actual fixture/connection pattern. + +- [ ] **Step 3: Complete the test for real, run, confirm pass** + +Run: `cd server/python && pytest tests/integration/test_enum_intvaluemap_roundtrip.py -v -m integration` +Expected: PASS. + +- [ ] **Step 4: Run the full Python test suite** + +Run: `cd server/python && pytest` +Expected: 100% pass. + +- [ ] **Step 5: Commit** + +```bash +git add server/python/tests/integration/test_enum_intvaluemap_roundtrip.py +git commit -m "test(python): int-backed field.enum round-trips through real Postgres via ObjectManager" +``` + +--- + +## After all five plans land + +Once every port's persistence plan is done (this one; TS; C#; Java+Kotlin), the last remaining item from the design spec's conformance plan is the shared `fixtures/persistence-conformance/roundtrip-all-types.yaml` scenario carrying `intEnumVal` (added in the TS persistence plan's Task 6) passing identically across **all five** ports' real-engine runners — that is the final, cross-language proof this feature is genuinely done, not just done-per-port. Run each port's persistence-conformance suite one more time after all five plans are merged to confirm. diff --git a/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md new file mode 100644 index 000000000..fce0cd804 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-int-backed-enum-values-ts-persistence.md @@ -0,0 +1,843 @@ +# Int-Backed Enum Values — TypeScript Persistence Implementation Plan + +> **STATUS — SUPERSEDED, kept for provenance.** This plan is IMPLEMENTED; the shipped +> behaviour is in +> [`docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md`](../specs/2026-07-23-int-backed-enum-values-design.md), +> which is the source of truth. Two things below are now WRONG and must not be followed: +> **(1) D7 is reversed** — int-backing is scalar-only, and `@intValueMap` with `isArray` +> is a load error (`ERR_ENUM_INT_VALUE_MAP_ARRAY`) in every port, so every array-of-enum +> fixture, column shape and element-wise codec sketched here describes vocabulary that +> cannot load. **(2) Some sketched tests call APIs that do not exist** (e.g. +> `MetaRoot.find_object`, `MetaObject.field(name)`) or assume test libraries a module does +> not depend on. Read the shipped code and its tests, not these snippets. + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Wire `field.enum`'s `@intValueMap` (metamodel layer already shipped in `docs/superpowers/plans/2026-07-23-int-backed-enum-values-metamodel.md`) into TypeScript's persistence stack: migrate-ts emits `integer` + a numeric `CHECK` instead of `text` + a string `CHECK`, codegen-ts's Drizzle column mapper follows suit, and the generated entity/query code translates symbol↔int at the DB boundary while every TS-facing type (Zod schema, `EntityType`, wire JSON) stays exactly the string union it is today. + +**Architecture:** `@intValueMap`'s presence is read in exactly three places, each already dispatching on `field.subType === FIELD_SUBTYPE_ENUM`: migrate-ts's `subtypeToSqlType`/`arrayElementSqlType`/`buildChecks` (schema/DDL), codegen-ts's `column-mapper.ts` (Drizzle column declaration), and a **new** generated symbol↔int lookup consumed by two new template hooks — a Zod `.transform()` on write (an established pattern already used for `@autoSet` timestamps) and a small generated `decode` step on read (a genuinely new pattern for this codebase — no field has ever needed a wire type that differs from its storage type before). The migration-safety guard from the design's D8 needs **no new code at all**: this codebase's existing `isWidening`/`allow.typeChange` mechanism (`packages/migrate-ts/src/sql-type.ts` + `packages/migrate-ts/src/diff/status.ts`) already treats any cross-`kind` `change-column-type` (text↔integer included) as blocked-by-default requiring an explicit `allow.typeChange` pass — confirmed by reading `isWidening`'s `if (from.kind !== to.kind) return false` and `blockedReasonFor`'s `case "change-column-type"` branch. This task's job here is a **test proving that's true**, not new gating logic. + +**Tech Stack:** TypeScript, Bun test runner, Drizzle ORM, Kysely (via `@metaobjectsdev/runtime-ts`), Zod. + +## Global Constraints + +- Every TS-facing type (Zod schema output type, generated `EntityType`, the JSON wire payload) is BYTE-IDENTICAL between string-backed and int-backed enums — string in, string out, everywhere except the literal DB column. +- `@intValueMap` presence alone is the trigger — no new codegen option, no new CLI flag. +- Metamodel/validation changes need explicit justification, but are NOT forbidden. (This constraint originally read "do not touch the metamodel/validation layer — it's already done." That premise was falsified by #246 before this plan ever ran; see Amendment 1.) Prefer deriving from what's already registered; adding a new attr still requires the ADR-0023 can't-be-computed justification. +- The migration-safety guard requires ZERO new gating code (see Architecture) — this plan's migration-safety task is a test, not an implementation. + +--- + +## Amendments (2026-08-12) + +This plan was written 2026-07-23 against a pre-#246 / pre-FR-019-hardening tree. Everything below post-dates it and MUST be folded in before executing. Tasks 1-6 are otherwise still accurate — their file anchors were re-verified against the current tree and all resolve. + +**Amendment 1 — the metamodel layer moved under this plan's feet.** Three loader/codegen changes landed after this plan was authored: +- **#246 int-backed twin** — a field may NOT declare its own `@intValueMap` when its immediate super is a root-level abstract (SHARED, FR-019) `field.enum`. The map lives on the SHARED DECLARATION and is inherited. +- **`@provided` is declaration-layer** (ADR-0039 amended) — read own-only in all five ports. +- **Chained abstract enum declarations** are legal and each materializes as its own type, named for its own declaration. + +**Consequence for Tasks 4 + 5, and it is the load-bearing one:** every read of `@intValueMap` in codegen MUST be a RESOLVING read (`field.attr(FIELD_ATTR_INT_VALUE_MAP)`), never `ownAttrs()`. Post-#246 the common authoring shape is the map on a shared abstract declaration with N consuming fields inheriting it — an own-only read sees `undefined` on every one of those fields and silently emits a STRING codec into an INTEGER column. That is a silent data-corruption path, not a compile error. ADR-0039 is the governing rule; the two deliberately-own-only attrs (`@dbColumnType`, `@provided`) do NOT include `@intValueMap`. + +**Also required:** the shared-enum path emits ONE materialized type per declaration. A per-TYPE codec artifact (a lookup const/table named for the enum type) must be emitted ONCE per shared declaration, not once per consuming field — the Kotlin plan's `${enumClassName}_TO_INT` shape collides under sharing. TS's per-entity `ENTITY_FIELD_TO_INT` naming is per-field and does NOT collide, so Task 5's naming is safe as written; keep it that way deliberately rather than by accident. + +**Amendment 2 — three persistence surfaces are missing from Tasks 1-6.** Each is a real, verified gap; they are added as Tasks 7-9 below. + +**Amendment 3 — Task 6 breaks the other four ports on landing.** It adds `intEnumVal` to the SHARED `persistence-conformance` corpus, which every port runs. Until each port's codec ships, their round-trip lanes go red. Run Task 6 LAST, and treat the resulting cross-port red as expected and tracked — or hold Task 6 until the C#/Java+Kotlin/Python plans are ready to land in the same train. Per the release ruling, `@intValueMap` must NOT reach a published registry while inert, so the whole program merges as one train anyway. + +**Amendment 4 — array-of-enum (`@isArray`) is under-specified.** D7 says an int-backed array is `integer[]`. Task 1 covers the column type, but Task 5's codec must encode/decode ELEMENT-WISE, and the existing enum `CHECK` is skipped for arrays (`buildChecks` returns early on `field.resolvedIsArray()`), so array membership stays app-level exactly as it is for string-backed arrays. Add an explicit array case to Task 5's tests rather than leaving it implied. + +--- + +### Task 1: migrate-ts — `integer` column type for int-backed enums + +**Files:** +- Modify: `packages/migrate-ts/src/expected-schema.ts` +- Test: `packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: `FIELD_ATTR_INT_VALUE_MAP` (metamodel plan, already shipped). +- Produces: `buildExpectedSchema(...)` returns `{ kind: "integer", bits: 32 }` for a scalar int-backed enum column, `{ kind: "array", element: { kind: "integer", bits: 32 } }` for an array one — consumed by Task 2 (CHECK) and by `diff/index.ts`'s existing (unmodified) column-type comparison. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts +import { describe, test, expect } from "bun:test"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { loadFixture } from "../fixtures/load.js"; // match the existing helper used by expected-schema.test.ts + +describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { + test("scalar int-backed enum maps to integer, not text", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const col = snapshot.tables[0]!.columns.find((c) => c.name === "status")!; + expect(col.sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("string-backed enum (no @intValueMap) is unchanged", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const col = snapshot.tables[0]!.columns.find((c) => c.name === "status")!; + expect(col.sqlType).toEqual({ kind: "text" }); + }); + + test("array-of-int-backed-enum maps to integer[]", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "labels", "isArray": true, "@values": ["LOW","MEDIUM","HIGH"], "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const col = snapshot.tables[0]!.columns.find((c) => c.name === "labels")!; + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "integer", bits: 32 } }); + }); +}); +``` + +> Check `packages/migrate-ts/test/unit/expected-schema.test.ts`'s actual fixture-loading helper (`loadFixture`, or whatever it's really called) before finalizing — mirror its exact signature. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: FAIL — the first and third tests currently get `{ kind: "text" }`/`{ kind: "array", element: { kind: "text" } }`. + +- [ ] **Step 3: Add the scalar case to `subtypeToSqlType`** + +Edit `packages/migrate-ts/src/expected-schema.ts` — in `subtypeToSqlType` (the switch shown in research, ending at the `default: return { kind: "text" }` around line 1045), add a case immediately before `default`: + +```typescript + case FIELD_SUBTYPE_ENUM: + // @intValueMap present → this enum is int-backed (docs/superpowers/specs/ + // 2026-07-23-int-backed-enum-values-design.md D5/D6). ADR-0039: resolving — + // @intValueMap may be inherited via extends, same as @values. + return field.attr(FIELD_ATTR_INT_VALUE_MAP) !== undefined + ? { kind: "integer", bits: 32 } + : { kind: "text" }; +``` + +Add `FIELD_SUBTYPE_ENUM` and `FIELD_ATTR_INT_VALUE_MAP` to this file's existing `@metaobjectsdev/metadata` import block (both already exist as exports — `FIELD_SUBTYPE_ENUM` is likely already imported for other uses in this file; check before adding a duplicate). + +- [ ] **Step 4: Add the array-element case to `arrayElementSqlType`** + +Edit the same file's `arrayElementSqlType` function (research found the enum branch around line 949, grouped with `FIELD_SUBTYPE_URI`): + +```typescript + case FIELD_SUBTYPE_ENUM: + // enum[] stores as text[] (string-backed) or integer[] (int-backed, when + // @intValueMap is present); membership is app-level either way (no CHECK + // on array columns — see buildChecks). + return field.attr(FIELD_ATTR_INT_VALUE_MAP) !== undefined + ? { kind: "integer", bits: 32 } + : { kind: "text" }; + case FIELD_SUBTYPE_URI: + return { kind: "text" }; +``` + +(Remove the old combined `case FIELD_SUBTYPE_ENUM: case FIELD_SUBTYPE_URI: return { kind: "text" };` line and replace with the two separate cases above.) + +- [ ] **Step 5: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: PASS — all 3 tests green. + +- [ ] **Step 6: Run the full migrate-ts suite** + +Run: `cd server/typescript && bun test packages/migrate-ts` +Expected: all pass, no regressions (this step only adds a new case; every existing string-backed-enum test is untouched). + +- [ ] **Step 7: Commit** + +```bash +git add server/typescript/packages/migrate-ts/src/expected-schema.ts server/typescript/packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts +git commit -m "feat(migrate-ts): int-backed field.enum (@intValueMap) maps to integer, not text" +``` + +--- + +### Task 2: migrate-ts — numeric `CHECK` constraint for int-backed enums + +**Files:** +- Modify: `packages/migrate-ts/src/expected-schema.ts` +- Test: extend `packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: Task 1. + +- [ ] **Step 1: Write the failing test** + +Append to `expected-schema-enum-intvaluemap.test.ts`: + +```typescript +test("int-backed enum gets a numeric CHECK (unquoted literals)", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + const check = snapshot.tables[0]!.checks.find((c) => c.name === "orders_status_chk")!; + expect(check.expression).toBe('"status" IN (0, 5, 9)'); +}); + +test("int-backed array-of-enum gets NO check (array columns never get a CHECK)", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Ticket", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "labels", "isArray": true, "@values": ["LOW","MEDIUM"], "@intValueMap": { "LOW": 1, "MEDIUM": 2 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}}`); + const snapshot = buildExpectedSchema(root); + expect(snapshot.tables[0]!.checks.find((c) => c.name === "tickets_labels_chk")).toBeUndefined(); +}); +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: FAIL — the CHECK expression is currently the quoted-string form (`"status" IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')`), since `buildChecks` doesn't yet know about `@intValueMap`. + +- [ ] **Step 3: Update `buildChecks`'s enum branch** + +Edit `packages/migrate-ts/src/expected-schema.ts`'s `buildChecks` function (the block shown in research at lines 653-664): + +```typescript + // Enum membership check. + if (field.subType === FIELD_SUBTYPE_ENUM) { + const intValueMap = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (intValueMap !== undefined && typeof intValueMap === "object" && intValueMap !== null) { + // Int-backed: unquoted numeric literals, in the SAME member order as @values + // (cosmetic only — SQL IN() is order-independent, but matching declaration + // order keeps the emitted CHECK stable/predictable for diffing). + const raw = field.attr(FIELD_ATTR_VALUES); + const members: string[] = Array.isArray(raw) ? raw.map((v) => String(v)) : []; + const ints = members.map((m) => (intValueMap as Record)[m]); + const expression = `${qcol} IN (${ints.join(", ")})`; + checks.push({ name: `${tableName}_${col}_chk`, expression }); + } else { + const raw = field.attr(FIELD_ATTR_VALUES); + if (Array.isArray(raw) && raw.length > 0) { + const values = raw.map((v) => String(v)); + const expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; + checks.push({ name: `${tableName}_${col}_chk`, expression }); + } + } + } +``` + +- [ ] **Step 4: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` +Expected: PASS — all 5 tests in this file green. + +- [ ] **Step 5: Run the full migrate-ts suite** + +Run: `cd server/typescript && bun test packages/migrate-ts` +Expected: all pass, no regressions. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/migrate-ts/src/expected-schema.ts server/typescript/packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts +git commit -m "feat(migrate-ts): int-backed field.enum CHECK uses unquoted numeric literals" +``` + +--- + +### Task 3: migrate-ts — prove the migration-safety guard (no new code) + +**Files:** +- Test: `packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts` + +**Interfaces:** +- Consumes: Task 1 (column type), the existing `isWidening`/`applyStatus` mechanism (unmodified). + +- [ ] **Step 1: Write the real-engine test** + +```typescript +// packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts +import { describe, test, expect } from "bun:test"; +import Database from "better-sqlite3"; // match whatever driver this test dir's other integration tests already use +import { diff } from "../../src/diff/index.js"; +import { buildExpectedSchema } from "../../src/expected-schema.js"; +import { introspect } from "../../src/introspect/sqlite.js"; // match the actual introspect module path used by sibling tests +import { loadFixture } from "../fixtures/load.js"; +import { apply } from "../../src/act/sqlite.js"; // match the actual apply/act module path + +describe("SQLite enum backing-mode change — real-engine migration-safety guard", () => { + test("adding @intValueMap to an EXISTING string-backed enum column is BLOCKED by default", async () => { + const db = new Database(":memory:"); + const stringBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const initial = buildExpectedSchema(stringBackedRoot, { dialect: "sqlite" }); + // Apply the initial (string-backed) schema for real, so introspection sees a real + // existing text column — this is what makes the guard fire (there is no diff at + // all against an empty DB; the guard is specifically about an EXISTING column). + const firstDiff = diff({ actual: { tables: [], views: [] }, expected: initial, dialect: "sqlite" }); + await apply(db, firstDiff.changes); + + const intBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const expected = buildExpectedSchema(intBackedRoot, { dialect: "sqlite" }); + const actual = await introspect(db, "sqlite"); + const secondDiff = diff({ actual, expected, dialect: "sqlite" }); + + const typeChange = secondDiff.changes.find((c) => c.kind === "change-column-type" && c.column === "status"); + expect(typeChange).toBeDefined(); + expect(typeChange!.status.state).toBe("blocked"); + expect(typeChange!.status.blockedReason).toContain("allow.typeChange"); + expect(secondDiff.blocked).toContain(typeChange); + }); + + test("passing allow.typeChange unblocks it (operator opt-in, per D8 — no silent auto-migration)", async () => { + const db = new Database(":memory:"); + const stringBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const initial = buildExpectedSchema(stringBackedRoot, { dialect: "sqlite" }); + const firstDiff = diff({ actual: { tables: [], views: [] }, expected: initial, dialect: "sqlite" }); + await apply(db, firstDiff.changes); + + const intBackedRoot = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ]}} + ]}}`); + const expected = buildExpectedSchema(intBackedRoot, { dialect: "sqlite" }); + const actual = await introspect(db, "sqlite"); + const secondDiff = diff({ actual, expected, dialect: "sqlite", allow: { typeChange: true } }); + + const typeChange = secondDiff.changes.find((c) => c.kind === "change-column-type" && c.column === "status"); + expect(typeChange!.status.state).toBe("allowed"); + }); +}); +``` + +> This test's exact imports (`introspect`, `apply`, the SQLite driver) must match whatever `packages/migrate-ts/test/integration/sqlite-autoset-default.test.ts` (found during earlier investigation of this same package) actually uses — read that file first and mirror its setup/teardown and import paths exactly; the sketch above captures the SCENARIO, not necessarily the exact API surface. + +- [ ] **Step 2: Run to verify current behavior** + +Run: `cd server/typescript && bun test packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts` +Expected: Both tests should already PASS once Task 1 lands, since `isWidening`/`blockedReasonFor` need no changes — this step is confirming that expectation, not driving new implementation. If either test fails, the failure itself is the signal that D8's assumption (the existing generic mechanism already covers this) was wrong, and this task's scope needs to grow into an actual new guard (re-read `blockedReasonFor` in `packages/migrate-ts/src/diff/status.ts` and add a `change-column-type` sub-case specific to enum `text↔integer` transitions before proceeding). + +- [ ] **Step 3: Run the full migrate-ts suite** + +Run: `cd server/typescript && bun test packages/migrate-ts` +Expected: all pass. + +- [ ] **Step 4: Commit** + +```bash +git add server/typescript/packages/migrate-ts/test/integration/sqlite-enum-backing-mode-change.test.ts +git commit -m "test(migrate-ts): prove the existing allow.typeChange guard blocks enum backing-mode changes" +``` + +--- + +### Task 4: codegen-ts — Drizzle column mapper + +**Files:** +- Modify: `packages/codegen-ts/src/column-mapper.ts` +- Test: `packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts` + +**Interfaces:** +- Consumes: `enumValues(field)` from `packages/codegen-ts/src/enum-meta.ts` (existing) — this task adds a sibling `intValueMap(field)` reader to the same file. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts +import { describe, test, expect } from "bun:test"; +import { mapColumnType } from "../../src/column-mapper.js"; +import { loadFixture } from "../fixtures/load.js"; // match whatever helper column-mapper.test.ts already uses + +describe("column-mapper — int-backed field.enum (@intValueMap)", () => { + test("postgres: emits integer(...), not text(..., {enum:[...]})", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } } + ]}} + ]}}`); + const field = root.findObject("Order")!.fields().find((f) => f.name === "status")!; + const result = mapColumnType(field, "postgres"); + expect(result.fnName).toBe("integer"); + expect(result.fnOptions).toBeUndefined(); + expect(result.checkConstraint).toBe('"status" IN (0, 5)'); + }); + + test("sqlite: emits integer(...), not text(..., {enum:[...]})", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"], "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } } } + ]}} + ]}}`); + const field = root.findObject("Order")!.fields().find((f) => f.name === "status")!; + const result = mapColumnType(field, "sqlite"); + expect(result.fnName).toBe("integer"); + }); + + test("string-backed enum (no @intValueMap) is unchanged on both dialects", async () => { + const root = await loadFixture(`{ "metadata.root": { "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED"] } } + ]}} + ]}}`); + const field = root.findObject("Order")!.fields().find((f) => f.name === "status")!; + expect(mapColumnType(field, "postgres").fnName).toBe("text"); + expect(mapColumnType(field, "sqlite").fnName).toBe("text"); + }); +}); +``` + +> `mapColumnType`'s exact exported name/signature and its result shape (`fnName`/`fnOptions`/`checkConstraint`) are inferred from the research report's excerpts — confirm against the actual function signature in `column-mapper.ts` (search for `export function mapColumnType` or similar) before finalizing. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts` +Expected: FAIL — currently `fnName` is `"text"` in every case. + +- [ ] **Step 3: Add `intValueMap(field)` to `enum-meta.ts`** + +Edit `packages/codegen-ts/src/enum-meta.ts` — add a sibling to `enumValues`: + +```typescript +export function intValueMap(field: MetaField): Record | undefined { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + return typeof raw === "object" && raw !== null ? (raw as Record) : undefined; +} +``` + +- [ ] **Step 4: Update `column-mapper.ts`'s dialect switches** + +Edit both the SQLite switch (research: `:399`) and the Postgres switch (research: `:513-516`) to check `intValueMap(field)` before falling into the text case: + +```typescript + case FIELD_SUBTYPE_ENUM: { + fnName = intValueMap(field) !== undefined ? "integer" : "text"; + break; + } +``` + +(For Postgres, this REPLACES the current `default: fnName = "text"` grouping for enum — give enum its own case above `default` so it no longer falls into the shared default branch.) + +- [ ] **Step 5: Guard the literal-union `{enum:[...]}` option and the CHECK-constraint emission** + +Edit the existing blocks at research lines 527-532 and 663-674 — both are already gated on `fnName === "text"`/`subType === FIELD_SUBTYPE_ENUM && !isArray`, so once Step 4 makes `fnName` become `"integer"` for int-backed enums, the `{enum:[...]}` block already naturally skips (its `fnName === "text"` guard is now false). The CHECK-constraint block, however, unconditionally quotes string values — update it: + +```typescript +if (subType === FIELD_SUBTYPE_ENUM && !isArray) { + const map = intValueMap(field); + if (map !== undefined) { + const members = enumValues(field) ?? []; + const ints = members.map((m) => map[m]); + result.checkConstraint = `${dbName} IN (${ints.join(", ")})`; + } else { + const values = enumValues(field); + if (values !== undefined && values.length > 0) { + const list = values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", "); + result.checkConstraint = `${dbName} IN (${list})`; + } + } +} +``` + +- [ ] **Step 6: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts` +Expected: PASS — all 3 tests green. + +- [ ] **Step 7: Run the full codegen-ts suite** + +Run: `cd server/typescript && bun test packages/codegen-ts` +Expected: all pass, no regressions in existing golden-file snapshots (no existing fixture uses `@intValueMap`, so no snapshot should change). + +- [ ] **Step 8: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/enum-meta.ts server/typescript/packages/codegen-ts/src/column-mapper.ts server/typescript/packages/codegen-ts/test/templates/column-mapper-enum-intvaluemap.test.ts +git commit -m "feat(codegen-ts): Drizzle column-mapper emits integer(...) for int-backed field.enum" +``` + +--- + +### Task 5: codegen-ts — symbol↔int codec at the write/read boundary + +**Files:** +- Modify: `packages/codegen-ts/src/templates/zod-validators.ts` +- Modify: `packages/codegen-ts/src/templates/entity-file.ts` +- Modify: `packages/codegen-ts/src/templates/queries-file.ts` +- Test: `packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts` + +**Interfaces:** +- Consumes: `intValueMap(field)` (Task 4). +- Produces: for each int-backed enum field, a generated `_TO_INT`/`_FROM_INT` lookup pair (emitted in the entity file), a Zod `.transform()` on the Insert/Update schema (write), and a `decodeRow` helper applied at every read call site in `queries-file.ts` (read). + +**Note — this is a genuinely new pattern for this codebase.** Research confirmed no existing field type needs a wire type that differs from its DB storage type; every existing "conversion" (SQLite boolean `mode`, Drizzle `numeric`→`string`) is the driver's own built-in behavior, not codegen-authored translation. Read every file this task touches in full before editing — the excerpts below are the exact, verified current content at the cited lines, but the surrounding template has more call sites than shown (e.g. TPH polymorphic reads) that may need the identical treatment; this task covers the vanilla (non-TPH) path completely and flags TPH as a follow-up if discovered incomplete. + +- [ ] **Step 1: Write the failing tests** + +```typescript +// packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts +import { describe, test, expect } from "bun:test"; +import { runGen } from "../../src/runner.js"; // match whatever golden-file tests already use to drive a full gen pass +import { loadFixtureProject } from "../fixtures/load-project.js"; // match the existing golden-test harness + +describe("int-backed field.enum — generated codec", () => { + test("entity file emits a symbol<->int lookup pair", async () => { + const output = await runGen(await loadFixtureProject("enum-int-backed")); + const entityFile = output.find((f) => f.path.endsWith("Order.ts"))!.contents; + expect(entityFile).toContain('const ORDER_STATUS_TO_INT'); + expect(entityFile).toContain('const ORDER_STATUS_FROM_INT'); + }); + + test("insert schema transforms the symbol to its int before Drizzle sees it", async () => { + const output = await runGen(await loadFixtureProject("enum-int-backed")); + const entityFile = output.find((f) => f.path.endsWith("Order.ts"))!.contents; + expect(entityFile).toContain('.transform((v) => ORDER_STATUS_TO_INT[v])'); + }); + + test("read paths decode the int back to its symbol", async () => { + const output = await runGen(await loadFixtureProject("enum-int-backed")); + const queriesFile = output.find((f) => f.path.endsWith("Order.queries.ts"))!.contents; + expect(queriesFile).toContain("decodeOrderRow"); + }); + + test("string-backed enum (no @intValueMap) generates no codec at all", async () => { + const output = await runGen(await loadFixtureProject("enum-inline")); // the pre-existing string-backed fixture + const entityFile = output.find((f) => f.path.endsWith(".ts") && !f.path.includes("queries"))!.contents; + expect(entityFile).not.toContain("_TO_INT"); + expect(entityFile).not.toContain("_FROM_INT"); + }); +}); +``` + +> `runGen`/`loadFixtureProject` are placeholders for whatever harness `packages/codegen-ts/test/golden/*.test.ts` actually uses to drive a full generator pass against a fixture project — read one of those tests first and mirror its exact setup. You will also need a new fixture project under wherever `enum-inline`-equivalent codegen fixtures live for THIS package (distinct from the `fixtures/conformance/` ones — codegen-ts likely has its own `test/golden/` or `test/fixtures/` project layout) declaring an `Order` entity with an int-backed `status` field. + +- [ ] **Step 2: Run to verify failure** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts` +Expected: FAIL — none of this generated content exists yet. + +- [ ] **Step 3: Emit the lookup pair in `entity-file.ts`** + +Edit `packages/codegen-ts/src/templates/entity-file.ts` — for each field where `intValueMap(field) !== undefined`, emit (near the top of the file, alongside other per-entity constants): + +```typescript +function intBackedEnumConstants(entity: MetaObject, ctx: GenContext): Code[] { + const blocks: Code[] = []; + for (const field of entity.fields()) { + if (field.subType !== FIELD_SUBTYPE_ENUM) continue; + const map = intValueMap(field); + if (map === undefined) continue; + const constName = `${screamingSnake(entity.name)}_${screamingSnake(field.name)}`; + const toIntEntries = Object.entries(map).map(([k, v]) => ` ${JSON.stringify(k)}: ${v},`).join("\n"); + const fromIntEntries = Object.entries(map).map(([k, v]) => ` ${v}: ${JSON.stringify(k)},`).join("\n"); + blocks.push(code` +const ${constName}_TO_INT: Record = { +${toIntEntries} +}; +const ${constName}_FROM_INT: Record = { +${fromIntEntries} +}; +`); + } + return blocks; +} +``` + +Call `intBackedEnumConstants(entity, ctx)` from this file's main generator function and splice its output into the emitted file (alongside the existing Zod schema / type declarations — match how this file already assembles its output `Code[]` array). + +> `screamingSnake` is a naming helper this codebase may already have (check `naming.ts` or similar in `packages/codegen-ts/src/`) — use the existing helper rather than writing a new one if one exists. + +- [ ] **Step 4: Add the write-side `.transform()` in `zod-validators.ts`** + +Edit `packages/codegen-ts/src/templates/zod-validators.ts` — in the same loop shown in research (lines 189-197, the `autoSet` check), add an `else if` branch before the final `else`: + +```typescript + const autoSet = child.attr(FIELD_ATTR_AUTO_SET); + const map = child.subType === FIELD_SUBTYPE_ENUM ? intValueMap(child) : undefined; + + if (autoSet === AUTO_SET_ON_CREATE || autoSet === AUTO_SET_ON_UPDATE) { + insertFieldLines.push( + code` ${child.name}: z.string().optional().transform(() => new Date().toISOString())`, + ); + } else if (map !== undefined) { + const constName = `${screamingSnake(obj.name)}_${screamingSnake(child.name)}`; + insertFieldLines.push( + code` ${child.name}: ${zodFieldExpr(child, obj, ctx)}.transform((v) => ${constName}_TO_INT[v])`, + ); + } else { + insertFieldLines.push(code` ${child.name}: ${zodFieldExpr(child, obj, ctx)}`); + } +``` + +Apply the identical `else if (map !== undefined)` branch to whichever sibling loop builds the Update schema's field lines (search this file for the second occurrence of a field-line-building loop — Insert and Update schemas are built by parallel loops in this file per the earlier research on `zod-validators.ts`). + +- [ ] **Step 5: Add the read-side `decodeRow` helper and wire it into `queries-file.ts`** + +Edit `packages/codegen-ts/src/templates/entity-file.ts` — emit one decode helper per entity that has ANY int-backed enum field: + +```typescript +function decodeRowHelper(entity: MetaObject): Code | null { + const intBackedFields = entity.fields().filter( + (f) => f.subType === FIELD_SUBTYPE_ENUM && intValueMap(f) !== undefined, + ); + if (intBackedFields.length === 0) return null; + const assignments = intBackedFields.map((f) => { + const constName = `${screamingSnake(entity.name)}_${screamingSnake(f.name)}`; + return ` ${f.name}: ${constName}_FROM_INT[row.${f.name}],`; + }).join("\n"); + return code` +export function decode${entity.name}Row `${f.name}: number`).join("; ")} }>(row: T) { + return { + ...row, +${assignments} + }; +} +`; +} +``` + +Edit `packages/codegen-ts/src/templates/queries-file.ts` — wrap every read return with `decode${entityName}Row(...)` when the entity has at least one int-backed enum field (check once at the top of this file's generator function, e.g. `const hasIntBackedEnum = entity.fields().some((f) => f.subType === FIELD_SUBTYPE_ENUM && intValueMap(f) !== undefined)`), updating the `reads` block (research lines 165-175): + +```typescript +export async function ${findByIdFnName(entityName)}(db: Db, ${pkField}: ${pkType}): Promise<${entityName} | null> { + const [row] = await db.select().from(${viewVar}).where(${eqSym}(${viewVar}.${pkField}, ${pkField})).limit(1); + return row ${hasIntBackedEnum ? `? decode${entityName}Row(row)` : "??"} : null; +} + +export async function ${listFnName(entityName)}(db: Db, opts?: { limit?: number; offset?: number }): Promise<${entityName}[]> { + let q = db.select().from(${viewVar}).$dynamic(); + if (opts?.limit !== undefined) q = q.limit(opts.limit); + if (opts?.offset !== undefined) q = q.offset(opts.offset); + ${hasIntBackedEnum ? "return (await q).map(decode" + "${entityName}" + "Row);" : "return q;"} +} +``` + +> Write this as real template-string interpolation matching this file's actual `code\`...\`` tagged-template style — the snippet above is illustrative of the LOGIC (conditionally wrap with decode), not literal copy-paste, since the exact ternary-inside-template-literal syntax needs to match this codebase's `ts-poet`/`code` helper conventions. Also update `insertReturningView`'s second `db.select()` (research lines 232-251) with the same wrapping. + +- [ ] **Step 6: Run tests — confirm all pass** + +Run: `cd server/typescript && bun test packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts` +Expected: PASS — all 4 tests green. + +- [ ] **Step 7: Run the full codegen-ts suite** + +Run: `cd server/typescript && bun test packages/codegen-ts` +Expected: all pass, no golden-snapshot regressions (no existing fixture has `@intValueMap`). + +- [ ] **Step 8: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/templates/entity-file.ts server/typescript/packages/codegen-ts/src/templates/zod-validators.ts server/typescript/packages/codegen-ts/src/templates/queries-file.ts server/typescript/packages/codegen-ts/test/templates/enum-intvaluemap-codec.test.ts +git commit -m "feat(codegen-ts): symbol<->int codec for int-backed field.enum at the Drizzle boundary" +``` + +--- + +### Task 6: persistence-conformance — real round-trip + +**Files:** +- Modify: `fixtures/persistence-conformance/canonical/meta.fitness.json` +- Modify: `fixtures/persistence-conformance/queries/roundtrip-all-types.yaml` +- Verify: TS's real-engine round-trip test run + +**Interfaces:** +- Consumes: Tasks 1-5. + +- [ ] **Step 1: Add an int-backed enum field to the `AllTypes` entity** + +Edit `fixtures/persistence-conformance/canonical/meta.fitness.json` — add a sibling field next to the existing `enumVal` (found at line 243): + +```json +{ "field.enum": { "name": "intEnumVal", "@required": true, "@values": ["LOW", "MEDIUM", "HIGH"], "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } } }, +``` + +- [ ] **Step 2: Regenerate the canonical schema SQL** + +Run whatever command produces `fixtures/persistence-conformance/canonical/schema.postgres.sql` from `meta.fitness.json` (per CLAUDE.md, this file is TS-produced and committed) — likely `meta migrate` or a dedicated script; check `fixtures/persistence-conformance/README.md` for the exact regeneration command. +Expected: the regenerated SQL declares `int_enum_val integer NOT NULL CHECK (int_enum_val IN (1, 2, 3))` (naming per this project's `columnNamingStrategy`). + +- [ ] **Step 3: Add the round-trip scenario** + +Edit `fixtures/persistence-conformance/queries/roundtrip-all-types.yaml` — add `intEnumVal` to the existing `insert:`/`expect:` blocks (research found them at lines 74/98) as a sibling key: + +```yaml +insert: + enumVal: "MEDIUM" + intEnumVal: "MEDIUM" +expect: + enumVal: "MEDIUM" + intEnumVal: "MEDIUM" +``` + +(The wire value is the member STRING on both insert and expect, per this design's D6/D2 — the DB stores `2`, but every port's runtime translates transparently.) + +- [ ] **Step 4: Run TS's real-engine round-trip test** + +Run: `cd server/typescript && bun test packages/migrate-ts -t roundtrip` (or whichever package actually runs the `persistence-conformance` corpus against TS — check `fixtures/persistence-conformance/README.md` for the exact per-port runner) +Expected: PASS — `intEnumVal: "MEDIUM"` round-trips through insert → Postgres `2` → decoded back to `"MEDIUM"` on read. + +- [ ] **Step 5: Commit** + +```bash +git add fixtures/persistence-conformance/canonical/meta.fitness.json fixtures/persistence-conformance/canonical/schema.postgres.sql fixtures/persistence-conformance/queries/roundtrip-all-types.yaml +git commit -m "test(persistence-conformance): int-backed field.enum round-trips through TS's real runtime" +``` + +--- + +### Task 7: migrate-ts + codegen-ts — `@default` must lower to the INT literal + +**Why (verified 2026-08-12):** `buildColumn` reads `@default` and, for a string, emits `col.default = { kind: "literal", value: "DRAFT" }` (`packages/migrate-ts/src/expected-schema.ts:943-949`). On an int-backed enum that produces `DEFAULT 'DRAFT'` on an `integer` column — un-appliable DDL, and permanent false drift on any DB that somehow has it. The enum-member `@default` is already validated as a member of `@values` (Check 5 / FR-011), so the mapping is always available. + +**Files:** +- Modify: `packages/migrate-ts/src/expected-schema.ts` (`buildColumn`) +- Modify: `packages/codegen-ts/src/column-mapper.ts` (Drizzle `.default(...)` emission — same defect, same fix) +- Test: `packages/migrate-ts/test/unit/expected-schema-enum-intvaluemap.test.ts` (extend Task 1's file) + +- [ ] **Step 1: Failing test** — an int-backed enum with `@default: "DRAFT"` and `@intValueMap: {DRAFT: 0, …}` must yield `col.default === { kind: "literal", value: "0" }`, NOT `"DRAFT"`. Add the string-backed control asserting `"DRAFT"` is unchanged. +- [ ] **Step 2:** In `buildColumn`, when the field is an int-backed enum, map the `@default` member symbol through the RESOLVING `@intValueMap` before building the descriptor. An authored default that is not a member is already a load-time error — do not re-validate, but do not silently pass an unmapped value through either; throw a codegen error naming the field if the lookup misses (defensive, unreachable). +- [ ] **Step 3:** Same in `column-mapper.ts` so the Drizzle `.default()` and the DDL agree — a mismatch is exactly the class of drift `meta verify` exists to catch. +- [ ] **Step 4:** Run `bun test packages/migrate-ts packages/codegen-ts`. +- [ ] **Step 5: Commit** — `fix(migrate-ts,codegen-ts): int-backed enum @default lowers to its integer literal` + +--- + +### Task 8: runtime-ts + codegen-ts — the filter path must encode symbol→int — **DONE** + +**Outcome (2026-08-14), and it diverged from the plan in both directions.** + +**Steps 1-3 turned out to be unnecessary.** The premise — that `parseFilterParams` binds a +raw member symbol against an integer column — is false once Task 5's Drizzle `customType` +is in the column definition: the comparison value goes through `toDriver` and encodes for +free (empirically confirmed). The `dateValues` precedent was NOT followed; `FilterFieldRule` +is unchanged, and the parser stays metadata-free as intended. + +**Step 4 was the whole task, and its optimistic reading was wrong.** The existing gating +does NOT exclude `like` and structurally cannot: `opsForSubType` is keyed by subtype and +only ever sees `"enum"`, so the generated allowlist offered `like` on an int-backed field +byte-identically to a string-backed one. The band is a property of the FIELD. + +Fixed in `e8dca0b4d` as **one loader rule per port**, not five codegen filters — the #210 / +`@objectRef` precedent — so an authored `attr.filter` / dataGrid `@filter` using `like` on +an int-backed enum fails at LOAD, not later at the SQL layer: + +- TS `opsForField` (query-constants) · Java `FilterOps.opsForField` · C# + `QueryConstants.OpsForField` · Python `ops_for_field` + `ops_for_field_ordered` · Kotlin + reuses the JVM band through its own generator call site. +- `opsForSubType` is deliberately KEPT for the one caller with no field in hand (the + expression grammar's declared operand type). +- C#'s codegen carried its OWN duplicate per-subtype band table; it was **deleted** rather + than extended. +- Gated cross-port by a new `fEnumInt` case in `fixtures/conformance/filter-ops-matrix` + (`field.filter-ops` was already a field-level capability in all five ports, so no runner + change was needed). + +**Task 8b (NOT in the original plan) — the view-DDL blocker.** Probing Step 4 surfaced a +strictly worse bug: a projection row-scope `@filter` (#207) and an `origin.aggregate +@filter` render as literal SQL TEXT and never touch Drizzle, so the customType does nothing +for them. Both emitted `WHERE p.status = 'PUBLISHED'` against an `integer` column — +rejected by Postgres at CREATE VIEW time, aborting the migration. A `meta migrate` blocker, +affecting every operator rather than just `like`. Fixed in `2fd177f2d`; note it needed BOTH +`resolveViewFilter` and the separate `resolveAggregateFilter`. Gated by 10 unit tests plus a +real-Postgres apply-and-converge test, itself verified load-bearing by disabling the encode +and confirming red. + +**Durable lesson for the other three port plans:** a column-level codec seam does not reach +anywhere the port renders SQL text by hand. Each port needs the encoding in both places. + +--- + +### Task 9: codegen-ts — TPH per-subtype read schemas must decode — **DONE (no product change)** + +**Outcome (2026-08-14).** The premise is false, and for a structural reason worth +recording: `renderTphSubtypeReadSchema` does NOT see raw DB rows. Every TPH path goes +through Drizzle — `db.select()` for both the polymorphic and per-subtype reads, +`eq(auths.type, "Bridge")` for the subtype predicate, `.values()` for the insert, and +the routes tier's `discriminatorCond` (`runtime-ts/src/drizzle-fastify/index.ts:94`) +likewise. So Task 5's `customType` encodes and decodes **at the column**, and every +schema only ever sees member symbols. `z.literal("Bridge")` and `parseAuth`'s `z.enum` +head parse are correct exactly as emitted; there is no decode to wire and no lookup to +reuse. + +**Step 3 — the discriminator case is DECIDED: SUPPORTED, not rejected.** An int-backed +enum used AS a TPH discriminator needs no special handling for the same reason: the +discriminator column gets the same `customType`, its `CHECK` lists unquoted integers, +and every comparison against it is a Drizzle `eq`. The documented fallback (reject +`@intValueMap` on a discriminator with a named loader error) was NOT taken. + +**Verified by running it, not by reading it** (`da535f95d`) — #203/#229 is the precedent +for TPH being a separate code path everyone assumes is covered, and the 0.15.21 line is +what "the generated source looks right" is worth. Seven real-Postgres tests in +`integration-tests/test/enum-intvaluemap-pg.test.ts` over a hierarchy whose discriminator +is int-backed 1/2 and which carries a second int-backed enum (0/7, so the zero member is +live): DDL applies + converges; the discriminator column is `integer` with an integer +CHECK; a generated per-subtype create stores both enums as integers (asserted with raw +SQL, bypassing the codec); the per-subtype read schema decodes a raw-SQL-inserted integer +row; the per-subtype filter compares the integer; the polymorphic read dispatches on the +decoded value; and find-by-id is discriminator-scoped (a Copay row asked for as a Bridge +must MISS). + +**Two things the run surfaced, both pre-existing and both identical for a string-backed +enum** — neither is int-backing-specific, neither was changed: migrate-ts names the check +constraint `auths_type_chk` while codegen's Drizzle `check()` uses `chk_auths_type`; and a +TPH `InsertSchema` requires its `z.literal` discriminator (the ROUTES tier is what +omits it and re-adds it from the URL). + +--- + +## After this plan lands + +TS is the reference port; the same shape (DDL/column-type dispatch → runtime codec → persistence-conformance round-trip) repeats in the C#, Java+Kotlin, and Python persistence plans, each adapted to that port's own ORM/codec idiom (EF Core `HasConversion`, OMDB `JdbcFieldCodec`/Exposed `customEnumeration`, Python `ObjectManager` coercion). Only TS needed a genuinely new "wire type differs from storage type" pattern — the other ports' `MetaField`-level runtime access already made a symbol↔int translation point available without inventing new template plumbing. + +**The other three plans are deliberately NOT amended yet** (ruling, 2026-08-12). They are rewritten from what THIS execution actually learns, not from paper analysis — TS owns the schema layer (ADR-0015), so the hard questions (integer DDL, CHECK evolution, filter lowering, migration gating against a real engine) can only be *answered* here. Amending all four up front would bake speculation into three ports that would then be re-amended anyway. + +Known port-specific defects already identified, to fold in during that rewrite: +- **C#** — the array branch emits `ElementType().HasConversion()` unconditionally, ignoring `@intValueMap` (violates D7); and its per-entity `EnumTypeName` naming needs re-checking against FR-019's shared/provided materialization. +- **Java/Kotlin** — Kotlin's per-package `${enumClassName}_TO_INT` support-file emission collides under a shared enum (two consuming fields → two same-named top-level `val`s, even with identical maps: the emitter iterates `(class, field)` pairs with no dedupe). Emit per TYPE, once. A `@provided` Kotlin enum additionally needs its class imported into the support file. Java's `hasMetaAttr(name)` defaults to `includeParentData=true` and DOES resolve through `extends` (verified) — so its codec read is correct by default, but keep it that way deliberately. +- **Python** — the write branch's `int_value_map[value]` raises `KeyError` on a non-member (should be a clean validation error) and `TypeError` on an array-of-enum value (a list is unhashable); D7 array handling is absent entirely. The query/WHERE path is unaddressed (same class as Task 8). +- **Every port, from Task 9's result** — if a port's TPH surface (and its read/filter/insert + paths generally) goes through its ORM, a COLUMN-level codec seam makes int-backing work + with no TPH-specific code at all. Prefer that seam over a query-layer one and the TPH + fan-out cost drops to zero. Where a port instead hand-builds SQL for TPH (a raw + discriminator predicate, a hand-written `WHERE type = ?`), it needs the encoding + explicitly — the same split as Task 8b's view bodies. +- **All ports** — a column-level codec seam (the TS `customType`, EF Core `HasConversion`, + `JdbcFieldCodec`, Exposed `customEnumeration`) does NOT reach anywhere the port renders + SQL **text** by hand — view bodies above all. TS needed the member→integer encoding in + BOTH places (see Task 8b); assume every port does. +- **All ports** — `@provided` + `@intValueMap` is a REAL adopter case, not an edge case: ADR-0026's motivating example is literally a hand-written enum with int backing. Materialization is suppressed; the codec is NOT. Every port must map by member SYMBOL through the metadata map, never through the provided native type's own underlying integer values (a hand-written `ContactMethod.Email = 3` with `@intValueMap {Email: 1}` must store `1`). C#'s name-keyed dictionary gets this right by construction but has no test pinning it. diff --git a/docs/superpowers/specs/2026-05-23-enum-datatype-design.md b/docs/superpowers/specs/2026-05-23-enum-datatype-design.md index 0f3f4db67..494fbd5ba 100644 --- a/docs/superpowers/specs/2026-05-23-enum-datatype-design.md +++ b/docs/superpowers/specs/2026-05-23-enum-datatype-design.md @@ -1,7 +1,7 @@ # Design: `field.enum` — first-class enum datatype **Date:** 2026-05-23 -**Status:** Implemented across TS, C#, Java, Python (2026-05-23) +**Status:** Implemented across TS, C#, Java, Python, Kotlin (TypeScript int-backed in progress) **Author:** Doug Mealing (with Claude) ## Problem @@ -33,9 +33,7 @@ metamodel-as-spine payoff. A per-language regex is strictly worse output. ## Non-goals (out of scope) -- **Integer-backed enums** (where a member's symbol name differs from a stored number). - This needs per-member symbol→value assignment and is materially more complex; deferred - to a later design. v1 members are symbols stored as their own string. +- **Integer-backed enums** — Now supported via the `@intValueMap` attribute. See "Int-backed enum storage" below. - **Display labels.** Human-facing labels for members (e.g. `DRAFT` → "Draft") belong to the presentation/view layer, not the enum datatype. The enum stays a pure domain concept. - **Native Postgres `CREATE TYPE ... AS ENUM`.** Breaks PG/SQLite parity and is a @@ -68,9 +66,11 @@ metamodel-as-spine payoff. A per-language regex is strictly worse output. - **D3 — Members declared via `@values`.** A required string array on the `field.enum`. Declaration order is significant (it is the canonical member order for every port). -- **D4 — v1 is string-backed; int-backed deferred.** Each member's symbol *is* its stored - and transmitted string value. No `@backing` knob. Int-backed enums are a future codegen - mapping of the *same* subtype, not a new datatype — the model already accommodates them. +- **D4 — v1 is string-backed; int-backed via `@intValueMap`.** Each member's symbol *is* its stored + and transmitted string value by default. The optional `@intValueMap` attribute switches a field to + integer storage while preserving the string wire format and generated enum type. Keys must match + `@values` exactly; values must be unique integers. The generated type and wire format are unchanged + in every language. - **D5 — DB representation: `varchar` + `CHECK`.** Portable across Postgres and SQLite; adding/removing a member is a cheap CHECK swap. Native PG enum is explicitly out (see @@ -241,7 +241,8 @@ the metadata layer. The empty-`@values` fixture, surfaced during review, replace ## Remaining follow-ups -- Integer-backed enums (per-member symbol→value) — no current consumer. +- ~~Integer-backed enums (per-member symbol→value)~~ — **DONE via `@intValueMap` (2026-08)**. + TypeScript persistence layer complete; other ports pending. See "Int-backed enum storage" below. - Non-identifier-safe member strings (kebab-case, leading digit, etc.) needing a symbol↔stored-value mapping — no current consumer. - Display labels (presentation/view layer) — no current consumer. @@ -249,3 +250,36 @@ the metadata layer. The empty-`@values` fixture, surfaced during review, replace `varchar`+`CHECK` covers current needs. - Extend the EF Core compile-check to the **Routes/ASP.NET** surface (needs the `Microsoft.AspNetCore.App` shared framework) — optional, lower priority. + +## Int-backed enum storage (2026-08) + +**Problem:** Some databases (especially legacy schemas) store enums as integers rather than strings, +but the generated enum type and wire format should remain string-based for type safety. + +**Solution:** The `@intValueMap` attribute on `field.enum`: + +```json +{ "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 1, "ARCHIVED": 2 } +}} +``` + +**Behavior:** + +- **Loader validation:** Keys must exactly match `@values`; values must be unique integers. +- **Generated type:** Unchanged — still a union/enum of string symbols. +- **Wire format:** Unchanged — members still transmit as strings. +- **Database column:** Integer instead of `varchar`; CHECK constraint enforces valid integers. +- **ORM mapping:** Custom codec/convertor layer translates between in-memory enum and DB integer. +- **Filter operators:** `like` is unsupported on int-backed enum fields (meaningless against integers). + +**Cross-port status:** + +- TypeScript: Complete (Drizzle customType, Zod, migrate, filter bands, TPH discriminators) +- Java/Kotlin: Metamodel registered, codecs pending +- C#: Metamodel registered, EF Core converters pending +- Python: Metamodel registered, codecs pending + +See plan docs: `docs/superpowers/plans/2026-07-23-int-backed-enum-values-*.md` diff --git a/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md b/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md index 5c828a67e..f456cae81 100644 --- a/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md +++ b/docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md @@ -145,4 +145,4 @@ The corpus is the oracle: TS reference green first, then each port matches. ## Cross-references - [ADR-0026](../../../spec/decisions/ADR-0026-shared-and-provided-named-types.md) — `@provided` as a cross-type provenance flag (enums + value objects); shared vs provided orthogonal; Option 2 upgrade path. - [ADR-0001](../../../spec/decisions/ADR-0001-cross-language-type-binding.md) — metadata→native type binding is per-port build-time config, not metadata. -- [enum datatype design](2026-05-23-enum-datatype-design.md) — D6 abstract-enum reuse; deferred int-backed/display-label/native-PG-enum. +- [enum datatype design](2026-05-23-enum-datatype-design.md) — D6 abstract-enum reuse; int-backed now supported via `@intValueMap`. diff --git a/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md new file mode 100644 index 000000000..fdce8ff9d --- /dev/null +++ b/docs/superpowers/specs/2026-07-23-int-backed-enum-values-design.md @@ -0,0 +1,300 @@ +# Design: int-backed `field.enum` values (`@intValueMap`) + +**Date:** 2026-07-23 +**Status:** Proposed +**Author:** Doug Mealing (with Claude) +**Supersedes deferral in:** `docs/superpowers/specs/2026-05-23-enum-datatype-design.md` (D4 — +"Integer-backed enums... deferred to a later design") + +## Problem + +`field.enum` (shipped 2026-05-23) is string-backed only: each member's symbol *is* its +stored and transmitted string value (`varchar` + `CHECK`). There is no way to declare +that an enum's members persist as integers in the database — the deferred v1 non-goal. +A consumer that wants a compact/indexable integer column (e.g. matching an existing +legacy schema, or optimizing storage/index size for a high-cardinality table) has no +metadata-driven path today; they would have to hand-roll a converter outside the +generated code, defeating the "declare once → idiomatic type + DB constraint in every +language" payoff that is `field.enum`'s whole reason to exist. + +**Provenance (recorded 2026-08-13).** This is a **live requirement from a downstream +consumer**, not a speculative feature — the adopter needs int-backed enum maps to model +an existing integer-coded schema. Recorded here explicitly because the requirement was +absent from this repo's issues and roadmap, which made the work look demand-less on +review and nearly got it de-scoped. If the driving need is purely *storage size* rather +than matching an existing integer encoding, note that native Postgres `CREATE TYPE … +AS ENUM` is the better instrument (also 4 bytes, keeps string semantics, needs no codec +in any port) — it is deferred for PG/SQLite parity reasons, see the enum design's D-list. +Int-backing's unique value is matching an encoding you do not control. + +## Goals + +1. Let a `field.enum` declare an explicit, possibly-sparse, per-member integer value for + database storage. +2. Every language's *generated native type* (TS union, C# `enum`, Java/Python/Kotlin + equivalents) and the *wire format* (JSON API payloads) stay **exactly as they are + today** for both string- and int-backed enums — this is a persistence-layer-only + concern, invisible to any client of the generated code. +3. Avoid positional/index correspondence between two parallel structures — the + documented weak point of the one ecosystem (OpenAPI's `x-enum-varnames` extension) + that uses that shape (see Prior art below). A metadata-only reorder of an unrelated + array must never be able to silently corrupt which stored int a symbol means. +4. Ship as a strictly additive, opt-in overlay on the existing `field.enum` contract — + zero change to any currently-shipped string-backed enum. +5. Preserve cross-language conformance: the new vocabulary is identical across + TS / C# / Java / Python / Kotlin, gated by `registry-conformance`. + +## Non-goals (out of scope) + +- **Toggling an existing, data-bearing field between string- and int-backed.** That is a + genuine column-type-changing migration (`varchar` → `integer`, with a data cast), not a + cheap `CHECK` swap. migrate-ts refuses to auto-generate it (see Migration safety). +- **Value aliasing** (two members sharing one stored int, à la Python `enum` without + `@unique`, or protobuf's `allow_alias`). No current consumer need; duplicate values are + a load-time error (matches protobuf's default, the strictest of the surveyed + frameworks — see Prior art). +- **Display labels** (still deferred from the original enum design; unrelated to storage). +- **Native Postgres `CREATE TYPE ... AS ENUM`** (still out — same PG/SQLite parity and + migration-footgun reasoning as the original design's D5/Non-goals). +- **A `@kind` discriminator.** Considered and rejected — see Decision D2. + +## Prior art + +Researched before finalizing the shape (frameworks that let a symbol carry an explicit, +possibly-sparse integer value): + +| Framework | Shape | Uniqueness enforced? | +|---|---|---| +| Ruby on Rails `ActiveRecord::Enum` | name-keyed hash: `enum :status, { draft: 0, published: 5 }` | No (hash keys unique by construction; duplicate *values* not checked pre-7.1) | +| Protocol Buffers | inline per-value assignment: `enum Status { DRAFT = 0; PUBLISHED = 5; }` | **Yes** — compile error unless `allow_alias = true` | +| C# (native language enum) | inline per-value assignment: `enum Status { Draft = 0, Published = 5 }` | No | +| Django `IntegerChoices` | inline per-member `(value, label)` tuple | No (Python `enum` aliasing unless `@unique`) | +| graphql-js | name-keyed map (server-side only): `values: { DRAFT: { value: 0 } }` | No | +| OpenAPI (`x-enum-varnames`, NSwag `x-enumNames`) | **parallel array** of names alongside the plain `enum: [...]` values array | No — and explicitly called out as the fragile, index-alignment-error-prone approach | +| JPA `@Enumerated` | no explicit-int support at all; ordinal-or-string only | n/a | +| Prisma | no int-backed enums; long-standing open feature request | n/a | + +**Takeaway:** name-keyed pairing (a literal map, or inline per-value assignment — the +same shape in declaration-order form) is the dominant, safe pattern. Parallel arrays are +the one shape the survey found, and it is the ecosystem's own acknowledged weak point. +This is why `@intValueMap` is a map, not a second array parallel to `@values`. + +## Decisions + +- **D1 — `@intValueMap`, not a second array.** A new object-shaped attribute on + `field.enum`: `@intValueMap: { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 }`. Keys are + member symbols, values are integers. Optional; absence keeps today's string+`CHECK` + default unchanged. Rejected the parallel-array shape (`@intValues: [0, 5, 9]`, + index-matched to `@values`) specifically because index correspondence lets a + metadata-only reorder of `@values` silently reassign stored meanings with no + validation able to catch it — the exact failure mode the OpenAPI-ecosystem prior art + confirms is real. Named `@intValueMap` (not `@valueMap`) so the attribute name states + its value type — no framework surveyed uses this exact name, but it unambiguously + states its own contents, which every alternative in the survey (`values`, `x-enum-varnames`) + does not. + +- **D2 — No `@kind` discriminator.** Presence of `@intValueMap` alone signals int-backed + persistence. Per ADR-0037, `@kind` is chartered for structural variants that change a + subtype's *generated shape* — and per Goal 2, the generated shape (native + union/enum type, wire format) is identical in both modes; only the DB persistence + codec differs. Using `@kind` here would stretch its charter for a distinction that + produces no codegen-shape difference, and would add a second attribute that must be + kept in sync with the first for no benefit. + +- **D3 — `@values` is untouched.** It remains required, and remains the sole source of + canonical member order (used for TS union member order, C# `enum` declaration order, + etc., in both string- and int-backed modes). `@intValueMap` never replaces it and + carries no ordering significance of its own (object key order is not relied upon). + +- **D4 — Validation.** At load time (own-only, eager-throw per the pattern the original + `field.enum` design used for Java's post-load `ValidationPhase`): + - `@intValueMap`'s key set must be **exactly** the member set in `@values` — no + missing member, no extra key. + - Every value must be a JSON integer (reject strings, floats, booleans, `null`). + - No two keys may share the same value (rejected outright — no alias opt-in; matches + protobuf's default, the strictest surveyed). + - `@intValueMap` is invalid on any subtype other than `enum` (mirrors `@values`' own + subtype restriction). + +- **D5 — DB representation: `integer` + `CHECK`.** `CHECK (col IN (0, 5, 9))`, portable + across Postgres and SQLite exactly like the string-backed `varchar` + `CHECK` — adding + or removing a member (within the same backing mode) is the same cheap `CHECK` swap + migrate-ts already supports for string-backed enums. + +- **D6 — Codec boundary: persistence only.** Every language's generated *native* type + (TS union + `z.enum`, C# `enum`, Java/Python/Kotlin equivalents) is **byte-identical** + between string- and int-backed modes. Each port's persistence layer builds a + bidirectional symbol↔int lookup table from `@intValueMap` at codegen/build time (never + runtime reflection, per ADR-0001) and translates at the DB read/write boundary only — + e.g. C# gets a custom `HasConversion` built from the table instead of + `HasConversion()`; TS/Drizzle gets an explicit encode/decode pair around the + Kysely column instead of a passthrough string column. The wire format (JSON API + payloads) is the member string in both modes, unchanged from the original design's + cross-language contract. + + **A stored integer that maps to no member THROWS, in all five ports.** The row holds + data the model says is impossible — a hand-written INSERT, or a member removed without a + migration — and neither alternative is honest. Surfacing the raw value hands the caller a + "member" that is not one, and it is not even representable in C#, Kotlin or TypeScript, + which type the property as a closed enum; returning null hides the corruption behind a + nullable column. C# reaches this through a generated static helper called from the + provider→model lambda: CS8188 bans a throw-EXPRESSION inside an expression tree, but a + method CALL is legal there and the throw happens in the helper's ordinary body. The + WRITE side is the mirror case and is deliberately left to the database: an unmapped + symbol binds unchanged, so the column type and its `CHECK` reject it. + +- **D7 — Int-backing is scalar-only; `@isArray` + `@intValueMap` is a LOAD ERROR.** + `ERR_ENUM_INT_VALUE_MAP_ARRAY`, in all five ports. An array-of-enum stays + string-backed. + + This reverses D7 as originally written ("array-of-enum composes unchanged"), which + assumed the element codec would fall out of the scalar one. It does not. Int-backing is + a persistence-layer CODEC, and each port's codec seam is scalar by construction: OMDB's + `EnumCodec` and Kotlin's `customEnumeration` bind one value; Python's `ObjectManager` + tests `value in int_map`, which is false for a list, so it binds the symbol LIST into an + `integer[]`; and TypeScript's sqlite branch serializes an array as JSON text before the + enum case is reached, storing symbols. Only TS/Postgres (`customType(...).array()`) and + C# (`PrimitiveCollection().ElementType()`) compose — and two ports composing while four + silently get it wrong is not a feature, it is the `field.byte`/`field.short`/ + `field.class` mistake: vocabulary that reads as supported and is not. + + Rejected at LOAD rather than fixed per-port because there is no consumer need for the + array form (the provenance is a scalar integer-coded column), and because the guarantee + a load error gives — *identical behaviour in every port* — is the one that was missing. + Both halves are read RESOLVING in every loader: post-#246 the map must live on the + shared abstract declaration while `isArray` is declared by the consuming field, so an + own-only read would see the two halves on different nodes and never fire. Gated by + `error-enum-intvaluemap-array` (own map) and `error-enum-intvaluemap-array-inherited` + (the canonical shared-enum shape). + +- **D8 — Migration safety: no auto-recast.** Adding `@intValueMap` to a *new* field (no + existing column) is a normal create. Adding or removing `@intValueMap` on a field that + **already has a table/column** is a backing-mode change — migrate-ts detects it in the + diff and surfaces a manual-intervention-required error rather than auto-generating a + `varchar`↔`integer` `ALTER COLUMN TYPE`, consistent with how this codebase already + refuses to auto-generate other genuinely risky migrations (e.g. the auto-allowed + drop-view guard). A consumer needing this must do the two-step backfill migration by + hand (add new column, backfill, swap, drop old) — no new safe-recast subsystem is being + built for v1, since there is no current consumer need for that path specifically. + +## Metamodel addition + +``` +FIELD_ATTR_INT_VALUE_MAP = "intValueMap" // @intValueMap: object (optional, `enum` subtype only) +``` + +### Authoring + +```yaml +field.enum: + name: status + values: ["DRAFT", "PUBLISHED", "ARCHIVED"] + intValueMap: { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 } +``` + +Canonical JSON: `{ "field.enum": { "name": "status", "@values": [...], "@intValueMap": { +"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }`. + +Reuse via abstract field + `extends` works exactly as it does for `@values` today — an +abstract `field.enum` may carry both `@values` and `@intValueMap`; concrete fields +`extends` it and inherit both. + +## Codegen mappings + +| Concern | String-backed (unchanged) | Int-backed (new) | +|---|---|---| +| TS type | `type Status = "DRAFT" \| "PUBLISHED" \| "ARCHIVED"` | **identical** | +| TS validation | `z.enum([...])` | **identical** | +| C# type | `enum Status { DRAFT, PUBLISHED, ARCHIVED }` | **identical** | +| Wire (all langs) | the member string | **identical** | +| DB column | `varchar` | `integer` | +| DB constraint | `CHECK (status IN ('DRAFT', ...))` | `CHECK (status IN (0, 5, 9))` | +| C# persistence | EF Core `HasConversion()` | EF Core `HasConversion` via a generated symbol↔int table | +| TS persistence | passthrough string column | explicit encode/decode around the Kysely column, generated from `@intValueMap` | + +Java/Python/Kotlin: vocabulary + validation ship now; each port's persistence codec +follows the same table-driven pattern as C#/TS, in the same release (per the "all five +ports up front" scope call — no cross-port gap period for this feature, unlike the +original `field.enum` rollout). + +## Cross-language contract (must be identical across ports) + +- Attribute name: `intValueMap` (canonical JSON `@intValueMap`). +- Value shape: object, string keys (member symbols), integer values. +- Key-set-must-equal-`@values` and no-duplicate-value rules enforced identically in every + loader. +- Wire format is unaffected: the member string, on every endpoint, in both backing modes. + +## Conformance fixtures + +1. **`enum-int-backed`** — a `field.enum` with `@values` + `@intValueMap`; asserts DB + column is `integer` + int `CHECK`, and the native type in every port is unchanged from + the string-backed case. +2. **`error-enum-intvaluemap-array`** (negative) — `field.enum[]` + an own `@intValueMap` + → `ERR_ENUM_INT_VALUE_MAP_ARRAY` (D7: int-backing is scalar-only). +2b. **`error-enum-intvaluemap-array-inherited`** (negative) — the same rejection where the + map is INHERITED from a shared abstract enum and only `isArray` is declared locally. + This is the canonical authoring shape post-#246, and the case an own-only read misses. +3. **`error-enum-intvaluemap-key-mismatch`** (negative) — `@intValueMap` keys don't + exactly match `@values` members → load error. +4. **`error-enum-intvaluemap-non-int`** (negative) — a non-integer value in + `@intValueMap` → load error. +5. **`error-enum-intvaluemap-duplicate-value`** (negative) — two members share one int → + load error. + +Persistence-conformance: extend the existing round-trip write/read gate (the `AllTypes` +entity family, `fixtures/persistence-conformance/roundtrip-all-types.yaml`) with an +int-backed enum field, inserting/reading through each port's real runtime codec — not +just golden-snapshot codegen. + +## Testing + +- Metadata package (all 5 ports): load/validate unit tests for D4's rules, abstract + + extends inheritance of `@intValueMap`, the negative cases. +- Per-port codegen: DDL emission (`integer` + int `CHECK`), native-type-unchanged + assertion (byte-diff the generated union/enum type between a string-backed and + int-backed fixture — they must be identical modulo the field name). +- Per-port persistence: the symbol↔int codec round-trips through the real runtime/ORM + (not just golden snapshots) via the extended `roundtrip-all-types` scenario. +- migrate-ts: a real-engine test that adding `@intValueMap` to a field with no existing + column succeeds normally, and that adding/removing it on a field with an existing + column surfaces the manual-intervention error rather than silently altering the + column type. +- Conformance: the fixtures above run across every port (`registry-conformance` for the + new attribute + `ERR_*` codes, `persistence-conformance` for the round-trip). + +## Remaining follow-ups (explicitly out of scope for this design) + +- Safe backing-mode migration (varchar↔integer recast with data preservation) — no + current consumer; D8's manual path covers the only known need. +- **RE-mapping an existing member's integer is not understood by anything; what saves you + is incidental.** D8 covers ADDING or REMOVING `@intValueMap`, not changing a value inside + a map that stays present. Measured against the real diff rather than reasoned about: + + - **A remap that changes the rendered `CHECK` list** — which is every remap, since the + list renders in `@values` order — emits `drop-check` + `add-check`, and the `drop-check` + is **BLOCKED by default** (`allow.dropCheck`). So `meta migrate` refuses. That refusal + is a happy accident: it fires because dropping a `CHECK` is destructive, not because + anything recognises that the meaning of stored data just changed. + - **Once allowed, the migration only refreshes the constraint — it never touches the + data.** Moving a member to an int not already in the set (`DRAFT: 0` → `1`) then applies + a `CHECK` every existing `0` row violates, and the database refuses it: loud, at apply + time. But **swapping** two members' ints (`DRAFT: 0, PUBLISHED: 5` → `5, 0`) leaves the + admitted SET identical, so the new `CHECK` applies cleanly and every stored row has + quietly changed meaning. + - **One shape is invisible even to the diff:** a remap combined with a compensating + `@values` reorder renders a byte-identical `CHECK`, so the diff is EMPTY and no + migration is emitted at all. + + Nothing in the pipeline can see the meaning change in any of these: the column holds bare + integers, and neither introspection nor the committed schema snapshot records which member + an integer stood for. Closing it needs the mapping itself carried in gen-state or the + snapshot so a diff can compare member→int PAIRS rather than the value set — a design + decision, not a patch. No current consumer remaps; documented here rather than left + implied by D8's "migration safety" heading, and pinned by + `expected-schema-enum-intvaluemap.test.ts` so the accident that currently protects the + common case cannot be removed silently. +- Value aliasing (`allow_alias`-style opt-out of the duplicate-value rejection) — no + current consumer. +- Native Postgres `CREATE TYPE ... AS ENUM` — unrelated to this design, still deferred + per the original enum design's D5/Non-goals. diff --git a/fixtures/codegen-conformance/shared-provided-enum/input/meta.json b/fixtures/codegen-conformance/shared-provided-enum/input/meta.json index b355526f1..eb87f21f3 100644 --- a/fixtures/codegen-conformance/shared-provided-enum/input/meta.json +++ b/fixtures/codegen-conformance/shared-provided-enum/input/meta.json @@ -4,6 +4,7 @@ "children": [ { "field.enum": { "name": "Priority", "abstract": true, "@values": ["LOW", "MEDIUM", "HIGH"] } }, { "field.enum": { "name": "Currency", "abstract": true, "@provided": true, "@values": ["USD", "EUR", "GBP"] } }, + { "field.enum": { "name": "Money", "abstract": true, "extends": "Currency" } }, { "object.entity": { "name": "Ticket", "children": [ { "field.long": { "name": "id" } }, { "field.enum": { "name": "priority", "extends": "Priority" } }, @@ -16,6 +17,12 @@ { "field.enum": { "name": "priority", "extends": "Priority" } }, { "source.rdb": { "@table": "orders" } }, { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ] } }, + { "object.entity": { "name": "Invoice", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "money", "extends": "Money" } }, + { "source.rdb": { "@table": "invoices" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } ] } } ] } diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 694f731b2..8bd3a0dee 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -76,6 +76,7 @@ "ERR_SQL_BODY_WITH_UNMANAGED": "#208: a source.rdb declares both @sql (author-supplied body) and @unmanaged (DDL owned elsewhere). The two markers are the mutually exclusive non-default states of one DDL-ownership axis \u2014 contradictory on a single source.", "ERR_SQL_BODY_ON_WRITABLE_KIND": "#208: a source.rdb declares @sql with a writable @kind (\"table\", the default). A hand-written CREATE TABLE would bypass the column-diff machinery; tables are fully modeled or @unmanaged, never opaque-bodied.", "ERR_ORIGIN_UNDER_SQL_BODY": "#208: a host object whose read source carries @sql also declares an origin.*-bearing (derived) field, or (on object.projection) an @filter (#207) \u2014 two sources of truth for the same body (the synthesized derivation/outer-WHERE vs. the author's verbatim SQL). Fail-closed.", - "ERR_ENUM_EXTENDS_VALUES_CONFLICT": "A field.enum both extends a shared package-level abstract enum and declares its own @values. One shared enum type has one member set \u2014 the own @values would be silently dropped in codegen. Remove the own @values to inherit the shared set, or extend a concrete (non-shared) enum instead." + "ERR_ENUM_EXTENDS_VALUES_CONFLICT": "A field.enum both extends a shared package-level abstract enum and declares its own @values. One shared enum type has one member set \u2014 the own @values would be silently dropped in codegen. Remove the own @values to inherit the shared set, or extend a concrete (non-shared) enum instead.", + "ERR_ENUM_INT_VALUE_MAP_ARRAY": "A field.enum carries @intValueMap together with isArray=true. Int-backing is a persistence-layer codec and no port implements it element-wise over an array column, so the combination would silently persist member SYMBOLS into an integer array. An array-of-enum stays string-backed: drop @intValueMap, or make the field scalar." } } diff --git a/fixtures/conformance/enum-abstract-chained-extends/expected.json b/fixtures/conformance/enum-abstract-chained-extends/expected.json new file mode 100644 index 000000000..4c0f5a5c2 --- /dev/null +++ b/fixtures/conformance/enum-abstract-chained-extends/expected.json @@ -0,0 +1,52 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Currency", + "package": "acme", + "abstract": true, + "@values": [ + "USD", + "EUR" + ] + } + }, + { + "field.enum": { + "name": "Money", + "package": "acme", + "extends": "acme::Currency", + "abstract": true + } + }, + { + "object.entity": { + "name": "Invoice", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "amount", + "extends": "acme::Money" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json b/fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json new file mode 100644 index 000000000..fd5f5f5d1 --- /dev/null +++ b/fixtures/conformance/enum-abstract-chained-extends/input/meta.enums.json @@ -0,0 +1,19 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Currency", "abstract": true, "@values": ["USD", "EUR"] } }, + { "field.enum": { "name": "Money", "abstract": true, "extends": "acme::Currency" } }, + { + "object.entity": { + "name": "Invoice", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "amount", "extends": "acme::Money" } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed-inherited-values/expected.json b/fixtures/conformance/enum-int-backed-inherited-values/expected.json new file mode 100644 index 000000000..8b232a297 --- /dev/null +++ b/fixtures/conformance/enum-int-backed-inherited-values/expected.json @@ -0,0 +1,50 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Status", + "package": "acme", + "abstract": true, + "@intValueMap": { + "ARCHIVED": 9, + "DRAFT": 0, + "PUBLISHED": 5 + }, + "@values": [ + "DRAFT", + "PUBLISHED", + "ARCHIVED" + ] + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "status", + "extends": "acme::Status" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json b/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json new file mode 100644 index 000000000..24ba90cd8 --- /dev/null +++ b/fixtures/conformance/enum-int-backed-inherited-values/input/meta.enums.json @@ -0,0 +1,25 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Status", + "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status" } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed/expected.json b/fixtures/conformance/enum-int-backed/expected.json new file mode 100644 index 000000000..97b09cd0c --- /dev/null +++ b/fixtures/conformance/enum-int-backed/expected.json @@ -0,0 +1,42 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.enum": { + "name": "status", + "@intValueMap": { + "ARCHIVED": 9, + "DRAFT": 0, + "PUBLISHED": 5 + }, + "@values": [ + "DRAFT", + "PUBLISHED", + "ARCHIVED" + ] + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/enum-int-backed/input/meta.enums.json b/fixtures/conformance/enum-int-backed/input/meta.enums.json new file mode 100644 index 000000000..843c7e898 --- /dev/null +++ b/fixtures/conformance/enum-int-backed/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json b/fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json new file mode 100644 index 000000000..2de10d20a --- /dev/null +++ b/fixtures/conformance/error-enum-chained-extends-values-conflict/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json b/fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json new file mode 100644 index 000000000..9105a4bdb --- /dev/null +++ b/fixtures/conformance/error-enum-chained-extends-values-conflict/input/meta.enums.json @@ -0,0 +1,9 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Currency", "abstract": true, "@values": ["USD", "EUR"] } }, + { "field.enum": { "name": "Money", "abstract": true, "extends": "acme::Currency", "@values": ["USD", "EUR", "JPY"] } } + ] + } +} diff --git a/fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json new file mode 100644 index 000000000..bfa5c11ee --- /dev/null +++ b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json new file mode 100644 index 000000000..1bb47c7a1 --- /dev/null +++ b/fixtures/conformance/error-enum-extends-intvaluemap-conflict/input/meta.enums.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status", "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json new file mode 100644 index 000000000..dacdbb47d --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array-inherited/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_INT_VALUE_MAP_ARRAY", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[1]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json new file mode 100644 index 000000000..1924ebfa6 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array-inherited/input/meta.enums.json @@ -0,0 +1,25 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "field.enum": { + "name": "Status", + "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status", "isArray": true } }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json new file mode 100644 index 000000000..6b7767484 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ENUM_INT_VALUE_MAP_ARRAY", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-array/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-array/input/meta.enums.json new file mode 100644 index 000000000..590587edb --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-array/input/meta.enums.json @@ -0,0 +1,24 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Ticket", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "labels", + "isArray": true, + "@values": ["LOW", "MEDIUM", "HIGH"], + "@intValueMap": { "LOW": 1, "MEDIUM": 2, "HIGH": 3 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json new file mode 100644 index 000000000..a5f2e7920 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json new file mode 100644 index 000000000..d09dff04a --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-duplicate-value/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 0 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json new file mode 100644 index 000000000..a5f2e7920 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json new file mode 100644 index 000000000..45b545a17 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-key-mismatch/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json b/fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json new file mode 100644 index 000000000..a5f2e7920 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-non-int/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.enums.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['field.enum']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json b/fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json new file mode 100644 index 000000000..ae2b940f4 --- /dev/null +++ b/fixtures/conformance/error-enum-intvaluemap-non-int/input/meta.enums.json @@ -0,0 +1,23 @@ +{ + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "status", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { "DRAFT": "zero", "PUBLISHED": 5 } + } + }, + { "identity.primary": { "name": "id", "@fields": "id" } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json b/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json index 22dd427e0..e305a32cf 100644 --- a/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json +++ b/fixtures/conformance/filter-ops-matrix/input/meta.matrix.json @@ -19,6 +19,7 @@ { "field.time": { "name": "fTime", "@filterable": true } }, { "field.timestamp": { "name": "fTimestamp", "@filterable": true } }, { "field.enum": { "name": "fEnum", "@values": ["a", "b", "c"], "@filterable": true } }, + { "field.enum": { "name": "fEnumInt", "@values": ["a", "b", "c"], "@intValueMap": { "a": 0, "b": 1, "c": 2 }, "@filterable": true } }, { "field.uuid": { "name": "fUuid", "@filterable": true } }, { "source.rdb": { "@table": "matrix" } }, { "identity.primary": { "name": "pk", "@fields": ["id"] } }, @@ -34,6 +35,7 @@ { "identity.secondary": { "name": "byTime", "@fields": ["fTime"] } }, { "identity.secondary": { "name": "byTimestamp", "@fields": ["fTimestamp"] } }, { "identity.secondary": { "name": "byEnum", "@fields": ["fEnum"] } }, + { "identity.secondary": { "name": "byEnumInt", "@fields": ["fEnumInt"] } }, { "identity.secondary": { "name": "byUuid", "@fields": ["fUuid"] } } ] } diff --git a/fixtures/conformance/filter-ops-matrix/script.json b/fixtures/conformance/filter-ops-matrix/script.json index da22b0a0b..acd7fe841 100644 --- a/fixtures/conformance/filter-ops-matrix/script.json +++ b/fixtures/conformance/filter-ops-matrix/script.json @@ -6,6 +6,9 @@ { "navigate": ["object:Matrix", "field:fEnum"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "in", "like", "isNull"] } }, + { "navigate": ["object:Matrix", "field:fEnumInt"], + "invoke": "field.filter-ops", + "expect": { "names": ["eq", "ne", "in", "isNull"] } }, { "navigate": ["object:Matrix", "field:fUuid"], "invoke": "field.filter-ops", "expect": { "names": ["eq", "ne", "in", "isNull"] } }, diff --git a/fixtures/metamodel-docs/expected/INDEX.md b/fixtures/metamodel-docs/expected/INDEX.md index 8726d4025..eb3b0fd22 100644 --- a/fixtures/metamodel-docs/expected/INDEX.md +++ b/fixtures/metamodel-docs/expected/INDEX.md @@ -22,6 +22,7 @@ children, and cardinality of a subtype. Universal documentation attributes | `attr.expression` | A structured expression tree over a base entity's own fields (closed node grammar: field/value refs, comparisons sharing the filter op vocabulary, isNull/isNotNull, and/or/not, coalesce). Backs origin.computed; a filter object embeds canonically. Additive node kinds (arithmetic/case/via-joined refs) are #159. | [types/attr.md#attrexpression](types/attr.md#attrexpression) | | `attr.filter` | A filter-expression-valued metadata attribute. Object-shaped value that desugars a preset filter to the canonical { field: { op: value } } form (scalar→eq, array→in, null→isNull; or/and recurse). | [types/attr.md#attrfilter](types/attr.md#attrfilter) | | `attr.int` | A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number. | [types/attr.md#attrint](types/attr.md#attrint) | +| `attr.intMap` | An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation. | [types/attr.md#attrintmap](types/attr.md#attrintmap) | | `attr.long` | A 64-bit-integer-valued metadata attribute. Coerces to and validates as a number. | [types/attr.md#attrlong](types/attr.md#attrlong) | | `attr.properties` | A key/value map attribute (a bag of arbitrary author-supplied properties). Object-shaped value; the registered escape hatch for author-supplied properties (exempt from the strict-attr check, ADR-0023). | [types/attr.md#attrproperties](types/attr.md#attrproperties) | | `attr.string` | A string-valued metadata attribute. Coerces to and validates as text; the default value-type for inline @-syntax attrs (array-of-string is the same subtype with isArray). | [types/attr.md#attrstring](types/attr.md#attrstring) | diff --git a/fixtures/metamodel-docs/expected/providers.md b/fixtures/metamodel-docs/expected/providers.md index 76fa7e2b2..e4b4abbe1 100644 --- a/fixtures/metamodel-docs/expected/providers.md +++ b/fixtures/metamodel-docs/expected/providers.md @@ -13,14 +13,14 @@ provider owns. This is the ownership lens over the same vocabulary Core metaobjects metamodel types and subtypes. -**Owns (registers):** `attr.base`, `attr.boolean`, `attr.class`, `attr.double`, `attr.expression`, `attr.filter`, `attr.int`, `attr.long`, `attr.properties`, `attr.string`, `field.base`, `field.boolean`, `field.currency`, `field.date`, `field.decimal`, `field.double`, `field.enum`, `field.float`, `field.inet`, `field.int`, `field.long`, `field.map`, `field.object`, `field.string`, `field.time`, `field.timestamp`, `field.uri`, `field.uuid`, `identity.primary`, `identity.reference`, `identity.secondary`, `layout.base`, `layout.dataGrid`, `object.base`, `object.entity`, `object.projection`, `object.value`, `origin.aggregate`, `origin.base`, `origin.collection`, `origin.computed`, `origin.first`, `origin.passthrough`, `relationship.aggregation`, `relationship.association`, `relationship.base`, `relationship.composition`, `source.base`, `source.rdb`, `template.base`, `template.output`, `template.prompt`, `template.toolcall`, `validator.array`, `validator.atLeastOne`, `validator.base`, `validator.comparison`, `validator.length`, `validator.numeric`, `validator.presentIff`, `validator.regex`, `validator.required`, `validator.requiredWhen`, `view.base`, `view.currency` +**Owns (registers):** `attr.base`, `attr.boolean`, `attr.class`, `attr.double`, `attr.expression`, `attr.filter`, `attr.int`, `attr.intMap`, `attr.long`, `attr.properties`, `attr.string`, `field.base`, `field.boolean`, `field.currency`, `field.date`, `field.decimal`, `field.double`, `field.enum`, `field.float`, `field.inet`, `field.int`, `field.long`, `field.map`, `field.object`, `field.string`, `field.time`, `field.timestamp`, `field.uri`, `field.uuid`, `identity.primary`, `identity.reference`, `identity.secondary`, `layout.base`, `layout.dataGrid`, `object.base`, `object.entity`, `object.projection`, `object.value`, `origin.aggregate`, `origin.base`, `origin.collection`, `origin.computed`, `origin.first`, `origin.passthrough`, `relationship.aggregation`, `relationship.association`, `relationship.base`, `relationship.composition`, `source.base`, `source.rdb`, `template.base`, `template.output`, `template.prompt`, `template.toolcall`, `validator.array`, `validator.atLeastOne`, `validator.base`, `validator.comparison`, `validator.length`, `validator.numeric`, `validator.presentIff`, `validator.regex`, `validator.required`, `validator.requiredWhen`, `view.base`, `view.currency` **Contributes attributes:** - `field.base`: `@default`, `@readOnly`, `@required`, `@unique` - `field.currency`: `@currency` - `field.decimal`: `@precision`, `@scale` -- `field.enum`: `@provided`, `@values` +- `field.enum`: `@intValueMap`, `@provided`, `@values` - `field.inet`: `@lenient` - `field.map`: `@objectRef`, `@valueType` - `field.object`: `@objectRef` diff --git a/fixtures/metamodel-docs/expected/types/attr.md b/fixtures/metamodel-docs/expected/types/attr.md index b6ca05de8..8276f40e5 100644 --- a/fixtures/metamodel-docs/expected/types/attr.md +++ b/fixtures/metamodel-docs/expected/types/attr.md @@ -108,6 +108,20 @@ _No subtype-specific attributes._ _No structural children._ +### attr.intMap + +An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation. + +**Owning provider:** metaobjects-core-types + +**Attributes** + +_No subtype-specific attributes._ + +**Allowed children** + +_No structural children._ + ### attr.long A 64-bit-integer-valued metadata attribute. Coerces to and validates as a number. diff --git a/fixtures/metamodel-docs/expected/types/field.md b/fixtures/metamodel-docs/expected/types/field.md index e52004dcd..2476f5a7f 100644 --- a/fixtures/metamodel-docs/expected/types/field.md +++ b/fixtures/metamodel-docs/expected/types/field.md @@ -239,6 +239,7 @@ String-backed enumeration constrained to a closed set of member symbols (@values | `@filterable` | boolean | no | | | metaobjects-ui | When true, the field is exposed in generated CRUD filter allowlists (Project D filter layer). | | `@formExclude` | boolean | no | | | metaobjects-ui | When true, the field is omitted from generated forms. Inert on fields for which no form is generated (e.g. projection/derived fields). | | `@instruction` | string | no | | | metaobjects-prompt | FR-010: a short instruction for this field, shown in the generated output-format prompt fragment. | +| `@intValueMap` | intMap | no | | | metaobjects-core-types | Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language. | | `@normalize` | string | no | `strip` | `none`, `collapse`, `strip` | metaobjects-prompt | ASCII normalization mode for tolerant enum extract (none\|collapse\|strip, default strip). On field.enum it is per-field; on object.value it is the default for the object's enum fields. | | `@provided` | boolean | no | | | metaobjects-core-types | FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration. | | `@readOnly` | boolean | no | | | metaobjects-core-types | FR-013: when true, the field is read-only — codegen emits no setter / writable property, the persistence layer skips the column on INSERT/UPDATE, and Zod/Pydantic/class-validator schemas mark it read-only on input variants. The value is populated by the database (computed column, default expression, trigger), by replication, or by another external owner. | diff --git a/fixtures/persistence-conformance/canonical/meta.fitness.json b/fixtures/persistence-conformance/canonical/meta.fitness.json index de39f2e59..ccbb328dd 100644 --- a/fixtures/persistence-conformance/canonical/meta.fitness.json +++ b/fixtures/persistence-conformance/canonical/meta.fitness.json @@ -241,6 +241,8 @@ { "field.timestamp": { "name": "tsTzVal", "@required": true } }, { "field.currency": { "name": "moneyVal", "@required": true, "@currency": "USD" } }, { "field.enum": { "name": "enumVal", "@required": true, "@values": ["LOW", "MEDIUM", "HIGH"] } }, + { "field.enum": { "name": "intEnumVal", "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, { "field.uuid": { "name": "uuidVal", "@required": true } }, { "field.uri": { "name": "uriVal", "@required": true } }, { "field.inet": { "name": "inetVal", "@required": true } }, diff --git a/fixtures/persistence-conformance/canonical/schema.postgres.sql b/fixtures/persistence-conformance/canonical/schema.postgres.sql index 2cb6d9852..86b60bb61 100644 --- a/fixtures/persistence-conformance/canonical/schema.postgres.sql +++ b/fixtures/persistence-conformance/canonical/schema.postgres.sql @@ -112,6 +112,7 @@ CREATE TABLE "all_types" ( "tsTzVal" TIMESTAMPTZ NOT NULL, "moneyVal" BIGINT NOT NULL, "enumVal" TEXT NOT NULL, + "intEnumVal" INTEGER, "uuidVal" UUID NOT NULL, "uriVal" TEXT NOT NULL, "inetVal" INET NOT NULL, @@ -119,7 +120,10 @@ CREATE TABLE "all_types" ( "settings" JSONB, "labels" JSONB, CONSTRAINT "all_types_pkey" PRIMARY KEY ("id"), - CONSTRAINT "all_types_enumVal_chk" CHECK ("enumVal" IN ('LOW', 'MEDIUM', 'HIGH')) + CONSTRAINT "all_types_enumVal_chk" CHECK ("enumVal" IN ('LOW', 'MEDIUM', 'HIGH')), + -- int-backed enum (@intValueMap): the column stores the mapped INTEGER, so the + -- membership CHECK lists the integers unquoted, not the member symbols. + CONSTRAINT "all_types_intEnumVal_chk" CHECK ("intEnumVal" IN (0, 5, 9)) ); CREATE UNIQUE INDEX "byTitle" ON "programs" ("title"); diff --git a/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml b/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml index 2a6761dfe..af362329c 100644 --- a/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml +++ b/fixtures/persistence-conformance/migrations/bootstrap-canonical-from-empty.yaml @@ -40,6 +40,11 @@ expect: - 'CREATE TABLE "all_types"' - '"bVal" BOOLEAN' - 'CHECK ("enumVal" IN (''LOW'', ''MEDIUM'', ''HIGH''))' + # int-backed enum (@intValueMap): the column is INTEGER and the membership CHECK + # lists the mapped integers UNQUOTED. Emitting the member symbols here would be + # un-appliable DDL against an integer column, so this pins the lowering. + - '"intEnumVal" INTEGER' + - 'CHECK ("intEnumVal" IN (0, 5, 9))' - 'CREATE UNIQUE INDEX "byTitle" ON "programs" ("title");' - 'ALTER TABLE "weeks" ADD CONSTRAINT "weeks_programId_fk"' # Match both TS (`CREATE VIEW`) and C# (`CREATE OR REPLACE VIEW`, idempotent) diff --git a/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml b/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml index 85ef825f9..93ff84047 100644 --- a/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml +++ b/fixtures/persistence-conformance/queries/roundtrip-all-types.yaml @@ -72,6 +72,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: 199900 enumVal: "MEDIUM" + intEnumVal: "PUBLISHED" uuidVal: "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA" uriVal: "https://example.com/a/b?q=1" inetVal: "192.168.1.1" @@ -96,6 +97,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: "199900" enumVal: "MEDIUM" + intEnumVal: "PUBLISHED" uuidVal: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" uriVal: "https://example.com/a/b?q=1" inetVal: "192.168.1.1" @@ -122,6 +124,7 @@ queries: tsTzVal: "2026-01-15T09:00:00Z" moneyVal: 0 enumVal: "LOW" + intEnumVal: "DRAFT" uuidVal: "11111111-1111-4111-8111-111111111111" uriVal: "urn:isbn:0451450523" inetVal: "10.0.0.5" @@ -143,6 +146,7 @@ queries: tsTzVal: "2026-01-15T09:00:00Z" moneyVal: "0" enumVal: "LOW" + intEnumVal: "DRAFT" uuidVal: "11111111-1111-4111-8111-111111111111" uriVal: "urn:isbn:0451450523" inetVal: "10.0.0.5" @@ -179,6 +183,7 @@ queries: tsTzVal: "2026-12-31T23:59:59.999Z" moneyVal: "9223372036854775807" enumVal: "HIGH" + intEnumVal: "ARCHIVED" uuidVal: "ffffffff-ffff-4fff-bfff-ffffffffffff" uriVal: "https://example.org/path#frag" inetVal: "255.255.255.255" @@ -202,6 +207,7 @@ queries: tsTzVal: "2026-12-31T23:59:59.999Z" moneyVal: "9223372036854775807" enumVal: "HIGH" + intEnumVal: "ARCHIVED" uuidVal: "ffffffff-ffff-4fff-bfff-ffffffffffff" uriVal: "https://example.org/path#frag" inetVal: "255.255.255.255" diff --git a/fixtures/persistence-conformance/queries/update-delete-all-types.yaml b/fixtures/persistence-conformance/queries/update-delete-all-types.yaml index 8683ac16d..35edbcfe3 100644 --- a/fixtures/persistence-conformance/queries/update-delete-all-types.yaml +++ b/fixtures/persistence-conformance/queries/update-delete-all-types.yaml @@ -75,6 +75,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: 250000 enumVal: "HIGH" + intEnumVal: "PUBLISHED" uuidVal: "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA" uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" @@ -95,6 +96,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: "250000" enumVal: "HIGH" + intEnumVal: "PUBLISHED" uuidVal: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" @@ -123,6 +125,7 @@ queries: tsTzVal: "2026-06-03T14:30:00.123Z" moneyVal: "250000" enumVal: "HIGH" + intEnumVal: "PUBLISHED" uuidVal: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" uriVal: "https://example.com/after?x=1" inetVal: "192.168.10.20" diff --git a/fixtures/registry-conformance/coverage-report.json b/fixtures/registry-conformance/coverage-report.json index 4b702f4e0..cfc50bf37 100644 --- a/fixtures/registry-conformance/coverage-report.json +++ b/fixtures/registry-conformance/coverage-report.json @@ -1,5 +1,5 @@ { - "registeredSubTypeCount": 69, + "registeredSubTypeCount": 70, "exercisedSubTypeCount": 47, "untestedSubTypes": [ "attr.base", @@ -9,6 +9,7 @@ "attr.expression", "attr.filter", "attr.int", + "attr.intMap", "attr.long", "attr.string", "field.base", diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index 6bd8d96a0..4066da028 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -50,6 +50,13 @@ "attrs": [], "children": [] }, + { + "type": "attr", + "subType": "intMap", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation.", + "attrs": [], + "children": [] + }, { "type": "attr", "subType": "long", @@ -1014,6 +1021,13 @@ "required": false, "description": "FR-010: a short instruction for this field, shown in the generated output-format prompt fragment." }, + { + "name": "intValueMap", + "valueType": "intMap", + "isArray": false, + "required": false, + "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." + }, { "name": "normalize", "valueType": "string", diff --git a/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs b/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs index b42728bfd..2873773bf 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/DbContextCompileTests.cs @@ -48,6 +48,9 @@ public class DbContextCompileTests // (see the file-header comment for the field-by-field breakdown). private const string Model = """ { "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "Priority", "abstract": true, + "@values": ["LOW", "HIGH"], + "@intValueMap": { "LOW": 1, "HIGH": 9 } } }, { "object.value": { "name": "Address", "children": [ { "field.string": { "name": "street", "@required": true, "@maxLength": 120 } }, { "field.string": { "name": "city", "@maxLength": 80 } } @@ -57,6 +60,8 @@ public class DbContextCompileTests { "field.long": { "name": "id" } }, { "field.enum": { "name": "status", "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, { "field.enum": { "name": "statuses", "isArray": true, "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { "field.enum": { "name": "priority", "extends": "Priority" } }, + { "field.enum": { "name": "rank", "@values": ["A", "B"], "@intValueMap": { "A": 0, "B": 7 } } }, { "field.string": { "name": "tags", "isArray": true } }, { "field.object": { "name": "homeAddress", "@objectRef": "Address", "@storage": "flattened" } }, { "field.object": { "name": "config", "@objectRef": "Address" } }, diff --git a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs index 596729f0b..7584f0b89 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/DbContextGenerator.cs @@ -64,8 +64,11 @@ public virtual IEnumerable Generate(GenContext ctx) // column as Int32 at materialization — an InvalidCastException. Mirror the // entity-side HasConversion() so a projection that passes an enum // through (e.g. ProgramView.status over v_program) round-trips. + // An int-backed enum (@intValueMap) column in the view holds the declared INTEGER, + // so it takes the same custom converter pair the table side gets — reading it as + // a string would fail materialization exactly as the ordinal default does here. foreach (var f in p.Fields().Where(f => f.SubType == FIELD_SUBTYPE_ENUM && !f.ResolvedIsArray())) - modelLines.Add($" modelBuilder.Entity<{name}>().Property(x => x.{CSharpNaming.Pascal(f.Name)}).HasConversion();"); + modelLines.Add($" modelBuilder.Entity<{name}>().Property(x => x.{CSharpNaming.Pascal(f.Name)}).{EnumConversionCall(name, p, f, ctx.Config)};"); } foreach (var e in objects.Where(o => o.IsEntity() && !o.IsReadOnlyProjection())) { @@ -150,6 +153,7 @@ public virtual IEnumerable Generate(GenContext ctx) EmitUsings(sb, needsMetadataUsing, ctx); EmitDbSetDeclarations(sb, objects, ctx); EmitOnModelCreatingBody(sb, modelLines, ctx); + if (NeedsUnmappedEnumHelper(objects)) EmitUnmappedEnumHelper(sb); sb.AppendLine("}"); return [new EmittedFile("AppDbContext.g.cs", sb.ToString())]; } @@ -322,6 +326,119 @@ private static string UsingEntityConfig(M2MNavigation nav) $"r => r.HasOne<{source}>().WithMany().HasForeignKey(nameof({through}.{sourceFkProp})));"; } + /// + /// The declared symbol→int map (@intValueMap) for an int-backed + /// field.enum, or null when the enum is string-backed. + /// + /// + /// ADR-0039 RESOLVING (Attr, not OwnAttr): the map is @values' + /// numeric half — a logical property of the enum vocabulary that inherits through + /// extends — so a field extending a shared abstract enum is int-backed too. + /// The loader's validation reads it own-only, which is correct there: it validates + /// what a declaration itself declares. + /// + private static IReadOnlyDictionary? IntValueMapOf(MetaField f) => + f.Attr(FIELD_ATTR_INT_VALUE_MAP) as IReadOnlyDictionary; + + /// Name of the generated fail-fast helper the read converters end in. + private const string UnmappedEnumHelperName = "UnmappedEnumValue"; + + /// + /// Emits the fail-fast helper every int-backed enum's provider→model converter ends + /// in. Emitted only when at least one such converter was generated, so a model with + /// no @intValueMap produces byte-identical output. + /// + private static void EmitUnmappedEnumHelper(StringBuilder sb) + { + sb.AppendLine(); + sb.AppendLine(" /// "); + sb.AppendLine(" /// An int-backed field.enum column held a value that maps to no member: the"); + sb.AppendLine(" /// database holds data the model says is impossible (a hand-written INSERT, or a"); + sb.AppendLine(" /// member removed without a migration). Materializing the last member instead"); + sb.AppendLine(" /// would hand the caller a wrong-but-valid value, silently."); + sb.AppendLine(" /// "); + sb.AppendLine($" private static T {UnmappedEnumHelperName}(int stored, string field) =>"); + // Fully qualified: the generated file's usings are a fixed set (EmitUsings), and + // adding `using System;` there would change byte-identical output for every model. + sb.AppendLine(" throw new System.InvalidOperationException("); + sb.AppendLine(" $\"field.enum '{field}' read stored value {stored} with no member in \" +"); + sb.AppendLine(" \"@intValueMap — the database holds a value the model does not describe.\");"); + } + + /// + /// True when any emitted converter will reference + /// — i.e. the model carries at least one int-backed field.enum. Mirrors the + /// fields configures (enum fields of every emitted + /// object, plus a write-through entity's read-model view over the same field set). + /// + private static bool NeedsUnmappedEnumHelper(IEnumerable objects) => + objects.Any(o => o.Fields().Any(f => + f.SubType == FIELD_SUBTYPE_ENUM && IntValueMapOf(f) is not null)); + + /// + /// The complete HasConversion call for an enum property: the generic + /// HasConversion<string>() for a string-backed enum, or + /// HasConversion(model→provider, provider→model) built from + /// @intValueMap for an int-backed one. + /// + /// + /// The mapping is emitted as a TERNARY CHAIN rather than a switch + /// expression because EF converts these lambdas to EXPRESSION TREES, and a switch + /// expression is not legal in one (CS8155). A conditional is. + /// The provider→model chain gives EVERY member its own branch and ends in a + /// call to the generated UnmappedEnumValue<T> helper, so a stored int + /// with no member THROWS rather than silently materializing as the last member — + /// matching all four sibling ports. CS8188 bans a throw-EXPRESSION inside an + /// expression tree, but a method CALL is legal there and the throw itself happens in + /// the helper's ordinary body. Only the read side needs this: the model→provider + /// chain is exhaustive over the enum by construction, since @intValueMap's + /// keys are loader-validated to match @values exactly. + /// + private static string EnumConversionCall(string owner, MetaObject entity, MetaField f, GenConfig config) + { + var intMap = IntValueMapOf(f); + if (intMap is null) return "HasConversion()"; + + var members = f.EffectiveEnumValues ?? new List(); + if (members.Count == 0) return "HasConversion()"; + + // FR-019: a SHARED (root-level abstract) or @provided enum is NOT nested inside the + // entity class — EntityGenerator references it instead (see its EnumPropertyTypeName) + // — so it must be named unqualified here. Qualifying it as {owner}.{Name} emits + // CS0426 ("the type name does not exist in the type"), which is not an edge case: + // ERR_ENUM_EXTENDS_VALUES_CONFLICT makes declaring @intValueMap on the CONSUMING + // field a load error, so hanging it on the shared declaration is the only legal way + // to int-back a shared enum. String-backed shared enums never showed this because + // HasConversion() names no type at all. + var type = Fr019SharedEnum.SharedEnumForField(f) is { } shared + ? Fr019SharedEnum.SharedEnumTypeReference(shared, config) + : $"{owner}.{CSharpNaming.EnumTypeName(entity, f)}"; + // Read the ints THROUGH the map, keyed by member, so @values stays the SSOT and a + // member with no mapping cannot silently vanish from the conversion. + var ints = new List(members.Count); + foreach (var m in members) + { + if (!intMap.TryGetValue(m, out var raw) || raw is null) + throw new InvalidOperationException( + $"field.enum '{f.Name}' @{FIELD_ATTR_INT_VALUE_MAP} has no integer for member '{m}' — " + + "cannot build the EF value conversion."); + ints.Add(Convert.ToInt64(raw).ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + + var toProvider = new System.Text.StringBuilder("v => "); + var fromProvider = new System.Text.StringBuilder("v => "); + for (var i = 0; i < members.Count - 1; i++) + toProvider.Append($"v == {type}.{members[i]} ? {ints[i]} : "); + toProvider.Append(ints[^1]); + // Every member gets its own branch here (no last-member fallthrough) so the + // final else can reject an int the model does not describe. + for (var i = 0; i < members.Count; i++) + fromProvider.Append($"v == {ints[i]} ? {type}.{members[i]} : "); + fromProvider.Append($"{UnmappedEnumHelperName}<{type}>(v, \"{f.Name}\")"); + + return $"HasConversion({toProvider}, {fromProvider})"; + } + // #214 [0] — the per-field EF TYPE-converter emission, factored out so the SAME set of // converters configures BOTH the write entity (its derived-EXCLUDED field set) AND a // write-through entity's View read model (ALL fields, incl. the derived origin.* @@ -353,13 +470,18 @@ private void EmitFieldTypeConfig( foreach (var f in fieldList.Where(f => f.SubType == FIELD_SUBTYPE_ENUM)) { var prop = CSharpNaming.Pascal(f.Name); + // An int-backed enum (@intValueMap) persists the declared INTEGER instead of the + // member symbol, so it needs a custom converter pair rather than HasConversion(). + // The generated C# `enum` declaration is byte-identical either way — int-backing is a + // persistence concern, invisible in the entity's API. + var conversion = EnumConversionCall(className, entity, f, ctx.Config); // ADR-0039: resolving — array-ness inheritable via extends. Array-of-enum uses the - // EF Core 8 primitive collection with a per-element string conversion so members - // persist as symbols (["DRAFT"]), not int ordinals ([0]). + // EF Core 8 primitive collection with a per-element conversion so members persist as + // symbols (["DRAFT"]) — or as their declared ints — not as int ordinals ([0]). if (f.ResolvedIsArray()) - modelLines.Add($" modelBuilder.Entity<{className}>().PrimitiveCollection(x => x.{prop}).ElementType().HasConversion();"); + modelLines.Add($" modelBuilder.Entity<{className}>().PrimitiveCollection(x => x.{prop}).ElementType().{conversion};"); else - modelLines.Add($" modelBuilder.Entity<{className}>().Property(x => x.{prop}).HasConversion();"); + modelLines.Add($" modelBuilder.Entity<{className}>().Property(x => x.{prop}).{conversion};"); } foreach (var f in fieldList.Where(f => f.ResolvedIsArray() && CSharpNaming.ScalarFor(f.SubType) is not null)) diff --git a/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs index c67a53859..349451fa9 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/FilterAllowlistGenerator.cs @@ -28,21 +28,6 @@ public class FilterAllowlistGenerator : PerEntityGenerator { public override string Name => "filter-allowlist-generator"; - /// Operator set for string-shaped subtypes. - internal static readonly string[] OpsString = - { "eq", "ne", "in", "like", "isNull" }; - - /// Operator set for uuid — identity comparison only, no like (not a substring type) and no ordering. - internal static readonly string[] OpsUuid = - { "eq", "ne", "in", "isNull" }; - - /// Operator set for numeric / date / timestamp / currency subtypes. - internal static readonly string[] OpsNumeric = - { "eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull" }; - - /// Operator set for boolean subtype. - internal static readonly string[] OpsBoolean = { "eq", "isNull" }; - // Read-only projections (views/etc.) are not filterable in the routes // generator today — skip them here to match. protected override bool Filter(MetaObject entity) => AppliesTo(entity); @@ -119,7 +104,7 @@ protected virtual Dictionary> ComputeFilterableOps { if (field.SubType == FIELD_SUBTYPE_OBJECT) continue; if (!IsFilterable(field)) continue; - var ops = OpsForSubtype(field.SubType); + var ops = OpsForField(field); if (ops.Count == 0) continue; out_[field.Name] = ops; } @@ -135,18 +120,20 @@ protected virtual Dictionary> ComputeFilterableOps _ => false, }; - /// Operator set for , or empty for unsupported subtypes. - internal static IReadOnlyList OpsForSubtype(string? subType) => subType switch - { - // ADR-0036 Wave 3 — field.uri is string-like (eq/ne/in/like/isNull); field.inet is - // uuid-like (eq/ne/in/isNull — no like / ordering on an opaque address value). - FIELD_SUBTYPE_STRING or FIELD_SUBTYPE_ENUM or FIELD_SUBTYPE_URI => OpsString, - FIELD_SUBTYPE_UUID or FIELD_SUBTYPE_INET => OpsUuid, - FIELD_SUBTYPE_INT or FIELD_SUBTYPE_LONG - or FIELD_SUBTYPE_FLOAT or FIELD_SUBTYPE_DOUBLE or FIELD_SUBTYPE_DECIMAL - or FIELD_SUBTYPE_CURRENCY - or FIELD_SUBTYPE_DATE or FIELD_SUBTYPE_TIMESTAMP or FIELD_SUBTYPE_TIME => OpsNumeric, - FIELD_SUBTYPE_BOOLEAN => OpsBoolean, - _ => System.Array.Empty(), - }; + /// + /// Operator set for , or empty for unsupported subtypes. + /// + /// Delegates wholesale to QueryConstants.OpsForField — the cross-port single + /// source of truth, pinned by fixtures/conformance/filter-ops-matrix. This + /// generator used to carry its OWN copy of the per-subtype band table; that copy was + /// removed rather than extended, because two tables is exactly how the bands drift. + /// + /// + /// Field-level, not subtype-level: an int-backed field.enum + /// (@intValueMap, design D5) persists as an INTEGER column, so like — + /// a substring match — is dropped. + /// + /// + internal static IReadOnlyList OpsForField(MetaField field) => + MetaObjects.Core.Query.QueryConstants.OpsForField(field); } diff --git a/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs b/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs index 1375498df..7835bf7d2 100644 --- a/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs +++ b/server/csharp/MetaObjects.Conformance.Tests/CapabilityBinding.cs @@ -70,12 +70,14 @@ private delegate NormalizedResult CapabilityFn( ["field.effective-tree"] = (node, _) => NormalizedResult.EffectiveTree(MetaObjects.SerializerJson.CanonicalSerialize(AsField(node))), - // field.filter-ops → the canonical per-subtype filter-operator band - // (QueryConstants.OPS_BY_SUBTYPE). Returns { names: [...] } in canonical - // operator order. Single source of truth — the same map the server - // allowlist + codegen consume. + // field.filter-ops → the canonical per-FIELD filter-operator band. + // Returns { names: [...] } in canonical operator order. Single source of + // truth — the same function the server allowlist + codegen consume. + // + // OpsForField, not OpsForSubType: the band is field-level because an + // int-backed field.enum (@intValueMap) stores as an integer, dropping `like`. ["field.filter-ops"] = (node, _) => - NormalizedResult.Names(QueryConstants.OpsForSubType(AsField(node).SubType).ToList()), + NormalizedResult.Names(QueryConstants.OpsForField(AsField(node)).ToList()), }; /// diff --git a/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs b/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs new file mode 100644 index 000000000..84d9a0f93 --- /dev/null +++ b/server/csharp/MetaObjects.Conformance.Tests/EnumIntValueMapTests.cs @@ -0,0 +1,89 @@ +// Int-backed-enum-values plan, Task 6 — C# port of field.enum's @intValueMap. +// +// Mirrors the TS reference (Task 1/2/3): a new attr.intMap subtype (object-shaped, +// all-integer values) plus field.enum-specific content-rule validation (key-set +// equals @values, no duplicate values), reusing ERR_BAD_ATTR_VALUE. + +using MetaObjects.Loader; +using Xunit; + +namespace MetaObjects.Conformance.Tests; + +public class EnumIntValueMapTests +{ + private static string Model(string extra) => """ + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] EXTRA_PLACEHOLDER } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """.Replace("EXTRA_PLACEHOLDER", extra); + + private static LoadResult TryLoad(string json) + { + var registry = FullCoreRegistry.Compose(); + var loader = new MetaDataLoader(registry); + return loader.Load(new IMetaDataSource[] + { + new InMemoryStringSource(json, format: MetaDataFormat.Json, id: "test.json"), + }); + } + + [Fact] + public void Valid_intValueMap_with_matching_keys_and_unique_ints_loads_clean() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.Empty(res.Errors); + } + + [Fact] + public void No_intValueMap_still_loads_clean_string_backed_default() + { + var res = TryLoad(Model("")); + Assert.Empty(res.Errors); + } + + [Fact] + public void Missing_member_key_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("ARCHIVED")); + } + + [Fact] + public void Extra_key_not_in_values_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("RETRACTED")); + } + + [Fact] + public void Non_integer_value_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE); + } + + [Fact] + public void Duplicate_int_value_across_members_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE + && e.Message.Contains("DRAFT") && e.Message.Contains("PUBLISHED")); + } + + // Final-review fix: the eventual DB column for an int-backed enum is a + // 32-bit Postgres/SQLite `integer` (design doc D5) — a value outside that + // range can never actually be persisted, so it must be rejected at load + // time. Mirrors Java's IntMapAttribute#setValueAsString bound check + // (this port's generic type check parses integral JSON numbers as + // `long`, which has no fixed 32-bit width on its own). + [Fact] + public void Value_outside_32bit_range_is_rejected() + { + var res = TryLoad(Model(""", "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9999999999}""")); + Assert.Contains(res.Errors, e => e.Code == ErrorCode.ERR_BAD_ATTR_VALUE); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs b/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs index ac4859a36..8a76d60fa 100644 --- a/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs +++ b/server/csharp/MetaObjects.IntegrationTests/AppDbContextModelTests.cs @@ -36,4 +36,38 @@ public void Model_builds_without_error() Assert.NotNull(model.FindEntityType(typeof(PriorAuthAuth))); Assert.Equal("auths", auth!.GetTableName()); } + + /// + /// The int-backed field.enum value converter, EXECUTED in both directions off + /// the finalized model — the emitted lambdas are the whole feature, and a string + /// assertion over the generated source cannot tell a working converter from one that + /// compiles and computes the wrong thing. + /// + [Fact] + public void Int_backed_enum_converter_maps_both_ways_and_rejects_an_unmapped_stored_value() + { + var options = new DbContextOptionsBuilder() + .UseNpgsql("Host=localhost;Database=unused") + .Options; + using var db = new AppDbContext(options); + + var prop = db.Model.FindEntityType(typeof(AllTypes))!.FindProperty(nameof(AllTypes.IntEnumVal)); + var converter = prop!.GetValueConverter(); + Assert.NotNull(converter); + + // @intValueMap declares DRAFT=0, PUBLISHED=5, ARCHIVED=9 — NOT the C# ordinals + // (which would make PUBLISHED 1 and ARCHIVED 2), so these assertions also prove + // the declared map is what reaches the column rather than EF's ordinal default. + Assert.Equal(5, converter!.ConvertToProvider(AllTypes.AllTypesIntEnumVal.PUBLISHED)); + Assert.Equal(9, converter.ConvertToProvider(AllTypes.AllTypesIntEnumVal.ARCHIVED)); + Assert.Equal(AllTypes.AllTypesIntEnumVal.PUBLISHED, converter.ConvertFromProvider(5)); + Assert.Equal(AllTypes.AllTypesIntEnumVal.ARCHIVED, converter.ConvertFromProvider(9)); + + // 7 maps to no member. Materializing the last member instead — which is what the + // ternary chain did before it grew a final else — would hand the caller + // ARCHIVED for a row that is not archived, silently. + var ex = Assert.Throws(() => converter.ConvertFromProvider(7)); + Assert.Contains("7", ex.Message); + Assert.Contains("intValueMap", ex.Message); + } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs index cafb2c3e6..80116f4a0 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AllTypes.g.cs @@ -14,6 +14,7 @@ namespace MetaObjects.IntegrationTests.Generated; public class AllTypes { public enum AllTypesEnumVal { LOW, MEDIUM, HIGH } + public enum AllTypesIntEnumVal { DRAFT, PUBLISHED, ARCHIVED } [Key] [Column("id")] public Guid Id { get; set; } @@ -46,6 +47,8 @@ public enum AllTypesEnumVal { LOW, MEDIUM, HIGH } public long MoneyVal { get; set; } [Column("enumVal")] public AllTypesEnumVal EnumVal { get; set; } + [Column("intEnumVal")] + public AllTypesIntEnumVal? IntEnumVal { get; set; } [Column("uuidVal")] public Guid UuidVal { get; set; } [Column("uriVal")] diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs index 65b3c5979..24b7bbc94 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs @@ -33,6 +33,7 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) modelBuilder.Entity().OwnsOne(x => x.Settings, b => b.ToJson("settings")); modelBuilder.Entity().OwnsMany(x => x.Labels, b => b.ToJson("labels")); modelBuilder.Entity().Property(x => x.EnumVal).HasConversion(); + modelBuilder.Entity().Property(x => x.IntEnumVal).HasConversion(v => v == AllTypes.AllTypesIntEnumVal.DRAFT ? 0 : v == AllTypes.AllTypesIntEnumVal.PUBLISHED ? 5 : 9, v => v == 0 ? AllTypes.AllTypesIntEnumVal.DRAFT : v == 5 ? AllTypes.AllTypesIntEnumVal.PUBLISHED : v == 9 ? AllTypes.AllTypesIntEnumVal.ARCHIVED : UnmappedEnumValue(v, "intEnumVal")); modelBuilder.Entity().Property(x => x.DecVal).HasPrecision(18, 6); modelBuilder.Entity().Property(x => x.TsVal).HasColumnType("timestamp without time zone"); modelBuilder.Entity().Property(x => x.TsTzVal).HasColumnType("timestamp with time zone"); @@ -50,4 +51,15 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) modelBuilder.Entity().Property(x => x.Status).HasConversion(); modelBuilder.Entity().Property(x => x.CreatedAt).HasColumnType("timestamp without time zone"); } + + /// + /// An int-backed field.enum column held a value that maps to no member: the + /// database holds data the model says is impossible (a hand-written INSERT, or a + /// member removed without a migration). Materializing the last member instead + /// would hand the caller a wrong-but-valid value, silently. + /// + private static T UnmappedEnumValue(int stored, string field) => + throw new System.InvalidOperationException( + $"field.enum '{field}' read stored value {stored} with no member in " + + "@intValueMap — the database holds a value the model does not describe."); } diff --git a/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs b/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs index 5c8a554d3..79fc53166 100644 --- a/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs +++ b/server/csharp/MetaObjects/Core/Attr/AttrConstants.cs @@ -8,7 +8,7 @@ namespace MetaObjects.Core.Attr; /// -/// Attr concern constants — the 9 attr subtypes (plus the universal base). +/// Attr concern constants — the 10 attr subtypes (plus the universal base). /// Wire-format identifiers; do not rename. /// public static class AttrConstants @@ -24,6 +24,12 @@ public static class AttrConstants // #195 — a structured expression tree over a base entity's own fields (backs // origin.computed). Object-shaped (a closed node grammar); mirrors attr.filter. public const string ATTR_SUBTYPE_EXPRESSION = "expression"; + // Int-backed-enum-values plan, Task 6 — an object-shaped attr whose values are + // all integers (e.g. field.enum's @intValueMap). No shape reuse of `properties` + // — that would silently stringify ints in some ports. Generic shape check only; + // a consumer field type layers its own semantic rules (key-set membership, + // uniqueness) in its own content-rule validation. + public const string ATTR_SUBTYPE_INT_MAP = "intMap"; /// /// The retired stringarray array attr subtype. It is NO LONGER a @@ -47,5 +53,6 @@ public static class AttrConstants ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, ]; } diff --git a/server/csharp/MetaObjects/Core/Field/FieldConstants.cs b/server/csharp/MetaObjects/Core/Field/FieldConstants.cs index 2d4d8e801..326a7f668 100644 --- a/server/csharp/MetaObjects/Core/Field/FieldConstants.cs +++ b/server/csharp/MetaObjects/Core/Field/FieldConstants.cs @@ -199,6 +199,16 @@ public static class FieldConstants /// public const string FIELD_ATTR_PROVIDED = "provided"; + /// + /// Optional per-member int values ({member: int}) on a field.enum, + /// switching that field's DB persistence from string+CHECK to integer+CHECK. + /// attr.intMap-shaped. Keys must exactly match ; + /// values must be unique integers (content-rule validated in + /// ValidationPasses.ValidateEnumValues, Rule 5). The generated native type + /// and wire format are unchanged in every language. + /// + public const string FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; + /// /// Regex pattern each enum member symbol must satisfy: must start with /// a letter or underscore, followed by letters, digits, or underscores. diff --git a/server/csharp/MetaObjects/Core/Field/FieldSchema.cs b/server/csharp/MetaObjects/Core/Field/FieldSchema.cs index 82eda8b4f..3afaf4942 100644 --- a/server/csharp/MetaObjects/Core/Field/FieldSchema.cs +++ b/server/csharp/MetaObjects/Core/Field/FieldSchema.cs @@ -198,6 +198,13 @@ public static class FieldSchema "codegen references the type (resolved via per-port codegen config) instead of " + "materializing it. Default false. Not a field attr — it lives on the type declaration."); + /// The @intValueMap attr — only on field.enum. Optional object of integers. + public static readonly AttrSchema IntValueMapAttr = new AttrSchema( + Name: FieldConstants.FIELD_ATTR_INT_VALUE_MAP, + ValueType: AttrConstants.ATTR_SUBTYPE_INT_MAP, + Required: false, + Description: "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers."); + // FR-033 — the field.enum tolerant-extract overlays (@enumAlias / @enumDoc / // @coerceDefault / @normalize) and object.value's @normalize default are NO LONGER // declared here. They are re-homed to the metaobjects-prompt concern provider diff --git a/server/csharp/MetaObjects/Core/Query/QueryConstants.cs b/server/csharp/MetaObjects/Core/Query/QueryConstants.cs index 6aeaf5cc1..722a630b0 100644 --- a/server/csharp/MetaObjects/Core/Query/QueryConstants.cs +++ b/server/csharp/MetaObjects/Core/Query/QueryConstants.cs @@ -71,6 +71,58 @@ public static class QueryConstants public static string[] OpsForSubType(string subType) => OPS_BY_SUBTYPE.TryGetValue(subType, out string[]? ops) ? ops : []; + /// + /// The int-backed-enum band: the enum band minus like. Hoisted + /// so the narrowing is one named constant rather than an array filtered at every call. + /// + public static readonly string[] OPS_ENUM_INT_BACKED = + [FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL]; + + /// + /// The filter-operator band for a FIELD — the entry point every consumer that has a + /// field in hand must use (loader validation, the codegen filter-allowlist generator, + /// the cross-port field.filter-ops capability). + /// + /// Identical to except for ONE case: an int-backed + /// field.enum (one declaring @intValueMap, design D5) persists as an + /// INTEGER column, so like — a substring match — is dropped. + /// eq/ne/in survive because the member symbol encodes to its + /// integer before it reaches SQL; like has no such encoding, and an unencoded + /// LIKE 'DRAFT' against an integer column is a request-time type error. + /// + /// + /// cannot express this — it only ever sees the subtype + /// "enum" — and is deliberately left unchanged for the one caller that + /// genuinely has no field: ExpressionGrammar's declared operand type. + /// + /// + /// ADR-0039: the @intValueMap read is RESOLVING (Attr, not + /// OwnAttr). Post-#246 the map lives on a shared root-level abstract + /// declaration and consuming fields INHERIT it, so an own-only read would see it + /// absent on exactly the shape adopters are steered toward and wrongly keep + /// like. + /// + /// + /// Cross-port: fixtures/conformance/filter-ops-matrix pins fEnum vs + /// fEnumInt in all five ports. + /// + /// + /// Takes MetaData rather than MetaField so the loader's dataGrid pass — + /// which iterates untyped children — can call it directly, mirroring the TS + /// structural parameter. + /// + /// + public static string[] OpsForField(MetaObjects.Meta.MetaData field) + { + if (field is null) return []; + if (field.SubType == MetaObjects.Core.Field.FieldConstants.FIELD_SUBTYPE_ENUM && + field.Attr(MetaObjects.Core.Field.FieldConstants.FIELD_ATTR_INT_VALUE_MAP) is not null) + { + return OPS_ENUM_INT_BACKED; + } + return OpsForSubType(field.SubType); + } + // ----------------------------------------------------------------------- // Sort order values (used by @sortableDefaultOrder on fields and // @defaultSortOrder on dataGrid layouts) diff --git a/server/csharp/MetaObjects/CoreTypes.cs b/server/csharp/MetaObjects/CoreTypes.cs index 87c152594..6033a021e 100644 --- a/server/csharp/MetaObjects/CoreTypes.cs +++ b/server/csharp/MetaObjects/CoreTypes.cs @@ -174,6 +174,7 @@ private static TypeDefinition Def( [ATTR_SUBTYPE_PROPERTIES] = DataType.Object, [ATTR_SUBTYPE_FILTER] = DataType.Object, [ATTR_SUBTYPE_EXPRESSION] = DataType.Object, + [ATTR_SUBTYPE_INT_MAP] = DataType.Object, }; // ------------------------------------------------------------------------- @@ -347,7 +348,7 @@ private static void RegisterCoreTypeDefs(TypeRegistry registry) // tolerant-extract overlays (@enumAlias / @enumDoc / @coerceDefault / // @normalize) are re-homed to the metaobjects-prompt concern provider // (reads prompt.json's field.enum extends). - FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr], + FIELD_SUBTYPE_ENUM => [.. FieldSchema.CommonFieldAttrs, FieldSchema.EnumValuesAttr, FieldSchema.ProvidedAttr, FieldSchema.IntValueMapAttr], _ => FieldSchema.CommonFieldAttrs.ToList(), }; @@ -362,7 +363,7 @@ private static void RegisterCoreTypeDefs(TypeRegistry registry) DataTypeFor(FieldDataType, subType, "field"))); } - // attr — 9 subtypes (base + 8), no children allowed + // attr — 10 subtypes (base + 9), no children allowed foreach (string subType in ATTR_SUBTYPES) { registry.Register( diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index a8880dc1e..2c6c8e6f5 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -187,6 +187,12 @@ public enum ErrorCode // own @values would be silently dropped in codegen. Remove the own @values // to inherit the shared set, or extend a concrete (non-shared) enum instead. ERR_ENUM_EXTENDS_VALUES_CONFLICT, + // A field.enum carries @intValueMap together with isArray=true. Int-backing is + // a persistence-layer codec and no port implements it element-wise over an + // array column, so the combination would silently persist member SYMBOLS into + // an integer array. An array-of-enum stays string-backed: drop @intValueMap, + // or make the field scalar. + ERR_ENUM_INT_VALUE_MAP_ARRAY, ERR_UNKNOWN, } diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 83047ceb0..c4c903579 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -1476,7 +1476,9 @@ public static IReadOnlyList ValidateDataGridFilterValues(MetaData roo // ADR-0039: resolving — a concrete field may inherit @filterable via extends (TS validation-passes.ts:1252). if (f.Attr(FIELD_ATTR_FILTERABLE) is true) { - allow[f.Name] = OpsForSubType(f.SubType); + // OpsForField, not OpsForSubType — an int-backed field.enum + // (@intValueMap) stores as an integer, so `like` is not in its band. + allow[f.Name] = OpsForField(f); } } @@ -1934,6 +1936,27 @@ ATTR_SUBTYPE_FILTER or // an object @expr then flows to the closed-grammar check in the origin pass. value is IReadOnlyDictionary, + // Int-backed-enum-values plan, Task 6 — attr.intMap: object-shaped AND every + // member value must be an integer (JSON integral numbers parse as `long`/`int` + // in this port's DataConverter; a fractional number parses as `double` and fails + // here). Mirrors TS's generic IntMapAttr.validateValue per-member type check — + // in this port that generic check lives here rather than in a per-subtype class. + // field.enum's own key-set/uniqueness content rules run separately (Pass 10). + // + // Final-review fix: also bound every value to the 32-bit signed int range + // (inclusive) — the eventual DB column for an int-backed enum is a 32-bit + // Postgres/SQLite `integer` (design doc D5), matching Java's + // IntMapAttribute#setValueAsString bound check exactly. A `long` here can + // exceed Int32 range even though it's a whole number. + ATTR_SUBTYPE_INT_MAP => + value is IReadOnlyDictionary intMap + && intMap.Values.All(v => v switch + { + int => true, + long l => l >= int.MinValue && l <= int.MaxValue, + _ => false, + }), + _ => true, // SUBTYPE_BASE or unknown → accept anything }; } @@ -2303,15 +2326,21 @@ public static SourceEscapeValidationResult ValidateSourceEscapes(MetaData root) // ========================================================================= // Pass 10: ValidateEnumValues - // Enforces the three cross-language @values rules on every field.enum node: + // Enforces the cross-language @values / @intValueMap rules on every + // field.enum node: // 1. @values must be non-empty. // 2. Every member must match ENUM_MEMBER_PATTERN (identifier-safe). // 3. No duplicate members. - // Error: ERR_BAD_ATTR_VALUE for all three. + // 4. FR-011: @coerceDefault / @default (when set) must name a @values member. + // 5. Int-backed-enum-values plan: @intValueMap (when set) keys must exactly + // match @values, and no two members may share the same int. + // Error: ERR_BAD_ATTR_VALUE for all five. // // Note: Pass 6 (ValidateAttrSchema) already enforces that @values is // present (Required: true → ERR_MISSING_REQUIRED_ATTR) and that it is a - // stringarray. This pass runs after that and handles the content rules. + // stringarray, and that @intValueMap (when present) is intMap-shaped with + // every member value an integer. This pass runs after that and handles the + // content rules. // ========================================================================= private static readonly Regex EnumMemberRegex = @@ -2324,6 +2353,24 @@ public static IReadOnlyList ValidateEnumValues(MetaData root) return errors.AsReadOnly(); } + /// + /// The shared-enum super of , or null when it has none. + /// "Shared" (FR-019 / #246) means the immediate super is abstract AND declared at + /// metadata-root — a metadata.root child, not one nested under an object. Such a + /// declaration is materialized ONCE per port as a single named type, so anything a + /// consuming field re-declares that belongs to the shared TYPE's contract (its + /// @values member set, or the @intValueMap integer backing of that set) is a conflict. + /// Immediate-super-only, matching codegen's Fr019SharedEnum.ResolveSharedEnumDecl so + /// the validator and the shared-enum collapse agree on what "shared" means. + /// + private static MetaData? SharedEnumSuper(MetaField field) + { + var sup = field.SuperData; + return sup is not null && sup.IsAbstract && sup.Parent is { } p && p.Type == TYPE_METADATA + ? sup + : null; + } + private static void WalkEnumValues(MetaData node, List errors) { if (node is MetaField { SubType: FIELD_SUBTYPE_ENUM } field) @@ -2377,11 +2424,10 @@ private static void WalkEnumValues(MetaData node, List errors) // collapse would silently drop this field's own @values in favor of the shared // type's. Own-attrs-only (matches Rules 1-3 above): only fires when THIS node // declares @values itself, not when it merely inherits. - var sup = field.SuperData; - if (sup is not null && sup.IsAbstract && sup.Parent is { } p && p.Type == TYPE_METADATA) + if (SharedEnumSuper(field) is { } sharedSuper) { errors.Add(new MetaError( - $"field.enum '{field.Name}' extends shared abstract enum '{sup.Name}' AND declares its own " + + $"field.enum '{field.Name}' extends shared abstract enum '{sharedSuper.Name}' AND declares its own " + $"@{FIELD_ATTR_VALUES} — a shared enum's member set is owned by the shared declaration; " + $"remove the own @{FIELD_ATTR_VALUES} to inherit it, or extend a non-shared enum instead.", ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, @@ -2416,6 +2462,96 @@ private static void WalkEnumValues(MetaData node, List errors) } } } + + // Rule 5 (int-backed-enum-values plan, Task 6): @intValueMap content rules + // (optional attr — nothing to check when absent). + // a. Key set must exactly match @values. + // b. No two members may share the same int (protobuf's stance — no alias + // opt-in). (Every-value-is-an-integer is already enforced by + // ValueMatchesType's ATTR_SUBTYPE_INT_MAP case in Pass 6, which runs + // before this pass.) + if (field.OwnAttr(FIELD_ATTR_INT_VALUE_MAP) is IReadOnlyDictionary intValueMap) + { + // #246 (int-backed twin): the symbol→int mapping is a property of the enum + // VOCABULARY, not of one column that uses it — it is @values' numeric half. + // A shared enum is materialized once as a single type, so a per-field map + // would give one logical type N storage encodings (and, where a port emits + // per-TYPE codec artifacts, two same-named declarations). Same remedy as the + // @values half: declare it on the shared declaration and inherit it. + if (SharedEnumSuper(field) is { } sharedIntSuper) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' extends shared abstract enum '{sharedIntSuper.Name}' AND declares its own " + + $"@{FIELD_ATTR_INT_VALUE_MAP} — a shared enum's integer backing is owned by the shared declaration; " + + $"move @{FIELD_ATTR_INT_VALUE_MAP} onto '{sharedIntSuper.Name}' to inherit it, or extend a non-shared enum instead.", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, + Envelope: field.Source)); + } + + var effective = field.EffectiveEnumValues ?? new List(); + var memberSet = new HashSet(effective, StringComparer.Ordinal); + var mapKeys = intValueMap.Keys.ToList(); + var keySet = new HashSet(mapKeys, StringComparer.Ordinal); + + var missing = effective.Where(m => !keySet.Contains(m)).ToList(); + var extra = mapKeys.Where(k => !memberSet.Contains(k)).ToList(); + if (missing.Count > 0 || extra.Count > 0) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@{FIELD_ATTR_VALUES}' members" + + (missing.Count > 0 ? $" (missing: {string.Join(", ", missing)})" : "") + + (extra.Count > 0 ? $" (unknown: {string.Join(", ", extra)})" : "") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + + var seenValues = new Dictionary(); + foreach (var key in mapKeys) + { + // Skip a non-integer member value here — ValueMatchesType (Pass 6) + // already reported it; avoid a redundant/misleading double-report. + if (intValueMap[key] is not (long or int)) continue; + long value = System.Convert.ToInt64(intValueMap[key]); + if (seenValues.TryGetValue(value, out var owner)) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members '{owner}' and '{key}' " + + $"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + Envelope: field.Source)); + } + else + { + seenValues[value] = key; + } + } + } + + // Design D7, narrowed: @intValueMap is scalar-only. Int-backing is a + // persistence-layer CODEC and no port implements it element-wise over an + // array column: OMDB's EnumCodec and Kotlin's customEnumeration are scalar + // by construction, Python would bind the symbol LIST into an integer[], and + // TypeScript's sqlite branch serializes an array as JSON text before the enum + // case is reached. Two ports that happen to compose (this one, via + // PrimitiveCollection().ElementType(), and TS/Postgres) are not a feature — + // shipping a claim four ports silently get wrong is the + // field.byte/short/class mistake. + // + // BOTH halves are read RESOLVING, unlike the content rules above: the illegal + // thing is the EFFECTIVE combination. Post-#246 the map must live on the + // shared abstract declaration, so the field that inherits it is exactly where + // isArray gets declared — an own-only read would see the two halves on + // different nodes and never fire. + if (field.Attr(FIELD_ATTR_INT_VALUE_MAP) is IReadOnlyDictionary + && field.ResolvedIsArray()) + { + errors.Add(new MetaError( + $"field.enum '{field.Name}' declares '@{FIELD_ATTR_INT_VALUE_MAP}' with isArray=true; " + + "int-backing is scalar-only — an array-of-enum persists its member symbols. " + + $"Remove '@{FIELD_ATTR_INT_VALUE_MAP}', or make the field scalar.", + ErrorCode.ERR_ENUM_INT_VALUE_MAP_ARRAY, + Envelope: field.Source)); + } } foreach (var child in node.OwnChildren()) diff --git a/server/csharp/MetaObjects/SpecMetamodel/attr.json b/server/csharp/MetaObjects/SpecMetamodel/attr.json index 45f4cea6b..a57e1d6a9 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/attr.json +++ b/server/csharp/MetaObjects/SpecMetamodel/attr.json @@ -19,6 +19,12 @@ "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/server/csharp/MetaObjects/SpecMetamodel/field.json b/server/csharp/MetaObjects/SpecMetamodel/field.json index 01be74308..a80d56cf5 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/field.json +++ b/server/csharp/MetaObjects/SpecMetamodel/field.json @@ -150,7 +150,8 @@ "rules": "Required @values is a non-empty, duplicate-free set; each member must match ^[A-Za-z_][A-Za-z0-9_]*$ so symbol == stored string in every target language. Optional FR-010/FR-011 overlays add tolerant-extract aliasing (@enumAlias), per-member docs (@enumDoc), an uncoercible-value fallback (@coerceDefault, must be one of @values), and ASCII normalization mode (@normalize).", "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, - { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." } + { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, { diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt index f3e1c12c7..f68dd5598 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableGenerator.kt @@ -607,18 +607,14 @@ open class KotlinExposedTableGenerator : MultiFileDirectGeneratorBase null } + /** + * Exposed column spec for a `field.enum`. + * + * String-backed (the default) → `enumerationByName`: a VARCHAR column holding the + * member symbol. Int-backed (`@intValueMap`) → `customEnumeration` over an INTEGER + * column, with the declared symbol↔int mapping inlined as `when` expressions. + * + * The mapping is emitted INLINE rather than through a generated lookup-map support + * file, for two reasons. A `when` over an enum is exhaustive, so a member with no + * mapping is a COMPILE error in the adopter's build rather than a runtime surprise; + * and it allocates nothing per row, where a `mapOf(...)` inside the lambda would + * rebuild the map on every read and every write. + * + * ADR-0039: `@values` and `@intValueMap` are both read RESOLVING, so a field that + * `extends` a shared abstract enum inherits the members AND their mapping. + */ + private fun enumColumnSpec(field: EnumField, entity: MetaObject): String { + val enumCn = KotlinTypeMapper.enumTypeName(field, entity) + ?: error("enumTypeName returned null for EnumField '${field.name}' on ${entity.name}") + val colName = KotlinGenUtil.camelToSnake(field.name) + val simple = enumCn.simpleName + val intMap = readIntValueMap(field) + ?: return "enumerationByName(\"$colName\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, $simple::class)" + + // `; ` separates the branches: this is a one-line `when`, and space-separated + // branches do not parse (`Foo.DRAFT 5 -> ...`). + val fromDb = intMap.entries.joinToString("; ") { (sym, i) -> "$i -> $simple.$sym" } + val toDb = intMap.entries.joinToString("; ") { (sym, i) -> "$simple.$sym -> $i" } + // An int with no member is corrupt data the model does not describe: fail loudly + // rather than substituting a member. The write side needs no `else` — it is + // exhaustive over the enum by construction, since @intValueMap's keys are + // validated to match @values exactly. + return "customEnumeration(\"$colName\", \"INTEGER\", " + + "{ v -> when ((v as Number).toInt()) { $fromDb; " + + "else -> error(\"unmapped stored value \$v for $simple\") } }, " + + "{ e -> when (e) { $toDb } })" + } + + /** The declared symbol→int map (`@intValueMap`), or null when the enum is string-backed. */ + private fun readIntValueMap(field: EnumField): Map? { + if (!field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null + val raw = runCatching { field.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).value }.getOrNull() + val m = (raw as? Map<*, *>) ?: return null + val out = LinkedHashMap() + for ((k, v) in m) { + val key = k?.toString() ?: continue + val i = (v as? Number)?.toInt() ?: continue + out[key] = i + } + return out.ifEmpty { null } + } + // === MultiFileDirectGeneratorBase abstract-method stubs ==================== override fun writeSingleFile(md: MetaObject, writer: GeneratorIOWriter<*>?) { /* unused */ } override fun ?> getSingleWriter( diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt index 174e10665..7d2788d99 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinFilterAllowlistGenerator.kt @@ -173,7 +173,7 @@ open class KotlinFilterAllowlistGenerator : MultiFileDirectGeneratorBase = - com.metaobjects.query.FilterOps.opsForSubType(subType) + // + // Field-level, not subtype-level: an int-backed field.enum (@intValueMap, + // design D5) stores as an INTEGER column, so `like` — a substring match — + // is not in its band. Pinned cross-port by + // fixtures/conformance/filter-ops-matrix (fEnum vs fEnumInt). + private fun opsForField(field: MetaField<*>): Set = + com.metaobjects.query.FilterOps.opsForField(field) } // === MultiFileDirectGeneratorBase abstract-method stubs ==================== diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt index af7878882..97e275691 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/KotlinTypeMapper.kt @@ -196,12 +196,27 @@ object KotlinTypeMapper { // `splitFqn` collapsed to a root-package `Status` — colliding across every entity with a `status`. val immediateSuper = field.superField if (immediateSuper != null && runCatching { immediateSuper.declaringObject }.getOrNull() == null) { - // The DIRECT super is a package-level abstract enum → collapse onto the shared type. Name it - // for the top-most abstract root so a chain of abstract enums (abstract extends abstract) still - // collapses onto one type (resolveSuperRoot walks the abstract chain; == immediateSuper when - // there is no further super). - val superRoot = resolveSuperRoot(field) ?: immediateSuper - val (superPkg, superShort) = PackageMapping.splitFqn(superRoot.name) + // The DIRECT super is a package-level abstract enum → collapse onto the shared type, named + // for THAT declaration — the immediate super, NOT the top-most root of an abstract chain. + // + // Naming by the top-most root (the previous behaviour) contradicted this file's own FR-019 + // arm, which resolves the shared declaration from the IMMEDIATE super + // ([Fr019SharedEnum.resolveSharedEnumDecl]) exactly as TS/C#/Java/Python do. On a CHAINED + // declaration (root abstract `Money extends` root abstract `@provided Currency`) the two + // halves disagreed: the materialize-vs-reference decision saw `Money` (own @provided absent + // → materialize) while the NAME collapsed to `Currency`, so Kotlin emitted a local + // `Currency.kt` that collided with the `com.acme.ext.Currency` reference emitted for fields + // extending `Currency` directly. + // + // Per ADR-0026 §2 a materialized type is named for ITS OWN declaration, so a chain yields one + // type per declaration, each carrying the member set it inherits ([KotlinEnumEmitter + // .readEnumValues] is inheritance-aware across any number of hops). #246 guarantees a chained + // declaration can never MUTATE the vocabulary it inherits — it may rename it, nothing more. + // + // Non-chained output is byte-identical: with no further super the root walk returned the + // immediate super anyway. The #259 two-hop projection guard is the `declaringObject == null` + // condition above, which is evaluated FIRST and is unaffected. + val (superPkg, superShort) = PackageMapping.splitFqn(immediateSuper.name) return ClassName(superPkg, superShort.replaceFirstChar { it.uppercase() }) } @@ -214,23 +229,6 @@ object KotlinTypeMapper { } } - /** - * Walk a field's `extends` (super-field) chain to the top-most ancestor, returning it, or - * `null` when the field has no super. The top-most super is an abstract enum declared at the - * metadata root (e.g. `field.enum Priority @abstract`); naming the generated enum class after - * it makes every extending field share one type. Defensive against cycles via a visited set. - */ - private fun resolveSuperRoot(field: EnumField): MetaField<*>? { - var current: MetaField<*>? = field.superField ?: return null - val seen = HashSet() - while (true) { - val next = current?.superField ?: break - if (!seen.add(current.name)) break // cycle guard - current = next - } - return current - } - /** Map a MetaField to its KotlinPoet data-class property TypeName. */ fun kotlinTypeName(field: MetaField<*>): TypeName = when (field) { // This is the SCALAR/element mapper — `isArray` List<…> wrapping is applied by the diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt new file mode 100644 index 000000000..6746d9f5c --- /dev/null +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinExposedTableIntBackedEnumTest.kt @@ -0,0 +1,96 @@ +package com.metaobjects.generator.kotlin + +import com.metaobjects.metadata.ktx.loadDirectory +import com.tschuchort.compiletesting.KotlinCompilation +import com.tschuchort.compiletesting.SourceFile +import java.nio.file.Files +import java.nio.file.Path +import kotlin.io.path.isRegularFile +import kotlin.io.path.readText +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * Int-backed `field.enum` (`@intValueMap`) on the Exposed table generator. + * + * A string-backed enum persists its member symbol into a VARCHAR via + * `enumerationByName`. One declaring `@intValueMap` must instead persist the declared + * INTEGER via `customEnumeration`, while the Kotlin-side property type stays the very + * same generated enum class — int-backing is a persistence concern, invisible in the + * entity's API. + * + * The fixture carries BOTH modes on one entity (`priority` int-backed, `status` + * string-backed) so the test also pins that adding the feature did not change the + * string-backed emission. + * + * The compile gate is the load-bearing part. The generator hand-rolls its file body as + * a string, so `customEnumeration`'s two lambdas are only proven to type-check against + * the real Exposed API — the `Number`→`Int` narrowing on read, and the exhaustive `when` + * over the enum on write — by compiling the emitted tree. A text assertion alone would + * happily pass on a column spec that does not compile in the adopter's build. + */ +@OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) +class KotlinExposedTableIntBackedEnumTest { + + private val models: Path = run { + var p: Path? = Path.of(System.getProperty("user.dir")).toAbsolutePath() + while (p != null && !Files.exists(p.resolve("src/test/resources/models"))) { + p = p.parent + } + assertTrue(p != null, "could not locate src/test/resources/models from user.dir") + p!!.resolve("src/test/resources/models") + } + + private fun compile(outDir: Path): KotlinCompilation.Result { + val sources = Files.walk(outDir).filter { it.isRegularFile() }.sorted().toList() + .map { path -> SourceFile.kotlin(path.parent.relativize(path).toString().replace('/', '_'), path.readText()) } + return KotlinCompilation().apply { + this.sources = sources + inheritClassPath = true + messageOutputStream = System.out + }.compile() + } + + @Test fun `int-backed enum emits customEnumeration over INTEGER and still compiles`() { + val outDir = Files.createTempDirectory("ktbl-int-enum-") + try { + val loader = loadDirectory("enum-int-backed", models.resolve("enum-int-backed")) + for (gen in listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator())) { + gen.setArgs(mapOf("outputDir" to outDir.toString())) + gen.execute(loader) + } + + val table = outDir.resolve("acme/shop/OrderTable.kt") + assertTrue(Files.exists(table), "expected $table; files=${Files.walk(outDir).toList()}") + val src = table.readText() + + // int-backed → customEnumeration over an INTEGER column, carrying the + // declared mapping in BOTH directions. + assertTrue("customEnumeration(\"priority\", \"INTEGER\"" in src, + "priority must use customEnumeration over INTEGER; saw:\n$src") + assertTrue("0 -> OrderPriority.DRAFT" in src && "5 -> OrderPriority.PUBLISHED" in src && + "9 -> OrderPriority.ARCHIVED" in src, + "read lambda must map every declared int to its member; saw:\n$src") + assertTrue("OrderPriority.DRAFT -> 0" in src && "OrderPriority.PUBLISHED -> 5" in src && + "OrderPriority.ARCHIVED -> 9" in src, + "write lambda must map every member to its declared int; saw:\n$src") + + // ...and the int-backed column must NOT take the string form. + assertTrue("enumerationByName(\"priority\"" !in src, + "priority must not emit the string-backed column; saw:\n$src") + + // The string-backed sibling is unchanged — this feature is additive. + assertTrue("enumerationByName(\"status\", ${KotlinTypeMapper.ENUM_VARCHAR_LEN}, OrderStatus::class)" in src, + "string-backed status must keep enumerationByName; saw:\n$src") + assertTrue("customEnumeration(\"status\"" !in src, + "string-backed status must not become customEnumeration; saw:\n$src") + + // Compile gate: proves both lambdas type-check against the real Exposed API. + val result = compile(outDir) + assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode, result.messages) + } finally { + outDir.toFile().deleteRecursively() + } + } +} diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt index e920c77e8..9d7a36982 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinFr019SharedProvidedConformanceTest.kt @@ -79,6 +79,40 @@ class KotlinFr019SharedProvidedConformanceTest { } } + /** + * A CHAINED declaration: root abstract `Money extends` root abstract `@provided Currency`. + * + * Per ADR-0026 §2 a materialized type is named for ITS OWN declaration, and ADR-0039 makes + * `@provided` declaration-layer (it does not flow down the chain), so `Money` is its own type, + * materialized from the members it inherits. #246 guarantees the chain can rename the + * vocabulary but never mutate it. + * + * Kotlin used to name a chained enum after the TOP-MOST root while its FR-019 arm decided + * materialize-vs-reference from the IMMEDIATE super — so it emitted a local `Currency.kt` + * that collided with the `com.acme.ext.Currency` reference asserted above. All five ports + * must agree here. + */ + @Test + fun `chained abstract enum is materialized under its OWN name, not the roots`() { + val out = generate("com.acme.ext") + try { + val money = out.resolve("acme/shop/Money.kt") + assertTrue(Files.exists(money), "chained Money must be materialized under its own name") + val src = money.readText() + for (member in listOf("USD", "EUR", "GBP")) { + assertTrue(member in src, "Money must carry the inherited member $member; saw:\n$src") + } + // The chain must not drag the provided root into materialization. + assertFalse(Files.exists(out.resolve("acme/shop/Currency.kt")), + "provided Currency must still NOT be materialized") + val invoice = out.resolve("acme/shop/Invoice.kt").readText() + assertFalse("com.acme.ext.Money" in invoice, + "Money is materialized locally, not referenced externally; saw:\n$invoice") + } finally { + out.toFile().deleteRecursively() + } + } + @Test fun `provided enum with no namespace config is a codegen error naming the enum`() { val ex = assertThrows { generate(null) } diff --git a/server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json b/server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json new file mode 100644 index 000000000..ca4fc6f36 --- /dev/null +++ b/server/java/codegen-kotlin/src/test/resources/models/enum-int-backed/meta.shop.json @@ -0,0 +1,30 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Order", + "children": [ + { "source.rdb": { "name": "src", "@table": "orders", "@kind": "table" } }, + { "field.long": { "name": "id" } }, + { + "field.enum": { + "name": "priority", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } + } + }, + { + "field.enum": { + "name": "status", + "@values": ["LOW", "MEDIUM", "HIGH"] + } + }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } } + ] + } + } + ] + } +} diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java index f71fb47a8..519c5c631 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/spring/SpringFilterAllowlistGenerator.java @@ -115,7 +115,7 @@ static Map> computeFilterableOps(Iterable fields) for (MetaField field : fields) { if (field instanceof ObjectField) continue; if (!isFilterable(field)) continue; - Set ops = opsForSubtype(field.getSubType()); + Set ops = opsForField(field); if (ops.isEmpty()) continue; out.putIfAbsent(field.getName(), ops); // dedup base/subtype column names } @@ -130,12 +130,15 @@ protected static boolean isFilterable(MetaField field) { return Boolean.parseBoolean(String.valueOf(raw)); } - private static Set opsForSubtype(String subType) { + private static Set opsForField(MetaField field) { // Single source of truth — com.metaobjects.query.FilterOps (the same // band the loader's validation path reads). Returns a canonical-ordered // set so the emitted source is stable; an unbanded subtype → empty set, // which computeFilterableOps drops. - return com.metaobjects.query.FilterOps.opsForSubType(subType); + // + // Field-level, not subtype-level: an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. + return com.metaobjects.query.FilterOps.opsForField(field); } protected void emit(MetaObject entity, Path outRoot, MetaDataLoader loader) { diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt index 88ec7e9ed..081ccb599 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt @@ -33,6 +33,7 @@ import org.jetbrains.exposed.sql.update import org.jetbrains.exposed.sql.statements.InsertStatement import org.jetbrains.exposed.sql.transactions.transaction import com.metaobjects.integration.kotlin.tables.AllTypesTable +import com.metaobjects.integration.kotlin.tables.IntBackedEnumColumnType import java.math.BigDecimal import java.net.URI import java.sql.DriverManager @@ -418,6 +419,9 @@ object QueryScenarioRunner { */ private fun coerceForWrite(raw: Any?, col: Column<*>): Any? { if (raw == null) return null + // Int-backed field.enum: bind the member SYMBOL and let the column's codec encode it to + // the stored int (see the same guard in `coerce`). + if (col.columnType is IntBackedEnumColumnType) return raw.toString() val type = col.columnType.sqlType().lowercase() return when { type == "uuid" -> if (raw is UUID) raw else UUID.fromString(raw.toString().lowercase()) @@ -607,6 +611,10 @@ object QueryScenarioRunner { */ private fun coerce(raw: Any?, col: Column<*>): Any? { if (raw == null) return null + // An int-backed field.enum is a Column over INTEGER: the authoring value is the + // member SYMBOL and the column's own codec maps it to the stored int. Checked BEFORE the + // sqlType dispatch, whose `int` branch would try "PUBLISHED".toInt(). + if (col.columnType is IntBackedEnumColumnType) return raw.toString() val type = col.columnType.sqlType().lowercase() return when { // uuid columns compare against java.util.UUID — the YAML supplies a string diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt index 5c1a86473..9b29270d4 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/AllTypesTable.kt @@ -29,6 +29,7 @@ import org.jetbrains.exposed.sql.json.jsonb * - default `field.timestamp` (instant/TZ-aware, ADR-0036 Wave 2) → `instantWithTimeZone("tsTzVal")` (TIMESTAMPTZ) * - `field.currency` (@currency USD) → `long("moneyVal")` (BIGINT minor units) * - `field.enum` (@values LOW/MEDIUM/HIGH) → `varchar("enumVal", 64)` (text + CHECK in DDL) + * - `field.enum` + `@intValueMap` (int-backed) → `intBackedEnum("intEnumVal", …)` (INTEGER + int CHECK) * - `field.uuid` (non-key, @required) → `uuid("uuidVal")` (Postgres native uuid) * - `field.object` (@objectRef Settings, @storage jsonb) → `jsonb("settings", …)` (real Postgres JSONB) * - `field.object` (@objectRef Label, @storage jsonb, isArray) → `jsonb("labels", …)` (JSONB array) @@ -64,6 +65,13 @@ object AllTypesTable : Table("all_types") { val tsTzVal = instantWithTimeZone("tsTzVal") val moneyVal = long("moneyVal") val enumVal = varchar("enumVal", 64) + // INT-BACKED `field.enum` (@intValueMap): physically INTEGER (+ the canonical DDL's + // CHECK (… IN (0, 5, 9))), carrying the member SYMBOL in and out — storage changes, the + // wire format does not. Nullable to match the canonical DDL (`"intEnumVal" INTEGER`, no + // NOT NULL). See [IntBackedEnumColumnType] for why this is a Column here while the + // GENERATED form binds a real enum through customEnumeration. + val intEnumVal = + intBackedEnum("intEnumVal", mapOf("DRAFT" to 0, "PUBLISHED" to 5, "ARCHIVED" to 9)).nullable() val uuidVal = uuid("uuidVal") // `field.uri` → plain `text` column carrying the verbatim URI string (Postgres has no uri // type). See [MetaUriColumnType] — round-trips the URI unchanged. diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt new file mode 100644 index 000000000..2dd0d6d0a --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/IntBackedEnumColumnType.kt @@ -0,0 +1,60 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Column +import org.jetbrains.exposed.sql.ColumnType +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed column type for an INT-BACKED `field.enum` (`@intValueMap`) — + * the test-harness analogue of the `customEnumeration(...)` call + * [com.metaobjects.generator.kotlin.KotlinExposedTableGenerator] emits for such a field. + * + * The column is physically `INTEGER` (plus the canonical DDL's `CHECK (… IN (0, 5, 9))`) while + * the value carried in and out is the member SYMBOL — which is the whole contract of int-backing: + * storage changes, the wire format does not. + * + * It is a `Column` rather than a `Column` because this cross-port oracle has no + * generated enum class to bind — the generic [com.metaobjects.integration.kotlin.QueryScenarioRunner] + * moves plain YAML scalars. The GENERATED form binds a real Kotlin enum through + * `customEnumeration`, and is covered separately by the codegen tests; the divergence is the same + * one already documented for the jsonb columns in [AllTypesTable]. + * + * **A stored int that maps to no member THROWS** rather than surfacing the raw value or nulling + * it, matching the generated `customEnumeration`'s `else ->` branch and every sibling port: the + * row holds data the model says is impossible, and both alternatives hand the caller something + * untrue. The write side is exhaustive by construction upstream (`@intValueMap`'s keys are + * loader-validated to equal `@values`), so an unmapped SYMBOL here can only be a harness bug — + * it fails loudly for the same reason. + */ +internal class IntBackedEnumColumnType( + private val intByMember: Map, +) : ColumnType() { + + private val memberByInt: Map = + intByMember.entries.associate { (member, stored) -> stored to member } + + override fun sqlType(): String = "INTEGER" + + override fun valueFromDB(value: Any): String { + val stored = (value as? Number)?.toInt() + ?: error("int-backed field.enum column read a non-numeric value: $value") + return memberByInt[stored] + ?: error( + "field.enum read stored value $stored with no member in @intValueMap " + + "(declared: $intByMember) — the database holds a value the model does not describe." + ) + } + + override fun notNullValueToDB(value: String): Any = + intByMember[value] + ?: error("field.enum has no @intValueMap entry for member '$value' (declared: $intByMember).") + + override fun nonNullValueToString(value: String): String = notNullValueToDB(value).toString() +} + +/** + * Column builder for an int-backed `field.enum`: a `Column` carrying the member symbol + * over an `INTEGER` column, translating through [intByMember] (`@intValueMap`) in both directions. + */ +internal fun Table.intBackedEnum(name: String, intByMember: Map): Column = + registerColumn(name, IntBackedEnumColumnType(intByMember)) diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 06960b096..ef64e5921 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -430,6 +430,15 @@ public enum ErrorCode { */ ERR_ENUM_EXTENDS_VALUES_CONFLICT, + /** + * A {@code field.enum} carries {@code @intValueMap} together with + * {@code isArray=true}. Int-backing is a persistence-layer codec and no port + * implements it element-wise over an array column, so the combination would + * silently persist member SYMBOLS into an integer array. An array-of-enum + * stays string-backed: drop {@code @intValueMap}, or make the field scalar. + */ + ERR_ENUM_INT_VALUE_MAP_ARRAY, + /** An internal loader error with no stable error code. */ ERR_UNKNOWN, } diff --git a/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java b/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java index 2f37a4cd9..1af221b57 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java +++ b/server/java/metadata/src/main/java/com/metaobjects/attr/AttributeTypesMetaDataProvider.java @@ -21,6 +21,7 @@ *
  • REMOVED attr.stringarray: Use StringAttribute with @isArray=true instead
    • *
  • attr.class: Class attributes
    • *
  • attr.properties: Properties attributes
    • + *
  • attr.intMap: Object-shaped attributes whose values are all integers (e.g. field.enum's @intValueMap)
    • *
  • attr.filter: Filter constraint attributes (object-valued, owns desugar)
    • *
  • attr.expression: Expression tree attributes (object-valued, stored verbatim)
    • * @@ -56,6 +57,7 @@ public void registerTypes(MetaDataRegistry registry) { // StringArrayAttribute removed - use StringAttribute with @isArray instead ClassAttribute.registerTypes(registry); PropertiesAttribute.registerTypes(registry); + IntMapAttribute.registerTypes(registry); FilterAttribute.registerTypes(registry); ExpressionAttribute.registerTypes(registry); diff --git a/server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java b/server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java new file mode 100644 index 000000000..20aff58b2 --- /dev/null +++ b/server/java/metadata/src/main/java/com/metaobjects/attr/IntMapAttribute.java @@ -0,0 +1,152 @@ +package com.metaobjects.attr; + +import com.google.gson.Gson; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; +import com.google.gson.JsonSyntaxException; +import com.metaobjects.DataTypes; +import com.metaobjects.registry.MetaDataRegistry; + +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * An object-shaped attribute whose values are all integers (e.g. field.enum's + * {@code @intValueMap}: {@code {member: int}}). + * + *

    Mirrors {@link PropertiesAttribute}'s role as an object-shaped key/value + * attribute, but is backed by {@code Map} rather than + * {@code java.util.Properties} — {@code Properties} coerces every value to a + * {@code String} on load, which would silently corrupt int fidelity on + * canonical-JSON round-trip (TS/C#/Python all preserve real integers here).

    + * + *

    Declares {@link DataTypes#OBJECT} — the same data type {@link FilterAttribute} + * uses for its object-shaped value — rather than {@link DataTypes#CUSTOM}. This + * matters beyond typing: {@code CanonicalJsonSerializer#attrValueToJson} special-cases + * {@code DataTypes.OBJECT} values that are a {@code Map} and serializes them via + * {@code Gson.toJsonTree} (preserving integer values as JSON numbers); a + * {@code DataTypes.CUSTOM} attribute falls through to {@code Object#toString()}, + * which would emit Java's {@code Map.toString()} form ({@code "{DRAFT=0}"}) — not + * valid JSON. {@code PropertiesAttribute} gets away with {@code CUSTOM} only because + * the serializer has a SEPARATE special case keyed on {@code instanceof Properties}; + * there is no such case for a plain {@code Map}, so this class must use + * {@code DataTypes.OBJECT} to round-trip correctly.

    + * + *

    Generic shape check only here (object, every value an integer); a consumer + * field type (field.enum) layers its own semantic content rules (key-set membership + * against {@code @values}, no-duplicate-values) in its own post-load validation + * pass ({@code ValidationPhase}).

    + */ +public class IntMapAttribute extends MetaAttribute> { + + public static final String SUBTYPE_INT_MAP = "intMap"; + + private static final Gson GSON = new Gson(); + + /** + * Register this type with the MetaDataRegistry (called by + * {@link AttributeTypesMetaDataProvider}). + */ + public static void registerTypes(MetaDataRegistry registry) { + registry.registerType(IntMapAttribute.class, def -> def + .type(TYPE_ATTR).subType(SUBTYPE_INT_MAP) + .description("An object-shaped attribute whose values are all integers " + + "(e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape " + + "check only; a consumer field type layers its own semantic rules " + + "(key-set membership, uniqueness) in its own content-rule validation.") + .inheritsFrom(TYPE_ATTR, SUBTYPE_BASE) + ); + } + + public IntMapAttribute(String name) { + super(SUBTYPE_INT_MAP, name, DataTypes.OBJECT); + } + + /** + * Manually create an IntMap MetaAttribute with a value + */ + public static IntMapAttribute create(String name, Map value) { + IntMapAttribute a = new IntMapAttribute(name); + a.setValue(value); + return a; + } + + @Override + public void setValueAsObject(Object value) { + if (value == null) { + setValue(null); + } else if (value instanceof String) { + setValueAsString((String) value); + } else if (value instanceof Map) { + Map m = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) value).entrySet()) { + if (e.getKey() == null || e.getValue() == null) continue; + String key = e.getKey().toString(); + m.put(key, coerceInt(key, e.getValue())); + } + setValue(m); + } else { + throw new InvalidAttributeValueException( + "Can not set value with class [" + value.getClass() + "] for object: " + value); + } + } + + @Override + public void setValueAsString(String value) { + if (value == null) { setValue(null); return; } + String trimmed = value.trim(); + if (!(trimmed.startsWith("{") && trimmed.endsWith("}"))) { + throw new InvalidAttributeValueException( + "Could not parse intMap attribute '@" + getName() + + "' value (expected a JSON object): " + value); + } + JsonElement parsed; + try { + parsed = JsonParser.parseString(trimmed); + } catch (JsonSyntaxException e) { + throw new InvalidAttributeValueException( + "Could not parse intMap attribute '@" + getName() + "' value as JSON: " + value, e); + } + if (!parsed.isJsonObject()) { + throw new InvalidAttributeValueException( + "intMap attribute '@" + getName() + "' value must be a JSON object: " + value); + } + JsonObject obj = parsed.getAsJsonObject(); + Map m = new LinkedHashMap<>(); + for (Map.Entry e : obj.entrySet()) { + JsonElement el = e.getValue(); + if (el.isJsonPrimitive() && el.getAsJsonPrimitive().isNumber()) { + double d = el.getAsDouble(); + if (d != Math.floor(d) || Double.isInfinite(d) + || d < Integer.MIN_VALUE || d > Integer.MAX_VALUE) { + throw new InvalidAttributeValueException( + "attribute '@" + getName() + "' member '" + e.getKey() + + "' has value '" + el + "' which is not an integer"); + } + m.put(e.getKey(), (int) d); + } else { + throw new InvalidAttributeValueException( + "attribute '@" + getName() + "' member '" + e.getKey() + + "' has value '" + el + "' which is not an integer"); + } + } + setValue(m); + } + + private static int coerceInt(String key, Object value) { + if (value instanceof Integer i) return i; + if (value instanceof Number n && n.doubleValue() == Math.floor(n.doubleValue())) { + return n.intValue(); + } + throw new InvalidAttributeValueException( + "intMap value for member '" + key + "' is not an integer: " + value); + } + + @Override + public String getValueAsString() { + Map val = getValue(); + if (val == null) return "{}"; + return GSON.toJson(val); + } +} diff --git a/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java b/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java index 2700203d8..6a0374521 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java +++ b/server/java/metadata/src/main/java/com/metaobjects/field/EnumField.java @@ -78,6 +78,18 @@ public class EnumField extends PrimitiveField { */ public static final String ATTR_PROVIDED = "provided"; + /** + * Name of the optional per-member explicit-integer-value attribute + * ({@code {member: int}}), switching this enum field's DB persistence from + * string+CHECK to integer+CHECK. Keys must exactly match the field's effective + * {@code @values}; values must be unique integers ({@code ERR_BAD_ATTR_VALUE} + * otherwise — enforced post-load in + * {@link com.metaobjects.loader.ValidationPhase}). The generic "is this an + * object of integers" shape check runs in {@link com.metaobjects.attr.IntMapAttribute} + * itself. Cross-language vocabulary: {@code @intValueMap} in canonical JSON. + */ + public static final String ATTR_INT_VALUE_MAP = "intValueMap"; + /** * Name of the optional per-member description map (properties). * Each key is an enum member symbol from {@code @values}; the value is a @@ -198,6 +210,13 @@ public static void registerTypes(MetaDataRegistry registry) { .ofType(BooleanAttribute.SUBTYPE_BOOLEAN) .asSingle(); + // Optional @intValueMap — an object-shaped attribute whose values + // are all integers. Key-set-matches-@values and uniqueness are + // validated post-load in ValidationPhase (own-only, same as @values). + def.optionalAttributeWithConstraints(ATTR_INT_VALUE_MAP) + .ofType(com.metaobjects.attr.IntMapAttribute.SUBTYPE_INT_MAP) + .asSingle(); + // FR-033: @enumDoc and @coerceDefault are re-homed to the // metaobjects-prompt concern provider (reads // spec/metamodel/prompt.json's field.enum extends). diff --git a/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java b/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java index c596263b0..a646a617a 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java +++ b/server/java/metadata/src/main/java/com/metaobjects/io/json/CanonicalJsonSerializer.java @@ -651,10 +651,31 @@ private static JsonElement attrValueToJson(MetaAttribute attr) { // OBJECT-datatype attr with a Map value: emit as a JSON object. // Guard: DataTypes.OBJECT ensures only attrs that explicitly declare object - // semantics (e.g. FilterAttribute) take this path. Gson.toJsonTree handles - // nested Maps/Lists/primitives natively, so the full desugared filter - // structure ({field: {op: value}, ...}) round-trips correctly. + // semantics (e.g. FilterAttribute, IntMapAttribute) take this path. + // Gson.toJsonTree handles nested Maps/Lists/primitives natively, so the full + // desugared filter structure ({field: {op: value}, ...}) round-trips correctly. + // + // Cross-port value-shape heuristic (mirrors TS's sortAttrValue / C#'s + // AttrObjectToJsonNode): the schema-free serializer distinguishes canonical + // key order by value SHAPE, not by attribute subtype — + // • a flat scalar-valued map (e.g. attr.intMap's @intValueMap: {member: int}) + // sorts its keys alphabetically in canonical form (matches the Java + // PropertiesAttribute-style sorted-object precedent below). + // • a map carrying at least one nested object/array value (e.g. attr.filter's + // desugared {field: {op: value}} clauses, or/and composition arrays, or + // attr.expression's expression tree) preserves declaration order — that + // order is semantically significant and must not be reordered. if (attr.getDataType() == DataTypes.OBJECT && value instanceof Map) { + Map map = (Map) value; + boolean allScalar = map.values().stream().allMatch(v -> + v == null || v instanceof String || v instanceof Boolean || v instanceof Number); + if (allScalar) { + Map sorted = new java.util.TreeMap<>(); + for (Map.Entry e : map.entrySet()) { + sorted.put(String.valueOf(e.getKey()), e.getValue()); + } + return GSON.toJsonTree(sorted); + } return GSON.toJsonTree(value); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index c1087d362..fd2383668 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -69,6 +69,7 @@ import java.util.ArrayList; import java.util.Collections; +import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Locale; @@ -681,6 +682,17 @@ private static void validateEnumNode(MetaData node) { return; } + // --- Own @intValueMap content check (optional) --- + // Independent of the @values own/inherited branching below — an @intValueMap + // owned by this node is validated here against the node's EFFECTIVE @values + // (own or inherited via extends), mirroring the TS/C# port structure exactly. + validateEnumIntValueMap(node); + + // --- @intValueMap is scalar-only (design D7, narrowed) --- + // Separate from the content check above because it must fire on the node that + // combines the two halves, which is NOT necessarily the node declaring the map. + validateEnumIntValueMapNotArray(node); + // --- Own @values content check --- if (node.hasMetaAttr(EnumField.ATTR_VALUES, false)) { MetaAttribute valuesAttr = node.getMetaAttr(EnumField.ATTR_VALUES, false); @@ -698,8 +710,8 @@ private static void validateEnumNode(MetaData node) { // favor of the shared type's. Own-attrs-only (matches the check above): // only fires when THIS node declares @values itself, not when it merely // inherits. - MetaData sup = node.getSuperData(); - if (sup != null && isAbstract(sup) && sup.getParent() instanceof MetaRoot) { + MetaData sup = sharedEnumSuper(node); + if (sup != null) { throw new MetaDataException( ErrorMessageConstants.ERR_ENUM_EXTENDS_VALUES_CONFLICT + ": field.enum '" + node.getName() @@ -730,6 +742,128 @@ private static void validateEnumNode(MetaData node) { validateEnumFr011Attrs(node); } + /** + * Own {@code @intValueMap} content validation for a {@code field.enum} node. + * + *

    Optional. Own-only (mirrors the {@code @values}/FR-011 own-attrs-only policy) + * — an inherited {@code @intValueMap} is validated on its declaring node. The + * generic "is this an object of integers" shape check already ran via + * {@link com.metaobjects.attr.IntMapAttribute} at parse time (its + * {@code setValueAsString}/{@code setValueAsObject} reject a non-integer member + * with {@link com.metaobjects.attr.InvalidAttributeValueException}, which the + * parser's strict-mode catch re-wraps as {@code ERR_BAD_ATTR_VALUE}); this method + * validates the field.enum-SPECIFIC semantics: key-set-equals-effective-@values, + * and no two members share the same int value.

    + */ + /** + * The shared-enum super of {@code node}, or null when it has none. + * + *

    "Shared" (FR-019 / #246) means the immediate super is abstract AND declared at + * metadata-root — its parent is the {@link MetaRoot}, not an object. Such a declaration + * is materialized ONCE per port as a single named type, so anything a consuming field + * re-declares that belongs to the shared TYPE's contract (its {@code @values} member + * set, or the {@code @intValueMap} integer backing of that set) is a conflict. + * + *

    Immediate-super-only, matching codegen's {@code Fr019SharedEnum.resolveSharedEnumDecl} + * so the validator and the shared-enum collapse agree on what "shared" means. + */ + private static MetaData sharedEnumSuper(MetaData node) { + MetaData sup = node.getSuperData(); + return (sup != null && isAbstract(sup) && sup.getParent() instanceof MetaRoot) ? sup : null; + } + + /** + * {@code @intValueMap} is scalar-only (design D7, narrowed). + * + *

    Int-backing is a persistence-layer CODEC, and no port implements it element-wise + * over an array column: OMDB's {@code EnumCodec} and Kotlin's {@code customEnumeration} + * are scalar by construction, Python would bind the symbol LIST straight into an + * {@code integer[]}, and TypeScript's sqlite branch serializes an array as JSON text + * before the enum case is reached. Two ports that happen to compose (TS/Postgres, C#) + * are not a feature — shipping a claim four ports silently get wrong is the + * {@code field.byte}/{@code short}/{@code class} mistake.

    + * + *

    BOTH halves are read RESOLVING, unlike {@link #validateEnumIntValueMap}: the + * illegal thing is the EFFECTIVE combination. Post-#246 the map must live on the shared + * abstract declaration, so the field that inherits it is exactly where {@code isArray} + * gets declared — an own-only read would see the two halves on different nodes and + * never fire.

    + */ + private static void validateEnumIntValueMapNotArray(MetaData node) { + // hasMetaAttr defaults to includeParentData=true → RESOLVING. + if (!node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return; + if (!(node instanceof MetaField) || !((MetaField) node).isArrayType()) return; + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_INT_VALUE_MAP_ARRAY + + ": field.enum '" + node.getName() + "' declares @" + + EnumField.ATTR_INT_VALUE_MAP + " with isArray=true; int-backing is" + + " scalar-only - an array-of-enum persists its member symbols." + + " Remove @" + EnumField.ATTR_INT_VALUE_MAP + ", or make the field scalar.", + ErrorCode.ERR_ENUM_INT_VALUE_MAP_ARRAY, node.getSource()); + } + + private static void validateEnumIntValueMap(MetaData node) { + if (!node.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false)) { + return; + } + + // #246 (int-backed twin): the symbol->int mapping is a property of the enum + // VOCABULARY, not of one column that uses it - it is @values' numeric half. A + // shared enum is materialized once as a single type, so a per-field map would give + // one logical type N storage encodings (and, where a port emits per-TYPE codec + // artifacts, two same-named declarations). Same remedy as the @values half: + // declare it on the shared declaration and inherit it. + MetaData sharedSuper = sharedEnumSuper(node); + if (sharedSuper != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_ENUM_EXTENDS_VALUES_CONFLICT + + ": field.enum '" + node.getName() + + "' extends shared abstract enum '" + sharedSuper.getName() + + "' AND declares its own @" + EnumField.ATTR_INT_VALUE_MAP + + " - a shared enum's integer backing is owned by the shared declaration;" + + " move @" + EnumField.ATTR_INT_VALUE_MAP + " onto '" + sharedSuper.getName() + + "' to inherit it, or extend a non-shared enum instead", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, node.getSource()); + } + + @SuppressWarnings("unchecked") + Map intValueMap = (Map) + node.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP, false).getValue(); + if (intValueMap == null) { + return; + } + + List effective = effectiveEnumValues(node); + Set memberSet = new HashSet<>(effective); + Set keySet = intValueMap.keySet(); + + List missing = effective.stream().filter(m -> !keySet.contains(m)).toList(); + List extra = keySet.stream().filter(k -> !memberSet.contains(k)).toList(); + if (!missing.isEmpty() || !extra.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' keys must exactly match '@" + EnumField.ATTR_VALUES + "' members" + + (missing.isEmpty() ? "" : " (missing: " + String.join(", ", missing) + ")") + + (extra.isEmpty() ? "" : " (unknown: " + String.join(", ", extra) + ")") + ".", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + + Map seenValues = new HashMap<>(); + for (Map.Entry entry : intValueMap.entrySet()) { + Integer value = entry.getValue(); + String owner = seenValues.putIfAbsent(value, entry.getKey()); + if (owner != null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_BAD_ATTR_VALUE + + ": field.enum '" + node.getName() + "' attribute '@" + EnumField.ATTR_INT_VALUE_MAP + + "' members '" + owner + "' and '" + entry.getKey() + + "' share the same value " + value + "; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); + } + } + } + /** * FR-011 own-attr validation for a {@code field.enum} node: *
      @@ -2673,8 +2807,10 @@ private static void validateFilterClause(MetaObject obj, DataGridLayout grid, } private static java.util.Set allowedOpsFor(MetaField field) { + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. java.util.Set band = - com.metaobjects.query.FilterOps.opsForSubType(field.getSubType()); + com.metaobjects.query.FilterOps.opsForField(field); // Any subtype without a declared band (already rejected upstream by // validateFilterableHasSupportedOps) falls through to the string-shape // band, preserving the prior default. diff --git a/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java b/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java index 62a74e85b..e97085717 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java +++ b/server/java/metadata/src/main/java/com/metaobjects/query/FilterOps.java @@ -145,4 +145,47 @@ public static Set opsForSubType(String subType) { public static boolean supportsFiltering(String subType) { return subType != null && OPS_BY_SUBTYPE.containsKey(subType); } + + /** + * The int-backed-{@code enum} band: the {@code enum} band minus {@code like}. + * Hoisted so the narrowing is one named constant rather than a set filtered at + * every call site. + */ + public static final Set OPS_ENUM_INT_BACKED = + ordered(FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL); + + /** + * The filter-operator band for a FIELD — the entry point every consumer that has + * a field in hand must use (loader validation, the codegen-spring allowlist + * generator, the cross-port {@code field.filter-ops} capability). + * + *

      Identical to {@link #opsForSubType} except for ONE case: an int-backed + * {@code field.enum} (one declaring {@code @intValueMap}, design D5) persists as + * an INTEGER column, so {@code like} — a substring match — is dropped. + * {@code eq}/{@code ne}/{@code in} survive because the member symbol encodes to + * its integer before it reaches SQL; {@code like} has no such encoding, and an + * unencoded {@code LIKE 'DRAFT'} against an integer column is a request-time + * type error.

      + * + *

      {@link #opsForSubType} cannot express this — it only ever sees the subtype + * {@code "enum"} — and is deliberately left unchanged for the one caller that + * genuinely has no field: {@code ExpressionAttribute}'s declared operand type.

      + * + *

      ADR-0039: {@code hasMetaAttr(String)} defaults to {@code includeParentData = + * true}, so this read RESOLVES through {@code extends}. That is load-bearing, not + * incidental: post-#246 the map lives on a shared root-level abstract declaration + * and consuming fields INHERIT it, so an own-only read would see it absent on + * exactly the shape adopters are steered toward and wrongly keep {@code like}.

      + * + *

      Cross-port: {@code fixtures/conformance/filter-ops-matrix} pins + * {@code fEnum} vs {@code fEnumInt} in all five ports.

      + */ + public static Set opsForField(com.metaobjects.field.MetaField field) { + if (field == null) return Collections.emptySet(); + if (EnumField.SUBTYPE_ENUM.equals(field.getSubType()) + && field.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) { + return OPS_ENUM_INT_BACKED; + } + return opsForSubType(field.getSubType()); + } } diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index 9ee138b89..3b7e5f70d 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -255,6 +255,16 @@ private ErrorMessageConstants() { */ public static final String ERR_ENUM_EXTENDS_VALUES_CONFLICT = "ERR_ENUM_EXTENDS_VALUES_CONFLICT"; + /** + * A {@code field.enum} carries {@code @intValueMap} together with {@code isArray=true}. + * Int-backing is a persistence-layer codec and no port implements it element-wise over + * an array column, so the combination would silently persist member SYMBOLS into an + * integer array. An array-of-enum stays string-backed. + * + *

      Cross-language contract: {@code ERR_ENUM_INT_VALUE_MAP_ARRAY}. + */ + public static final String ERR_ENUM_INT_VALUE_MAP_ARRAY = "ERR_ENUM_INT_VALUE_MAP_ARRAY"; + /** * #208 (design doc §5 R6) warning: an {@code origin.*}-bearing (derived) own * field lives under a host object that declares an {@code @unmanaged} source. diff --git a/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java b/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java index 1e73b0232..f495fa7d7 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java +++ b/server/java/metadata/src/test/java/com/metaobjects/conformance/ScriptRunner.java @@ -149,7 +149,7 @@ private static void runOperation(MetaDataLoader loader, JsonObject op, } MetaField field = (MetaField) target; List got = new ArrayList<>( - com.metaobjects.query.FilterOps.opsForSubType(field.getSubType())); + com.metaobjects.query.FilterOps.opsForField(field)); assertStringNames(got, expect, index, invoke, failures); return; } diff --git a/server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java b/server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java new file mode 100644 index 000000000..d1f38a0a0 --- /dev/null +++ b/server/java/metadata/src/test/java/com/metaobjects/field/EnumFieldIntValueMapTest.java @@ -0,0 +1,113 @@ +package com.metaobjects.field; + +import com.metaobjects.ErrorCode; +import com.metaobjects.MetaDataException; +import com.metaobjects.loader.InMemoryStringSource; +import com.metaobjects.loader.MetaDataLoader; +import com.metaobjects.registry.SharedRegistryTestBase; +import org.junit.Test; + +import java.util.Collections; +import java.util.List; + +import static org.junit.Assert.*; + +/** + * Tests for {@code field.enum}'s optional {@code @intValueMap} attribute — an + * explicit per-member integer value map ({@code {member: int}}) that switches the + * field's DB persistence from string+CHECK to integer+CHECK. + * + *

      Cross-language contract (mirrors the TS/C# ports): {@code attr.intMap} + * (Java: {@link com.metaobjects.attr.IntMapAttribute}) is a generic object-shaped + * attribute whose values must all be integers; {@code field.enum} layers its own + * semantic rules on top (key-set exactly equals {@code @values}, no duplicate int + * values) — both enforced via {@code ERR_BAD_ATTR_VALUE}, no new error code.

      + */ +public class EnumFieldIntValueMapTest extends SharedRegistryTestBase { + + // ----------------------------------------------------------------------- + // Helpers — mirrors EnumFieldTest's loading idiom exactly. + // ----------------------------------------------------------------------- + + private MetaDataLoader newTestLoader() { + return createTestLoader("EnumFieldIntValueMapTest", Collections.emptyList()); + } + + private MetaDataLoader loadThrough(String canonical, String id) { + MetaDataLoader loader = newTestLoader(); + loader.load(List.of(new InMemoryStringSource(canonical, id))); + return loader; + } + + private static String model(String extra) { + return "{ \"metadata.root\": { \"package\": \"acme\", \"children\": [" + + "{ \"object.entity\": { \"name\": \"Order\", \"children\": [" + + "{ \"field.long\": { \"name\": \"id\" } }," + + "{ \"field.enum\": { \"name\": \"status\", \"@values\": [\"DRAFT\",\"PUBLISHED\",\"ARCHIVED\"]" + extra + " } }," + + "{ \"identity.primary\": { \"@fields\": \"id\" } }" + + "]}}]}}"; + } + + private MetaDataException loadExpectingError(String json, String id) { + try { + loadThrough(json, id); + fail("Expected MetaDataException"); + throw new AssertionError("unreachable"); + } catch (MetaDataException e) { + return e; + } + } + + // ----------------------------------------------------------------------- + // Tests + // ----------------------------------------------------------------------- + + @Test + public void validIntValueMapWithMatchingKeysAndUniqueIntsLoadsClean() { + loadThrough( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9}"), + "intvaluemap-valid-test.json"); + // No exception — success is the assertion. + } + + @Test + public void noIntValueMapStillLoadsCleanStringBackedDefault() { + loadThrough(model(""), "intvaluemap-absent-test.json"); + // No exception — success is the assertion. + } + + @Test + public void missingMemberKeyIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5}"), + "intvaluemap-missing-member-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + assertTrue(ex.getMessage().contains("ARCHIVED")); + } + + @Test + public void extraKeyNotInValuesIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 5, \"ARCHIVED\": 9, \"RETRACTED\": 12}"), + "intvaluemap-extra-key-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + assertTrue(ex.getMessage().contains("RETRACTED")); + } + + @Test + public void nonIntegerValueIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": \"zero\", \"PUBLISHED\": 5, \"ARCHIVED\": 9}"), + "intvaluemap-non-integer-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + } + + @Test + public void duplicateIntValueAcrossMembersIsRejected() { + MetaDataException ex = loadExpectingError( + model(", \"@intValueMap\": {\"DRAFT\": 0, \"PUBLISHED\": 0, \"ARCHIVED\": 9}"), + "intvaluemap-duplicate-test.json"); + assertEquals(ErrorCode.ERR_BAD_ATTR_VALUE, ex.getCode().orElseThrow()); + assertTrue(ex.getMessage().contains("DRAFT") && ex.getMessage().contains("PUBLISHED")); + } +} diff --git a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java index 45e09c788..cab32aa87 100644 --- a/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java +++ b/server/java/omdb/src/main/java/com/metaobjects/manager/db/codec/JdbcCodecs.java @@ -57,6 +57,7 @@ import java.time.OffsetDateTime; import java.time.ZoneOffset; import java.time.format.DateTimeParseException; +import java.util.LinkedHashMap; import java.util.Map; import java.util.UUID; import java.util.concurrent.ConcurrentHashMap; @@ -406,19 +407,87 @@ static final class CurrencyCodec implements JdbcFieldCodec { } /** - * {@code field.enum} ⇄ a text column. The enum is string-backed (its member symbol is the - * stored value — see {@code EnumField}); EnumField is backed by {@code DataTypes.STRING}, so - * this is bind-as-string / read-as-string, identical to {@link StringCodec} but registered - * explicitly so enum does not ride the generic {@link ObjectCodec} fallback. The DB - * {@code CHECK (col IN (...))} (emitted from {@code @values}) enforces membership. + * {@code field.enum} ⇄ a text OR integer column, string- or int-backed. + * + *

      A plain enum persists its member symbol as text (see {@code EnumField}, backed by + * {@code DataTypes.STRING}) — bind-as-string / read-as-string, identical to + * {@link StringCodec} but registered explicitly so enum does not ride the generic + * {@link ObjectCodec} fallback. The DB {@code CHECK (col IN (...))} (emitted from + * {@code @values}) enforces membership.

      + * + *

      One declaring {@code @intValueMap} persists the member's declared INTEGER instead, + * while the caller's contract stays the SYMBOL in both directions — int-backing is a + * persistence-layer concern, invisible above this codec.

      + * + *

      The map is read RESOLVING ({@code getMetaAttr}, ADR-0039): it is + * {@code @values}' numeric half, a logical property of the enum vocabulary that + * inherits through {@code extends}, so a concrete field extending a shared + * abstract enum encodes with the inherited map.

      */ static final class EnumCodec implements JdbcFieldCodec { @Override public void readInto(Object o, MetaField f, ResultSet rs, int j) throws SQLException { - f.setString(o, rs.getString(j)); + Map intMap = intValueMap(f); + if (intMap == null) { + f.setString(o, rs.getString(j)); + return; + } + int stored = rs.getInt(j); + if (rs.wasNull()) { + f.setString(o, null); + return; + } + for (Map.Entry e : intMap.entrySet()) { + if (e.getValue() != null && e.getValue() == stored) { + f.setString(o, e.getKey()); + return; + } + } + // A stored int with no member is data the model says is impossible — a + // hand-written INSERT, or a member removed without a migration. Throw: + // returning String.valueOf(stored) would hand the caller a "member" that is + // not one, and C#/Kotlin/TS type this property as a CLOSED enum, so a value + // like "7" is not even representable there. Nulling it would hide the + // corruption behind a nullable column. Matches every sibling port. + throw new SQLException( + "field.enum '" + f.getName() + "' read stored value " + stored + + " with no member in @" + EnumField.ATTR_INT_VALUE_MAP + + " (declared: " + intMap + ") — the database holds a value the model " + + "does not describe."); } + @Override public void write(PreparedStatement s, MetaField f, int j, Object v) throws SQLException { - if (v == null) s.setNull(j, Types.VARCHAR); - else s.setString(j, v.toString()); + Map intMap = intValueMap(f); + if (intMap == null) { + if (v == null) s.setNull(j, Types.VARCHAR); + else s.setString(j, v.toString()); + return; + } + if (v == null) { + s.setNull(j, Types.INTEGER); + return; + } + Integer mapped = intMap.get(v.toString()); + if (mapped != null) { + s.setInt(j, mapped); + return; + } + // Unmapped symbol: bind it unchanged so the column/CHECK rejects it. + // Membership is the database's to enforce; inventing a value here + // would turn a loud error into a silently wrong row. + s.setString(j, v.toString()); + } + + /** The declared symbol→int map, or null when the enum is string-backed. */ + private static Map intValueMap(MetaField f) { + if (!f.hasMetaAttr(EnumField.ATTR_INT_VALUE_MAP)) return null; + Object raw = f.getMetaAttr(EnumField.ATTR_INT_VALUE_MAP).getValue(); + if (!(raw instanceof Map)) return null; + Map out = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) raw).entrySet()) { + if (e.getKey() == null || !(e.getValue() instanceof Number)) continue; + out.put(String.valueOf(e.getKey()), ((Number) e.getValue()).intValue()); + } + return out.isEmpty() ? null : out; } } diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java index 912be69c7..d28371d04 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/codec/JdbcCodecRoundTripTest.java @@ -242,7 +242,8 @@ public void timestampCurrencyEnumRoundTripThroughOMDB() throws Exception { // The SP-H subtypes under test. vo.setDate("tsVal", ts); // TimestampCodec vo.setLong("moneyVal", 199900L); // CurrencyCodec (integer minor units) - vo.setString("status", "MEDIUM"); // EnumCodec + vo.setString("status", "MEDIUM"); // EnumCodec (string-backed) + vo.setString("priority", "PUBLISHED"); // EnumCodec (int-backed, @intValueMap) omdb.createObject(oc, vo); @@ -266,11 +267,96 @@ public void timestampCurrencyEnumRoundTripThroughOMDB() throws Exception { Long.valueOf(199900L), read.getLong("moneyVal")); assertEquals("EnumCodec must round-trip the member symbol", "MEDIUM", read.getString("status")); + + // int-backed enum (@intValueMap): the caller's contract is the SYMBOL in + // both directions — int-backing is invisible above the codec. + assertEquals("EnumCodec must round-trip an int-backed member as its symbol", + "PUBLISHED", read.getString("priority")); + + // ...and ask the DATABASE what it actually stored. A round-trip alone + // cannot tell a working int codec from one that wrote the symbol both + // ways, because a symmetric bug is self-consistent. PUBLISHED is declared + // as 5, and the column is INTEGER. + try (Connection c = getConnection(); + PreparedStatement ps = c.prepareStatement( + "SELECT priority FROM CODEC_SAMPLE WHERE label = ?")) { + ps.setString(1, label); + try (ResultSet rs = ps.executeQuery()) { + assertTrue("row present for raw column read", rs.next()); + assertEquals("the column must hold the declared int, not the symbol", + 5, rs.getInt(1)); + } + } } finally { omdb.releaseConnection(oc); } } + /** + * An int-backed {@code field.enum} column holding an integer that maps to NO member is + * data the model says is impossible. The codec must THROW rather than surface the raw + * int as a pseudo-symbol: the sibling ports type this property as a CLOSED enum, so a + * value like {@code "7"} is not representable there, and returning null would hide the + * corruption behind a nullable column. + * + *

      Written through OMDB with a legal member, then corrupted with raw SQL — the same + * shape as the drift this guards against (a hand-written INSERT, or a member removed + * without a migration).

      + */ + @Test + public void intBackedEnumThrowsOnAnUnmappedStoredValue() throws Exception { + MetaObject mo = registry.findMetaObjectByName("codectest::Sample"); + assertNotNull(mo); + + ObjectConnection oc = omdb.getConnection(); + try { + ValueObject vo = (ValueObject) mo.newInstance(); + String label = "unmapped-int-enum-" + System.currentTimeMillis(); + vo.setString("label", label); + vo.setInt("count", 1); + vo.setLong("bignum", 1L); + vo.setBoolean("active", false); + vo.setDouble("ratio", 0d); + vo.setFloat("rate", 0f); + vo.setObject("amount", java.math.BigDecimal.ZERO); + vo.setDate("createdAt", new Date(0)); + vo.setObject("startTime", LocalTime.of(0, 0, 0)); + vo.setString("priority", "PUBLISHED"); + omdb.createObject(oc, vo); + + // 7 is in no member's @intValueMap (DRAFT=0, PUBLISHED=5, ARCHIVED=9). + try (Connection c = getConnection(); + PreparedStatement ps = c.prepareStatement( + "UPDATE CODEC_SAMPLE SET priority = 7 WHERE label = ?")) { + ps.setString(1, label); + assertEquals("exactly one row corrupted", 1, ps.executeUpdate()); + } + + try { + omdb.getObjects(oc, mo, + new QueryOptions(new Expression("label", label, Expression.EQUAL))); + fail("reading an unmapped int-backed enum value must throw, not surface it"); + } catch (Exception e) { + assertTrue("the failure must name the unmapped value: " + messageChain(e), + messageChain(e).contains("7")); + assertTrue("the failure must name the attribute: " + messageChain(e), + messageChain(e).contains("intValueMap")); + } + } finally { + omdb.releaseConnection(oc); + } + } + + /** Every message in a throwable's cause chain, joined — OMDB wraps driver exceptions. */ + private static String messageChain(Throwable t) { + StringBuilder sb = new StringBuilder(); + for (Throwable c = t; c != null; c = c.getCause()) { + sb.append(c.getMessage()).append(" | "); + if (c.getCause() == c) break; + } + return sb.toString(); + } + /** * {@link UuidCodec} read-back-lowercase contract at the raw codec/JDBC boundary. The * native-uuid WRITE bind ({@code setObject(.., Types.OTHER)}) is Postgres-only (Derby diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java index 3f6549503..a1d8aab0e 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/test/CodecSchema.java @@ -53,7 +53,11 @@ private CodecSchema() {} + " startTime TIME,\n" + " tsVal TIMESTAMP,\n" + " moneyVal BIGINT,\n" - + " status VARCHAR(20)\n" + + " status VARCHAR(20),\n" + // int-backed field.enum (@intValueMap): the member symbol persists as + // its declared INTEGER, so this column's type differs from `status` + // above even though both are field.enum. + + " priority INTEGER\n" + ")"; /** Executes the CODEC_SAMPLE DDL on a fresh connection from {@code connector}. */ diff --git a/server/java/omdb/src/test/resources/meta.codec.json b/server/java/omdb/src/test/resources/meta.codec.json index ce2eb1124..a717669e0 100644 --- a/server/java/omdb/src/test/resources/meta.codec.json +++ b/server/java/omdb/src/test/resources/meta.codec.json @@ -95,6 +95,14 @@ "@values": ["LOW", "MEDIUM", "HIGH"] } }, + { + "field.enum": { + "name": "priority", + "@column": "priority", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9} + } + }, { "identity.primary": { "name": "primary", diff --git a/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py b/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py index 6510ffcee..f7dd76b40 100644 --- a/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py +++ b/server/python/src/metaobjects/codegen/generators/filter_allowlist_generator.py @@ -53,6 +53,11 @@ # uuid: identity-comparison only — no `like` (not a substring type), no ordering. _OPS_UUID: tuple[str, ...] = ("eq", "ne", "in", "isNull") _OPS_NUMERIC: tuple[str, ...] = ("eq", "ne", "gt", "gte", "lt", "lte", "in", "isNull") +# int-backed enum (@intValueMap, design D5): the enum band minus `like` — the column +# is an integer, and a substring match against an integer is meaningless. Distinct +# constant from _OPS_UUID despite the identical members: they narrow for different +# reasons and would diverge independently. +_OPS_ENUM_INT_BACKED: tuple[str, ...] = ("eq", "ne", "in", "isNull") _OPS_BOOLEAN: tuple[str, ...] = ("eq", "isNull") @@ -118,6 +123,36 @@ def ops_for_subtype_ordered(sub_type: str | None) -> tuple[str, ...]: return () +def ops_for_field_ordered(field: MetaField) -> tuple[str, ...]: + """The operator tuple for a FIELD, in canonical operator order. + + The ordered counterpart of ``validation_passes.ops_for_field``, and the entry + point for every consumer that has a field in hand: the codegen allowlist emit + and the cross-port ``field.filter-ops`` conformance capability. + + Identical to :func:`ops_for_subtype_ordered` except for ONE case: an int-backed + ``field.enum`` (one declaring ``@intValueMap``, design D5) persists as an INTEGER + column, so ``like`` -- a substring match -- is dropped. ``eq``/``ne``/``in`` + survive because the member symbol encodes to its integer before it reaches SQL; + ``like`` has no such encoding, and an unencoded ``LIKE 'DRAFT'`` against an integer + column is a request-time type error. + + ADR-0039: the ``@intValueMap`` read is RESOLVING (``attrs()``, NOT ``attr()`` -- + Python inverts the TS naming, ``attr()`` here is own-only). Post-#246 the map lives + on a shared root-level abstract declaration and consuming fields INHERIT it, so an + own-only read would see it absent on exactly the shape adopters are steered toward + and wrongly keep ``like``. + + Cross-port: ``fixtures/conformance/filter-ops-matrix`` pins ``fEnum`` vs + ``fEnumInt`` in all five ports. + """ + if field.sub_type == fc.FIELD_SUBTYPE_ENUM and isinstance( + field.attrs().get(fc.FIELD_ATTR_INT_VALUE_MAP), dict + ): + return _OPS_ENUM_INT_BACKED + return ops_for_subtype_ordered(field.sub_type) + + def _is_filterable(field: MetaField) -> bool: """True iff ``field`` carries ``@filterable: true`` as a metadata attribute. @@ -166,10 +201,10 @@ class FilterAllowlistGenerator: name = "filter-allowlist-generator" def _ops_for_field(self, field: MetaField) -> tuple[str, ...]: - """The operator tuple allowed for ``field`` (FR-009 §5 per-subtype matrix by - default). Override to customize the per-field operator vocabulary; returning - an empty tuple excludes the field from the allowlist.""" - return ops_for_subtype_ordered(field.sub_type) + """The operator tuple allowed for ``field`` (FR-009 §5 matrix by default). + Override to customize the per-field operator vocabulary; returning an empty + tuple excludes the field from the allowlist.""" + return ops_for_field_ordered(field) def _compute_filterable_ops( self, entity: MetaObject, object_index: dict[str, MetaObject] | None = None diff --git a/server/python/src/metaobjects/core_types.py b/server/python/src/metaobjects/core_types.py index 80f22c974..0ea8c9955 100644 --- a/server/python/src/metaobjects/core_types.py +++ b/server/python/src/metaobjects/core_types.py @@ -9,6 +9,7 @@ ATTR_SUBTYPE_EXPRESSION, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_INT_MAP, ATTR_SUBTYPE_STRING, ATTR_SUBTYPES, ) @@ -19,6 +20,7 @@ FIELD_ATTR_AUTO_SET, FIELD_ATTR_CURRENCY, FIELD_ATTR_DEFAULT, + FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_PROVIDED, FIELD_ATTR_MAX_LENGTH, FIELD_ATTR_OBJECT_REF, @@ -472,6 +474,17 @@ def _register_subtypes( value_type=ATTR_SUBTYPE_BOOLEAN, required=False, ), + # int-backed-enum-values plan — @intValueMap: optional per-member int + # values ({member: int}) switching this enum field's DB persistence + # from string+CHECK to integer+CHECK. Keys must exactly match @values; + # values must be unique integers (enforced in _validate_enum_values, + # Rule 4). Structural (core), like @values / @provided — not a + # TS-web-only / prompt-domain overlay. + AttrSchema( + name=FIELD_ATTR_INT_VALUE_MAP, + value_type=ATTR_SUBTYPE_INT_MAP, + required=False, + ), # FR-033 — the field.enum tolerant-extract overlays (@enumAlias / @enumDoc / # @coerceDefault / @normalize) are NO LONGER declared here; they are re-homed # to the prompt domain provider (`prompt_provider`, metaobjects-prompt) via diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index db5ef7139..d98915089 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -194,6 +194,12 @@ class ErrorCode(str, Enum): # own @values would be silently dropped in codegen. Remove the own @values # to inherit the shared set, or extend a concrete (non-shared) enum instead. ERR_ENUM_EXTENDS_VALUES_CONFLICT = "ERR_ENUM_EXTENDS_VALUES_CONFLICT" + # A field.enum carries @intValueMap together with isArray=true. Int-backing is + # a persistence-layer codec and no port implements it element-wise over an + # array column, so the combination would silently persist member SYMBOLS into + # an integer array. An array-of-enum stays string-backed: drop @intValueMap, + # or make the field scalar. + ERR_ENUM_INT_VALUE_MAP_ARRAY = "ERR_ENUM_INT_VALUE_MAP_ARRAY" ERR_UNKNOWN = "ERR_UNKNOWN" diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 1febddf78..893ee79a6 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -23,6 +23,7 @@ ENUM_MEMBER_PATTERN, FIELD_ATTR_COERCE_DEFAULT, FIELD_ATTR_DEFAULT, + FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_REQUIRED, FIELD_ATTR_STORAGE, @@ -60,6 +61,7 @@ SOURCE_ROLE_PRIMARY, ) from ..meta.core.attr.attr_constants import ( + ATTR_SUBTYPE_INT_MAP, ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_STRINGARRAY, ) @@ -281,6 +283,26 @@ def _type_ok(value: object, value_type: str) -> bool: # fails here → ERR_BAD_ATTR_VALUE, matching TS + Java (fail-closed); an object # @expr then flows to the closed-grammar check in the origin pass. return isinstance(value, dict) + if value_type == ATTR_SUBTYPE_INT_MAP: + # attr.intMap (int-backed-enum-values plan): an object whose every member + # value is an integer (e.g. field.enum's @intValueMap). This port has no + # per-attr-class validateValue dispatch (that's TS's architecture); the + # generic shape check lives here, mirroring C#'s ValueMatchesType + # ATTR_SUBTYPE_INT_MAP case. field.enum's own key-set/uniqueness content + # rules run separately in _validate_enum_values (Rule 4), which skips + # re-reporting a non-integer member already caught here. + # + # Final-review fix: also bound every value to the 32-bit signed int range + # (inclusive) — the eventual DB column for an int-backed enum is a 32-bit + # Postgres/SQLite `integer` (design doc D5), matching Java's + # IntMapAttribute#setValueAsString bound check exactly. Python ints have + # no fixed width, so this is the only place that boundary is enforced. + return isinstance(value, dict) and all( + isinstance(v, int) + and not isinstance(v, bool) + and -2147483648 <= v <= 2147483647 + for v in value.values() + ) # Unknown value types (e.g. "class") — allow anything. return True @@ -544,6 +566,32 @@ def _validate_attr_schema( _ENUM_MEMBER_RE = re.compile(ENUM_MEMBER_PATTERN) +def _shared_enum_super(node: MetaData) -> MetaData | None: + """The shared-enum super of ``node``, or ``None`` when it has none. + + "Shared" (FR-019 / #246) means the immediate super is abstract AND declared at + metadata-root — its parent is the ``metadata.root`` node, not an object. Such a + declaration is materialized ONCE per port as a single named type, so anything a + consuming field re-declares that belongs to the shared TYPE's contract (its + ``@values`` member set, or the ``@intValueMap`` integer backing of that set) is a + conflict. A concrete super, or a non-root abstract super (e.g. one nested inside + an object), is legal and not flagged. + + Immediate-super-only, matching codegen's ``resolve_shared_enum_decl`` + (``codegen/generators/fr019_shared_enum.py``) so the validator and the + shared-enum collapse agree on what "shared" means. + """ + sup = node.super_data + if ( + sup is not None + and sup.is_abstract + and sup.parent is not None + and sup.parent.type == TYPE_METADATA + ): + return sup + return None + + def _validate_enum_values( root: MetaData, errors: list[MetaError], @@ -556,6 +604,21 @@ def _validate_enum_values( # @coerceDefault / @default / @normalize while inheriting @values). _validate_enum_fr011_attrs(node, errors) + # Rule 4 (@intValueMap content rules) is independent of whether @values is + # own or inherited on THIS node — it must run whenever this node owns + # @intValueMap, using the EFFECTIVE @values (own-or-inherited) as the + # membership set. Mirrors TS/C#/Java, which all check this unconditionally + # (Java's `validateEnumIntValueMap` is a standalone call at the top of + # `validateEnumNode`). Deliberately called BEFORE the `own_values is None: + # continue` gate below so a concrete field.enum that inherits @values via + # `extends` but declares its own @intValueMap is still validated. + _validate_enum_int_value_map(node, errors) + + # Design D7, narrowed: int-backing is scalar-only. Separate from the content + # rules above because it must fire on the node that combines the two halves, + # which is NOT necessarily the node that declares the map. + _validate_enum_int_value_map_not_array(node, errors) + # ADR-0039 sanctioned own: validates the AUTHORED @values membership on THIS # node (mirrors the TS attr-schema-validate `node.ownAttrs()`); an inherited # @values yields None here and is validated on its declaring node. @@ -607,19 +670,9 @@ def _validate_enum_values( # #246: own @values AND extends a shared package-level abstract enum — # one shared enum type has one member set, so the own @values would be - # silently dropped by the shared-enum codegen collapse. "Shared" means - # the resolved super is abstract AND declared at metadata-root (its - # parent is the metadata.root node, not nested inside an object) — a - # concrete super, or a non-root abstract super (e.g. nested inside an - # object), is legal and not flagged. Mirrors the TS reference - # (attr-schema-validate.ts). - sup = node.super_data - if ( - sup is not None - and sup.is_abstract - and sup.parent is not None - and sup.parent.type == TYPE_METADATA - ): + # silently dropped by the shared-enum codegen collapse. Mirrors the TS + # reference (attr-schema-validate.ts). + if _shared_enum_super(node) is not None: errors.append( MetaError( f"{label} declares its own '@{FIELD_ATTR_VALUES}' but extends " @@ -633,6 +686,113 @@ def _validate_enum_values( ) +def _validate_enum_int_value_map_not_array(node: MetaData, errors: list[MetaError]) -> None: + """``@intValueMap`` is scalar-only (design D7, narrowed). + + Int-backing is a persistence-layer CODEC, and no port implements it + element-wise over an array column: this port's ``ObjectManager`` would bind the + symbol LIST straight into an ``integer[]``, OMDB's ``EnumCodec`` and Kotlin's + ``customEnumeration`` are scalar by construction, and TypeScript's sqlite branch + serializes an array as JSON text before the enum case is reached. Two ports that + happen to compose (TS/Postgres, C#) are not a feature — shipping a claim four + ports silently get wrong is the ``field.byte``/``short``/``class`` mistake. + + BOTH halves are read RESOLVING, unlike the content rules: the illegal thing is + the EFFECTIVE combination. Post-#246 the map must live on the shared abstract + declaration, so the field that inherits it is exactly where ``isArray`` gets + declared — an own-only read would see the two halves on different nodes and + never fire. + """ + if not isinstance(node.get_meta_attr(FIELD_ATTR_INT_VALUE_MAP), dict): + return + if not node.resolved_is_array(): + return + errors.append( + MetaError( + f"{_node_label(node)} declares '@{FIELD_ATTR_INT_VALUE_MAP}' with isArray=true; " + f"int-backing is scalar-only — an array-of-enum persists its member symbols. " + f"Remove '@{FIELD_ATTR_INT_VALUE_MAP}', or make the field scalar.", + ErrorCode.ERR_ENUM_INT_VALUE_MAP_ARRAY, + envelope=node.source, + ) + ) + + +def _validate_enum_int_value_map(node: MetaData, errors: list[MetaError]) -> None: + """``@intValueMap`` content rules (optional), independent of whether ``@values`` + is own or inherited on this node. Mirrors TS/C#/Java, which all run this check + unconditionally against the node's EFFECTIVE ``@values`` — a concrete + ``field.enum`` that inherits ``@values`` via ``extends`` but declares its own + ``@intValueMap`` locally must still have that map validated. + + a. Key set must exactly match the EFFECTIVE ``@values`` (own or inherited). + b. No two members may share the same int (protobuf's stance — no alias + opt-in). Every-value-is-an-integer is already enforced by ``_type_ok``'s + ATTR_SUBTYPE_INT_MAP case at parse time. + """ + int_value_map = node.attr(FIELD_ATTR_INT_VALUE_MAP) + if not isinstance(int_value_map, dict): + return + + label = _node_label(node) + + # #246 (int-backed twin): the symbol->int mapping is a property of the enum + # VOCABULARY, not of one column that uses it -- it is @values' numeric half. A + # shared enum is materialized once as a single type, so a per-field map would + # give one logical type N storage encodings (and, where a port emits per-TYPE + # codec artifacts, two same-named declarations). Same remedy as the @values + # half: declare it on the shared declaration and inherit it. + shared_super = _shared_enum_super(node) + if shared_super is not None: + errors.append( + MetaError( + f"{label} declares its own '@{FIELD_ATTR_INT_VALUE_MAP}' but extends " + f"a shared package-level abstract enum — a shared enum's integer " + f"backing is owned by the shared declaration. Move " + f"'@{FIELD_ATTR_INT_VALUE_MAP}' onto the shared declaration to " + f"inherit it, or extend a concrete (non-shared) enum instead", + ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT, + envelope=node.source, + ) + ) + effective_values = _effective_enum_values(node) + member_set = set(effective_values) + key_set = set(int_value_map.keys()) + missing = [m for m in effective_values if m not in key_set] + extra = [k for k in int_value_map if k not in member_set] + if missing or extra: + parts = [] + if missing: + parts.append(f"missing: {', '.join(missing)}") + if extra: + parts.append(f"unknown: {', '.join(extra)}") + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match " + f"'@{FIELD_ATTR_VALUES}' members ({'; '.join(parts)}).", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + + seen_values: dict[int, str] = {} + for member, value in int_value_map.items(): + if not isinstance(value, int) or isinstance(value, bool): + continue # _type_ok already reported this + owner = seen_values.get(value) + if owner is not None: + errors.append( + MetaError( + f"{label} attribute '@{FIELD_ATTR_INT_VALUE_MAP}' members {owner!r} and {member!r} " + f"share the same value {value}; every member must have a unique int.", + ErrorCode.ERR_BAD_ATTR_VALUE, + envelope=node.source, + ) + ) + else: + seen_values[value] = member + + def _effective_enum_values(node: MetaData) -> list[str]: """The effective ``@values`` members of an enum node (own or inherited via ``extends:``). Empty list when absent. Mirrors Java ``effectiveEnumValues``.""" @@ -913,6 +1073,45 @@ def ops_for_subtype(field_subtype: str) -> frozenset[str]: return frozenset() +# The int-backed-enum band: the enum band minus `like`. Named so the narrowing is +# one constant rather than a set rebuilt at every call. +_OPS_ENUM_INT_BACKED: frozenset[str] = frozenset({"eq", "ne", "in", "isNull"}) + + +def ops_for_field(field: MetaData) -> frozenset[str]: + """The filter-operator band for a FIELD. + + The entry point every consumer that has a field in hand must use (loader + validation, the codegen filter-allowlist generator, the cross-port + ``field.filter-ops`` capability). + + Identical to :func:`ops_for_subtype` except for ONE case: an int-backed + ``field.enum`` (one declaring ``@intValueMap``, design D5) persists as an + INTEGER column, so ``like`` -- a substring match -- is dropped. + ``eq``/``ne``/``in`` survive because the member symbol encodes to its integer + before it reaches SQL; ``like`` has no such encoding, and an unencoded + ``LIKE 'DRAFT'`` against an integer column is a request-time type error. + + ``ops_for_subtype`` cannot express this -- it only ever sees the subtype + ``"enum"`` -- and is deliberately left unchanged for the one caller that + genuinely has no field: the expression grammar's declared operand type. + + ADR-0039: the ``@intValueMap`` read is RESOLVING (``attrs()``, NOT ``attr()`` + -- Python inverts the TS naming, ``attr()`` here is own-only). Post-#246 the + map lives on a shared root-level abstract declaration and consuming fields + INHERIT it, so an own-only read would see it absent on exactly the shape + adopters are steered toward and wrongly keep ``like``. + + Cross-port: ``fixtures/conformance/filter-ops-matrix`` pins ``fEnum`` vs + ``fEnumInt`` in all five ports. + """ + if field.sub_type == FIELD_SUBTYPE_ENUM and isinstance( + field.attrs().get(FIELD_ATTR_INT_VALUE_MAP), dict + ): + return _OPS_ENUM_INT_BACKED + return ops_for_subtype(field.sub_type) + + # --------------------------------------------------------------------------- # #195 — attr.expression closed grammar (validate + infer) # --------------------------------------------------------------------------- @@ -1095,8 +1294,10 @@ def _validate_datagrid_filter_values( continue # Build filterable map: field_name → allowed ops set + # ops_for_field, not ops_for_subtype — an int-backed field.enum + # (@intValueMap) stores as an integer, so `like` is not in its band. filterable: dict[str, frozenset[str]] = { - f.name: ops_for_subtype(f.sub_type) + f.name: ops_for_field(f) for f in node.fields() if f.attrs().get("filterable") is True } diff --git a/server/python/src/metaobjects/meta/core/attr/attr_constants.py b/server/python/src/metaobjects/meta/core/attr/attr_constants.py index 3b76a80fe..ab2e4ce7f 100644 --- a/server/python/src/metaobjects/meta/core/attr/attr_constants.py +++ b/server/python/src/metaobjects/meta/core/attr/attr_constants.py @@ -14,6 +14,13 @@ # ATTR_SUBTYPE_EXPRESSION / meta-attr-expression.ts. ATTR_SUBTYPE_EXPRESSION = "expression" +# attr.intMap (int-backed-enum-values plan) — an object-shaped attribute whose +# values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). +# Generic shape check only; field.enum layers its own semantic rules (key-set +# membership, uniqueness) in its own content-rule validation pass. Mirrors TS +# ATTR_SUBTYPE_INT_MAP / C# AttrConstants.ATTR_SUBTYPE_INT_MAP. +ATTR_SUBTYPE_INT_MAP = "intMap" + # The retired "stringarray" array attr subtype. It is NO LONGER a registered # (attr, sub_type) (not in ATTR_SUBTYPES) — array-ness is modeled as a "string" # attr with the orthogonal AttrSchema.is_array flag (matching Java's @@ -33,4 +40,5 @@ ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_CLASS, ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, ) diff --git a/server/python/src/metaobjects/meta/core/attr/meta_attr.py b/server/python/src/metaobjects/meta/core/attr/meta_attr.py index 034d437f1..f219bbf59 100644 --- a/server/python/src/metaobjects/meta/core/attr/meta_attr.py +++ b/server/python/src/metaobjects/meta/core/attr/meta_attr.py @@ -12,6 +12,7 @@ ATTR_SUBTYPE_EXPRESSION, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_INT_MAP, ATTR_SUBTYPE_LONG, ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_STRING, @@ -134,6 +135,21 @@ def data_type(self) -> DataType: return DataType.OBJECT +class IntMapAttr(MetaAttr): + """attr.intMap (int-backed-enum-values plan) — an object-shaped attribute + whose values are all integers (e.g. field.enum's @intValueMap). Generic + shape check only; the deeper "every value is an int" check is enforced by + `_type_ok`'s ATTR_SUBTYPE_INT_MAP case in validation_passes.py (this port + has no per-attr-class validate_value dispatch — TS's is the one port that + does; C# mirrors the same "static type-check function" architecture this + port uses). field.enum's own key-set/uniqueness content rules run in a + separate pass. Mirrors TS IntMapAttr / C# ATTR_SUBTYPE_INT_MAP shape.""" + + @property + def data_type(self) -> DataType: + return DataType.OBJECT + + class ClassAttr(MetaAttr): @property def data_type(self) -> DataType: @@ -145,4 +161,5 @@ def data_type(self) -> DataType: register_attr_class(ATTR_SUBTYPE_FILTER, FilterAttr) register_attr_class(ATTR_SUBTYPE_PROPERTIES, PropertiesAttr) register_attr_class(ATTR_SUBTYPE_EXPRESSION, ExpressionAttr) +register_attr_class(ATTR_SUBTYPE_INT_MAP, IntMapAttr) register_attr_class(ATTR_SUBTYPE_CLASS, ClassAttr) diff --git a/server/python/src/metaobjects/meta/core/field/field_constants.py b/server/python/src/metaobjects/meta/core/field/field_constants.py index 8eda7ea82..2eca57fa3 100644 --- a/server/python/src/metaobjects/meta/core/field/field_constants.py +++ b/server/python/src/metaobjects/meta/core/field/field_constants.py @@ -150,3 +150,10 @@ # Regex pattern for enum member symbols — must be identifier-safe. # Cross-language contract: every port enforces this pattern. ENUM_MEMBER_PATTERN = r"^[A-Za-z_][A-Za-z0-9_]*$" + +# int-backed-enum-values plan — @intValueMap (field.enum only): optional +# per-member int values ({member: int}) switching this enum field's DB +# persistence from string+CHECK to integer+CHECK. Keys must exactly match +# @values; values must be unique integers. The generated native type and wire +# format are unchanged in every language. Mirrors TS FIELD_ATTR_INT_VALUE_MAP. +FIELD_ATTR_INT_VALUE_MAP = "intValueMap" diff --git a/server/python/src/metaobjects/runtime/object_manager.py b/server/python/src/metaobjects/runtime/object_manager.py index 43eb930df..b82620b05 100644 --- a/server/python/src/metaobjects/runtime/object_manager.py +++ b/server/python/src/metaobjects/runtime/object_manager.py @@ -311,6 +311,8 @@ def _insert_row( } row = result.rows[0] mapped = {col_to_field.get(k, k): v for k, v in row.items()} + # int-backed enums come back as integers; hand the caller the symbol. + mapped = _decode_read_row({f.name: f for f in returning_fields}, mapped) # #214: a write-through entity's INSERT RETURNING covers only the table (non-derived) # columns; re-read the persisted row through the replica VIEW by PK so the returned row # carries the derived origin.* fields (read-your-writes). find_by_id routes to the view. @@ -423,6 +425,8 @@ def update( } row = result.rows[0] mapped = {col_to_field.get(k, k): v for k, v in row.items()} + # int-backed enums come back as integers; hand the caller the symbol. + mapped = _decode_read_row({f.name: f for f in returning_fields}, mapped) # #214: re-read the updated row through the replica VIEW by PK so the returned row # carries the derived origin.* fields (write targeted the table, which lacks them). if entity.is_write_through(): @@ -509,8 +513,11 @@ def find_many( self.last_column_oids = { col_to_field.get(c, c): oid for c, oid in result.column_oids.items() } + # int-backed enums come back as integers; hand the caller the symbol. + fields_by_name = {f.name: f for f in entity.fields()} return [ - {col_to_field.get(k, k): v for k, v in row.items()} for row in result.rows + _decode_read_row(fields_by_name, {col_to_field.get(k, k): v for k, v in row.items()}) + for row in result.rows ] def count(self, entity_name: str, filter: Filter | None = None) -> int: @@ -711,12 +718,13 @@ def _compile_filter(f: Filter | None, entity: MetaObject) -> tuple[str, list[Any mf = entity.find_field(field_name) col = _column_of(mf) if mf is not None else field_name if not isinstance(ops, dict): - # Shortcut: {field: value} → equality + # Shortcut: {field: value} → equality. Encoded like any other bound + # value (see _op_clause) so an int-backed enum reaches SQL as its int. parts.append(f"{_q(col)} = %s") - params.append(ops) + params.append(_coerce_write_value(mf, ops) if mf is not None else ops) continue for op, value in ops.items(): - sql, p = _op_clause(col, op, value) + sql, p = _op_clause(col, op, value, mf) parts.append(sql) params.extend(p) if not parts: @@ -724,9 +732,21 @@ def _compile_filter(f: Filter | None, entity: MetaObject) -> tuple[str, list[Any return " AND ".join(parts), params -def _op_clause(col: str, op: str, value: Any) -> tuple[str, list[Any]]: - """Translate one operator → SQL + params. Mirrors TS/C#/Java semantics.""" +def _op_clause(col: str, op: str, value: Any, field: MetaField | None = None) -> tuple[str, list[Any]]: + """Translate one operator → SQL + params. Mirrors TS/C#/Java semantics. + + Bound values go through the WRITE codec, exactly as an INSERT's do: the four + sibling ports all encode on this path (TS through the Drizzle customType, Java + through GenericSQLDriver.setStatementValue → EnumCodec.write, Kotlin through + Exposed's toDb, C# through the EF converter). Without it an int-backed enum's + filter binds the member SYMBOL against an INTEGER column — the filter band + keeps eq/ne/in for int-backed enums precisely BECAUSE the symbol is supposed + to encode to its integer before reaching SQL. + """ qc = _q(col) + if field is not None and op != "isNull": + value = ([_coerce_write_value(field, v) for v in value] + if op == "in" and value else _coerce_write_value(field, value)) if op == "eq": return f"{qc} = %s", [value] if op == "ne": return f"{qc} <> %s", [value] if op == "gt": return f"{qc} > %s", [value] @@ -823,11 +843,79 @@ def _coerce_write_value(field: MetaField, value: Any) -> Any: storage = field.get_meta_attr(fc.FIELD_ATTR_STORAGE) # ADR-0039 resolving if storage != "flattened": # None / "jsonb" / "subdocument" → single jsonb column return _json.dumps(value) + # field.enum carrying @intValueMap: the column is an INTEGER while the native + # and wire contract stays the member SYMBOL (int-backing is a persistence-layer + # concern, invisible above this codec), so encode symbol -> declared int here. + # + # An unmapped symbol is passed through untouched: membership is the column's + # CHECK constraint to enforce, and inventing a value here would hide the drift. + if sub == fc.FIELD_SUBTYPE_ENUM: + int_map = _int_value_map(field) + if int_map is not None and value in int_map: + return int_map[value] + # Everything else (string / int / long / double / float / boolean / enum) # is already the native type pg8000 binds directly. return value +def _int_value_map(field: MetaField) -> dict[Any, Any] | None: + """The declared ``@intValueMap`` (symbol → int), or ``None`` when the enum is + string-backed. + + ADR-0039 resolving: the map is ``@values``' numeric half — a logical property + of the enum vocabulary that inherits through ``extends`` — so it is read with + ``get_meta_attr``, NOT the own-only accessor (contrast ``@dbColumnType`` in + :func:`_coerce_write_value`, the one field attribute that is deliberately + own-only). Shared by the write and read halves of the enum codec, mirroring + the dedicated int-map helper each sibling port's codec keeps (Java + ``EnumCodec.intValueMap``, Kotlin ``readIntValueMap``, C# ``IntValueMapOf``). + """ + m = field.get_meta_attr(fc.FIELD_ATTR_INT_VALUE_MAP) + return m if isinstance(m, dict) else None + + +def _decode_read_value(field: MetaField, value: Any) -> Any: + """Decode a stored value back to its authoring form on read. + + The inverse of :func:`_coerce_write_value`'s int-backed-enum branch, and + today its only case: the column holds the member's integer, callers expect + the member SYMBOL. Every other field subtype is returned verbatim, keeping + ADR-0019's "runtime returns native in-process types" contract intact. + + An int with no member RAISES. The database then holds a value the model says + is impossible — a hand-written INSERT, or a member removed without a + migration — and neither alternative is honest: returning the raw int hands + the caller a "member" that is not one (C#, Kotlin and TypeScript type this + property as a CLOSED enum, where ``7`` is not even representable), and + returning ``None`` hides the corruption behind a nullable column. Every port + throws here. + """ + if value is None: + return None + if field.sub_type != fc.FIELD_SUBTYPE_ENUM: + return value + int_map = _int_value_map(field) + if int_map is None or not isinstance(value, int) or isinstance(value, bool): + return value + for symbol, stored in int_map.items(): + if stored == value: + return symbol + raise ValueError( + f"field.enum '{field.name}' read stored value {value} with no member in " + f"@{fc.FIELD_ATTR_INT_VALUE_MAP} (declared: {int_map}) — the database holds " + f"a value the model does not describe." + ) + + +def _decode_read_row(fields_by_name: dict[str, MetaField], row: dict[str, Any]) -> dict[str, Any]: + """Apply :func:`_decode_read_value` to every field-keyed value in a mapped row.""" + return { + k: (_decode_read_value(fields_by_name[k], v) if k in fields_by_name else v) + for k, v in row.items() + } + + # ---------------------------------------------------------------------------- # #203 — @autoSet CRUD stamping. A ``field.timestamp`` (or any temporal field) # marked ``@autoSet: onCreate|onUpdate`` declares "the runtime owns this diff --git a/server/python/src/metaobjects/spec_metamodel/attr.json b/server/python/src/metaobjects/spec_metamodel/attr.json index 45f4cea6b..a57e1d6a9 100644 --- a/server/python/src/metaobjects/spec_metamodel/attr.json +++ b/server/python/src/metaobjects/spec_metamodel/attr.json @@ -19,6 +19,12 @@ "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/server/python/src/metaobjects/spec_metamodel/field.json b/server/python/src/metaobjects/spec_metamodel/field.json index 01be74308..a80d56cf5 100644 --- a/server/python/src/metaobjects/spec_metamodel/field.json +++ b/server/python/src/metaobjects/spec_metamodel/field.json @@ -150,7 +150,8 @@ "rules": "Required @values is a non-empty, duplicate-free set; each member must match ^[A-Za-z_][A-Za-z0-9_]*$ so symbol == stored string in every target language. Optional FR-010/FR-011 overlays add tolerant-extract aliasing (@enumAlias), per-member docs (@enumDoc), an uncoercible-value fallback (@coerceDefault, must be one of @values), and ASCII normalization mode (@normalize).", "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, - { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." } + { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, { diff --git a/server/python/tests/conformance/capabilities.py b/server/python/tests/conformance/capabilities.py index b03d96d51..8bf36e556 100644 --- a/server/python/tests/conformance/capabilities.py +++ b/server/python/tests/conformance/capabilities.py @@ -52,15 +52,19 @@ def invoke(node: MetaData, capability: str, args: dict[str, Any]) -> dict[str, A return {"subtype": identity.sub_type} if capability == "field.filter-ops": - # The canonical per-subtype filter-operator band (in canonical operator + # The canonical per-FIELD filter-operator band (in canonical operator # order). Single source of truth — the same ordered helper the codegen # filter-allowlist generator consumes. Returns {"names": [...]}. + # + # ops_for_field_ordered, not ops_for_subtype_ordered: the band is + # field-level because an int-backed field.enum (@intValueMap) stores as + # an integer and so drops `like`. from metaobjects.meta.core.field.meta_field import MetaField from metaobjects.codegen.generators.filter_allowlist_generator import ( - ops_for_subtype_ordered, + ops_for_field_ordered, ) if not isinstance(node, MetaField): raise TypeError(f"field.filter-ops requires a MetaField, got {type(node)}") - return {"names": list(ops_for_subtype_ordered(node.sub_type))} + return {"names": list(ops_for_field_ordered(node))} raise ValueError(f"Unknown capability: {capability!r}") diff --git a/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py new file mode 100644 index 000000000..df44e4028 --- /dev/null +++ b/server/python/tests/runtime/test_object_manager_enum_intvaluemap.py @@ -0,0 +1,192 @@ +"""field.enum @intValueMap — Python ObjectManager persistence codec. + +The metamodel layer ships the attribute; this pins the RUNTIME halves: +write encodes the member symbol to its declared int, read decodes it back. + +The native/wire contract is unchanged in both directions — a caller always +passes and receives the member SYMBOL, exactly as for a string-backed enum +(design goal 2: int-backing is a persistence-layer-only concern). Only the +value handed to / received from the driver differs. +""" +from __future__ import annotations + +import pytest + +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource +from metaobjects.runtime.object_manager import _coerce_write_value, _decode_read_value + +INT_MAP = ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}' + + +def _field_of(json_str: str, entity_name: str, field_name: str): + """Load a model and pull one field. MetaRoot has no find_object and + MetaObject no field(name) — children()/fields() are the real accessors.""" + result = MetaDataLoader().load([InMemoryStringSource(json_str, "test.json")]) + assert result.errors == [] + entity = next(c for c in result.root.children() if c.name == entity_name) + return next(f for f in entity.fields() if f.name == field_name) + + +def _status_field(extra: str): + json_str = f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + return _field_of(json_str, "Order", "status") + + +# --- write side ----------------------------------------------------------- + + +@pytest.mark.parametrize("symbol,stored", [("DRAFT", 0), ("PUBLISHED", 5), ("ARCHIVED", 9)]) +def test_int_backed_enum_write_encodes_symbol_to_declared_int(symbol, stored): + assert _coerce_write_value(_status_field(INT_MAP), symbol) == stored + + +def test_int_backed_enum_write_encodes_zero_not_falsy_dropped(): + """DRAFT maps to 0 — a falsy int. Guards the `if not mapped` class of bug.""" + assert _coerce_write_value(_status_field(INT_MAP), "DRAFT") == 0 + + +def test_string_backed_enum_write_is_unchanged(): + assert _coerce_write_value(_status_field(""), "PUBLISHED") == "PUBLISHED" + + +def test_none_stays_none_on_write(): + assert _coerce_write_value(_status_field(INT_MAP), None) is None + + +def test_unknown_symbol_on_write_is_left_alone_for_the_db_to_reject(): + """Not the codec's job to validate membership — the CHECK constraint is.""" + assert _coerce_write_value(_status_field(INT_MAP), "NOPE") == "NOPE" + + +# --- read side ------------------------------------------------------------ + + +@pytest.mark.parametrize("stored,symbol", [(0, "DRAFT"), (5, "PUBLISHED"), (9, "ARCHIVED")]) +def test_int_backed_enum_read_decodes_int_to_symbol(stored, symbol): + assert _decode_read_value(_status_field(INT_MAP), stored) == symbol + + +def test_int_backed_enum_read_decodes_zero(): + assert _decode_read_value(_status_field(INT_MAP), 0) == "DRAFT" + + +def test_string_backed_enum_read_is_unchanged(): + assert _decode_read_value(_status_field(""), "PUBLISHED") == "PUBLISHED" + + +def test_none_stays_none_on_read(): + assert _decode_read_value(_status_field(INT_MAP), None) is None + + +def test_unmapped_int_on_read_raises(): + """A row holding an int outside the map is data the model says is impossible. + + Neither alternative is honest: returning the raw int hands the caller a + "member" that is not one (C#, Kotlin and TypeScript type this as a CLOSED + enum, where 7 is unrepresentable), and returning None hides the corruption + behind a nullable column. Every port throws. + """ + with pytest.raises(ValueError) as exc: + _decode_read_value(_status_field(INT_MAP), 7) + assert "7" in str(exc.value) + assert "intValueMap" in str(exc.value) + + +def test_non_enum_field_read_is_untouched(): + json_str = """{ "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]} } + ]} }""" + assert _decode_read_value(_field_of(json_str, "Order", "id"), 5) == 5 + + +# --- round trip ----------------------------------------------------------- + + +@pytest.mark.parametrize("symbol", ["DRAFT", "PUBLISHED", "ARCHIVED"]) +def test_write_then_read_round_trips_to_the_same_symbol(symbol): + field = _status_field(INT_MAP) + assert _decode_read_value(field, _coerce_write_value(field, symbol)) == symbol + + +# --- inheritance (ADR-0039: @intValueMap is read RESOLVING) --------------- + + +def test_intvaluemap_inherited_through_extends_is_honoured(): + """A concrete field extending a shared abstract enum must encode with the + inherited map — an own-only read here would silently store the symbol.""" + json_str = """{ "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "StatusEnum", "abstract": true, + "@values": ["DRAFT","PUBLISHED","ARCHIVED"], + "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9} } }, + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "StatusEnum" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]} } + ]} }""" + field = _field_of(json_str, "Order", "status") + assert _coerce_write_value(field, "PUBLISHED") == 5 + assert _decode_read_value(field, 5) == "PUBLISHED" + + +# --- filter encoding ------------------------------------------------------ +# +# The filter band keeps eq/ne/in for an int-backed enum precisely BECAUSE the +# member symbol is supposed to encode to its integer before reaching SQL. Python +# was the only port not encoding on this path, so every query bound the SYMBOL +# against an INTEGER column (pg 22P02). These pin the encoding at the SQL boundary. + + +def _order_entity(extra: str): + json_str = f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "source.rdb": {{ "name": "src", "@table": "orders", "@kind": "table" }} }}, + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + result = MetaDataLoader().load([InMemoryStringSource(json_str, "test.json")]) + assert result.errors == [] + return next(c for c in result.root.children() if c.name == "Order") + + +def test_filter_eq_on_int_backed_enum_binds_the_int(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": {"eq": "PUBLISHED"}}, _order_entity(INT_MAP)) + assert params == [5] + + +def test_filter_shortcut_equality_on_int_backed_enum_binds_the_int(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": "DRAFT"}, _order_entity(INT_MAP)) + assert params == [0] + + +def test_filter_in_on_int_backed_enum_binds_each_int(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": {"in": ["DRAFT", "ARCHIVED"]}}, _order_entity(INT_MAP)) + assert params == [0, 9] + + +def test_filter_on_string_backed_enum_is_unchanged(): + from metaobjects.runtime.object_manager import _compile_filter + _, params = _compile_filter({"status": {"eq": "PUBLISHED"}}, _order_entity("")) + assert params == ["PUBLISHED"] + + +def test_filter_isnull_binds_no_param_and_is_not_coerced(): + from metaobjects.runtime.object_manager import _compile_filter + sql, params = _compile_filter({"status": {"isNull": True}}, _order_entity(INT_MAP)) + assert params == [] + assert "IS NULL" in sql diff --git a/server/python/tests/unit/test_field_enum_intvaluemap.py b/server/python/tests/unit/test_field_enum_intvaluemap.py new file mode 100644 index 000000000..61741c2da --- /dev/null +++ b/server/python/tests/unit/test_field_enum_intvaluemap.py @@ -0,0 +1,185 @@ +"""field.enum's @intValueMap (int-backed-enum-values plan) — Python port. + +Mirrors the TS/C#/Java conformance surface: an optional attr.intMap-shaped +@intValueMap ({member: int}) whose keys must exactly match @values and whose +values must be unique integers. Reuses ERR_BAD_ATTR_VALUE — no new error code. +""" +from __future__ import annotations + +from metaobjects.errors import ErrorCode +from metaobjects.loader.meta_data_loader import MetaDataLoader +from metaobjects.loader.sources import InMemoryStringSource + + +def _model(extra: str) -> str: + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def _load(json_str: str): + loader = MetaDataLoader() + return loader.load([InMemoryStringSource(json_str, "test.json")]) + + +def test_valid_intvaluemap_with_matching_keys_and_unique_ints_loads_clean(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')) + assert result.errors == [] + + +def test_no_intvaluemap_still_loads_clean_string_backed_default(): + result = _load(_model("")) + assert result.errors == [] + + +def test_missing_member_key_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "ARCHIVED" in result.errors[0].message + + +def test_extra_key_not_in_values_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "RETRACTED" in result.errors[0].message + + +def test_non_integer_value_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + + +def test_duplicate_int_value_across_members_is_rejected(): + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "DRAFT" in result.errors[0].message and "PUBLISHED" in result.errors[0].message + + +def test_value_outside_32bit_range_is_rejected(): + # Final-review fix: the eventual DB column for an int-backed enum is a + # 32-bit Postgres/SQLite `integer` (design doc D5) — a value outside that + # range can never actually be persisted, so it must be rejected at load + # time. Mirrors Java's IntMapAttribute#setValueAsString bound check + # (Python ints have no fixed width, so nothing else in this port would + # otherwise catch this). + result = _load(_model(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9999999999}')) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + + +# --------------------------------------------------------------------------- +# Regression: @intValueMap validation when @values is INHERITED via extends +# (bug — Rule 4 lived behind an `own_values is None: continue` early-return, +# so a concrete field.enum that inherits @values from an abstract parent but +# owns its own @intValueMap got that map skipped entirely). +# --------------------------------------------------------------------------- + + +def _model_inherited_values(extra: str) -> str: + """An abstract field.enum owning @values, and a concrete field.enum on + Order.status that `extends` it (inheriting @values) while owning its own + @intValueMap directly. + + The abstract enum is nested inside an abstract object.entity, NOT declared at + metadata-root: a root-level abstract enum is a SHARED enum (FR-019), and #246's + int-backed twin forbids a consuming field from owning an @intValueMap against + one. A non-root abstract super stays legal, so this is the shape that still + exercises "own @intValueMap validated against INHERITED @values". + """ + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "object.entity": {{ "name": "Container", "abstract": true, "children": [ + {{ "field.enum": {{ "name": "kind", "abstract": true, "@values": ["DRAFT","PUBLISHED","ARCHIVED"] }} }} + ]}} }}, + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "extends": "acme::Container.kind" {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def test_intvaluemap_on_node_with_inherited_values_missing_key_is_rejected(): + # status.status has no own @values (inherited from abstract Status) but + # owns its own @intValueMap missing the "ARCHIVED" key — must still be + # validated against the EFFECTIVE (inherited) @values, not skipped. + result = _load( + _model_inherited_values(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}') + ) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "ARCHIVED" in result.errors[0].message + + +def test_intvaluemap_on_node_with_inherited_values_extra_key_is_rejected(): + result = _load( + _model_inherited_values( + ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}' + ) + ) + assert len(result.errors) > 0 + assert result.errors[0].code == ErrorCode.ERR_BAD_ATTR_VALUE + assert "RETRACTED" in result.errors[0].message + + +def test_intvaluemap_on_node_with_inherited_values_valid_map_loads_clean(): + # Positive-case sibling: a valid @intValueMap (keys exactly matching the + # inherited @values, unique ints) must NOT be rejected — proving the fix + # doesn't just make every inherited-@values node fail. + result = _load( + _model_inherited_values( + ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}' + ) + ) + assert result.errors == [] + + +def _model_shared_enum(extra: str) -> str: + """A ROOT-level abstract field.enum (a SHARED enum per FR-019) owning @values, + and a concrete Order.status that `extends` it.""" + return f"""{{ "metadata.root": {{ "package": "acme", "children": [ + {{ "field.enum": {{ "name": "Status", "abstract": true, "@values": ["DRAFT","PUBLISHED","ARCHIVED"] }} }}, + {{ "object.entity": {{ "name": "Order", "children": [ + {{ "field.long": {{ "name": "id" }} }}, + {{ "field.enum": {{ "name": "status", "extends": "acme::Status" {extra} }} }}, + {{ "identity.primary": {{ "name": "pk", "@fields": ["id"] }} }} + ]}} }} + ]}} }}""" + + +def test_own_intvaluemap_against_shared_enum_is_rejected(): + # #246 int-backed twin: a shared enum is materialized ONCE as a single type, so + # its integer backing belongs on the shared declaration. A consuming field that + # owns an @intValueMap would give one logical type N storage encodings. + result = _load( + _model_shared_enum( + ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}' + ) + ) + codes = [e.code for e in result.errors] + assert ErrorCode.ERR_ENUM_EXTENDS_VALUES_CONFLICT in codes + + +def test_intvaluemap_on_the_shared_declaration_itself_loads_clean(): + # The sanctioned shape: the shared declaration owns BOTH @values and the + # integer backing; the consuming field inherits both and declares neither. + result = _load( + """{ "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT","PUBLISHED","ARCHIVED"], + "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9} } }, + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "acme::Status" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]} } + ]} }""" + ) + assert result.errors == [] diff --git a/server/python/uv.lock b/server/python/uv.lock index 8b7d7b7c1..3e9a8aee2 100644 --- a/server/python/uv.lock +++ b/server/python/uv.lock @@ -250,7 +250,7 @@ wheels = [ [[package]] name = "metaobjects" -version = "0.23.0" +version = "0.23.1" source = { editable = "." } dependencies = [ { name = "pyyaml" }, diff --git a/server/typescript/packages/codegen-ts/src/column-mapper.ts b/server/typescript/packages/codegen-ts/src/column-mapper.ts index 87f2854f4..4967713ca 100644 --- a/server/typescript/packages/codegen-ts/src/column-mapper.ts +++ b/server/typescript/packages/codegen-ts/src/column-mapper.ts @@ -46,7 +46,7 @@ import { AGG_COLLECT, } from "@metaobjectsdev/metadata"; import { columnNameFromField } from "./naming.js"; -import { enumValues } from "./enum-meta.js"; +import { enumValues, intValueMapOf, intValueForMember } from "./enum-meta.js"; import { DEFAULT_COLUMN_NAMING_STRATEGY, stripPackage } from "@metaobjectsdev/metadata"; import type { Dialect, ColumnNamingStrategy } from "./metaobjects-config.js"; @@ -212,9 +212,42 @@ function canonicalizeSqlExpr(value: string): string { return value; // unrecognized — pass through (function calls etc.) } +/** + * An int-backed `field.enum` column: a generated Drizzle `customType` whose + * `toDriver`/`fromDriver` translate member symbol <-> stored integer, so the + * codec lives in the COLUMN definition rather than in the query layer. + * + * This is the TS analogue of what every other port already does at its own + * `MetaField` codec seam (EF Core `HasConversion`, OMDB `JdbcFieldCodec`, Exposed + * `customEnumeration`, Python `ObjectManager` coercion) — which is why it was + * chosen over a Zod write-transform plus a bespoke read-decode: TS's generated + * queries hand back raw Drizzle rows and have no decode seam at all, so a + * query-layer codec would have meant inventing one and wrapping every generated + * read. Binding through the column type also makes filter values encode for free. + */ +export interface EnumIntCustomType { + /** Local const name for the customType column helper, e.g. `orderStatusEnumCol`. */ + fnConstName: string; + /** Local const name for the symbol->int map, e.g. `ORDER_STATUS_TO_INT`. */ + toIntConstName: string; + /** Local const name for the int->symbol map, e.g. `ORDER_STATUS_FROM_INT`. */ + fromIntConstName: string; + /** Physical column type for `dataType()` — always integer for an int-backed enum. */ + dataType: string; + /** Member symbols, in `@values` order (the TS union and the map key order). */ + members: string[]; + /** Member symbol -> stored integer. */ + intByMember: Record; +} + export interface ColumnSpec { /** Drizzle function name, e.g., "text", "integer", "varchar". */ fnName: string; + /** + * When set, `fnName` names a LOCAL generated const (this spec's customType + * helper) rather than a Drizzle export — the renderer must NOT `imp()` it. + */ + enumIntCustomType?: EnumIntCustomType; /** DB column name (snake_case from field name, or @column override). */ dbName: string; /** Positional args after dbName (currently always empty; reserved). */ @@ -342,6 +375,48 @@ function objectRefBaseName(field: MetaField): string | undefined { return undefined; } +/** SCREAMING_SNAKE_CASE for a generated map const name. */ +function screamingSnake(s: string): string { + return s + .replace(/([a-z0-9])([A-Z])/g, "$1_$2") + .replace(/[^A-Za-z0-9]+/g, "_") + .toUpperCase(); +} + +/** + * Build the customType descriptor for an int-backed `field.enum`, or undefined + * when `@values` is missing (the field then degrades to a plain integer column + * rather than emitting a codec over an unknown member set). + * + * Names are derived from the FIELD name, so the consts are per-entity-file and + * self-contained. A shared enum consumed by N entities therefore emits N small + * identical helpers rather than requiring a cross-module import — the same + * self-contained tradeoff the per-entity enum union already makes. + */ +function buildEnumIntCustomType( + field: MetaField, + intByMember: Record, +): EnumIntCustomType | undefined { + const members = enumValues(field); + if (members === undefined || members.length === 0) return undefined; + // Every member must map — the loader pins key-set-equals-@values (Check 5b), so a + // miss is unreachable; throwing beats emitting a codec with a hole in it. + for (const m of members) { + intValueForMember(intByMember, m, `customType codec for field '${field.name}'`); + } + const base = field.name.replace(/[^A-Za-z0-9]/g, ""); + const camel = base.charAt(0).toLowerCase() + base.slice(1); + const screaming = screamingSnake(base); + return { + fnConstName: `${camel}IntEnum`, + toIntConstName: `${screaming}_TO_INT`, + fromIntConstName: `${screaming}_FROM_INT`, + dataType: "integer", + members, + intByMember, + }; +} + export function mapColumnType( field: MetaField, dialect: Dialect, @@ -355,6 +430,8 @@ export function mapColumnType( let fnName: string; let fnOptions: Record | undefined; + // Set only for an int-backed field.enum — see EnumIntCustomType. + let enumIntCustomType: EnumIntCustomType | undefined; let leadingComment: string | undefined; if (dialect === "sqlite") { @@ -405,8 +482,20 @@ export function mapColumnType( // "string" by the time it reaches here for this dialect. fnName = "text"; break; - case FIELD_SUBTYPE_STRING: case FIELD_SUBTYPE_ENUM: + // An INT-BACKED enum stores the mapped integer on SQLite too — SQLite has + // one integer storage class, so this matches migrate-ts's integer{32}. + { + const im = intValueMapOf(field); + if (im !== undefined) { + enumIntCustomType = buildEnumIntCustomType(field, im); + fnName = enumIntCustomType?.fnConstName ?? "integer"; + } else { + fnName = "text"; + } + } + break; + case FIELD_SUBTYPE_STRING: case FIELD_SUBTYPE_UUID: case FIELD_SUBTYPE_URI: case FIELD_SUBTYPE_INET: @@ -524,6 +613,22 @@ export function mapColumnType( fnName = "jsonb"; break; case FIELD_SUBTYPE_ENUM: + // An INT-BACKED enum (@intValueMap, design D5) stores the mapped integer, + // so the Drizzle column is integer — matching migrate-ts's expected-schema. + // The TS-facing type stays the member-string union; the symbol<->int + // translation happens at the write/read boundary. Scalar only: D7 makes + // @intValueMap + isArray ERR_ENUM_INT_VALUE_MAP_ARRAY at load, so an array + // enum reaching here is always string-backed. + { + const im = intValueMapOf(field); + if (im !== undefined) { + enumIntCustomType = buildEnumIntCustomType(field, im); + fnName = enumIntCustomType?.fnConstName ?? "integer"; + } else { + fnName = "text"; + } + } + break; default: fnName = "text"; break; @@ -669,6 +774,7 @@ export function mapColumnType( }; if (fnOptions !== undefined) result.fnOptions = fnOptions; if (defaultExpr !== undefined) result.defaultExpr = defaultExpr; + if (enumIntCustomType !== undefined) result.enumIntCustomType = enumIntCustomType; if (dollarTypeRef !== undefined) result.dollarTypeRef = dollarTypeRef; if (leadingComment !== undefined) result.leadingComment = leadingComment; @@ -676,12 +782,22 @@ export function mapColumnType( if (subType === FIELD_SUBTYPE_ENUM && !isArray) { const values = enumValues(field); if (values !== undefined && values.length > 0) { - // Single-quote escaping is belt-and-suspenders: the loader's - // ENUM_MEMBER_PATTERN already rejects quote-bearing members (members are - // validated to be identifier-safe), so this never fires in practice. - const list = values - .map((v) => `'${v.replace(/'/g, "''")}'`) - .join(", "); + const intMap = intValueMapOf(field); + let list: string; + if (intMap !== undefined) { + // Int-backed: the column holds integers, so the CHECK lists them unquoted. + // Keyed BY MEMBER through the map (not Object.values) so the constraint can + // never disagree with @values, which stays the SSOT. Must match + // migrate-ts's buildChecks exactly or `meta verify` reports permanent drift. + list = values + .map((v) => String(intValueForMember(intMap, v, `CHECK for column '${dbName}'`))) + .join(", "); + } else { + // Single-quote escaping is belt-and-suspenders: the loader's + // ENUM_MEMBER_PATTERN already rejects quote-bearing members (members are + // validated to be identifier-safe), so this never fires in practice. + list = values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", "); + } result.checkConstraint = `${dbName} IN (${list})`; } } diff --git a/server/typescript/packages/codegen-ts/src/enum-meta.ts b/server/typescript/packages/codegen-ts/src/enum-meta.ts index 256c72129..3233a3012 100644 --- a/server/typescript/packages/codegen-ts/src/enum-meta.ts +++ b/server/typescript/packages/codegen-ts/src/enum-meta.ts @@ -3,7 +3,7 @@ // extraction and the z.enum([...]) expression are derived in exactly one place. import type { MetaField } from "@metaobjectsdev/metadata"; -import { FIELD_ATTR_VALUES } from "@metaobjectsdev/metadata"; +import { FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP } from "@metaobjectsdev/metadata"; /** * Effective enum member values (`@values`) for a field, as strings. @@ -20,6 +20,42 @@ export function enumValues(field: MetaField): string[] | undefined { return values.map((v) => String(v)); } +/** + * The effective `@intValueMap` (member symbol → integer) for an int-backed enum, + * or undefined when the enum is string-backed. Its PRESENCE is the whole trigger + * for integer persistence (design D5) — there is no separate flag or config. + * + * ADR-0039: RESOLVING (`attr`, not `ownAttr`), and this is load-bearing rather + * than incidental. Post-#246 an own `@intValueMap` declared against a shared + * (root-level abstract) enum is `ERR_ENUM_EXTENDS_VALUES_CONFLICT`, so the map + * lives on the SHARED DECLARATION and every consuming field INHERITS it. An + * own-only read would therefore see undefined on exactly the shape adopters are + * steered toward, and silently emit a string codec into an integer column. + */ +export function intValueMapOf(field: MetaField): Record | undefined { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return raw as Record; +} + +/** + * The integer a member symbol persists as, for an int-backed enum. Throws when the + * member has no mapping — the loader pins key-set-equals-`@values` (Check 5b) in + * every port, so a miss is unreachable and must not be papered over: emitting the + * symbol instead would fail only at INSERT time, against a live database. + */ +export function intValueForMember( + intMap: Record, + member: string, + context: string, +): number { + const n = intMap[member]; + if (typeof n !== "number") { + throw new Error(`@intValueMap has no integer for member '${member}' (${context}).`); + } + return n; +} + /** Build the Zod expression for a set of enum members, e.g. `z.enum(["A", "B"])`. */ export function zodEnumExpr(values: string[]): string { return `z.enum([${values.map((v) => JSON.stringify(v)).join(", ")}])`; diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts index 2ec52bcb0..3654d3a52 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts @@ -32,6 +32,8 @@ import { FILTER_OP_LT, FILTER_OP_LTE, FILTER_OP_IS_NULL, + FILTER_OP_LIKE, + FIELD_SUBTYPE_ENUM, FILTER_COMPOSE_AND, FILTER_COMPOSE_OR, SORT_ORDER_DESC, @@ -47,6 +49,7 @@ import { type AggregateFunction, } from "@metaobjectsdev/metadata"; import { type MetaData, type MetaField, type MetaRoot, MetaObject } from "@metaobjectsdev/metadata"; +import { intValueMapOf } from "../enum-meta.js"; import { columnNameFromField, viewNameFromProjection, @@ -126,13 +129,84 @@ function resolveAggregateFilter( kind: "cmp", ref: `${alias}.${sourceColumnNameFor(field, ctx)}`, op, - value: opObj[op], + // Same int-backed-enum encoding as the row-scope @filter below: this scoping + // filter renders as a SQL literal too (FILTER (WHERE …) / CASE WHEN), so a + // member symbol would land unencoded in an integer comparison. + value: encodeIntEnumFilterValue( + opObj[op], + op, + field.subType === FIELD_SUBTYPE_ENUM ? intValueMapOf(field) : undefined, + key, + entity.name, + ), }); } if (clauses.length === 0) return undefined; return clauses.length === 1 ? clauses[0]! : { kind: "and", clauses }; } +/** + * The `@intValueMap` of every int-backed `field.enum` the projection declares, keyed + * by field name. Only int-backed enums appear, so a lookup miss means "no encoding". + * + * `fields()` (effective) and `intValueMapOf` (which reads `attr`, RESOLVING) — a + * projection's fields are bound through `extends` to the base entity's, and post-#246 + * the map itself commonly lives one hop further up on a shared abstract declaration. + * Own-only at either hop would silently emit the member symbol into an integer column + * (ADR-0039). + */ +function intEnumMapsOf(projection: MetaObject): ReadonlyMap> { + const out = new Map>(); + for (const f of projection.fields()) { + if (f.subType !== FIELD_SUBTYPE_ENUM) continue; + const map = intValueMapOf(f); + if (map !== undefined) out.set(f.name, map); + } + return out; +} + +/** + * Lower a filter value for an int-backed `field.enum` from its member SYMBOL to the + * INTEGER it persists as. A no-op for every other field (`intMap` undefined), so a + * string-backed enum's SQL is byte-identical. + * + * `isNull` is skipped — its value is a boolean, not a member. `like` is unreachable: + * `opsForField` removes it from an int-backed enum's band, so the loader rejects it + * before codegen; the explicit throw makes that a loud failure rather than a + * `LIKE NaN`. An unmapped member is likewise loader-unreachable (the key set is + * pinned equal to `@values`) and throws for the same reason — silently emitting the + * symbol would produce DDL that fails only at apply time, against a live database. + */ +function encodeIntEnumFilterValue( + value: unknown, + op: string, + intMap: Record | undefined, + fieldName: string, + projectionName: string, +): unknown { + if (intMap === undefined) return value; + if (op === FILTER_OP_IS_NULL) return value; + if (op === FILTER_OP_LIKE) { + throw new Error( + `Projection ${projectionName}: view @filter uses "like" on "${fieldName}", an ` + + `int-backed field.enum (@intValueMap) — it stores as an integer column, so a ` + + `substring match is not expressible. Use eq/ne/in.`, + ); + } + const encode = (v: unknown): unknown => { + if (typeof v !== "string") return v; + const n = intMap[v]; + if (typeof n !== "number") { + throw new Error( + `Projection ${projectionName}: view @filter value "${v}" for "${fieldName}" has no ` + + `entry in @intValueMap.`, + ); + } + return n; + }; + return Array.isArray(value) ? value.map(encode) : encode(value); +} + /** * #207 — resolve a projection's row-scope `@filter` (the desugared canonical * `{ field: { op: value }, and?, or? }`) into a {@link ViewFilterClause} whose @@ -152,13 +226,14 @@ function resolveViewFilter( filter: unknown, columnsByField: ReadonlyMap, projectionName: string, + intMapsByField: ReadonlyMap>, ): ViewFilterClause | undefined { if (typeof filter !== "object" || filter === null || Array.isArray(filter)) return undefined; const clauses: ViewFilterClause[] = []; for (const [key, val] of Object.entries(filter as Record)) { if (key === FILTER_AND || key === FILTER_OR) { const subs = (Array.isArray(val) ? val : []) - .map((s) => resolveViewFilter(s, columnsByField, projectionName)) + .map((s) => resolveViewFilter(s, columnsByField, projectionName, intMapsByField)) .filter((c): c is ViewFilterClause => c !== undefined); if (subs.length > 0) clauses.push({ kind: key === FILTER_AND ? "and" : "or", clauses: subs }); continue; @@ -177,7 +252,14 @@ function resolveViewFilter( // becomes its own comparison, AND-composed (dropping all-but-the-first would silently // widen the exposed row set). The loader has already validated every op for this // field's subtype. - for (const [op, value] of Object.entries(desugarClause(val))) { + for (const [op, rawValue] of Object.entries(desugarClause(val))) { + // An INT-BACKED field.enum (@intValueMap, design D5) stores as an INTEGER + // column, so the authored member SYMBOL must become its integer before it is + // rendered as a SQL literal. The Drizzle customType handles the runtime query + // path, but view DDL is emitted as literal SQL text and never touches Drizzle. + const value = encodeIntEnumFilterValue( + rawValue, op, intMapsByField.get(key), key, projectionName, + ); if (col.kind === "passthrough") { clauses.push({ kind: "cmp", ref: `${col.sourceAlias}.${col.sourceColumn}`, op, value }); } else if (col.kind === "computed") { @@ -990,7 +1072,9 @@ export function extractViewSpec( const columnsByField = new Map( selectSpec.columns.map((c) => [c.fieldName, c] as const), ); - where = resolveViewFilter(rawFilter, columnsByField, projection.name); + where = resolveViewFilter( + rawFilter, columnsByField, projection.name, intEnumMapsOf(projection), + ); } } diff --git a/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts b/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts index 90d11c043..b3b1d546c 100644 --- a/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts +++ b/server/typescript/packages/codegen-ts/src/templates/drizzle-schema.ts @@ -13,7 +13,7 @@ import { } from "@metaobjectsdev/metadata"; import { fieldDeclaringPackage, type RenderContext } from "../render-context.js"; import { crossEntitySpecifier, valueObjectModuleSpecifier } from "../import-path.js"; -import { mapColumnType, type ColumnSpec } from "../column-mapper.js"; +import { mapColumnType, type ColumnSpec, type EnumIntCustomType } from "../column-mapper.js"; import { tableNameFromEntity, columnNameFromField } from "../naming.js"; import { renderRelationsBlock } from "./relations-block.js"; import { renderDocsFor } from "./jsdoc.js"; @@ -66,6 +66,9 @@ export function renderDrizzleSchema(obj: MetaObject, ctx: RenderContext): Code { const columnLines: Code[] = []; // Collect CHECK constraints for enum columns; emitted as table-level check() callbacks. const checkConstraints: Array<{ name: string; expr: string }> = []; + // Int-backed field.enum customType helpers, emitted ahead of the table. Keyed by + // const name so a shared enum used by two fields of the SAME entity emits once. + const enumIntTypes = new Map(); for (const child of obj.fields()) { // #213 — a derived (origin-bearing) field is read-only, materialized on the // read (view) side, NOT a column on the entity's write table (FR-024 §7). @@ -78,6 +81,9 @@ export function renderDrizzleSchema(obj: MetaObject, ctx: RenderContext): Code { // Compute the column spec once per field and reuse it for both the column // line and the CHECK collection. const spec = mapColumnType(child, ctx.dialect, ctx.columnNamingStrategy, ctx.timestampMode); + if (spec.enumIntCustomType !== undefined) { + enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); + } const fieldDocs = renderDocsFor(child); const columnLine = renderColumn(spec, child, ctx, isPk, pkGeneration, fkInfo, isComposite, isUnique, obj.package, obj.name); columnLines.push(fieldDocs ? code` ${fieldDocs}\n${columnLine}` : columnLine); @@ -103,6 +109,9 @@ export function renderDrizzleSchema(obj: MetaObject, ctx: RenderContext): Code { // #213 — a TPH subtype's derived field is read-only too; never a table column. if (child.isDerived()) continue; const spec = mapColumnType(child, ctx.dialect, ctx.columnNamingStrategy, ctx.timestampMode); + if (spec.enumIntCustomType !== undefined) { + enumIntTypes.set(spec.enumIntCustomType.fnConstName, spec.enumIntCustomType); + } const fieldDocs = renderDocsFor(child); const columnLine = renderColumn( spec, child, ctx, false, undefined, fkMap.get(child.name), isComposite, false, obj.package, obj.name, true, @@ -177,11 +186,60 @@ ${joinCode(columnLines, { on: ",\n", trim: false })} // Emit the relations() block (returns null if no relations). const relationsBlock = renderRelationsBlock(obj, ctx); - if (relationsBlock === null) { - return tableBlock; - } + // Int-backed enum codecs are declared BEFORE the table that references them. + // Sorted by const name so output is deterministic regardless of field order. + const enumIntBlocks = [...enumIntTypes.values()] + .sort((a, b) => a.fnConstName.localeCompare(b.fnConstName)) + .map((t) => renderEnumIntCustomType(t, importModule)); - return joinCode([tableBlock, relationsBlock], { on: "\n" }); + const blocks: Code[] = [...enumIntBlocks, tableBlock]; + if (relationsBlock !== null) blocks.push(relationsBlock); + return blocks.length === 1 ? blocks[0]! : joinCode(blocks, { on: "\n" }); +} + +/** + * Render an int-backed `field.enum`'s Drizzle `customType` helper plus its two + * lookup maps. + * + * The codec lives HERE, in the column definition, so nothing downstream needs to + * know about it: `db.insert().values()` encodes on bind, a selected row decodes on + * read, and a filter comparison encodes because Drizzle binds through the column + * type. That is why this shape was chosen over a Zod write-transform plus a + * generated read-decode — TS's generated queries return raw Drizzle rows and have + * no decode seam, so the query-layer approach meant inventing one and wrapping + * every generated read. It is also the direct analogue of what the other four + * ports already do (EF Core `HasConversion`, OMDB `JdbcFieldCodec`, Exposed + * `customEnumeration`, Python `ObjectManager` coercion). + * + * `fromDriver` throws on an unmapped integer rather than returning undefined: a + * value outside the map means the DB holds data the model says is impossible + * (a hand-written INSERT, or a member removed without a migration), and silently + * yielding `undefined` for a non-nullable field would surface far from the cause. + */ +function renderEnumIntCustomType(t: EnumIntCustomType, importModule: string): Code { + const customTypeSym = imp(`customType@${importModule}`); + const union = t.members.map((m) => JSON.stringify(m)).join(" | "); + const toEntries = t.members + .map((m) => `${JSON.stringify(m)}: ${t.intByMember[m]}`) + .join(", "); + const fromEntries = t.members + .map((m) => `${t.intByMember[m]}: ${JSON.stringify(m)}`) + .join(", "); + return code` +const ${t.toIntConstName} = { ${toEntries} } as const satisfies Record<${union}, number>; +const ${t.fromIntConstName}: Record = { ${fromEntries} }; +const ${t.fnConstName} = ${customTypeSym}<{ data: ${union}; driverData: number }>({ + dataType: () => ${JSON.stringify(t.dataType)}, + toDriver: (value) => ${t.toIntConstName}[value], + fromDriver: (value) => { + const member = ${t.fromIntConstName}[value]; + if (member === undefined) { + throw new Error(\`unmapped ${t.fnConstName} value: \${value}\`); + } + return member; + }, +}); +`; } interface FkInfo { @@ -266,7 +324,13 @@ function renderColumn( // and suppress any DB default (other-subtype rows must stay NULL here). forceNullable: boolean = false, ): Code { - const fnSym = imp(`${spec.fnName}@${spec.importModule}`); + // An int-backed field.enum's column function is a LOCAL generated const (the + // customType helper emitted into this same file), so it must not be imported + // from drizzle-orm/*-core like a built-in column type would be. + const fnSym = + spec.enumIntCustomType !== undefined + ? spec.enumIntCustomType.fnConstName + : imp(`${spec.fnName}@${spec.importModule}`); const dbNameLit = JSON.stringify(spec.dbName); let baseCall: Code; diff --git a/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts b/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts index 03883f01f..640e42293 100644 --- a/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts +++ b/server/typescript/packages/codegen-ts/src/templates/filter-allowlist.ts @@ -13,7 +13,7 @@ import { FIELD_SUBTYPE_TIME, FIELD_SUBTYPE_TIMESTAMP, FIELD_SUBTYPE_CURRENCY, - opsForSubType, + opsForField, } from "@metaobjectsdev/metadata"; import { sortableFields } from "./filter-shared.js"; import type { RenderContext } from "../render-context.js"; @@ -72,7 +72,9 @@ export const ${entity.name}FilterAllowlist = {} as const satisfies FilterAllowli } const rows = fields .map((f) => { - const ops = opsForSubType(f.subType).map((o) => JSON.stringify(o)).join(", "); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` (a substring match) is not in its band. + const ops = opsForField(f).map((o) => JSON.stringify(o)).join(", "); const sub = filterSubTypeFor(f.subType); // Only field.timestamp is governed by timestampMode — Drizzle types // field.date / field.time as strings under every dialect. diff --git a/server/typescript/packages/codegen-ts/src/templates/filter-type.ts b/server/typescript/packages/codegen-ts/src/templates/filter-type.ts index 148a8d58b..b5744711c 100644 --- a/server/typescript/packages/codegen-ts/src/templates/filter-type.ts +++ b/server/typescript/packages/codegen-ts/src/templates/filter-type.ts @@ -11,7 +11,7 @@ import { FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT, - opsForSubType, + opsForField, } from "@metaobjectsdev/metadata"; import { isSortableField } from "./filter-shared.js"; @@ -34,7 +34,11 @@ function tsNameFor(fieldSubType: string): string { } function renderFieldUnion(field: MetaField): string { - const ops = opsForSubType(field.subType); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) stores + // as an integer, so `like` is not in its band. The client type and the server + // allowlist MUST agree: offering `like` here that the allowlist 400s is a + // client/server mismatch of exactly the kind filter-shared.ts exists to prevent. + const ops = opsForField(field); const tsName = tsNameFor(field.subType); const opEntries = ops.map((op) => { if (op === "in") return `in?: ${tsName}[]`; diff --git a/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts new file mode 100644 index 000000000..33ae78646 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/column-mapper-enum-intvaluemap.test.ts @@ -0,0 +1,123 @@ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import type { MetaField } from "@metaobjectsdev/metadata"; +import { mapColumnType } from "../src/column-mapper.js"; + +// Int-backed field.enum (@intValueMap, design D5/D7): the Drizzle column becomes +// integer / integer[] and its CHECK lists unquoted integers, matching migrate-ts's +// expected-schema exactly. A mismatch between the two is precisely the drift +// `meta verify --codegen` exists to catch, so these assertions are deliberately +// the mirror of expected-schema-enum-intvaluemap.test.ts in migrate-ts. + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +async function statusField( + statusDecl: Record, + extraRoots: unknown[] = [], +): Promise { + const json = JSON.stringify({ + "metadata.root": { + children: [ + ...extraRoots, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": statusDecl }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (errors.length > 0) throw new Error(`fixture failed to load: ${errors.map(String).join("; ")}`); + const order = root.objects().find((o) => o.name === "Order")!; + return order.fields().find((f) => f.name === "status")!; +} + +describe("mapColumnType — int-backed field.enum (@intValueMap)", () => { + test("scalar int-backed enum → a generated customType column, no literal-union option", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "postgres"); + // fnName names a LOCAL generated const, not a Drizzle export — the renderer + // must emit it rather than imp() it. + expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.enumIntCustomType).toEqual({ + fnConstName: "statusIntEnum", + toIntConstName: "STATUS_TO_INT", + fromIntConstName: "STATUS_FROM_INT", + dataType: "integer", + members: VALUES, + intByMember: INT_MAP, + }); + // The `{ enum: [...] }` literal-union option is a TEXT-column affordance; on an + // integer column it would type the column as a string union over a numeric value. + expect(spec.fnOptions?.enum).toBeUndefined(); + }); + + test("string-backed enum carries NO customType (byte-identical to today)", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); + expect(spec.enumIntCustomType).toBeUndefined(); + }); + + test("string-backed enum is unchanged — text + literal union", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); + expect(spec.fnName).toBe("text"); + expect(spec.fnOptions?.enum).toEqual(VALUES); + }); + + test("CHECK lists unquoted integers, matching migrate-ts", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "postgres"); + expect(spec.checkConstraint).toBe("status IN (0, 5, 9)"); + }); + + test("string-backed CHECK keeps quoted members", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES }), "postgres"); + expect(spec.checkConstraint).toBe("status IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')"); + }); + + test("sqlite int-backed enum gets the same customType (integer storage class)", async () => { + const spec = mapColumnType(await statusField({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), "sqlite"); + expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.enumIntCustomType?.dataType).toBe("integer"); + }); + + // Amendment 1 / #246 — the canonical authoring shape. An own-only read of + // @intValueMap would emit `text` here and silently disagree with migrate-ts, + // which reads it resolving. + test("map INHERITED from a shared abstract declaration still yields integer", async () => { + const spec = mapColumnType( + await statusField({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]), + "postgres", + ); + expect(spec.fnName).toBe("statusIntEnum"); + expect(spec.enumIntCustomType?.intByMember).toEqual(INT_MAP); + expect(spec.checkConstraint).toBe("status IN (0, 5, 9)"); + }); + + // Design D7, narrowed: int-backing is scalar-only, so there is no array codegen + // shape to assert — the combination never reaches a generator. This test replaced + // one that asserted `statusIntEnum(...).array()`, which composed on Postgres while + // four ports silently got it wrong (Python bound the symbol list into an integer[], + // Java and Kotlin emitted a scalar codec, and the sqlite branch stored symbols as + // JSON text). The rejection itself is gated by the loader tests + the cross-port + // error-enum-intvaluemap-array fixtures; this pins the CODEGEN-side consequence: + // an array enum reaching a generator is always string-backed. + test("an array-of-enum reaching codegen is string-backed — @intValueMap cannot load with isArray", async () => { + const spec = mapColumnType( + await statusField({ name: "status", isArray: true, "@values": VALUES }), + "postgres", + ); + expect(spec.fnName).toBe("text"); + expect(spec.enumIntCustomType).toBeUndefined(); + expect(spec.modifiers).toContain(".array()"); + // Membership on arrays stays app-level, exactly as for string-backed enum[]. + expect(spec.checkConstraint).toBeUndefined(); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts b/server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts new file mode 100644 index 000000000..4946d344d --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/drizzle-enum-intvaluemap.test.ts @@ -0,0 +1,93 @@ +// Task 5 — an int-backed field.enum's symbol<->int codec lives in the COLUMN +// definition, as a generated Drizzle customType. Nothing downstream changes: +// db.insert().values() encodes on bind, a selected row decodes on read, and a +// filter comparison encodes because Drizzle binds through the column type. +// +// Chosen over a Zod write-transform + generated read-decode because TS's +// generated queries return raw Drizzle rows and have NO decode seam — that route +// meant inventing one and wrapping every generated read. This is also the direct +// analogue of the other four ports' codec seams (EF Core HasConversion, OMDB +// JdbcFieldCodec, Exposed customEnumeration, Python ObjectManager coercion). + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { renderDrizzleSchema } from "../src/templates/drizzle-schema.js"; +import { makeRenderContext } from "../src/render-context.js"; +import { buildPkMap } from "../src/pk-resolver.js"; +import { buildRelationMap } from "../src/relation-resolver.js"; +import type { Dialect } from "../src/metaobjects-config.js"; + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +async function emit(statusDecl: Record, extraRoots: unknown[] = [], dialect: Dialect = "postgres") { + const json = JSON.stringify({ + "metadata.root": { + children: [ + ...extraRoots, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": statusDecl }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const res = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(res.errors).toEqual([]); + const ctx = makeRenderContext({ + dialect, loadedRoot: res.root, outDir: "/x", dbImport: "~/db", + pkMap: buildPkMap(res.root), relationMap: buildRelationMap(res.root), + }); + return renderDrizzleSchema(res.root.findObject("Order")!, ctx).toString(); +} + +describe("Drizzle codegen — int-backed field.enum customType", () => { + test("emits the two lookup maps and a customType whose column is integer", async () => { + const out = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + expect(out).toContain('const STATUS_TO_INT = { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 }'); + expect(out).toContain('const STATUS_FROM_INT: Record'); + expect(out).toContain('dataType: () => "integer"'); + expect(out).toContain("toDriver: (value) => STATUS_TO_INT[value]"); + // The column uses the local const, NOT a drizzle `integer(...)` call. + expect(out).toContain('statusIntEnum("status")'); + }); + + test("the TS-facing data type stays the member-string union, never number", async () => { + const out = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + expect(out).toContain('data: "DRAFT" | "PUBLISHED" | "ARCHIVED"; driverData: number'); + }); + + test("an unmapped stored integer throws rather than yielding undefined", async () => { + const out = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + expect(out).toContain("unmapped statusIntEnum value"); + }); + + test("customType is imported from the dialect's core module", async () => { + const pg = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }, [], "postgres"); + expect(pg).toContain("drizzle-orm/pg-core"); + const sqlite = await emit({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }, [], "sqlite"); + expect(sqlite).toContain("drizzle-orm/sqlite-core"); + }); + + test("the codec is emitted for a map INHERITED from a shared declaration (#246 shape)", async () => { + const out = await emit({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]); + expect(out).toContain("const statusIntEnum = "); + expect(out).toContain("STATUS_TO_INT[value]"); + }); + + test("a string-backed enum emits NO codec at all (byte-identical to today)", async () => { + const out = await emit({ name: "status", "@values": VALUES }); + expect(out).not.toContain("customType"); + expect(out).not.toContain("STATUS_TO_INT"); + expect(out).toContain('text("status"'); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts b/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts index 2e143f554..0c8290d58 100644 --- a/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts +++ b/server/typescript/packages/codegen-ts/test/golden/fr019-shared-provided-conformance.test.ts @@ -89,6 +89,24 @@ describe("FR-019 shared + provided enum conformance (TS)", () => { expect(t).toContain("currency IN ('USD', 'EUR', 'GBP')"); }); + // A CHAINED declaration: root abstract `Money extends` root abstract `@provided Currency`. + // Per ADR-0026 §2 a materialized type is named for ITS OWN declaration, and ADR-0039 makes + // @provided declaration-layer (it does not flow down the chain), so `Money` is its own type, + // materialized from the members it inherits — NOT a reference to a hand-written `Money` the + // adopter never declared. #246 guarantees the chain can rename the vocabulary but never + // mutate it. All five ports must agree here. + test("a root abstract enum extending a @provided enum is materialized under its OWN name", async () => { + const files = await gen(await loadSharedFixture(), "@acme/ext-enums"); + const enums = files["enums.ts"]!; + expect(enums).toContain('export type Money = "USD" | "EUR" | "GBP";'); + // Currency itself stays provided — the chain must not drag it into materialization. + expect(enums).not.toContain("export type Currency ="); + const inv = files["Invoice.ts"]!; + expect(inv).toBeDefined(); + expect(inv).toContain('from "./enums"'); + expect(inv).not.toContain('from "@acme/ext-enums"'); + }); + test("a @provided enum with no providedEnumModule config is a codegen-time error naming the enum", async () => { let err: unknown; try { diff --git a/server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts b/server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts new file mode 100644 index 000000000..a4ba9e58b --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/view-level-filter-int-enum.test.ts @@ -0,0 +1,259 @@ +// A projection row-scope `@filter` (#207) on an INT-BACKED field.enum (@intValueMap, +// design D5) must lower to the INTEGER literal, not the member symbol. +// +// The column is `integer`, so `WHERE p.status = 'PUBLISHED'` is not merely wrong rows — +// on Postgres it is `invalid input syntax for type integer`, raised at CREATE VIEW time, +// which aborts the whole migration. This is the same defect class as Task 7's `@default` +// (`DEFAULT 'DRAFT'` on an integer column): everywhere metadata puts an enum member into +// SQL, the member has to go through @intValueMap first. +// +// The Drizzle customType covers the RUNTIME query path for free, but view DDL is emitted +// as literal SQL text and never touches Drizzle — so it needs its own encoding. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { extractViewSpec } from "../../src/projection/extract-view-spec.js"; +import { emitViewDdl } from "../../src/projection/view-ddl-emit.js"; +import { buildProjectionViews } from "../../src/projection/build-projection-views.js"; + +type CodedError = Error & { readonly code?: string }; +const codeOf = (e: Error): string | undefined => (e as CodedError).code; + +async function load(children: unknown[]) { + const json = JSON.stringify({ "metadata.root": { package: "test", children } }); + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error( + `Loader errors:\n${result.errors.map((e) => `${codeOf(e)}: ${e.message}`).join("\n")}`, + ); + } + return result.root; +} + +/** `Program` with one int-backed enum + one string-backed enum (the control). */ +function programEntity() { + return { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { + "field.enum": { + name: "status", + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }, + }, + }, + { "field.enum": { name: "tier", "@values": ["FREE", "PAID"] } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }; +} + +function projection(name: string, view: string, filter: unknown) { + return { + "object.projection": { + name, + "@filter": filter, + children: [ + { "source.rdb": { "@kind": "view", "@view": view } }, + { "field.int": { name: "id", extends: "test::Program.id" } }, + { "field.enum": { name: "status", extends: "test::Program.status" } }, + { "field.enum": { name: "tier", extends: "test::Program.tier" } }, + { "identity.primary": { extends: "test::Program.id" } }, + ], + }, + }; +} + +async function sqlFor(filter: unknown): Promise { + const root = await load([programEntity(), projection("P", "v_p", filter)]); + const proj = root.objects().find((o) => o.name === "P")!; + const spec = extractViewSpec(proj, root, { columnNamingStrategy: "snake_case" }); + return emitViewDdl(spec, { + dialect: "postgres", + baseTableName: "programs", + joinTables: {}, + bodyOnly: true, + }); +} + +describe("#207 view @filter on an int-backed enum lowers to the integer", () => { + test("eq encodes the member symbol to its integer", async () => { + const sql = await sqlFor({ status: { eq: "PUBLISHED" } }); + expect(sql).toContain("WHERE p.status = 5"); + // The member symbol must NOT survive into the SQL. + expect(sql).not.toContain("'PUBLISHED'"); + }); + + test("ne encodes too", async () => { + const sql = await sqlFor({ status: { ne: "ARCHIVED" } }); + expect(sql).toContain("WHERE p.status <> 9"); + }); + + test("in encodes element-wise", async () => { + const sql = await sqlFor({ status: { in: ["DRAFT", "ARCHIVED"] } }); + expect(sql).toContain("WHERE p.status IN (0, 9)"); + }); + + test("the zero member encodes as 0, not dropped as falsy", async () => { + // DRAFT → 0. A truthiness-guarded encode would leave 'DRAFT' here; #235 is the + // precedent for a falsy-value bug in exactly this shape. + const sql = await sqlFor({ status: { eq: "DRAFT" } }); + expect(sql).toContain("WHERE p.status = 0"); + }); + + test("isNull is untouched by the encoding", async () => { + const sql = await sqlFor({ status: { isNull: true } }); + expect(sql).toContain("WHERE p.status IS NULL"); + }); + + test("a STRING-backed enum is byte-identical to before (quoted symbol)", async () => { + const sql = await sqlFor({ tier: { eq: "PAID" } }); + expect(sql).toContain("WHERE p.tier = 'PAID'"); + }); + + test("a composed and/or filter encodes inside both arms", async () => { + const sql = await sqlFor({ + or: [{ status: { eq: "DRAFT" } }, { status: { eq: "PUBLISHED" } }], + }); + expect(sql).toContain("(p.status = 0 OR p.status = 5)"); + }); + + test("the map is read RESOLVING — an inherited @intValueMap still encodes", async () => { + // The canonical post-#246 shape: the map on a shared root-level abstract enum, + // inherited by the consuming field. An own-only read would emit 'PUBLISHED'. + const root = await load([ + { + "field.enum": { + name: "SharedStatus", + abstract: true, + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5 }, + }, + }, + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { "field.enum": { name: "status", extends: "test::SharedStatus" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + { + "object.projection": { + name: "P", + "@filter": { status: { eq: "PUBLISHED" } }, + children: [ + { "source.rdb": { "@kind": "view", "@view": "v_p" } }, + { "field.int": { name: "id", extends: "test::Program.id" } }, + { "field.enum": { name: "status", extends: "test::Program.status" } }, + { "identity.primary": { extends: "test::Program.id" } }, + ], + }, + }, + ]); + const proj = root.objects().find((o) => o.name === "P")!; + const spec = extractViewSpec(proj, root, { columnNamingStrategy: "snake_case" }); + const sql = emitViewDdl(spec, { + dialect: "postgres", + baseTableName: "programs", + joinTables: {}, + bodyOnly: true, + }); + expect(sql).toContain("WHERE p.status = 5"); + expect(sql).not.toContain("'PUBLISHED'"); + }); +}); + +// The `origin.aggregate @filter` scoping filter is a SEPARATE resolver +// (resolveAggregateFilter) from the row-scope one, and renders as a SQL literal too — +// `FILTER (WHERE …)` on Postgres, `CASE WHEN` on SQLite. Assume a sibling code path +// exists: the row-scope fix above does not reach this one. +describe("origin.aggregate @filter on an int-backed enum lowers to the integer", () => { + const model = () => [ + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { + "relationship.association": { + name: "weeks", "@objectRef": "Week", "@cardinality": "many", + }, + }, + ], + }, + }, + { + "object.entity": { + name: "Week", + children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.int": { name: "id" } }, + { "field.int": { name: "programId" } }, + { "field.int": { name: "ordinal" } }, + { + "field.enum": { + name: "status", + "@values": ["DRAFT", "ACTIVE"], + "@intValueMap": { DRAFT: 0, ACTIVE: 7 }, + }, + }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { + "identity.reference": { + name: "ref_program", "@fields": "programId", "@references": "Program", + }, + }, + ], + }, + }, + { + "object.projection": { + name: "ProgramSummary", + children: [ + { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, + { "field.int": { name: "id", extends: "Program.id" } }, + { + "field.int": { + name: "activeMax", + children: [{ + "origin.aggregate": { + "@agg": "max", "@of": "Week.ordinal", "@via": "Program.weeks", + "@filter": { status: { eq: "ACTIVE" } }, + }, + }], + }, + }, + { "identity.primary": { name: "id", extends: "Program.id" } }, + ], + }, + }, + ]; + + test("postgres FILTER (WHERE …) compares the integer", async () => { + const root = await load(model()); + const [v] = buildProjectionViews(root, { + dialect: "postgres", columnNamingStrategy: "snake_case", + }); + expect(v!.sql).toMatch(/FILTER \(WHERE [a-z0-9]+\.status = 7\)/); + expect(v!.sql).not.toContain("'ACTIVE'"); + }); + + test("sqlite CASE WHEN compares the integer", async () => { + const root = await load(model()); + const [v] = buildProjectionViews(root, { + dialect: "sqlite", columnNamingStrategy: "snake_case", + }); + expect(v!.sql).toMatch(/CASE WHEN [a-z0-9]+\.status = 7 THEN/); + expect(v!.sql).not.toContain("'ACTIVE'"); + }); +}); diff --git a/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts new file mode 100644 index 000000000..6f9b1375f --- /dev/null +++ b/server/typescript/packages/integration-tests/test/enum-intvaluemap-pg.test.ts @@ -0,0 +1,613 @@ +/** + * Int-backed `field.enum` (`@intValueMap`) — the REAL gate, against a live Postgres. + * + * WHY THIS EXISTS + * + * Everything else about int-backed enums is verified by unit assertions and by + * inspecting generated source: migrate-ts says the column is `integer`, codegen-ts + * says the Drizzle column is a `customType`, and both say the CHECK lists unquoted + * integers. None of that proves the DDL APPLIES, that a second migrate CONVERGES, or + * that the codec actually round-trips a member symbol through a real integer column. + * + * This repo has a monument to exactly that gap: the 0.15.21 line, where a family of + * destructive migrate bugs survived a suite of thousands because nothing ever ran the + * pipeline twice against a real engine. `emit` and `introspect` had never been in the + * same room. The same is true here until this file runs. + * + * So: apply to a REAL engine, RE-DIFF, and then prove the value semantics both ways — + * a symbol written through the generated codec must land as the mapped INTEGER in the + * physical column, and an integer already in the column must read back as the symbol. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from "bun:test"; +import { + buildExpectedSchema, diff, emit, introspectPostgres, + type SchemaSnapshot, +} from "@metaobjectsdev/migrate-ts"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { runGen, defineConfig, buildProjectionViews } from "@metaobjectsdev/codegen-ts"; +import { entityFile, queriesFile } from "@metaobjectsdev/codegen-ts/generators"; +import { Kysely, PostgresDialect, sql } from "kysely"; +import pg, { Pool } from "pg"; +import { drizzle } from "drizzle-orm/node-postgres"; +import { eq, inArray } from "drizzle-orm"; +import { mkdirSync, mkdtempSync, rmSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { startPostgres, type RunningPg } from "../src/postgres-container.ts"; + +// The map is deliberately SPARSE and non-ordinal (0/5/9) so any accidental +// index-of-@values correspondence shows up as a wrong number rather than passing +// by coincidence — the failure mode the design's Goal 3 calls out. +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 } as const; + +/** An entity with an int-backed enum. The map lives on a SHARED root-level abstract + * declaration and the field inherits it — the shape #246 steers authors toward, and + * the one an own-only read would silently get wrong. */ +function meta(opts: { withDefault?: boolean } = {}): string { + const dflt = opts.withDefault ? `, "@default": "PUBLISHED"` : ""; + return `{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.enum": { "name": "status", "extends": "Status", "@required": true${dflt} } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } } + ] + } + }`; +} + +/** The string-backed control: identical model minus @intValueMap. */ +function metaStringBacked(): string { + return `{ + "metadata.root": { + "package": "acme", + "children": [ + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.enum": { "name": "status", "@required": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"] } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } } + ] + } + }`; +} + +let runningPg: RunningPg; +let pool: Pool; +let k: Kysely; +/** Shared with the generated-codec block below, which needs its own Drizzle pool + * against the SAME database the DDL was applied to. */ +let pg2Uri: string; + +beforeAll(async () => { + runningPg = await startPostgres(); + pg2Uri = runningPg.connectionUri; + pool = new Pool({ connectionString: runningPg.connectionUri }); + k = new Kysely({ dialect: new PostgresDialect({ pool }) }); +}, 120_000); + +afterAll(async () => { + await k?.destroy(); + await runningPg?.stop(); +}); + +beforeEach(async () => { + await sql.raw(`DROP TABLE IF EXISTS "orders" CASCADE;`).execute(k); + await sql.raw(`DROP TABLE IF EXISTS orders CASCADE;`).execute(k); + // The TPH block below builds its own base table; CASCADE also clears any + // dependent view left by the projection block. + await sql.raw(`DROP TABLE IF EXISTS "auths" CASCADE;`).execute(k); +}); + +async function applyRaw(ddl: string): Promise { + for (const stmt of ddl.split(/;\s*\n/).map((s) => s.trim()).filter(Boolean)) { + await sql.raw(stmt.endsWith(";") ? stmt : `${stmt};`).execute(k); + } +} + +async function expectedFor(metaJson: string): Promise { + const root = (await new MetaDataLoader().load([new InMemoryStringSource(metaJson)])).root; + return buildExpectedSchema(root, { columnNamingStrategy: "literal", dialect: "postgres" }); +} + +/** build → introspect → diff → emit → apply, then return the expected side. */ +async function migrate(metaJson: string): Promise { + const expected = await expectedFor(metaJson); + const result = await diff({ + expected, actual: await introspectPostgres(k), dialect: "postgres", + }); + expect(result.blocked).toEqual([]); + const { up } = result.changes.length === 0 + ? { up: "" } + : emit(result.changes, { dialect: "postgres" }); + if (up.trim().length > 0) await applyRaw(up); + return expected; +} + +/** THE gate: a second migrate against the just-migrated DB must be a no-op. */ +async function assertConverged(expected: SchemaSnapshot): Promise { + const followup = await diff({ + expected, actual: await introspectPostgres(k), dialect: "postgres", + }); + if (followup.changes.length > 0) { + console.error("NOT CONVERGED — a second migrate would emit:"); + for (const c of followup.changes) console.error(" -", c.kind, JSON.stringify(c).slice(0, 300)); + } + expect(followup.changes).toEqual([]); +} + +describe("int-backed field.enum — real Postgres", () => { + test("the emitted DDL APPLIES and a second migrate converges", async () => { + const expected = await migrate(meta()); + await assertConverged(expected); + }, 120_000); + + test("the physical column is integer, not text", async () => { + await migrate(meta()); + const rows = await sql<{ data_type: string }>` + SELECT data_type FROM information_schema.columns + WHERE table_name = 'orders' AND column_name = 'status' + `.execute(k); + expect(rows.rows[0]?.data_type).toBe("integer"); + }, 120_000); + + test("the CHECK enforces the mapped INTEGERS — a valid member's int is accepted, a non-member int rejected", async () => { + await migrate(meta()); + // 5 === PUBLISHED, so this must be accepted. + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('ok', 5);`).execute(k); + // 7 maps to no member. If the CHECK had been emitted over the member STRINGS + // (or omitted), this would succeed and the column would hold an impossible value. + let rejected = false; + try { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('bad', 7);`).execute(k); + } catch { + rejected = true; + } + expect(rejected).toBe(true); + }, 120_000); + + test("an ordinal-looking wrong value is rejected — the map is sparse, not positional", async () => { + await migrate(meta()); + // ARCHIVED is index 2 in @values but maps to 9. If anything derived the stored + // int from the member's POSITION (design Goal 3's hazard), 2 would be valid. + let rejected = false; + try { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('ordinal', 2);`).execute(k); + } catch { + rejected = true; + } + expect(rejected).toBe(true); + }, 120_000); + + test("@default lands as the mapped integer, appliably", async () => { + const expected = await migrate(meta({ withDefault: true })); + await assertConverged(expected); + await sql.raw(`INSERT INTO "orders" ("title") VALUES ('defaulted');`).execute(k); + const rows = await sql<{ status: number }>` + SELECT "status" FROM "orders" WHERE "title" = 'defaulted' + `.execute(k); + // PUBLISHED === 5. A DEFAULT 'PUBLISHED' on an integer column would not have + // applied at all, so reaching this assertion is itself part of the proof. + expect(rows.rows[0]?.status).toBe(5); + }, 120_000); + + test("the string-backed control still gets a text column and converges", async () => { + const expected = await migrate(metaStringBacked()); + await assertConverged(expected); + const rows = await sql<{ data_type: string }>` + SELECT data_type FROM information_schema.columns + WHERE table_name = 'orders' AND column_name = 'status' + `.execute(k); + expect(rows.rows[0]?.data_type).toBe("text"); + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('s', 'DRAFT');`).execute(k); + }, 120_000); + + test("toggling the backing on an existing table is BLOCKED (no silent destructive recast)", async () => { + await migrate(metaStringBacked()); + const intExpected = await expectedFor(meta()); + const result = await diff({ + expected: intExpected, actual: await introspectPostgres(k), dialect: "postgres", + }); + const typeChange = result.changes.find((c) => c.kind === "change-column-type"); + expect(typeChange).toBeDefined(); + expect(typeChange!.status.state).toBe("blocked"); + }, 120_000); +}); + +// --------------------------------------------------------------------------- +// The CODEC half. Everything above exercises the SCHEMA through raw SQL, so +// customType's toDriver/fromDriver had still never run against a database. This +// block emits the REAL entity file via runGen, imports it unmodified, and drives +// Drizzle through it — the only way to prove the generated codec, as opposed to +// a hand-written mirror of what we think it generates. +// --------------------------------------------------------------------------- + +describe("int-backed field.enum — generated codec against real Postgres", () => { + // Emit INSIDE the package tree (.gen-tmp/, gitignored): the generated module + // resolves bare specifiers like `drizzle-orm/pg-core` by walking up to a + // node_modules chain, which the OS tmpdir never reaches. + let tmp: string; + let ordersTable: any; + let gdb: any; + let gpool: pg.Pool; + + beforeAll(async () => { + const here = dirname(fileURLToPath(import.meta.url)); + const genTmpRoot = join(here, "..", ".gen-tmp"); + mkdirSync(genTmpRoot, { recursive: true }); + tmp = mkdtempSync(join(genTmpRoot, "enum-intmap-")); + + const root = (await new MetaDataLoader().load([new InMemoryStringSource(meta())])).root; + const lr = await runGen({ + config: defineConfig({ + outDir: tmp, + extStyle: "none", + dbImport: "./db", + dialect: "postgres", + generators: [entityFile()], + }), + metadata: root, + }); + if (lr.warnings.length > 0) throw new Error(`codegen warnings: ${lr.warnings.join("; ")}`); + + // The emitted entity module, imported UNMODIFIED — codec included. + const entityUrl = pathToFileURL(join(tmp, "Order.ts")).href; + const mod: any = await import(entityUrl); + ordersTable = mod.orders; + expect(ordersTable).toBeDefined(); + + gpool = new pg.Pool({ connectionString: pg2Uri }); + gdb = drizzle(gpool); + }, 180_000); + + afterAll(async () => { + await gpool?.end(); + rmSync(tmp, { recursive: true, force: true }); + }); + + beforeEach(async () => { + // Schema provisioned from the SAME metadata via migrate-ts, so the physical + // shape the codec writes into is the one the DDL pipeline produces. + await migrate(meta()); + }); + + test("a member symbol written through Drizzle lands as the mapped INTEGER", async () => { + await gdb.insert(ordersTable).values({ title: "w", status: "PUBLISHED" }); + // Read the PHYSICAL value with raw SQL — bypassing the codec entirely, so this + // asserts what is actually on disk rather than what the codec round-trips. + const raw = await sql<{ status: number }>`SELECT "status" FROM "orders" WHERE "title" = 'w'`.execute(k); + expect(raw.rows[0]?.status).toBe(5); + }, 120_000); + + test("an integer already in the column reads back as its member symbol", async () => { + // Insert 9 (ARCHIVED) with raw SQL so the value never passes through toDriver — + // proving fromDriver decodes, not merely that the two directions cancel out. + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('r', 9);`).execute(k); + const rows = await gdb.select().from(ordersTable); + expect(rows).toHaveLength(1); + expect(rows[0].status).toBe("ARCHIVED"); + }, 120_000); + + test("round-trips every member, including the ZERO-valued one", async () => { + for (const m of ["DRAFT", "PUBLISHED", "ARCHIVED"]) { + await gdb.insert(ordersTable).values({ title: m, status: m }); + } + const rows = await gdb.select().from(ordersTable); + const byTitle = new Map(rows.map((r: any) => [r.title, r.status])); + // DRAFT maps to 0 — falsy, and therefore the value any truthiness-based codec + // silently drops or coerces (the #235 bug class). + expect(byTitle.get("DRAFT")).toBe("DRAFT"); + expect(byTitle.get("PUBLISHED")).toBe("PUBLISHED"); + expect(byTitle.get("ARCHIVED")).toBe("ARCHIVED"); + const raws = await sql<{ title: string; status: number }>`SELECT "title", "status" FROM "orders"`.execute(k); + expect(new Map(raws.rows.map((r) => [r.title, r.status])).get("DRAFT")).toBe(0); + }, 120_000); + + // Task 8 — the filter path. This is the claim that customType makes the + // filter work "for free": Drizzle binds a WHERE comparison through the column + // type, so a member symbol encodes without the filter layer knowing anything. + test("a WHERE comparison on a member symbol encodes through the column type", async () => { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('a', 0), ('b', 9);`).execute(k); + const archived = await gdb.select().from(ordersTable).where(eq(ordersTable.status, "ARCHIVED")); + expect(archived).toHaveLength(1); + expect(archived[0].title).toBe("b"); + const drafts = await gdb.select().from(ordersTable).where(eq(ordersTable.status, "DRAFT")); + expect(drafts).toHaveLength(1); + expect(drafts[0].title).toBe("a"); + }, 120_000); + + test("an `in` comparison encodes every member in the list", async () => { + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('a', 0), ('b', 5), ('c', 9);`).execute(k); + const some = await gdb.select().from(ordersTable) + .where(inArray(ordersTable.status, ["DRAFT", "ARCHIVED"])); + expect(some.map((r: any) => r.title).sort()).toEqual(["a", "c"]); + }, 120_000); + + test("an unmapped stored integer throws on read instead of yielding undefined", async () => { + // The CHECK normally makes this unreachable; drop it to simulate data written + // before a member was removed, or by a hand-written migration. + await sql.raw(`ALTER TABLE "orders" DROP CONSTRAINT "orders_status_chk";`).execute(k); + await sql.raw(`INSERT INTO "orders" ("title", "status") VALUES ('bogus', 42);`).execute(k); + let threw = false; + try { + await gdb.select().from(ordersTable); + } catch (e) { + threw = true; + expect(String(e)).toContain("unmapped"); + } + expect(threw).toBe(true); + }, 120_000); +}); + +/** + * A projection row-scope `@filter` (#207) on an int-backed enum. + * + * The view body is emitted as LITERAL SQL text and never touches Drizzle, so the + * customType that rescues the runtime query path does nothing here. Before the fix + * this emitted `WHERE p.status = 'PUBLISHED'` against an `integer` column, which + * Postgres rejects at CREATE VIEW time — `invalid input syntax for type integer` — + * aborting the migration. A unit assertion on the emitted string cannot show that; + * only applying it can. + */ +describe("int-backed field.enum in a projection view — real Postgres", () => { + function metaWithView(): string { + return `{ + "metadata.root": { + "package": "acme", + "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], + "@intValueMap": { "DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9 } } }, + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": {} }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.enum": { "name": "status", "extends": "Status", "@required": true } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.projection": { "name": "PublishedOrders", + "@filter": { "status": { "eq": "PUBLISHED" } }, "children": [ + { "source.rdb": { "@kind": "view", "@table": "v_published_orders" } }, + { "identity.primary": { "name": "id", "extends": "Order.id", "@fields": "id" } }, + { "field.long": { "name": "id", "extends": "Order.id", "children": [ + { "origin.passthrough": { "@from": "Order.id" } } ] } }, + { "field.string": { "name": "title", "children": [ + { "origin.passthrough": { "@from": "Order.title" } } ] } }, + { "field.enum": { "name": "status", "extends": "Status", "children": [ + { "origin.passthrough": { "@from": "Order.status" } } ] } } + ] } } + ] + } + }`; + } + + /** Views are NOT derived by buildExpectedSchema — they must be passed in + * explicitly, so this block needs its own migrate helper rather than the + * table-only one above. */ + async function migrateWithViews(metaJson: string): Promise { + const root = (await new MetaDataLoader().load([new InMemoryStringSource(metaJson)])).root; + const expected = buildExpectedSchema(root, { + columnNamingStrategy: "literal", + dialect: "postgres", + views: buildProjectionViews(root, { + dialect: "postgres", columnNamingStrategy: "literal", + }), + }); + const result = await diff({ + expected, actual: await introspectPostgres(k), dialect: "postgres", + }); + expect(result.blocked).toEqual([]); + const { up } = result.changes.length === 0 + ? { up: "" } + : emit(result.changes, { dialect: "postgres" }); + if (up.trim().length > 0) await applyRaw(up); + return expected; + } + + test("the filtered view APPLIES, converges, and selects by the INTEGER", async () => { + const expected = await migrateWithViews(metaWithView()); + // The view exists — i.e. the CREATE VIEW did not blow up on a text-vs-integer + // comparison. This is the assertion the whole test exists for. + await assertConverged(expected); + + await sql.raw( + `INSERT INTO "orders" ("title", "status") VALUES ('a', 0), ('b', 5), ('c', 9), ('d', 5);`, + ).execute(k); + + const rows = await sql<{ title: string }>` + SELECT "title" FROM "v_published_orders" ORDER BY "title" + `.execute(k); + // Only the two PUBLISHED (5) rows — proving the WHERE compared 5, not 'PUBLISHED'. + expect(rows.rows.map((r) => r.title)).toEqual(["b", "d"]); + }, 120_000); + + test("the emitted view body carries the integer literal, not the member symbol", async () => { + const root = (await new MetaDataLoader().load([ + new InMemoryStringSource(metaWithView()), + ])).root; + const views = buildProjectionViews(root, { + dialect: "postgres", columnNamingStrategy: "literal", + }); + const body = views.find((v) => v.name === "v_published_orders")?.sql ?? ""; + expect(body).toMatch(/WHERE o\.status = 5/); + expect(body).not.toContain("'PUBLISHED'"); + }, 120_000); +}); + +/** + * TPH (single-table discriminator) + int-backed enums — Task 9. + * + * Two distinct questions, and the second is the one the plan flagged as possibly + * needing its own design: + * 1. a per-subtype read schema must tolerate an int-backed enum COLUMN; + * 2. an int-backed enum used AS the DISCRIMINATOR must still pin, filter and insert. + * + * Reading the generated source says both work: every TPH path goes through Drizzle + * (`db.select()`, `eq(auths.type, "Bridge")`, `.values()`), so the Task 5 customType + * encodes and decodes at the column, and the schemas only ever see member symbols. + * But #203/#229 is the precedent for TPH being a separate code path that everyone + * assumes is covered — and this repo's 0.15.21 line is what "the source looks right" + * is worth. So: run it. + * + * If this had needed its own design, the documented fallback was to REJECT + * @intValueMap on a discriminator with a named loader error. It does not — the + * discriminator case is SUPPORTED, and these tests are what pins that. + */ +describe("int-backed field.enum under TPH — real Postgres", () => { + // The discriminator (`type`) is int-backed 1/2, and there is ALSO a non- + // discriminator int-backed enum (`status`, 0/7) so both questions are live in one + // model. Both maps are non-ordinal so an accidental index-of-@values correspondence + // shows up as a wrong number rather than passing by coincidence. + const TPH_META = `{ + "metadata.root": { "package": "demo", "children": [ + { "object.entity": { "name": "Auth", "@discriminator": "type", "children": [ + { "source.rdb": { "@table": "auths" } }, + { "field.enum": { "name": "type", "@values": ["Bridge", "Copay"], + "@intValueMap": { "Bridge": 1, "Copay": 2 } } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "field.enum": { "name": "status", "@values": ["OPEN", "SHUT"], + "@intValueMap": { "OPEN": 0, "SHUT": 7 } } }, + { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "BridgeAuth", "extends": "demo::Auth", + "@discriminatorValue": "Bridge", "children": [ + { "field.int": { "name": "quantity" } } ] } }, + { "object.entity": { "name": "CopayAuth", "extends": "demo::Auth", + "@discriminatorValue": "Copay", "children": [ + { "field.int": { "name": "amount" } } ] } } + ] } }`; + + let tphTmp: string; + let q: any; // the generated Auth.queries module + let tphDb: any; // Drizzle handle against the same database + let tphPool: pg.Pool; + + beforeAll(async () => { + const here = dirname(fileURLToPath(import.meta.url)); + const genTmpRoot = join(here, "..", ".gen-tmp"); + mkdirSync(genTmpRoot, { recursive: true }); + tphTmp = mkdtempSync(join(genTmpRoot, "enum-tph-")); + + const root = (await new MetaDataLoader().load([new InMemoryStringSource(TPH_META)])).root; + const lr = await runGen({ + config: defineConfig({ + outDir: tphTmp, extStyle: "none", dbImport: "./db", dialect: "postgres", + generators: [entityFile(), queriesFile()], + }), + metadata: root, + }); + if (lr.warnings.length > 0) throw new Error(`codegen warnings: ${lr.warnings.join("; ")}`); + + // The queries module takes its Db as a PARAMETER, so it needs no db module — + // it is imported exactly as emitted. + q = await import(pathToFileURL(join(tphTmp, "Auth.queries.ts")).href); + tphPool = new pg.Pool({ connectionString: pg2Uri }); + tphDb = drizzle(tphPool); + }, 180_000); + + afterAll(async () => { + await tphPool?.end(); + rmSync(tphTmp, { recursive: true, force: true }); + }); + + beforeEach(async () => { + await migrate(TPH_META); + }); + + test("the TPH base table DDL applies and a second migrate converges", async () => { + const expected = await expectedFor(TPH_META); + await assertConverged(expected); + }, 120_000); + + test("the discriminator column is integer with an INTEGER check", async () => { + const col = await sql<{ data_type: string }>` + SELECT data_type FROM information_schema.columns + WHERE table_name = 'auths' AND column_name = 'type' + `.execute(k); + expect(col.rows[0]?.data_type).toBe("integer"); + + const chk = await sql<{ def: string }>` + SELECT pg_get_constraintdef(oid) AS def FROM pg_constraint + WHERE conname = 'auths_type_chk' + `.execute(k); + // Unquoted integers, not 'Bridge'/'Copay' — the discriminator's CHECK is + // subject to the same int-backing as any other enum column. + expect(chk.rows[0]?.def).toContain("1"); + expect(chk.rows[0]?.def).toContain("2"); + expect(chk.rows[0]?.def).not.toContain("Bridge"); + }, 120_000); + + test("create through the generated per-subtype fn stores BOTH enums as integers", async () => { + // `type` is required by BridgeAuthInsertSchema (z.literal("Bridge")) — that is + // pre-existing TPH behaviour, identical for a string-backed discriminator; the + // ROUTES layer is what omits it and re-adds it from the URL. + await q.createBridgeAuth(tphDb, { + type: "Bridge", title: "a", status: "SHUT", quantity: 3, + }); + // Raw SQL, bypassing the codec — this is what is actually on disk. + const raw = await sql<{ type: number; status: number }>` + SELECT "type", "status" FROM "auths" WHERE "title" = 'a' + `.execute(k); + expect(raw.rows[0]?.type).toBe(1); // Bridge, not 'Bridge' + expect(raw.rows[0]?.status).toBe(7); // SHUT + }, 120_000); + + test("the per-subtype read schema decodes an int-backed enum column", async () => { + // Insert with raw SQL so neither value passes through toDriver — proving the + // read path decodes rather than the two directions cancelling out. + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status", "quantity") VALUES (1, 'b', 0, 9);`, + ).execute(k); + const rows = await q.listBridgeAuths(tphDb); + expect(rows).toHaveLength(1); + expect(rows[0].type).toBe("Bridge"); // z.literal("Bridge") accepted the decoded value + expect(rows[0].status).toBe("OPEN"); // the ZERO-valued member + expect(rows[0].quantity).toBe(9); + }, 120_000); + + test("the per-subtype filter compares the INTEGER discriminator", async () => { + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status") VALUES (1, 'b', 0), (2, 'c', 7), (1, 'd', 7);`, + ).execute(k); + const bridges = await q.listBridgeAuths(tphDb); + expect(bridges.map((r: any) => r.title).sort()).toEqual(["b", "d"]); + const copays = await q.listCopayAuths(tphDb); + expect(copays.map((r: any) => r.title)).toEqual(["c"]); + }, 120_000); + + test("the polymorphic read dispatches on the DECODED discriminator", async () => { + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status", "amount") VALUES (2, 'c', 7, 42);`, + ).execute(k); + const all = await q.listAuths(tphDb); + expect(all).toHaveLength(1); + // parseAuth read `type` as "Copay" and dispatched to CopayAuthSchema — a raw 2 + // would have thrown on the z.enum head parse. + expect(all[0].type).toBe("Copay"); + expect(all[0].amount).toBe(42); + }, 120_000); + + test("find-by-id is scoped by the integer discriminator, not just the PK", async () => { + await sql.raw( + `INSERT INTO "auths" ("type", "title", "status") VALUES (2, 'c', 7);`, + ).execute(k); + const [{ id }] = (await sql<{ id: string }>`SELECT "id" FROM "auths"`.execute(k)).rows as any; + // The row IS a Copay, so asking for it as a Bridge must miss — proving the AND'd + // discriminator predicate encoded to 1 rather than binding 'Bridge'. + expect(await q.findBridgeAuthById(tphDb, Number(id))).toBeNull(); + expect((await q.findCopayAuthById(tphDb, Number(id)))?.title).toBe("c"); + }, 120_000); +}); diff --git a/server/typescript/packages/integration-tests/test/validation-conformance.test.ts b/server/typescript/packages/integration-tests/test/validation-conformance.test.ts index 4fb095588..8493b3d20 100644 --- a/server/typescript/packages/integration-tests/test/validation-conformance.test.ts +++ b/server/typescript/packages/integration-tests/test/validation-conformance.test.ts @@ -35,6 +35,12 @@ beforeAll(async () => { const zodPath = pathToFileURL(Bun.resolveSync("zod", import.meta.dir)).href; tmpDir = mkdtempSync(join(tmpdir(), "validation-conformance-")); + // Write EVERY module before importing ANY of them. Bun caches a directory's + // listing at the first import out of it, so a sibling written after that import + // is invisible to the resolver and fails with `Cannot find module ` even + // though the file is on disk — which is exactly what a write-then-import loop + // produced here (Account resolved, Ledger did not). + const modulePaths: Array = []; for (const entityName of ENTITY_NAMES) { const entity = root.findObject(entityName); if (!entity) throw new Error(`corpus meta.json has no object named ${entityName}`); @@ -46,10 +52,14 @@ beforeAll(async () => { generated = generated.replace(/(['"])zod\1/g, JSON.stringify(zodPath)); // Emit to a temp module (outside the package tree, so it can't be picked up by - // a later test glob) and import it so we exercise the real generated code. + // a later test glob). const modulePath = join(tmpDir, `${entityName}.ts`); writeFileSync(modulePath, generated, "utf8"); + modulePaths.push([entityName, modulePath] as const); + } + for (const [entityName, modulePath] of modulePaths) { + // Import so we exercise the real generated code. const mod = (await import(pathToFileURL(modulePath).href)) as Record; const schema = mod[`${entityName}InsertSchema`]; if (!schema) throw new Error(`generated module did not export ${entityName}InsertSchema`); diff --git a/server/typescript/packages/metadata/src/attr-schema-validate.ts b/server/typescript/packages/metadata/src/attr-schema-validate.ts index 479b5e8df..0728d1295 100644 --- a/server/typescript/packages/metadata/src/attr-schema-validate.ts +++ b/server/typescript/packages/metadata/src/attr-schema-validate.ts @@ -52,6 +52,7 @@ import { FIELD_ATTR_VALUES, FIELD_ATTR_COERCE_DEFAULT, FIELD_ATTR_DEFAULT, + FIELD_ATTR_INT_VALUE_MAP, ENUM_MEMBER_PATTERN, } from "./core/field/field-constants.js"; import { @@ -309,6 +310,19 @@ function validateNode( // only own @values need checking here (mirrors the own-attrs-only policy of // Checks 2+3 above — inherited attrs were validated on the declaring node). if (node.type === TYPE_FIELD && node.subType === FIELD_SUBTYPE_ENUM) { + // #246: the shared-enum super, if any — a root-level abstract field.enum (one + // whose parent is the metadata root, not an object). FR-019 materializes such a + // declaration ONCE per port as a single named type, so anything a consuming + // field re-declares that is part of the shared TYPE's contract is a conflict. + // Immediate-super-only, matching codegen's resolveSharedEnumDecl (enum-shared.ts) + // so the validator and the collapse agree on what "shared" means. + const sharedSuper = + node.superData !== undefined && + node.superData.isAbstract && + node.superData.parent?.type === TYPE_METADATA + ? node.superData + : undefined; + // ADR-0039: own — validates the @values DECLARED on this node; a concrete enum // extending an abstract one inherits already-validated @values (own-attrs-only). const rawValues = node.ownAttrs().get(FIELD_ATTR_VALUES); @@ -347,18 +361,16 @@ function validateNode( } } - // #246: a field.enum extending a shared package-level abstract enum - // (a root-level abstract field — one whose parent is the metadata root, - // not an object) that ALSO declares its own @values is a conflict: one - // shared enum type has one member set, so codegen's shared-enum collapse - // would silently drop this field's own @values in favor of the shared - // type's. Own-attrs-only (matches the rest of Check 4): only fires when - // THIS node declares @values itself, not when it merely inherits. - const sup = node.superData; - if (sup !== undefined && sup.isAbstract && sup.parent?.type === TYPE_METADATA) { + // #246: a field.enum extending a shared package-level abstract enum that + // ALSO declares its own @values is a conflict: one shared enum type has one + // member set, so codegen's shared-enum collapse would silently drop this + // field's own @values in favor of the shared type's. Own-attrs-only + // (matches the rest of Check 4): only fires when THIS node declares + // @values itself, not when it merely inherits. + if (sharedSuper !== undefined) { errors.push( new ParseError( - `${nodeLabel(node)} extends shared abstract enum '${nodeLabel(sup)}' AND declares its own ` + + `${nodeLabel(node)} extends shared abstract enum '${nodeLabel(sharedSuper)}' AND declares its own ` + `'@${FIELD_ATTR_VALUES}' — a shared enum's member set is owned by the shared declaration; ` + `remove the own '@${FIELD_ATTR_VALUES}' to inherit it, or extend a non-shared enum instead.`, { code: "ERR_ENUM_EXTENDS_VALUES_CONFLICT", source: node.source }, @@ -399,6 +411,97 @@ function validateNode( } } } + + // --- Check 5a: @intValueMap is scalar-only (design D7) --- + // + // Int-backing is a persistence-layer CODEC, and no port implements it + // element-wise over an array column: OMDB's EnumCodec and Kotlin's + // customEnumeration are scalar by construction, Python would bind the symbol + // list straight into an integer[], and TS's sqlite branch serializes an array + // as JSON text before the enum case is ever reached. Two ports that DO compose + // (TS/Postgres via .array(), C# via PrimitiveCollection) are not a feature — + // shipping a claim four ports silently get wrong is the field.byte/short/class + // mistake. Rejected at LOAD so it fails identically everywhere. + // + // BOTH reads are RESOLVING, unlike Check 5b below: the illegal thing is the + // EFFECTIVE combination. Post-#246 the map must live on the shared abstract + // declaration, so the field that inherits it is exactly where isArray gets + // declared — an own-only read would see the two halves on different nodes and + // never fire. + if (node.attrs().get(FIELD_ATTR_INT_VALUE_MAP) !== undefined && node.resolvedIsArray()) { + errors.push( + new ParseError( + `${nodeLabel(node)} declares '@${FIELD_ATTR_INT_VALUE_MAP}' with isArray=true; ` + + `int-backing is scalar-only — an array-of-enum persists its member symbols. ` + + `Remove '@${FIELD_ATTR_INT_VALUE_MAP}', or make the field scalar.`, + { code: "ERR_ENUM_INT_VALUE_MAP_ARRAY", source: node.source }, + ), + ); + } + + // --- Check 5b: field.enum @intValueMap content rules --- + // + // Optional. Own-only (mirrors Checks 4/5's own-attrs-only policy) — an + // inherited @intValueMap is validated on its declaring node. The generic + // "is this an object of integers" shape check already ran via IntMapAttr + // (attr subtype `intMap`); this validates the field.enum-SPECIFIC + // semantics: key-set-equals-@values, and no two members share a value. + const rawIntValueMap = node.ownAttrs().get(FIELD_ATTR_INT_VALUE_MAP); + if (rawIntValueMap !== undefined && typeof rawIntValueMap === "object" && rawIntValueMap !== null) { + // #246 (int-backed twin): the symbol→int mapping is a property of the enum + // VOCABULARY, not of one column that uses it — it is @values' numeric half. + // A shared enum is materialized once as a single type, so a per-field map + // would give one logical type N storage encodings (and, where a port emits + // per-TYPE codec artifacts, two same-named declarations). Same remedy as the + // @values half: declare it on the shared declaration and inherit it. + if (sharedSuper !== undefined) { + errors.push( + new ParseError( + `${nodeLabel(node)} extends shared abstract enum '${nodeLabel(sharedSuper)}' AND declares its own ` + + `'@${FIELD_ATTR_INT_VALUE_MAP}' — a shared enum's integer backing is owned by the shared ` + + `declaration; move '@${FIELD_ATTR_INT_VALUE_MAP}' onto '${nodeLabel(sharedSuper)}' to inherit it, ` + + `or extend a non-shared enum instead.`, + { code: "ERR_ENUM_EXTENDS_VALUES_CONFLICT", source: node.source }, + ), + ); + } + const map = rawIntValueMap as Record; + const effectiveValues = node.attrs().get(FIELD_ATTR_VALUES); + const declaredMembers: string[] = Array.isArray(effectiveValues) ? effectiveValues : []; + const memberSet = new Set(declaredMembers); + const mapKeys = Object.keys(map); + const keySet = new Set(mapKeys); + + const missing = declaredMembers.filter((m) => !keySet.has(m)); + const extra = mapKeys.filter((k) => !memberSet.has(k)); + if (missing.length > 0 || extra.length > 0) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' keys must exactly match '@${FIELD_ATTR_VALUES}' members` + + (missing.length > 0 ? ` (missing: ${missing.join(", ")})` : "") + + (extra.length > 0 ? ` (unknown: ${extra.join(", ")})` : "") + ".", + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } + + const seenValues = new Map(); + for (const [member, value] of Object.entries(map)) { + if (typeof value !== "number" || !Number.isInteger(value)) continue; // IntMapAttr already reported this + const owner = seenValues.get(value); + if (owner !== undefined) { + errors.push( + new ParseError( + `${nodeLabel(node)} attribute '@${FIELD_ATTR_INT_VALUE_MAP}' members '${owner}' and '${member}' ` + + `share the same value ${value}; every member must have a unique int.`, + { code: "ERR_BAD_ATTR_VALUE", source: node.source }, + ), + ); + } else { + seenValues.set(value, member); + } + } + } } // --- Check 6 (R6 Plan 2b): @dbColumnType (logical subtype × value) pairing --- diff --git a/server/typescript/packages/metadata/src/core-types.ts b/server/typescript/packages/metadata/src/core-types.ts index ced9aac7b..b12cb787b 100644 --- a/server/typescript/packages/metadata/src/core-types.ts +++ b/server/typescript/packages/metadata/src/core-types.ts @@ -20,6 +20,7 @@ import "./core/attr/meta-attr-stringarray.js"; import "./core/attr/meta-attr-filter.js"; import "./core/attr/meta-attr-properties.js"; import "./core/attr/meta-attr-expression.js"; +import "./core/attr/meta-attr-int-map.js"; import { MetaValidator, MetaRequiredValidator, diff --git a/server/typescript/packages/metadata/src/core/attr/attr-constants.ts b/server/typescript/packages/metadata/src/core/attr/attr-constants.ts index 8b1cbcfc1..4c7faff72 100644 --- a/server/typescript/packages/metadata/src/core/attr/attr-constants.ts +++ b/server/typescript/packages/metadata/src/core/attr/attr-constants.ts @@ -17,6 +17,11 @@ export const ATTR_SUBTYPE_FILTER = "filter"; // #195 — a structured expression tree over a base entity's own fields (backs // origin.computed). Object-shaped; a closed node grammar (see meta-attr-expression.ts). export const ATTR_SUBTYPE_EXPRESSION = "expression"; +// An object-shaped attr whose values are all integers (e.g. field.enum's +// @intValueMap: {memberSymbol: int}). Generic shape check only — semantic +// rules specific to a consumer (key-set membership, uniqueness) are that +// consumer's own content-rule validation, not this attr's. +export const ATTR_SUBTYPE_INT_MAP = "intMap"; /** * The retired `stringarray` array attr subtype. It is NO LONGER a registered @@ -41,6 +46,7 @@ export const ATTR_SUBTYPES = [ ATTR_SUBTYPE_PROPERTIES, ATTR_SUBTYPE_FILTER, ATTR_SUBTYPE_EXPRESSION, + ATTR_SUBTYPE_INT_MAP, ] as const; export type AttrSubType = | (typeof ATTR_SUBTYPES)[number] diff --git a/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts b/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts index 0fdf7dcdc..fa0afaade 100644 --- a/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/attr/attr-definition.embedded.ts @@ -27,6 +27,12 @@ export const ATTR_DEFINITION: ProviderDefinition = { "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts b/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts new file mode 100644 index 000000000..00b14755b --- /dev/null +++ b/server/typescript/packages/metadata/src/core/attr/meta-attr-int-map.ts @@ -0,0 +1,54 @@ +// IntMapAttr — attr subtype `intMap`. Object-shaped value whose members must +// all be integers (e.g. field.enum's @intValueMap). No desugar; validates +// shape (object, not array) and every value's type (integer). A consumer's +// own semantic rules (key-set membership, uniqueness) are validated by that +// consumer, not here — mirrors how StringArrayAttr validates shape while +// field.enum's own content-rule pass validates its @values semantics. + +import { MetaAttr, type ValueError, runtimeTypeName } from "./meta-attr.js"; +import { type AttrValue } from "../../shared/meta-data.js"; +import { DATA_TYPE_OBJECT, type DataType } from "../../data-type.js"; +import { registerAttrClass } from "../../attr-class-map.js"; +import { ATTR_SUBTYPE_INT_MAP } from "./attr-constants.js"; + +// 32-bit signed integer bounds (inclusive) — the eventual DB column for an +// int-backed enum is a 32-bit Postgres/SQLite `integer` (design doc D5). +const INT32_MIN = -2147483648; +const INT32_MAX = 2147483647; + +export class IntMapAttr extends MetaAttr { + override get dataType(): DataType { + return DATA_TYPE_OBJECT; + } + + override coerce(raw: unknown): AttrValue { + return raw as AttrValue; + } + + override validateValue(value: AttrValue): ValueError[] { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return [{ message: `attribute '@${this.name}' must be of type 'intMap' but got ${runtimeTypeName(value)}` }]; + } + const errors: ValueError[] = []; + for (const [key, member] of Object.entries(value as Record)) { + if (typeof member !== "number" || !Number.isInteger(member)) { + errors.push({ + message: `attribute '@${this.name}' member '${key}' has value '${String(member)}' which is not an integer`, + }); + } else if (member < INT32_MIN || member > INT32_MAX) { + // The eventual DB column for an int-backed enum is a 32-bit Postgres/ + // SQLite `integer` (design doc D5; matches field.int's existing + // 32-bit mapping in expected-schema.ts) — mirrors Java's + // IntMapAttribute#setValueAsString bound check exactly (inclusive at + // both ends) so a value that could never be persisted fails at load + // time, not silently, on every port. + errors.push({ + message: `attribute '@${this.name}' member '${key}' has value '${member}' which is outside the 32-bit signed integer range`, + }); + } + } + return errors; + } +} + +registerAttrClass(ATTR_SUBTYPE_INT_MAP, IntMapAttr); diff --git a/server/typescript/packages/metadata/src/core/field/field-constants.ts b/server/typescript/packages/metadata/src/core/field/field-constants.ts index 7b9197266..52de625fa 100644 --- a/server/typescript/packages/metadata/src/core/field/field-constants.ts +++ b/server/typescript/packages/metadata/src/core/field/field-constants.ts @@ -166,6 +166,16 @@ export const FIELD_ATTR_CURRENCY_DEFAULT = "USD"; /** Member symbols of an enum-subtype field. Required, string array. */ export const FIELD_ATTR_VALUES = "values"; +/** + * Optional per-member explicit integer value ({memberSymbol: int}) switching + * this enum field's DB persistence from string+CHECK to integer+CHECK. Keys + * must exactly match @values; values must be unique integers. The generated + * native type and wire format are UNCHANGED in every language — this is a + * persistence-layer-only concern (docs/superpowers/specs/2026-07-23-int-backed- + * enum-values-design.md). + */ +export const FIELD_ATTR_INT_VALUE_MAP = "intValueMap"; + /** * Pattern every enum member must satisfy: a legal identifier in all target * languages (TS union member, Java/C#/Python enum member). Ensures symbol == diff --git a/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts b/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts index 86a71cf16..e627cdad0 100644 --- a/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/field/field-definition.embedded.ts @@ -279,6 +279,14 @@ export const FIELD_DEFINITION: ProviderDefinition = { "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." + }, + { + "type": "attr", + "subType": "intMap", + "name": "intValueMap", + "min": 0, + "max": 1, + "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, diff --git a/server/typescript/packages/metadata/src/core/query/query-constants.ts b/server/typescript/packages/metadata/src/core/query/query-constants.ts index 3f6e2d110..8eba954b2 100644 --- a/server/typescript/packages/metadata/src/core/query/query-constants.ts +++ b/server/typescript/packages/metadata/src/core/query/query-constants.ts @@ -1,4 +1,4 @@ -import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET } from "../field/field-constants.js"; +import { FIELD_SUBTYPE_UUID, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_URI, FIELD_SUBTYPE_INET, FIELD_ATTR_INT_VALUE_MAP } from "../field/field-constants.js"; // Query concern constants — filter operators, sort order values. // @@ -67,6 +67,52 @@ export function opsForSubType(subType: string): readonly FilterOp[] { return OPS_BY_SUBTYPE[subType] ?? []; } +/** The int-backed-enum band: the enum band minus `like`. Hoisted so the narrowing + * is one named constant rather than a filter re-derived at every call. */ +const OPS_ENUM_INT_BACKED: readonly FilterOp[] = [ + FILTER_OP_EQ, FILTER_OP_NE, FILTER_OP_IN, FILTER_OP_IS_NULL, +]; + +/** + * The structural shape `opsForField` needs. Declared here rather than importing + * `MetaField`: `query-constants` is foundational and `core/field` imports it, so a + * type import back the other way would close a cycle. + */ +export interface FilterOpBandField { + readonly subType: string; + attr(name: string): unknown; +} + +/** + * The filter-operator band for a FIELD — the entry point every consumer that has a + * field in hand must use (loader validation, generated allowlists, generated client + * filter types, the cross-port `field.filter-ops` capability). + * + * Identical to {@link opsForSubType} except for ONE case: an int-backed `field.enum` + * (one declaring `@intValueMap`, design D5) persists as an INTEGER column, so `like` + * — a substring match — is dropped. `eq`/`ne`/`in` survive because the member symbol + * encodes to its integer before it reaches SQL; `like` has no such encoding, and an + * unencoded `LIKE 'DRAFT'` against an integer column is a request-time type error. + * + * `opsForSubType` cannot express this: it only ever sees the subtype `"enum"`. It is + * deliberately left unchanged for the one caller that genuinely has no field — the + * expression grammar's declared operand type. + * + * ADR-0039: the `@intValueMap` read is RESOLVING. Post-#246 the map lives on a shared + * root-level abstract declaration and consuming fields INHERIT it, so an own-only read + * would see `undefined` on exactly the shape adopters are steered toward and wrongly + * keep `like` in the band. + */ +export function opsForField(field: FilterOpBandField): readonly FilterOp[] { + if (field.subType === FIELD_SUBTYPE_ENUM && isIntBacked(field)) return OPS_ENUM_INT_BACKED; + return opsForSubType(field.subType); +} + +function isIntBacked(field: FilterOpBandField): boolean { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + return raw !== undefined && raw !== null && typeof raw === "object"; +} + // --------------------------------------------------------------------------- // Sort order values (used by @sortableDefaultOrder on fields and // @defaultSortOrder on dataGrid layouts) diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index 284b4a55c..70beb35a7 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -202,6 +202,12 @@ export const ERROR_CODES = [ // own @values would be silently dropped in codegen. Remove the own @values // to inherit the shared set, or extend a concrete (non-shared) enum instead. "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + // A field.enum carries @intValueMap together with isArray=true. Int-backing is + // a persistence-layer codec and no port implements it element-wise over an + // array column, so the combination would silently persist member SYMBOLS into + // an integer array. An array-of-enum stays string-backed: drop @intValueMap, + // or make the field scalar. + "ERR_ENUM_INT_VALUE_MAP_ARRAY", "ERR_UNKNOWN", ] as const; diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index e0c0e73f2..12ff01248 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -130,6 +130,7 @@ import { FILTER_COMPOSE_OR, FILTER_COMPOSE_AND, opsForSubType, + opsForField, } from "../core/query/query-constants.js"; // --------------------------------------------------------------------------- @@ -1684,7 +1685,9 @@ export function validateDataGridFilterValues(root: MetaData): ParseError[] { for (const f of effective.filter((c) => c.type === TYPE_FIELD)) { // ADR-0039: resolving — a concrete field may inherit @filterable via extends. if (f.attr(FIELD_ATTR_FILTERABLE) === true) { - allow.set(f.name, opsForSubType(f.subType)); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. + allow.set(f.name, opsForField(f)); } } for (const layout of effective.filter( @@ -2034,7 +2037,9 @@ export function validateProjectionFilter(root: MetaData): ParseError[] { origin !== undefined && origin.subType !== ORIGIN_SUBTYPE_PASSTHROUGH && origin.subType !== ORIGIN_SUBTYPE_COMPUTED; - fields.set(f.name, { derived, ops: opsForSubType(f.subType) }); + // opsForField, not opsForSubType — an int-backed field.enum (@intValueMap) + // stores as an integer, so `like` is not in its band. + fields.set(f.name, { derived, ops: opsForField(f) }); } checkProjectionFilterRefs(filter as Record, fields, obj.name, obj.source, errors); } diff --git a/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts new file mode 100644 index 000000000..5b6a3e568 --- /dev/null +++ b/server/typescript/packages/metadata/test/attr-schema-validate-enum-intvaluemap.test.ts @@ -0,0 +1,117 @@ +// field.enum @intValueMap — content-rule tests (Task 2 of the int-backed-enum- +// values plan). Key set must exactly match @values; values must be unique +// integers. The generic "is this an object of integers" shape check is +// IntMapAttr's job (attr subtype `intMap`, see meta-attr-int-map.test.ts); +// these tests cover the field.enum-SPECIFIC semantics layered in +// attr-schema-validate.ts Check 5b. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +async function load(json: string) { + // strict:true so an unregistered @intValueMap surfaces as ERR_UNKNOWN_ATTR + // (ADR-0023) before Step 3/4 register it — matching how the library's own + // loader and the conformance runner load. + const loader = new MetaDataLoader({ strict: true }); + return loader.load([new InMemoryStringSource(json, { id: "test.json" })]); +} + +const base = (extra: string) => `{ + "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "@values": ["DRAFT","PUBLISHED","ARCHIVED"] ${extra} } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]} +}`; + +describe("field.enum @intValueMap content rules", () => { + test("accepts a valid map — key set matches @values, unique ints", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors).toEqual([]); + }); + + test("field.enum with no @intValueMap is still valid (string-backed default)", async () => { + const result = await load(base("")); + expect(result.errors).toEqual([]); + }); + + test("rejects a missing member key", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("ARCHIVED"); + }); + + test("rejects an extra key not in @values", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9, "RETRACTED": 12}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("RETRACTED"); + }); + + test("rejects a non-integer value", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": "zero", "PUBLISHED": 5, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + }); + + test("rejects two members sharing the same int", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 0, "ARCHIVED": 9}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("DRAFT"); + expect(result.errors[0]?.message).toContain("PUBLISHED"); + }); + + // Final-review fix: the eventual DB column for an int-backed enum is a + // 32-bit Postgres/SQLite `integer` (design doc D5) — a value outside that + // range can never actually be persisted, so it must be rejected at load + // time rather than silently accepted (TS numbers have no fixed width). + // Mirrors Java's IntMapAttribute#setValueAsString bound check. + test("rejects a value outside the 32-bit signed integer range", async () => { + const result = await load(base(', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9999999999}')); + expect(result.errors.length).toBeGreaterThan(0); + expect((result.errors[0] as { code?: string })?.code).toBe("ERR_BAD_ATTR_VALUE"); + expect(result.errors[0]?.message).toContain("ARCHIVED"); + }); +}); + +// Design D7, narrowed: int-backing is scalar-only. No port implements the codec +// element-wise over an array column, and two ports that happen to compose are +// not a feature — so the combination is rejected at LOAD, in every port. +describe("field.enum @intValueMap is scalar-only", () => { + const MAP = ', "@intValueMap": {"DRAFT": 0, "PUBLISHED": 5, "ARCHIVED": 9}'; + + test("rejects @intValueMap on an isArray field", async () => { + const result = await load(base(`, "isArray": true${MAP}`)); + const codes = result.errors.map((e) => (e as { code?: string })?.code); + expect(codes).toContain("ERR_ENUM_INT_VALUE_MAP_ARRAY"); + }); + + test("an array enum with no @intValueMap stays valid (string-backed)", async () => { + const result = await load(base(', "isArray": true')); + expect(result.errors).toEqual([]); + }); + + // The two halves land on DIFFERENT nodes on the canonical authoring shape: + // #246 forces @intValueMap onto the shared abstract declaration, and isArray + // is declared by the consuming field. An own-only read would never see both. + test("rejects an inherited @intValueMap combined with a locally-declared isArray", async () => { + const result = await load(`{ + "metadata.root": { "package": "acme", "children": [ + { "field.enum": { "name": "Status", "abstract": true, + "@values": ["DRAFT","PUBLISHED","ARCHIVED"]${MAP} } }, + { "object.entity": { "name": "Order", "children": [ + { "field.long": { "name": "id" } }, + { "field.enum": { "name": "status", "extends": "Status", "isArray": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]} + }`); + const codes = result.errors.map((e) => (e as { code?: string })?.code); + expect(codes).toContain("ERR_ENUM_INT_VALUE_MAP_ARRAY"); + }); +}); diff --git a/server/typescript/packages/metadata/test/conformance/binding.ts b/server/typescript/packages/metadata/test/conformance/binding.ts index 343587149..dd8098e91 100644 --- a/server/typescript/packages/metadata/test/conformance/binding.ts +++ b/server/typescript/packages/metadata/test/conformance/binding.ts @@ -12,7 +12,7 @@ import type { MetaData } from "../../src/shared/meta-data.js"; import { MetaObject } from "../../src/core/object/meta-object.js"; import { MetaField } from "../../src/core/field/meta-field.js"; import { canonicalSerialize } from "../../src/serializer-json.js"; -import { opsForSubType } from "../../src/core/query/query-constants.js"; +import { opsForField } from "../../src/core/query/query-constants.js"; type CapabilityArgs = Record; type CapabilityFn = (node: MetaData, args: CapabilityArgs) => NormalizedResult; @@ -85,12 +85,14 @@ export const binding: Readonly> = { "effective-tree": canonicalSerialize(asField(node)), }), - // field.filter-ops → the canonical per-subtype filter-operator band - // (query-constants OPS_BY_SUBTYPE). Returns `{ names: [...] }` in canonical - // operator order so the cross-port matrix fixture compares order-sensitively. - // The single source of truth for the band is opsForSubType — the same map - // the server allowlist + codegen consume. + // field.filter-ops → the canonical per-FIELD filter-operator band. Returns + // `{ names: [...] }` in canonical operator order so the cross-port matrix + // fixture compares order-sensitively. The single source of truth for the band + // is opsForField — the same function the server allowlist + codegen consume. + // + // opsForField, not opsForSubType: the band is field-level because an int-backed + // field.enum (@intValueMap) stores as an integer and so drops `like`. "field.filter-ops": (node) => ({ - names: [...opsForSubType(asField(node).subType)], + names: [...opsForField(asField(node))], }), }; diff --git a/server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts b/server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts new file mode 100644 index 000000000..b037a9ad8 --- /dev/null +++ b/server/typescript/packages/metadata/test/core/attr/meta-attr-int-map.test.ts @@ -0,0 +1,47 @@ +// attr.intMap — a generic object-shaped attr whose values must all be +// integers (e.g. field.enum's future @intValueMap). Shape-only validation; +// a consumer's own semantic rules (key-set membership, uniqueness) are +// layered by that consumer, not here. + +import { describe, test, expect } from "bun:test"; +import { TypeId } from "../../../src/registry.js"; +import { IntMapAttr } from "../../../src/core/attr/meta-attr-int-map.js"; +import { TYPE_ATTR, ATTR_SUBTYPE_INT_MAP } from "../../../src/index.js"; +import { type AttrValue } from "../../../src/shared/meta-data.js"; + +function intMapAttr(name = "intValueMap"): IntMapAttr { + return new IntMapAttr(new TypeId(TYPE_ATTR, ATTR_SUBTYPE_INT_MAP), name); +} + +describe("IntMapAttr", () => { + test("accepts a plain object with integer values", () => { + const attr = intMapAttr(); + expect(attr.validateValue({ DRAFT: 0, PUBLISHED: 5 })).toEqual([]); + }); + + test("rejects a non-object value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue("not-an-object" as unknown as AttrValue); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("must be of type 'intMap'"); + }); + + test("rejects an array value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue([0, 1] as unknown as AttrValue); + expect(errors.length).toBe(1); + }); + + test("rejects a non-integer value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue({ DRAFT: "0" }); + expect(errors.length).toBe(1); + expect(errors[0]?.message).toContain("DRAFT"); + }); + + test("rejects a float value", () => { + const attr = intMapAttr(); + const errors = attr.validateValue({ DRAFT: 0.5 }); + expect(errors.length).toBe(1); + }); +}); diff --git a/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts b/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts index 40d6623c3..bfbf166a9 100644 --- a/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts +++ b/server/typescript/packages/metadata/test/enum-extends-values-conflict.test.ts @@ -135,3 +135,121 @@ describe("field.enum — extends a shared abstract enum AND declares own @values expect(conflictErrors).toHaveLength(0); }); }); + +// The int-backed twin of the rule above. @intValueMap is @values' numeric half — +// the symbol→int mapping belongs to the enum VOCABULARY, not to one column that +// uses it. A shared enum is materialized ONCE as a single named type, so a +// per-field map would give one logical type N storage encodings (and, in ports +// that emit per-TYPE codec artifacts, two same-named declarations). +describe("field.enum — extends a shared abstract enum AND declares own @intValueMap", () => { + const sharedDecl = { + "field.enum": { + name: "Status", + abstract: true, + "@values": ["A", "B"], + }, + }; + + function conflicts(errors: unknown[]) { + return errors.filter( + (e) => (e as { code?: string }).code === "ERR_ENUM_EXTENDS_VALUES_CONFLICT", + ); + } + + it("emits ERR_ENUM_EXTENDS_VALUES_CONFLICT when the consuming field owns @intValueMap", async () => { + const { errors } = await load({ + "metadata.root": { + package: "acme", + children: [ + sharedDecl, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { + "field.enum": { + name: "status", + extends: "acme::Status", + "@intValueMap": { A: 0, B: 1 }, + }, + }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + ], + }, + }); + + expect(conflicts(errors)).toHaveLength(1); + }); + + it("does NOT emit the conflict when @intValueMap sits on the shared declaration and the field inherits it", async () => { + const { errors } = await load({ + "metadata.root": { + package: "acme", + children: [ + { + "field.enum": { + name: "Status", + abstract: true, + "@values": ["A", "B"], + "@intValueMap": { A: 0, B: 1 }, + }, + }, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": { name: "status", extends: "acme::Status" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + ], + }, + }); + + expect(conflicts(errors)).toHaveLength(0); + expect(errors).toHaveLength(0); + }); + + it("does NOT emit the conflict for an ABSTRACT but NON-ROOT super (nested inside an object), where a per-field map stays legal", async () => { + const { errors } = await load({ + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "Container", + abstract: true, + children: [ + { "field.enum": { name: "kind", abstract: true, "@values": ["A", "B"] } }, + ], + }, + }, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { + "field.enum": { + name: "status", + extends: "acme::Container.kind", + "@intValueMap": { A: 0, B: 1 }, + }, + }, + { "identity.primary": { name: "id", "@fields": "id" } }, + ], + }, + }, + ], + }, + }); + + expect(conflicts(errors)).toHaveLength(0); + }); +}); diff --git a/server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts b/server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts new file mode 100644 index 000000000..077bc721e --- /dev/null +++ b/server/typescript/packages/metadata/test/enum-int-backed-filter-ops.test.ts @@ -0,0 +1,146 @@ +// An INT-BACKED field.enum (@intValueMap, design D5) persists as an INTEGER column. +// `like` is a substring match — meaningless against an integer, and it is the one +// operator in the string/enum band that cannot be made to work by encoding the +// member symbol to its integer (eq/ne/in all can, and do). +// +// So the op band is a property of the FIELD, not of the subtype alone: a +// string-backed enum keeps `like`, an int-backed one does not. `opsForSubType` +// cannot express that — it only ever sees "enum". `opsForField` is the field-level +// entry point every op-band consumer must use when it has a field in hand. +// +// Cross-port: fixtures/conformance/filter-ops-matrix pins fEnum vs fEnumInt in all +// five ports, so this narrowing can never become a TS-only divergence. + +import { describe, test, expect } from "bun:test"; +import { + MetaDataLoader, + InMemoryStringSource, + opsForField, + opsForSubType, + FIELD_SUBTYPE_ENUM, +} from "../src/index.js"; +import type { MetaField } from "../src/index.js"; + +async function loadMatrix(): Promise> { + const json = JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { + "object.entity": { + name: "Matrix", + children: [ + { "source.rdb": { "@table": "matrix" } }, + { "field.long": { name: "id" } }, + // String-backed enum — the control. Keeps `like`. + { + "field.enum": { + name: "strEnum", + "@values": ["DRAFT", "PUBLISHED"], + "@filterable": true, + }, + }, + // Int-backed enum — declares @intValueMap, so it stores as integer. + { + "field.enum": { + name: "intEnum", + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5 }, + "@filterable": true, + }, + }, + { "field.string": { name: "title", "@filterable": true } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(errors).toEqual([]); + const entity = root.objects()[0]!; + const out = new Map(); + for (const f of entity.fields()) out.set(f.name, f); + return out; +} + +describe("opsForField — int-backed enum drops `like`", () => { + test("a string-backed enum keeps the full string band", async () => { + const fields = await loadMatrix(); + expect([...opsForField(fields.get("strEnum")!)]).toEqual([ + "eq", + "ne", + "in", + "like", + "isNull", + ]); + }); + + test("an int-backed enum drops `like`, keeping eq/ne/in/isNull", async () => { + const fields = await loadMatrix(); + expect([...opsForField(fields.get("intEnum")!)]).toEqual(["eq", "ne", "in", "isNull"]); + }); + + test("a plain string is untouched by the narrowing", async () => { + const fields = await loadMatrix(); + expect([...opsForField(fields.get("title")!)]).toEqual( + [...opsForSubType("string")], + ); + }); + + test("opsForSubType(enum) is unchanged — the narrowing is field-level only", () => { + // The subtype-keyed band stays the string band. Callers that only have a + // subtype string (the expression grammar's declared operand type) are + // deliberately unaffected. + expect([...opsForSubType(FIELD_SUBTYPE_ENUM)]).toEqual([ + "eq", + "ne", + "in", + "like", + "isNull", + ]); + }); + + test("the map is read RESOLVING — an inherited @intValueMap narrows too", async () => { + // Post-#246 this is the CANONICAL authoring shape: the map lives on the shared + // root-level abstract declaration and consuming fields inherit it. An own-only + // read would see undefined here and wrongly keep `like`. + const json = JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { + "field.enum": { + name: "SharedStatus", + abstract: true, + "@values": ["DRAFT", "PUBLISHED"], + "@intValueMap": { DRAFT: 0, PUBLISHED: 5 }, + }, + }, + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { name: "id" } }, + { + "field.enum": { + name: "status", + extends: "acme::SharedStatus", + "@filterable": true, + }, + }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + expect(errors).toEqual([]); + const status = root.objects()[0]!.fields().find((f) => f.name === "status")!; + expect([...opsForField(status)]).toEqual(["eq", "ne", "in", "isNull"]); + }); +}); diff --git a/server/typescript/packages/metadata/test/field-definition-completeness.test.ts b/server/typescript/packages/metadata/test/field-definition-completeness.test.ts index 5b8726aae..e64d08c24 100644 --- a/server/typescript/packages/metadata/test/field-definition-completeness.test.ts +++ b/server/typescript/packages/metadata/test/field-definition-completeness.test.ts @@ -12,7 +12,7 @@ // decimal → @precision, @scale // object → @objectRef // currency → @currency -// enum → @values, @provided +// enum → @values, @provided, @intValueMap // int/long/double/float/boolean/date/time/timestamp/uuid → none // - the "any attr" wildcard is gone; a misplaced attr is now ERR_UNKNOWN_ATTR. // @@ -103,6 +103,7 @@ const MAP_EXTRA: Record = { const ENUM_CORE_EXTRA: Record = { values: { valueType: "string", required: true }, provided: { valueType: "boolean", required: false }, + intValueMap: { valueType: "intMap", required: false }, }; /** Scoped concern attrs (db / prompt). */ diff --git a/server/typescript/packages/migrate-ts/src/diff/index.ts b/server/typescript/packages/migrate-ts/src/diff/index.ts index 3d9b06a43..4da7ea67f 100644 --- a/server/typescript/packages/migrate-ts/src/diff/index.ts +++ b/server/typescript/packages/migrate-ts/src/diff/index.ts @@ -148,9 +148,16 @@ export async function diff( // schemas it mentions), else null = no scoping (empty model → prior whole-DB // behavior). A table outside the scope is excluded from both sides, so a // co-located schema owned by another app is neither dropped nor reported. - const declaredSchemas = new Set( - args.expected.tables.map((t) => t.schema ?? DEFAULT_DB_SCHEMA_POSTGRES), - ); + const declaredSchemas = new Set([ + ...args.expected.tables.map((t) => t.schema ?? DEFAULT_DB_SCHEMA_POSTGRES), + // A model that declares views in a schema with no table of its own (e.g. an + // API/read-model schema like `p3_api` sitting alongside an all-`public` + // entity model) must still bring that schema into scope — otherwise its + // views are silently excluded from BOTH sides of the diff (never compared, + // so real drift in an opaque @sql body or a genuine missing/extra view goes + // undetected) rather than gated on it as an owned schema. + ...args.expected.views.map((v) => v.schema ?? DEFAULT_DB_SCHEMA_POSTGRES), + ]); const scopeSchemas: Set | null = args.scopeSchemas !== undefined ? new Set(args.scopeSchemas) diff --git a/server/typescript/packages/migrate-ts/src/expected-schema.ts b/server/typescript/packages/migrate-ts/src/expected-schema.ts index b79019ac0..fe25f1cbb 100644 --- a/server/typescript/packages/migrate-ts/src/expected-schema.ts +++ b/server/typescript/packages/migrate-ts/src/expected-schema.ts @@ -43,6 +43,7 @@ import { FIELD_SUBTYPE_INET, FIELD_SUBTYPE_ENUM, FIELD_ATTR_VALUES, + FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STORAGE, FIELD_ATTR_DB_COLUMN_TYPE, @@ -698,12 +699,35 @@ function buildChecks( if (field.resolvedIsArray()) continue; const col = resolveColumnName(field, strategy); const qcol = quoteCheckCol(col); - // Enum membership check. + // Enum membership check. An INT-BACKED enum (@intValueMap, design D5) stores + // the mapped integers, so the CHECK lists those integers unquoted rather than + // the member strings — `IN (0, 5, 9)`, not `IN ('DRAFT', …)`. The members are + // still the SSOT: the integers are read THROUGH the map, keyed by member, so a + // member with no mapping cannot silently vanish from the constraint. if (field.subType === FIELD_SUBTYPE_ENUM) { const raw = field.attr(FIELD_ATTR_VALUES); if (Array.isArray(raw) && raw.length > 0) { const values = raw.map((v) => String(v)); - const expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; + const intMap = intValueMapOf(field); + let expression: string; + if (intMap !== undefined) { + // The loader pins key-set-equals-@values (Check 5b) in every port, so every + // member resolves. Guard anyway: emitting a partial IN list would silently + // reject rows the model considers valid. + const ints = values.map((v) => { + const n = intMap[v]; + if (typeof n !== "number") { + throw new Error( + `field.enum '${field.name}' @intValueMap has no integer for member '${v}' — ` + + `cannot build the CHECK constraint for column '${col}'.`, + ); + } + return String(n); + }); + expression = `${qcol} IN (${ints.join(", ")})`; + } else { + expression = `${qcol} IN (${values.map((v) => `'${v.replace(/'/g, "''")}'`).join(", ")})`; + } checks.push({ name: `${tableName}_${col}_chk`, expression }); } } @@ -842,11 +866,27 @@ function buildForeignKeys( // Target columns: prefer explicit multi-field dotted form, else delegate // to MetaReferenceIdentity.resolvedTargetPkField (single field → target's - // primary identity → "id" fallback). + // primary identity → "id" fallback). Each target FIELD name must resolve to + // its PHYSICAL column via the target entity's own @column override (e.g. a + // PK field `id` with `@column: "Id"`), exactly like fkCols above — the raw + // naming strategy alone would emit the logical name and phantom-diff every + // FK into that table (expected ["id"] vs actual ["Id"]). + // targetEntity may be package-qualified (FQN); findObject is keyed by bare + // name — same fallback as resolvedTargetPkField/resolveTargetTable. + const targetObj = root.findObject(targetEntity) + ?? (targetEntity.includes("::") + ? root.findObject(targetEntity.slice(targetEntity.lastIndexOf("::") + 2)) + : undefined); const explicitTargetFields = refChild.targetFields; - const refColumns = explicitTargetFields.length > 1 - ? explicitTargetFields.map((n) => applyColumnNamingStrategy(n, strategy)) - : [applyColumnNamingStrategy(refChild.resolvedTargetPkField(root) ?? "id", strategy)]; + const targetFieldNames = explicitTargetFields.length > 1 + ? explicitTargetFields + : [refChild.resolvedTargetPkField(root) ?? "id"]; + const refColumns = targetFieldNames.map((jsName) => { + const targetField = targetObj ? findField(targetObj, jsName) : undefined; + return targetField + ? resolveColumnName(targetField, strategy) + : applyColumnNamingStrategy(jsName, strategy); + }); const { onDelete, onUpdate } = resolveReferentialActions(entity, refChild); // An explicit @constraintName adopts an existing FK name (e.g. a database @@ -925,12 +965,29 @@ function buildColumn( }; if (typeof defaultRaw === "string") { - // #235: an EMPTY-string default (`@default: ""`) is a real literal default — - // codegen emits `.default("")` and the DB gets `DEFAULT ''`, so dropping it here - // (a falsy `.length > 0` check) made the column drift forever on sqlite/d1 and - // disagree with codegen. Keep it as a literal; only `undefined` means "no default". - const isExpr = defaultRaw.length > 0 && EXPR_DEFAULT_PATTERNS.some((re) => re.test(defaultRaw)); - col.default = { kind: isExpr ? "expr" : "literal", value: defaultRaw }; + // An INT-BACKED enum's @default names a MEMBER SYMBOL, but the column holds the + // mapped integer — emitting `DEFAULT 'DRAFT'` on an `integer` column is + // un-appliable DDL. Lower it through the map. The member is already validated + // against @values by the loader (FR-011 Check 5), so a miss here is unreachable; + // throw rather than silently emit the symbol, which would fail only at apply time. + const enumIntMap = field.subType === FIELD_SUBTYPE_ENUM ? intValueMapOf(field) : undefined; + if (enumIntMap !== undefined) { + const mapped = enumIntMap[defaultRaw]; + if (typeof mapped !== "number") { + throw new Error( + `field.enum '${field.name}' @default '${defaultRaw}' has no entry in @intValueMap — ` + + `cannot lower the column default for '${col.name}'.`, + ); + } + col.default = { kind: "literal", value: String(mapped) }; + } else { + // #235: an EMPTY-string default (`@default: ""`) is a real literal default — + // codegen emits `.default("")` and the DB gets `DEFAULT ''`, so dropping it here + // (a falsy `.length > 0` check) made the column drift forever on sqlite/d1 and + // disagree with codegen. Keep it as a literal; only `undefined` means "no default". + const isExpr = defaultRaw.length > 0 && EXPR_DEFAULT_PATTERNS.some((re) => re.test(defaultRaw)); + col.default = { kind: isExpr ? "expr" : "literal", value: defaultRaw }; + } } else if (typeof defaultRaw === "boolean" || typeof defaultRaw === "number") { col.default = { kind: "literal", value: String(defaultRaw) }; } else { @@ -978,8 +1035,10 @@ function buildColumn( */ function arrayElementSqlType(field: MetaData): SqlType | undefined { switch (field.subType) { + // enum[] stores as text[] — membership is app-level (no CHECK — see buildChecks). + // An INT-BACKED enum[] (@intValueMap, design D7) stores as integer[] instead. + case FIELD_SUBTYPE_ENUM: return isIntBackedEnum(field) ? { kind: "integer", bits: 32 } : { kind: "text" }; case FIELD_SUBTYPE_STRING: - case FIELD_SUBTYPE_ENUM: // enum[] stores as text[]; membership is app-level (no CHECK — see buildChecks) case FIELD_SUBTYPE_URI: return { kind: "text" }; case FIELD_SUBTYPE_UUID: return { kind: "uuid" }; case FIELD_SUBTYPE_INT: return { kind: "integer", bits: 32 }; @@ -1078,7 +1137,33 @@ function subtypeToSqlType(field: MetaData): SqlType { // stores as text (the native inet column would reject a not-strictly-valid // value at INSERT). ADR-0039: resolving — @lenient may be inherited via extends. case FIELD_SUBTYPE_INET: return field.attr(FIELD_ATTR_LENIENT) === true ? { kind: "text" } : { kind: "inet" }; + // A string-backed field.enum is a text column with a membership CHECK; an + // INT-BACKED one (@intValueMap, design D5) stores the mapped integer instead. + // The TS/wire type is the member string either way — only the column differs. + case FIELD_SUBTYPE_ENUM: return isIntBackedEnum(field) ? { kind: "integer", bits: 32 } : { kind: "text" }; default: return { kind: "text" }; // unknown → text fallback } } +/** + * True when this `field.enum` persists as an integer — i.e. it carries an + * `@intValueMap` (design D5). + * + * ADR-0039: RESOLVING (`attr`, not `ownAttr`). Post-#246 a shared (root-level + * abstract) enum OWNS the map and consuming fields inherit it — declaring an own + * `@intValueMap` against a shared super is `ERR_ENUM_EXTENDS_VALUES_CONFLICT`. So + * the inherited case is not an edge case, it is the CANONICAL authoring shape, and + * an own-only read here would emit a `text` column for an integer-encoded value on + * every consuming field of every shared enum. + */ +function isIntBackedEnum(field: MetaData): boolean { + return intValueMapOf(field) !== undefined; +} + +/** The resolved `@intValueMap` as a plain record, or undefined when absent. */ +export function intValueMapOf(field: MetaData): Record | undefined { + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return raw as Record; +} + diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts new file mode 100644 index 000000000..b950f3de5 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/expected-schema-enum-intvaluemap.test.ts @@ -0,0 +1,253 @@ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../src/expected-schema.js"; +import { diff } from "../src/diff/index.js"; + +// Int-backed field.enum (@intValueMap, design D5): the column is `integer`, not +// `text`. The wire/TS type is unchanged — only the physical column differs. + +async function loadJson(json: string): Promise { + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error(`fixture failed to load: ${result.errors.map((e) => String(e)).join("; ")}`); + } + return result.root; +} + +function entityModel(statusField: Record, extraRoots: unknown[] = []): string { + return JSON.stringify({ + "metadata.root": { + children: [ + ...extraRoots, + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.enum": statusField }, + { "source.rdb": { name: "src", "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); +} + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +async function statusColumn(json: string) { + const snapshot = buildExpectedSchema(await loadJson(json)); + const table = snapshot.tables.find((t) => t.name === "orders")!; + return table.columns.find((c) => c.name === "status")!; +} + +describe("buildExpectedSchema — int-backed field.enum (@intValueMap)", () => { + test("scalar int-backed enum maps to integer, not text", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }), + ); + expect(col.sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("string-backed enum (no @intValueMap) is unchanged", async () => { + const col = await statusColumn(entityModel({ name: "status", "@values": VALUES })); + expect(col.sqlType).toEqual({ kind: "text" }); + }); + + // D7, narrowed: int-backing is scalar-only — @intValueMap with isArray is + // ERR_ENUM_INT_VALUE_MAP_ARRAY at LOAD, so no int-backed array column shape exists + // to assert. The rejection is gated by the loader tests + the cross-port + // error-enum-intvaluemap-array fixtures. + test("array-of-enum string-backed stays text[]", async () => { + const col = await statusColumn(entityModel({ name: "status", isArray: true, "@values": VALUES })); + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "text" } }); + }); + + // Amendment 1 / #246: post-#246 the map CANNOT live on the consuming field when + // the field extends a shared (root-level abstract) enum — it lives on the SHARED + // DECLARATION and is inherited. An own-only read here would see undefined and + // silently emit a text column for an integer-encoded value. This is the shape + // real adopters will author, so it is the one that most needs pinning. + test("map inherited from a SHARED abstract declaration still yields integer", async () => { + const col = await statusColumn( + entityModel({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]), + ); + expect(col.sqlType).toEqual({ kind: "integer", bits: 32 }); + }); + + test("the membership CHECK lists the mapped INTEGERS, unquoted", async () => { + const snapshot = buildExpectedSchema( + await loadJson(entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP })), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + const chk = table.checks!.find((c) => c.name === "orders_status_chk")!; + expect(chk.expression).toBe(`"status" IN (0, 5, 9)`); + }); + + test("string-backed enum keeps its quoted-string CHECK", async () => { + const snapshot = buildExpectedSchema( + await loadJson(entityModel({ name: "status", "@values": VALUES })), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + const chk = table.checks!.find((c) => c.name === "orders_status_chk")!; + expect(chk.expression).toBe(`"status" IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')`); + }); + + test("an int-backed enum inheriting its map from a shared decl gets the integer CHECK", async () => { + const snapshot = buildExpectedSchema( + await loadJson( + entityModel({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, "@values": VALUES, "@intValueMap": INT_MAP } }, + ]), + ), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + const chk = table.checks!.find((c) => c.name === "orders_status_chk")!; + expect(chk.expression).toBe(`"status" IN (0, 5, 9)`); + }); + + test("array-of-enum still gets NO field-level CHECK (membership stays app-level)", async () => { + const snapshot = buildExpectedSchema( + await loadJson(entityModel({ name: "status", isArray: true, "@values": VALUES })), + ); + const table = snapshot.tables.find((t) => t.name === "orders")!; + expect((table.checks ?? []).find((c) => c.name === "orders_status_chk")).toBeUndefined(); + }); + + // Task 7 — @default names a MEMBER, the column holds the mapped INT. Emitting + // DEFAULT 'DRAFT' on an integer column is un-appliable DDL. + test("@default lowers to the mapped integer literal", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP, "@default": "PUBLISHED" }), + ); + expect(col.default).toEqual({ kind: "literal", value: "5" }); + }); + + test("@default on a string-backed enum is unchanged", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@default": "PUBLISHED" }), + ); + expect(col.default).toEqual({ kind: "literal", value: "PUBLISHED" }); + }); + + test("a zero-valued member default survives (0 is falsy — must not be dropped)", async () => { + const col = await statusColumn( + entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP, "@default": "DRAFT" }), + ); + expect(col.default).toEqual({ kind: "literal", value: "0" }); + }); + + // Array-ness inherited from a shared declaration still resolves here. This + // replaced a test that inherited array-ness AND @intValueMap together, which is + // now ERR_ENUM_INT_VALUE_MAP_ARRAY at load (D7, narrowed) — that rejection is + // gated by the loader tests and the error-enum-intvaluemap-array-inherited + // fixture. What remains worth pinning at THIS layer is the resolving read of + // array-ness itself, since an own-only read would emit a scalar column. + test("array-ness inherited from the shared declaration still yields an array column", async () => { + const col = await statusColumn( + entityModel({ name: "status", extends: "Status" }, [ + { "field.enum": { name: "Status", abstract: true, isArray: true, "@values": VALUES } }, + ]), + ); + expect(col.sqlType).toEqual({ kind: "array", element: { kind: "text" } }); + }); +}); + +// Task 3 — the D8 migration-safety guard needs NO new gating code. Toggling +// @intValueMap on a field that already has a column is a text<->integer +// change-column-type, and `isWidening` returns false for ANY cross-kind pair +// (sql-type.ts: `if (from.kind !== to.kind) return false`), so +// `blockedReasonFor`'s change-column-type branch blocks it unless allow.typeChange +// is passed. This proves that end-to-end through the real metadata path rather +// than asserting it about hand-built snapshots. +describe("int-backed enum — backing-mode change is blocked by the existing guard", () => { + const STRING_BACKED = entityModel({ name: "status", "@values": VALUES }); + const INT_BACKED = entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP }); + + async function backingModeDiff(expected: string, actual: string, allowTypeChange = false) { + const e = buildExpectedSchema(await loadJson(expected)); + const a = buildExpectedSchema(await loadJson(actual)); + return diff(e, a, allowTypeChange ? { allow: { typeChange: true } } : undefined); + } + + test("ADDING @intValueMap (text → integer) is blocked without allow.typeChange", async () => { + const r = await backingModeDiff(INT_BACKED, STRING_BACKED); + const change = r.changes.find((c) => c.kind === "change-column-type"); + expect(change).toBeDefined(); + expect(change!.status.state).toBe("blocked"); + expect(r.blocked).toContain(change!); + }); + + test("REMOVING @intValueMap (integer → text) is blocked too", async () => { + const r = await backingModeDiff(STRING_BACKED, INT_BACKED); + const change = r.changes.find((c) => c.kind === "change-column-type"); + expect(change).toBeDefined(); + expect(change!.status.state).toBe("blocked"); + }); + + test("an explicit allow.typeChange unblocks it (the documented manual-recast path)", async () => { + const r = await backingModeDiff(INT_BACKED, STRING_BACKED, true); + const change = r.changes.find((c) => c.kind === "change-column-type"); + expect(change!.status.state).toBe("allowed"); + expect(r.blocked).toHaveLength(0); + }); + + test("no backing-mode change → no change-column-type at all", async () => { + const r = await backingModeDiff(INT_BACKED, INT_BACKED); + expect(r.changes.find((c) => c.kind === "change-column-type")).toBeUndefined(); + }); +}); + +// RE-mapping a member inside a map that stays present is a different animal from D8's +// add/remove, and nothing in the pipeline understands it: the column holds bare integers, +// and neither introspection nor the committed snapshot records which member an integer +// stood for. What currently protects the common case is an ACCIDENT — dropping a CHECK is +// destructive, so the remap trips `allow.dropCheck` on its way past. These tests pin that +// accident (so it cannot be relaxed silently) and pin the one shape it does not cover. +describe("int-backed enum — re-mapping a member's integer", () => { + const MAP_SWAPPED = { DRAFT: 5, PUBLISHED: 0, ARCHIVED: 9 }; + const MAP_MOVED = { DRAFT: 1, PUBLISHED: 5, ARCHIVED: 9 }; + + async function remapDiff(afterMap: Record, afterValues = VALUES) { + const before = buildExpectedSchema( + await loadJson(entityModel({ name: "status", "@values": VALUES, "@intValueMap": INT_MAP })), + ); + const after = buildExpectedSchema( + await loadJson( + entityModel({ name: "status", "@values": afterValues, "@intValueMap": afterMap }), + ), + ); + return diff(after, before, { dialect: "postgres" }); + } + + // The CHECK list renders in @values order, so ANY remap changes its text — which is the + // only reason migrate sees a remap at all. + test("a swap trips the blocked drop-check rather than passing silently", async () => { + const r = await remapDiff(MAP_SWAPPED); + const drop = r.changes.find((c) => c.kind === "drop-check"); + expect(drop).toBeDefined(); + expect(drop!.status.state).toBe("blocked"); + expect(r.changes.find((c) => c.kind === "add-check")).toBeDefined(); + }); + + test("moving a member to an unused int behaves the same way", async () => { + const r = await remapDiff(MAP_MOVED); + expect(r.changes.find((c) => c.kind === "drop-check")!.status.state).toBe("blocked"); + }); + + // The gap the two tests above do NOT cover: reorder @values to compensate for the swap + // and the rendered CHECK is byte-identical, so there is no diff to block. Every stored + // row changes meaning with no migration emitted at all. Pinned as KNOWN, not as desired + // — if a future change makes this produce a diff, this test should be updated, not the + // behaviour reverted. + test("KNOWN GAP: a compensating @values reorder makes the remap invisible to the diff", async () => { + const r = await remapDiff(MAP_SWAPPED, ["PUBLISHED", "DRAFT", "ARCHIVED"]); + expect(r.changes).toEqual([]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts new file mode 100644 index 000000000..617307d71 --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/expected-schema-fk-refcolumn-override.test.ts @@ -0,0 +1,76 @@ +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../src/expected-schema.js"; + +// Regression: an FK's refColumns must resolve the TARGET entity's physical +// @column override, not the raw field name. An adopted database (e.g. an EF +// Core schema) commonly has a PK field `id` stored as column "Id"; before the +// fix the expected side emitted refColumns ["id"] while introspection read +// ["Id"], phantom-diffing every FK into that table as drop-fk + add-fk. + +async function loadJson(json: string): Promise { + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + return result.root; +} + +function model(clientIdField: Record): string { + return JSON.stringify({ + "metadata.root": { + "children": [ + { + "object.entity": { + "name": "Client", + "children": [ + { "field.uuid": clientIdField }, + { "source.rdb": { "name": "src", "@table": "Clients" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.entity": { + "name": "Patient", + "children": [ + { "field.uuid": { "name": "id" } }, + { "field.uuid": { "name": "clientId", "@column": "ClientId" } }, + { "source.rdb": { "name": "src", "@table": "Patients" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { + "identity.reference": { + "name": "fk_client", + "@fields": ["clientId"], + "@references": "Client", + }, + }, + ], + }, + }, + ], + }, + }); +} + +function patientFk(root: MetaData) { + const snapshot = buildExpectedSchema(root, { dialect: "postgres" }); + const table = snapshot.tables.find((t) => t.name === "Patients"); + expect(table).toBeDefined(); + expect(table!.foreignKeys).toHaveLength(1); + return table!.foreignKeys[0]!; +} + +describe("buildExpectedSchema — FK refColumns resolve the target PK's @column override", () => { + test("target PK field with @column override → refColumns uses the physical name", async () => { + const root = await loadJson(model({ "name": "id", "@column": "Id" })); + const fk = patientFk(root); + expect(fk.columns).toEqual(["ClientId"]); + expect(fk.refTable).toBe("Clients"); + expect(fk.refColumns).toEqual(["Id"]); + }); + + test("target PK field without @column override → refColumns keeps the field name", async () => { + const root = await loadJson(model({ "name": "id" })); + const fk = patientFk(root); + expect(fk.refColumns).toEqual(["id"]); + }); +}); diff --git a/server/typescript/packages/runtime-ts/src/query-builder.ts b/server/typescript/packages/runtime-ts/src/query-builder.ts index 2562e0e0c..8650316ff 100644 --- a/server/typescript/packages/runtime-ts/src/query-builder.ts +++ b/server/typescript/packages/runtime-ts/src/query-builder.ts @@ -7,6 +7,7 @@ import { resolveTableName, resolveColumnName, } from "@metaobjectsdev/metadata"; import { MetadataError } from "./errors.js"; +import { intValueMapOf } from "./type-coercer.js"; import type { WhereClause, OrderBy, PrimitiveValue, Row, SelectSpec, InsertSpec, UpdateSpec, DeleteSpec, CountSpec, @@ -118,12 +119,20 @@ function compileEntry( const field = getField(entity, fieldName); const column = resolveColumnName(field, strategy); + // An int-backed field.enum (@intValueMap) stores an INTEGER while its filter value + // is the member SYMBOL, so every comparison value must be encoded before it is + // bound — otherwise Postgres rejects the statement outright ("invalid input syntax + // for type integer"). Applied at this ONE seam because every operator arrives here + // with the field in hand; `like` is not reachable for such a field (the loader's + // field-level operator band drops it), and `isNull` carries no value to encode. + const enc = (v: T): T => encodeFilterValue(field, v); + if (value === null) return { kind: "isNull", column, not: false }; if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") { - return { kind: "eq", column, value }; + return { kind: "eq", column, value: enc(value) }; } if (Array.isArray(value)) { - return { kind: "in", column, values: value as PrimitiveValue[] }; + return { kind: "in", column, values: (value as PrimitiveValue[]).map(enc) }; } // If multiple operators are set on one field ({ $gte: 5, $lte: 10 }), the FIRST match wins @@ -131,24 +140,39 @@ function compileEntry( const op = value; if ("$eq" in op && op.$eq !== undefined) { if (op.$eq === null) return { kind: "isNull", column, not: false }; - return { kind: "eq", column, value: op.$eq }; + return { kind: "eq", column, value: enc(op.$eq) }; } if ("$ne" in op && op.$ne !== undefined) { if (op.$ne === null) return { kind: "isNull", column, not: true }; - return { kind: "ne", column, value: op.$ne }; + return { kind: "ne", column, value: enc(op.$ne) }; } - if ("$gt" in op && op.$gt !== undefined) return { kind: "gt", column, value: op.$gt }; - if ("$gte" in op && op.$gte !== undefined) return { kind: "gte", column, value: op.$gte }; - if ("$lt" in op && op.$lt !== undefined) return { kind: "lt", column, value: op.$lt }; - if ("$lte" in op && op.$lte !== undefined) return { kind: "lte", column, value: op.$lte }; + if ("$gt" in op && op.$gt !== undefined) return { kind: "gt", column, value: enc(op.$gt) }; + if ("$gte" in op && op.$gte !== undefined) return { kind: "gte", column, value: enc(op.$gte) }; + if ("$lt" in op && op.$lt !== undefined) return { kind: "lt", column, value: enc(op.$lt) }; + if ("$lte" in op && op.$lte !== undefined) return { kind: "lte", column, value: enc(op.$lte) }; if ("$like" in op && op.$like !== undefined) return { kind: "like", column, pattern: op.$like }; - if ("$in" in op && op.$in !== undefined) return { kind: "in", column, values: op.$in }; + if ("$in" in op && op.$in !== undefined) return { kind: "in", column, values: op.$in.map(enc) }; if ("$isNull" in op && op.$isNull !== undefined) { return { kind: "isNull", column, not: !op.$isNull }; } throw new MetadataError(`No recognized operator on filter for field '${fieldName}'`); } +/** + * Encode one filter comparison value for an int-backed `field.enum`: the member + * SYMBOL becomes its declared integer. Anything else — a string-backed enum, a + * non-string value, or a symbol with no mapping — passes through untouched. An + * unmapped symbol is deliberately NOT rejected here: a filter that matches nothing + * is the honest answer for a member that does not exist, and the loader already + * pins `@intValueMap`'s key set to `@values`. + */ +function encodeFilterValue(field: MetaData, value: T): T { + const intMap = intValueMapOf(field); + if (intMap === undefined || typeof value !== "string") return value; + const stored = intMap[value]; + return (typeof stored === "number" ? stored : value) as T; +} + function normalizeOrderBy( input: QueryOpts["orderBy"], entity: MetaData, strategy: ColumnNamingStrategy, ): OrderBy[] | undefined { diff --git a/server/typescript/packages/runtime-ts/src/type-coercer.ts b/server/typescript/packages/runtime-ts/src/type-coercer.ts index ab5cc203a..3fbdfece9 100644 --- a/server/typescript/packages/runtime-ts/src/type-coercer.ts +++ b/server/typescript/packages/runtime-ts/src/type-coercer.ts @@ -17,9 +17,11 @@ import type { MetaData } from "@metaobjectsdev/metadata"; import { TYPE_FIELD, FIELD_SUBTYPE_BOOLEAN, + FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_OBJECT, FIELD_ATTR_STORAGE, FIELD_ATTR_DB_COLUMN_TYPE, + FIELD_ATTR_INT_VALUE_MAP, STORAGE_JSONB, STORAGE_FLATTENED, DB_COLUMN_TYPE_JSONB, @@ -28,14 +30,111 @@ import type { Dialect, Row } from "./persistence-driver.js"; export function coerceRowOnRead(entity: MetaData, row: Row, dialect: Dialect): Row { const hydrated = deserializeJsonbObjectFields(entity, row); - if (dialect !== "sqlite") return hydrated; - return mapBooleansFromInt(entity, hydrated); + const decoded = decodeIntBackedEnums(entity, hydrated); + if (dialect !== "sqlite") return decoded; + return mapBooleansFromInt(entity, decoded); +} + +/** + * The effective `@intValueMap` (member symbol → integer) for an int-backed + * `field.enum`, or undefined when the enum is string-backed. Its PRESENCE is the + * whole trigger for integer persistence (design D5) — there is no separate flag. + * + * ADR-0039: RESOLVING (`attr`, not `ownAttr`), and load-bearing rather than + * incidental. Post-#246 an own `@intValueMap` against a shared (root-level + * abstract) enum is `ERR_ENUM_EXTENDS_VALUES_CONFLICT`, so the map lives on the + * SHARED DECLARATION and every consuming field INHERITS it — an own-only read + * would see undefined on exactly the shape adopters are steered toward and bind + * the symbol straight into an integer column. + * + * Mirrors `codegen-ts`'s `intValueMapOf`; duplicated rather than shared because + * `runtime-ts` must not depend on a codegen package (the two are disjoint trees, + * and only one of them ships to a server at runtime). + */ +export function intValueMapOf(field: MetaData): Record | undefined { + if (field.subType !== FIELD_SUBTYPE_ENUM) return undefined; + const raw = field.attr(FIELD_ATTR_INT_VALUE_MAP); + if (raw === undefined || raw === null || typeof raw !== "object") return undefined; + return raw as Record; +} + +/** + * Encode a member SYMBOL to its declared integer for an int-backed `field.enum`. + * + * An UNMAPPED symbol is passed through untouched rather than rejected here: the + * column's `CHECK` is what enforces membership, and inventing a value would hide + * the drift. Matching Python's write codec exactly — every port leaves the write + * side to the database. + */ +function encodeIntBackedEnum(intMap: Record, value: unknown): unknown { + if (typeof value !== "string") return value; + const stored = intMap[value]; + return typeof stored === "number" ? stored : value; +} + +/** + * Decode the stored integer back to its member symbol, so the runtime's return + * value is the symbol in both backing modes — int-backing is a persistence-layer + * concern and must be invisible above this codec (ADR-0019). + * + * An int with no member THROWS. The row holds data the model says is impossible + * (a hand-written INSERT, or a member removed without a migration); surfacing the + * raw integer would hand the caller a "member" that is not one, and returning + * null would hide the corruption behind a nullable column. Every port throws + * here — the generated Drizzle `customType`'s `fromDriver` included. + */ +function decodeIntBackedEnums(entity: MetaData, row: Row): Row { + let out: Row | null = null; + // ADR-0039: resolving — an int-backed enum may be inherited from a base via extends. + for (const child of entity.children()) { + if (child.type !== TYPE_FIELD) continue; + const intMap = intValueMapOf(child); + if (intMap === undefined) continue; + if (!(child.name in row)) continue; + const raw = row[child.name]; + if (raw === null || raw === undefined) continue; + // A driver may hand back a BIGINT-ish string; Number() covers both shapes. + const stored = typeof raw === "number" ? raw : Number(raw); + if (!Number.isInteger(stored)) continue; + const member = Object.keys(intMap).find((k) => intMap[k] === stored); + if (member === undefined) { + throw new Error( + `field.enum '${child.name}' read stored value ${stored} with no member in ` + + `@intValueMap (declared: ${JSON.stringify(intMap)}) — the database holds a ` + + `value the model does not describe.`, + ); + } + out ??= { ...row }; + out[child.name] = member; + } + return out ?? row; +} + +function encodeIntBackedEnums(entity: MetaData, row: Row): Row { + let out: Row | null = null; + for (const child of entity.children()) { + if (child.type !== TYPE_FIELD) continue; + const intMap = intValueMapOf(child); + if (intMap === undefined) continue; + if (!(child.name in row)) continue; + const raw = row[child.name]; + if (raw === null || raw === undefined) continue; + const encoded = encodeIntBackedEnum(intMap, raw); + if (encoded === raw) continue; + out ??= { ...row }; + out[child.name] = encoded; + } + return out ?? row; } export function coerceRowOnWrite(entity: MetaData, row: Row, dialect: Dialect): Row { const jsonbColumned = serializeJsonbColumns(entity, row); - if (dialect !== "sqlite") return jsonbColumned; - return mapBooleansToInt(entity, jsonbColumned); + // Dialect-independent: an int-backed enum's column is INTEGER on every dialect + // (SQLite has one integer storage class), so the symbol→int encode is not gated + // on the dialect the way the boolean mapping below is. + const encoded = encodeIntBackedEnums(entity, jsonbColumned); + if (dialect !== "sqlite") return encoded; + return mapBooleansToInt(entity, encoded); } /** diff --git a/server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts b/server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts new file mode 100644 index 000000000..b48994c3e --- /dev/null +++ b/server/typescript/packages/runtime-ts/test/int-backed-enum-codec.test.ts @@ -0,0 +1,162 @@ +import { describe, test, expect } from "bun:test"; +import type { MetaData } from "@metaobjectsdev/metadata"; +import { TypeId, TYPE_OBJECT, TYPE_FIELD, TYPE_IDENTITY, + FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_ENUM, + FIELD_ATTR_VALUES, FIELD_ATTR_INT_VALUE_MAP, + IDENTITY_SUBTYPE_PRIMARY, OBJECT_SUBTYPE_ENTITY } from "@metaobjectsdev/metadata"; +import { meta } from "./_meta-build.js"; +import { coerceRowOnRead, coerceRowOnWrite } from "../src/type-coercer.js"; +import { compileFilter } from "../src/query-builder.js"; + +// An int-backed field.enum (@intValueMap) persists as an INTEGER while the runtime's +// value — in, out, and in a filter — stays the member SYMBOL. This is the ObjectManager +// half of the codec: the generated Drizzle `customType` covers the codegen path, and +// nothing covered this one, so `create()` bound "PUBLISHED" straight into an integer +// column and Postgres rejected the statement. + +const VALUES = ["DRAFT", "PUBLISHED", "ARCHIVED"]; +const INT_MAP = { DRAFT: 0, PUBLISHED: 5, ARCHIVED: 9 }; + +/** + * An Order whose `status` is int-backed, plus a string-backed `kind` for contrast. + * Takes `null` — not `undefined` — for "no map": passing `undefined` explicitly would + * trigger the default parameter and silently hand back the int-backed shape, which is + * exactly how the first draft of the string-backed test passed for the wrong reason. + */ +function makeOrder(intMap: Record | null = INT_MAP): MetaData { + const order = meta(new TypeId(TYPE_OBJECT, OBJECT_SUBTYPE_ENTITY), "Order"); + order.addChild(meta(new TypeId(TYPE_FIELD, FIELD_SUBTYPE_LONG), "id")); + + const status = meta(new TypeId(TYPE_FIELD, FIELD_SUBTYPE_ENUM), "status"); + status.setAttr(FIELD_ATTR_VALUES, VALUES); + if (intMap !== null) status.setAttr(FIELD_ATTR_INT_VALUE_MAP, intMap); + order.addChild(status); + + const kind = meta(new TypeId(TYPE_FIELD, FIELD_SUBTYPE_ENUM), "kind"); + kind.setAttr(FIELD_ATTR_VALUES, VALUES); + order.addChild(kind); + + const primary = meta(new TypeId(TYPE_IDENTITY, IDENTITY_SUBTYPE_PRIMARY), "primary"); + primary.setAttr("fields", ["id"]); + order.addChild(primary); + return order; +} + +describe("int-backed field.enum — ObjectManager write codec", () => { + test("encodes the member symbol to its declared int", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: "PUBLISHED" }, "postgres"); + expect(row.status).toBe(5); + }); + + test("a string-backed enum on the same entity is untouched", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, kind: "PUBLISHED" }, "postgres"); + expect(row.kind).toBe("PUBLISHED"); + }); + + test("encodes on sqlite too — the column is INTEGER on every dialect", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: "ARCHIVED" }, "sqlite"); + expect(row.status).toBe(9); + }); + + test("null passes through", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: null }, "postgres"); + expect(row.status).toBeNull(); + }); + + // Membership is the column's CHECK to enforce; inventing a value here would hide drift. + test("an unmapped symbol is passed through for the database to reject", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1, status: "NOPE" }, "postgres"); + expect(row.status).toBe("NOPE"); + }); + + test("a field absent from the row stays absent", () => { + const row = coerceRowOnWrite(makeOrder(), { id: 1 }, "postgres"); + expect("status" in row).toBe(false); + }); +}); + +describe("int-backed field.enum — ObjectManager read codec", () => { + test("decodes the stored int back to the member symbol", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: 5 }, "postgres"); + expect(row.status).toBe("PUBLISHED"); + }); + + test("decodes 0 — the falsy member the naive guard drops", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: 0 }, "postgres"); + expect(row.status).toBe("DRAFT"); + }); + + test("a driver-stringified integer still decodes", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: "9" }, "postgres"); + expect(row.status).toBe("ARCHIVED"); + }); + + test("null stays null", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, status: null }, "postgres"); + expect(row.status).toBeNull(); + }); + + test("a string-backed enum reads through unchanged", () => { + const row = coerceRowOnRead(makeOrder(), { id: 1, kind: "DRAFT" }, "postgres"); + expect(row.kind).toBe("DRAFT"); + }); + + // The row holds data the model says is impossible. Surfacing the raw int would hand + // the caller a "member" that is not one; null would hide the corruption. Every port throws. + test("a stored int with no member THROWS", () => { + expect(() => coerceRowOnRead(makeOrder(), { id: 1, status: 7 }, "postgres")).toThrow( + /stored value 7 with no member in @intValueMap/, + ); + }); + + test("an enum with no @intValueMap is left alone entirely", () => { + const row = coerceRowOnRead(makeOrder(null), { id: 1, status: 5 }, "postgres"); + expect(row.status).toBe(5); + }); +}); + +describe("int-backed field.enum — filter values", () => { + test("eq encodes the symbol", () => { + expect(compileFilter(makeOrder(), { status: "PUBLISHED" })).toEqual({ + kind: "eq", column: "status", value: 5, + }); + }); + + test("$ne encodes", () => { + expect(compileFilter(makeOrder(), { status: { $ne: "DRAFT" } })).toEqual({ + kind: "ne", column: "status", value: 0, + }); + }); + + test("$in encodes every member", () => { + expect(compileFilter(makeOrder(), { status: { $in: ["DRAFT", "ARCHIVED"] } })).toEqual({ + kind: "in", column: "status", values: [0, 9], + }); + }); + + test("a bare array encodes every member", () => { + expect(compileFilter(makeOrder(), { status: ["PUBLISHED"] })).toEqual({ + kind: "in", column: "status", values: [5], + }); + }); + + test("a string-backed enum filter is untouched", () => { + expect(compileFilter(makeOrder(), { kind: "PUBLISHED" })).toEqual({ + kind: "eq", column: "kind", value: "PUBLISHED", + }); + }); + + // A filter naming a member that does not exist should match nothing, not throw — + // and it must not silently become a WRONG integer. + test("an unmapped symbol passes through rather than becoming a wrong int", () => { + expect(compileFilter(makeOrder(), { status: "NOPE" })).toEqual({ + kind: "eq", column: "status", value: "NOPE", + }); + }); + + test("isNull carries no value to encode", () => { + expect(compileFilter(makeOrder(), { status: null })).toEqual({ + kind: "isNull", column: "status", not: false, + }); + }); +}); diff --git a/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md b/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md index 7e8b127aa..749efcca0 100644 --- a/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md +++ b/spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md @@ -165,7 +165,9 @@ still want, then freeze. Before cutting 1.0: (§2) fixes the underlying churn: most releases stop being metamodel events. Apply the semver rule strictly — the trigger is *new public surface, not code size*: a **package MINOR** adds surface a consumer can newly depend on (codegen output, a CLI - flag, a newly-supported vocab member); a **package PATCH** is a bugfix/refactor with + flag, a newly-supported vocab member — **but see Amendment 1: "vocab member" is too + coarse, and taking it literally re-created exactly the minor churn this bullet was + written to stop**); a **package PATCH** is a bugfix/refactor with no new surface (e.g. the `0.15.2` output-prompt fix — one-port, no surface); the **Metamodel spec version** bumps only when the shared vocabulary/wire contract itself changes. A run of minors for non-additive changes is the "minor churn" @@ -175,6 +177,44 @@ still want, then freeze. Before cutting 1.0: - **Migration + policy docs:** a `0.x → 1.0` migration guide and a published compatibility policy (this ADR's §1, consumer-facing). +## Amendment 1 (2026-08-17) — "a newly-supported vocab member" splits three ways + +The cadence-discipline bullet above lists "a newly-supported vocab member" among the +triggers for a package MINOR. Applied literally it became a rule that **any** registry +addition is a MINOR — and that rule produced the churn this ADR exists to prevent: +`0.22.0` and `0.23.0` were both cut MINOR for additions a project declaring no +`requirement.*` nodes could not observe at all, each changelog saying so in its own +opening paragraph. Four registries move per cut here, so a wasted minor is not free. + +The error is treating `expected-registry.json` as consumer surface. It is an **internal** +gate: five ports byte-matching one manifest is how we stop the ports drifting from each +other. Its churn is evidence about *us*, not about an adopter's project. "New public +surface, not code size" was the right instinct; "vocab member" was the wrong unit. +Vocabulary sorts by what it can do to a consumer: + +- **A new ATTRIBUTE ⇒ PATCH.** Reachable only by authoring it; every existing document + loads unchanged and emits byte-identical output. Nothing to adopt deliberately. +- **A new top-level TYPE ⇒ MINOR.** A new modeling concept with its own children, + validation, and usually tooling surface (`requirement.*` brought a `verify` pass and + summary output). Genuinely new surface to depend on. +- **A new SUBTYPE ⇒ either, and the test is whether it is INERT.** PATCH when only + authoring it can reach it: no existing valid document changes meaning or output, + nothing previously permitted is narrowed, nothing reserved is consumed. MINOR when + any of those fails — closing a wildcard, promoting a reserved-not-registered member + (ADR-0007 Amendment 2 / ADR-0040), or shifting what the recommended shape for an + existing field is — or when it headlines a release and you want the range bump on + purpose. + +The caret rule is not to be inverted. "Pre-1.0 `^0.22.x` resolves `<0.23.0`, so a +consumer adopts a MINOR deliberately" is a reason to **choose** MINOR when that gate is +wanted; it is not a reason additive vocabulary must be MINOR. A minor spent on a change +nobody can observe is a gate you no longer have when something real needs it. + +This amendment changes cadence policy only. It does not touch §1's post-1.0 compat +promise: after 1.0, a **breaking** change to the metamodel vocabulary still requires a +MAJOR, whether the break is an attribute, a subtype, or a type. Operational form of the +rule (with worked rows): `docs/RELEASING.md` → "Versioning policy (pre-1.0)". + ## Consequences - After 1.0, a breaking change to the metamodel vocabulary, the canonical/wire diff --git a/spec/decisions/ADR-0039-own-accessor-discipline.md b/spec/decisions/ADR-0039-own-accessor-discipline.md index df77ec4a4..68c627570 100644 --- a/spec/decisions/ADR-0039-own-accessor-discipline.md +++ b/spec/decisions/ADR-0039-own-accessor-discipline.md @@ -34,7 +34,10 @@ Two attributes are read own-only by explicit policy, outside the emit-declared-h The distinction is only observable on a **chained declaration**: a root-level abstract enum `B extends` a root-level abstract `@provided` enum `A`. `@provided` is read on the resolved *declaration*, never on the consuming field, so for the ordinary `field extends @provided decl` shape own and resolving agree. On the chained shape a resolving read reports `B` as provided and emits a reference to a hand-written `B` **the adopter never declared** (the marker was authored on `A`), instead of materializing `B` from its inherited `@values`. Own-only matches authored intent. - Note this is a *provenance* marker and not a value: the member set it accompanies (`@values`, and its numeric half `@intValueMap`) is a logical property and is still read **resolving**, so a declaration inheriting `@values` from its super materializes correctly. + Note this is a *provenance* marker and not a value: the member set it accompanies (`@values`, and its numeric half `@intValueMap`) + is a logical property and is still read **resolving**, so a declaration inheriting `@values` from its super materializes correctly. + The `@intValueMap` attribute itself (registered on `field.enum` since 2026-08) is also read resolving, enabling inheritance through + `extends` chains. ### Naming Where a port's default-named accessor is own-only (Python `attr()` is own; TS `attr()` resolves — an inversion), the port SHOULD make the **resolving** form the default-named one and the own form explicitly `own*`, so "the obvious call" is the correct one. Any `own*()` call MUST carry a one-line comment stating which sanctioned case it is. diff --git a/spec/metamodel/attr.json b/spec/metamodel/attr.json index 45f4cea6b..a57e1d6a9 100644 --- a/spec/metamodel/attr.json +++ b/spec/metamodel/attr.json @@ -19,6 +19,12 @@ "dataType": "int", "description": "A 32-bit-integer-valued metadata attribute. Coerces to and validates as a number." }, + { + "type": "attr", + "subType": "intMap", + "dataType": "object", + "description": "An object-shaped attribute whose values are all integers (e.g. field.enum's @intValueMap: {memberSymbol: int}). Generic shape check only; a consumer field type layers its own semantic rules (key-set membership, uniqueness) in its own content-rule validation." + }, { "type": "attr", "subType": "long", diff --git a/spec/metamodel/field.json b/spec/metamodel/field.json index 01be74308..a80d56cf5 100644 --- a/spec/metamodel/field.json +++ b/spec/metamodel/field.json @@ -150,7 +150,8 @@ "rules": "Required @values is a non-empty, duplicate-free set; each member must match ^[A-Za-z_][A-Za-z0-9_]*$ so symbol == stored string in every target language. Optional FR-010/FR-011 overlays add tolerant-extract aliasing (@enumAlias), per-member docs (@enumDoc), an uncoercible-value fallback (@coerceDefault, must be one of @values), and ASCII normalization mode (@normalize).", "children": [ { "type": "attr", "subType": "string", "name": "values", "isArray": true, "min": 1, "max": 1, "description": "Member symbols of an enum-subtype field. Declaration order is significant; each is a legal identifier and its own stored string." }, - { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." } + { "type": "attr", "subType": "boolean", "name": "provided", "min": 0, "max": 1, "description": "FR-019: marks an abstract package-level field.enum as externally provided — codegen references the type (resolved via per-port codegen config) instead of materializing it. Default false. Not a field attr — it lives on the type declaration." }, + { "type": "attr", "subType": "intMap", "name": "intValueMap", "min": 0, "max": 1, "description": "Optional per-member int values ({member: int}) switching this enum field's DB persistence from string+CHECK to integer+CHECK. Keys must exactly match @values; values must be unique integers. The generated native type and wire format are unchanged in every language." } ] }, {