fix(codegen-ts): an assigned primary key must be required on insert

An entity whose primary key carries no `identity.primary @generation` -- a
natural key, or an id issued by something upstream -- generated code that did
not compile.

The InsertSchema read a PK field's optionality off `@required`, like any other
field, so a PK with no `@required` became `id: z.string().optional()`. The
Drizzle column for that same PK is `text("id").primaryKey()` with NO default and
is therefore required on insert, and the generated `create<Entity>` pipes
`InsertSchema.parse(data)` straight into `.values()`. tsc reported TS2769 ("No
overload matches this call") on the queries file. It was a semantic hole as well
as a typing one: the schema accepted a create payload with no primary key.

The reasoning already existed one function away. `primaryKeyFieldNames` excludes
PKs from the UpdateSchema's `.nullable()` treatment because "a PK column is
never NULL" -- the same fact says PK optionality can never be read off
`@required` alone. This applies it to the insert shape.

Narrow by construction. A `@generation: increment|uuid` PK is still omitted from
the schema entirely (the caller never supplies it) and a PK with a `@default`
stays optional (its column carries that default), so the only shape that moves
is the one that could not compile. Output is byte-identical for every model in
this repo -- all of them generate their PKs, which is precisely why nothing here
caught it; `verify --codegen` on the advanced-modeling example reports no drift.

Gated by a compile test that runs the real TypeScript compiler over the entity
AND queries files TOGETHER: the defect is a mismatch BETWEEN those two files, so
no single-file gate could have seen it. It pins both already-correct arms
(generated uuid PK, int rowid PK) against regression, on both dialects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: dmealing

CHANGELOG.md

@@ -7,6 +7,8 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ## [Unreleased]
 
+## [0.22.0] — npm `0.22.0` · PyPI `0.22.0` · NuGet `0.22.0` · Maven `7.22.0`
+
 A **MINOR**: a new registered type family (new vocabulary in all five ports), plus a
 registry tightening that turns a previously-permitted provider shape into a hard error.
 Nothing here changes runtime behaviour on an existing database.
@@ -127,6 +129,35 @@ structurally.
 The conformance corpus's committed canonical schema is regenerated: a parent-side
 `relationship.composition` was contributing no referential action.
 
+### Fixed — an assigned primary key generated code that did not compile
+
+**Generated-output change — regenerate to pick it up; three-way merge preserves hand edits.**
+
+An entity whose primary key carries no `identity.primary @generation` — a natural key, or an
+id issued by something upstream — emitted a `<Entity>InsertSchema` that made the PK
+`.optional()`, because a PK's optionality was read off `@required` like any other field's.
+The Drizzle column for that same PK is `text("id").primaryKey()` with **no default**, so it
+is required on insert, and the generated `create<Entity>` pipes `InsertSchema.parse(data)`
+straight into `.values()`. The result did not typecheck (`TS2769`), and the schema also
+accepted a create payload carrying no primary key at all.
+
+An assigned PK with no `@default` is now required in the insert shape. Unchanged: a
+`@generation: increment|uuid` PK is still omitted from the schema entirely (the caller never
+supplies it), a PK with a `@default` stays optional (the column has that default), and the
+`UpdateSchema` keeps every field optional under PATCH semantics.
+
+Output is byte-identical for any model whose PKs are generated or `@required` — which is
+every model in this repository, and why nothing here caught it. Gated by a new compile test
+that runs the real TypeScript compiler over the entity **and** queries files together, since
+the defect was a mismatch *between* the two and no single-file gate could see it.
+
+### Fixed — `@metaobjectsdev/cli` no longer pulls a `yaml` package it never imports
+
+`yaml` entered the CLI's runtime `dependencies` while a requirement was still a YAML side
+file the CLI parsed itself. Requirements became registered vocabulary two commits later and
+the only file importing it was deleted; the dependency was not. Nothing in the package
+imports the module today, so an install of the CLI no longer resolves it.
+
 
 ## [0.21.6] — npm `0.21.6` · PyPI `0.21.6` · NuGet `0.21.6` · Maven `7.21.6`
 

server/typescript/packages/codegen-ts/src/templates/zod-validators.ts

@@ -159,6 +159,44 @@ function primaryKeyFieldNames(obj: MetaObject): Set<string> {
   return new Set(primaryIdentityFieldNames(obj));
 }
 
+/** PK field names the CALLER must supply on insert: an ASSIGNED primary key —
+ *  a natural key or an externally-issued id, i.e. a primary identity carrying no
+ *  @generation — that also has no @default to fill it.
+ *
+ *  These are insert-REQUIRED regardless of @required, and that is a typing
+ *  necessity rather than a policy choice: the Drizzle column for such a PK is
+ *  `text("id").primaryKey()` with no default, so its inferred insert type
+ *  requires the value. Deriving the PK's optionality from @required like any
+ *  other field emitted `id: z.string().optional()`, and the generated
+ *  `create<Entity>` — which pipes `InsertSchema.parse(data)` straight into
+ *  `.values()` — then failed to compile (TS2769). It was also a semantic hole:
+ *  the schema accepted a create payload with no primary key at all.
+ *
+ *  Excluded, because each is already insert-optional for a real reason:
+ *    • @generation increment|uuid PKs — omitted from the schema entirely
+ *      (autoGenPkFieldNames), the caller never supplies them;
+ *    • a PK carrying @default — the column gets that default, so omitting is
+ *      legal and forcing it required would make callers repeat the default.
+ *  Mirrors primaryKeyFieldNames' reasoning for the UpdateSchema: a PK column is
+ *  never NULL, so PK optionality can never be read off @required alone. */
+function assignedPkFieldNames(obj: MetaObject): Set<string> {
+  const autoGen = autoGenPkFieldNames(obj);
+  const out = new Set<string>();
+  for (const name of primaryIdentityFieldNames(obj)) {
+    if (autoGen.has(name)) continue;
+    out.add(name);
+  }
+  return out;
+}
+
+/** True when this field is an assigned PK with no @default — see
+ *  assignedPkFieldNames. Kept as one predicate so the emitters and the
+ *  documented-shape function (insertSchemaFields) cannot drift. */
+function isInsertRequiredPk(field: MetaField, assignedPk: Set<string>): boolean {
+  if (!assignedPk.has(field.name)) return false;
+  return field.attr(FIELD_ATTR_DEFAULT) === undefined;
+}
+
 /**
  * Emit ONLY the `<Name>InsertSchema`. Used by the value-object file emitter
  * for metaobjects with no writable source.rdb — those have no PATCH/update
@@ -171,6 +209,7 @@ function primaryKeyFieldNames(obj: MetaObject): Set<string> {
 export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Code {
   const z = imp("z@zod");
   const autoGenPkFields = autoGenPkFieldNames(obj);
+  const assignedPkFields = assignedPkFieldNames(obj);
   const tphPin = tphDiscriminatorPin(obj);
 
   const insertFieldLines: Code[] = [];
@@ -204,7 +243,9 @@ export function renderInsertSchemaOnly(obj: MetaObject, ctx?: RenderContext): Co
           : code`  ${child.name}: z.string().optional().transform(() => new Date().toISOString())`,
       );
     } else {
-      insertFieldLines.push(code`  ${child.name}: ${zodFieldExpr(child, obj, ctx)}`);
+      insertFieldLines.push(
+        code`  ${child.name}: ${zodFieldExpr(child, obj, ctx, isInsertRequiredPk(child, assignedPkFields))}`,
+      );
     }
   }
 
@@ -246,11 +287,14 @@ export interface SchemaFieldShape {
  *   • @readOnly fields are omitted (DB / replication owns the write path);
  *   • a TPH subtype's @discriminator field is a pinned `z.literal(value)`;
  *   • @autoSet fields are present but optional (server fills them);
+ *   • an ASSIGNED primary key with no @default is required (see
+ *     assignedPkFieldNames — its Drizzle column has no default);
  *   • every other field's optionality is `fieldWillBeOptional` (not required, or
  *     carries a @default).
  */
 export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] {
   const autoGenPkFields = autoGenPkFieldNames(obj);
+  const assignedPkFields = assignedPkFieldNames(obj);
   const tphPin = tphDiscriminatorPin(obj);
   const out: SchemaFieldShape[] = [];
   for (const child of obj.fields()) {
@@ -266,7 +310,10 @@ export function insertSchemaFields(obj: MetaObject): SchemaFieldShape[] {
     if (autoSet === AUTO_SET_ON_CREATE || autoSet === AUTO_SET_ON_UPDATE) {
       out.push({ name: child.name, optional: true, autoSet: true });
     } else {
-      out.push({ name: child.name, optional: fieldWillBeOptional(child) });
+      out.push({
+        name: child.name,
+        optional: isInsertRequiredPk(child, assignedPkFields) ? false : fieldWillBeOptional(child),
+      });
     }
   }
   return out;
@@ -311,6 +358,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code
   const z = imp("z@zod");
   const autoGenPkFields = autoGenPkFieldNames(obj);
   const pkFields = primaryKeyFieldNames(obj);
+  const assignedPkFields = assignedPkFieldNames(obj);
   const tphPin = tphDiscriminatorPin(obj);
 
   const insertFieldLines: Code[] = [];
@@ -358,7 +406,9 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code
       // field expr) so an import/restore keeps the caller's original timestamp.
       preservingFieldLines.push(code`  ${child.name}: ${zodFieldExpr(child, obj, ctx)}`);
     } else {
-      const fieldLine = code`  ${child.name}: ${zodFieldExpr(child, obj, ctx)}`;
+      // The preserving schema is an INSERT shape (import / restore / replication),
+      // so an assigned PK is required there for the same typing reason.
+      const fieldLine = code`  ${child.name}: ${zodFieldExpr(child, obj, ctx, isInsertRequiredPk(child, assignedPkFields))}`;
       insertFieldLines.push(fieldLine);
       preservingFieldLines.push(fieldLine);
     }
@@ -452,7 +502,14 @@ function zodScalarFor(subType: string): string {
   return "z.string()"; // string/uuid/date/time/timestamp/decimal/enum on the wire
 }
 
-function zodFieldExpr(field: MetaField, owner?: MetaObject, ctx?: RenderContext): Code {
+function zodFieldExpr(
+  field: MetaField,
+  owner?: MetaObject,
+  ctx?: RenderContext,
+  /** Suppress the trailing `.optional()` — see assignedPkFieldNames. Set ONLY
+   *  by the insert-shape emitters; the update/read shapes must stay optional. */
+  forceRequired = false,
+): Code {
   // `@dbColumnType: jsonb` on a scalar (legal only on field.string) is the
   // sanctioned "open JSON bag" escape hatch — a genuinely untyped JSON column
   // with no value-object to reference. Its Drizzle column is a bare `jsonb()`,
@@ -465,7 +522,7 @@ function zodFieldExpr(field: MetaField, owner?: MetaObject, ctx?: RenderContext)
   if (field.attr(FIELD_ATTR_DB_COLUMN_TYPE) === DB_COLUMN_TYPE_JSONB) {
     let base: Code = code`z.unknown()`;
     if (field.resolvedIsArray()) base = code`z.array(${base})`;
-    return appendValidatorChain(base, field);
+    return appendValidatorChain(base, field, forceRequired);
   }
 
   // FIELD_SUBTYPE_OBJECT: emit z.array(<Ref>InsertSchema) / <Ref>InsertSchema
@@ -490,13 +547,13 @@ function zodFieldExpr(field: MetaField, owner?: MetaObject, ctx?: RenderContext)
       const refImp = imp(`${refName}InsertSchema@${moduleSpec}`);
       let base: Code = code`${refImp}`;
       if (field.resolvedIsArray()) base = code`z.array(${base})`;
-      return appendValidatorChain(base, field);
+      return appendValidatorChain(base, field, forceRequired);
     }
     // No resolvable @objectRef — fall through to z.unknown(); downstream code
     // can still pass a value through but loses validation.
     let base: Code = code`z.unknown()`;
     if (field.resolvedIsArray()) base = code`z.array(${base})`;
-    return appendValidatorChain(base, field);
+    return appendValidatorChain(base, field, forceRequired);
   }
 
   // field.map → z.record(z.string(), V): value is a VO's InsertSchema (@objectRef)
@@ -509,10 +566,10 @@ function zodFieldExpr(field: MetaField, owner?: MetaObject, ctx?: RenderContext)
         ? valueObjectModuleSpecifier(refName, ctx.packageOf, owner.package, ctx.outputLayout, ctx.extStyle)
         : `./${refName}.js`;
       const refImp = imp(`${refName}InsertSchema@${moduleSpec}`);
-      return appendValidatorChain(code`z.record(z.string(), ${refImp})`, field);
+      return appendValidatorChain(code`z.record(z.string(), ${refImp})`, field, forceRequired);
     }
     const vt = field.attr(FIELD_ATTR_VALUE_TYPE);
-    return appendValidatorChain(code`z.record(z.string(), ${zodScalarFor(typeof vt === "string" ? vt : "string")})`, field);
+    return appendValidatorChain(code`z.record(z.string(), ${zodScalarFor(typeof vt === "string" ? vt : "string")})`, field, forceRequired);
   }
 
   let baseStr: string;
@@ -573,7 +630,7 @@ function zodFieldExpr(field: MetaField, owner?: MetaObject, ctx?: RenderContext)
           const sharedConst = imp(`${constName}@${spec}`);
           let base: Code = code`${sharedConst}`;
           if (field.resolvedIsArray()) base = code`z.array(${base})`;
-          return appendValidatorChain(base, field);
+          return appendValidatorChain(base, field, forceRequired);
         }
       }
       baseStr = zodEnumExpr(values);
@@ -612,7 +669,7 @@ function zodFieldExpr(field: MetaField, owner?: MetaObject, ctx?: RenderContext)
   }
 
   if (field.resolvedIsArray()) baseStr = `z.array(${baseStr})`;
-  return appendValidatorChain(code`${baseStr}`, field);
+  return appendValidatorChain(code`${baseStr}`, field, forceRequired);
 }
 
 /** Mirrors the optional-or-not decision inside appendValidatorChain so the update-schema
@@ -635,7 +692,7 @@ const NUMERIC_FIELD_SUBTYPES = new Set<string>([
  *  - numeric (scalar)       → .min/.max = numeric value     (validator.numeric)
  *  - array (any element)    → .min/.max = element count     (validator.array)
  */
-function appendValidatorChain(base: Code, field: MetaField): Code {
+function appendValidatorChain(base: Code, field: MetaField, forceRequired = false): Code {
   let isRequired = field.attr(FIELD_ATTR_REQUIRED) === true;
   let maxLen: number | undefined = field.attr(FIELD_ATTR_MAX_LENGTH) as number | undefined;
   let minLen: number | undefined;
@@ -709,6 +766,12 @@ function appendValidatorChain(base: Code, field: MetaField): Code {
     if (numMax !== undefined) chain = code`${chain}.max(${numMax})`;
   }
 
+  // An assigned PK is insert-REQUIRED whatever @required says — its Drizzle
+  // column has no default, so an optional value cannot typecheck into
+  // `.values()`. Only the caller knows this (the update/read shapes must stay
+  // optional), hence the explicit opt-in. See assignedPkFieldNames.
+  if (forceRequired) return chain;
+
   // Fields with DB-level defaults are optional in the InsertSchema: the caller
   // can omit them and the DB will fill in. Otherwise required-with-default
   // would force callers to repeat the default at every call site.

server/typescript/packages/codegen-ts/test/assigned-pk-insert-compile.test.ts

@@ -0,0 +1,144 @@
+// Regression guard: an ASSIGNED primary key (an entity whose PK carries no
+// `identity.primary @generation`, e.g. a natural key or an externally-issued id)
+// emitted generated code that did not compile.
+//
+// The InsertSchema derived a PK field's optionality from `@required` like any
+// other field, so a PK with no `@required` became `id: z.string().optional()`.
+// The Drizzle column for that same PK is `text("id").primaryKey()` — no default,
+// therefore REQUIRED on insert. `createOrder` pipes one straight into the other
+// (`db.insert(t).values(InsertSchema.parse(data))`), so `tsc` reported TS2769
+// ("No overload matches this call") on the generated queries file.
+//
+// Two arms are already correct and are pinned here so a fix can't regress them:
+//   • a GENERATED pk (@generation: uuid|increment) is OMITTED from the
+//     InsertSchema entirely — the caller never supplies it;
+//   • an int rowid pk compiles even when optional, because Drizzle's
+//     `integer().primaryKey()` is rowid-aliased and thus insert-optional.
+//
+// This compiles entity + queries TOGETHER with the real TS compiler — the bug is
+// a mismatch BETWEEN those two files, so a single-file gate cannot see it.
+
+import { describe, test, expect } from "bun:test";
+import { mkdtempSync, writeFileSync, rmSync } from "node:fs";
+import { join } from "node:path";
+import ts from "typescript";
+import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata";
+import { entityFile, queriesFile } from "../src/generators/index.js";
+import { makeRenderContext } from "../src/render-context.js";
+import { buildPkMap } from "../src/pk-resolver.js";
+import { buildRelationMap } from "../src/relation-resolver.js";
+import type { GenContext } from "../src/generator.js";
+import type { Dialect } from "../src/metaobjects-config.js";
+
+/** @param pkField the PK field node; @param identity the identity.primary node. */
+async function loadRoot(pkField: unknown, identity: unknown) {
+  const json = JSON.stringify({
+    "metadata.root": {
+      package: "test",
+      children: [
+        {
+          "object.entity": {
+            name: "Order",
+            children: [
+              { "source.rdb": { "@kind": "table", "@table": "orders" } },
+              pkField,
+              { "field.string": { name: "buyer", "@required": true } },
+              identity,
+            ],
+          },
+        },
+      ],
+    },
+  });
+  const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]);
+  if (result.errors.length > 0) {
+    throw new Error(`Loader errors:\n${result.errors.map((e) => e.message).join("\n")}`);
+  }
+  return result.root;
+}
+
+// Real TS compiler gate. Files land under this package so the program resolves
+// drizzle-orm / zod from the package's own node_modules (the REAL types).
+function compile(dir: string, files: string[]): readonly ts.Diagnostic[] {
+  const program = ts.createProgram(
+    files.map((f) => join(dir, f)),
+    {
+      strict: true,
+      noEmit: true,
+      target: ts.ScriptTarget.ES2022,
+      module: ts.ModuleKind.ESNext,
+      moduleResolution: ts.ModuleResolutionKind.Bundler,
+      skipLibCheck: true,
+    },
+  );
+  return ts.getPreEmitDiagnostics(program);
+}
+
+async function genAndCompile(
+  dialect: Dialect,
+  pkField: unknown,
+  identity: unknown,
+): Promise<string[]> {
+  const root = await loadRoot(pkField, identity);
+  const dir = mkdtempSync(join(import.meta.dir, "tmp-assigned-pk-"));
+  try {
+    const renderContext = makeRenderContext({
+      dialect,
+      loadedRoot: root,
+      outDir: dir,
+      dbImport: "~/db",
+      pkMap: buildPkMap(root),
+      relationMap: buildRelationMap(root),
+    });
+    const ctx: GenContext = {
+      entities: root.objects(),
+      loadedRoot: root,
+      matches: () => true,
+      projectRoot: dir,
+      config: { outDir: dir, extStyle: "none", dbImport: "~/db", dialect } as never,
+      renderContext,
+      warn: () => {},
+    };
+    // The queries file declares its own `Db` type from drizzle, so entity+queries
+    // is a self-contained program — no dbImport module needs to resolve.
+    const files = [
+      ...(await entityFile({ allowlists: false }).generate(ctx)),
+      ...(await queriesFile().generate(ctx)),
+    ];
+    for (const f of files) writeFileSync(join(dir, f.path), f.content);
+    return compile(dir, files.map((f) => f.path)).map((d) =>
+      ts.flattenDiagnosticMessageText(d.messageText, "\n"),
+    );
+  } finally {
+    rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+const ASSIGNED_PK = { "identity.primary": { name: "pk", "@fields": "id" } };
+const UUID_GEN_PK = { "identity.primary": { name: "pk", "@fields": "id", "@generation": "uuid" } };
+const INCR_PK = { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } };
+
+const UUID_FIELD = { "field.uuid": { name: "id" } };
+const STRING_FIELD = { "field.string": { name: "id" } };
+const INT_FIELD = { "field.long": { name: "id" } };
+
+describe("assigned (non-generated) PK emits typechecking insert code (TS2769 guard)", () => {
+  for (const dialect of ["postgres", "sqlite"] as const) {
+    test(`${dialect}: uuid PK with NO @generation compiles`, async () => {
+      expect(await genAndCompile(dialect, UUID_FIELD, ASSIGNED_PK)).toEqual([]);
+    });
+
+    test(`${dialect}: string natural-key PK with NO @generation compiles`, async () => {
+      expect(await genAndCompile(dialect, STRING_FIELD, ASSIGNED_PK)).toEqual([]);
+    });
+
+    // Already-correct arms, pinned so the fix cannot regress them.
+    test(`${dialect}: uuid PK with @generation: uuid compiles (PK omitted from InsertSchema)`, async () => {
+      expect(await genAndCompile(dialect, UUID_FIELD, UUID_GEN_PK)).toEqual([]);
+    });
+
+    test(`${dialect}: int PK with @generation: increment compiles`, async () => {
+      expect(await genAndCompile(dialect, INT_FIELD, INCR_PK)).toEqual([]);
+    });
+  }
+});