fix(formula)!: the formula gate refuses bare date subtraction, which reads null or {} at runtime

.changeset/28944-formula-date-subtraction-refused.md

@@ -0,0 +1,34 @@
+---
+'@objectstack/formula': minor
+'@objectstack/spec': minor
+---
+
+fix(formula)!: the formula gate refuses bare date subtraction, which reads null or `{}` at runtime
+
+Clause-②: yes (narrowing)
+Metadata-protocol: narrowed
+
+<!-- adr-0087: registered formula-date-subtraction-refused -->
+
+**BREAKING** accept-set narrowing at the authoring type-soundness gate (`@objectstack/formula` `validateExpression`), shipped as `minor`. The gate declared a `date` / `datetime` field a CEL Timestamp, so `record.end_date - record.start_date` checked as a legal Duration. The runtime never holds a Timestamp there: every read door serves a date field as its `YYYY-MM-DD` text and a datetime field as its ISO-8601 instant text. So the subtraction faulted and the formula read `null` on every record, and `today() - record.start_date` evaluated to a Duration, which a field serialises as `{}`. Both shapes now get the same verdict at every door that runs the gate: `os validate`, `os build`, `os lint`, and the object save door with the other doors of the active authoring gate (`runRuntimeAuthoringRules`). Every door asks one judge.
+
+- **Before.** A formula `record.end_date - record.start_date` or `today() - record.start_date` built green (exit 0), and `PUT /api/v1/meta/object/NAME` answered 200. A read then returned `null` (with a server WARN) or `{}`. The same was true for two datetime fields, and for `now() - record.start_at`.
+- **Now.** The same formula is refused at build (exit 1) and at the save door (422 `INVALID_METADATA`, rule `expression-invalid`), located at the field's `expression`. There are two new refusal codes. `date-text-arithmetic` covers arithmetic on a date field's text that the evaluator faults on, in any role, so a predicate `record.end_date - record.start_date > duration("72h")` is refused too. `date-duration-value` covers a formula whose value is a `google.protobuf.Duration`, whether or not a date field is named (`today() - daysAgo(3)` too). Each message names the runtime outcome and prescribes `daysBetween(a, b)`.
+- **Unchanged.** `(record.end_date - record.start_date) + 1` and `record.d + 1` keep their `date-arithmetic` refusal word for word, because the existing check still answers first. Date comparisons (`<`, `==`, against a literal, against `today()`), concatenation, `daysBetween` / `addDays`, number arithmetic, and a predicate that compares a duration (`today() - record.start_date > duration("72h")`) all stay accepted.
+- **Who sees it.** An author whose formula subtracts dates. That formula already reads `null` or `{}` for every record. No shipped formula in this repository's `packages/` or `examples/` has this shape.
+
+## FROM → TO
+
+```ts
+// before: built green, read null (end minus start) or {} (since a date)
+span: Field.formula({ expression: 'record.end_date - record.start_date' }),
+age: Field.formula({ expression: 'today() - record.start_date' }),
+
+// after: the span in whole days
+span: Field.formula({ expression: 'daysBetween(record.start_date, record.end_date)' }),
+age: Field.formula({ expression: 'daysBetween(record.start_date, today())' }),
+```
+
+The upgrade guide entry is `formula-date-subtraction-refused`. There is no automatic rewrite, because which span the author meant (days, inclusive or not, or hours) is a judgement about the field.
+
+The closed refusal-code set (`EXPRESSION_REFUSAL_CODES`) gains `date-text-arithmetic` (`params: { operands, reference? }`) and `date-duration-value` (`params: { reference? }`).

content/docs/data-modeling/formulas.mdx

@@ -244,6 +244,7 @@ come at two severities: **errors** fail the build; **warnings** are advisory
 | **Unknown function** — a misspelled or nonexistent stdlib call. | `isBlnk(record.x)` | error |
 | **Type soundness** — a text or boolean field used with an arithmetic (`+ - * / %`) or ordering (`< > <= >=`) operator against a number faults at runtime and evaluates to `null`. Store it as a number field, or drop the arithmetic. | `record.title * 2` | warning |
 | **Date arithmetic** — a `date` / `datetime` field used with `+` / `-` against a number (a `YYYY-MM-DD` value is a string at runtime, so this always faulted to `null`). Use the date helpers instead. | `record.end_date - record.start_date + 1` → `daysBetween(record.start_date, record.end_date) + 1`; `today() + 30` → `daysFromNow(30)`; `record.date + n` → `addDays(record.date, n)` | error (#3306) |
+| **Date subtraction** — the runtime holds a `date` / `datetime` field as its text (`YYYY-MM-DD`, or an ISO-8601 instant), so subtracting two of them faults and the formula reads `null`; a timestamp minus one computes a duration, which a field reads as `{}`. Use `daysBetween`. A condition may still compare a duration (`today() - record.start_date > duration("72h")`). | `record.end_date - record.start_date` → `daysBetween(record.start_date, record.end_date)`; `today() - record.start_date` → `daysBetween(record.start_date, today())` | error |
 
 > **Integer literals in arithmetic are fine.** `record.amount / 100`,
 > `record.price * 2`, `record.total - fee` all work — the engine handles mixed

packages/formula/src/cel-engine.ts

@@ -965,7 +965,13 @@ const UNSOUND_OVERLOAD_RE = /no such overload:\s*([\w.]+)\s*(<=|>=|<|>|\+|-|\*|\
 function buildTypedEnv(
   fieldCelTypes: Readonly<Record<string, FieldCelType>>,
   scope: 'record' | 'flattened',
+  temporalAsText = false,
 ): Environment {
+  // `temporalAsText` declares a `date`/`datetime` field as the text the runtime
+  // holds ({@link DATE_TEXT_CEL_TYPE}) instead of a Timestamp — see
+  // {@link firstDateTextVerdict}.
+  const typeName = (t: FieldCelType): string =>
+    temporalAsText && t === 'timestamp' ? DATE_TEXT_CEL_TYPE : celTypeName(t);
   if (scope === 'flattened') {
     const env = new Environment({
       unlistedVariablesAreDyn: true,
@@ -974,13 +980,14 @@ function buildTypedEnv(
     });
     registerStdLib(env, () => new Date(0));
     registerNullComparisons(env);
+    if (temporalAsText) registerDateTextOverloads(env);
     for (const root of SCOPE_ROOTS) {
       try { env.registerVariable(root, 'map'); } catch { /* duplicate — ignore */ }
     }
     // Fields are bound bare at top level; a name that collides with a root
     // (unlikely) is skipped by the duplicate guard.
     for (const [name, t] of Object.entries(fieldCelTypes)) {
-      try { env.registerVariable(name, celTypeName(t)); } catch { /* duplicate / reserved — ignore */ }
+      try { env.registerVariable(name, typeName(t)); } catch { /* duplicate / reserved — ignore */ }
     }
     return env;
   }
@@ -991,8 +998,9 @@ function buildTypedEnv(
   });
   registerStdLib(env, () => new Date(0));
   registerNullComparisons(env);
+  if (temporalAsText) registerDateTextOverloads(env);
   const fields: Record<string, string> = {};
-  for (const [name, t] of Object.entries(fieldCelTypes)) fields[name] = celTypeName(t);
+  for (const [name, t] of Object.entries(fieldCelTypes)) fields[name] = typeName(t);
   try { env.registerType('OsRecordScope', { fields }); } catch { /* invalid field name — ignore */ }
   // The record namespaces carry the typed struct; every other root stays a
   // `map` (dyn members) so a reference through it never faults.
@@ -1051,7 +1059,10 @@ function offendingField(
  *    {@link UNSOUND_OVERLOAD_RE}), and concatenation (`"Due: " + date` → runtime
  *    string+string) are all runtime-tolerated, so they are never flagged. Two
  *    date fields (`date − date` → Duration, `date + date` → runtime string concat)
- *    also aren't flagged — only date/duration paired with a NUMBER.
+ *    also aren't flagged HERE — only date/duration paired with a NUMBER. This
+ *    typing declares a date field a Timestamp, which the runtime never holds;
+ *    {@link firstDateTextVerdict} re-checks what this passes with the field
+ *    declared as its text, and catches `date − date` there (#28944).
  *  - Equality (`==` / `!=`) is excluded ({@link UNSOUND_OVERLOAD_RE}): a
  *    heterogeneous equality is runtime-safe.
  *
@@ -1111,6 +1122,115 @@ export function firstTypeMismatch(
   }
 }
 
+/**
+ * The CEL type a `date` / `datetime` field is declared as by
+ * {@link firstDateTextVerdict}: what the runtime actually holds for it. Every
+ * read door hands a `Field.date` out as its `YYYY-MM-DD` text and a
+ * `Field.datetime` as its canonical ISO-8601 instant text (the SQL driver folds
+ * every dialect's storage to that text, ADR-0053 D-F1), so in an evaluated
+ * expression the operand is a string, never a Timestamp. A named type rather
+ * than plain `string` because the evaluator treats this text differently from a
+ * text field's: it coerces it to a date where the other operand is a Timestamp
+ * (`rewriteFaultedOperands`), which a text field's value never gets.
+ */
+const DATE_TEXT_CEL_TYPE = 'DateText';
+
+/**
+ * Declare, for the check-only env, the overloads the evaluator ANSWERS on a
+ * date field's text — and only those, so every other operator on it faults the
+ * check exactly where it faults the evaluator (measured on cel-js 8.0.0 through
+ * `celEngine.evaluate` over `{ d: '2026-11-01', at: '2026-11-01T10:00:00.000Z' }`):
+ *  - ordering against another date field or a text literal → string-lex, and
+ *    against a Timestamp → the text operand is coerced to a date: `bool`;
+ *  - equality against the same → `bool` (equality never faults);
+ *  - a Timestamp minus the text, or the text minus a Timestamp → the text
+ *    operand is coerced to a date, and the answer is a `Duration`;
+ *  - `+` with text, or with another date field's text → string concatenation.
+ * NOT declared, because the evaluator faults on them and the field reads null:
+ * the text minus the text (`record.end_date - record.start_date`, either kind),
+ * the text with a number (`record.d + 1`), with a Duration
+ * (`record.d - duration('24h')`) and a Timestamp plus the text.
+ * Handlers are no-ops: this env is checked, never evaluated.
+ */
+function registerDateTextOverloads(env: Environment): void {
+  const T = DATE_TEXT_CEL_TYPE;
+  const TS = 'google.protobuf.Timestamp';
+  const declare = (signature: string): void => {
+    try { env.registerOperator(signature, () => null); } catch { /* already/invalid — ignore */ }
+  };
+  try { env.registerType(T, { fields: {} }); } catch { /* already registered — ignore */ }
+  for (const op of ['<', '<=', '>', '>=', '==']) {
+    for (const other of [T, TS, 'string']) {
+      declare(`${T} ${op} ${other}: bool`);
+      if (other !== T) declare(`${other} ${op} ${T}: bool`);
+    }
+  }
+  declare(`${T} == null: bool`);
+  declare(`null == ${T}: bool`);
+  declare(`${TS} - ${T}: google.protobuf.Duration`);
+  declare(`${T} - ${TS}: google.protobuf.Duration`);
+  declare(`${T} + ${T}: string`);
+  declare(`${T} + string: string`);
+  declare(`string + ${T}: string`);
+}
+
+/**
+ * A verdict of the date-text check ({@link firstDateTextVerdict}):
+ *  - `fault` — arithmetic on a date field's text that the evaluator cannot
+ *    answer, so the expression faults and the field reads null;
+ *  - `type` — the expression checks clean, and `type` is the CEL type it
+ *    computes (`google.protobuf.Duration` for `today() - record.start_date`).
+ */
+export type DateTextVerdict =
+  | { readonly kind: 'fault'; readonly operator: string; readonly operands: string; readonly field: string | null }
+  | { readonly kind: 'type'; readonly type: string; readonly field: string | null };
+
+/**
+ * #28944 — type-check `source` with every `date`/`datetime` field declared as
+ * the text the runtime holds ({@link DATE_TEXT_CEL_TYPE}), not as the Timestamp
+ * {@link firstTypeMismatch} declares it.
+ *
+ * The Timestamp typing reads `record.end_date - record.start_date` as a legal
+ * `Duration`, so it is blind to the two shapes that cannot produce a value: the
+ * evaluator faults on the subtraction of two date texts (the field reads null),
+ * and `today() - record.start_date` evaluates to a `Duration`, which a field
+ * serialises as `{}`. Declared as what it is, the first faults this check
+ * exactly where it faults the evaluator, and the second's computed type is
+ * `Duration`; the caller words both. Nothing is matched in the source text.
+ *
+ * It answers only what the Timestamp typing passes: the caller asks
+ * {@link firstTypeMismatch} first, so a shape that check already refuses (the
+ * `… + 1` of `(record.end_date - record.start_date) + 1`) keeps its words.
+ *
+ * Returns `null` when the source does not parse, or the check faults on
+ * anything other than arithmetic with a date field's text — that is another
+ * check's to report, and this one never guesses.
+ */
+export function firstDateTextVerdict(
+  source: string,
+  fieldCelTypes: Readonly<Record<string, FieldCelType>>,
+  scope: 'record' | 'flattened' = 'record',
+): DateTextVerdict | null {
+  if (typeof source !== 'string' || !source.trim()) return null;
+  try {
+    const env = buildTypedEnv(fieldCelTypes, scope, true);
+    const result = env.parse(rewriteNullableTernary(rewriteIndexedHas(source))).check?.() as
+      | { valid?: boolean; type?: unknown; error?: { message?: string } }
+      | undefined;
+    if (!result) return null;
+    const field = offendingField(source, fieldCelTypes, 'timestamp', scope);
+    if (result.valid !== false) {
+      return typeof result.type === 'string' ? { kind: 'type', type: result.type, field } : null;
+    }
+    const m = UNSOUND_OVERLOAD_RE.exec(result.error?.message ?? '');
+    if (!m || !ARITH_OPS.has(m[2]) || (m[1] !== DATE_TEXT_CEL_TYPE && m[3] !== DATE_TEXT_CEL_TYPE)) return null;
+    return { kind: 'fault', operator: m[2], operands: `${m[1]} ${m[2]} ${m[3]}`, field };
+  } catch {
+    // A parse/other fault is the syntax checker's job (celEngine.compile).
+    return null;
+  }
+}
+
 /** cel-js temporal functions that return a calendar Timestamp (for #3183). */
 const TEMPORAL_FNS = new Set(['today', 'daysFromNow', 'daysAgo', 'now']);
 

packages/formula/src/expression-refusal-codes.test.ts

@@ -305,6 +305,53 @@ const PINS: { readonly [C in ExpressionRefusalCode]: readonly [Pin<C>, ...Pin<C>
         'in whole days, and `daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.',
     },
   ],
+  'date-text-arithmetic': [
+    {
+      produce: () =>
+        onlyError('value', 'record.end_date - record.start_date', {
+          scope: 'record',
+          fieldTypes: { start_date: 'date', end_date: 'date' },
+        }),
+      params: { operands: 'DateText - DateText', reference: 'record.start_date' },
+      message:
+        'date arithmetic `DateText - DateText` — `record.start_date` is a date field, and the runtime holds a date field ' +
+        'as its text (`YYYY-MM-DD`, or an ISO-8601 instant for a date-time), which CEL can\'t do arithmetic on: this ' +
+        'faults at runtime, so the field silently evaluates to null. Use `daysBetween(a, b)` for the span in whole days, ' +
+        'and `daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.',
+    },
+    {
+      produce: () =>
+        onlyError('predicate', 'end_at - start_at > duration("1h")', {
+          fields: ['start_at', 'end_at'],
+          fieldTypes: { start_at: 'datetime', end_at: 'datetime' },
+        }),
+      params: { operands: 'DateText - DateText', reference: 'start_at' },
+      message:
+        'date arithmetic `DateText - DateText` — `start_at` is a date field, and the runtime holds a date field ' +
+        'as its text (`YYYY-MM-DD`, or an ISO-8601 instant for a date-time), which CEL can\'t do arithmetic on: this ' +
+        'faults at runtime, so the field silently evaluates to null. Use `daysBetween(a, b)` for the span in whole days, ' +
+        'and `daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.',
+    },
+  ],
+  'date-duration-value': [
+    {
+      produce: () =>
+        onlyError('value', 'today() - record.start_date', { scope: 'record', fieldTypes: { start_date: 'date' } }),
+      params: { reference: 'record.start_date' },
+      message:
+        'date arithmetic — this computes a `google.protobuf.Duration` from `record.start_date`, and a field can\'t hold ' +
+        'a duration: it reads `{}` at runtime. Use `daysBetween(a, b)` for the span in whole days, and ' +
+        '`daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.',
+    },
+    {
+      produce: () => onlyError('value', 'today() - daysAgo(3)', { scope: 'record', fieldTypes: { name: 'text' } }),
+      params: {},
+      message:
+        'date arithmetic — this computes a `google.protobuf.Duration`, and a field can\'t hold ' +
+        'a duration: it reads `{}` at runtime. Use `daysBetween(a, b)` for the span in whole days, and ' +
+        '`daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.',
+    },
+  ],
   'type-mismatch': [
     {
       produce: () => onlyWarning('predicate', 'record.name > 5', { scope: 'record', fieldTypes: { name: 'text' } }),
@@ -460,6 +507,8 @@ const SWEEP_SOURCES: readonly string[] = [
   "current_user.positions.contains('zzz')",
   'record.close_date + 1',
   'close_date + 1 > 3',
+  'record.close_date - record.close_date',
+  'today() - record.close_date',
   'record.name > 5',
   'record.active + 1 > 5',
   "record.account == 'a1' && record.account.name == 'x'",

packages/formula/src/expression-refusal.ts

@@ -155,6 +155,19 @@ export interface ExpressionRefusalParams {
    * operand types; `reference` is the field as written, when it is known.
    */
   'date-arithmetic': { readonly operands: string; readonly reference?: string };
+  /**
+   * Arithmetic on a date field's runtime TEXT that the evaluator cannot answer
+   * (`record.end_date - record.start_date`), judged with the field declared as
+   * that text. `operands` are the CEL operand types (the text is `DateText`);
+   * `reference` is the field as written, when it is known.
+   */
+  'date-text-arithmetic': { readonly operands: string; readonly reference?: string };
+  /**
+   * A formula that computes a `google.protobuf.Duration`
+   * (`today() - record.start_date`), which a field reads as `{}`. `reference`
+   * is the date field it is computed from, when the expression names one.
+   */
+  'date-duration-value': { readonly reference?: string };
   /**
    * WARNING, never an error: a text or boolean field used with an arithmetic
    * or ordering `operator` against a number.
@@ -224,6 +237,8 @@ const CODE_TABLE = {
   'unbound-root': true,
   'bare-reference': true,
   'date-arithmetic': true,
+  'date-text-arithmetic': true,
+  'date-duration-value': true,
   'type-mismatch': true,
   'field-near-miss': true,
   'traversal-bare-and-traversed': true,