diff --git a/.changeset/28944-formula-date-subtraction-refused.md b/.changeset/28944-formula-date-subtraction-refused.md new file mode 100644 index 00000000000..22d8c3623f0 --- /dev/null +++ b/.changeset/28944-formula-date-subtraction-refused.md @@ -0,0 +1,34 @@ +--- +'@objectstack/formula': minor +'@objectstack/spec': minor +--- + +fix(formula)!: the formula gate refuses bare date subtraction, which reads null or `{}` at runtime + +Clause-②: yes (narrowing) +Metadata-protocol: narrowed + + + +**BREAKING** accept-set narrowing at the authoring type-soundness gate (`@objectstack/formula` `validateExpression`), shipped as `minor`. The gate declared a `date` / `datetime` field a CEL Timestamp, so `record.end_date - record.start_date` checked as a legal Duration. The runtime never holds a Timestamp there: every read door serves a date field as its `YYYY-MM-DD` text and a datetime field as its ISO-8601 instant text. So the subtraction faulted and the formula read `null` on every record, and `today() - record.start_date` evaluated to a Duration, which a field serialises as `{}`. Both shapes now get the same verdict at every door that runs the gate: `os validate`, `os build`, `os lint`, and the object save door with the other doors of the active authoring gate (`runRuntimeAuthoringRules`). Every door asks one judge. + +- **Before.** A formula `record.end_date - record.start_date` or `today() - record.start_date` built green (exit 0), and `PUT /api/v1/meta/object/NAME` answered 200. A read then returned `null` (with a server WARN) or `{}`. The same was true for two datetime fields, and for `now() - record.start_at`. +- **Now.** The same formula is refused at build (exit 1) and at the save door (422 `INVALID_METADATA`, rule `expression-invalid`), located at the field's `expression`. There are two new refusal codes. `date-text-arithmetic` covers arithmetic on a date field's text that the evaluator faults on, in any role, so a predicate `record.end_date - record.start_date > duration("72h")` is refused too. `date-duration-value` covers a formula whose value is a `google.protobuf.Duration`, whether or not a date field is named (`today() - daysAgo(3)` too). Each message names the runtime outcome and prescribes `daysBetween(a, b)`. +- **Unchanged.** `(record.end_date - record.start_date) + 1` and `record.d + 1` keep their `date-arithmetic` refusal word for word, because the existing check still answers first. Date comparisons (`<`, `==`, against a literal, against `today()`), concatenation, `daysBetween` / `addDays`, number arithmetic, and a predicate that compares a duration (`today() - record.start_date > duration("72h")`) all stay accepted. +- **Who sees it.** An author whose formula subtracts dates. That formula already reads `null` or `{}` for every record. No shipped formula in this repository's `packages/` or `examples/` has this shape. + +## FROM → TO + +```ts +// before: built green, read null (end minus start) or {} (since a date) +span: Field.formula({ expression: 'record.end_date - record.start_date' }), +age: Field.formula({ expression: 'today() - record.start_date' }), + +// after: the span in whole days +span: Field.formula({ expression: 'daysBetween(record.start_date, record.end_date)' }), +age: Field.formula({ expression: 'daysBetween(record.start_date, today())' }), +``` + +The upgrade guide entry is `formula-date-subtraction-refused`. There is no automatic rewrite, because which span the author meant (days, inclusive or not, or hours) is a judgement about the field. + +The closed refusal-code set (`EXPRESSION_REFUSAL_CODES`) gains `date-text-arithmetic` (`params: { operands, reference? }`) and `date-duration-value` (`params: { reference? }`). diff --git a/content/docs/data-modeling/formulas.mdx b/content/docs/data-modeling/formulas.mdx index e71f44f370d..cb0804af1e2 100644 --- a/content/docs/data-modeling/formulas.mdx +++ b/content/docs/data-modeling/formulas.mdx @@ -244,6 +244,7 @@ come at two severities: **errors** fail the build; **warnings** are advisory | **Unknown function** — a misspelled or nonexistent stdlib call. | `isBlnk(record.x)` | error | | **Type soundness** — a text or boolean field used with an arithmetic (`+ - * / %`) or ordering (`< > <= >=`) operator against a number faults at runtime and evaluates to `null`. Store it as a number field, or drop the arithmetic. | `record.title * 2` | warning | | **Date arithmetic** — a `date` / `datetime` field used with `+` / `-` against a number (a `YYYY-MM-DD` value is a string at runtime, so this always faulted to `null`). Use the date helpers instead. | `record.end_date - record.start_date + 1` → `daysBetween(record.start_date, record.end_date) + 1`; `today() + 30` → `daysFromNow(30)`; `record.date + n` → `addDays(record.date, n)` | error (#3306) | +| **Date subtraction** — the runtime holds a `date` / `datetime` field as its text (`YYYY-MM-DD`, or an ISO-8601 instant), so subtracting two of them faults and the formula reads `null`; a timestamp minus one computes a duration, which a field reads as `{}`. Use `daysBetween`. A condition may still compare a duration (`today() - record.start_date > duration("72h")`). | `record.end_date - record.start_date` → `daysBetween(record.start_date, record.end_date)`; `today() - record.start_date` → `daysBetween(record.start_date, today())` | error | > **Integer literals in arithmetic are fine.** `record.amount / 100`, > `record.price * 2`, `record.total - fee` all work — the engine handles mixed diff --git a/packages/formula/src/cel-engine.ts b/packages/formula/src/cel-engine.ts index 98c3e2e7f76..d48b48384db 100644 --- a/packages/formula/src/cel-engine.ts +++ b/packages/formula/src/cel-engine.ts @@ -965,7 +965,13 @@ const UNSOUND_OVERLOAD_RE = /no such overload:\s*([\w.]+)\s*(<=|>=|<|>|\+|-|\*|\ function buildTypedEnv( fieldCelTypes: Readonly>, scope: 'record' | 'flattened', + temporalAsText = false, ): Environment { + // `temporalAsText` declares a `date`/`datetime` field as the text the runtime + // holds ({@link DATE_TEXT_CEL_TYPE}) instead of a Timestamp — see + // {@link firstDateTextVerdict}. + const typeName = (t: FieldCelType): string => + temporalAsText && t === 'timestamp' ? DATE_TEXT_CEL_TYPE : celTypeName(t); if (scope === 'flattened') { const env = new Environment({ unlistedVariablesAreDyn: true, @@ -974,13 +980,14 @@ function buildTypedEnv( }); registerStdLib(env, () => new Date(0)); registerNullComparisons(env); + if (temporalAsText) registerDateTextOverloads(env); for (const root of SCOPE_ROOTS) { try { env.registerVariable(root, 'map'); } catch { /* duplicate — ignore */ } } // Fields are bound bare at top level; a name that collides with a root // (unlikely) is skipped by the duplicate guard. for (const [name, t] of Object.entries(fieldCelTypes)) { - try { env.registerVariable(name, celTypeName(t)); } catch { /* duplicate / reserved — ignore */ } + try { env.registerVariable(name, typeName(t)); } catch { /* duplicate / reserved — ignore */ } } return env; } @@ -991,8 +998,9 @@ function buildTypedEnv( }); registerStdLib(env, () => new Date(0)); registerNullComparisons(env); + if (temporalAsText) registerDateTextOverloads(env); const fields: Record = {}; - for (const [name, t] of Object.entries(fieldCelTypes)) fields[name] = celTypeName(t); + for (const [name, t] of Object.entries(fieldCelTypes)) fields[name] = typeName(t); try { env.registerType('OsRecordScope', { fields }); } catch { /* invalid field name — ignore */ } // The record namespaces carry the typed struct; every other root stays a // `map` (dyn members) so a reference through it never faults. @@ -1051,7 +1059,10 @@ function offendingField( * {@link UNSOUND_OVERLOAD_RE}), and concatenation (`"Due: " + date` → runtime * string+string) are all runtime-tolerated, so they are never flagged. Two * date fields (`date − date` → Duration, `date + date` → runtime string concat) - * also aren't flagged — only date/duration paired with a NUMBER. + * also aren't flagged HERE — only date/duration paired with a NUMBER. This + * typing declares a date field a Timestamp, which the runtime never holds; + * {@link firstDateTextVerdict} re-checks what this passes with the field + * declared as its text, and catches `date − date` there (#28944). * - Equality (`==` / `!=`) is excluded ({@link UNSOUND_OVERLOAD_RE}): a * heterogeneous equality is runtime-safe. * @@ -1111,6 +1122,115 @@ export function firstTypeMismatch( } } +/** + * The CEL type a `date` / `datetime` field is declared as by + * {@link firstDateTextVerdict}: what the runtime actually holds for it. Every + * read door hands a `Field.date` out as its `YYYY-MM-DD` text and a + * `Field.datetime` as its canonical ISO-8601 instant text (the SQL driver folds + * every dialect's storage to that text, ADR-0053 D-F1), so in an evaluated + * expression the operand is a string, never a Timestamp. A named type rather + * than plain `string` because the evaluator treats this text differently from a + * text field's: it coerces it to a date where the other operand is a Timestamp + * (`rewriteFaultedOperands`), which a text field's value never gets. + */ +const DATE_TEXT_CEL_TYPE = 'DateText'; + +/** + * Declare, for the check-only env, the overloads the evaluator ANSWERS on a + * date field's text — and only those, so every other operator on it faults the + * check exactly where it faults the evaluator (measured on cel-js 8.0.0 through + * `celEngine.evaluate` over `{ d: '2026-11-01', at: '2026-11-01T10:00:00.000Z' }`): + * - ordering against another date field or a text literal → string-lex, and + * against a Timestamp → the text operand is coerced to a date: `bool`; + * - equality against the same → `bool` (equality never faults); + * - a Timestamp minus the text, or the text minus a Timestamp → the text + * operand is coerced to a date, and the answer is a `Duration`; + * - `+` with text, or with another date field's text → string concatenation. + * NOT declared, because the evaluator faults on them and the field reads null: + * the text minus the text (`record.end_date - record.start_date`, either kind), + * the text with a number (`record.d + 1`), with a Duration + * (`record.d - duration('24h')`) and a Timestamp plus the text. + * Handlers are no-ops: this env is checked, never evaluated. + */ +function registerDateTextOverloads(env: Environment): void { + const T = DATE_TEXT_CEL_TYPE; + const TS = 'google.protobuf.Timestamp'; + const declare = (signature: string): void => { + try { env.registerOperator(signature, () => null); } catch { /* already/invalid — ignore */ } + }; + try { env.registerType(T, { fields: {} }); } catch { /* already registered — ignore */ } + for (const op of ['<', '<=', '>', '>=', '==']) { + for (const other of [T, TS, 'string']) { + declare(`${T} ${op} ${other}: bool`); + if (other !== T) declare(`${other} ${op} ${T}: bool`); + } + } + declare(`${T} == null: bool`); + declare(`null == ${T}: bool`); + declare(`${TS} - ${T}: google.protobuf.Duration`); + declare(`${T} - ${TS}: google.protobuf.Duration`); + declare(`${T} + ${T}: string`); + declare(`${T} + string: string`); + declare(`string + ${T}: string`); +} + +/** + * A verdict of the date-text check ({@link firstDateTextVerdict}): + * - `fault` — arithmetic on a date field's text that the evaluator cannot + * answer, so the expression faults and the field reads null; + * - `type` — the expression checks clean, and `type` is the CEL type it + * computes (`google.protobuf.Duration` for `today() - record.start_date`). + */ +export type DateTextVerdict = + | { readonly kind: 'fault'; readonly operator: string; readonly operands: string; readonly field: string | null } + | { readonly kind: 'type'; readonly type: string; readonly field: string | null }; + +/** + * #28944 — type-check `source` with every `date`/`datetime` field declared as + * the text the runtime holds ({@link DATE_TEXT_CEL_TYPE}), not as the Timestamp + * {@link firstTypeMismatch} declares it. + * + * The Timestamp typing reads `record.end_date - record.start_date` as a legal + * `Duration`, so it is blind to the two shapes that cannot produce a value: the + * evaluator faults on the subtraction of two date texts (the field reads null), + * and `today() - record.start_date` evaluates to a `Duration`, which a field + * serialises as `{}`. Declared as what it is, the first faults this check + * exactly where it faults the evaluator, and the second's computed type is + * `Duration`; the caller words both. Nothing is matched in the source text. + * + * It answers only what the Timestamp typing passes: the caller asks + * {@link firstTypeMismatch} first, so a shape that check already refuses (the + * `… + 1` of `(record.end_date - record.start_date) + 1`) keeps its words. + * + * Returns `null` when the source does not parse, or the check faults on + * anything other than arithmetic with a date field's text — that is another + * check's to report, and this one never guesses. + */ +export function firstDateTextVerdict( + source: string, + fieldCelTypes: Readonly>, + scope: 'record' | 'flattened' = 'record', +): DateTextVerdict | null { + if (typeof source !== 'string' || !source.trim()) return null; + try { + const env = buildTypedEnv(fieldCelTypes, scope, true); + const result = env.parse(rewriteNullableTernary(rewriteIndexedHas(source))).check?.() as + | { valid?: boolean; type?: unknown; error?: { message?: string } } + | undefined; + if (!result) return null; + const field = offendingField(source, fieldCelTypes, 'timestamp', scope); + if (result.valid !== false) { + return typeof result.type === 'string' ? { kind: 'type', type: result.type, field } : null; + } + const m = UNSOUND_OVERLOAD_RE.exec(result.error?.message ?? ''); + if (!m || !ARITH_OPS.has(m[2]) || (m[1] !== DATE_TEXT_CEL_TYPE && m[3] !== DATE_TEXT_CEL_TYPE)) return null; + return { kind: 'fault', operator: m[2], operands: `${m[1]} ${m[2]} ${m[3]}`, field }; + } catch { + // A parse/other fault is the syntax checker's job (celEngine.compile). + return null; + } +} + /** cel-js temporal functions that return a calendar Timestamp (for #3183). */ const TEMPORAL_FNS = new Set(['today', 'daysFromNow', 'daysAgo', 'now']); diff --git a/packages/formula/src/expression-refusal-codes.test.ts b/packages/formula/src/expression-refusal-codes.test.ts index 9776bfbb1f9..f9ff4df6c64 100644 --- a/packages/formula/src/expression-refusal-codes.test.ts +++ b/packages/formula/src/expression-refusal-codes.test.ts @@ -305,6 +305,53 @@ const PINS: { readonly [C in ExpressionRefusalCode]: readonly [Pin, ...Pin 'in whole days, and `daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.', }, ], + 'date-text-arithmetic': [ + { + produce: () => + onlyError('value', 'record.end_date - record.start_date', { + scope: 'record', + fieldTypes: { start_date: 'date', end_date: 'date' }, + }), + params: { operands: 'DateText - DateText', reference: 'record.start_date' }, + message: + 'date arithmetic `DateText - DateText` — `record.start_date` is a date field, and the runtime holds a date field ' + + 'as its text (`YYYY-MM-DD`, or an ISO-8601 instant for a date-time), which CEL can\'t do arithmetic on: this ' + + 'faults at runtime, so the field silently evaluates to null. Use `daysBetween(a, b)` for the span in whole days, ' + + 'and `daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.', + }, + { + produce: () => + onlyError('predicate', 'end_at - start_at > duration("1h")', { + fields: ['start_at', 'end_at'], + fieldTypes: { start_at: 'datetime', end_at: 'datetime' }, + }), + params: { operands: 'DateText - DateText', reference: 'start_at' }, + message: + 'date arithmetic `DateText - DateText` — `start_at` is a date field, and the runtime holds a date field ' + + 'as its text (`YYYY-MM-DD`, or an ISO-8601 instant for a date-time), which CEL can\'t do arithmetic on: this ' + + 'faults at runtime, so the field silently evaluates to null. Use `daysBetween(a, b)` for the span in whole days, ' + + 'and `daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.', + }, + ], + 'date-duration-value': [ + { + produce: () => + onlyError('value', 'today() - record.start_date', { scope: 'record', fieldTypes: { start_date: 'date' } }), + params: { reference: 'record.start_date' }, + message: + 'date arithmetic — this computes a `google.protobuf.Duration` from `record.start_date`, and a field can\'t hold ' + + 'a duration: it reads `{}` at runtime. Use `daysBetween(a, b)` for the span in whole days, and ' + + '`daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.', + }, + { + produce: () => onlyError('value', 'today() - daysAgo(3)', { scope: 'record', fieldTypes: { name: 'text' } }), + params: {}, + message: + 'date arithmetic — this computes a `google.protobuf.Duration`, and a field can\'t hold ' + + 'a duration: it reads `{}` at runtime. Use `daysBetween(a, b)` for the span in whole days, and ' + + '`daysFromNow(n)` / `addDays(d, n)` / `addMonths(d, n)` to shift a date.', + }, + ], 'type-mismatch': [ { produce: () => onlyWarning('predicate', 'record.name > 5', { scope: 'record', fieldTypes: { name: 'text' } }), @@ -460,6 +507,8 @@ const SWEEP_SOURCES: readonly string[] = [ "current_user.positions.contains('zzz')", 'record.close_date + 1', 'close_date + 1 > 3', + 'record.close_date - record.close_date', + 'today() - record.close_date', 'record.name > 5', 'record.active + 1 > 5', "record.account == 'a1' && record.account.name == 'x'", diff --git a/packages/formula/src/expression-refusal.ts b/packages/formula/src/expression-refusal.ts index dff221e9926..0f369f678f8 100644 --- a/packages/formula/src/expression-refusal.ts +++ b/packages/formula/src/expression-refusal.ts @@ -155,6 +155,19 @@ export interface ExpressionRefusalParams { * operand types; `reference` is the field as written, when it is known. */ 'date-arithmetic': { readonly operands: string; readonly reference?: string }; + /** + * Arithmetic on a date field's runtime TEXT that the evaluator cannot answer + * (`record.end_date - record.start_date`), judged with the field declared as + * that text. `operands` are the CEL operand types (the text is `DateText`); + * `reference` is the field as written, when it is known. + */ + 'date-text-arithmetic': { readonly operands: string; readonly reference?: string }; + /** + * A formula that computes a `google.protobuf.Duration` + * (`today() - record.start_date`), which a field reads as `{}`. `reference` + * is the date field it is computed from, when the expression names one. + */ + 'date-duration-value': { readonly reference?: string }; /** * WARNING, never an error: a text or boolean field used with an arithmetic * or ordering `operator` against a number. @@ -224,6 +237,8 @@ const CODE_TABLE = { 'unbound-root': true, 'bare-reference': true, 'date-arithmetic': true, + 'date-text-arithmetic': true, + 'date-duration-value': true, 'type-mismatch': true, 'field-near-miss': true, 'traversal-bare-and-traversed': true, diff --git a/packages/formula/src/validate-date-text-arithmetic.test.ts b/packages/formula/src/validate-date-text-arithmetic.test.ts new file mode 100644 index 00000000000..3603db7787c --- /dev/null +++ b/packages/formula/src/validate-date-text-arithmetic.test.ts @@ -0,0 +1,143 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Date subtraction the runtime cannot answer is refused at author time (#28944). + * + * The type-soundness gate declared a `date` / `datetime` field a Timestamp, so + * `record.end_date - record.start_date` checked as a legal `Duration` and + * built. The runtime never holds a Timestamp there: every read door hands a + * date field out as its text (`YYYY-MM-DD`, or the ISO-8601 instant for a + * date-time), so the evaluator faults on the subtraction (the formula reads + * null), and `today() - record.start_date` (the text coerced against a + * Timestamp) computes a `Duration`, which a field serialises as `{}`. + * `firstDateTextVerdict` re-checks what the Timestamp typing passes with the + * field declared as that text, so the gate's verdict falls out of the same + * typing the evaluator applies. + * + * Every row is judged twice — by the gate and by the evaluator over the row a + * read door serves — so the table pins that the gate refuses exactly the + * shapes that cannot produce a value, and nothing the evaluator answers. + * Message pins live with the other code pins in + * `expression-refusal-codes.test.ts`. + */ +import { describe, expect, it } from 'vitest'; +import { celEngine } from './cel-engine'; +import { validateExpression, type ExprSchemaHint } from './validate'; + +const RECORD_SCOPE: ExprSchemaHint = { + scope: 'record', + objectName: 'qa_doctor_time_off', + fields: ['title', 'start_date', 'end_date', 'start_at', 'end_at', 'amount', 'cost'], + fieldTypes: { + title: 'text', + start_date: 'date', + end_date: 'date', + start_at: 'datetime', + end_at: 'datetime', + amount: 'number', + cost: 'currency', + }, +}; + +/** The row a read door serves for those fields: dates as their text. */ +const SERVED_ROW = { + title: 't1', + start_date: '2026-11-01', + end_date: '2026-11-05', + start_at: '2026-11-01T10:00:00.000Z', + end_at: '2026-11-05T12:00:00.000Z', + amount: 10, + cost: 4, +}; + +/** What the evaluator gives a field computed from `source` over the served row. */ +function runtimeReading(source: string): 'null' | '{}' | 'value' { + const r = celEngine.evaluate({ dialect: 'cel', source }, { record: SERVED_ROW }); + if (!r.ok) return 'null'; + return JSON.stringify(r.value) === '{}' ? '{}' : 'value'; +} + +function codesOf(role: 'value' | 'predicate', source: string): string[] { + const r = validateExpression(role, source, RECORD_SCOPE); + return r.errors.map((e) => e.code); +} + +describe('date subtraction the runtime cannot answer (#28944)', () => { + // [role, source, the code the gate answers (null = clean), what the evaluator reads] + const ROWS: ReadonlyArray = [ + // The card's shapes, refused. + ['value', 'record.end_date - record.start_date', 'date-text-arithmetic', 'null'], + ['value', 'today() - record.start_date', 'date-duration-value', '{}'], + // Datetime, measured: its text faults the same way, and a Timestamp minus it is a Duration. + ['value', 'record.end_at - record.start_at', 'date-text-arithmetic', 'null'], + ['value', 'now() - record.start_at', 'date-duration-value', '{}'], + ['value', 'record.start_date - today()', 'date-duration-value', '{}'], + // The subtraction feeding a comparison still cannot produce a value. + ['predicate', 'record.end_date - record.start_date > duration("72h")', 'date-text-arithmetic', 'null'], + ['value', 'record.start_date - duration("24h")', 'date-text-arithmetic', 'null'], + // The `+ 1` shape keeps its own refusal (asserted word for word below). + ['value', '(record.end_date - record.start_date) + 1', 'date-arithmetic', 'null'], + ['value', '(today() - record.start_date) + 1', 'date-arithmetic', 'null'], + // Controls: every shape the evaluator answers stays clean. + ['value', 'daysBetween(record.start_date, record.end_date)', null, 'value'], + ['value', 'daysBetween(record.start_date, today())', null, 'value'], + ['predicate', 'today() - record.start_date > duration("72h")', null, 'value'], + ['value', '(today() - record.start_date).getHours()', null, 'value'], + ['predicate', 'record.end_date < record.start_date', null, 'value'], + ['predicate', 'record.start_date < today()', null, 'value'], + ['predicate', 'record.start_at > now()', null, 'value'], + ['predicate', 'record.start_date >= "2026-01-01"', null, 'value'], + ['predicate', 'record.start_date == "2026-11-01"', null, 'value'], + ['predicate', 'record.start_date != null && record.end_date != null', null, 'value'], + ['value', '"Due: " + record.end_date', null, 'value'], + ['value', 'addDays(record.start_date, 3)', null, 'value'], + ['value', 'record.amount - record.cost', null, 'value'], + ]; + + for (const [role, source, code, reads] of ROWS) { + it(`${role} \`${source}\` → ${code ?? 'clean'} (the evaluator reads ${reads})`, () => { + expect(runtimeReading(source)).toBe(reads); + expect(codesOf(role, source)).toEqual(code === null ? [] : [code]); + }); + } + + it('the `+ 1` shape keeps its words: the Timestamp typing still answers first', () => { + const r = validateExpression('value', '(record.end_date - record.start_date) + 1', RECORD_SCOPE); + expect(r.errors).toHaveLength(1); + expect(r.errors[0].code).toBe('date-arithmetic'); + expect(r.errors[0].params).toEqual({ operands: 'google.protobuf.Duration + int', reference: 'record.start_date' }); + }); + + it('names the field and prescribes daysBetween', () => { + const [span] = validateExpression('value', 'record.end_date - record.start_date', RECORD_SCOPE).errors; + expect(span.params).toEqual({ operands: 'DateText - DateText', reference: 'record.start_date' }); + expect(span.message).toMatch(/evaluates to null/); + expect(span.message).toMatch(/daysBetween\(a, b\)/); + const [since] = validateExpression('value', 'today() - record.start_date', RECORD_SCOPE).errors; + expect(since.params).toEqual({ reference: 'record.start_date' }); + expect(since.message).toMatch(/reads `\{\}`/); + expect(since.message).toMatch(/daysBetween\(a, b\)/); + }); + + it('judges a flattened (flow) condition the same way', () => { + const flattened: ExprSchemaHint = { + fields: ['start_date', 'end_date'], + fieldTypes: { start_date: 'date', end_date: 'date' }, + }; + const refused = validateExpression('predicate', 'end_date - start_date > duration("72h")', flattened); + expect(refused.errors.map((e) => e.code)).toEqual(['date-text-arithmetic']); + expect(refused.errors[0].params).toEqual({ operands: 'DateText - DateText', reference: 'start_date' }); + expect(validateExpression('predicate', 'today() - start_date > duration("72h")', flattened).errors).toEqual([]); + }); + + it('a predicate that computes a duration is not a formula value, so only `value` refuses it', () => { + // A predicate's computed type is another check's question; the `{}` reading is a field's. + expect(codesOf('predicate', 'today() - record.start_date')).toEqual([]); + expect(codesOf('value', 'today() - record.start_date')).toEqual(['date-duration-value']); + }); + + it('without field types the check does not run, as before', () => { + const r = validateExpression('value', 'record.end_date - record.start_date', { scope: 'record' }); + expect(r.errors).toEqual([]); + }); +}); diff --git a/packages/formula/src/validate.test.ts b/packages/formula/src/validate.test.ts index 832c4f069ec..89d835302bf 100644 --- a/packages/formula/src/validate.test.ts +++ b/packages/formula/src/validate.test.ts @@ -900,7 +900,9 @@ describe('validateExpression (ADR-0032)', () => { 'record.end_date < "2026-01-01"', 'record.end_date == today()', '"Due: " + record.end_date', - 'record.end_date - record.start_date', // → Duration, no further arith + // `record.end_date - record.start_date` is no longer here: the runtime + // holds a date field as its text and faults on it (#28944, + // `validate-date-text-arithmetic.test.ts`). ]) { const r = validateExpression('value', src, schema); expect(r.errors, `should be clean: ${src}`).toHaveLength(0); diff --git a/packages/formula/src/validate.ts b/packages/formula/src/validate.ts index 131a3ef5605..787a1532d20 100644 --- a/packages/formula/src/validate.ts +++ b/packages/formula/src/validate.ts @@ -20,6 +20,7 @@ import { compileCel, firstUndeclaredReference, + firstDateTextVerdict, firstTypeMismatch, inferCelType, parseCelToAst, @@ -308,15 +309,20 @@ function toCelFieldTypes(fieldTypes: Readonly>): Record` vs bare field binding, shaping the message. */ function typeSoundnessIssue( source: string, fieldTypes: Readonly>, scope: 'record' | 'flattened', + role: 'predicate' | 'value', ): { issue: ExprValidationError; severity: 'error' | 'warning' } | null { - const mismatch = firstTypeMismatch(source, toCelFieldTypes(fieldTypes), scope); - if (!mismatch) return null; + const celFieldTypes = toCelFieldTypes(fieldTypes); + const mismatch = firstTypeMismatch(source, celFieldTypes, scope); + if (!mismatch) return dateTextIssue(source, celFieldTypes, scope, role); const reference = mismatch.field ? (scope === 'record' ? `record.${mismatch.field}` : mismatch.field) : undefined; @@ -357,6 +363,63 @@ function typeSoundnessIssue( }; } +/** + * #28944 — the date shapes the Timestamp typing of {@link firstTypeMismatch} + * passes and the runtime cannot answer, judged by {@link firstDateTextVerdict} + * with every `date`/`datetime` field declared as the text the runtime holds: + * - `date-text-arithmetic`: arithmetic on that text the evaluator faults on — + * `record.end_date - record.start_date` (either field kind), `record.d - + * duration('24h')` — so the expression reads null. Any role. + * - `date-duration-value`: a formula (`value` role) that computes a + * `google.protobuf.Duration` — `today() - record.start_date` — which a field + * serialises as `{}`. A predicate that COMPARES a duration + * (`today() - record.d > duration('72h')`) evaluates, and is not touched. + * Runs only when there is something to find: a date field to retype, or a + * `value` whose computed type is the question. + */ +function dateTextIssue( + source: string, + celFieldTypes: Readonly>, + scope: 'record' | 'flattened', + role: 'predicate' | 'value', +): { issue: ExprValidationError; severity: 'error' | 'warning' } | null { + if (role !== 'value' && !Object.values(celFieldTypes).includes('timestamp')) return null; + const verdict = firstDateTextVerdict(source, celFieldTypes, scope); + if (!verdict) return null; + const reference = verdict.field + ? (scope === 'record' ? `record.${verdict.field}` : verdict.field) + : undefined; + const ref = reference !== undefined ? `\`${reference}\`` : null; + const shift = `\`daysFromNow(n)\` / \`addDays(d, n)\` / \`addMonths(d, n)\` to shift a date.`; + if (verdict.kind === 'fault') { + const subject = ref ? `${ref} is a date field` : 'a date field is an operand'; + return { + severity: 'error', + issue: refusal( + 'date-text-arithmetic', + { operands: verdict.operands, ...(reference !== undefined ? { reference } : {}) }, + source, + `date arithmetic \`${verdict.operands}\` — ${subject}, and the runtime holds a date field as ` + + `its text (\`YYYY-MM-DD\`, or an ISO-8601 instant for a date-time), which CEL can't do ` + + `arithmetic on: this faults at runtime, so the field silently evaluates to null. Use ` + + `\`daysBetween(a, b)\` for the span in whole days, and ${shift}`, + ), + }; + } + if (role !== 'value' || verdict.type !== 'google.protobuf.Duration') return null; + return { + severity: 'error', + issue: refusal( + 'date-duration-value', + reference !== undefined ? { reference } : {}, + source, + `date arithmetic — this computes a \`google.protobuf.Duration\`${ref ? ` from ${ref}` : ''}, and a ` + + `field can't hold a duration: it reads \`{}\` at runtime. Use \`daysBetween(a, b)\` for the ` + + `span in whole days, and ${shift}`, + ), + }; +} + /** A bare `{x}` that is NOT part of a `{{x}}` mustache hole. */ const SINGLE_BRACE_RE = /(?:^|[^{])\{\s*([A-Za-z_$][\w.$]*)\s*\}(?!\})/; /** The roots whose members the field-existence pass judges against `schema.fields`. */ @@ -1148,7 +1211,7 @@ export function validateExpression( // numeric text value can succeed), or date-field arithmetic (hard error — // always nulls). Only runs when there is no bare-ref error (the typed // check needs the canonical `record.` form). - const r = typeSoundnessIssue(source, schema.fieldTypes, 'record'); + const r = typeSoundnessIssue(source, schema.fieldTypes, 'record', role); if (r) (r.severity === 'error' ? errors : warnings).push(r.issue); } } else if (schema?.fields && schema.fields.length > 0) { @@ -1183,7 +1246,7 @@ export function validateExpression( // date-field arithmetic (error). Flow variables stay `dyn` (never flagged); // equality/ordering of a date is runtime-safe (never flagged). if (schema.fieldTypes) { - const r = typeSoundnessIssue(source, schema.fieldTypes, 'flattened'); + const r = typeSoundnessIssue(source, schema.fieldTypes, 'flattened', role); if (r) (r.severity === 'error' ? errors : warnings).push(r.issue); } } diff --git a/packages/lint/src/validate-expressions.test.ts b/packages/lint/src/validate-expressions.test.ts index 4e39adda3cb..4deb11c026f 100644 --- a/packages/lint/src/validate-expressions.test.ts +++ b/packages/lint/src/validate-expressions.test.ts @@ -302,6 +302,34 @@ describe('validateStackExpressions (ADR-0032 build-time)', () => { expect(issues[0].message).toMatch(/daysBetween/); }); + // #28944 — the bare subtraction, and a timestamp minus a date field: the + // runtime holds a date field as its text, so the first reads null and the + // second a Duration (`{}`). Both are refused by the stack gate, located at + // the formula's expression; the daysBetween form beside them stays clean. + it('flags bare date subtraction and a timestamp minus a date field in a formula field', () => { + const issues = validateStackExpressions({ + objects: [{ + name: 'qa_doctor_time_off', + fields: { + start_date: { type: 'date' }, + end_date: { type: 'date' }, + qa_span_bare: { type: 'formula', name: 'qa_span_bare', expression: 'record.end_date - record.start_date' }, + qa_span_daysbetween: { + type: 'formula', name: 'qa_span_daysbetween', expression: 'daysBetween(record.start_date, record.end_date)', + }, + qa_since_today: { type: 'formula', name: 'qa_since_today', expression: 'today() - record.start_date' }, + }, + }], + }); + expect(issues.map((i) => [i.where, i.severity])).toEqual([ + ["object 'qa_doctor_time_off' · field 'qa_span_bare' expression", 'error'], + ["object 'qa_doctor_time_off' · field 'qa_since_today' expression", 'error'], + ]); + expect(issues[0].message).toMatch(/evaluates to null/); + expect(issues[1].message).toMatch(/reads `\{\}`/); + for (const issue of issues) expect(issue.message).toMatch(/daysBetween\(a, b\)/); + }); + it('accepts the daysBetween rewrite of that formula', () => { const issues = validateStackExpressions({ objects: [{ diff --git a/packages/spec/src/migrations/entries/semantic/18.formula-date-subtraction-refused.ts b/packages/spec/src/migrations/entries/semantic/18.formula-date-subtraction-refused.ts new file mode 100644 index 00000000000..a83d32113e4 --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.formula-date-subtraction-refused.ts @@ -0,0 +1,35 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +export const entry: SemanticMigration = { + id: 'formula-date-subtraction-refused', + surface: + 'data.Field.expression of a formula field (and any CEL predicate judged with the object\'s field types: ' + + 'validation conditions, flow conditions) — a date or datetime field subtracted from another ' + + '(record.end_date - record.start_date), or arithmetic on one with a duration ' + + '(record.start_date - duration("24h")); and, as a formula value, an expression that computes a ' + + 'duration (today() - record.start_date, now() - record.start_at)', + replacement: + 'daysBetween(a, b) for the span in whole days: daysBetween(record.start_date, record.end_date) for end ' + + 'minus start, daysBetween(record.start_date, today()) for the days since a date. addDays(d, n) / ' + + 'addMonths(d, n) / daysFromNow(n) to shift a date. A predicate may still compare a duration, ' + + 'today() - record.start_date > duration("72h")', + reason: + 'The authoring type-soundness gate declared a date or datetime field a CEL Timestamp, so the ' + + 'subtraction of two of them checked as a legal Duration and every door passed it: os validate, ' + + 'os build, os lint and the object save door. The runtime never holds a Timestamp there: every read ' + + 'door hands a date field out as its YYYY-MM-DD text and a datetime field as its ISO-8601 instant ' + + 'text, so the evaluator faulted on the subtraction and the field read null for every record, with ' + + 'a server WARN as the only trace; a timestamp minus a date field evaluated to a Duration, which a ' + + 'field serialises as {}. The gate now re-checks what the Timestamp typing passes with the field ' + + 'declared as that text, and refuses both shapes (date-text-arithmetic, and date-duration-value for a ' + + 'formula). The (record.end_date - record.start_date) + 1 shape keeps its existing date-arithmetic ' + + 'refusal and words. There is no mechanical rewrite: which span the author meant (the days between, ' + + 'inclusive or not, or hours) is a judgement about the field, and a stored formula of the refused ' + + 'shape read null or {} for every record, so no stored value depends on it.', + acceptanceCriteria: + 'os validate and os build report no date-text-arithmetic or date-duration-value finding; each ' + + 'rewritten formula field reads a number on GET /api/v1/data/OBJECT/ID where it read null or {} ' + + 'before; no stored sys_metadata object row reports either finding in its _diagnostics.', +};