From f853185879c294028b63896cce49866f5e7bc7ad Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 27 Oct 2026 23:30:52 +0000 Subject: [PATCH] docs(skills): objectstack-formula eval case 4 names the refused date forms without spelling its own must_not_contain strings Case 4's expected_output carried both strings its own must_not_contain refuses (`new Date(` and `today() + `), so a reference-grade answer was graded wrong. The reference now says "never a JS `Date` / `Date.now()`" and "never date arithmetic on `today()` (it faults)"; every assertion and fact stays, the file keeps its 5407 bytes, and a per-case cross-check of all seven cases reports 0 failures. Patch changeset for @objectstack/skills. Claude-Session: https://claude.ai/code/session_01ADHkmPQ8p3cVKear7Mchi2 Co-authored-by: Claude --- .../26814-formula-eval-case4-self-consistent.md | 12 ++++++++++++ .../evals/cel-predicates-formulas.json | 2 +- 2 files changed, 13 insertions(+), 1 deletion(-) create mode 100644 .changeset/26814-formula-eval-case4-self-consistent.md diff --git a/.changeset/26814-formula-eval-case4-self-consistent.md b/.changeset/26814-formula-eval-case4-self-consistent.md new file mode 100644 index 00000000000..3e3b117b320 --- /dev/null +++ b/.changeset/26814-formula-eval-case4-self-consistent.md @@ -0,0 +1,12 @@ +--- +'@objectstack/skills': patch +--- + +docs(skills): objectstack-formula eval case 4's reference answer names the refused date forms without spelling its own `must_not_contain` strings + +Clause-②: no +Metadata-protocol: none + +- **Before.** Case 4 of `skills/objectstack-formula/evals/cel-predicates-formulas.json` refuses `new Date(` and `today() + ` in an answer, yet its own `expected_output` spelled both (`` `new Date()` `` and `` `today() + 45` ``), so a reference-grade answer was graded wrong. +- **Now.** The reference answer says "never a JS `Date` / `Date.now()`" and "never date arithmetic on `today()` (it faults)", as case 7 names "the `label` macro". Every assertion and every fact stays: the CEL envelope seed, the determinism gate, `daysBetween(today(), record.close_date)`, the `{}` duration trap. A per-case cross-check (no `must_not_contain` string inside its own `expected_output`, every `must_contain` string present) reports 0 failures over all 7 cases; the file keeps its byte size, so its token-ratchet row does not move. +- No schema, key or behaviour changes. diff --git a/skills/objectstack-formula/evals/cel-predicates-formulas.json b/skills/objectstack-formula/evals/cel-predicates-formulas.json index d69be1ddfc7..e4389a466f8 100644 --- a/skills/objectstack-formula/evals/cel-predicates-formulas.json +++ b/skills/objectstack-formula/evals/cel-predicates-formulas.json @@ -34,7 +34,7 @@ { "id": 4, "prompt": "Demo opportunities should close 45 days after the package is installed, and a formula should show how many days remain until close.", - "expected_output": "Seed value as a CEL envelope, `close_date: cel`daysFromNow(45)``, never `new Date()` / `Date.now()` (the determinism gate: two `os build` runs must be byte-identical). Days remaining is `daysBetween(today(), record.close_date)` — never `record.close_date - today()` (a duration stored as `{}`) and never `today() + 45` (date arithmetic faults).", + "expected_output": "Seed value as a CEL envelope, `close_date: cel`daysFromNow(45)``, never a JS `Date` / `Date.now()` (the determinism gate: two `os build` runs must be byte-identical). Days remaining is `daysBetween(today(), record.close_date)` — never `record.close_date - today()` (a duration stored as `{}`) and never date arithmetic on `today()` (it faults).", "files": [], "assertions": { "must_contain": ["cel`", "daysFromNow(", "daysBetween("],