skills(objectstack-formula): eval case 4's expected_output carries both strings its own must_not_contain refuses

Filing-gate class ① — a defect found by a contract review in flight. The director seat's review of PR #26810 (record 6448530347, should-fix outside that diff) measured it; the domain:skills#1 seat re-measured on main. Filed by the domain:skills#1 seat (session_01ADHkmSrRBdu9MdC6fc33Uu, seat post #7623). ⛔ Not a claim.

Measured (origin/main): skills/objectstack-formula/evals/cel-predicates-formulas.json case 4 has must_not_contain: ['new Date(', 'today() + '], and its own expected_output contains both strings (「never new Date()」, 「never today() + 45」). A correct answer that repeats the teaching's own words is graded wrong. This is the same class #26515 repaired in the automation eval file, and that PR #26810 removed from case 3 (ISCHANGED().

Business reason: a grader that fails the ideal answer cannot tell a good agent from a bad one, so the eval stops measuring the skill.

Wanted: rewrite case 4's expected_output so it names the refused forms without spelling the refused strings (as case 7 says "the label macro"), every assertion kept. Budget: the file is at its ceiling (1352 after #26810, headroom 0) — a same-byte rewrite first; if it cannot be byte-neutral, raise at the measurement under the standing ruling 「A:先削重复,余额按实测提高」.

Order: after PR #26810 lands (same file).

Acceptance: in every case of the file, no must_not_contain string occurs in that case's expected_output, and every must_contain string does; check:skills-token-ratchet green.
Size: XS. Tier H (skills/**), patch changeset for @objectstack/skills. Clause-②: no · Metadata-protocol: none.

Dedupe: open domain:skills issues read by REST for cel-predicates-formulas → only #26760 (case 6–7, in flight).

Generated by Claude Code

Triage: first grade — `documentation` · `priority:p2` · `domain:skills` · `area:records` · `pm:queue` · triage seat · `session_01AavokzJ5DndAwitDXvKy4U` · 2026-10-27T17:55Z

⛔ Not a claim, ⛔ not a dispatch.

p2: eval case 4 of skills/objectstack-formula/evals/cel-predicates-formulas.json carries in its own expected_output both strings its must_not_contain refuses, so a correct answer is graded wrong. This is the same class as skills(objectstack-automation): eval fixtures 5, 7 and 8 of flows-triggers-approvals.json contradict their own assertions — expected_output spells a must_not_contain string or lacks a must_contain one, so the ideal answer fails its own check #26515. Fix the fixture against its assertions, and run the same consistency check over the file. Tier H.

Generated by Claude Code

Claim: PM loop round 1
Session: session_01ADHkmSrRBdu9MdC6fc33Uu
Account: os-steve (the seat's linked user as GET /user answers it; the card's assignee)
Branch: claude/project-thread-htz15g (the card thread's harness push branch, self-reported at 2026-10-27T23:27Z — the measured platform exception to claude/issue-N-slug)
Worktree: objectstack-issue-26814
Domain: domain:skills
Seat: domain:skills#1
File surface: skills/objectstack-formula/evals/cel-predicates-formulas.json (case 4's expected_output only: name the refused forms without spelling its own must_not_contain strings; every assertion and fact kept), its row in scripts/check-skills-token-ratchet.mjs only if a same-byte rewrite is impossible (then raised at the measurement under the standing ruling 「A:先削重复,余额按实测提高」), and a patch changeset for @objectstack/skills; nothing else (stop on breach; explain in the report)
Container & model: XS, mode:thread, model: CONTRACT_REVIEW_TIER (dispatch-gates --tier: MANDATORY — skills/**, clause ①, no one-line exit), effort: high. Tier H (skills/**): draft PR; ⛔ the dev never readies, arms or enqueues it.
Clause-②: no
Metadata-protocol: none
Responsibility: n/a — not a defect card (an eval-consistency defect found by the director's review of PR #26810)
Thread-read: 6449631119
Premise: holds (seat re-measure on origin/main after #26810: a per-case cross-check finds exactly one failure, case 4, new Date( and today() + inside its own expected_output; the file sits at 5407 B / 1352 tokens, headroom 0)
Serial constraints cleared: PR #26810 (same file) merged at 40264beec4

os-dev-report

{
  "issue": 26814,
  "status": "done",
  "branch": "claude/project-thread-htz15g",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/26978",
  "session": "session_01ADHkmPQ8p3cVKear7Mchi2",
  "premise_still_valid": true,
  "summary": "Case 4's expected_output in skills/objectstack-formula/evals/cel-predicates-formulas.json now names the refused forms without spelling its own must_not_contain strings: 'never a JS `Date` / `Date.now()`' and 'never date arithmetic on `today()` (it faults)'. Every assertion and fact kept; the file stays 5407 B / 1352 tokens, so the ratchet row does not move and the standing ruling was not needed. Patch changeset for @objectstack/skills. Draft PR, Tier H (skills/**), waits for a GOVERNED_APPROVERS approval; never readied, armed or enqueued.",
  "tests": "Per-case cross-check (each case: no must_not_contain string inside its own expected_output, every must_contain string present) — origin/main 912076f8: 'case 4: must_not_contain \"new Date(\" PRESENT / case 4: must_not_contain \"today() + \" PRESENT / cases=7 failures=2'; head f8531858: 'cases=7 failures=0'. JSON.parse ok. wc -c before/after 5407/5407; whole package skills/*/SKILL.md 4605 lines / 224961 B before and after; skills/*/evals/*.json 1479 lines / 94793 B before and after. No test file edited; no ablation (no tested behaviour changed).",
  "gates": "Local subset at f8531858: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack → 42 commands, each run with its exit code captured before any pipe: 42/42 exit 0 (incl. 'node scripts/check-skills-token-ratchet.mjs' → '✓ … cel-predicates-formulas.json is 1352 tokens (ceiling 1352; headroom 0)', check:nul-bytes, check:empty-changeset, check:adr-0087-registration, check:pm-governed-merges). --ran reconciliation: 'Run reconciliation — 42 derived, 42 run, 0 NOT-MEASURED, 0 UNRUN' (exit 0, no UNRUN on stderr). Build closure first under the lock: 'VERDICT command-exit 0 · held the lock 841s'. CI families (Lint & Repo Gates, TypeScript Type Check, Check Changeset, Governed Surface Queue Guard, …) are read from PR #26978's check-runs: in_progress at report time, not re-run locally. test:repo is CI's (no test edited).",
  "dogfood_battery": "n/a — no `!` in the PR title and no (narrowing) arm",
  "mcp_calls": "2 — mcp__github__issue_read (get_comments #26814, claim check), mcp__github__pull_request_read (get #26978, draft read-back); no MCP write",
  "api_writes": "2 relay strokes + git push — (1) fleet-write pr_create (draft create + assignee leg os-steve + the relay's Tier H governed-review request) run 40605464767, read-back 5350/5350 bytes identical; (2) post-stamped.mjs --comment=26814 for this report. git push -u origin claude/project-thread-htz15g is not a REST write.",
  "open_questions": [
    { "question": "Wording of the JS Date refusal: 'never a JS `Date` / `Date.now()`' (byte-neutral) vs the card's example 'never a JavaScript Date constructor' (+15 B, would need a paid cut elsewhere in the file).", "options": ["A keep 'a JS `Date` / `Date.now()`' — byte-neutral, both refused forms named", "B 'a JS `Date` constructor or `Date.now()`' and pay 14 B by dropping 'the' before 'determinism gate' plus the em dash → comma"], "recommendation": "A, because the row stays at 1352 with zero arithmetic and the sentence still names both forms the assertion refuses" },
    { "question": "The PR body opens with the harness-required claude-projects-attribution comment line (an HTML comment) before 'Fixes #26814'.", "options": ["A leave as the platform wrote it", "B seat strips it via issue_patch"], "recommendation": "A, the os-dev exception for the harness-written first-line attribution comment; reported, not changed" }
  ],
  "out_of_scope_findings": []
}

Generated by Claude Code

ACCEPT → PR #26978 · domain:skills#1 · 2026-10-27T23:59Z · Tier H; draft until a GOVERNED_APPROVERS approval; contract review requested from the director seat.

Landed · domain:skills#1 · session_01ADHkmSrRBdu9MdC6fc33Uu · 2026-10-28T00:47Z — PR #26978 merged through the queue (merge commit 1bd7574722) on the os-zhuang approval, after the director's PASS: objectstack-formula eval case 4 names the refused date forms without spelling its own must_not_contain strings. The card closed by its Fixes line; pm:dispatched stripped in this act.

Navigation Menu

Repository navigation

skills(objectstack-formula): eval case 4's expected_output carries both strings its own must_not_contain refuses #26814

Triage: first grade — `documentation` · `priority:p2` · `domain:skills` · `area:records` · `pm:queue` · triage seat · `session_01AavokzJ5DndAwitDXvKy4U` · 2026-10-27T17:55Z

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

skills(objectstack-formula): eval case 4's expected_output carries both strings its own must_not_contain refuses #26814

Description

Activity

objectstack-fleet commented on Oct 27, 2026

Triage: first grade — documentation · priority:p2 · domain:skills · area:records · pm:queue · triage seat · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-27T17:55Z

objectstack-fleet commented on Oct 27, 2026

objectstack-fleet commented on Oct 27, 2026

objectstack-fleet commented on Oct 27, 2026

objectstack-fleet commented on Oct 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Triage: first grade — `documentation` · `priority:p2` · `domain:skills` · `area:records` · `pm:queue` · triage seat · `session_01AavokzJ5DndAwitDXvKy4U` · 2026-10-27T17:55Z