diff --git a/CHANGELOG.md b/CHANGELOG.md index ff60ea83..a3a3ac25 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- process(context): **stories and old measurements move out of four always-loaded skills; baseline 75,028 → 74,223 words (−805).** Verbatim moves into `process/baseline-incident-history.md` (new sections UI1–UI4, TO1–TO3, MF1, MP1), each leaving a one-line pointer: `ui-preflight-pages.md` (the 2026-09-09 design-reach story, the ToeicBuddy fidelity numbers, the fidelity-vs-checklist table, the repeated 2026-08-31 A/B paragraph), `tool-output-is-not-ground-truth.md` (the three 2026-07-29 instance narratives, the `$?` instance table, the Part 3 workflow story), `microflow-preflight.md` (`mxcli layout` probe history) and `learned-mdl-preflight.md` (#879 re-probe numbers). Rules, STOP rows and their numbers, denominators and one-line failure examples stay in place. New `bin/check-moved-text.sh` makes the #315 line-by-line review mechanical: every removed line must be found with `grep -F` in its new home (83 of 83 here). It exits 2 on a source with no diff or on 0 removed lines (never "0 of 0" green), lists lines under 15 characters as trivially found, and has a fixture (`tests/wave2/test-check-moved-text.sh`). Behaviour A/B still owed before merge. — T4 of the stub-diet research report - process(build): **Stage 5 drafts the ready set in parallel and applies serially.** The runbook dispatch row said "sequential within a module", which serialised drafting even for independent scripts; it now points to `iterative-build-loop.md` → Gate cost, the single statement of the rule. The unit is the build-plan row: every `BUILD` row whose "Depends on" rows are all applied, minus overlapping writers (two rows that replace, alter or drop+create the same document never share a ready set, and no row is drafted while it overlaps a queued unapplied one; `brd-to-build-plan.md` has Produces name every document a row writes), runs at once, its scripts in order, one `mdl-agent` per script. Applies stay one at a time through `bin/exec.sh`, and the ready set is recomputed after each. A queued draft made before a sibling apply landed is re-checked with `mxcli diff --format struct` (preflight STOP rule 23); any removal line (checked against `describe` per BUG-146) or STOP rule 21 target means re-draft, never patch, because a stale restatement of a shared document is exactly rule 21's silent seed-call loss and passes `check --references`. Report `ready set N, drafted N, applied k`; the `mdl-agent` template says a drafter never applies and never depends on an unapplied draft. Field evidence: none yet. — MendixMau - fix(platform-link): **CLI commits now show their changed documents in Studio Pro History.** Studio Pro's History → Model changes pane reads a `ModelChanges` array from the commit's `refs/notes/mx_metadata` note. Trap 6c's hand-written template set it to `[]`, so every mxcli/agent commit showed "Model changes 0" (field project: 270 of 270 notes empty). New `project-bin/mx-metadata-note.py` (stdlib only) computes the array from the commit's changed `.mxunit` files, in Studio Pro's entry shape: Status, UnitID, UnitType, UnitName and Module (walked up the `.mpr` Unit tree). It has `changes` / `write` / `backfill` / `sync ` subcommands. `install-project-hooks.sh` now also installs post-commit and post-rewrite hooks that write the note and never block, and `--check` covers them, so `sync-project.sh` rolls them out to wired projects. Notes are published with `mx-metadata-note.py sync ` after the branch push. Push config is never touched. With mxcli v0.26+ in the project root the hooks call `mxcli git note --write` instead (mendixlabs/mxcli#1337); the Python script is the fallback for older pins, and on 160 field commits both wrote the same 3,594 entries. Trap 6c rewritten. Measured against Studio Pro's own notes, 51 commits in 5 apps: 3387/3389 entries exact on all five fields. Field run on a 357-commit Mendix 11.15.0 app: 158 notes backfilled and pushed, and Studio Pro History lists the documents per revision. Review fixes: an earlier revision added `HEAD` + the notes ref to `remote..push`. It was dropped because any push refspec replaces `push.default`. A branch tracking a differently named upstream got a new same-named branch on Team Server where `simple` refuses, a missing local notes ref failed the whole push, and nobody knows whether Studio Pro's own push reads that config. The installer now only fetches the remote's notes (non-forced). It removes that exact `HEAD` + notes pair if present, and leaves and reports any other entry. `sync` handles a remote with no notes ref (it used to exit 1) and retries a rejected push once after re-merging. A note that already lists changes (Studio Pro's) is never overwritten by `write`/`backfill` without `--force`, and `sync` keeps a colleague's non-empty note over ours. The mxcli probe greps `git note --help` for `--write`. Hooks and docs call the script through `resolve_py`. Fixture `tests/wave2/test-mx-metadata-note.sh` (40 cases, over real captured template units; single- and two-tree layouts, amend, backfill and `--force`, push exit codes identical to a no-hooks clone, leftover-refspec cleanup, sync to an empty remote and its retry, notes fetch at install, a diverged notes merge; green under `init.defaultBranch` master and main). — field project (Mendix 11.15 app), MendixMau - learn(fixture-seeding): **explore the data structure first, choose rows by use case, scan actions for resets before any run.** Two seeding runs on public template apps: seeding raised journey passes 33 -> 58 and 20 -> 51 of the same sets (FAIL 93 -> 68, 163 -> 132; INVALID unchanged), and a "Reset all demodata" journey wiped accounts so 104 of 127 journeys reported `The username or password you entered is incorrect`, identical on empty and seeded databases. Skill gains three questions a seed must answer, a use-case checklist, a destructive-action scan, a generic traps table with symptoms, measured expectations and a denominator checklist; routing trigger now fires on seed/test/demo data and empty pages. Independent review fixes: object ids follow the Mendix bigint pattern with a business-key guard (no uuids), associations are seeded where the page's path stores them, four new traps (enum names, inheritance chain, calculated attributes, view/external entities), and the port-offset and container-label traps moved to `harness-architecture.md`. — MendixMau diff --git a/bin/check-moved-text.sh b/bin/check-moved-text.sh new file mode 100755 index 00000000..74273691 --- /dev/null +++ b/bin/check-moved-text.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# check-moved-text.sh — prove a "move, never delete" trim lost no line. +# +# WHY. CLAUDE.md "Always-loaded skill text — the word budget": room in the baseline is made by +# MOVING stories, examples and old numbers to an on-demand file, never by deleting the rules. +# The first such trim (#315) was checked by hand, line by line. This makes the mechanical half of +# that review a command: every line the diff removes from the source files must still exist, +# verbatim, in the file it moved to. What it cannot judge is whether a line that stayed behind +# should have — that half stays with the reviewer. +# +# bin/check-moved-text.sh [--base ] ... +# bin/check-moved-text.sh process/baseline-incident-history.md skills/ui-preflight-pages.md +# +# Compares the working tree against merge-base(, HEAD) (default ref origin/master), so it +# runs the same before and after the commit. A removed line counts as found when `grep -F` finds +# it (trailing whitespace trimmed) anywhere in the destination's working-tree copy. Blank lines +# are not counted. Each line not found is printed for hand review — a line rewritten in place +# (a sentence kept, its neighbour moved) shows up here, and the PR says why. +# +# LIMIT. `grep -F` matches substrings, and a line that occurs twice in the sources is satisfied +# by one copy in the destination. So a very short line (`|---|`, "```", "---") is found almost +# anywhere and proves nothing. Lines under 15 characters are listed separately as "trivially +# found, review by hand"; they count as found, but the reviewer reads them. +# +# A named source with no diff (a mistyped path, or a file nothing was removed from) exits 2, +# and so does a run that removed 0 lines in total: "0 of 0 found" is not a pass. +# +# Field run: the T4 stub-diet trim, 2026-10-14 (four baseline skills into +# process/baseline-incident-history.md). +# +# Exit: 0 every removed line found · 1 some not found, listed · 2 could not run, a source with +# no diff, or nothing removed at all. +set -u + +BASE="origin/master" +if [ "${1:-}" = "--base" ]; then BASE="${2:-}"; shift 2; fi +DEST="${1:-}" +[ -n "$DEST" ] && [ "$#" -ge 2 ] || { grep '^# bin/check-moved-text.sh' "$0" | sed 's/^# *//' >&2; exit 2; } +shift +[ -f "$DEST" ] || { echo "check-moved-text: destination '$DEST' not found." >&2; exit 2; } +MB="$(git merge-base "$BASE" HEAD 2>/dev/null)" || { + echo "check-moved-text: no merge base with '$BASE' (fetch it first)." >&2; exit 2; } + +TOTAL=0; FOUND=0; MISSING=""; SHORT=""; NODIFF="" +for SRC in "$@"; do + REMOVED="$(git diff --no-color -U0 "$MB" -- "$SRC" | awk ' + /^--- (a\/|\/dev\/null)/ { next } + /^-/ { print substr($0, 2) }')" + if [ -z "$(printf '%s' "$REMOVED" | tr -d '[:space:]')" ]; then + NODIFF="${NODIFF} ${SRC} +" + continue + fi + while IFS= read -r line; do + t="$(printf '%s' "$line" | sed 's/[[:space:]]*$//')" + [ -n "$(printf '%s' "$t" | tr -d '[:space:]')" ] || continue + TOTAL=$((TOTAL + 1)) + if grep -F -q -- "$t" "$DEST"; then + FOUND=$((FOUND + 1)) + if [ "${#t}" -lt 15 ]; then SHORT="${SHORT} ${SRC}: ${t} +"; fi + else + MISSING="${MISSING} ${SRC}: ${t} +" + fi + done <&2 + printf '%s' "$NODIFF" >&2 + exit 2 +fi +if [ "$TOTAL" -eq 0 ]; then + echo "check-moved-text: 0 removed lines in total — nothing was checked, which is not a pass." >&2 + exit 2 +fi +echo "check-moved-text: $FOUND of $TOTAL removed non-blank line(s) found verbatim in $DEST." +if [ -n "$SHORT" ]; then + echo "Trivially found (under 15 characters, a substring match proves little — review by hand):" + printf '%s' "$SHORT" +fi +if [ -n "$MISSING" ]; then + echo "Not found (review by hand; each needs a reason in the PR):" + printf '%s' "$MISSING" + exit 1 +fi +exit 0 diff --git a/process/baseline-incident-history.md b/process/baseline-incident-history.md index c863ea2d..9a941b3a 100644 --- a/process/baseline-incident-history.md +++ b/process/baseline-incident-history.md @@ -268,3 +268,177 @@ Verbatim, from a real intake (2026-09-02, Q4 "documents not yet accounted for?") The triage never named the deck. Two months of downstream artifacts inherited the gap, because once Q4 read "answered" nothing had a reason to look again. The answer *cited a document*, which is exactly why it was believed. + +## UI1 — Design system ported, never reached the app (2026-09-09) + +_From `skills/ui-preflight-pages.md`; moved verbatim 2026-10-14._ + +Measured 2026-09-09: 55 tokens were ported into the right file at the right stage, 0 of 35 +framework knobs were bound, 0 of 20 component classes existed in the built stylesheet, and two +build phases of pages were written on top of it. The pages were structurally correct. `mx +check`, `mxcli lint`, the MDL suite and two e2e journeys were all green, because none of them +reads a stylesheet. It was found by a human opening a screenshot and asking why the UI was +poor. The three failures were a bridge written against the wrong layer (SCSS variables where +the theme runs `$use-css-variables: true`), component classes never ported at all, and a theme +module whose `:root` loses the cascade to the framework's own customization file. + +## UI2 — First-build page fidelity, ToeicBuddy field run (2026-08-25/26) + +_From `skills/ui-preflight-pages.md`; moved verbatim 2026-10-14._ + +**Measured, ToeicBuddy field run 2026-08-25/26** (`process/first-build-page-fidelity-2026-08-27.md`): +first-build fidelity across 10 pages was **32% median** against a target of 80%. All 14 +wireframes declared a 900px page column and **0 of 10** pages capped their width — repaired +wholesale at script 67, fifty-five scripts later, after rendering at 1360px. All 10 were +built on a sidebar layout against wireframes drawing a top bar — repaired at script 20. +The script's header carries the full evidence for each check. + +## UI3 — Fidelity score vs. element checklist, seven screens (2026-09-19) + +_From `skills/ui-preflight-pages.md`; moved verbatim 2026-10-14._ + +**A fidelity score is not a LOOK pass, and must never be accepted as one.** +`page-fidelity.js` matches identifiers in the page MDL against the wireframe's text. It +cannot see where an element sits, only that the name occurs. Measured on a +field build of 2026-09-19, scoring +seven screens both ways on the same commit: + +| Screen | Fidelity (text match) | Element checklist (present / placed / behaves) | +|---|---:|---:| +| S1 Requests overview | 100% | 58% | +| S4 Approval queue | 100% | 40% | +| S3 Request detail | 55% | 69% | + +Two screens scored a perfect 100 while missing more than half their wireframe: the elements +were named in the MDL and laid out wrongly. S3 runs the other way, because its grid's decimal +formatting is not expressible as text the instrument greps for. So a high fidelity score +closes nothing — the `look` obligation is discharged by a person or an agent opening the +rendered page, at 1280px and at 390px, and writing down what it saw. Use the fidelity score +for the rework curve between LOOK passes, never in place of one. + +## UI4 — Controlled A/B, and the build script that cited Step 3 (2026-08-31) + +_From `skills/ui-preflight-pages.md`; moved verbatim 2026-10-14._ + +**Measured again, controlled A/B, 2026-08-31** (`process/preflight-skill-baseline-2026-08-31.md`): +5 fresh drafting agents with this file inlined produced **zero** judgement-step violations +(layout, H1, inline styles, invented classes) across all 5 reps; 5 without it reproduced every +field defect. The content works when read — the failure mode is dispatching a page build with +a *reference* to this file instead of its text (see the delivery rule at the top). + +The build script that produced those pages **cited this file's Step 3 by name**. That is +the point: the pre-flight was run, and its output was a paragraph nobody could fail. A +shell defect is uniform across every page built the same way, so no page stands out as a +diff and per-page review does not surface it — which is exactly what a mechanical check +is for, and exactly what human judgement is worst at. + +## TO1 — Three false findings from one session (2026-07-29) + +_From `skills/tool-output-is-not-ground-truth.md`; moved verbatim 2026-10-14, its headings demoted one level._ + +### Three real instances from one session + +#### 1. "All your work is gone" — read during a write + +Read the `.mpr` while Studio Pro was mid-rewrite, saw 6 microflows instead of 13, and told the user +the whole session's work was lost. It was not; the file was simply being written at that instant. + +**Antidote:** confirm against an independent source before declaring loss. +```bash +git hash-object Project.mpr # working tree +git rev-parse :Project.mpr # committed +``` +Identical hashes mean the tree *is* the checkpoint, whatever a stale read said. Also re-run the +query — caches (`.mxcli/catalog.db`) lag. + +#### 2. "mxcli converts v2→v1 on read" — correlation asserted as cause + +The project kept flipping from MPR v2 to a single-file form. Noticed the conversions happened +around read-only `mxcli` commands and reported mxcli as the culprit — plausible, and wrong. A +subagent later checked mtime before and after its own mxcli calls: unchanged. Something else was +doing it. + +**Antidote:** to claim tool T causes effect E, run T in isolation and observe E. Timing correlation +in a tree with multiple concurrent writers is not evidence. Instrument (record state before/after +your own calls) rather than infer. + +#### 3. "These filters are unbound" — a lossy DESCRIBE + +`DESCRIBE PAGE` emitted `dropdownfilter fEquipType` with no `Attributes:` property. Reported as a +live shipping bug across two pages, and a corrective script was drafted. The user pushed back: +the filters had been tested working many times. They were right. + +Two compounding errors: +- **`grep` flattened the hierarchy**, hiding that the filters sat in a deliberately built filter bar + with paired labels. +- **Disconfirming evidence was already in hand and ignored.** `mxcli check` on the corrective script + emitted `MDL-WIDGET07: property columnsFilterable is not recognized and will be silently dropped` + — an explicit admission that mxcli's model of that widget family is incomplete. A tool that does + not understand a sibling property cannot be trusted to round-trip a child one. + +Had it been executed, it would have rebuilt a working page containing `customContent` columns — +the single highest-risk construct for load-time BSON corruption in this stack. **All risk, zero +benefit, from a finding that never existed.** + +## TO2 — `$?` after a pipeline — three instances (2026-08-04) + +_From `skills/tool-output-is-not-ground-truth.md`; moved verbatim 2026-10-14._ + +Real instances from the one session: + +| Observed | Concluded | Actually | +|---|---|---| +| `coverage-check ... \| tail; exit=0` with 323 UNCLAIMED | "exit 0 on a dirty ledger — new bug!" | exit **1**, correct | +| `gate-check ... \| head; exit=0` printing "Gate BLOCKED" | "a blocked gate passes — 5th false-PASS!" | exit **1**, correct | +| `exec.sh ... \| tail; EXIT=` (empty) | inconclusive | exit **3**, correct | + +## TO3 — A passing gate is not a working model — the workflow red pins (2026-08-05) + +_From `skills/tool-output-is-not-ground-truth.md`; moved verbatim 2026-10-14, its headings demoted one level._ + +### The instance + +Scripted a Mendix workflow with five `CALL MICROFLOW` activities. + +- `mxcli check --references` → `Check passed!` +- `mxcli exec` → `Created workflow` +- `mx check` (Studio Pro's own validator) → **`0 errors`** +- Studio Pro **opened the model** without complaint + +Four green signals. Every activity was a **red pin** that could not be clicked, and the app would +have refused to boot: `Class 'Workflows$CallMicroflowTask' could not be found`. + +mxcli `v0.16.0` writes a `$Type` that Mendix renamed at 11.9. `mx check` does not validate that +name. The upstream fix commit states this outright — *"Both checkers passed... the runtime refused +to load the ENTIRE model at boot."* + +### Why it cost a day rather than an hour + +A six-variant probe had been run to find the trigger boundary, varying `WITH`-clause forms. **Every +arm was graded by `mx check`.** All six passed, so the conclusion was "syntax X is safe" — and all +six were broken in exactly the same way, because the variable that mattered was never varied. + +> A control that cannot distinguish your arms will confirm whichever hypothesis you brought. + +The probe was well-designed in every respect except the one that counted: nobody asked *what is +this gate actually sensitive to?* + +## MF1 — `mxcli layout` probe history (2026-09-25) + +_From `skills/microflow-preflight.md`; moved verbatim 2026-10-14._ + +**`mxcli layout` is a domain-model command.** The mxcli team's answer to microflow +positioning (2026-09-25) is a standalone layout command rather than anything in `create +microflow`. On v0.24.0 and on upstream `main` at that date, `mxcli layout --help` says +*"Arrange the entities of a domain model"*; it moves entities only, and there is no microflow or +nanoflow mode yet. Measured: `mxcli layout -p app.mpr --dry-run` on a scratch copy of a PoC +model listed 10 entity moves and no microflow. **Before relying on it for a microflow, probe +your binary:** `mxcli layout --help` — the capability-probe rule. Until a microflow mode ships, +the remedy for a bad microflow layout stays the one in this file: no `@position`, and the +`SUB_` split for the defect below. + +## MP1 — `marketplace install` #879 — the v0.21.0 re-probe (2026-09-10) + +_From `skills/learned-mdl-preflight.md`; moved verbatim 2026-10-14._ + +- ~~`marketplace install` on a split-model project (#879)~~ — **RESOLVED in v0.21.0, STOP LIFTED (probed 2026-09-10).** Install no longer shells out to `mx module-import`; it copies units with mxcli's own writer and MPR v2 survives. The destructive path is now opt-in behind `--allow-format-change`, so the failure can only happen on purpose. Measured on a real 501-unit project: 501 → 1075 `.mxunit`, `.mpr` 84 KB → 156 KB (an index, not the 35 MB blob #879 produced), native mxbuild **0 errors**. **Read `bug-logs/mxcli-bugs.md` → the #879 entry before your first install** — the corruption is gone, and three *new* operational traps replaced it (packages do not carry their dependencies; two `mxcli fix` repairs are mandatory, not advisory; a bundled widget can silently fail to land and the installer cannot self-repair it). diff --git a/skills/learned-mdl-preflight.md b/skills/learned-mdl-preflight.md index 8acf7c76..0cafca48 100644 --- a/skills/learned-mdl-preflight.md +++ b/skills/learned-mdl-preflight.md @@ -11,7 +11,7 @@ Re-probed on mxcli v0.24.0 / Mendix 11.13.0 (2026-09-30); rules that now work or **Version notes (mxcli ≥ v0.20.0):** - ALTER PAGE addresses DataGrid2 columns by **derived** name (bound attribute, else caption — MDL-WIDGET16); a bare column name in REPLACE is refused in favour of `grid.column`. - **Still corrupting, STOP stays:** workflow `DECISION` activities (BUG-76 — project unloadable, `EnumerationValueIdentifier`). **v0.24.0 narrows it:** the bare-enum form is refused by `MDL-WF03` (never force it with `--no-check`), and a Boolean decision with `true`/`false` outcome keywords builds at 0 native errors — `learned-workflow-patterns.md` §8 Warning 1. -- ~~`marketplace install` on a split-model project (#879)~~ — **RESOLVED in v0.21.0, STOP LIFTED (probed 2026-09-10).** Install no longer shells out to `mx module-import`; it copies units with mxcli's own writer and MPR v2 survives. The destructive path is now opt-in behind `--allow-format-change`, so the failure can only happen on purpose. Measured on a real 501-unit project: 501 → 1075 `.mxunit`, `.mpr` 84 KB → 156 KB (an index, not the 35 MB blob #879 produced), native mxbuild **0 errors**. **Read `bug-logs/mxcli-bugs.md` → the #879 entry before your first install** — the corruption is gone, and three *new* operational traps replaced it (packages do not carry their dependencies; two `mxcli fix` repairs are mandatory, not advisory; a bundled widget can silently fail to land and the installer cannot self-repair it). +- ~~`marketplace install` on a split-model project (#879)~~ — **RESOLVED in v0.21.0, STOP LIFTED (probed 2026-09-10).** The destructive path is now opt-in behind `--allow-format-change`, so the failure can only happen on purpose. **Read `bug-logs/mxcli-bugs.md` → the #879 entry before your first install** — the corruption is gone, and three *new* operational traps replaced it (packages do not carry their dependencies; two `mxcli fix` repairs are mandatory, not advisory; a bundled widget can silently fail to land and the installer cannot self-repair it). Probe measurements: `process/baseline-incident-history.md` § MP1. - The 10–12-writes-per-session cap from BUG-97 is downgraded to snapshot-plus-read-back: 16 sequential writes were clean on v0.20.0. ## Step 0 — choose the write mode (do this first, before the STOP table) diff --git a/skills/microflow-preflight.md b/skills/microflow-preflight.md index b6863319..fbbda739 100644 --- a/skills/microflow-preflight.md +++ b/skills/microflow-preflight.md @@ -120,12 +120,8 @@ coordinates are relative to the loop box's top-left corner; n = activities in th | Next activity after the loop | loop centre + 80·n + 180 | | Sequential pitch | 160 | -**`mxcli layout` is a domain-model command.** The mxcli team's answer to microflow -positioning (2026-09-25) is a standalone layout command rather than anything in `create -microflow`. On v0.24.0 and on upstream `main` at that date, `mxcli layout --help` says -*"Arrange the entities of a domain model"*; it moves entities only, and there is no microflow or -nanoflow mode yet. Measured: `mxcli layout -p app.mpr --dry-run` on a scratch copy of a PoC -model listed 10 entity moves and no microflow. **Before relying on it for a microflow, probe +**`mxcli layout` is a domain-model command.** On v0.24.0 (as of 2026-09-25) it moves entities only; there is no +microflow or nanoflow mode (history: `process/baseline-incident-history.md` § MF1). **Before relying on it for a microflow, probe your binary:** `mxcli layout --help` — the capability-probe rule. Until a microflow mode ships, the remedy for a bad microflow layout stays the one in this file: no `@position`, and the `SUB_` split for the defect below. diff --git a/skills/tool-output-is-not-ground-truth.md b/skills/tool-output-is-not-ground-truth.md index 86e734fe..c0977b05 100644 --- a/skills/tool-output-is-not-ground-truth.md +++ b/skills/tool-output-is-not-ground-truth.md @@ -54,10 +54,11 @@ was used. ## Three real instances from one session +Full narratives: `process/baseline-incident-history.md` § TO1. + ### 1. "All your work is gone" — read during a write -Read the `.mpr` while Studio Pro was mid-rewrite, saw 6 microflows instead of 13, and told the user -the whole session's work was lost. It was not; the file was simply being written at that instant. +Read the `.mpr` while Studio Pro was mid-rewrite and declared the session's work lost. It was not. **Antidote:** confirm against an independent source before declaring loss. ```bash @@ -69,10 +70,7 @@ query — caches (`.mxcli/catalog.db`) lag. ### 2. "mxcli converts v2→v1 on read" — correlation asserted as cause -The project kept flipping from MPR v2 to a single-file form. Noticed the conversions happened -around read-only `mxcli` commands and reported mxcli as the culprit — plausible, and wrong. A -subagent later checked mtime before and after its own mxcli calls: unchanged. Something else was -doing it. +Format flips coincided with read-only `mxcli` calls; mtime before and after showed mxcli was not the writer. **Antidote:** to claim tool T causes effect E, run T in isolation and observe E. Timing correlation in a tree with multiple concurrent writers is not evidence. Instrument (record state before/after @@ -80,21 +78,10 @@ your own calls) rather than infer. ### 3. "These filters are unbound" — a lossy DESCRIBE -`DESCRIBE PAGE` emitted `dropdownfilter fEquipType` with no `Attributes:` property. Reported as a -live shipping bug across two pages, and a corrective script was drafted. The user pushed back: -the filters had been tested working many times. They were right. - -Two compounding errors: -- **`grep` flattened the hierarchy**, hiding that the filters sat in a deliberately built filter bar - with paired labels. -- **Disconfirming evidence was already in hand and ignored.** `mxcli check` on the corrective script - emitted `MDL-WIDGET07: property columnsFilterable is not recognized and will be silently dropped` - — an explicit admission that mxcli's model of that widget family is incomplete. A tool that does - not understand a sibling property cannot be trusted to round-trip a child one. - -Had it been executed, it would have rebuilt a working page containing `customContent` columns — -the single highest-risk construct for load-time BSON corruption in this stack. **All risk, zero -benefit, from a finding that never existed.** +`DESCRIBE PAGE` showed a `dropdownfilter` with no `Attributes:`; the filters worked, and the drafted +fix would have rebuilt a working page with `customContent` columns. `mxcli check` had already +warned `property columnsFilterable is not recognized and will be silently dropped`: a tool that does +not understand a sibling property cannot be trusted to round-trip a child one. --- @@ -232,13 +219,7 @@ measuring**, and replaces it with a constant 0. A failing gate reads as passing. It is worse than a plain wrong answer, because the failure mode is *biased toward green*. You will never see a spurious failure from this; you will only ever see spurious success. -Real instances from the one session: - -| Observed | Concluded | Actually | -|---|---|---| -| `coverage-check ... \| tail; exit=0` with 323 UNCLAIMED | "exit 0 on a dirty ledger — new bug!" | exit **1**, correct | -| `gate-check ... \| head; exit=0` printing "Gate BLOCKED" | "a blocked gate passes — 5th false-PASS!" | exit **1**, correct | -| `exec.sh ... \| tail; EXIT=` (empty) | inconclusive | exit **3**, correct | +Three real instances from that session: `process/baseline-incident-history.md` § TO2. **THE RULE: measure the exit code on an unpiped run.** @@ -289,30 +270,14 @@ that runs correctly, reports honestly, and is simply **not sensitive to the defe ## The instance -Scripted a Mendix workflow with five `CALL MICROFLOW` activities. - -- `mxcli check --references` → `Check passed!` -- `mxcli exec` → `Created workflow` -- `mx check` (Studio Pro's own validator) → **`0 errors`** -- Studio Pro **opened the model** without complaint - -Four green signals. Every activity was a **red pin** that could not be clicked, and the app would -have refused to boot: `Class 'Workflows$CallMicroflowTask' could not be found`. - -mxcli `v0.16.0` writes a `$Type` that Mendix renamed at 11.9. `mx check` does not validate that -name. The upstream fix commit states this outright — *"Both checkers passed... the runtime refused -to load the ENTIRE model at boot."* - -## Why it cost a day rather than an hour - -A six-variant probe had been run to find the trigger boundary, varying `WITH`-clause forms. **Every -arm was graded by `mx check`.** All six passed, so the conclusion was "syntax X is safe" — and all -six were broken in exactly the same way, because the variable that mattered was never varied. +A scripted workflow with five `CALL MICROFLOW` activities passed `mxcli check --references`, +`mxcli exec`, `mx check` (**0 errors**) and Studio Pro's load; every activity was a red pin and the +runtime would have refused to boot (`Workflows$CallMicroflowTask`, renamed at Mendix 11.9). A +six-arm probe graded by `mx check` passed all six arms, all broken the same way. > A control that cannot distinguish your arms will confirm whichever hypothesis you brought. -The probe was well-designed in every respect except the one that counted: nobody asked *what is -this gate actually sensitive to?* +Full story: `process/baseline-incident-history.md` § TO3. ## The rule diff --git a/skills/ui-preflight-pages.md b/skills/ui-preflight-pages.md index 335c79f8..bce50711 100644 --- a/skills/ui-preflight-pages.md +++ b/skills/ui-preflight-pages.md @@ -113,14 +113,8 @@ Write the report block to **`docs/design-reaches-app.md`**, first line carrying path) — that file is the mark that discharges the `design-reaches-app` obligation in `bin/lib/obligations.tsv`; without it the pass reads as never performed. -Measured 2026-09-09: 55 tokens were ported into the right file at the right stage, 0 of 35 -framework knobs were bound, 0 of 20 component classes existed in the built stylesheet, and two -build phases of pages were written on top of it. The pages were structurally correct. `mx -check`, `mxcli lint`, the MDL suite and two e2e journeys were all green, because none of them -reads a stylesheet. It was found by a human opening a screenshot and asking why the UI was -poor. The three failures were a bridge written against the wrong layer (SCSS variables where -the theme runs `$use-css-variables: true`), component classes never ported at all, and a theme -module whose `:root` loses the cascade to the framework's own customization file. +(2026-09-09 field build: 0 of 35 framework knobs bound and 0 of 20 component classes built, +while `mx check`, lint and e2e stayed green. Full story: `process/baseline-incident-history.md` § UI1.) If there is no build yet, say so in the report block — "binding UNPROVEN, no build" is a legal answer and silence is not. @@ -253,43 +247,16 @@ gate waiver: if you didn't declare it a stub at scoring time, it scores as what it is. "It was meant as a stub" after a bad score is not a category — it is the bad score. -**Measured, ToeicBuddy field run 2026-08-25/26** (`process/first-build-page-fidelity-2026-08-27.md`): -first-build fidelity across 10 pages was **32% median** against a target of 80%. All 14 -wireframes declared a 900px page column and **0 of 10** pages capped their width — repaired -wholesale at script 67, fifty-five scripts later, after rendering at 1360px. All 10 were -built on a sidebar layout against wireframes drawing a top bar — repaired at script 20. -The script's header carries the full evidence for each check. - **A fidelity score is not a LOOK pass, and must never be accepted as one.** `page-fidelity.js` matches identifiers in the page MDL against the wireframe's text. It -cannot see where an element sits, only that the name occurs. Measured on a -field build of 2026-09-19, scoring -seven screens both ways on the same commit: - -| Screen | Fidelity (text match) | Element checklist (present / placed / behaves) | -|---|---:|---:| -| S1 Requests overview | 100% | 58% | -| S4 Approval queue | 100% | 40% | -| S3 Request detail | 55% | 69% | - -Two screens scored a perfect 100 while missing more than half their wireframe: the elements -were named in the MDL and laid out wrongly. S3 runs the other way, because its grid's decimal -formatting is not expressible as text the instrument greps for. So a high fidelity score +cannot see where an element sits, only that the name occurs (2026-09-19: two screens scored +100% fidelity with 58% and 40% of their wireframe elements in place). So a high fidelity score closes nothing — the `look` obligation is discharged by a person or an agent opening the rendered page, at 1280px and at 390px, and writing down what it saw. Use the fidelity score for the rework curve between LOOK passes, never in place of one. -**Measured again, controlled A/B, 2026-08-31** (`process/preflight-skill-baseline-2026-08-31.md`): -5 fresh drafting agents with this file inlined produced **zero** judgement-step violations -(layout, H1, inline styles, invented classes) across all 5 reps; 5 without it reproduced every -field defect. The content works when read — the failure mode is dispatching a page build with -a *reference* to this file instead of its text (see the delivery rule at the top). - -The build script that produced those pages **cited this file's Step 3 by name**. That is -the point: the pre-flight was run, and its output was a paragraph nobody could fail. A -shell defect is uniform across every page built the same way, so no page stands out as a -diff and per-page review does not surface it — which is exactly what a mechanical check -is for, and exactly what human judgement is worst at. +The measurements behind this step — first-build fidelity, fidelity vs. element checklist, the +2026-08-31 A/B — are in `process/baseline-incident-history.md` § UI2–UI4. --- diff --git a/tests/wave2/test-check-moved-text.sh b/tests/wave2/test-check-moved-text.sh new file mode 100755 index 00000000..463cd2a6 --- /dev/null +++ b/tests/wave2/test-check-moved-text.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +# test-check-moved-text.sh — bin/check-moved-text.sh must fail loudly, never green-by-absence. +# +# Born in review of the T4 stub-diet trim (#369): a mistyped source path produced "0 of 0" and +# exit 0. Builds a throwaway git repo (base commit tagged `base`), edits its working tree, and +# runs the real script there with --base base. Cases: every moved line found (exit 0) · a removed +# line not in the destination (exit 1, line named) · a mistyped or unchanged source (exit 2) · +# lines carrying regex/shell metacharacters and a leading dash · short lines listed as trivial. +# +# Usage: bash tests/wave2/test-check-moved-text.sh [path-to-check-moved-text.sh] + +set -u + +TOOLKIT="$(cd "$(dirname "$0")/../.." && pwd)" +SUT="${1:-$TOOLKIT/bin/check-moved-text.sh}" +SUT="$(cd "$(dirname "$SUT")" && pwd)/$(basename "$SUT")" +[ -r "$SUT" ] || { echo "test-check-moved-text: no script at $SUT"; exit 2; } + +PASS=0; FAIL=0 +TMP="$(mktemp -d "${TMPDIR:-/tmp}/moved-text.XXXXXX")" +trap 'rm -rf "$TMP"' EXIT + +ok() { PASS=$((PASS+1)); printf ' ok %s\n' "$1"; } +bad() { FAIL=$((FAIL+1)); printf ' FAIL %s\n %s\n' "$1" "${2:-}"; } + +printf 'test-check-moved-text: %s\n' "$SUT" + +# fresh repo: src.md and other.md at base, dest.md empty-ish +setup() { + rm -rf "$TMP/r"; mkdir -p "$TMP/r"; cd "$TMP/r" || exit 2 + git init -q . && git config user.email t@t && git config user.name t + cat > src.md <<'SRC' +# Skill +The rule stays here and is long enough. +This story line moves to the history file. +- a leading dash line with $HOME and `backticks` moves too +Regex chars [a-z]* .+ ? ^ $ \d and "quotes" 'single' move +|---| +A line that is deleted and found nowhere else at all. +SRC + printf 'unchanged file\n' > other.md + printf '# History\n' > dest.md + git add src.md other.md dest.md && git commit -q -m base && git tag base +} +run() { OUT="$("$SUT" --base base "$@" 2>&1)"; RC=$?; } + +# 1 — found: move four lines verbatim (special characters included) +setup +grep -v -e 'story line' -e 'leading dash' -e 'Regex chars' -e '^|---|$' -e 'deleted and found nowhere' src.md > s && mv s src.md +printf '%s\n' 'This story line moves to the history file.' \ + '- a leading dash line with $HOME and `backticks` moves too' \ + 'Regex chars [a-z]* .+ ? ^ $ \d and "quotes" '"'"'single'"'"' move' '|---|' \ + 'A line that is deleted and found nowhere else at all.' >> dest.md +run dest.md src.md +[ "$RC" -eq 0 ] && ok "all moved lines found -> exit 0" || bad "found case exit $RC" "$OUT" +printf '%s' "$OUT" | grep -q '5 of 5' && ok "count reads 5 of 5" || bad "count" "$OUT" +printf '%s' "$OUT" | grep -q 'Trivially found' && printf '%s' "$OUT" | grep -qF '|---|' \ + && ok "short line listed as trivially found" || bad "short line not listed" "$OUT" + +# 2 — missing: one removed line absent from dest -> exit 1 and the line is named +setup +grep -v -e 'story line' -e 'deleted and found nowhere' src.md > s && mv s src.md +printf '%s\n' 'This story line moves to the history file.' >> dest.md +run dest.md src.md +[ "$RC" -eq 1 ] && ok "missing line -> exit 1" || bad "missing case exit $RC" "$OUT" +printf '%s' "$OUT" | grep -qF 'A line that is deleted and found nowhere else at all.' \ + && ok "missing line is named" || bad "missing line not named" "$OUT" + +# 2b — missing line with special characters is named verbatim, not treated as a regex +setup +grep -v -e 'Regex chars' src.md > s && mv s src.md +printf '%s\n' 'Regex chars [a-z] different text' >> dest.md +run dest.md src.md +[ "$RC" -eq 1 ] && printf '%s' "$OUT" | grep -qF 'Regex chars [a-z]* .+ ? ^ $ \d and "quotes"' \ + && ok "special-character line missing -> exit 1, named verbatim" || bad "special missing" "rc=$RC $OUT" + +# 3 — typo path -> exit 2, even alongside a valid source +setup +grep -v -e 'story line' src.md > s && mv s src.md +printf '%s\n' 'This story line moves to the history file.' >> dest.md +run dest.md src.md srcc.md +[ "$RC" -eq 2 ] && printf '%s' "$OUT" | grep -q 'srcc.md' \ + && ok "mistyped source -> exit 2, path named" || bad "typo case exit $RC" "$OUT" + +# 4 — unchanged source only -> exit 2 (never "0 of 0" green) +setup +run dest.md other.md +[ "$RC" -eq 2 ] && ok "unchanged source -> exit 2" || bad "unchanged case exit $RC" "$OUT" +printf '%s' "$OUT" | grep -q '0 of 0' && bad "printed a 0 of 0 verdict" "$OUT" || ok "no '0 of 0' verdict printed" + +# 5 — usage +cd "$TMP" || exit 2 +"$SUT" >/dev/null 2>&1; RC=$? +[ "$RC" -eq 2 ] && ok "no arguments -> exit 2" || bad "usage exit $RC" + +printf '\ntest-check-moved-text.sh: %d ok, %d FAIL\n' "$PASS" "$FAIL" +[ "$FAIL" -eq 0 ]