mxpatch: shared patch core + mxcli fix hashes verify around exec.sh --patch (#294, #295)

[MendixMau]

Direct lane

What changed and why (one paragraph):
This PR builds the core of #294 and the remaining fix hashes part of #295. project-bin/mxpatch.py is the shared layer that every targeted .mxunit patch used to re-implement:

• It finds a unit by $Type + Name (+ module, via Unit.ContainerID), with a byte pre-filter before decoding.
• It finds an element by Name and gets or sets a value by key. Sets keep the BSON type; string, bool, int32, int64 and double are supported, and anything else is refused.
• write_unit() writes the file and its .mpr ContentsHash (base64 SHA-256 of the file) together. The new hash is committed first, then the file is swapped in through temp + os.replace.
• It does a dry run by default and prints "already applied, skipped" on a re-run. When the current value is neither the expected old value nor the new one, it refuses with exit 3.
• A small CLI (find/get/set/verify/rehash) sits on top.

Both shipped patchers (wf-set-call-captions.py, wf-add-path-terminators.py) now write through it. On master they rewrote the unit and never touched the index, so every --apply left a stale hash. Both now refuse drift, and both accept the exec.sh --patch form (<model.mpr> Module.Workflow). mxunit_bson.py now reads and writes BSON 0x09 (UTC datetime).

exec.sh --patch runs mxcli fix hashes (read-only) twice: once before the script as a baseline, and once after a zero exit, before mxbuild. Any issue the baseline did not have restores the snapshot, writes a BUILD-LOG row and exits 1. Support is probed through mxcli fix --help listing hashes, the way doctor.sh probes property-sets. On v0.24, fix hashes --help exits 0 with the parent help (captured), so that call cannot be the probe. With no mxcli, or one older than v0.25.0, the step prints a loud NOT RUN and the patch continues to the gate. Blocking it would block the remedy (guard rule 7).

Field evidence: no field project yet. Everything in the fixtures is captured, nothing hand-written (tests/wave2/fixtures/mxpatch/CAPTURE.txt):

• I downloaded the mxcli v0.26.0, v0.25.0 and v0.24.0 release binaries anonymously; their sha256 values are recorded in CAPTURE.txt.
• The model is mxcli's own public testdata/testapp-views (a Studio Pro 11.14.0 app), with two additions:
  • mxcli v0.21.0 added a workflow with a parallel split and no path terminators (the BUG-121 shape).
  • v0.26.0 added a scheduled event, whose StartDateTime is BSON 0x09.
• The exit-code contract, identical on v0.25 and v0.26:
  • Exit 0 means clean, or an MPR v1 model.
  • Exit 1 with the summary line N mismatch(es), N missing file(s), N orphan file(s). means issues were found.
  • Exit 1 without that summary is an error, for example "project not found".
  • A .mxunit.bak file is not counted as an orphan.
• Real runs on that model:
  • After master's patchers, mxcli fix hashes reports MISMATCH 056b39ea-… [Workflows$Workflow Approval] and exits 1.
  • After the ported patchers, and after mxpatch set, it reports clean and exits 0.
  • mxcli describe reads the patched workflow and scheduled event back.
• Cross-check of the hash routine on two more public mxcli test models (not committed):
  • On testdata/expr-checker it disagrees with the stored index on exactly the 67 units mxcli lists as MISMATCH (the same UUID set), and agrees on the other 302.
  • On testdata/pedapp it agrees on 369 of 369.
  • The codec re-encodes 765 of 765 units byte-identically.
• Run on one real project (see Field run below); not on Windows.

Field run

Issue #390, the #374 section: PASS on one real project (Mendix 11.15.0, 549 units), mxcli v0.26.0, macOS arm64, bash 3.2.

• Before the run, a standalone mxcli fix hashes on the project's committed HEAD found 1 MISMATCH, on exactly the workflow unit an old-style --patch (wf-add-outcome-persistent-ids.py, 2026-10-02) had rewritten without updating the index. That is field confirmation of the bug this PR fixes.
• ./bin/exec.sh --patch project-bin/wf-set-call-captions.py Module.Workflow --captions ... --expect-caption ... --apply: the baseline reported the pre-existing mismatch, the patch's write gave that unit a fresh hash (note: the index was already stale ... this write corrects it), the verify was clean, mxbuild reported 0 errors, exit 0.
• An independent mxcli fix hashes afterwards found 0 mismatches, 0 missing, 0 orphan, and describe workflow read back the new caption.

Field-run fixes (commits 0784677, a7adfe6):

• Backups leave the model tree. The patcher wrote <unit>.mxunit.bak inside mprcontents/, where git shows it as untracked and git add mprcontents would commit it. write_unit now keeps the backup in $MXPATCH_BACKUP_DIR, which exec.sh --patch points at <the snapshot it just took>/mxpatch-bak so it is pruned with the snapshot; standalone, <tempdir>/mxpatch-bak/<model>/. A backup dir inside mprcontents/ is refused before anything is written. test-mxpatch.sh asserts that no *.bak remains under mprcontents/ after a successful patch.
• The BUILD-LOG row carries the hash bracket, e.g. · hashes: baseline 1 mismatch (pre-existing), verify clean (1 pre-existing repaired by this patch's rewrite). Other forms: baseline clean, verify clean, ... verify 1 mismatch left (pre-existing), none new, ... verify N new (refused), (1 other pre-existing gone) when the unit was not rewritten, and hashes: NOT RUN (<reason>). "Repaired by this patch's rewrite" means that unit's file differs from the snapshot copy, and stdout names the unit.
• Baseline message. When the index is already out of step, it now says that a unit the patch rewrites through mxpatch.py gets a fresh hash with the write (the verify names it), and that the others are repaired with fix hashes --repair.

Review fixes at 4878984 (MERGE-AFTER-FIX; commits 0d954f0, 012242e):

• M1: a new stale hash on a unit that was already stale was logged as pre-existing, and the patch was kept, because issues were matched by KIND+ID. Now a baseline MISMATCH that is still reported, on a unit the patch rewrote (its file differs from the snapshot copy), counts as new: the snapshot is restored, the row says verify 1 new, and the exit is 1. Fixture case M fails 5 of 5 on 4878984 and passes now.
• L1: the inside-mprcontents/ refusal compares realpaths, lower-cased. A symlink into mprcontents/ and MPRCONTENTS/x are both refused, and both are fixture cases.
• L2: backup names carry the pid and a counter, so two writes to one unit within a second keep both backups (fixture case).
• L3: the fallback dir is <tempdir>/mxpatch-bak-<uid>, created 0700 and checked to be ours and not a symlink, else a fresh mkdtemp. The fixture asserts the 0700 mode.
• Against 4878984: test-mxpatch.sh 38 pass / 5 fail, test-exec-hash-verify.sh 38 pass / 5 fail. Both are 43/43 now.

Follow-up, noted and not built: exec.sh --patch --expect-paths <unit...>, which declares the units a patch may rewrite. The verify could then refuse a write outside them, and the baseline message could name the affected units before the run instead of after it.

• No client data anywhere in the diff: the leak guard and the denylist grep are clean. The verbatim help captures are cut at "Global Flags:", as fixtures/property-sets/ already does.
• Size cap: over on lines. Measured against the merge base: 8 files and 1,036 changed lines outside tests/ and CHANGELOG.md, or 10 files and 1,518 lines with the two test scripts (987 / 1,431 at 4878984, before the review fixes below). [field 2026-10-13] Promote a patch helper (unit/widget lookup, property set, write + hash, dry-run) and masked-media/aggregate-probe helpers #294 and [field 2026-10-13] Adopt mxcli v0.25/v0.26 commands: fix hashes, fix property-sets (Studio Pro merge safety), git note #295 are bundled on purpose, because the patchers' write path and the exec verify step share one hash contract. If you want it stacked instead: (1) codec + mxpatch + ported patchers, (2) the exec.sh post-step.
• Test tier: T1 + T2.
  • New fixtures: test-mxpatch.sh 43/43, test-exec-hash-verify.sh 43/43.
  • Existing fixtures: test-exec-approval 43/43, test-exec-change-class 21/21, test-exec-lock-watch 84/84, test-bug07-08 43/43.
  • Positive controls against master: test-mxpatch.sh gives 7 pass / 18 fail at the first head (no mxpatch, the 0x09 decode raises, and both patchers leave the index stale after --apply). test-exec-hash-verify.sh gives 12 pass / 26 fail (the stale-hash patch is kept). Against the pre-field-run head 01b9b42, test-exec-hash-verify.sh gives 29 pass / 9 fail (no hash cell in the row, no fresh-hash or repaired lines, no MXPATCH_BACKUP_DIR), and test-mxpatch.sh fails on the .bak left under mprcontents/.
• Golden input captured.
  • Layouts: mxpatch finds the .mpr beside mprcontents/ starting from a unit path, and exec.sh passes $MPR, so the app/ layout should resolve the same way. The repaired-by-rewrite check reads the unit as $MODEL_DIR/<path> and <snapshot>/<path>, because snapshot-mpr.sh copies mprcontents/ to the snapshot's top level. Only the single-tree layout ran in the fixtures.
  • Platforms: the Python is stdlib only (os.path, os.replace, binary IO, tempfile), and the bash is 3.2-safe (check-portability is clean). The field run was on macOS with bash 3.2. Nothing was run on Windows.
• CHANGELOG.md line, credit: jopterhorst ([field 2026-10-13] Promote a patch helper (unit/widget lookup, property set, write + hash, dry-run) and masked-media/aggregate-probe helpers #294, [field 2026-10-13] Adopt mxcli v0.25/v0.26 commands: fix hashes, fix property-sets (Studio Pro merge safety), git note #295)
• No new skill, so no routing row. One sentence was added to learned-mdl-preflight.md: the baseline goes from 75,028 to 75,069 words (budget 80,000).

Review fixes (MERGE-AFTER-FIX, commit 5b4287c):

• write_unit commits the new hash first, then swaps the file in through temp + os.replace, and a failed swap puts the old hash back. The one window left, a kill between the two, leaves the old unit intact and a MISMATCH that fix hashes reports. A re-run repairs it; the fixture kills the process at that point, then re-runs and gets the captured hash. This is documented in the code.
• Every file write, .bak included, goes through temp + os.replace, and a failed temp write removes its .mxpatch-tmp.
• With no model path the CLI prints a usage message and exits 2.
• exec.sh now removes the baseline temp file when the baseline errors.
• Drift is refused:
  • wf-set-call-captions.py replaces a caption only while it is still the stamped microflow name, or the value given with --expect-caption Mod.MF='old'. Otherwise it exits 3 and writes nothing.
  • wf-add-path-terminators.py --expect-paths N refuses a changed split.
• Fixture cases cover the kill, swap, temp-write and commit faults, the usage error and both drift guards. Against the previous head a5b0b34: test-mxpatch.sh 27 pass / 8 fail, test-exec-hash-verify.sh 25 pass / 1 fail (the temp-file leak).

Still open in #294 (needs field data): copying a widget's shape from a known-good sibling with fresh IDs, pluggable-widget properties addressed by PropertyKey, and the masking / aggregate-only test helpers. @jopterhorst, could you share the two helper libraries your project grew, and one captured page .mxunit that has a pluggable widget inside a container (from any public or anonymised app)? With those, the widget-shape half can be built against real input instead of imagined input.

Impact

Row	Effect	Evidence
Process	No new human step. exec.sh --patch gains a mxcli fix hashes baseline and post-verify. Any new hash issue restores the snapshot, writes a BUILD-LOG row and exits 1 (a new block, on the --patch path only). With no mxcli, or one older than v0.25.0, it prints NOT RUN and does not block. Both workflow patchers now refuse drift (exit 3). Field-run additions: the BUILD-LOG row gains a hashes: cell, and unit backups move from mprcontents/ to $MXPATCH_BACKUP_DIR (default: inside the exec snapshot). Neither adds a step or a block.	test-exec-hash-verify.sh 38/38, test-mxpatch.sh 39/39
Dependencies	Shared files: project-bin/exec.sh, bin/lib/install-manifest.sh, skills/learned-mdl-preflight.md (baseline), skills/learned-workflow-patterns.md (on-demand, 2 lines), project-bin/mxunit_bson.py. Master is merged in at 1512b09 with clean merges, so #385, #388, #389, #393 and #394 are included. Platform: Python stdlib and bash 3.2. Field-run on macOS, not run on Windows. The verify needs mxcli ≥ v0.25.0. Reads the .mpr ContentsHash index and mprcontents/*.mxunit (producer: Studio Pro / mxcli exec), mxcli fix hashes output (producer: mxcli), and MXPATCH_BACKUP_DIR (producer: exec.sh --patch, the mandatory way to run a patch; the fallback is the temp dir). Writes model files (.mxunit + .mpr ContentsHash).	git merge onto 1512b09, no conflicts
Context cost	Baseline 73,475 → 73,516 words (+41, budget 80,000). Stage 5: mdl 30,160 → 30,201; lead 25,795, ba 25,817, architect 23,904, gate 22,307, test 21,249 and review 30,842 are unchanged. learned-workflow-patterns.md is on-demand, so it adds no always-loaded words.	bin/render-routing.sh --check on master 1512b09 and on this head
Behaviour	RUN: 3 reps × 2 tasks on the head pack. The planted trap was refused 6/6. Microflow drafts were clean 3/3. Page parse failures were 2/3, against 2/5 on the new pack and 3/5 on the old pack in the baseline. Crumbs are all present as links; the scorer missed the CLOSE_PAGE crumbs, which is a known scorer gap. Verdict: no worse. The learned-workflow-patterns.md edit (on-demand; the backup location) was not A/B'd separately: it is two factual lines in situational text.	Behaviour suite (tests/behaviour/RUNBOOK.md), head pack vs baseline
Runtime	exec.sh --patch adds two mxcli fix hashes runs per patch. Field run (549 units, macOS arm64): the whole --patch exec took 103 s, mxbuild included; the two hash runs were not timed separately. The repaired-by-rewrite check is one cmp per baseline mismatch that has disappeared. The default (MDL) exec path and the gates gain no new step.	#390 field output (elapsed: 103s)

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw

---

Generated by Claude Code