Skip to content

Sync ako/mxcli: describe performance and round-trips on MPR v1, git notes, test --db-type - #1340

Merged
ako merged 35 commits into
mendixlabs:mainfrom
ako:main
Oct 8, 2026
Merged

ako merged 35 commits into
mendixlabs:mainfrom
ako:main

Conversation

@ako

@ako ako commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Syncs ako/mxcli:main into mendixlabs/mxcli:main: 15 commits since #1336, landed in the fork through PRs ako#1035–ako#1053.

Describe round-trips on large real-world projects

Most of these were found by describing every flow of a 1,875-flow MPR v1 project.

  • describe microflow no longer takes minutes when a flow calls a rule. One IsRule lookup resolved the module of every rule in the project and took 53 s on MPR v1. The rebuild ran 9 times, so describe went past a 5-minute timeout. Rule lookups are now memoised and compare the bare name first.
  • MPR v1 typed listings filter on $Type before reading unit blobs, through a type index peeked from each blob's first 256 bytes. Previously every listing read the whole .mpr.
  • An unspaced minus after a name is subtraction: $P/Offset-1 and $I/OEE-$B/OEE lexed as hyphenated names, so the describe output did not parse.
  • A blank call argument describes as Param = nothing, which parses and round-trips Studio Pro's Argument: "". Before, describe printed TraceID = ,, which doesn't parse.
  • MDL049 accepts $obj/Assoc/Module.Entity as an object value, the form Studio Pro writes. 42 untouched flows were refused, so exec would not re-apply their own describe output. Measured on 10.24.15 and 11.13.0.

Pages / OData

Commands

Robustness / CI

claude and others added 30 commits October 8, 2026 06:55
SessionMembers rebuilt a process's start time as btime + starttime and
skipped it if that fell more than 1s before notBefore. btime is whole
seconds (up to 1s early) and starttime whole ticks (up to 10ms early),
so the rebuild runs up to ~1.01s early: a process started a few ms
after notBefore was skipped. frac(btime) is fixed per machine, so on a
VM that booted late in a second every session lookup missed a run's own
leader — `run stop` reporting "was not running" for a live run, and
TestRunStop_* / TestSessionMembers_* failing together on CI
(runs 37733157296, 37688181946) while green elsewhere.

The comparison is now startedBefore with a 2s slack; a reused pid comes
from a session that began long before, never seconds. The new test pins
the worst case (boot at X.9999, start 9.9ms into a tick, 1ms after
notBefore) and fails against the 1s slack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw
`make test` ran `go test ./...` with no -timeout, so every test binary got
Go's 10m default. Under ./... all packages run at once and a package's wall
time follows machine load: mdl/linter passes in ~25s alone and was measured
at 552s and 872s inside `make test`, the latter failing with
`panic: test timed out after 10m0s` in a package the PR had not touched.

- test: `-timeout $(TEST_TIMEOUT)`, default 30m (twice the slowest
  measurement), overridable per run.
- check-conformance / conformance-shrink: explicit -timeout 20m (88s alone).
- scripts/check-test-timeouts.sh, wired into `make lint`: refuses any go test
  in the Makefile, direct or through a variable such as INTEGRATION_GO_TEST,
  without a -timeout, and fails if it cannot see the `test` target at all.
- finding appended to findings/other.jsonl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RL7V4XDekBvPTYuKgF96sn
… or role

mendixlabs#1333 reported check --references passing a grant on a
missing entity / to a missing module role, and exec writing the valid
grant above it before failing. Already fixed by 6731d9d
(validateGrantReferences), which landed after v0.25.0; the pre-fix
binary reproduces the report verbatim.

What was not pinned is the issue's harm: the existing tests drive the
reference check, not exec's pre-flight. This test runs the mixed script
through execPreflight and fails when validateGrantReferences is unwired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GSX6tDpzzWLUfX7VDKVfkA
fix(run): tolerate btime truncation in the session pid-reuse guard
…utes

`describe microflow AgentCommons.PromptToUse_ApplyVariable` on Evora
(MPR v1, 29 rules) ran past a 5-minute timeout. The flow is small - three
decisions, no loops - but two of its decisions call a rule.

The canonical describe rebuilds the flow once per derived-layout round
(9 here), and each build asks backend.IsRule for every `if Module.Name(...)`.
The derivation's read cache did not memoise IsRule, and modelsdk IsRule
resolved the module of every rule in the project before comparing names;
each resolution listed the whole project, then the modules once per folder
level. On MPR v1 a listing reads every unit's contents from SQLite, so one
IsRule took 53 s.

- IsRule compares the bare name first and resolves the module only for a
  rule whose name fits.
- moduleNameFor lists the modules once instead of once per ancestor.
- layoutCheckBackend memoises IsRule for the derivation, which writes
  nothing.

Describe output is unchanged; the flow now describes in 6-10 s, like any
other flow in the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…phenated name

describe microflow printed `offset $Pagination/PageSize*($Pagination/Offset-1)`
and `(($I/OEE-$B/OEE) div $B/OEE)*100` verbatim, as Studio Pro stores them,
and the output failed to parse with `missing '('`. HYPHENATED_ID (there for
`starts-with` and Atlas icon names) matched `Offset-1` and the trailing-hyphen
`OEE-`, fusing the member and the operator into one name token.

Every hyphen in HYPHENATED_ID must now be followed by a letter or underscore;
a hyphen before a digit, `$`, space or nothing lexes as MINUS. The stored
expression text is unchanged, so the description re-executes as Unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Studio Pro stores a call microflow / call nanoflow argument field left blank
as a parameter mapping with `Argument: ""`. describe printed it as
`TraceID = ,` / `ID = )`, which does not parse (10 of 1,875 Evora flows).

No existing spelling round-trips it: `Param = empty` stores the Mendix value
"empty" (a different document), omitting the argument drops the mapping, and
`Param = nothing` used to store the text "nothing". callArgument now accepts
`Param = nothing` (ast.BlankExpr, rendered as ""), the visitor refuses it
outside call microflow / call nanoflow, and both call printers render a blank
argument as `nothing` (describeCallArgument in
mdl/executor/cmd_microflows_format_action.go). mx check reports the stored
state as CE0127 "Missing argument", as it does for the Studio Pro original,
so the doctype example lists CE0127 as known.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI runs `make lint-go`, not `make lint`, so the guard added for
mendixlabs#1291 would never have run on a PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RL7V4XDekBvPTYuKgF96sn
MDL049 flagged every call argument whose path ended in a module-qualified
segment as "an association path … CE0117". Mendix paths alternate
association and entity, so a qualified last segment after an association
is the entity step that names the object — the form Studio Pro writes
(`$PageHelper/AgentCommons.PageHelper_Agent/AgentCommons.Agent`). 42
untouched Evora Factory Management microflows were refused, so exec would
not re-apply their own describe output without --no-check.

Measured on mxbuild 10.24.15 and 11.13.0, one microflow per argument:
`$A/M.A_B` CE0117 (still refused); `$A/M.A_B/M.B`, a reference set into a
List parameter, two hops with entity steps, and `$A/M.A_B/M.B/M.B_C` all
0 errors. The rule now fires on the bare one-segment form only, and its
Fix: line names the entity-step spelling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…base

`mxcli test --local` always provisioned a scratch PostgreSQL, so on a
machine without one it stopped with "ensuring database: no local
PostgreSQL superuser available to create the role/database" while
`run --local --db-type hsqldb` served the same project. Carries
mendixlabs#1274 with its review points addressed.

- `mxcli test --db-type postgresql|hsqldb`, same values as run --local,
  validated by testrunner.ResolveTestDBType before anything is built.
  Refused with --attach and on the Docker path, which boot no database
  of their own (accepted-and-ignored is the failure --constant refuses
  for the same reason).
- The headless boot (docker.StartLocalApp) learns the file database:
  applyDefaults no longer fills PostgreSQL's host/credentials into it,
  and the database step (now prepareLocalAppDatabase) skips the ping and
  refuses EnsureDB for it. Before, the type passed through still failed
  with "database not reachable at 127.0.0.1:5432".
- The _test scratch suffix is kept for HSQLDB: the name selects the
  database files (measured: deployment/data/database/hsqldb/probe_test/),
  so the suite never shares data or the file lock with a
  `run --local --db-type hsqldb` dev loop.
- RunOptions.DBType documents the canonical spelling it holds; dbConfig
  translates it to the runtime's, since DBConfig.IsFileBased keys on it.
- test-microflows skill and docs-site running-tests page document it.

Verified on Mendix 11.13.0 with no PostgreSQL running: a suite that
commits a write passes under `--local --db-type hsqldb` and its log
never mentions PostgreSQL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GUDyoCt5SBft1ptKKnRX1a
On an MPR v1 project every typed listing (ListModules, ListFolders,
ListMicroflows, ListConsumedRestServices, ...) and ListUnits read the
Contents blob of every unit and only then filtered by $Type, so each cost a
full read of the .mpr. One `describe microflow` on a 140 MB v1 app did 14 of
them.

Add a v1 type index on the Reader, the counterpart of v2's unitCache: the
type is peeked from the first 256 bytes of the blob ($ID then $Type head
every unit; a full read is the fallback), keyed by UnitID with the blob's
length. Each listing re-reads only ID/container/name/length(Contents) --
answered from the record header, no overflow pages -- re-types new or
resized units, and reads contents by primary key for matches only. The
index needs no invalidation, which matters because v1's updateUnit never
calls InvalidateCache and another process can change the file.

ListUnits now uses the index (v1) or unitCache (v2) and reads no contents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test(exec): pin pre-flight refusal of a grant naming a missing entity or role (mendixlabs#1333)
fix(make): give `make test` an explicit -timeout (mendixlabs#1291)
Bumps [source-map-js](https://github.com/7rulnik/source-map-js) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/7rulnik/source-map-js/releases)
- [Changelog](https://github.com/7rulnik/source-map-js/blob/main/CHANGELOG.md)
- [Commits](7rulnik/source-map-js@v1.2.1...v1.2.2)

---
updated-dependencies:
- dependency-name: source-map-js
  dependency-version: 1.2.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [shell-quote](https://github.com/ljharb/shell-quote) from 1.10.0 to 1.12.0.
- [Changelog](https://github.com/ljharb/shell-quote/blob/main/CHANGELOG.md)
- [Commits](ljharb/shell-quote@v1.10.0...v1.12.0)

---
updated-dependencies:
- dependency-name: shell-quote
  dependency-version: 1.12.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Studio Pro attaches a git note under refs/notes/mx_metadata to every
commit: the Mendix version, the branch, and the model units the commit
changed. Commits an agent made with plain git had none; Studio Pro 11.13
back-fills an "(unknown)" placeholder for them after each background
fetch, and that fetch force-replaces the local notes ref, discarding
notes that were not pushed.

`mxcli git note -p app.mpr [--write]` computes the note from the commit
(the .mpr unit index and mprcontents at the commit and its parent), for
the unpushed commits by default. It keeps notes Studio Pro wrote
(--force replaces them) and replaces placeholders. Measured against the
8 notes Studio Pro wrote on a Team Server repository's MPR v2 history:
header identical on all 8, change list identical on every ordinary
commit; on version upgrades (which Studio Pro annotates itself) it
lists more units. MPR v1 is refused.

The teamserver-git skill teaches the workflow: commit from Studio Pro
when it is open, create branches on the Team Server, push the notes ref
with the branch, never merge/rebase/force-push model history with git.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the findings-file conflict GitHub reports: both sides appended to
mdl-executor.jsonl, which merge=union keeps locally but GitHub's
server-side merge does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the findings-file conflict GitHub reports: both sides appended to
the findings shards, which merge=union keeps locally but GitHub's
server-side merge does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(describe): stop rule splits making describe microflow run for minutes
…endixlabs#1338)

A DataGrid 2 column bound to `changedBy/Name` was written as the flat
attribute 'MyFirstModule.Item.changedBy/Name' and mxbuild failed CE1613.
changedBy and owner are the associations System.changedBy / System.owner
to System.User, which no domain model lists (an entity carries them as
HasChangedBy / HasOwner), so the association lookup found nothing and the
builder fell back to a flat path.

resolveAssociationAttributePath now falls back to systemMemberAssociation
when the modelled lookup fails, gated on the entity (or its generalization
root) storing the member. Covers columns, input widgets and dynamic-text
params, which share the resolver.

Verified on 11.12.5: fixed build mx check clean, describe -> exec reports
Unchanged; the pre-fix binary reproduces both CE1613s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0189h4HBKF5NSe6j4bseaNVD
Brings in #1036 and resolves the findings-file conflict GitHub reports for
appends to mdl-executor.jsonl (merge=union keeps both locally; GitHub's
server-side merge does not run merge drivers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(describe): make every Evora microflow description parse (unspaced minus, blank call argument)
…skills/packs/mendix-vega-charts/widget/source-map-js-1.2.2

build(deps-dev): Bump source-map-js from 1.2.1 to 1.2.2 in /.claude/skills/packs/mendix-vega-charts/widget
…skills/packs/mendix-vega-charts/widget/shell-quote-1.12.0

build(deps-dev): Bump shell-quote from 1.10.0 to 1.12.0 in /.claude/skills/packs/mendix-vega-charts/widget
perf(mpr): filter MPR v1 typed listings before reading unit blobs
mxcli git note + teamserver-git skill: work with Mendix Team Server like Studio Pro (mendixlabs#1337)
…1613)

Exposing a cross-module association on a published OData entity wrote a
PublishedAttribute named after it; mx check then reported
CE1613 "The selected attribute 'Od.Child.Child_Parent' no longer exists."

lookupEntityMembers read only dm.Associations. A cross-module association
is a DomainModels$CrossAssociation in the FROM entity's module, so it is
now looked up there, and in other modules' domain models when published
from the TO side. The executor qualifies the association with its own
module, since the writer could only prefix the published entity's module
(Op.Child_Parent -> CE1613 "selected association" from the TO side).

Fixes mendixlabs#1335

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wqjt38s8gXxwbzkt2dspAN
Brings in #1036, #1037 and #1042 and resolves the findings-file conflict
GitHub reports for appends to mdl-executor.jsonl (merge=union keeps both
locally; GitHub's server-side merge does not run merge drivers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ako and others added 5 commits October 8, 2026 15:09
fix(check): MDL049 accepts `$obj/Assoc/TargetEntity` as an object value
Fix DataGrid columns over System.changedBy/owner associations
feat(test): add --db-type so a --local suite can run on the file database
…tion

fix: publish cross-module associations as PublishedAssociationEnd (CE1613)
@ako
ako merged commit 91ae198 into mendixlabs:main Oct 8, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants