diff --git a/.claude/skills/debug-bson.md b/.claude/skills/debug-bson.md index dc0f5007fd..85a1c19a16 100644 --- a/.claude/skills/debug-bson.md +++ b/.claude/skills/debug-bson.md @@ -360,7 +360,7 @@ for t, props in crash_props.items(): **Investigation methodology used for v0.10 CE0463 fixes** — see [WIDGET_BSON_VERSION_COMPATIBILITY.md](../../docs/03-development/WIDGET_BSON_VERSION_COMPATIBILITY.md) for the full case study and version-resilience model. -**Quick workaround** (if you can't fix the root cause): normalize with `mxcli docker check`/`build`, which run the widget update **and preserve MPRv2 storage** (they snapshot/restore `.mpr` + `mprcontents/`). Do **not** run bare `mx update-widgets` on a v2 project you care about — it converts to single-file v1 and deletes `mprcontents/` (corrupts git, breaks `mxcli run --local`). Raw `mx update-widgets` is fine only on a throwaway diagnostic copy or a v1 project. +**Quick workaround** (if you can't fix the root cause): normalize with `mxcli fix widgets -p app.mpr`, which runs the widget update on the project **and preserves MPRv2 storage**. Not `mxcli docker check`/`build`: they run the update on a **temporary copy**, so a CE0463 the normalization clears is not reported while the stored project still fails MxBuild — use `docker check --no-update-widgets` to see the project as stored (and [`diagnose-ce0463.md`](./diagnose-ce0463.md) before calling it widget drift). Do **not** run bare `mx update-widgets` on a v2 project you care about — it converts to single-file v1 and deletes `mprcontents/` (corrupts git, breaks `mxcli run --local`). Raw `mx update-widgets` is fine only on a throwaway diagnostic copy or a v1 project. ### CE0642: Property X Is Required @@ -440,7 +440,7 @@ for _, pm := range a.ParameterMappings { 2. **Mode-dependent properties must be consistent**: When changing a mode-switching property (e.g., `showContentAs`), all dependent properties must be updated to match. -3. **Widget normalization is the safety net — via `mxcli docker check`/`build`**: they run the update-widgets normalization *and* preserve MPRv2 storage (snapshot/restore). Bare `mx update-widgets` does the same normalization but rewrites a v2 project to v1 and deletes `mprcontents/` — only use it on a v1 project or a throwaway diagnostic copy. +3. **Widget normalization is the safety net — via `mxcli fix widgets`**: it applies the update-widgets normalization to the project *and* preserves MPRv2 storage. `mxcli docker check`/`build` normalize only a temporary copy, which hides CE0463 without fixing the stored project. Bare `mx update-widgets` does the same normalization but rewrites a v2 project to v1 and deletes `mprcontents/` — only use it on a v1 project or a throwaway diagnostic copy. 4. **The mpk is the source of truth**: The XML schema defines property types/defaults, the editorConfig.js defines visibility rules. Together they specify the complete expected Object structure. diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-08-describe-subcommand-rejects-published-rest-service.json b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-08-describe-subcommand-rejects-published-rest-service.json new file mode 100644 index 0000000000..714a6ebdde --- /dev/null +++ b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-08-describe-subcommand-rejects-published-rest-service.json @@ -0,0 +1 @@ +{"area": "cmd/mxcli", "date": "2026-10-08", "symptom": "`mxcli describe published rest service M.Api` prints \"Unknown type: published rest service\", and auto-detect (`mxcli describe M.Api`) says no describable document is found even when the catalog lists the service, while `describe published rest service M.Api` works under `mxcli -c` / exec / the REPL", "cause": "The describe subcommand keeps its own type-keyword -> MDL DESCRIBE switch, plus two auto-detect maps (catalog ObjectType and unit $Type -> keyword), none of which had an entry for published REST services; the MDL grammar and executor did", "file": "cmd/mxcli/cmd_describe.go", "fix": "Dispatch moved into the pure describeMDLCommand(); added publishedrestservice / published rest service / restservice / rest service, PUBLISHED_REST_SERVICE and Rest$PublishedRestService", "insight": "The subcommand is a second, hand-kept copy of the DESCRIBE grammar's type list, so a new doctype in the grammar does not reach it. TestDescribeMDLCommand_AutoDetectKeywordsAccepted now asserts every auto-detect keyword dispatches and its MDL parses; it would also have caught the old maps pointing at a keyword the switch lacked. Wrong turn to skip: testing auto-detect without a catalog — the live-reader fallback (resolveViaReader) calls ListRawUnits, which the model engine does not implement, so it finds no document of any type; that is a separate defect.", "refs": ["mendixlabs/mxcli#1347"]} diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-08-run-local-watch-windows-deployment-web-in-use.json b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-08-run-local-watch-windows-deployment-web-in-use.json new file mode 100644 index 0000000000..e326533956 --- /dev/null +++ b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-08-run-local-watch-windows-deployment-web-in-use.json @@ -0,0 +1 @@ +{"date": "2026-10-08", "area": "cmd/mxcli", "symptom": "Windows, `run --local --watch` (Mendix 11.12): page and microflow edits hot-reload, but adding a persistent entity or attribute fails every rebuild with \"The process cannot access the file '...\\deployment\\web' because it is being used by another process\"; the build has emptied web/ and the app 404s until `run --local` is restarted", "cause": "The incremental rollup bundler is started with cmd.Dir = deployment/web, and Windows refuses to remove a directory that is any process's working directory. Page/microflow edits rewrite files inside web/ and are unaffected; a domain model change makes mxbuild recreate web/ itself. Killing rollup-runner by hand looked like no help because watchAndApply's EnsureAlive restarts the bundler just before the next serve build", "fix": "bundlerSupervisor.BuildReleasingWebDir wraps the watch loop's serve build: on a sharing-violation failure naming deployment/web (webDirInUse), stop the bundler, retry the build, then Restart a fresh bundler whose first build is a full bundle (so the AwaitRebuild wait is skipped). A failed retry leaves the bundler down for EnsureAlive", "insight": "Do not move the bundler's cwd out of web/ to dodge the lock: mxbuild's rollup-plugin-mendix-pages resolves page paths with path.relative(cwd, id) (see the 2026-10-02 page-glob finding), so that breaks page bundling. Retry-on-failure keeps the ~4s incremental path for page edits; stopping the bundler before every build would cost each of them a cold bundle. The Windows test proves both premises on a real process: os.Remove(web) fails with the same sharing violation while a child has it as cwd, and succeeds after WebClientWatcher.Stop. Wrong turn to skip: removing web/ straight after cmd.Start succeeds and looks like a disproof — the child takes its cwd handle during its own initialisation, after Start returns, so wait for the child to announce itself first. E2E on Windows 11 arm64: 11.13.0 (rollup.config.mjs present) with the retry disabled reproduces the report exactly (build failed on deployment\\web, then / and /dist/index.js 404); the fixed binary logs the retry and applies via restart (~26-32s, 200s after), twice in a row, and a page edit afterwards still re-bundles incrementally (reload 7.6s, gen 1->2, no bundler restart) — so nothing else (runtime JVM, Gradle daemon) holds web/. 11.12.6 does NOT reproduce: its mxbuild writes web/dist and no rollup config (planWebClient=prebuilt, no mxcli bundler), so the domain change applied on the unfixed binary — pick a version by the deployment shape, not the minor number", "file": "cmd/mxcli/docker/webclient_supervisor.go (BuildReleasingWebDir, webDirInUse); cmd/mxcli/docker/runlocal.go (watchAndApply)", "test": "cmd/mxcli/docker/webclient_supervisor_test.go; cmd/mxcli/docker/webclient_webdir_windows_test.go", "refs": ["mendixlabs/mxcli#1342"]} diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-09-check-script-set-files-resolved-in-isolation.json b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-09-check-script-set-files-resolved-in-isolation.json new file mode 100644 index 0000000000..9be2e3113a --- /dev/null +++ b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-09-check-script-set-files-resolved-in-isolation.json @@ -0,0 +1 @@ +{"area":"cmd/mxcli","date":"2026-10-09","symptom":"`mxcli check a.mdl b.mdl -p app.mpr` reported a combobox datasource in b.mdl naming an entity a.mdl creates as \"entity not found\", although `exec` on each file in order succeeds and the same statements in ONE file pass `check -p` (reported on v0.25.0, Mendix 11.12.1). On current main the page parameter's entity is reported first (\"parameter $Order of page …: entity … does not exist\"). Issue mendixlabs/mxcli#1355","cause":"runCheckFiles called runCheckFile once per file against --project as it stands; the only set-level pass was MDL-STUB01. \"References to objects created within the script are skipped\" held per file, never across the set, so every reference kind (entity, association, page, module) to an earlier file's creation failed — not just the widget datasource the report names.","file":"`cmd/mxcli/cmd_check.go` (runCheckFiles / runCheckFileAgainst / applyToScratch), `mdl/scriptdiff/scriptdiff.go` (Scratch: NewScratch / Apply / Close); test `cmd/mxcli/check_script_set_refs_test.go`","insight":"**A report naming one reference kind is the first symptom the reporter hit, not the blast radius** — the reported kind (widget datasource) was a red herring: the files were checked in isolation, so the fix belongs in the set runner, not in the widget resolver, and the reporter's \"the page parameter is not flagged\" did not hold on main. Skip the tempting fix of seeding earlier files' definitions into scriptContext: every reference pass in validateProgramWithWarnings (grants, forward refs, members, XPath, ALTER dry-runs, project conflicts) builds its own context from the program, and error messages mix `statement N:` with unprefixed text, so neither seeding nor filtering a concatenated program is faithful. Running the earlier files for real on a scratch copy (scriptdiff's copy + outsideGuard, which refuses SQL/IMPORT/foreign CONNECT) makes file N see exactly what exec leaves, for every reference kind at once. A file whose own check fails is NOT applied — exec's pre-flight would write nothing — and the output says which files the later ones were checked without. Controls in the test: wrong file order still fails, a datasource entity no file creates is still \"entity not found\", the project tree is byte-identical afterwards; with the scratch line removed the test fails with the parameter error. Repro `mdl-examples/bug-tests/1355-script-set-a-domain.mdl` + `1355-script-set-b-page.mdl`, checked together.","refs":["mdl-examples/bug-tests/1355-script-set-a-domain.mdl","mdl-examples/bug-tests/1355-script-set-b-page.mdl","cmd/mxcli/check_script_set_refs_test.go"]} diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-09-fmt-upgrade-test-file-header-docs-said-none.json b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-09-fmt-upgrade-test-file-header-docs-said-none.json new file mode 100644 index 0000000000..d5a9ea6af2 --- /dev/null +++ b/.claude/skills/fix-issue/findings/cmd-mxcli/2026-10-09-fmt-upgrade-test-file-header-docs-said-none.json @@ -0,0 +1 @@ +{"area": "cmd/mxcli", "date": "2026-10-09", "symptom": "fmt --upgrade adds `mdl 1;` to a .test.mdl, but fmt --help and the test-microflows skill say test files take no header", "cause": "ako/mxcli#847 made the runner and check read a test file header and made fmt --upgrade add it by default, and updated the CHANGELOG, docs-site test-formats.md and the code comment beside the branch; the fmt --help paragraph and the skill sentence describing the pre-#847 behaviour (\"takes no language header yet, so --header adds none to it\") were not touched, so the behaviour was right and the two texts an agent reads first contradicted it.", "file": "cmd/mxcli/cmd_fmt.go", "fix": "Both texts now say a test file gets the header by default and name --header=false as the way to decline it; cmd_fmt_testfile_header_doc_test.go runs the reporter file through fmt --upgrade and fails if either text again says a test file takes no header or stops naming --header=false.", "insight": "When a report pits behaviour against docs, settle which side is intended from the CHANGELOG entry that introduced the behaviour before touching code: here #847 said outright that --upgrade adds the header to test files, so the fix is text-only and changing fmt would have regressed #847. The tell for a stale sentence is the word \"yet\": a doc that says a feature does not exist yet must be grepped for when the feature lands, and a test that pins the doc to the behaviour on one input is what keeps the next such change from leaving it behind.", "refs": ["mendixlabs/mxcli#1356", "ako/mxcli#847"]} diff --git a/.claude/skills/fix-issue/findings/mdl-backend/2026-10-09-describe-autodetect-without-catalog-finds-nothing.json b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-09-describe-autodetect-without-catalog-finds-nothing.json new file mode 100644 index 0000000000..010da7dda7 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-backend/2026-10-09-describe-autodetect-without-catalog-finds-nothing.json @@ -0,0 +1 @@ +{"area": "mdl/backend", "date": "2026-10-09", "symptom": "With no .mxcli/catalog.db, `mxcli describe -p app.mpr Mod.Name` (type auto-detected) reports `no describable document named \"Mod.Name\" found in the project` for a microflow, page or any other document; only entities and associations resolve, and the same command works once `refresh catalog` has run", "cause": "describe's live fallback (resolveViaReader) calls ListRawUnits on the backend.FullBackend that the root package's Open returns; *Backend never declared it, so the unimplemented stub answered with an error the fallback discards. unimplemented_reachability_test.go recorded it as an accepted BYPASS ('describe holds a concrete reader'), true before Open stopped aliasing sdk/mpr and false after", "file": "mdl/backend/modelsdk/raw_lookup.go", "fix": "Backend.ListRawUnits delegates to the codec reader (which already had it); entry removed from unreachableUnimplemented", "insight": "The reachability record is a measurement with a date, and the reason column goes stale when a caller's type changes underneath it — `scripts/backend-reachability.sh ListRawUnits` said LIVE in one run and named the call site. Re-run the probe on any entry whose reason names a caller rather than trusting the column. The symptom hid because resolveViaReader ignores the ListRawUnits error (`if err == nil`), turning 'not implemented' into 'not found'.", "refs": ["mendixlabs/mxcli#1347"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-check-false-widget07-showfooter-builder-read-property.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-check-false-widget07-showfooter-builder-read-property.json new file mode 100644 index 0000000000..7aea6fac28 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-check-false-widget07-showfooter-builder-read-property.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-10-08", "symptom": "`mxcli check` warns MDL-WIDGET07 \"property `showFooter` is not recognized and will be silently dropped on write\" on a dataview, yet `exec` writes it and `describe page` emits `ShowFooter: true` — so re-executing describe's own output warns too. Same false warning for `Menu:` / `Profile:` / `Orientation:` on navigationtree / menubar and `SizeMode:` on a scroll-container `region`", "cause": "`staticWidgetKnownProps` (the MDL-WIDGET07 allow-list) is hand-maintained, and its drift guard `TestStaticWidgetKnownPropsCoverDescribe` was a second hand-typed list. #813 added the builder read (`lookupPropCI(w, \"ShowFooter\")`) and the describe emission but neither list", "file": "`mdl/executor/validate_widgets.go` (`staticWidgetKnownProps`), guard in `mdl/executor/validate_widgets_known_props_1346_test.go`", "insight": "Replace the hand-typed guard with a source scan: `go/parser` over `cmd_pages_builder*.go` for `Get*Prop(\"K\")`, `lookupPropCI(w, \"K\")` and `Properties[\"K\"]`, plus the `\"K: \"` strings in `cmd_pages_describe_output.go`. Its first run found the reported key and four more nobody had reported (SizeMode is used by a checked-in example, `bug-tests/573-simple-menu-bar.mdl`). The only exclusions are `ImageUrlParams` / `AlternativeTextParams`, emitted solely for the PLUGGABLE `image` widget, which MDL-WIDGET01 validates — probing with `staticimage` instead of `image` made them look like false positives too. Issue #1346", "refs": ["#1346", "#813"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-describe-nanoflow-omits-grant-execute-round-trip.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-describe-nanoflow-omits-grant-execute-round-trip.json new file mode 100644 index 0000000000..05b8af4b19 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-describe-nanoflow-omits-grant-execute-round-trip.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-10-08", "symptom": "`describe nanoflow` omits `grant execute on nanoflow \u2026 to \u2026;` for a nanoflow with allowed roles, so a describe -> exec round trip (or renamed copy) silently leaves it with no roles; `show access on nanoflow` lists them and `diff-local` prints the grant", "cause": "`describeNanoflow` kept its own copy of the flow renderer without the grant block that `describeMicroflowMode` and `renderMicroflowMDL` each carried as a hand-copied loop", "file": "`mdl/executor/cmd_microflows_show.go` (`describeNanoflow`, new `flowGrantLines`)", "insight": "When one doctype's output is right on one path and wrong on another, diff the renderers, not the backend: the data was read correctly (show access proved it), and the 2026-06-30 finding for the same symptom was a different cause (backend never populated roles). Three renderers for one output is the defect \u2014 the grant block is now one helper, and the test parses the describe output and asserts a GrantNanoflowAccessStmt rather than grepping for text.", "refs": ["mendixlabs/mxcli#1345"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-describe-navigation-menu-caption-apostrophe.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-describe-navigation-menu-caption-apostrophe.json new file mode 100644 index 0000000000..b9e91c79b4 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-describe-navigation-menu-caption-apostrophe.json @@ -0,0 +1 @@ +{"area":"mdl/executor","date":"2026-10-08","symptom":"`describe navigation` / `describe menu` writes a menu caption with an apostrophe unescaped (`menu item 'Customer's orders'`), so the output fails `mxcli check`/`exec` with `line 7:20 extraneous input 's' expecting {MENU_KW, '}'}` and a describe -> exec round trip loses the navigation statement","cause":"`printMenuMDL` formatted captions as `'%s'` instead of `mdlQuote`; the menu-item notes in `menuItemActionMDL` did the same. `cmd_navigation.go` was not in `TestDescribers_HaveNoHandRolledStringLiterals`' file list, so the #1006 guard never scanned it","file":"`mdl/executor/cmd_navigation.go` (`printMenuMDL`, `menuItemActionMDL`, new `menuCaptionNote`)","insight":"**The #1006 source-scan guard only covers the files on its list — a describer missing from the list is unguarded, and adding the file is the first step of any quoting fix (it listed all nine sites here at once).** Notes inside `--` comments quote with `mdl0Quote`, not `mdlQuote`: under mdl 1 a caption's line break is written verbatim and would end the comment. Prose status messages (`Navigation profile %s updated.`) switch to `mdlQuoted` rather than widening the guard's skip rule. Guard `TestDescribeNavigation_CaptionWithApostropheRoundTrips`; control: unfixed binary on testdata/testapp-views describes `menu 'Customer's menu'` and `check` fails at `line 7:17`; fixed, describe -> exec -> describe is byte-identical. Repro `mdl-examples/bug-tests/1343-navigation-caption-apostrophe.mdl`","refs":["mendixlabs/mxcli#1343","mendixlabs/mxcli#1006"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-restrict-delete-error-message-always-en-us.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-restrict-delete-error-message-always-en-us.json new file mode 100644 index 0000000000..4036ddf09c --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-08-restrict-delete-error-message-always-en-us.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-10-08", "symptom": "`on delete restrict error message '…'` (and `delete_behavior prevent error_message`, `alter association … set on delete restrict error message`) stores the ChildErrorMessage Texts$Text with one translation whose LanguageCode is always en_US; on a project whose default language is nl_NL the app has no message in its own language when a delete is refused. `mx check` reports 0 errors", "cause": "`deleteErrorText` hardcoded `{\"en_US\": msg}`; `sdk/domainmodel.DeleteBehavior` carried the message as a bare string with no language, so the executor could not pass `authoringLanguage(ctx)` down; `deleteErrorMessageFromGen` read en_US first, so DESCRIBE, the cross-module patch's no-op compare, and the alter read-back all looked at the wrong translation", "file": "`sdk/domainmodel/domainmodel.go` (`DeleteBehavior.ErrorMessageLanguage`, `ErrorMessageTranslations`), `mdl/executor/cmd_associations.go` (create/alter set the language, `deleteMessageIn` for describe + verify), `mdl/backend/modelsdk/domainmodel_write.go` (`deleteErrorText`), `domainmodel_alter.go` (`patchCrossDeleteErrorMessage` edits one language, keeps the rest), `domainmodel.go` (read side)", "insight": "**A #970-class miss: any text carried through a semantic struct as a bare `string` has nowhere to hold its language — grep the sdk types for `string` fields that the backend turns into a Texts$Text, not for `\"en_US\"`.** Fixing the backend alone does nothing: measured on 11.12.4 with a build that had the backend change but not the executor's, the stored code was still en_US — the language must be named where `authoringLanguage(ctx)` is in scope. The read side needs every translation, not one string, or DESCRIBE and the alter read-back compare against the stale en_US that CarryTranslations keeps. Oracle is the stored LanguageCode (`strings` on the .mxunit is enough), not `mx check`. Guards `TestCreateAssociation_DeleteMessageUsesProjectLanguage`, `TestDeleteErrorMessage_StoredInTheGivenLanguage`; repro `mdl-examples/bug-tests/1344-delete-message-default-language.mdl`", "refs": ["mendixlabs/mxcli#1344", "mendixlabs/mxcli#970"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-09-check-passed-create-module-role-in-missing-module.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-09-check-passed-create-module-role-in-missing-module.json new file mode 100644 index 0000000000..a32bf05be7 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-09-check-passed-create-module-role-in-missing-module.json @@ -0,0 +1 @@ +{"area":"mdl/executor","date":"2026-10-09","symptom":"`mxcli check -p` printed \"Check passed!\" (exit 0) for `create module role MyModule.Writer; create module role NoSuchModule.Reader;`; `exec` created MyModule.Writer, then failed with \"module not found: NoSuchModule\" (exit 1), leaving Writer in the model (reported on v0.25.0, Mendix 11.12.1). Issue mendixlabs/mxcli#1354","cause":"validateWithContext's per-statement switch resolves the module of every module-scoped CREATE (entity, enumeration, microflow, …) but had no case for CreateModuleRoleStmt, and validateGrantReferences only resolves the roles a grant NAMES, never the statement that creates one — so nothing at check time looked up the module execCreateModuleRole stops on.","file":"`mdl/executor/validate.go` (CreateModuleRoleStmt case); test `mdl/executor/validate_module_role_module_test.go`","insight":"**The check-tier gap is per statement type, so grep the switch for the AST type before reading any resolver.** `grep -n 'CreateModuleRoleStmt' mdl/executor/validate*.go` showed it only in the grant resolver's record/recordLater bookkeeping — the absence of a `case` in validateWithContext was the whole bug. Skip the plausible wrong turn of extending grantResolver: it resolves what a statement *names*, and widening it to the statement's own container would duplicate the findModule check every other CREATE already does in validate.go. Deliberately NOT extended to `drop module role`: execDropModuleRole calls findModule before honouring IF EXISTS, so exec refuses `drop module role if exists Missing.R` — which contradicts #531 (IF EXISTS tolerates a missing module); check mirroring that would lock in the exec inconsistency. Control: the pre-fix binary on PedApp printed \"Created module role: Administration.Writer\" then \"Error: module not found: NoSuchModule\"; fixed, the exec pre-flight refuses and Writer is absent. Repro `mdl-examples/bug-tests/1354-module-role-missing-module.mdl`.","refs":["mdl-examples/bug-tests/1354-module-role-missing-module.mdl","mdl/executor/validate_module_role_module_test.go"]} diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-10-09-odata-canbeempty-carried-onto-new-key-member.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-09-odata-canbeempty-carried-onto-new-key-member.json new file mode 100644 index 0000000000..a9f608f789 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-10-09-odata-canbeempty-carried-onto-new-key-member.json @@ -0,0 +1 @@ +{"area": "mdl-executor", "date": "2026-10-09", "symptom": "`create or modify published odata service` that adds `(KEY)` to a member already published without it reported 'Modified', `mxcli check` passed, and mxbuild refused it: CE0309 \"Exposed attribute 'Email' of entity 'Customer' that is part of the key cannot be marked as 'Can be empty'\"; a fresh `create` of the same statement built clean", "cause": "`carryPublishedEntityState` (#743) carries each member's stored CanBeEmpty by entity + member name, because MDL has no spelling for it. A non-key member is stored CanBeEmpty true; once it became a key, the carry wrote that true onto a key member instead of the derived `!IsPartOfKey`", "file": "`mdl/executor/cmd_odata.go` (`carryPublishedEntityState`)", "insight": "A carried value is valid only under the conditions it was stored in. When the statement CAN change one of its inputs (here the key status), the carry must be keyed on that input as well, or it turns a legitimate edit into an invalid model. The repro that settled it: same script, same project, old and new binary — CE0309 vs 0 errors. Plausible wrong turn: I first blamed the entity's NOT NULL change made between the runs, because that was the last thing I edited. The order of my own edits is not evidence; reading the carry was. Test: `TestModifyODataService_CanBeEmptyNotCarriedAcrossKeyChange` (with a control member whose key status is unchanged and still carries)", "refs": ["#743", "mendixlabs/mxcli#1357"], "ce": ["CE0309"]} diff --git a/.claude/skills/fix-issue/findings/modelsdk/2026-09-25-security-patch-dead-reconciler.json b/.claude/skills/fix-issue/findings/modelsdk/2026-09-25-security-patch-dead-reconciler.json new file mode 100644 index 0000000000..329adce4bb --- /dev/null +++ b/.claude/skills/fix-issue/findings/modelsdk/2026-09-25-security-patch-dead-reconciler.json @@ -0,0 +1 @@ +{"area": "modelsdk", "date": "2026-09-25", "symptom": "modelsdk/mpr/security_patch.go (PatchReconcileMemberAccesses) matched NoGeneralization keys \"HasOwner\"/\"HasChangedBy\"; the stored keys are \"HasOwnerAttr\"/\"HasChangedByAttr\", so its System.owner / System.changedBy branch never fired", "cause": "Not a live bug: the file had no production callers (UPDATE SECURITY and the marketplace step use the backend's ReconcileMemberAccesses in mdl/backend/modelsdk/domainmodel_security_write.go). And the dead branch encoded the OPPOSITE of the measured rule — it would preserve and add a MemberAccess for the audit associations, which mxbuild rejects", "file": "`modelsdk/mpr/security_patch.go` + `security_patch_test.go` (deleted). Live rule and its test: `mdl/backend/modelsdk/domainmodel_security_write.go` (ReconcileMemberAccesses), `access_rule_audit_member_test.go`", "ce": ["CE0066"], "insight": "Do NOT 'fix' a wrong BSON key without checking what the branch it unlocks does: here the typo was the only thing keeping dead code from writing CE0066. Re-measured on mxbuild 11.14.0, one entity with AutoOwner + AutoChangedBy, one rule: no audit MemberAccess -> 0 errors; + MemberAccess System.owner and System.changedBy -> CE0066 'Entity access is out of date' (matches ako/mxcli#554). Before fixing any function, grep for callers of every exported symbol in the file — a zero-caller file is a deletion, not a fix. Harness note for hand-injecting BSON with pymongo: element $ID must be Binary subtype 0; subtype 3 makes mx fail with the baffling \"Expected '$ID' as the first property of a storage object, but got '$ID'\"", "refs": ["ako/mxcli#554"]} diff --git a/.claude/skills/fix-issue/findings/other/2026-10-09-docs-site-examples-fail-check-and-build.json b/.claude/skills/fix-issue/findings/other/2026-10-09-docs-site-examples-fail-check-and-build.json new file mode 100644 index 0000000000..fa365fa4ac --- /dev/null +++ b/.claude/skills/fix-issue/findings/other/2026-10-09-docs-site-examples-fail-check-and-build.json @@ -0,0 +1 @@ +{"area": "skills", "date": "2026-10-09", "symptom": "24 complete MDL examples in docs-site/src parsed but failed `mxcli check` (MDL020 CreatedDate, MDL021 Id/Type, MDL023 unseeded AutoNumber, MDL-WIDGET34 attributes outside a data container in snippets, MDL-SEC20, MDL-GRANT02 grant to a user role, MDL-PAGE01, MDL-ODATA05, MDL008); building the fixed examples with mxbuild 11.12.1 found three more the linter passes: the CRM validation microflow did not parse at all (`NOT contains(…)`), the CRM demo-user password broke the new-project 12-character policy, and the quick-reference published OData service had no KEY (CE6585, then CE6624/CE0309 until the key attribute is NOT NULL UNIQUE)", "cause": "docs-site was outside every semantic gate; a headerless complete script is indistinguishable from a fragment, so a parse error there reads as 'fragment' to both check-skill-mdl.sh (which skips microflows) and the semantic gate, and only the conformance gate's `mdl 1;`-header rule holds a block to parse", "file": "`docs-site/src/**` examples; gate scope in Makefile `check-skill-mdl` and push-test.yml", "insight": "Check-clean is not build-clean: after the linter was satisfied, one mxbuild run over the fixed examples found three more defects, one of them a block that never parsed. Exec every example from a page into ONE fresh project (`mxcli new … --skip-init`, exec each block, `docker check`) — it costs one build and catches what no per-block rule can (missing KEY, project password policy, cross-block param names like `VAL_Customer(param = …)`). Give each complete example an `mdl 1;` header: that is what makes the conformance gate hold it to parse. Side observation, fixed in the same PR: the CE0309 seen while iterating came from re-running `create or modify published odata service` after adding `(KEY)` — the stored non-key CanBeEmpty true was carried onto the new key (see 2026-10-09-odata-canbeempty-carried-onto-new-key-member.json), not from the entity's NOT NULL change", "refs": ["mendixlabs/mxcli#1357"], "rules": ["MDL020", "MDL021", "MDL023", "MDL-WIDGET34", "MDL-SEC20", "MDL-GRANT02", "MDL-PAGE01", "MDL-ODATA05", "MDL008"], "ce": ["CE6585", "CE6624", "CE0309", "CE7247"]} diff --git a/.claude/skills/fix-issue/findings/other/2026-10-09-skill-examples-marked-correct-fail-mxcli-check.json b/.claude/skills/fix-issue/findings/other/2026-10-09-skill-examples-marked-correct-fail-mxcli-check.json new file mode 100644 index 0000000000..b6c12d942e --- /dev/null +++ b/.claude/skills/fix-issue/findings/other/2026-10-09-skill-examples-marked-correct-fail-mxcli-check.json @@ -0,0 +1 @@ +{"area": "skills", "date": "2026-10-09", "symptom": "v0.25.0 skill pack contradicted the binary: three examples marked correct failed `mxcli check` (`$response` created twice in an error handler, MDL063/CE0111; a view entity declaring String(400) and unlimited String over `a + ' ' + b`, MDL031), and prose taught `create or replace` as delete-and-recreate with a new UUID, a `Folder:` property for pages (MDL-DEPR105), and `docker check`/`build` as the way to clear CE0463", "cause": "`scripts/check-skill-mdl.sh` splits blocks into statements and fails only on syntax, so a whole example that parses but that check rejects was never judged. The prose claims were true once and never re-read when `or replace` became an alias (snippet replace now reuses the stored ID via UpdateSnippet) and when docker check moved to a temporary copy", "file": "`scripts/check-skill-mdl-semantics.sh` (new gate, wired into `make check-skill-mdl` and push-test.yml); skill files under `.claude/skills/mendix/`", "insight": "The 2026-10-01 mdl 1 pass already ran the real `mxcli check` over every whole block as a ONE-OFF and fixed 18 semantic failures — then nothing kept it true, and three came back. A one-off sweep is a measurement, not a guard: if it found anything, make it a gate the same day. Gate design that keeps it quiet: judge the WHOLE block (statement-splitting loses the variable context MDL063 needs); skip on `MDL-SYNTAX` (a fragment, owned by the syntax gate) and on check's 'produced no statements' (real SQL/OQL in a ```sql fence); ignore MDL-DUPDEF (alternatives shown side by side). Use `check --json` + jq on `ruleId`, not grep on the coloured text. For the prose claims, verify against the executor and `mxcli help`, not the linter alone: `cmd_pages_create_v3.go` shows snippet replace reusing the stored ID, and `help docker check` states the temp copy. docs-site still has ~24 whole blocks failing check (MDL020/021/023, MDL-WIDGET34…), left out of the gate's scope", "refs": ["mendixlabs/mxcli#1357"], "rules": ["MDL031", "MDL063", "MDL-DEPR001", "MDL-DEPR105"], "ce": ["CE0111", "CE6770", "CE0463"]} diff --git a/.claude/skills/mendix/check-syntax/SKILL.md b/.claude/skills/mendix/check-syntax/SKILL.md index 8dc587bc10..27c024c117 100644 --- a/.claude/skills/mendix/check-syntax/SKILL.md +++ b/.claude/skills/mendix/check-syntax/SKILL.md @@ -609,9 +609,9 @@ Declaration order that avoids most forward references: enumerations → entities → snippets (placeholder) → pages → snippets (fill-in) → microflows → navigation ``` -> **Never use `CREATE OR REPLACE` for the placeholder fill-in step.** OR REPLACE deletes -> the placeholder and creates a new document with a different UUID, silently breaking -> every page or snippet that references it. +> **Write the fill-in as `CREATE OR MODIFY`.** `CREATE OR REPLACE` is its deprecated +> spelling (`MDL-DEPR001`, keeps the ID too). Only `create or replace view entity` +> without an `mdl 1;` header deletes and recreates (`MDL-V1-REPLACE01`). ### Error: "mismatched input 'X'" diff --git a/.claude/skills/mendix/create-page/reference/widgets.md b/.claude/skills/mendix/create-page/reference/widgets.md index 123aa12e47..a1226aaa58 100644 --- a/.claude/skills/mendix/create-page/reference/widgets.md +++ b/.claude/skills/mendix/create-page/reference/widgets.md @@ -1115,14 +1115,17 @@ Notes: PieChart use the same `series` shape. - **CE0463 at `mx check`**: charts can report "widget definition changed" from widget-version drift (embedded template vs the installed `Charts.mpk`), even for a - chart with no series. Clear it with **`mxcli docker check`/`build`**, which normalize - the widgets and preserve your storage format. + chart with no series. Clear it with **`mxcli fix widgets -p app.mpr`**, which applies + the widget normalization to the project and keeps your storage format. + `mxcli docker check`/`build` normalize only a **temporary copy**: the check then + passes while the stored project still fails MxBuild and `mxcli run --local` with + CE0463. `docker check --no-update-widgets` checks the project as stored. **Do NOT run bare `mx update-widgets` on an MPRv2 project** (one with an `mprcontents/` folder — everything `mxcli new` creates). `mx update-widgets` rewrites the project into the single-file v1 format and **deletes `mprcontents/`**, which corrupts a git working tree, breaks a running `mxcli run --local` loop, and can leave - Studio Pro unable to open the project. `mxcli docker check`/`build` snapshot and - restore the v2 files around the normalization, so they are safe; raw + Studio Pro unable to open the project. `mxcli fix widgets` restores the v2 storage + around the normalization, so it is safe; raw `mx update-widgets` is only safe on a v1 project (or on a throwaway copy used purely for diagnosis). - LineChart/BubbleChart/HeatMap/TimeSeries are **also MDL-authorable** (via the diff --git a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md index 0eb3594d68..0e1c4cb961 100644 --- a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md +++ b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md @@ -81,7 +81,7 @@ mdl 1; */ create view entity ProductApi.CustomerAddressVE ( CustomerId: long, - CustomerName: string, + CustomerName: string(200), -- a derived string column is always String(200) Email: string, BillingStreet: string, BillingCity: string, diff --git a/.claude/skills/mendix/organize-project/SKILL.md b/.claude/skills/mendix/organize-project/SKILL.md index e6d04a0ff0..83682fd43d 100644 --- a/.claude/skills/mendix/organize-project/SKILL.md +++ b/.claude/skills/mendix/organize-project/SKILL.md @@ -335,7 +335,7 @@ drop folder 'Processing' in MyModule; ## Validation Checklist - [ ] Folder paths use `/` separator (not `\`) -- [ ] FOLDER keyword placement is correct (before BEGIN for microflows, inside properties for pages) +- [ ] FOLDER is a clause after the document name — before BEGIN for microflows, before the `(…)` properties for pages and snippets; a `Folder:` property is deprecated (`MDL-DEPR105`) - [ ] Cross-module moves: checked impact with `list impact of` first - [ ] Folder naming is consistent across modules - [ ] DROP FOLDER: verify folder is empty before dropping diff --git a/.claude/skills/mendix/overview-pages/SKILL.md b/.claude/skills/mendix/overview-pages/SKILL.md index f6cad29084..3d162f011a 100644 --- a/.claude/skills/mendix/overview-pages/SKILL.md +++ b/.claude/skills/mendix/overview-pages/SKILL.md @@ -632,7 +632,7 @@ create page Module.Customer_Overview -- Step 3: Fill in the snippet with real content (pages now exist) -- Use CREATE OR MODIFY (preserves the snippet's ID → page bindings stay valid) --- Do NOT use CREATE OR REPLACE — that would assign a new ID and break existing page references +-- (CREATE OR REPLACE is its deprecated spelling, MDL-DEPR001 — same meaning) create or modify snippet Module.NavigationMenu { layoutgrid navGrid { @@ -649,7 +649,7 @@ create or modify snippet Module.NavigationMenu - The placeholder snippet must exist before pages are created (for `snippetcall` to resolve) - Use `create or modify snippet` for the fill-in step — it preserves the snippet's UUID so pages that already reference it remain valid -- **Do not use `create or replace snippet`** — that deletes the placeholder and creates a fresh UUID, silently breaking every page that references the old one +- `create or replace snippet` is the deprecated spelling of the same statement (`MDL-DEPR001`) — it keeps the UUID too, but write `or modify` - Page references in the final snippet resolve correctly because pages already exist See [Resolve Forward References](../resolve-forward-references/SKILL.md) for the full pattern including page→page and microflow→page cases, declaration ordering rules, and the choice between `CREATE OR MODIFY` and `ALTER SNIPPET`. diff --git a/.claude/skills/mendix/resolve-forward-references/SKILL.md b/.claude/skills/mendix/resolve-forward-references/SKILL.md index d80f5df9c4..10ec9786c3 100644 --- a/.claude/skills/mendix/resolve-forward-references/SKILL.md +++ b/.claude/skills/mendix/resolve-forward-references/SKILL.md @@ -41,9 +41,12 @@ The standard workaround is a three-step sequence: 3. **Fill in the placeholder** using `CREATE OR MODIFY` or `ALTER` — both preserve the original ID so existing bindings remain valid. -> **Critical:** Never use `CREATE OR REPLACE` for the fill-in step. `OR REPLACE` deletes -> the placeholder and creates a new document with a different ID. Every page or snippet -> that references the placeholder immediately becomes a dangling reference. +> **Write `CREATE OR MODIFY`, not `CREATE OR REPLACE`.** `OR REPLACE` is the deprecated +> spelling of `OR MODIFY` (`MDL-DEPR001`, refused from `mdl 2`): it means the same and +> also keeps the ID, so it does not break the bindings — but it is not the spelling to +> teach. The one `OR REPLACE` that does delete and recreate is `create or replace view +> entity` in a script without an `mdl 1;` header (`MDL-V1-REPLACE01`); that is not a +> page or snippet. --- diff --git a/.claude/skills/mendix/test-microflows/SKILL.md b/.claude/skills/mendix/test-microflows/SKILL.md index 49c269dd63..12d37a099c 100644 --- a/.claude/skills/mendix/test-microflows/SKILL.md +++ b/.claude/skills/mendix/test-microflows/SKILL.md @@ -427,8 +427,11 @@ range to get a list, or use the variable as the object it is. `mxcli fmt --upgrade -w suite.test.mdl` reads the file the same way: it rewrites deprecated spellings in the blocks and keeps every doc comment, `--` comment and -`/` separator byte for byte. A test file takes **no `mdl 1;` header** yet — check -and the runner read its blocks as mdl 0 — so `--header` adds none to it and says so. +`/` separator byte for byte. Like a script, a test file **gets the `mdl 1;` +header by default** — check and the runner read every block behind it as mdl 1 — +and constructs whose meaning the header changes are rewritten to keep it +(`limit 1` becomes `first`, a bare `$x = …` becomes `set $x = …`). +`--header=false` upgrades the spellings and leaves the file headerless (mdl 0). --- diff --git a/.claude/skills/mendix/write-microflows/reference/control-flow.md b/.claude/skills/mendix/write-microflows/reference/control-flow.md index c0c5a862a3..164c079fcc 100644 --- a/.claude/skills/mendix/write-microflows/reference/control-flow.md +++ b/.claude/skills/mendix/write-microflows/reference/control-flow.md @@ -414,10 +414,13 @@ begin $response = call microflow Module.CallExternalAPI(data = $RequestData) on error without rollback begin log error node 'ExternalAPI' 'API call failed for: ' + $RequestData; - -- Create error response - $response = create Module.Response ( + -- Not `$response = create …`: a variable name is unique flow-wide, + -- handler included (MDL063, CE0111). The fallback gets its own name + -- and the handler returns it. + $fallbackResponse = create Module.Response ( success = false, message = 'External service unavailable'); + return $fallbackResponse; end error; return $response; diff --git a/.claude/skills/mendix/write-oql-queries/reference/patterns.md b/.claude/skills/mendix/write-oql-queries/reference/patterns.md index 2962e0b0cf..fa61880752 100644 --- a/.claude/skills/mendix/write-oql-queries/reference/patterns.md +++ b/.claude/skills/mendix/write-oql-queries/reference/patterns.md @@ -50,7 +50,7 @@ create view entity Finance.CategorySummary ( mdl 1; create view entity Shop.OrderDetails ( OrderId: long, - CustomerName: string(400), + CustomerName: string(200), -- a derived string column is always String(200) TotalItems: integer, TotalPrice: decimal ) as ( diff --git a/.claude/skills/packs/mendix-vega-charts/widget/package-lock.json b/.claude/skills/packs/mendix-vega-charts/widget/package-lock.json index 85cc7f0aa3..6a81d223a8 100644 --- a/.claude/skills/packs/mendix-vega-charts/widget/package-lock.json +++ b/.claude/skills/packs/mendix-vega-charts/widget/package-lock.json @@ -11352,13 +11352,13 @@ "license": "ISC" }, "node_modules/handlebars": { - "version": "4.7.9", - "resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.7.9.tgz", - "integrity": "sha512-4E71E0rpOaQuJR2A3xDZ+GM1HyWYv1clR58tC8emQNeQe3RH7MAzSbat+V0wG78LQBo6m6bzSG/L4pBuCsgnUQ==", + "version": "4.7.10", + "resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.7.10.tgz", + "integrity": "sha512-P5VJMVM7qgBn6vjXMw8WG9uVI+ncf2pi72j4de4yz5ZULLj2RGqLYaKOYGsgyrViQ0tePOVlN1tDCCXXtFqXKg==", "dev": true, "license": "MIT", "dependencies": { - "minimist": "^1.2.5", + "minimist": "^1.2.8", "neo-async": "^2.6.2", "source-map": "^0.6.1", "wordwrap": "^1.0.0" diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 0e4d9e5251..e2faf8b5f3 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -40,7 +40,7 @@ jobs: - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - name: Cache ANTLR4 JAR uses: actions/cache@v6 @@ -87,7 +87,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - name: Cache ANTLR4 JAR uses: actions/cache@v6 with: @@ -118,7 +118,7 @@ jobs: - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - uses: oven-sh/setup-bun@v2 diff --git a/.github/workflows/push-test.yml b/.github/workflows/push-test.yml index 7befaf521e..9ebc077c95 100644 --- a/.github/workflows/push-test.yml +++ b/.github/workflows/push-test.yml @@ -23,7 +23,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - name: Test the tunnel seam shell: bash # Scoped with -run to the seam's own tests. The full test binaries are still @@ -78,7 +78,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - name: Test the Windows process helpers shell: bash # Scoped with -run: the rest of the docker package's tests assert POSIX @@ -122,7 +122,7 @@ jobs: submodules: true - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - name: Cache ANTLR4 JAR uses: actions/cache@v6 with: @@ -175,6 +175,11 @@ jobs: run: ./scripts/check-skill-mdl.sh ./bin/mxcli .claude/skills/mendix - name: Check docs-site MDL blocks run: ./scripts/check-skill-mdl.sh ./bin/mxcli docs-site/src + - name: Check complete MDL examples (skills, docs-site) pass mxcli check + # The step above fails only on syntax. A whole example that parses and + # that `mxcli check` still rejects (MDL031, MDL063) shipped as correct + # until a user ran the pack through check (mendixlabs/mxcli#1357). + run: ./scripts/check-skill-mdl-semantics.sh ./bin/mxcli .claude/skills/mendix .claude/skills/packs docs/01-project/MDL_QUICK_REFERENCE.md docs-site/src - name: Check MDL conformance # Every MDL block in `mxcli syntax`, the skills, docs-site, the quick # reference and mdl-examples, parsed as mdl 1, with deprecated @@ -245,7 +250,7 @@ jobs: submodules: ${{ matrix.suite == 'roundtrip' || matrix.suite == 'parity' }} - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - name: Cache ANTLR4 JAR uses: actions/cache@v6 with: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d430adb50a..2f0b8d4021 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -17,7 +17,7 @@ jobs: fetch-depth: 0 - uses: actions/setup-go@v7 with: - go-version: '1.26.6' + go-version: '1.27.2' - uses: oven-sh/setup-bun@v2 - name: Cache ANTLR4 JAR uses: actions/cache@v6 diff --git a/Makefile b/Makefile index 2231d2994d..c6e5572ab2 100644 --- a/Makefile +++ b/Makefile @@ -275,6 +275,11 @@ check-skill-mdl: build echo " ok $$f"; \ done @./scripts/check-skill-mdl.sh ./$(BUILD_DIR)/$(BINARY_NAME) docs-site/src + @# The syntax pass above splits blocks into statements and ignores semantic + @# errors. Run each whole example through the full `mxcli check` as well: + @# examples marked correct that check rejects shipped until a user ran + @# the pack through check themselves (mendixlabs/mxcli#1357). + @./scripts/check-skill-mdl-semantics.sh ./$(BUILD_DIR)/$(BINARY_NAME) .claude/skills/mendix .claude/skills/packs docs/01-project/MDL_QUICK_REFERENCE.md docs-site/src # Canonical-form conformance gate (ako/mxcli#756, plan item 1.4 of # PROPOSAL_mdl_beta_syntax_freeze.md). Parses every MDL block in `mxcli syntax`, diff --git a/cmd/mxcli/check_script_set_refs_test.go b/cmd/mxcli/check_script_set_refs_test.go new file mode 100644 index 0000000000..643eadc06f --- /dev/null +++ b/cmd/mxcli/check_script_set_refs_test.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// mendixlabs/mxcli#1355: `check -p` over a script set reported a combobox +// datasource naming an entity an earlier file creates as "entity not found", +// although exec, run file by file in that order, succeeds. Every file was +// resolved against the project as it stands, so nothing an earlier file +// creates — entity, association, module — was visible to the files after it. +// The module is a stored one, as in the report, so what fails without the fix +// is the combobox's datasource and the page parameter, not the module. +// +// Controls: the same set in the wrong order still fails (the later file has +// not run yet), a datasource entity no file creates is still reported, and +// the project itself is left as it was. +func TestCheck_ScriptSetResolvesWhatEarlierFilesCreate(t *testing.T) { + src := filepath.Join("..", "..", "testdata", "pedapp") + if _, err := os.Stat(filepath.Join(src, "PedApp.mpr")); err != nil { + t.Skipf("PedApp fixture not found: %v", err) + } + dir := t.TempDir() + if err := copyTree(src, dir); err != nil { + t.Fatal(err) + } + mpr := filepath.Join(dir, "PedApp.mpr") + _ = checkCmd.InheritedFlags() + _ = rootCmd.PersistentFlags().Set("project", mpr) + defer func() { + _ = rootCmd.PersistentFlags().Set("project", "") + rootCmd.PersistentFlags().Lookup("project").Changed = false + }() + if f := checkCmd.Flags().Lookup("format"); f != nil { + _ = f.Value.Set(f.DefValue) + } + before := treeDigest(t, dir) + + scripts := t.TempDir() + domain := writeScript(t, scripts, "10-domain.mdl", `create persistent entity MyFirstModule.ShopCustomer ( Name: String(100) ); +create persistent entity MyFirstModule.ShopOrder ( Number: String(50) ); +create association MyFirstModule.ShopOrder_ShopCustomer from MyFirstModule.ShopOrder to MyFirstModule.ShopCustomer; +`) + page := func(name, entity string) string { + return writeScript(t, scripts, name, `create or modify page MyFirstModule.ShopOrderEdit +( Title: 'Order', Layout: Atlas_Core.Atlas_Default, Params: ( $Order: MyFirstModule.ShopOrder ) ) +{ + layoutgrid lg { row { column (desktopwidth: autofill) { + dataview dv (datasource: $Order) { + combobox cmbCustomer ( + label: 'Customer', + Association: MyFirstModule.ShopOrder_ShopCustomer, + datasource: database `+entity+`, + CaptionAttribute: Name + ) + } + } } } +}; +`) + } + good := page("20-pages.mdl", "MyFirstModule.ShopCustomer") + bad := page("20-bad.mdl", "MyFirstModule.ShopNoSuchCustomer") + + var code int + out := captureStd(t, func() { code = runCheckFiles(checkCmd, []string{domain, good}) }) + if code != 0 || strings.Contains(out, "not found") || strings.Contains(out, "does not exist") { + t.Errorf("a set exec runs cleanly failed its check (exit %d):\n%s", code, out) + } + + out = captureStd(t, func() { code = runCheckFiles(checkCmd, []string{good, domain}) }) + if code == 0 || !strings.Contains(out, "entity MyFirstModule.ShopOrder does not exist") { + t.Errorf("control, wrong order: the page ran before its entities and passed (exit %d):\n%s", code, out) + } + + out = captureStd(t, func() { code = runCheckFiles(checkCmd, []string{domain, bad}) }) + if code == 0 || !strings.Contains(out, "entity not found: MyFirstModule.ShopNoSuchCustomer") { + t.Errorf("control, missing datasource entity: not reported (exit %d):\n%s", code, out) + } + + if n := countDiff(before, treeDigest(t, dir)); n > 0 { + t.Errorf("check wrote to the project: %d file(s) changed", n) + } +} + +// treeDigest maps each file under dir (bar mxcli's own .mxcli cache) to its +// contents. +func treeDigest(t *testing.T, dir string) map[string]string { + t.Helper() + out := map[string]string{} + err := filepath.Walk(dir, func(p string, info os.FileInfo, err error) error { + if err != nil { + return err + } + if info.IsDir() { + if info.Name() == ".mxcli" { + return filepath.SkipDir + } + return nil + } + b, err := os.ReadFile(p) + out[p] = string(b) + return err + }) + if err != nil { + t.Fatal(err) + } + return out +} + +func countDiff(a, b map[string]string) int { + n := 0 + for k, v := range a { + if b[k] != v { + n++ + } + } + for k := range b { + if _, ok := a[k]; !ok { + n++ + } + } + return n +} diff --git a/cmd/mxcli/cmd_check.go b/cmd/mxcli/cmd_check.go index 4bbbe6a8f2..969f3fff4d 100644 --- a/cmd/mxcli/cmd_check.go +++ b/cmd/mxcli/cmd_check.go @@ -5,11 +5,15 @@ package main import ( "fmt" "os" + "path/filepath" "strings" "github.com/mendixlabs/mxcli/cmd/mxcli/testrunner" + "github.com/mendixlabs/mxcli/mdl/backend" + modelsdkbackend "github.com/mendixlabs/mxcli/mdl/backend/modelsdk" "github.com/mendixlabs/mxcli/mdl/executor" "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/mdl/scriptdiff" "github.com/mendixlabs/mxcli/mdl/visitor" "github.com/spf13/cobra" ) @@ -73,9 +77,13 @@ This is the verdict "mxcli diff" reports as "Refused:", computed by the same code. A statement on a flow an earlier statement of the script changes, or whose flow does not build until an earlier statement has run, is not predicted. -Several files are checked as one script set, run in the order given: each -file is checked as it would be alone, and the set is first read as a whole for -two "create or modify" statements declaring the same flow — a placeholder +Several files are checked as one script set, run in the order given, as +"mxcli exec" would run them one after the other. With a project, each file is +resolved against a scratch copy of it that the files before it have been +applied to, so a name an earlier file creates resolves in a later one; a file +that does not pass is left out of the copy, as exec's pre-flight would refuse +it and write nothing. The project itself is not changed. The set is also first +read as a whole for two "create or modify" statements declaring the same flow — a placeholder ("stub") followed by the real flow. That is reported as an MDL-STUB01 warning naming both statements: run in order, the stub replaces the real flow before the real statement restores it, on every run, and with the two under different @@ -126,7 +134,19 @@ Examples: // runCheckFile checks one script and returns the exit code: 0 when it passed. func runCheckFile(cmd *cobra.Command, filePath string) int { + return runCheckFileAgainst(cmd, filePath, "") +} + +// runCheckFileAgainst is runCheckFile resolving against the project at +// against instead of --project — a script set's scratch copy, holding what +// the files before this one write — while still naming --project in its +// report. "" means --project itself. +func runCheckFileAgainst(cmd *cobra.Command, filePath, against string) int { projectPath, _ := cmd.Flags().GetString("project") + shownProject := projectPath + if against != "" && projectPath != "" { + projectPath = against + } // A project makes reference resolution possible, so it runs. It used to // need --references as well, which meant `mxcli check script.mdl -p // app.mpr` printed an unqualified "Check passed!" having resolved @@ -317,7 +337,7 @@ func runCheckFile(cmd *cobra.Command, filePath string) int { } if !isStructured { - fmt.Printf("\nValidating references against: %s\n", projectPath) + fmt.Printf("\nValidating references against: %s\n", shownProject) fmt.Printf("(Note: References to objects created within the script are skipped)\n") } // exec was connected above, before the semantic report. @@ -448,7 +468,7 @@ func runCheckFile(cmd *cobra.Command, filePath string) int { return 1 } if !isStructured { - fmt.Printf("\nScanning project for legacy native widgets: %s\n", projectPath) + fmt.Printf("\nScanning project for legacy native widgets: %s\n", shownProject) } legacyViolations, err := scanLegacyWidgets(projectPath) if err != nil { @@ -513,18 +533,86 @@ func runCheckFiles(cmd *cobra.Command, files []string) int { linter.GetFormatter(linter.OutputFormat("text"), true).Format(v, os.Stderr) fmt.Fprintln(os.Stderr) } + // Against a project, each file is resolved against a scratch copy that the + // files before it have been applied to — what exec, run file by file in + // this order, leaves for it (mendixlabs/mxcli#1355). Checked against the + // project itself, every reference to an earlier file's entity, page or + // module was reported as missing, so a valid set failed its pre-flight. + var scratch *scriptdiff.Scratch + if projectPath, _ := cmd.Flags().GetString("project"); projectPath != "" { + sc, err := scriptdiff.NewScratch(projectPath) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: copy the project to check the script set against: %v\n", err) + return 1 + } + defer sc.Close() + scratch = sc + } worst := 0 + var notApplied []string for i, f := range files { if i > 0 { fmt.Println() } - if code := runCheckFile(cmd, f); code > worst { + against := "" + if scratch != nil { + against = scratch.Mpr + if i > 0 { + fmt.Printf("Checking %s against a scratch copy of the project holding what the %d file(s) before it write (the project itself is not changed)\n", f, i) + for _, n := range notApplied { + fmt.Printf(" (without %s)\n", n) + } + } + } + code := runCheckFileAgainst(cmd, f, against) + if code > worst { worst = code } + if scratch != nil && i < len(files)-1 { + if why := applyToScratch(scratch, f, code); why != "" { + notApplied = append(notApplied, why) + } + } } return worst } +// applyToScratch runs one checked file of a script set against the scratch +// copy, so the files after it resolve against what it writes. It returns "" +// when the whole file was applied, and otherwise what was left out and why. +// +// A file that did not pass is not applied: exec's pre-flight would refuse it +// and write nothing, so the files after it are checked as exec would run them +// — without it. A test file is not applied either: running it runs its tests. +func applyToScratch(scratch *scriptdiff.Scratch, file string, checkCode int) string { + if checkCode != 0 { + return file + ", which did not pass: exec would refuse it and write nothing" + } + if testrunner.IsTestFile(file) { + return file + ", a test file: it creates nothing for the files after it" + } + data, err := readMDLSource(file) + if err != nil { + return fmt.Sprintf("%s: %v", file, err) + } + prog, errs := visitor.Build(string(data)) + if len(errs) > 0 { + return fmt.Sprintf("%s: %v", file, errs[0]) + } + opts := scriptdiff.Options{ + // The file engine whatever --mcp says: the file is executed for real, + // and only the scratch copy may receive it. + NewBackend: func() backend.FullBackend { return modelsdkbackend.New() }, + } + if abs, err := filepath.Abs(file); err == nil { + opts.ScriptDir = filepath.Dir(abs) + } + if _, err := scratch.Apply(prog, opts); err != nil { + return fmt.Sprintf("what %s writes after: %v", file, err) + } + return "" +} + // parseScriptSet reads and parses each file for the set-level checks. A file // that cannot be read or parsed is left out (Prog nil): its own check reports // that. diff --git a/cmd/mxcli/cmd_describe.go b/cmd/mxcli/cmd_describe.go index 9ef5a9f4aa..d2c8377171 100644 --- a/cmd/mxcli/cmd_describe.go +++ b/cmd/mxcli/cmd_describe.go @@ -51,6 +51,7 @@ Types: restclient Describe a consumed REST service (also: "rest client") odataclient Describe a consumed OData service odataservice Describe a published OData service + publishedrestservice Describe a published REST service (also: "published rest service", "rest service") imagecollection Describe an image collection (also: "image collection") menu Describe a standalone menu document queue Describe a task queue @@ -124,92 +125,11 @@ Example: name := args[len(args)-1] objectType := strings.ToUpper(strings.Join(args[:len(args)-1], " ")) - var mdlCmd string - switch objectType { - case "MODULE": - mdlCmd = fmt.Sprintf("DESCRIBE MODULE %s", name) - case "ENTITY": - mdlCmd = fmt.Sprintf("DESCRIBE ENTITY %s", name) - case "ASSOCIATION": - mdlCmd = fmt.Sprintf("DESCRIBE ASSOCIATION %s", name) - case "ENUMERATION": - mdlCmd = fmt.Sprintf("DESCRIBE ENUMERATION %s", name) - case "MICROFLOW": - mdlCmd = fmt.Sprintf("DESCRIBE MICROFLOW %s", name) - case "NANOFLOW": - mdlCmd = fmt.Sprintf("DESCRIBE NANOFLOW %s", name) - case "WORKFLOW": - mdlCmd = fmt.Sprintf("DESCRIBE WORKFLOW %s", name) - case "PAGE": - mdlCmd = fmt.Sprintf("DESCRIBE PAGE %s", name) - case "SNIPPET": - mdlCmd = fmt.Sprintf("DESCRIBE SNIPPET %s", name) - case "BUILDINGBLOCK", "BUILDING BLOCK": - mdlCmd = fmt.Sprintf("DESCRIBE BUILDING BLOCK %s", name) - case "LAYOUT": - mdlCmd = fmt.Sprintf("DESCRIBE LAYOUT %s", name) - case "MODULEROLE", "MODULE ROLE": - mdlCmd = fmt.Sprintf("DESCRIBE MODULE ROLE %s", name) - case "USERROLE", "USER ROLE": - mdlCmd = fmt.Sprintf("DESCRIBE USER ROLE '%s'", name) - case "PROJECTSECURITY", "PROJECT SECURITY": - mdlCmd = "DESCRIBE APP SECURITY" - case "SETTINGS": - mdlCmd = "DESCRIBE SETTINGS" - case "DEMOUSER", "DEMO USER": - mdlCmd = fmt.Sprintf("DESCRIBE DEMO USER '%s'", name) - case "JAVAACTION", "JAVA ACTION": - mdlCmd = fmt.Sprintf("DESCRIBE JAVA ACTION %s", name) - case "CONSTANT": - mdlCmd = fmt.Sprintf("DESCRIBE CONSTANT %s", name) - case "JSONSTRUCTURE", "JSON STRUCTURE": - mdlCmd = fmt.Sprintf("DESCRIBE JSON STRUCTURE %s", name) - case "IMPORTMAPPING", "IMPORT MAPPING": - mdlCmd = fmt.Sprintf("DESCRIBE IMPORT MAPPING %s", name) - case "EXPORTMAPPING", "EXPORT MAPPING": - mdlCmd = fmt.Sprintf("DESCRIBE EXPORT MAPPING %s", name) - case "RESTCLIENT", "REST CLIENT": - mdlCmd = fmt.Sprintf("DESCRIBE CONSUMED REST SERVICE %s", name) - case "ODATACLIENT", "ODATA CLIENT": - mdlCmd = fmt.Sprintf("DESCRIBE CONSUMED ODATA SERVICE %s", name) - case "ODATASERVICE", "ODATA SERVICE": - mdlCmd = fmt.Sprintf("DESCRIBE PUBLISHED ODATA SERVICE %s", name) - case "IMAGECOLLECTION", "IMAGE COLLECTION": - mdlCmd = fmt.Sprintf("DESCRIBE IMAGE COLLECTION %s", name) - case "MENU": - mdlCmd = fmt.Sprintf("DESCRIBE MENU %s", name) - case "QUEUE": - mdlCmd = fmt.Sprintf("DESCRIBE TASK QUEUE %s", name) - case "SCHEDULEDEVENT", "SCHEDULED EVENT": - mdlCmd = fmt.Sprintf("DESCRIBE SCHEDULED EVENT %s", name) - case "REGULAREXPRESSION", "REGULAR EXPRESSION", "REGEX": - mdlCmd = fmt.Sprintf("DESCRIBE REGULAR EXPRESSION %s", name) - case "BUSINESSEVENTSERVICE", "BUSINESS EVENT SERVICE": - mdlCmd = fmt.Sprintf("DESCRIBE BUSINESS EVENT SERVICE %s", name) - case "DATABASECONNECTION", "DATABASE CONNECTION": - mdlCmd = fmt.Sprintf("DESCRIBE DATABASE CONNECTION %s", name) - case "EXTERNALENTITY", "EXTERNAL ENTITY": - mdlCmd = fmt.Sprintf("DESCRIBE ENTITY %s", name) - case "NAVIGATION": - mdlCmd = fmt.Sprintf("DESCRIBE NAVIGATION %s", name) - case "NAVPROFILE": - mdlCmd = fmt.Sprintf("DESCRIBE NAVIGATION %s", name) - case "AGENT": - mdlCmd = fmt.Sprintf("DESCRIBE AGENT %s", name) - case "AIMODEL", "AI MODEL", "MODEL": - mdlCmd = fmt.Sprintf("DESCRIBE MODEL %s", name) - case "KNOWLEDGEBASE", "KNOWLEDGE BASE": - mdlCmd = fmt.Sprintf("DESCRIBE KNOWLEDGE BASE %s", name) - case "CONSUMEDMCPSERVICE", "CONSUMED MCP SERVICE", "MCP SERVICE": - mdlCmd = fmt.Sprintf("DESCRIBE CONSUMED MCP SERVICE %s", name) - case "DATATRANSFORMER", "DATA TRANSFORMER": - mdlCmd = fmt.Sprintf("DESCRIBE DATA TRANSFORMER %s", name) - case "SYSTEMOVERVIEW": - mdlCmd = "" // handled directly by format-specific path - default: + mdlCmd, ok := describeMDLCommand(objectType, name) + if !ok { fmt.Fprintf(os.Stderr, "Unknown type: %s\n", strings.Join(args[:len(args)-1], " ")) - fmt.Fprintln(os.Stderr, "Valid types: module, entity, association, enumeration, constant, microflow, nanoflow, workflow, page, snippet, layout, javaaction, jsonstructure, importmapping, exportmapping, restclient, odataclient, odataservice, imagecollection, menu, queue, scheduledevent, businesseventservice, databaseconnection, agent, aimodel, knowledgebase, consumedmcpservice, datatransformer, modulerole, userrole, projectsecurity, settings, demouser, navigation, systemoverview") - fmt.Fprintln(os.Stderr, "Multi-word types also accepted: json structure, import mapping, export mapping, rest client, image collection, scheduled event, regular expression, business event service, agent, model, knowledge base, consumed mcp service, data transformer, etc.") + fmt.Fprintln(os.Stderr, "Valid types: module, entity, association, enumeration, constant, microflow, nanoflow, workflow, page, snippet, layout, javaaction, jsonstructure, importmapping, exportmapping, restclient, odataclient, odataservice, publishedrestservice, imagecollection, menu, queue, scheduledevent, businesseventservice, databaseconnection, agent, aimodel, knowledgebase, consumedmcpservice, datatransformer, modulerole, userrole, projectsecurity, settings, demouser, navigation, systemoverview") + fmt.Fprintln(os.Stderr, "Multi-word types also accepted: json structure, import mapping, export mapping, rest client, published rest service, image collection, scheduled event, regular expression, business event service, agent, model, knowledge base, consumed mcp service, data transformer, etc.") os.Exit(1) } @@ -303,6 +223,98 @@ Example: }, } +// describeMDLCommand maps a `describe` subcommand type (upper-cased, words joined +// by single spaces) and a name to the MDL DESCRIBE statement it runs. ok is false +// for an unknown type. SYSTEMOVERVIEW returns "" with ok true: it has no MDL form +// and is handled by the --format elk path. +func describeMDLCommand(objectType, name string) (mdlCmd string, ok bool) { + switch objectType { + case "MODULE": + return fmt.Sprintf("DESCRIBE MODULE %s", name), true + case "ENTITY": + return fmt.Sprintf("DESCRIBE ENTITY %s", name), true + case "ASSOCIATION": + return fmt.Sprintf("DESCRIBE ASSOCIATION %s", name), true + case "ENUMERATION": + return fmt.Sprintf("DESCRIBE ENUMERATION %s", name), true + case "MICROFLOW": + return fmt.Sprintf("DESCRIBE MICROFLOW %s", name), true + case "NANOFLOW": + return fmt.Sprintf("DESCRIBE NANOFLOW %s", name), true + case "WORKFLOW": + return fmt.Sprintf("DESCRIBE WORKFLOW %s", name), true + case "PAGE": + return fmt.Sprintf("DESCRIBE PAGE %s", name), true + case "SNIPPET": + return fmt.Sprintf("DESCRIBE SNIPPET %s", name), true + case "BUILDINGBLOCK", "BUILDING BLOCK": + return fmt.Sprintf("DESCRIBE BUILDING BLOCK %s", name), true + case "LAYOUT": + return fmt.Sprintf("DESCRIBE LAYOUT %s", name), true + case "MODULEROLE", "MODULE ROLE": + return fmt.Sprintf("DESCRIBE MODULE ROLE %s", name), true + case "USERROLE", "USER ROLE": + return fmt.Sprintf("DESCRIBE USER ROLE '%s'", name), true + case "PROJECTSECURITY", "PROJECT SECURITY": + return "DESCRIBE APP SECURITY", true + case "SETTINGS": + return "DESCRIBE SETTINGS", true + case "DEMOUSER", "DEMO USER": + return fmt.Sprintf("DESCRIBE DEMO USER '%s'", name), true + case "JAVAACTION", "JAVA ACTION": + return fmt.Sprintf("DESCRIBE JAVA ACTION %s", name), true + case "CONSTANT": + return fmt.Sprintf("DESCRIBE CONSTANT %s", name), true + case "JSONSTRUCTURE", "JSON STRUCTURE": + return fmt.Sprintf("DESCRIBE JSON STRUCTURE %s", name), true + case "IMPORTMAPPING", "IMPORT MAPPING": + return fmt.Sprintf("DESCRIBE IMPORT MAPPING %s", name), true + case "EXPORTMAPPING", "EXPORT MAPPING": + return fmt.Sprintf("DESCRIBE EXPORT MAPPING %s", name), true + case "RESTCLIENT", "REST CLIENT": + return fmt.Sprintf("DESCRIBE CONSUMED REST SERVICE %s", name), true + case "ODATACLIENT", "ODATA CLIENT": + return fmt.Sprintf("DESCRIBE CONSUMED ODATA SERVICE %s", name), true + case "ODATASERVICE", "ODATA SERVICE": + return fmt.Sprintf("DESCRIBE PUBLISHED ODATA SERVICE %s", name), true + case "PUBLISHEDRESTSERVICE", "PUBLISHED REST SERVICE", "RESTSERVICE", "REST SERVICE": + return fmt.Sprintf("DESCRIBE PUBLISHED REST SERVICE %s", name), true + case "IMAGECOLLECTION", "IMAGE COLLECTION": + return fmt.Sprintf("DESCRIBE IMAGE COLLECTION %s", name), true + case "MENU": + return fmt.Sprintf("DESCRIBE MENU %s", name), true + case "QUEUE": + return fmt.Sprintf("DESCRIBE TASK QUEUE %s", name), true + case "SCHEDULEDEVENT", "SCHEDULED EVENT": + return fmt.Sprintf("DESCRIBE SCHEDULED EVENT %s", name), true + case "REGULAREXPRESSION", "REGULAR EXPRESSION", "REGEX": + return fmt.Sprintf("DESCRIBE REGULAR EXPRESSION %s", name), true + case "BUSINESSEVENTSERVICE", "BUSINESS EVENT SERVICE": + return fmt.Sprintf("DESCRIBE BUSINESS EVENT SERVICE %s", name), true + case "DATABASECONNECTION", "DATABASE CONNECTION": + return fmt.Sprintf("DESCRIBE DATABASE CONNECTION %s", name), true + case "EXTERNALENTITY", "EXTERNAL ENTITY": + return fmt.Sprintf("DESCRIBE ENTITY %s", name), true + case "NAVIGATION": + return fmt.Sprintf("DESCRIBE NAVIGATION %s", name), true + case "NAVPROFILE": + return fmt.Sprintf("DESCRIBE NAVIGATION %s", name), true + case "AGENT": + return fmt.Sprintf("DESCRIBE AGENT %s", name), true + case "AIMODEL", "AI MODEL", "MODEL": + return fmt.Sprintf("DESCRIBE MODEL %s", name), true + case "KNOWLEDGEBASE", "KNOWLEDGE BASE": + return fmt.Sprintf("DESCRIBE KNOWLEDGE BASE %s", name), true + case "CONSUMEDMCPSERVICE", "CONSUMED MCP SERVICE", "MCP SERVICE": + return fmt.Sprintf("DESCRIBE CONSUMED MCP SERVICE %s", name), true + case "DATATRANSFORMER", "DATA TRANSFORMER": + return fmt.Sprintf("DESCRIBE DATA TRANSFORMER %s", name), true + case "SYSTEMOVERVIEW": + return "", true // handled directly by format-specific path + } + return "", false +} + // objectTypeToDescribe maps a catalog `objects` view ObjectType to the `describe` // keyword. Only directly describable types with a `Module.Name` shape are listed. var objectTypeToDescribe = map[string]string{ @@ -325,6 +337,7 @@ var objectTypeToDescribe = map[string]string{ "ODATA_CLIENT": "odataclient", "ODATA_SERVICE": "odataservice", "REST_CLIENT": "restclient", + "PUBLISHED_REST_SERVICE": "publishedrestservice", "BUSINESS_EVENT_SERVICE": "businesseventservice", "DATABASE_CONNECTION": "databaseconnection", "IMAGE_COLLECTION": "imagecollection", @@ -362,6 +375,7 @@ var unitTypeToDescribe = map[string]string{ "Queues$Queue": "queue", "ScheduledEvents$ScheduledEvent": "scheduledevent", "RegularExpressions$RegularExpression": "regularexpression", + "Rest$PublishedRestService": "publishedrestservice", } // resolveDescribeType auto-detects the `describe` type for a qualified document diff --git a/cmd/mxcli/cmd_describe_test.go b/cmd/mxcli/cmd_describe_test.go index 3c552bb7d5..4e37a7264e 100644 --- a/cmd/mxcli/cmd_describe_test.go +++ b/cmd/mxcli/cmd_describe_test.go @@ -4,7 +4,10 @@ package main import ( "slices" + "strings" "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" ) func TestChooseDescribeType(t *testing.T) { @@ -72,3 +75,52 @@ func TestTypeMaps_KnownEntries(t *testing.T) { } } } + +// TestDescribeMDLCommand_AutoDetectKeywordsAccepted pins that every keyword the +// auto-detect maps can produce is one the dispatch accepts and that the MDL it +// builds parses. A keyword missing from the dispatch reaches the user as +// "Unknown type" for a document the catalog lists (#1347). +func TestDescribeMDLCommand_AutoDetectKeywordsAccepted(t *testing.T) { + for _, m := range []map[string]string{objectTypeToDescribe, unitTypeToDescribe} { + for k, kw := range m { + name := "Mod.X" + if kw == "module" { + name = "Mod" // a module's name is not qualified + } + mdl, ok := describeMDLCommand(strings.ToUpper(kw), name) + if !ok { + t.Errorf("auto-detect keyword %q (from %q) is not accepted by describe", kw, k) + continue + } + if _, errs := visitor.Build(mdl); len(errs) > 0 { + t.Errorf("keyword %q builds %q, which does not parse: %v", kw, mdl, errs) + } + } + } +} + +// TestDescribeMDLCommand_PublishedRestService is the #1347 symptom: `mxcli +// describe published rest service M.Api` printed "Unknown type" although the +// MDL statement works under `-c`, and auto-detect could not resolve the name. +func TestDescribeMDLCommand_PublishedRestService(t *testing.T) { + const want = "DESCRIBE PUBLISHED REST SERVICE M.Api" + for _, typ := range []string{"PUBLISHED REST SERVICE", "PUBLISHEDRESTSERVICE", "REST SERVICE", "RESTSERVICE"} { + got, ok := describeMDLCommand(typ, "M.Api") + if !ok { + t.Errorf("describe %q: Unknown type", strings.ToLower(typ)) + continue + } + if got != want { + t.Errorf("describe %q = %q, want %q", strings.ToLower(typ), got, want) + } + } + if _, errs := visitor.Build(want); len(errs) > 0 { + t.Fatalf("%q does not parse: %v", want, errs) + } + if got := objectTypeToDescribe["PUBLISHED_REST_SERVICE"]; got == "" { + t.Error("catalog ObjectType PUBLISHED_REST_SERVICE has no auto-detect keyword") + } + if got := unitTypeToDescribe["Rest$PublishedRestService"]; got == "" { + t.Error("unit type Rest$PublishedRestService has no auto-detect keyword") + } +} diff --git a/cmd/mxcli/cmd_fmt.go b/cmd/mxcli/cmd_fmt.go index c5e075f834..c15e4c02f0 100644 --- a/cmd/mxcli/cmd_fmt.go +++ b/cmd/mxcli/cmd_fmt.go @@ -99,8 +99,10 @@ Upgrading (--upgrade): A test file (.test.mdl, .test.md) is upgraded the way check reads it: the statements in its blocks are rewritten, and its doc comments (@test, - @expect, …), separators and prose are kept byte for byte. It takes no - language header yet, so --header adds none to it and says so. + @expect, …), separators and prose are kept byte for byte. The header is + added as for a script — before the first line of a .test.mdl, as the first + line of each mdl-test block of a .test.md — since check and the runner read + a test file's header; --header=false upgrades the spellings without it. # Upgrade in place mxcli fmt --upgrade -w script.mdl diff --git a/cmd/mxcli/cmd_fmt_testfile_header_doc_test.go b/cmd/mxcli/cmd_fmt_testfile_header_doc_test.go new file mode 100644 index 0000000000..077b060d81 --- /dev/null +++ b/cmd/mxcli/cmd_fmt_testfile_header_doc_test.go @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "path/filepath" + "regexp" + "strings" + "testing" +) + +// mendixlabs/mxcli#1356: "fmt --upgrade adds `mdl 1;` to a .test.mdl, but fmt +// --help and the test-microflows skill say test files take no header". The +// behaviour is the intended one (ako/mxcli#847: the runner and check read a +// test file's header, and --upgrade adds it by default); the two texts an +// agent reads first were left describing the pre-#847 behaviour. This pins +// the behaviour on the issue's own file, then holds both texts to it. +func TestFmtUpgrade_TestFileHeaderDocsMatchBehaviour(t *testing.T) { + path := filepath.Join(t.TempDir(), "suite.test.mdl") + src := "/**\n * @test Finds a customer\n * @expect $Found = true\n */\n" + + "retrieve $Customer from MyModule.Customer where Name = 'Ann' limit 1;\n$Found = $Customer != empty;\n" + if err := os.WriteFile(path, []byte(src), 0o644); err != nil { + t.Fatal(err) + } + if _, err := runFmt(t, "--upgrade", "-w", path); err != nil { + t.Fatal(err) + } + if got, _ := os.ReadFile(path); !strings.HasPrefix(string(got), "mdl 1;\n") { + t.Fatalf("fmt --upgrade no longer adds the header to a test file; this test's premise is gone:\n%s", got) + } + + skill, err := os.ReadFile(filepath.Join("..", "..", ".claude", "skills", "mendix", "test-microflows", "SKILL.md")) + if err != nil { + t.Fatal(err) + } + docs := map[string]string{ + "fmt --help": fmtCmd.Long, + "test-microflows/SKILL.md": string(skill), + } + // "takes no language header", "takes **no `mdl 1;` header**", "adds none to it". + stale := regexp.MustCompile("(?i)takes\\s+(\\*\\*)?no\\b|adds\\s+none\\s+to\\s+it") + for name, body := range docs { + var said bool + for _, para := range regexp.MustCompile(`\n\s*\n`).Split(body, -1) { + p := strings.Join(strings.Fields(para), " ") + if !strings.Contains(strings.ToLower(p), "test file") || !strings.Contains(p, "header") { + continue + } + if m := stale.FindString(p); m != "" { + t.Errorf("%s says a test file takes no header (%q), but fmt --upgrade adds one:\n%s", name, m, p) + } + if strings.Contains(p, "--header=false") { + said = true + } + } + if !said { + t.Errorf("%s: no paragraph about a test file's header names --header=false, the way to decline it", name) + } + } +} diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index ca0d3f4ee1..984c6f94b9 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -1386,12 +1386,19 @@ func watchAndApply(opts LocalRunOptions, serve *ServeServer, rt *LocalRuntime, b genBefore := bundler.Generation() webBefore := webClientSourceMTime(opts.DeployDir) - build, err := serve.Build(BuildRequest{Target: TargetDeploy, ProjectFilePath: opts.ProjectPath}) + // On Windows the bundler's working directory pins deployment/web, so a + // change that makes mxbuild recreate web/ (a domain model change) fails + // until the bundler lets go; the supervisor stops it, rebuilds, and + // starts a fresh one on the new web/ (#1342). + build, freshBundle, err := bundler.BuildReleasingWebDir(func() (*BuildResult, error) { + return serve.Build(BuildRequest{Target: TargetDeploy, ProjectFilePath: opts.ProjectPath}) + }) if err != nil { fmt.Fprintf(opts.Stderr, " build error: %v\n", err) fail("build", err.Error(), nil) continue } + bundled = bundled || freshBundle if !build.OK() { // Surface the full serve response, not just the generic message — // it carries the real detail (e.g. the SCSS compiler's @@ -1414,8 +1421,9 @@ func watchAndApply(opts LocalRunOptions, serve *ServeServer, rt *LocalRuntime, b // incremental bundler to re-bundle. WaitForRebuild settles out cleanly if // no rebuild materializes (the touched file isn't a rollup input — e.g. a // microflow edit that rewrites a web metadata file but no page/widget), so - // this never hangs. A pure model change skips the wait entirely. - if webClientSourceMTime(opts.DeployDir).After(webBefore) { + // this never hangs. A pure model change skips the wait entirely, and so + // does a build after which a fresh bundler already bundled everything. + if !freshBundle && webClientSourceMTime(opts.DeployDir).After(webBefore) { // Detection is a reliable ~1s with polling, so a 2.5s settle is ample // margin to catch a rebuild that's going to start, while keeping the // no-rebuild case (a model edit that only grazed web/) snappy. diff --git a/cmd/mxcli/docker/tunnel_test.go b/cmd/mxcli/docker/tunnel_test.go index 93d71a3e8a..41928594b2 100644 --- a/cmd/mxcli/docker/tunnel_test.go +++ b/cmd/mxcli/docker/tunnel_test.go @@ -10,9 +10,14 @@ import ( // a loopback / NO_PROXY hub directly — otherwise a local hub (or an allow-listed // one) would be forced through a proxy that refuses it. func TestProxyForURL(t *testing.T) { - t.Setenv("HTTPS_PROXY", "http://127.0.0.1:33451") - t.Setenv("HTTP_PROXY", "http://127.0.0.1:33451") - t.Setenv("NO_PROXY", "127.0.0.1,localhost,.internal.example") + // Both spellings: x/net v0.60 reads the lowercase one first, so an ambient + // https_proxy (common on developer machines) would otherwise win. + for _, k := range []string{"HTTPS_PROXY", "https_proxy", "HTTP_PROXY", "http_proxy"} { + t.Setenv(k, "http://127.0.0.1:33451") + } + for _, k := range []string{"NO_PROXY", "no_proxy"} { + t.Setenv(k, "127.0.0.1,localhost,.internal.example") + } cases := []struct { url string diff --git a/cmd/mxcli/docker/webclient_supervisor.go b/cmd/mxcli/docker/webclient_supervisor.go index d45fb9257a..36b93b255a 100644 --- a/cmd/mxcli/docker/webclient_supervisor.go +++ b/cmd/mxcli/docker/webclient_supervisor.go @@ -5,6 +5,7 @@ package docker import ( "fmt" "io" + "regexp" "strings" "time" ) @@ -182,6 +183,56 @@ func (s *bundlerSupervisor) AwaitRebuild(sinceGen int, settle, buildTimeout time return s.want, nil } +// BuildReleasingWebDir runs a serve build, and when it fails because the +// bundler holds deployment/web open, stops the bundler and builds again (#1342). +// +// Windows refuses to delete a directory that is any process's working +// directory, and the bundler's is web/ — it has to be: mxbuild's pages plugin +// resolves page paths against cwd. A page or microflow edit rewrites files +// inside web/ and is unaffected, but a domain model change makes mxbuild +// recreate web/ itself, and that failed on every attempt with "The process +// cannot access the file '…\deployment\web' because it is being used by +// another process". Killing the bundler by hand did not help either: the watch +// loop's EnsureAlive restarted it just before the next build. +// +// After a successful retry a fresh bundler is started; its first build is a +// full bundle of the recreated web/, so rebundled tells the caller there is no +// incremental rebuild left to wait for. When the retry fails too, the bundler +// is left down for EnsureAlive to restart on the next change. On POSIX the +// directory is removed regardless and the retry never fires. +func (s *bundlerSupervisor) BuildReleasingWebDir(build func() (*BuildResult, error)) (res *BuildResult, rebundled bool, err error) { + res, err = build() + if err != nil || !s.Wanted() || s.cur == nil || !webDirInUse(res) { + return res, false, err + } + fmt.Fprintln(s.out, " deployment/web is held open by the web client bundler; stopping it and rebuilding...") + s.Stop() + res, err = build() + if err != nil || !res.OK() { + return res, false, err + } + if rerr := s.Restart(); rerr != nil { + fmt.Fprintf(s.out, " restarting web client bundler failed (next attempt in %s): %v\n", + bundlerRestartBackoff(s.failures), rerr) + return res, false, nil + } + return res, s.want, nil +} + +// webDirInUsePath matches a quoted path that is deployment/web or lies under +// it, in either separator style and in JSON-escaped form. +var webDirInUsePath = regexp.MustCompile(`[\\/]web([\\/][^'"]*)?['"]`) + +// webDirInUse reports whether a failed serve build failed on a sharing +// violation for deployment/web or a file under it. +func webDirInUse(res *BuildResult) bool { + if res == nil || res.OK() { + return false + } + text := res.Message + "\n" + string(res.Raw) + return strings.Contains(text, "being used by another process") && webDirInUsePath.MatchString(text) +} + // firstLine is s up to its first newline. func firstLine(s string) string { if i := strings.IndexByte(s, '\n'); i >= 0 { diff --git a/cmd/mxcli/docker/webclient_supervisor_test.go b/cmd/mxcli/docker/webclient_supervisor_test.go index 128405680b..bbbbf19c06 100644 --- a/cmd/mxcli/docker/webclient_supervisor_test.go +++ b/cmd/mxcli/docker/webclient_supervisor_test.go @@ -4,6 +4,7 @@ package docker import ( "bytes" + "encoding/json" "errors" "io" "net/http" @@ -410,3 +411,113 @@ func TestBundlerSupervisor_AwaitRebuildHealthyNoRestart(t *testing.T) { t.Fatalf("a healthy rebuild restarted the bundler (%d starts)", pool.started) } } + +// webDirLockedMessage is the serve build failure reported in #1342, verbatim +// but for the elided user path. +const webDirLockedMessage = `The process cannot access the file 'C:\Users\me\App\deployment\web' because it is being used by another process.` + +// lockingServeBuild models Windows for a build that recreates deployment/web: +// while any bundler is alive — its working directory is web/ — the delete +// fails with the #1342 message; with none alive the build succeeds. +func lockingServeBuild(pool *bundlerPool, calls *int) func() (*BuildResult, error) { + return func() (*BuildResult, error) { + *calls++ + if pool.liveCount() > 0 { + raw, _ := json.Marshal(map[string]string{"status": "Failure", "message": webDirLockedMessage}) + return &BuildResult{Status: "Failure", Message: webDirLockedMessage, Raw: raw}, nil + } + return &BuildResult{Status: "Success"}, nil + } +} + +// The reported symptom: a domain model change under `run --local --watch` on +// Windows failed with "being used by another process" on deployment\web, every +// time. The build must succeed, with the bundler stopped before the retry and a +// fresh one running afterwards. +func TestBundlerSupervisor_BuildReleasingWebDir_RetriesWithBundlerStopped(t *testing.T) { + pool := &bundlerPool{} + var out bytes.Buffer + sup, first := newFakeSupervisor(pool, &out) + calls := 0 + + res, rebundled, err := sup.BuildReleasingWebDir(lockingServeBuild(pool, &calls)) + if err != nil || !res.OK() { + t.Fatalf("build = (%+v, %v), want success once the bundler let go of web/", res, err) + } + if calls != 2 { + t.Fatalf("serve build ran %d time(s), want the failed one and one retry", calls) + } + if !first.stopped { + t.Fatal("the bundler holding web/ was not stopped before the retry") + } + if !rebundled || pool.started != 2 || pool.liveCount() != 1 || sup.cur == clientBundler(first) { + t.Fatalf("rebundled=%v started=%d live=%d, want a fresh bundler running on the new web/", + rebundled, pool.started, pool.liveCount()) + } + if !strings.Contains(out.String(), "held open by the web client bundler") { + t.Fatalf("the retry is not reported:\n%s", out.String()) + } +} + +// Every other failure is the user's model and must reach them unchanged, with +// the incremental bundler left alone. +func TestBundlerSupervisor_BuildReleasingWebDir_OtherFailureNoRetry(t *testing.T) { + pool := &bundlerPool{} + sup, first := newFakeSupervisor(pool, io.Discard) + calls := 0 + res, rebundled, err := sup.BuildReleasingWebDir(func() (*BuildResult, error) { + calls++ + return &BuildResult{Status: "Failure", Message: "The app contains: 1 error."}, nil + }) + if err != nil || res.OK() || rebundled { + t.Fatalf("build = (%+v, %v, %v), want the failure passed through", res, rebundled, err) + } + if calls != 1 || first.stopped || pool.started != 1 { + t.Fatalf("calls=%d stopped=%v started=%d, want no retry and the bundler untouched", calls, first.stopped, pool.started) + } +} + +// When the retry fails as well, the bundler stays down rather than being +// restarted onto a half-built web/; EnsureAlive brings it back on the next +// change. +func TestBundlerSupervisor_BuildReleasingWebDir_RetryFails(t *testing.T) { + pool := &bundlerPool{} + sup, _ := newFakeSupervisor(pool, io.Discard) + res, rebundled, err := sup.BuildReleasingWebDir(func() (*BuildResult, error) { + return &BuildResult{Status: "Failure", Message: webDirLockedMessage}, nil + }) + if err != nil || res.OK() || rebundled { + t.Fatalf("build = (%+v, %v, %v), want the retry's failure", res, rebundled, err) + } + if pool.started != 1 || sup.cur != nil || !sup.Wanted() { + t.Fatalf("started=%d cur=%v wanted=%v, want the bundler down but still wanted", pool.started, sup.cur, sup.Wanted()) + } + if restarted, err := sup.EnsureAlive(); err != nil || !restarted { + t.Fatalf("EnsureAlive = (%v, %v), want the bundler back", restarted, err) + } +} + +func TestWebDirInUse(t *testing.T) { + cases := map[string]struct { + res *BuildResult + want bool + }{ + "reported message": {&BuildResult{Status: "Failure", Message: webDirLockedMessage}, true}, + "JSON-escaped raw only": {&BuildResult{Status: "Failure", + Raw: json.RawMessage(`{"message":"The process cannot access the file 'C:\\App\\deployment\\web' because it is being used by another process."}`)}, true}, + "file under web/": {&BuildResult{Status: "Failure", + Message: `The process cannot access the file 'C:\App\deployment\web\dist\index.js' because it is being used by another process.`}, true}, + "other locked path": {&BuildResult{Status: "Failure", + Message: `The process cannot access the file 'C:\App\deployment\model\model.mdp' because it is being used by another process.`}, false}, + "webapp sibling": {&BuildResult{Status: "Failure", + Message: `The process cannot access the file 'C:\App\deployment\webapp' because it is being used by another process.`}, false}, + "model error": {&BuildResult{Status: "Failure", Message: "The app contains: 1 error."}, false}, + "success": {&BuildResult{Status: "Success", Message: webDirLockedMessage}, false}, + "nil response": {nil, false}, + } + for name, c := range cases { + if got := webDirInUse(c.res); got != c.want { + t.Errorf("%s: webDirInUse = %v, want %v", name, got, c.want) + } + } +} diff --git a/cmd/mxcli/docker/webclient_webdir_windows_test.go b/cmd/mxcli/docker/webclient_webdir_windows_test.go new file mode 100644 index 0000000000..afe4ef0580 --- /dev/null +++ b/cmd/mxcli/docker/webclient_webdir_windows_test.go @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: Apache-2.0 + +//go:build windows + +package docker + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// The mechanism behind #1342, on a real process: a bundler whose working +// directory is deployment/web keeps Windows from removing web/, with the same +// "being used by another process" the serve build reported, and Stop releases +// it. BuildReleasingWebDir rests on both halves; if either stops holding, its +// retry is either unnecessary or useless. +func TestWebClientWatcher_WorkingDirPinsWebDirUntilStop(t *testing.T) { + web := filepath.Join(t.TempDir(), "deployment", "web") + if err := os.MkdirAll(web, 0o755); err != nil { + t.Fatal(err) + } + // "grandchild" announces itself once running: the child opens its working + // directory during its own initialisation, after Start has returned, so + // removing web/ straight after Start races it. + cmd := helperCmd(t, "grandchild") + cmd.Dir = web // as StartWebClientWatch runs the rollup runner + setProcessGroup(cmd) + stdout, err := cmd.StdoutPipe() + if err != nil { + t.Fatal(err) + } + wc, err := launchWatcher(cmd, &syncBuffer{}, stdout) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = wc.Stop() }) + + for start := time.Now(); !strings.Contains(wc.Log(), "grandchild-started"); time.Sleep(20 * time.Millisecond) { + if time.Since(start) > 10*time.Second { + t.Fatalf("helper never announced itself:\n%s", wc.Log()) + } + } + + // Control: with the process up, web/ cannot go. + err = os.Remove(web) + if err == nil { + t.Fatal("removed deployment/web while a process had it as working directory; the #1342 premise no longer holds") + } + if !strings.Contains(err.Error(), "being used by another process") { + t.Fatalf("remove failed with %v, want the sharing violation #1342 reported", err) + } + + if err := wc.Stop(); err != nil { + t.Fatal(err) + } + // The kernel drops the handle as the process is torn down; allow a moment. + deadline := time.Now().Add(5 * time.Second) + for { + if err = os.Remove(web); err == nil { + return + } + if time.Now().After(deadline) { + t.Fatalf("deployment/web still pinned after Stop: %v", err) + } + time.Sleep(50 * time.Millisecond) + } +} diff --git a/docs-site/src/appendixes/data-type-mapping.md b/docs-site/src/appendixes/data-type-mapping.md index 9ccb3bacc8..5e7dc166df 100644 --- a/docs-site/src/appendixes/data-type-mapping.md +++ b/docs-site/src/appendixes/data-type-mapping.md @@ -96,7 +96,7 @@ MDL does not perform implicit type conversions. Types must match exactly. @Position(100, 100) CREATE PERSISTENT ENTITY Demo.AllTypes ( /** Auto-generated ID */ - Id: AutoNumber NOT NULL UNIQUE DEFAULT 1, + ItemNumber: AutoNumber NOT NULL UNIQUE DEFAULT 1, /** Short text field */ Code: String(10) NOT NULL UNIQUE, diff --git a/docs-site/src/appendixes/quick-reference.md b/docs-site/src/appendixes/quick-reference.md index 9e19709cdc..bb6a3328f1 100644 --- a/docs-site/src/appendixes/quick-reference.md +++ b/docs-site/src/appendixes/quick-reference.md @@ -122,7 +122,7 @@ CREATE CONSUMED ODATA SERVICE MyModule.ExternalAPI ( **OData Service Example:** ```sql CREATE PUBLISHED ODATA SERVICE MyModule.CustomerAPI ( - Path: '/odata/customers', + Path: 'odata/customers/', Version: '1.0.0', ODataVersion: OData4, Namespace: 'MyModule.Customers', @@ -137,7 +137,9 @@ CREATE PUBLISHED ODATA SERVICE MyModule.CustomerAPI ( UsePaging: Yes, PageSize: 100 ) - EXPOSE (Name, Email, Phone); + -- The KEY attribute must be unique on the entity + -- (Email: String(200) UNIQUE), or mxbuild reports CE6624. + EXPOSE (Email (KEY), Name, Phone); }; ``` diff --git a/docs-site/src/examples/crm-module.md b/docs-site/src/examples/crm-module.md index 311780eadc..e3667a683e 100644 --- a/docs-site/src/examples/crm-module.md +++ b/docs-site/src/examples/crm-module.md @@ -49,7 +49,7 @@ CREATE PERSISTENT ENTITY CRM.ContactLog ( /** Date and time of the interaction */ ContactDate: DateTime NOT NULL, /** Type of interaction */ - Type: Enumeration(CRM.ContactType) DEFAULT 'Email', + ContactType: Enumeration(CRM.ContactType) DEFAULT 'Email', /** Summary of what was discussed */ Summary: String(2000) NOT NULL ERROR MESSAGE 'Summary is required', /** Follow-up needed? */ @@ -68,6 +68,7 @@ CREATE ASSOCIATION CRM.ContactLog_Customer The two-microflow pattern: a validation microflow returns field-level feedback, and an action microflow calls it before saving. ```sql +mdl 1; CREATE MICROFLOW CRM.VAL_Customer ($Customer: CRM.Customer) RETURNS Boolean AS $IsValid BEGIN @@ -78,7 +79,7 @@ BEGIN VALIDATION FEEDBACK $Customer/Name MESSAGE 'Name cannot be empty'; END IF; - IF $Customer/Email != empty AND NOT contains($Customer/Email, '@') THEN + IF $Customer/Email != empty AND not(contains($Customer/Email, '@')) THEN SET $IsValid = false; VALIDATION FEEDBACK $Customer/Email MESSAGE 'Enter a valid email address'; END IF; @@ -90,12 +91,11 @@ BEGIN RETURN $IsValid; END; -/ CREATE MICROFLOW CRM.ACT_Customer_Save ($Customer: CRM.Customer) RETURNS Boolean AS $IsValid BEGIN - $IsValid = CALL MICROFLOW CRM.VAL_Customer(param = $Customer); + $IsValid = CALL MICROFLOW CRM.VAL_Customer(Customer = $Customer); IF $IsValid THEN COMMIT $Customer; @@ -104,31 +104,13 @@ BEGIN RETURN $IsValid; END; -/ ``` ## Pages ```sql mdl 1; --- Overview page with data grid -CREATE PAGE CRM.Customer_Overview ( - Title: 'Customers', - Layout: Atlas_Core.Atlas_Default -) { - DATAGRID dgCustomers (DataSource: DATABASE CRM.Customer, Selection: Single) { - COLUMN (Attribute: Name, Caption: 'Name') { TEXTFILTER fName } - COLUMN (Attribute: Email, Caption: 'Email') { TEXTFILTER fEmail } - COLUMN (Attribute: Phone, Caption: 'Phone') - COLUMN (Attribute: Status, Caption: 'Status') - COLUMN (Attribute: IsActive, Caption: 'Active') - CONTROLBAR { - ACTIONBUTTON btnNew (Caption: 'New', Action: SHOW PAGE CRM.Customer_NewEdit, ButtonStyle: Primary) - } - } -}; - --- NewEdit page with validation +-- NewEdit page with validation (first: the overview opens it) CREATE PAGE CRM.Customer_NewEdit ( Params: ( $Customer: CRM.Customer ), Title: 'Customer', @@ -156,6 +138,23 @@ CREATE PAGE CRM.Customer_NewEdit ( } }; +-- Overview page with data grid +CREATE PAGE CRM.Customer_Overview ( + Title: 'Customers', + Layout: Atlas_Core.Atlas_Default +) { + DATAGRID dgCustomers (DataSource: DATABASE CRM.Customer, Selection: Single) { + COLUMN (Attribute: Name, Caption: 'Name') { TEXTFILTER fName } + COLUMN (Attribute: Email, Caption: 'Email') { TEXTFILTER fEmail } + COLUMN (Attribute: Phone, Caption: 'Phone') + COLUMN (Attribute: Status, Caption: 'Status') + COLUMN (Attribute: IsActive, Caption: 'Active') + CONTROLBAR { + ACTIONBUTTON btnNew (Caption: 'New', Action: CREATE OBJECT CRM.Customer THEN SHOW PAGE CRM.Customer_NewEdit, ButtonStyle: Primary) + } + } +}; + ``` ## Security @@ -184,7 +183,7 @@ CREATE OR MODIFY USER ROLE CRMUser ( ModuleRoles: (System.User, CRM.User) ); CREATE OR MODIFY USER ROLE CRMAdmin ( ModuleRoles: (System.User, CRM.Admin) ); -- Demo users for testing -CREATE OR MODIFY DEMO USER 'crm_user' ( Password: 'Password1!', UserRoles: (CRMUser) ); -CREATE OR MODIFY DEMO USER 'crm_admin' ( Password: 'Password1!', UserRoles: (CRMAdmin) ); +CREATE OR MODIFY DEMO USER 'crm_user' ( Password: 'CrmPassword1!', UserRoles: (CRMUser) ); +CREATE OR MODIFY DEMO USER 'crm_admin' ( Password: 'CrmPassword1!', UserRoles: (CRMAdmin) ); ALTER APP SECURITY ( EnableDemoUsers: TRUE ); ``` diff --git a/docs-site/src/language/basics.md b/docs-site/src/language/basics.md index e4d88116aa..efc416aedd 100644 --- a/docs-site/src/language/basics.md +++ b/docs-site/src/language/basics.md @@ -25,7 +25,7 @@ mdl 1; CREATE MODULE OrderManagement; CREATE PERSISTENT ENTITY Sales.Order ( - OrderId: AutoNumber NOT NULL UNIQUE, + OrderId: AutoNumber NOT NULL UNIQUE DEFAULT 1, OrderDate: DateTime NOT NULL, ) INDEX (OrderDate DESC); diff --git a/docs-site/src/language/data-types.md b/docs-site/src/language/data-types.md index 1be6713cb6..e97539d9e1 100644 --- a/docs-site/src/language/data-types.md +++ b/docs-site/src/language/data-types.md @@ -17,7 +17,7 @@ MDL has a type system that maps to the Mendix metamodel attribute types. Every a ```sql CREATE PERSISTENT ENTITY Demo.AllTypes ( - Id: AutoNumber NOT NULL UNIQUE DEFAULT 1, + ItemNumber: AutoNumber NOT NULL UNIQUE DEFAULT 1, Code: String(10) NOT NULL UNIQUE, Name: String(200) NOT NULL, Description: String(unlimited), diff --git a/docs-site/src/language/entities.md b/docs-site/src/language/entities.md index e38a0bbfa6..52f53d4b25 100644 --- a/docs-site/src/language/entities.md +++ b/docs-site/src/language/entities.md @@ -43,7 +43,7 @@ CREATE PERSISTENT ENTITY Sales.Customer ( /** Whether the account is active */ IsActive: Boolean DEFAULT TRUE, /** Timestamp of account creation */ - CreatedDate: DateTime, + CreatedDate: AutoCreatedDate, /** Current lifecycle status */ Status: Enumeration(Sales.CustomerStatus) DEFAULT 'Active' ) @@ -109,7 +109,7 @@ Creates the entity if it does not exist, or updates it if it does. New attribute ```sql CREATE OR MODIFY PERSISTENT ENTITY Sales.Customer ( - CustomerId: AutoNumber NOT NULL UNIQUE, + CustomerId: AutoNumber NOT NULL UNIQUE DEFAULT 1, Name: String(200) NOT NULL, Email: String(200), Phone: String(50) -- new attribute added on modify @@ -122,7 +122,7 @@ Persistent entities can track who created/modified objects and when. Declare the ```sql CREATE PERSISTENT ENTITY Sales.Order ( - OrderNumber: AutoNumber, + OrderNumber: AutoNumber DEFAULT 1, TotalAmount: Decimal, Owner: AutoOwner, ChangedBy: AutoChangedBy, @@ -183,7 +183,7 @@ CREATE PERSISTENT ENTITY Sales.Customer ( IsActive: Boolean DEFAULT TRUE, /** Timestamp of account creation */ - CreatedDate: DateTime, + CreatedDate: AutoCreatedDate, /** Current lifecycle status */ Status: Enumeration(Sales.CustomerStatus) DEFAULT 'Active' diff --git a/docs-site/src/language/grant-revoke.md b/docs-site/src/language/grant-revoke.md index dfd6f75aeb..3ec0198e30 100644 --- a/docs-site/src/language/grant-revoke.md +++ b/docs-site/src/language/grant-revoke.md @@ -158,8 +158,8 @@ CREATE MODULE ROLE Shop.Viewer DESCRIPTION 'Read-only access'; -- User roles CREATE USER ROLE Administrator ( ModuleRoles: (Shop.Admin, System.Administrator), ManageAllRoles: true ); -CREATE USER ROLE Employee ( ModuleRoles: (Shop.User) ); -CREATE USER ROLE Guest ( ModuleRoles: (Shop.Viewer) ); +CREATE USER ROLE Employee ( ModuleRoles: (Shop.User, System.User) ); +CREATE USER ROLE Guest ( ModuleRoles: (Shop.Viewer, System.User) ); -- Entity access GRANT CREATE, DELETE, READ *, WRITE * ON ENTITY Shop.Customer TO Shop.Admin; diff --git a/docs-site/src/language/indexes.md b/docs-site/src/language/indexes.md index 635e7ed028..6a18674297 100644 --- a/docs-site/src/language/indexes.md +++ b/docs-site/src/language/indexes.md @@ -12,7 +12,7 @@ Indexes are placed after the closing parenthesis of the attribute list: ```sql CREATE PERSISTENT ENTITY Sales.Order ( - OrderId: AutoNumber NOT NULL UNIQUE, + OrderId: AutoNumber NOT NULL UNIQUE DEFAULT 1, OrderNumber: String(50) NOT NULL, CustomerId: Long, OrderDate: DateTime, diff --git a/docs-site/src/language/security.md b/docs-site/src/language/security.md index 5191d1c610..a7e37eeb2d 100644 --- a/docs-site/src/language/security.md +++ b/docs-site/src/language/security.md @@ -78,8 +78,9 @@ CREATE USER ROLE Anonymous ( ModuleRoles: (Shop.Viewer, System.User) ); ALTER APP SECURITY ( EnableGuestAccess: TRUE, GuestUserRole: Anonymous ); --- Grant exactly what should be public, and nothing else. -GRANT read * ON ENTITY Shop.Product TO Anonymous; +-- Grant exactly what should be public, and nothing else. Access is granted +-- to a module role (Shop.Viewer), never to the user role itself. +GRANT read * ON ENTITY Shop.Product TO Shop.Viewer; ``` Turning it off keeps the stored role, so switching it back on needs no `ROLE` diff --git a/docs-site/src/reference/domain-model/create-entity.md b/docs-site/src/reference/domain-model/create-entity.md index 4af79359dd..12a1a38e6d 100644 --- a/docs-site/src/reference/domain-model/create-entity.md +++ b/docs-site/src/reference/domain-model/create-entity.md @@ -124,7 +124,7 @@ CREATE PERSISTENT ENTITY Sales.Customer ( IsActive: Boolean DEFAULT TRUE, - CreatedDate: DateTime, + CreatedDate: AutoCreatedDate, Status: Enumeration(Sales.CustomerStatus) DEFAULT 'Active' ) @@ -172,7 +172,7 @@ CREATE NON-PERSISTENT ENTITY Sales.CustomerFilter ( ```sql CREATE OR MODIFY PERSISTENT ENTITY Sales.Customer ( - CustomerId: AutoNumber NOT NULL UNIQUE, + CustomerId: AutoNumber NOT NULL UNIQUE DEFAULT 1, Name: String(200) NOT NULL, Email: String(200), Phone: String(50) @@ -213,7 +213,7 @@ CREATE PERSISTENT ENTITY Sales.Order ( ```sql CREATE PERSISTENT ENTITY Sales.Order ( - OrderId: AutoNumber NOT NULL UNIQUE, + OrderId: AutoNumber NOT NULL UNIQUE DEFAULT 1, OrderNumber: String(50) NOT NULL UNIQUE, CustomerId: Long, OrderDate: DateTime, diff --git a/docs-site/src/reference/page/create-page.md b/docs-site/src/reference/page/create-page.md index 09260ab888..60b431f1b8 100644 --- a/docs-site/src/reference/page/create-page.md +++ b/docs-site/src/reference/page/create-page.md @@ -307,7 +307,9 @@ CREATE PAGE MyModule.Dashboard } COLUMN (Class: 'col-md-4') { CONTAINER cntStats (Class: 'card') { - DYNAMICTEXT txtCount (Attribute: TotalCount) + DATAVIEW dvStats (DataSource: MICROFLOW MyModule.DS_DashboardStats) { + DYNAMICTEXT txtCount (Attribute: TotalCount) + } } } } diff --git a/docs-site/src/reference/page/create-snippet.md b/docs-site/src/reference/page/create-snippet.md index 99623d447f..ab03319599 100644 --- a/docs-site/src/reference/page/create-snippet.md +++ b/docs-site/src/reference/page/create-snippet.md @@ -26,7 +26,7 @@ Snippets can optionally declare parameters. When a snippet has parameters, the ` ### Folder Placement -The optional `Folder` property places the snippet in a subfolder within the module. +The optional `FOLDER 'path'` clause, after the name, places the snippet in a subfolder within the module. ## Parameters @@ -58,8 +58,10 @@ CREATE SNIPPET MyModule.CustomerHeader ) { CONTAINER cntHeader (Class: 'card-header') { - DYNAMICTEXT txtName (Attribute: Name) - DYNAMICTEXT txtEmail (Attribute: Email) + DATAVIEW dvCustomer (DataSource: $Customer) { + DYNAMICTEXT txtName (Attribute: Name) + DYNAMICTEXT txtEmail (Attribute: Email) + } } }; ``` @@ -72,10 +74,12 @@ CREATE SNIPPET MyModule.AddressFields Params: ( $Address: MyModule.Address ) ) { - TEXTBOX txtStreet (Label: 'Street', Attribute: Street) - TEXTBOX txtCity (Label: 'City', Attribute: City) - TEXTBOX txtZip (Label: 'Zip Code', Attribute: ZipCode) - TEXTBOX txtCountry (Label: 'Country', Attribute: Country) + DATAVIEW dvAddress (DataSource: $Address) { + TEXTBOX txtStreet (Label: 'Street', Attribute: Street) + TEXTBOX txtCity (Label: 'City', Attribute: City) + TEXTBOX txtZip (Label: 'Zip Code', Attribute: ZipCode) + TEXTBOX txtCountry (Label: 'Country', Attribute: Country) + } }; ``` @@ -85,7 +89,7 @@ Snippet without parameters: CREATE SNIPPET MyModule.AppFooter { CONTAINER cntFooter (Class: 'app-footer') { - DYNAMICTEXT txtVersion (Attribute: Version) + DYNAMICTEXT txtVersion (Content: 'Version 1.0') } }; ``` diff --git a/docs-site/src/tutorial/create-entity.md b/docs-site/src/tutorial/create-entity.md index fb9367f6a5..675294b3f1 100644 --- a/docs-site/src/tutorial/create-entity.md +++ b/docs-site/src/tutorial/create-entity.md @@ -108,7 +108,7 @@ CREATE PERSISTENT ENTITY MyModule.Product ( Quantity: Integer, Weight: Long, IsActive: Boolean DEFAULT true, - CreatedDate: DateTime, + ReleaseDate: DateTime, Status: MyModule.ProductStatus ); ``` diff --git a/docs-site/src/tutorial/describe-search.md b/docs-site/src/tutorial/describe-search.md index 5128f3d9c5..1e4359b7f4 100644 --- a/docs-site/src/tutorial/describe-search.md +++ b/docs-site/src/tutorial/describe-search.md @@ -33,7 +33,7 @@ DESCRIBE ENTITY MyFirstModule.Order; ```sql mdl 1; CREATE PERSISTENT ENTITY MyFirstModule.Order ( - OrderNumber: AutoNumber, + OrderNumber: AutoNumber DEFAULT 1, OrderDate: DateTime, TotalAmount: Decimal, Status: MyFirstModule.OrderStatus diff --git a/docs-site/src/tutorial/skills.md b/docs-site/src/tutorial/skills.md index 464b95fa77..cfbe7791b5 100644 --- a/docs-site/src/tutorial/skills.md +++ b/docs-site/src/tutorial/skills.md @@ -82,8 +82,8 @@ The core MDL syntax for the topic, with all available options and keywords: ```sql -- From write-microflows.md: CREATE MICROFLOW Module.Name( - $param: EntityType -) RETURNS ReturnType AS $result + $param: Module.Entity +) RETURNS Boolean AS $result BEGIN -- activities here END; diff --git a/docs-wiki/bug-patterns/silent-property-drop.md b/docs-wiki/bug-patterns/silent-property-drop.md index 936db302d1..d4a43641f2 100644 --- a/docs-wiki/bug-patterns/silent-property-drop.md +++ b/docs-wiki/bug-patterns/silent-property-drop.md @@ -59,7 +59,13 @@ grammar keyword properties, every key the builders consume, and every property rather than per-type, because a per-type list would produce false warnings, and the describe half is included so a describe → create round trip never warns about its own output. A manually maintained list is a maintenance risk, which is why it -carries a drift test rather than a promise. +carries a drift test rather than a promise — and **the drift test must read the +source, not a second hand-typed list.** The first guard was a typed sample of the +describe vocabulary; when #813 taught the dataview builder and `describe` a +`ShowFooter`, neither list learned it, and `check` warned that a property it +wrote would be dropped (mendixlabs/mxcli#1346). A `go/parser` scan of the +builders' `Get*Prop` / `lookupPropCI` / `Properties["…"]` reads plus describe's +`"Key: "` strings found four more keys in the same state on its first run. **Warn; do not reject.** Neither the pluggable nor the built-in vocabulary can be proven complete, so an error would trade silent drops for false refusals. The diff --git a/go.mod b/go.mod index dfd3f2e60c..ba015505fb 100644 --- a/go.mod +++ b/go.mod @@ -2,7 +2,7 @@ module github.com/mendixlabs/mxcli go 1.26.0 -toolchain go1.26.6 +toolchain go1.27.2 require ( github.com/alecthomas/chroma/v2 v2.26.1 @@ -28,10 +28,10 @@ require ( go.mongodb.org/mongo-driver/v2 v2.6.0 go.starlark.net v0.0.0-20260102030733-3fee463870c9 go.uber.org/zap v1.28.0 - golang.org/x/crypto v0.56.0 - golang.org/x/net v0.57.0 - golang.org/x/sync v0.22.0 - golang.org/x/term v0.45.0 + golang.org/x/crypto v0.57.0 + golang.org/x/net v0.60.0 + golang.org/x/sync v0.23.0 + golang.org/x/term v0.46.0 gopkg.in/yaml.v3 v3.0.1 modernc.org/sqlite v1.51.0 ) @@ -81,8 +81,8 @@ require ( go.lsp.dev/pkg v0.0.0-20210717090340-384b27a52fb2 // indirect go.uber.org/multierr v1.10.0 // indirect golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect - golang.org/x/sys v0.47.0 // indirect - golang.org/x/text v0.41.0 // indirect + golang.org/x/sys v0.48.0 // indirect + golang.org/x/text v0.42.0 // indirect modernc.org/libc v1.72.3 // indirect modernc.org/mathutil v1.7.1 // indirect modernc.org/memory v1.11.0 // indirect diff --git a/go.sum b/go.sum index b1910872a8..9c71010487 100644 --- a/go.sum +++ b/go.sum @@ -194,26 +194,40 @@ go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/crypto v0.56.0 h1:GUh5Ii4J5jtcseSMiRqr1jXCNHoxjeV9Fmekc2oLy6Y= golang.org/x/crypto v0.56.0/go.mod h1:OMW5y6CY9l38uPLmxU6l6pwcXp1obtLo3e6gT7gQR2I= +golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M= +golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA= golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2wtOnGAHZWCHUM4KGzY= golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70= golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk= golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40= +golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c= golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE= golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU= +golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU= +golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg= golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk= +golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211110154304-99a53858aa08/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo= +golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0= golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w= +golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE= +golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc= golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8= golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M= +golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI= +golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E= golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE= golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk= +golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI= google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= diff --git a/mdl-examples/bug-tests/1342-watch-domain-model-change-windows.mdl b/mdl-examples/bug-tests/1342-watch-domain-model-change-windows.mdl new file mode 100644 index 0000000000..1d169d24c4 --- /dev/null +++ b/mdl-examples/bug-tests/1342-watch-domain-model-change-windows.mdl @@ -0,0 +1,35 @@ +-- ============================================================================ +-- Bug test: mendixlabs/mxcli#1342 — `run --local --watch` cannot apply a +-- domain model change on Windows +-- ============================================================================ +-- +-- Page and microflow edits hot-reloaded, but adding a persistent entity or an +-- attribute failed the rebuild every time with: +-- "The process cannot access the file '...\deployment\web' because it is +-- being used by another process" +-- and the app then returned 404. A domain model change makes mxbuild recreate +-- deployment/web, and the incremental web client bundler runs with web/ as its +-- working directory, which Windows will not let anyone delete. Fixed in +-- bundlerSupervisor.BuildReleasingWebDir: on that failure the bundler is +-- stopped, the build retried, and a fresh bundler started on the new web/. +-- +-- To verify (Windows, a Mendix whose deployment has web/rollup.config.mjs, so +-- mxcli runs the bundler: reported on 11.12.4, reproduced on 11.13.0; 11.12.6 +-- and 11.14+ bundle the client in mxbuild and never hit this): +-- mxcli run --local --watch --db-type hsqldb -p App.mpr +-- mxcli exec mdl-examples/bug-tests/1342-watch-domain-model-change-windows.mdl -p App.mpr +-- The watch log shows "deployment/web is held open by the web client bundler; +-- stopping it and rebuilding..." followed by "build #N applied", and the app +-- serves its pages again. Executing the script a second time adds nothing, so +-- to repeat, add another attribute. +-- ============================================================================ + +mdl 1; + +create module Bug1342; + +create persistent entity Bug1342."Order" ( + "Code": String(20) +); + +alter entity Bug1342."Order" add attribute if not exists "Label": String(100); diff --git a/mdl-examples/bug-tests/1343-navigation-caption-apostrophe.mdl b/mdl-examples/bug-tests/1343-navigation-caption-apostrophe.mdl new file mode 100644 index 0000000000..1fcd9cb277 --- /dev/null +++ b/mdl-examples/bug-tests/1343-navigation-caption-apostrophe.mdl @@ -0,0 +1,34 @@ +mdl 1; +-- ============================================================================ +-- mendixlabs/mxcli#1343: a menu caption with an apostrophe round-trips +-- ============================================================================ +-- +-- `describe navigation` (and `describe menu`, the same renderer) wrote a caption +-- as it was stored, apostrophe and all: +-- +-- menu item 'Customer's orders' ( OnClick: show page … ) +-- +-- which does not re-parse — `line 7:20 extraneous input 's' expecting +-- {MENU_KW, '}'}` — so a describe -> exec round trip lost the navigation +-- statement for any app with an ordinary possessive in its menu. +-- +-- Describing the profile after running this must print the doubled spelling +-- below, and that output must exec unchanged. + +create module NavCaptionQuote; + +create page NavCaptionQuote.Orders ( Title: 'Orders', Layout: Atlas_Core.Atlas_Default ) { + dynamictext t (Content: 'Orders') +}; + +create or modify navigation Responsive + home page NavCaptionQuote.Orders + { + menu 'Customer''s menu' { + menu item 'Customer''s orders' ( OnClick: show page NavCaptionQuote.Orders ) + } + }; + +create or modify menu NavCaptionQuote.Main_Menu { + menu item 'Customer''s orders' ( OnClick: show page NavCaptionQuote.Orders ) +}; diff --git a/mdl-examples/bug-tests/1344-delete-message-default-language.mdl b/mdl-examples/bug-tests/1344-delete-message-default-language.mdl new file mode 100644 index 0000000000..4129c78376 --- /dev/null +++ b/mdl-examples/bug-tests/1344-delete-message-default-language.mdl @@ -0,0 +1,31 @@ +mdl 1; +-- mendixlabs/mxcli#1344 — a restrict association's delete error message must +-- land in the project's language. +-- +-- Before the fix the message was stored as a Texts$Text whose one translation +-- was always en_US. On a project whose default language is not en_US the app had +-- no message in its own language when a delete was refused, and `mx check` +-- reported no error — so the stored LanguageCode is the oracle, not the check. +-- Regression tests: TestCreateAssociation_DeleteMessageUsesProjectLanguage, +-- TestAlterAssociation_DeleteMessageUsesProjectLanguage, +-- TestDeleteErrorMessage_StoredInTheGivenLanguage. + +-- The reporter's configuration: nl_NL as the default language. +alter settings LANGUAGE ADD 'nl_NL'; +alter settings LANGUAGE ( DefaultLanguageCode: 'nl_NL' ); + +create persistent entity MyFirstModule.Issue1344Klant ( Naam: String(200) ); +create persistent entity MyFirstModule.Issue1344Order ( Nummer: Integer ); + +-- CREATE: the message must be stored under nl_NL. +create association MyFirstModule.Issue1344Order_Klant + from MyFirstModule.Issue1344Order to MyFirstModule.Issue1344Klant + type Reference + on delete restrict error message 'Een klant met orders kan niet worden verwijderd'; + +-- ALTER: re-setting it must write nl_NL too. +alter association MyFirstModule.Issue1344Order_Klant + set on delete restrict error message 'Klant heeft nog orders'; + +-- DESCRIBE reads the language CREATE writes, so this prints the Dutch back. +describe association MyFirstModule.Issue1344Order_Klant; diff --git a/mdl-examples/bug-tests/1345-describe-nanoflow-grant-round-trip.mdl b/mdl-examples/bug-tests/1345-describe-nanoflow-grant-round-trip.mdl new file mode 100644 index 0000000000..4ceb4ce9eb --- /dev/null +++ b/mdl-examples/bug-tests/1345-describe-nanoflow-grant-round-trip.mdl @@ -0,0 +1,25 @@ +mdl 1; +-- ============================================================================ +-- Bug mendixlabs/mxcli#1345 — describe nanoflow omitted the grant line. +-- +-- `describe microflow` ends with `grant execute on microflow ... to ...;`, +-- but describe nanoflow had its own body renderer without that block. A +-- describe -> exec round trip (or a renamed copy) left the nanoflow with no +-- allowed roles, and nothing warned. `show access on nanoflow` listed both +-- roles all along: the data was stored and read; only describe dropped it. +-- +-- Verify: `describe nanoflow Issue1345.NF_Checkout;` after running this +-- script ends with the grant statement below. +-- ============================================================================ + +create module Issue1345; +create module role Issue1345.User; +create module role Issue1345.Admin; + +create or modify nanoflow Issue1345.NF_Checkout () +begin +end; + +grant execute on nanoflow Issue1345.NF_Checkout to Issue1345.User, Issue1345.Admin; + +describe nanoflow Issue1345.NF_Checkout; diff --git a/mdl-examples/bug-tests/1347-describe-autodetect-no-catalog.mdl b/mdl-examples/bug-tests/1347-describe-autodetect-no-catalog.mdl new file mode 100644 index 0000000000..04c27387ab --- /dev/null +++ b/mdl-examples/bug-tests/1347-describe-autodetect-no-catalog.mdl @@ -0,0 +1,18 @@ +mdl 1; +-- Found while fixing mendixlabs/mxcli#1347: with no .mxcli/catalog.db, +-- `mxcli describe -p app.mpr Mod.Name` (type auto-detected) reported +-- Error: no describable document named "Mod.Name" found in the project +-- for every document type. The live-project fallback called ListRawUnits on +-- the model engine, which left it to the "not implemented" stub. +-- +-- After running this script, delete .mxcli/catalog.db if present; then +-- mxcli describe -p app.mpr BugTest1347NoCat.MF_Hello +-- must print the microflow. + +create module BugTest1347NoCat; + +create microflow BugTest1347NoCat.MF_Hello () +begin +end; + +describe microflow BugTest1347NoCat.MF_Hello; diff --git a/mdl-examples/bug-tests/1347-describe-published-rest-service.mdl b/mdl-examples/bug-tests/1347-describe-published-rest-service.mdl new file mode 100644 index 0000000000..8199f5b1e5 --- /dev/null +++ b/mdl-examples/bug-tests/1347-describe-published-rest-service.mdl @@ -0,0 +1,29 @@ +mdl 1; +-- Bug #1347: the `mxcli describe` subcommand rejected a published REST +-- service with "Unknown type", and auto-detect (`mxcli describe M.Api`) +-- could not resolve it, although `describe published rest service` works +-- under `mxcli -c` / exec / the REPL. +-- +-- After running this script, all of these must print the service: +-- mxcli describe -p app.mpr published rest service BugTest1347.OrderApi +-- mxcli describe -p app.mpr rest service BugTest1347.OrderApi +-- mxcli describe -p app.mpr publishedrestservice BugTest1347.OrderApi +-- mxcli describe -p app.mpr BugTest1347.OrderApi -- auto-detect + +create module BugTest1347; + +create microflow BugTest1347.MF_GetOrders () +begin +end; + +create published rest service BugTest1347.OrderApi ( + path: 'rest/orders/v1', + version: '1.0.0' +) +{ + resource 'orders' { + get '' microflow BugTest1347.MF_GetOrders; + } +}; + +describe published rest service BugTest1347.OrderApi; diff --git a/mdl-examples/bug-tests/1354-module-role-missing-module.mdl b/mdl-examples/bug-tests/1354-module-role-missing-module.mdl new file mode 100644 index 0000000000..c277af23a0 --- /dev/null +++ b/mdl-examples/bug-tests/1354-module-role-missing-module.mdl @@ -0,0 +1,19 @@ +-- mendixlabs/mxcli#1354: check -p passed a module role created in a module +-- that does not exist; exec then wrote the statements above it and stopped. +-- +-- mxcli check this-file -p app.mpr +-- before (v0.25.0): "Check passed!", exit 0 +-- after: statement 2 "module not found: NoSuchModule", exit 1 +-- mxcli exec this-file -p app.mpr +-- before: "Created module role: Administration.Writer", then +-- "Error: module not found: NoSuchModule" (half-applied) +-- after: refused by the pre-flight; nothing was written +-- +-- Uses Administration (in every blank app). Not a `.fail.mdl`: the refusal +-- needs the project, and `make check-mdl` runs without one — this file parses, +-- and must keep parsing. + +mdl 1; + +create module role Administration.Writer; +create module role NoSuchModule.Reader; diff --git a/mdl-examples/bug-tests/1355-script-set-a-domain.mdl b/mdl-examples/bug-tests/1355-script-set-a-domain.mdl new file mode 100644 index 0000000000..cdb07115c3 --- /dev/null +++ b/mdl-examples/bug-tests/1355-script-set-a-domain.mdl @@ -0,0 +1,22 @@ +-- mendixlabs/mxcli#1355 (file 1 of 2): check -p over a script set resolved +-- every file against the project as it stands, so a later file's reference +-- to anything an earlier file creates was reported as missing — although +-- exec, run on each file in order, succeeds. +-- +-- mxcli check 1355-script-set-a-domain.mdl 1355-script-set-b-page.mdl -p app.mpr +-- before (v0.25.0): file 2 "entity not found: MyFirstModule.ShopCustomer" +-- (the combobox datasource); on later builds the page +-- parameter's entity is reported first +-- after: both files "Check passed!", exit 0 +-- Controls: the two files in the other order still fail (the page comes +-- before its entities), and the project is not changed by the check. +-- +-- Uses MyFirstModule (in every blank app). The pair is checked together; on +-- its own, without a project, each file only parses. + +mdl 1; + +create persistent entity MyFirstModule.ShopCustomer ( Name: String(100) ); +create persistent entity MyFirstModule.ShopOrder ( Number: String(50) ); +create association MyFirstModule.ShopOrder_ShopCustomer + from MyFirstModule.ShopOrder to MyFirstModule.ShopCustomer; diff --git a/mdl-examples/bug-tests/1355-script-set-b-page.mdl b/mdl-examples/bug-tests/1355-script-set-b-page.mdl new file mode 100644 index 0000000000..123a41fef7 --- /dev/null +++ b/mdl-examples/bug-tests/1355-script-set-b-page.mdl @@ -0,0 +1,20 @@ +-- mendixlabs/mxcli#1355 (file 2 of 2): a page whose parameter and combobox +-- datasource name entities 1355-script-set-a-domain.mdl creates. See that +-- file for the command and the before/after. + +mdl 1; + +create or modify page MyFirstModule.ShopOrderEdit +( Title: 'Order', Layout: Atlas_Core.Atlas_Default, Params: ( $Order: MyFirstModule.ShopOrder ) ) +{ + layoutgrid lg { row { column (desktopwidth: autofill) { + dataview dv (datasource: $Order) { + combobox cmbCustomer ( + label: 'Customer', + Association: MyFirstModule.ShopOrder_ShopCustomer, + datasource: database MyFirstModule.ShopCustomer, + CaptionAttribute: Name + ) + } + } } } +}; diff --git a/mdl-examples/bug-tests/1356-test-file-takes-mdl-1-header.test.mdl b/mdl-examples/bug-tests/1356-test-file-takes-mdl-1-header.test.mdl new file mode 100644 index 0000000000..41606d0c77 --- /dev/null +++ b/mdl-examples/bug-tests/1356-test-file-takes-mdl-1-header.test.mdl @@ -0,0 +1,15 @@ +mdl 1; +-- mendixlabs/mxcli#1356 — a .test.mdl takes the `mdl 1;` header. +-- +-- This is the reporter's file after `mxcli fmt --upgrade`, which adds the +-- header to a test file by default (ako/mxcli#847) and rewrites the constructs +-- whose meaning it changes: `limit 1` became `first`, `$Found = …` became +-- `set $Found = …`. `fmt --help` and the test-microflows skill still said a test +-- file takes no header; check and the runner read every block below as mdl 1. + +/** + * @test Finds a customer + * @expect $Found = true + */ +retrieve $Customer from MyModule.Customer where Name = 'Ann' first; +set $Found = $Customer != empty; diff --git a/mdl-examples/bug-tests/1357-odata-key-canbeempty-carry.mdl b/mdl-examples/bug-tests/1357-odata-key-canbeempty-carry.mdl new file mode 100644 index 0000000000..2457fa768b --- /dev/null +++ b/mdl-examples/bug-tests/1357-odata-key-canbeempty-carry.mdl @@ -0,0 +1,38 @@ +-- A published member that becomes part of the key must not keep the +-- CanBeEmpty it was stored with as a non-key member. Running the second +-- statement over the first used to carry CanBeEmpty true onto the key, and +-- mxbuild refused it: CE0309 "Exposed attribute 'Email' of entity 'Customer' +-- that is part of the key cannot be marked as 'Can be empty'". +mdl 1; +create module MyModule; + +create persistent entity MyModule.Customer ( + Name: String(200), + Email: String(200) not null unique, + Phone: String(50) +); + +create published odata service MyModule.CustomerAPI ( + Path: 'odata/customers/', + Version: '1.0.0', + ODataVersion: OData4, + Namespace: 'MyModule.Customers' +) +{ + publish entity MyModule.Customer as 'Customers' ( ReadMode: SOURCE ) + expose (Name, Email, Phone); +}; + +-- Re-run with Email as the key. Email was stored with CanBeEmpty true as a +-- non-key member; as a key it must be written false. (The first statement +-- alone has no key and is CE6585 — it is the stored state, not the target.) +create or modify published odata service MyModule.CustomerAPI ( + Path: 'odata/customers/', + Version: '1.0.0', + ODataVersion: OData4, + Namespace: 'MyModule.Customers' +) +{ + publish entity MyModule.Customer as 'Customers' ( ReadMode: SOURCE ) + expose (Email (KEY), Name, Phone); +}; diff --git a/mdl-examples/bug-tests/1357-skill-examples-pass-check.mdl b/mdl-examples/bug-tests/1357-skill-examples-pass-check.mdl new file mode 100644 index 0000000000..a8632500f7 --- /dev/null +++ b/mdl-examples/bug-tests/1357-skill-examples-pass-check.mdl @@ -0,0 +1,44 @@ +-- mendixlabs/mxcli#1357: three skill examples marked correct failed `mxcli check`. +-- These are the corrected forms; each must check clean. +mdl 1; + +-- write-microflows/reference/control-flow.md, "Robust External Call". +-- Was: `$response = create …` inside the handler, after the call already +-- created $response — MDL063 at check, CE0111 "Duplicate variable name" at build. +create microflow Module.SafeExternalCall ( + $RequestData: string +) +returns Module.Response as $response +begin + $response = call microflow Module.CallExternalAPI(data = $RequestData) + on error without rollback begin + log error node 'ExternalAPI' 'API call failed for: ' + $RequestData; + $fallbackResponse = create Module.Response ( + success = false, + message = 'External service unavailable'); + return $fallbackResponse; + end error; + + return $response; +end; + +-- write-oql-queries/reference/patterns.md, Pattern 3, and +-- odata-data-sharing/reference/walkthroughs.md, CustomerAddressVE. +-- Was: String(400) / unlimited String over a concatenation — MDL031 at check, +-- CE6770 at build. A derived string column is always String(200). +create view entity Shop.OrderDetails ( + OrderId: long, + CustomerName: string(200), + TotalItems: integer, + TotalPrice: decimal +) as ( + select + o.OrderId as OrderId, + c.FirstName + ' ' + c.LastName as CustomerName, + count(ol.OrderLineId) as TotalItems, + o.TotalPrice as TotalPrice + from Shop.CustomerOrder as o + inner join Shop.Order_Customer/Shop.Customer as c + left join Shop.OrderLine_Order/Shop.OrderLine as ol + group by o.OrderId, o.TotalPrice, c.FirstName, c.LastName +); diff --git a/mdl/backend/modelsdk/domainmodel.go b/mdl/backend/modelsdk/domainmodel.go index 1bcbec3668..8087a5fda5 100644 --- a/mdl/backend/modelsdk/domainmodel.go +++ b/mdl/backend/modelsdk/domainmodel.go @@ -4,6 +4,7 @@ package modelsdkbackend import ( "fmt" + "sort" "strconv" "strings" @@ -599,14 +600,16 @@ func assocFromGen(a *genDm.Association) *domainmodel.Association { // (assocToGen writes only the child side's, the side MDL's `on delete` // sets), so a statement that changes the parent behaviour drops it -- // exactly as before it was read. - ErrorMessage: deleteErrorMessageFromGen(db.ParentErrorMessage()), + ErrorMessage: deleteErrorMessageFromGen(db.ParentErrorMessage(), ""), + ErrorMessageTranslations: deleteErrorTranslationsFromGen(db.ParentErrorMessage()), } out.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ Type: domainmodel.DeleteBehaviorType(db.ChildDeleteBehavior()), // Read the refusal message back too, or DESCRIBE cannot emit it and a // describe -> exec round trip writes an association whose runtime will // not start (CapTrackV2 §1). - ErrorMessage: deleteErrorMessageFromGen(db.ChildErrorMessage()), + ErrorMessage: deleteErrorMessageFromGen(db.ChildErrorMessage(), ""), + ErrorMessageTranslations: deleteErrorTranslationsFromGen(db.ChildErrorMessage()), } } @@ -659,14 +662,16 @@ func crossAssocFromGen(ca *genDm.CrossAssociation) *domainmodel.CrossModuleAssoc // (assocToGen writes only the child side's, the side MDL's `on delete` // sets), so a statement that changes the parent behaviour drops it -- // exactly as before it was read. - ErrorMessage: deleteErrorMessageFromGen(db.ParentErrorMessage()), + ErrorMessage: deleteErrorMessageFromGen(db.ParentErrorMessage(), ""), + ErrorMessageTranslations: deleteErrorTranslationsFromGen(db.ParentErrorMessage()), } out.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ Type: domainmodel.DeleteBehaviorType(db.ChildDeleteBehavior()), // Read the refusal message back too, or DESCRIBE cannot emit it and a // describe -> exec round trip writes an association whose runtime will // not start (CapTrackV2 §1). - ErrorMessage: deleteErrorMessageFromGen(db.ChildErrorMessage()), + ErrorMessage: deleteErrorMessageFromGen(db.ChildErrorMessage(), ""), + ErrorMessageTranslations: deleteErrorTranslationsFromGen(db.ChildErrorMessage()), } } if src, ok := ca.Source().(*genDm.OqlViewAssociationSource); ok && src != nil { @@ -676,24 +681,44 @@ func crossAssocFromGen(ca *genDm.CrossAssociation) *domainmodel.CrossModuleAssoc return out } -// deleteErrorMessageFromGen reads the en_US text out of a delete behaviour's -// error message, or "" when there is none. The message is a Texts$Text like any -// caption; MDL carries one string, and CarryTranslations puts the other -// languages back on a rewrite. -func deleteErrorMessageFromGen(el element.Element) string { - txt, ok := el.(*genTexts.Text) - if !ok || txt == nil { - return "" - } - t := textFromGen(txt) - if t == nil { - return "" - } - if v, ok := t.Translations["en_US"]; ok { +// deleteErrorMessageFromGen reads one string out of a delete behaviour's error +// message, or "" when there is none: the translation in lang, else en_US, else +// the first by sorted language code (map order would make the read vary run to +// run). The message is a Texts$Text like any caption; MDL carries one string, +// and CarryTranslations puts the other languages back on a rewrite. +// +// lang is the language the caller is about to write. Reading en_US first +// regardless was half of mendixlabs/mxcli#1344: on a project holding both, the +// alter path compared the new nl_NL message against the stale en_US one. +func deleteErrorMessageFromGen(el element.Element, lang string) string { + tr := deleteErrorTranslationsFromGen(el) + if v, ok := tr[lang]; ok && lang != "" { return v } - for _, v := range t.Translations { + if v, ok := tr["en_US"]; ok { return v } + langs := make([]string, 0, len(tr)) + for l := range tr { + langs = append(langs, l) + } + sort.Strings(langs) + for _, l := range langs { + return tr[l] + } return "" } + +// deleteErrorTranslationsFromGen is every translation of a delete behaviour's +// error message, keyed by LanguageCode, or nil when there is no message. +func deleteErrorTranslationsFromGen(el element.Element) map[string]string { + txt, ok := el.(*genTexts.Text) + if !ok || txt == nil { + return nil + } + t := textFromGen(txt) + if t == nil || len(t.Translations) == 0 { + return nil + } + return t.Translations +} diff --git a/mdl/backend/modelsdk/domainmodel_alter.go b/mdl/backend/modelsdk/domainmodel_alter.go index d6687acd10..d780709514 100644 --- a/mdl/backend/modelsdk/domainmodel_alter.go +++ b/mdl/backend/modelsdk/domainmodel_alter.go @@ -511,10 +511,20 @@ func patchCrossDeleteErrorMessage(db *genDm.AssociationDeleteBehavior, child *do if child == nil || child.Type != domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences { return } - if db.ChildErrorMessage() != nil && deleteErrorMessageFromGen(db.ChildErrorMessage()) == child.ErrorMessage { + if db.ChildErrorMessage() != nil && deleteErrorMessageFromGen(db.ChildErrorMessage(), deleteErrorLanguage(child)) == child.ErrorMessage { return } - txt := textToGen(deleteErrorText(child)) + // Edit the translation in the written language and keep every other + // language's, as ALTER PAGE does for a caption — replacing the whole text + // would drop the Dutch the moment an English message was re-set, or the + // reverse. + sem := deleteErrorText(child) + for l, v := range deleteErrorTranslationsFromGen(db.ChildErrorMessage()) { + if _, ok := sem.Translations[l]; !ok { + sem.Translations[l] = v + } + } + txt := textToGen(sem) assignID(txt) for _, tr := range txt.TranslationsItems() { assignID(tr) @@ -522,6 +532,16 @@ func patchCrossDeleteErrorMessage(db *genDm.AssociationDeleteBehavior, child *do db.SetChildErrorMessage(txt) } +// deleteErrorLanguage is the language deleteErrorText stores db's message under: +// the one the executor named, else the process's authoring language (see +// model.AuthoringLanguage), which is en_US until a project's settings are read. +func deleteErrorLanguage(db *domainmodel.DeleteBehavior) string { + if db == nil || db.ErrorMessageLanguage == "" { + return model.AuthoringLanguage() + } + return db.ErrorMessageLanguage +} + // DeleteAssociation removes an association from a domain model by ID. Used by // DROP ASSOCIATION and by the executor's CREATE OR MODIFY ASSOCIATION (delete + // recreate) path. diff --git a/mdl/backend/modelsdk/domainmodel_write.go b/mdl/backend/modelsdk/domainmodel_write.go index 615de05738..027dd3b646 100644 --- a/mdl/backend/modelsdk/domainmodel_write.go +++ b/mdl/backend/modelsdk/domainmodel_write.go @@ -934,6 +934,12 @@ func assignID(elem element.Element) { // box be left empty, so an empty text is the closest thing to "no message" that // still boots. // +// The translation is keyed by db.ErrorMessageLanguage — the project's default +// language, set by the executor — and falls back to model.AuthoringLanguage +// (en_US until a project's settings are read) only when no language was given. Hardcoding en_US left a nl_NL app with no message in its own +// language when a delete was refused, with mx check reporting nothing +// (mendixlabs/mxcli#1344, the delete-message instance of #970). +// // NOTE: the empty case is INFERRED, not measured. The reference document // (ako/TestApp) captures a message that was filled in; nobody has yet saved a // restrict association with the box cleared, so what Studio Pro writes then is @@ -943,7 +949,7 @@ func deleteErrorText(db *domainmodel.DeleteBehavior) *model.Text { if db != nil { msg = db.ErrorMessage } - return &model.Text{Translations: map[string]string{"en_US": msg}} + return &model.Text{Translations: map[string]string{deleteErrorLanguage(db): msg}} } // copyAssignedIDs writes the identities minted on a gen entity back onto the diff --git a/mdl/backend/modelsdk/issue1344_delete_message_language_test.go b/mdl/backend/modelsdk/issue1344_delete_message_language_test.go new file mode 100644 index 0000000000..abddd1de16 --- /dev/null +++ b/mdl/backend/modelsdk/issue1344_delete_message_language_test.go @@ -0,0 +1,116 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "github.com/mendixlabs/mxcli/model" + genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels" + genTexts "github.com/mendixlabs/mxcli/modelsdk/gen/texts" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// mendixlabs/mxcli#1344: "on delete restrict error_message '…'" stored the +// message as a Texts$Text whose one translation was always en_US, so a nl_NL app +// had no message in its own language when a delete was refused — and mx check +// reported nothing. deleteErrorText hardcoded the language code. + +func restrictMessageTranslations(t *testing.T, db *domainmodel.DeleteBehavior) map[string]string { + t.Helper() + gen := assocToGen(&domainmodel.Association{Name: "Order_Customer", ChildDeleteBehavior: db}) + gdb, ok := gen.DeleteBehavior().(*genDm.AssociationDeleteBehavior) + if !ok || gdb == nil { + t.Fatalf("no AssociationDeleteBehavior on the generated association") + } + txt, ok := gdb.ChildErrorMessage().(*genTexts.Text) + if !ok || txt == nil { + t.Fatal("ChildErrorMessage is not a Texts$Text — the runtime will not start") + } + return textFromGen(txt).Translations +} + +// The reported case: the message lands under the language the executor names. +func TestDeleteErrorMessage_StoredInTheGivenLanguage(t *testing.T) { + got := restrictMessageTranslations(t, &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + ErrorMessage: "Een klant met orders kan niet verwijderd worden", + ErrorMessageLanguage: "nl_NL", + }) + if len(got) != 1 || got["nl_NL"] != "Een klant met orders kan niet verwijderd worden" { + t.Errorf("translations = %v, want exactly nl_NL", got) + } +} + +// CONTROL: no language named keeps en_US, so an en_US project and every caller +// that predates the field are unchanged. +func TestDeleteErrorMessage_NoLanguageKeepsEnUS(t *testing.T) { + got := restrictMessageTranslations(t, &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + ErrorMessage: "blocked", + }) + if len(got) != 1 || got["en_US"] != "blocked" { + t.Errorf("translations = %v, want exactly en_US", got) + } +} + +// The read side hands back every translation, so DESCRIBE can choose the +// project's default instead of en_US. +func TestDeleteErrorMessage_ReadBackCarriesEveryTranslation(t *testing.T) { + gen := assocToGen(&domainmodel.Association{Name: "A", ChildDeleteBehavior: &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + ErrorMessage: "Nee", + ErrorMessageLanguage: "nl_NL", + }}) + back := assocFromGen(gen) + if back.ChildDeleteBehavior == nil { + t.Fatal("no child delete behaviour read back") + } + if got := back.ChildDeleteBehavior.ErrorMessageTranslations; len(got) != 1 || got["nl_NL"] != "Nee" { + t.Errorf("ErrorMessageTranslations = %v, want nl_NL: Nee", got) + } +} + +// The cross-module alter path edits the message in place: it must compare and +// write in the statement's language, and keep the other languages' translations +// rather than replacing the whole text. +func TestPatchCrossDeleteErrorMessage_EditsOneLanguageKeepsTheOthers(t *testing.T) { + db := genDm.NewAssociationDeleteBehavior() + db.SetChildErrorMessage(textToGen(&model.Text{Translations: map[string]string{ + "en_US": "Cannot delete", "nl_NL": "Oud", + }})) + + patchCrossDeleteErrorMessage(db, &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + ErrorMessage: "Nieuw", + ErrorMessageLanguage: "nl_NL", + }) + + got := deleteErrorTranslationsFromGen(db.ChildErrorMessage()) + if got["nl_NL"] != "Nieuw" { + t.Errorf("nl_NL = %q, want the statement's message", got["nl_NL"]) + } + if got["en_US"] != "Cannot delete" { + t.Errorf("en_US = %q, want the stored translation kept", got["en_US"]) + } +} + +// CONTROL for the comparison: the same message re-set in nl_NL is a no-op even +// though the en_US translation differs — reading en_US first would see a change +// on every run and rewrite the text. +func TestPatchCrossDeleteErrorMessage_UnchangedInItsLanguageIsANoOp(t *testing.T) { + db := genDm.NewAssociationDeleteBehavior() + before := textToGen(&model.Text{Translations: map[string]string{ + "en_US": "Cannot delete", "nl_NL": "Nee", + }}) + db.SetChildErrorMessage(before) + + patchCrossDeleteErrorMessage(db, &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + ErrorMessage: "Nee", + ErrorMessageLanguage: "nl_NL", + }) + if db.ChildErrorMessage() != before { + t.Error("an unchanged nl_NL message replaced the stored text") + } +} diff --git a/mdl/backend/modelsdk/issue803_cross_assoc_restrict_test.go b/mdl/backend/modelsdk/issue803_cross_assoc_restrict_test.go index c2e69fa634..b0f3f6f8f4 100644 --- a/mdl/backend/modelsdk/issue803_cross_assoc_restrict_test.go +++ b/mdl/backend/modelsdk/issue803_cross_assoc_restrict_test.go @@ -93,7 +93,7 @@ func TestIssue803_CreateCrossAssociationWritesRestrictMessage(t *testing.T) { if msg == nil { t.Fatal("ChildErrorMessage is null on a restrict cross-module association — the runtime will not start (#803)") } - if got := deleteErrorMessageFromGen(msg); got != tc.msg { + if got := deleteErrorMessageFromGen(msg, ""); got != tc.msg { t.Errorf("ChildErrorMessage = %q, want %q", got, tc.msg) } if db.ParentErrorMessage() != nil { diff --git a/mdl/backend/modelsdk/raw_lookup.go b/mdl/backend/modelsdk/raw_lookup.go index ac228b992a..a15711cdae 100644 --- a/mdl/backend/modelsdk/raw_lookup.go +++ b/mdl/backend/modelsdk/raw_lookup.go @@ -36,6 +36,18 @@ func (b *Backend) GetRawUnitByName(objectType, qualifiedName string) (*types.Raw return b.reader.GetRawUnitByName(objectType, qualifiedName) } +// ListRawUnits returns every unit of an object type alias ("" for all units) +// with its qualified name and raw contents. +// +// Recorded in unimplemented_reachability_test.go as a bypass — "the describe +// command holds a concrete reader" — which stopped being true when the root +// package's Open began returning a backend value. describe's no-catalog +// auto-detect (resolveViaReader) then hit this stub and found no document of +// any type. The reader already had it; same gap as the two methods above. +func (b *Backend) ListRawUnits(objectType string) ([]*types.RawUnitInfo, error) { + return b.reader.ListRawUnits(objectType) +} + // GetRawMicroflowByName returns a microflow unit's raw BSON by qualified name. // // Unreachable through the interface today — the reachability probe found no diff --git a/mdl/backend/modelsdk/raw_lookup_test.go b/mdl/backend/modelsdk/raw_lookup_test.go index 7fa819e98c..88fe078a45 100644 --- a/mdl/backend/modelsdk/raw_lookup_test.go +++ b/mdl/backend/modelsdk/raw_lookup_test.go @@ -149,3 +149,46 @@ func TestFastPathReturnTypeMatchesTheSlowPath(t *testing.T) { t.Fatal("compared nothing") } } + +// ListRawUnits is reached through a backend value by describe's auto-detect +// fallback (resolveViaReader in cmd/mxcli/cmd_describe.go), which runs whenever +// the project has no catalog. Left to the stub it answered "not implemented" +// and an empty list, so `mxcli describe Mod.Doc` found no document of any type. +func TestListRawUnits(t *testing.T) { + b := connectedBackend(t) + + // "" is every unit — the form the describe fallback uses. + all, err := b.ListRawUnits("") + if err != nil { + t.Fatalf("ListRawUnits(\"\"): %v", err) + } + var found bool + for _, u := range all { + if u.QualifiedName == "FeedbackModule.ConvertBase64String" { + found = true + if u.Type != "Microflows$Microflow" { + t.Errorf("Type = %q, want Microflows$Microflow", u.Type) + } + if u.ModuleName != "FeedbackModule" { + t.Errorf("ModuleName = %q, want FeedbackModule", u.ModuleName) + } + } + } + if !found { + t.Fatalf("ListRawUnits(\"\") returned %d units, none named FeedbackModule.ConvertBase64String", len(all)) + } + + // A type alias narrows the list to that type and nothing else. + mfs, err := b.ListRawUnits("microflow") + if err != nil { + t.Fatalf("ListRawUnits(\"microflow\"): %v", err) + } + if len(mfs) == 0 || len(mfs) >= len(all) { + t.Fatalf("microflow filter returned %d of %d units", len(mfs), len(all)) + } + for _, u := range mfs { + if u.Type != "Microflows$Microflow" { + t.Errorf("microflow filter returned %s (%s)", u.QualifiedName, u.Type) + } + } +} diff --git a/mdl/backend/modelsdk/unimplemented_reachability_test.go b/mdl/backend/modelsdk/unimplemented_reachability_test.go index 169554396e..aedf6294b8 100644 --- a/mdl/backend/modelsdk/unimplemented_reachability_test.go +++ b/mdl/backend/modelsdk/unimplemented_reachability_test.go @@ -66,7 +66,7 @@ import ( // that hold a concrete *mpr.Reader / *mpr.Writer (the api/ package, examples/, // and cmd/mxcli commands that open a reader directly) — none of which route // through this engine. -// Every remaining entry is a BYPASS, and all five are the same kind of caller: +// Every remaining entry is a BYPASS, and all three are the same kind of caller: // a raw-unit debugging or export command that holds a concrete reader on // purpose. ACCEPTED as deliberate rather than queued for porting (2026-09-15, // closing Phase 3 of docs/plans/2026-09-14-retire-legacy-engine.md) — routing a @@ -82,7 +82,6 @@ var unreachableUnimplemented = map[string]string{ "FindAllCustomWidgetTypes": "BYPASS, accepted: reached only via the reader, inside modelsdk/mpr itself", "GetProjectRootID": "BYPASS, accepted: callers hold a reader; this package uses b.reader.GetProjectRootID directly", "ListAllUnitIDs": "BYPASS, accepted: cmd/mxcli/diag.go holds a concrete reader; infrastructure_write.go uses b.reader", - "ListRawUnits": "BYPASS, accepted: the bson dump/discover/describe commands hold a concrete reader", } func TestNoReachableUnimplementedBackendMethods(t *testing.T) { diff --git a/mdl/conformance/allowlist.txt b/mdl/conformance/allowlist.txt index df5da6ff79..83bc3e5af2 100644 --- a/mdl/conformance/allowlist.txt +++ b/mdl/conformance/allowlist.txt @@ -69,7 +69,6 @@ 7 syntax docs-site/src/appendixes/common-mistakes.md 1 syntax docs-site/src/appendixes/quick-reference.md 3 syntax docs-site/src/appendixes/version-compatibility.md -5 syntax docs-site/src/examples/crm-module.md 1 syntax docs-site/src/examples/data-import.md 1 syntax docs-site/src/examples/rest-integration.md 4 syntax docs-site/src/ide/customizing-skills.md diff --git a/mdl/executor/cmd_associations.go b/mdl/executor/cmd_associations.go index 69bfa520eb..72a4788402 100644 --- a/mdl/executor/cmd_associations.go +++ b/mdl/executor/cmd_associations.go @@ -86,6 +86,9 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error deleteBehavior := storageDeleteBehavior(s.DeleteBehavior) deleteMessage := s.DeleteErrorMessage + // The message is a Texts$Text, keyed by the project's default language like + // every other text mxcli authors (mendixlabs/mxcli#1344, #970). + deleteLang := authoringLanguage(ctx) // Convert storage type. A new association defaults to Column (foreign key on // the FROM entity's table); on OR MODIFY an unstated storage keeps what is @@ -115,7 +118,7 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error if s.Storage != ast.StorageDefault { assoc.StorageFormat = storageFormat } - assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior, ErrorMessage: deleteMessage} + assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior, ErrorMessage: deleteMessage, ErrorMessageLanguage: deleteLang} assoc.Documentation = carriedDocumentation( associationDocumentationStated(s), associationDocumentation(s), assoc.Documentation) // Anchors are applied only when the statement names them — @@ -141,7 +144,7 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error if s.Storage != ast.StorageDefault { // as above (#704) ca.StorageFormat = storageFormat } - ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior, ErrorMessage: deleteMessage} + ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{Type: deleteBehavior, ErrorMessage: deleteMessage, ErrorMessageLanguage: deleteLang} ca.ChildRef = childRef ca.Documentation = carriedDocumentation( associationDocumentationStated(s), associationDocumentation(s), ca.Documentation) @@ -193,8 +196,9 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error ParentID: parentID, ChildRef: childRef, ChildDeleteBehavior: &domainmodel.DeleteBehavior{ - Type: deleteBehavior, - ErrorMessage: deleteMessage, + Type: deleteBehavior, + ErrorMessage: deleteMessage, + ErrorMessageLanguage: deleteLang, }, } if err := ctx.Backend.CreateCrossAssociation(dm.ID, ca); err != nil { @@ -225,8 +229,9 @@ func execCreateAssociation(ctx *ExecContext, s *ast.CreateAssociationStmt) error ParentID: parentID, ChildID: childID, ChildDeleteBehavior: &domainmodel.DeleteBehavior{ - Type: deleteBehavior, - ErrorMessage: deleteMessage, + Type: deleteBehavior, + ErrorMessage: deleteMessage, + ErrorMessageLanguage: deleteLang, }, } applyAnchors(assoc, s.FromAnchor, s.ToAnchor) @@ -281,8 +286,9 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { switch s.Operation { case ast.AlterAssociationSetDeleteBehavior: assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: storageDeleteBehavior(s.DeleteBehavior), - ErrorMessage: s.DeleteErrorMessage, + Type: storageDeleteBehavior(s.DeleteBehavior), + ErrorMessage: s.DeleteErrorMessage, + ErrorMessageLanguage: authoringLanguage(ctx), } case ast.AlterAssociationSetOwner: assoc.Owner = domainmodel.AssociationOwner(s.Owner.String()) @@ -295,7 +301,7 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { } want := alteredAssociationValue(s.Operation, assocAlterView{ del: assoc.ChildDeleteBehavior, owner: string(assoc.Owner), storage: string(assoc.StorageFormat), - doc: assoc.Documentation, anchors: associationAnchors(assoc), + doc: assoc.Documentation, lang: authoringLanguage(ctx), anchors: associationAnchors(assoc), }) if err := ctx.Backend.UpdateDomainModel(dm); err != nil { return mdlerrors.NewBackend("update association", err) @@ -317,8 +323,9 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { switch s.Operation { case ast.AlterAssociationSetDeleteBehavior: ca.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ - Type: storageDeleteBehavior(s.DeleteBehavior), - ErrorMessage: s.DeleteErrorMessage, + Type: storageDeleteBehavior(s.DeleteBehavior), + ErrorMessage: s.DeleteErrorMessage, + ErrorMessageLanguage: authoringLanguage(ctx), } case ast.AlterAssociationSetOwner: ca.Owner = domainmodel.AssociationOwner(s.Owner.String()) @@ -336,7 +343,7 @@ func execAlterAssociation(ctx *ExecContext, s *ast.AlterAssociationStmt) error { } want := alteredAssociationValue(s.Operation, assocAlterView{ del: ca.ChildDeleteBehavior, owner: string(ca.Owner), storage: string(ca.StorageFormat), - doc: ca.Documentation, + doc: ca.Documentation, lang: authoringLanguage(ctx), }) if err := ctx.Backend.UpdateDomainModel(dm); err != nil { return mdlerrors.NewBackend("update cross-module association", err) @@ -423,6 +430,10 @@ func reconcileModuleAccess(ctx *ExecContext, moduleName, why string) error { type assocAlterView struct { del *domainmodel.DeleteBehavior owner, storage, doc, anchors string + // lang is the language the delete message is compared in — the one the + // alter wrote. Reading it back in en_US would compare against a stale + // translation the write carried over (mendixlabs/mxcli#1344). + lang string } // alteredAssociationValue renders the one property op changes, so the value the @@ -436,7 +447,7 @@ func alteredAssociationValue(op ast.AlterAssociationOperation, v assocAlterView) // The message is stored only on the restrict side (assocToGen), so it is // only part of the comparison there. if v.del.Type == domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences { - return string(v.del.Type) + "|" + v.del.ErrorMessage + return string(v.del.Type) + "|" + deleteMessageIn(v.del, v.lang) } return string(v.del.Type) case ast.AlterAssociationSetOwner: @@ -473,7 +484,7 @@ func verifyAssociationAltered(ctx *ExecContext, moduleID model.ID, s *ast.AlterA if a.Name == s.Name.Name { got, found = alteredAssociationValue(s.Operation, assocAlterView{ del: a.ChildDeleteBehavior, owner: string(a.Owner), storage: string(a.StorageFormat), - doc: a.Documentation, anchors: associationAnchors(a), + doc: a.Documentation, lang: authoringLanguage(ctx), anchors: associationAnchors(a), }), true break } @@ -483,7 +494,7 @@ func verifyAssociationAltered(ctx *ExecContext, moduleID model.ID, s *ast.AlterA if ca.Name == s.Name.Name { got, found = alteredAssociationValue(s.Operation, assocAlterView{ del: ca.ChildDeleteBehavior, owner: string(ca.Owner), storage: string(ca.StorageFormat), - doc: ca.Documentation, + doc: ca.Documentation, lang: authoringLanguage(ctx), }), true break } @@ -961,8 +972,23 @@ func describeDeleteClause(ctx *ExecContext, db *domainmodel.DeleteBehavior) stri action = "on delete restrict" } } - if db != nil && db.ErrorMessage != "" { - return action + " error message " + mdlQuote(ctx, db.ErrorMessage) + if msg := deleteMessageIn(db, describeDefaultLanguage(ctx)); msg != "" { + return action + " error message " + mdlQuote(ctx, msg) } return action } + +// deleteMessageIn is a delete behaviour's error message in lang: the language +// CREATE writes it under, so DESCRIBE reads back what was written and describe +// -> exec round-trips on a project whose default is not en_US +// (mendixlabs/mxcli#1344). Without the stored translations — a behaviour built +// in memory rather than read — it is the one string there is. +func deleteMessageIn(db *domainmodel.DeleteBehavior, lang string) string { + if db == nil { + return "" + } + if len(db.ErrorMessageTranslations) == 0 { + return db.ErrorMessage + } + return pickTextTranslation(&model.Text{Translations: db.ErrorMessageTranslations}, lang) +} diff --git a/mdl/executor/cmd_associations_delete_message_language_test.go b/mdl/executor/cmd_associations_delete_message_language_test.go new file mode 100644 index 0000000000..83a79118e3 --- /dev/null +++ b/mdl/executor/cmd_associations_delete_message_language_test.go @@ -0,0 +1,169 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// mendixlabs/mxcli#1344: "on delete restrict error_message '…'" (and +// "delete_behavior prevent error_message '…'") stored the message under en_US +// whatever the project's language. On a nl_NL project the app had no message in +// its own language when a delete was refused, and mx check reported nothing. The +// create, alter and describe handlers never asked for authoringLanguage, so +// fixing the backend alone would still have written en_US. + +// withProjectLanguage makes ctx's mock backend report code as the project's +// DefaultLanguageCode. It must run before anything resolves the language: the +// answer is cached on first use. +func withProjectLanguage(t *testing.T, ctx *ExecContext, code string) { + t.Helper() + mb, ok := ctx.Backend.(*mock.MockBackend) + if !ok { + t.Fatalf("backend is %T, want *mock.MockBackend", ctx.Backend) + } + mb.GetProjectSettingsFunc = func() (*model.ProjectSettings, error) { + return &model.ProjectSettings{Language: &model.LanguageSettings{DefaultLanguageCode: code}}, nil + } + t.Cleanup(func() { model.SetAuthoringLanguage("") }) +} + +func restrictStmt(msg string, createOrModify bool) *ast.CreateAssociationStmt { + return &ast.CreateAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Parent: ast.QualifiedName{Module: "M", Name: "Child"}, + Child: ast.QualifiedName{Module: "M", Name: "Parent"}, + Type: ast.AssocReference, + DeleteBehavior: ast.DeleteIfNoReferences, + DeleteErrorMessage: msg, + CreateOrModify: createOrModify, + } +} + +func assertMessageLanguage(t *testing.T, db *domainmodel.DeleteBehavior, wantLang, wantMsg string) { + t.Helper() + if db == nil { + t.Fatal("no delete behaviour written") + } + if db.ErrorMessage != wantMsg { + t.Errorf("ErrorMessage = %q, want %q", db.ErrorMessage, wantMsg) + } + if db.ErrorMessageLanguage != wantLang { + t.Errorf("ErrorMessageLanguage = %q, want %q — the message is stored in a language the app is not in", + db.ErrorMessageLanguage, wantLang) + } +} + +// The reported statement, as the first statement of a script: CREATE and +// CREATE OR MODIFY, same-module, on a nl_NL project and (control) an en_US one. +func TestCreateAssociation_DeleteMessageUsesProjectLanguage(t *testing.T) { + for _, lang := range []string{"nl_NL", "en_US"} { + t.Run("create-or-modify/"+lang, func(t *testing.T) { + ctx, assoc := assocFixture(t) + withProjectLanguage(t, ctx, lang) + assertNoError(t, execCreateAssociation(ctx, restrictStmt("Kan niet", true))) + assertMessageLanguage(t, assoc.ChildDeleteBehavior, lang, "Kan niet") + }) + + t.Run("create/"+lang, func(t *testing.T) { + ctx, _ := assocFixture(t) + withProjectLanguage(t, ctx, lang) + var created *domainmodel.Association + ctx.Backend.(*mock.MockBackend).CreateAssociationFunc = func(_ model.ID, a *domainmodel.Association) error { + created = a + return nil + } + s := restrictStmt("Kan niet", false) + s.Name.Name = "Child_Parent2" + assertNoError(t, execCreateAssociation(ctx, s)) + if created == nil { + t.Fatal("CreateAssociation was not called") + } + assertMessageLanguage(t, created.ChildDeleteBehavior, lang, "Kan niet") + }) + } +} + +// ALTER … SET DELETE_BEHAVIOR PREVENT ERROR_MESSAGE, same-module and +// cross-module, read back from a copy-on-read store. +func TestAlterAssociation_DeleteMessageUsesProjectLanguage(t *testing.T) { + for _, cross := range []bool{false, true} { + ctx := storeLikeAssocFixture(t, cross, true) + withProjectLanguage(t, ctx, "nl_NL") + assertNoError(t, execAlterAssociation(ctx, &ast.AlterAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Operation: ast.AlterAssociationSetDeleteBehavior, + DeleteBehavior: ast.DeleteIfNoReferences, + DeleteErrorMessage: "Kan niet", + })) + dm, _ := ctx.Backend.GetDomainModel("") + var db *domainmodel.DeleteBehavior + if cross { + db = dm.CrossAssociations[0].ChildDeleteBehavior + } else { + db = dm.Associations[0].ChildDeleteBehavior + } + assertMessageLanguage(t, db, "nl_NL", "Kan niet") + } +} + +// The alter's read-back must compare in the language it wrote. A real backend +// carries the stored en_US translation onto the rewritten text and reports +// ErrorMessage en_US-first, so comparing that string failed every nl_NL alter +// of a message that already had an English one with "change not persisted". +func TestAlterAssociation_VerifiesTheMessageInTheWrittenLanguage(t *testing.T) { + ctx := storeLikeAssocFixture(t, false, true) + withProjectLanguage(t, ctx, "nl_NL") + mb := ctx.Backend.(*mock.MockBackend) + persist := mb.UpdateDomainModelFunc + mb.UpdateDomainModelFunc = func(dm *domainmodel.DomainModel) error { + db := dm.Associations[0].ChildDeleteBehavior + db.ErrorMessageTranslations = map[string]string{"en_US": "Stale English", db.ErrorMessageLanguage: db.ErrorMessage} + db.ErrorMessage = "Stale English" // what the backend reads first + return persist(dm) + } + assertNoError(t, execAlterAssociation(ctx, &ast.AlterAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Child_Parent"}, + Operation: ast.AlterAssociationSetDeleteBehavior, + DeleteBehavior: ast.DeleteIfNoReferences, + DeleteErrorMessage: "Kan niet", + })) +} + +// DESCRIBE reads back the language CREATE writes, or describe -> exec stops +// round-tripping on a project whose default is not en_US. The stored text holds +// both, so picking en_US first would emit the stale English. +func TestDescribeAssociation_DeleteMessageInProjectLanguage(t *testing.T) { + for _, tc := range []struct{ lang, want string }{ + {"nl_NL", "Kan niet"}, + {"en_US", "Cannot"}, // control + } { + t.Run(tc.lang, func(t *testing.T) { + ctx, assoc := assocFixture(t) + withProjectLanguage(t, ctx, tc.lang) + assoc.ChildDeleteBehavior = &domainmodel.DeleteBehavior{ + Type: domainmodel.DeleteBehaviorTypeDeleteMeIfNoReferences, + ErrorMessage: "Cannot", + ErrorMessageTranslations: map[string]string{"en_US": "Cannot", "nl_NL": "Kan niet"}, + } + var buf bytes.Buffer + ctx.Output = &buf + assertNoError(t, describeAssociation(ctx, ast.QualifiedName{Module: "M", Name: "Child_Parent"})) + prog, errs := visitor.Build(buf.String()) + if len(errs) > 0 { + t.Fatalf("DESCRIBE emitted MDL the parser rejects: %v\n%s", errs, buf.String()) + } + stmt := prog.Statements[0].(*ast.CreateAssociationStmt) + if stmt.DeleteErrorMessage != tc.want { + t.Errorf("described message = %q, want %q\n%s", stmt.DeleteErrorMessage, tc.want, buf.String()) + } + }) + } +} diff --git a/mdl/executor/cmd_microflows_show.go b/mdl/executor/cmd_microflows_show.go index cb3e3cae88..2e7e933b42 100644 --- a/mdl/executor/cmd_microflows_show.go +++ b/mdl/executor/cmd_microflows_show.go @@ -364,16 +364,7 @@ func describeMicroflowMode(ctx *ExecContext, name ast.QualifiedName, opts descri lines = append(lines, "end;") - // Add GRANT EXECUTE if roles are assigned - if len(targetMf.AllowedModuleRoles) > 0 { - roles := make([]string, len(targetMf.AllowedModuleRoles)) - for i, r := range targetMf.AllowedModuleRoles { - roles[i] = string(r) - } - lines = append(lines, "") - lines = append(lines, fmt.Sprintf("grant execute on microflow %s.%s to %s;", - name.Module, name.Name, strings.Join(roles, ", "))) - } + lines = append(lines, flowGrantLines("microflow", name, targetMf.AllowedModuleRoles)...) // Output fmt.Fprintln(ctx.Output, strings.Join(lines, "\n")) @@ -516,6 +507,7 @@ func describeNanoflow(ctx *ExecContext, name ast.QualifiedName) error { } lines = append(lines, "end;") + lines = append(lines, flowGrantLines("nanoflow", name, targetNf.AllowedModuleRoles)...) fmt.Fprintln(ctx.Output, strings.Join(lines, "\n")) return nil @@ -721,19 +713,28 @@ func renderMicroflowMDL( lines = append(lines, "end;") - if len(mf.AllowedModuleRoles) > 0 { - roles := make([]string, len(mf.AllowedModuleRoles)) - for i, r := range mf.AllowedModuleRoles { - roles[i] = string(r) - } - lines = append(lines, "") - lines = append(lines, fmt.Sprintf("grant execute on %s %s.%s to %s;", - flowType, name.Module, name.Name, strings.Join(roles, ", "))) - } + lines = append(lines, flowGrantLines(flowType, name, mf.AllowedModuleRoles)...) return strings.Join(lines, "\n") } +// flowGrantLines renders the `grant execute` statement that follows a +// described microflow or nanoflow body, or nothing when no role is allowed. +// The three describe renderers share it: describe nanoflow kept its own copy +// of the body loop without this block, and a describe -> exec round trip +// silently dropped the nanoflow's access rules (mendixlabs/mxcli#1345). +func flowGrantLines(flowType string, name ast.QualifiedName, allowed []model.ID) []string { + if len(allowed) == 0 { + return nil + } + roles := make([]string, len(allowed)) + for i, r := range allowed { + roles[i] = string(r) + } + return []string{"", fmt.Sprintf("grant execute on %s %s.%s to %s;", + flowType, name.Module, name.Name, strings.Join(roles, ", "))} +} + func microflowHasReturnValue(mf *microflows.Microflow) bool { if mf == nil || mf.ReturnType == nil { return false diff --git a/mdl/executor/cmd_nanoflows_mock_test.go b/mdl/executor/cmd_nanoflows_mock_test.go index 6a154c5d6a..ed0edf862d 100644 --- a/mdl/executor/cmd_nanoflows_mock_test.go +++ b/mdl/executor/cmd_nanoflows_mock_test.go @@ -3,11 +3,13 @@ package executor import ( + "strings" "testing" "github.com/mendixlabs/mxcli/mdl/ast" "github.com/mendixlabs/mxcli/mdl/backend/mock" "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/mdl/visitor" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/domainmodel" "github.com/mendixlabs/mxcli/sdk/microflows" @@ -982,3 +984,50 @@ end;`) t.Error("MarkAsUsed = false — the stored true was cleared by the rewrite") } } + +// mendixlabs/mxcli#1345: describe nanoflow printed no grant line, so a +// describe -> exec round trip left the nanoflow with no allowed roles. +// Parse the output rather than grep it: the round trip is what was lost. +func TestDescribeNanoflow_Mock_EmitsGrantForAllowedRoles(t *testing.T) { + mod := mkModule("Shop") + nf := mkNanoflow(mod.ID, "NF_Checkout") + nf.AllowedModuleRoles = []model.ID{"Shop.User", "Shop.Admin"} + + h := mkHierarchy(mod) + withContainer(h, nf.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListNanoflowsFunc: func() ([]*microflows.Nanoflow, error) { return []*microflows.Nanoflow{nf}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return nil, nil }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, describeNanoflow(ctx, ast.QualifiedName{Module: "Shop", Name: "NF_Checkout"})) + + out := buf.String() + prog, errs := visitor.Build(out) + if len(errs) > 0 { + t.Fatalf("describe output does not parse: %v\n%s", errs, out) + } + var grant *ast.GrantNanoflowAccessStmt + for _, stmt := range prog.Statements { + if g, ok := stmt.(*ast.GrantNanoflowAccessStmt); ok { + grant = g + } + } + if grant == nil { + t.Fatalf("describe nanoflow emitted no grant execute statement:\n%s", out) + } + if grant.Nanoflow.String() != "Shop.NF_Checkout" { + t.Errorf("grant targets %s, want Shop.NF_Checkout", grant.Nanoflow.String()) + } + var roles []string + for _, r := range grant.Roles { + roles = append(roles, r.String()) + } + if got := strings.Join(roles, ", "); got != "Shop.User, Shop.Admin" { + t.Errorf("grant roles = %q, want %q", got, "Shop.User, Shop.Admin") + } +} diff --git a/mdl/executor/cmd_navigation.go b/mdl/executor/cmd_navigation.go index e217b21d4d..e857e2ff13 100644 --- a/mdl/executor/cmd_navigation.go +++ b/mdl/executor/cmd_navigation.go @@ -133,12 +133,12 @@ func execAlterNavigation(ctx *ExecContext, s *ast.AlterNavigationStmt) error { // CREATING a profile hides the fact that the project gained a routing target // it did not have — and a phone profile changes which pages a phone lands on. if createdProfile { - fmt.Fprintf(ctx.Output, "Navigation profile '%s' created.\n", s.ProfileName) + fmt.Fprintf(ctx.Output, "Navigation profile %s created.\n", mdlQuoted(s.ProfileName)) // An offline profile changes what the platform demands of pages this // statement never mentioned. Say so now, not at the next build. warnOfflineIncompatiblePages(ctx, createdKind) } else { - fmt.Fprintf(ctx.Output, "Navigation profile '%s' updated.\n", s.ProfileName) + fmt.Fprintf(ctx.Output, "Navigation profile %s updated.\n", mdlQuoted(s.ProfileName)) } reportKeptMenuActions(ctx, kept) return nil @@ -296,8 +296,8 @@ func menuItemActionMDL(ctx *ExecContext, item *types.NavMenuItem) (onClick, note doc := item.ActionDoc if doc == nil { if len(item.StoredAction) > 0 { - return "", fmt.Sprintf("-- menu item '%s': its action (%s) could not be read; "+ - "a rewrite keeps the stored action while the item states none", item.Caption, item.ActionType) + return "", fmt.Sprintf("-- menu item %s: its action (%s) could not be read; "+ + "a rewrite keeps the stored action while the item states none", menuCaptionNote(item.Caption), item.ActionType) } return "", "" } @@ -305,25 +305,33 @@ func menuItemActionMDL(ctx *ExecContext, item *types.NavMenuItem) (onClick, note rendered := renderClientActionMDL(ctx, doc) switch { case strings.HasPrefix(rendered, "--"): - return "", fmt.Sprintf("-- menu item '%s': %s; a rewrite keeps the stored action while the item states none", - item.Caption, strings.TrimSpace(strings.TrimPrefix(rendered, "--"))) + return "", fmt.Sprintf("-- menu item %s: %s; a rewrite keeps the stored action while the item states none", + menuCaptionNote(item.Caption), strings.TrimSpace(strings.TrimPrefix(rendered, "--"))) case rendered == "" && !isNoMenuAction(typeName): - return "", fmt.Sprintf("-- menu item '%s': its action (%s) has no MDL form; "+ - "a rewrite keeps the stored action while the item states none", item.Caption, typeName) + return "", fmt.Sprintf("-- menu item %s: its action (%s) has no MDL form; "+ + "a rewrite keeps the stored action while the item states none", menuCaptionNote(item.Caption), typeName) } if rendered != "" && !menuOnClickParses(rendered) { // A stored action with its target unset renders as a bare `show page` // or `create object`, which is not MDL: say so and keep it. - return "", fmt.Sprintf("-- menu item '%s': its action (%s, %s) has no target MDL can name; "+ - "a rewrite keeps the stored action while the item states none", item.Caption, typeName, rendered) + return "", fmt.Sprintf("-- menu item %s: its action (%s, %s) has no target MDL can name; "+ + "a rewrite keeps the stored action while the item states none", menuCaptionNote(item.Caption), typeName, rendered) } if extras := menuActionExtras(ctx, doc); len(extras) > 0 { - note = fmt.Sprintf("-- menu item '%s': %s has no MDL form; a rewrite keeps it while the item's action is unchanged", - item.Caption, strings.Join(extras, " and ")) + note = fmt.Sprintf("-- menu item %s: %s has no MDL form; a rewrite keeps it while the item's action is unchanged", + menuCaptionNote(item.Caption), strings.Join(extras, " and ")) } return rendered, note } +// menuCaptionNote writes a menu caption inside a `--` note the way the item +// line writes it, apostrophes doubled (mendixlabs/mxcli#1343), so a note reads +// against its item. The escaped spelling keeps a line break in the caption from +// ending the comment and leaving the rest of the note to be parsed as MDL. +func menuCaptionNote(caption string) string { + return mdl0Quote(caption) +} + // menuOnClickParses reports whether an OnClick value reads back as a menu // item's action. func menuOnClickParses(onClick string) bool { @@ -775,7 +783,7 @@ func printMenuMDL(ctx *ExecContext, w io.Writer, items []*types.NavMenuItem, dep } else if onClick != "" { // Studio Pro lets a sub-menu keep an action it no longer runs; MDL // has no OnClick on one, so the rewrite keeps it only as stored. - actionNote = fmt.Sprintf("-- menu '%s': its action (%s) belongs to a sub-menu and has no MDL form there", item.Caption, onClick) + actionNote = fmt.Sprintf("-- menu %s: its action (%s) belongs to a sub-menu and has no MDL form there", menuCaptionNote(item.Caption), onClick) } if icon := menuItemIconMDL(item); icon != "" { props = append(props, "Icon: "+icon) @@ -786,11 +794,11 @@ func printMenuMDL(ctx *ExecContext, w io.Writer, items []*types.NavMenuItem, dep } if len(item.Items) > 0 { // Sub-menu container - fmt.Fprintf(w, "%smenu '%s'%s {\n", indent, item.Caption, propList) + fmt.Fprintf(w, "%smenu %s%s {\n", indent, mdlQuote(ctx, item.Caption), propList) printMenuMDL(ctx, w, item.Items, depth+1, reproducer) fmt.Fprintf(w, "%s}\n", indent) } else { - fmt.Fprintf(w, "%smenu item '%s'%s\n", indent, item.Caption, propList) + fmt.Fprintf(w, "%smenu item %s%s\n", indent, mdlQuote(ctx, item.Caption), propList) } if note := menuItemIconNote(item, reproducer); note != "" { fmt.Fprintf(w, "%s%s\n", indent, note) diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 399699dbb0..362b442a44 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1701,7 +1701,8 @@ func createODataService(ctx *ExecContext, stmt *ast.CreateODataServiceStmt) erro // PageSize only with UsePaging, and Studio Pro stores 10000 either way); // - the stored order of the entity sets, which describe prints in entity-type // order; sets the stored service did not have follow, in statement order; -// - each member's CanBeEmpty, matched by entity and member name. +// - each member's CanBeEmpty, matched by entity and member name, while the +// member's key status is unchanged. func carryPublishedEntityState(storedTypes []*model.PublishedEntityType, storedSets []*model.PublishedEntitySet, svc *model.PublishedODataService) { setIndex := make(map[string]int, len(storedSets)) setByEntity := make(map[string]*model.PublishedEntitySet, len(storedSets)) @@ -1733,7 +1734,11 @@ func carryPublishedEntityState(storedTypes []*model.PublishedEntityType, storedS } for _, et := range svc.EntityTypes { for _, m := range et.Members { - if old, ok := members[et.Entity+"/"+bareMemberName(m.Name)]; ok && old.CanBeEmpty != nil { + // Only while the key status is the one it was stored with: a member + // that becomes a key would otherwise carry CanBeEmpty true onto a + // key, which mxbuild refuses (CE0309). On a change, the derived + // value (!IsPartOfKey) applies. + if old, ok := members[et.Entity+"/"+bareMemberName(m.Name)]; ok && old.CanBeEmpty != nil && old.IsPartOfKey == m.IsPartOfKey { v := *old.CanBeEmpty m.CanBeEmpty = &v } diff --git a/mdl/executor/cmd_odata_service_carry_test.go b/mdl/executor/cmd_odata_service_carry_test.go index 98cfbcc1ed..54c6bf6b16 100644 --- a/mdl/executor/cmd_odata_service_carry_test.go +++ b/mdl/executor/cmd_odata_service_carry_test.go @@ -75,3 +75,52 @@ func TestModifyODataService_CarriesPageSizeOrderAndCanBeEmpty(t *testing.T) { t.Errorf("new member Added got CanBeEmpty %v", *m.CanBeEmpty) } } + +// The CanBeEmpty carry above is keyed on the member's name, so it also carried +// across a change of key status. Exposing a member without (KEY) stores +// CanBeEmpty true; re-running the statement with `Email (KEY)` then wrote true +// onto a key member, and mxbuild refused it: CE0309 "Exposed attribute 'Email' +// of entity 'Customer' that is part of the key cannot be marked as 'Can be +// empty'". A fresh create of the same statement built clean. The stored value +// is the member's state only while its key status is the one it was stored +// with; when that changes, the derived value (!IsPartOfKey) applies. +func TestModifyODataService_CanBeEmptyNotCarriedAcrossKeyChange(t *testing.T) { + svc, mb, h := existingPublishedService() + yes, no := true, false + svc.EntityTypes[0].Members = []*model.PublishedMember{ + {Kind: "attribute", Name: "Label", ExposedName: "label", CanBeEmpty: &yes}, // becomes a key + {Kind: "attribute", Name: "Code", ExposedName: "code", IsPartOfKey: true, CanBeEmpty: &no}, // stops being one + {Kind: "attribute", Name: "Note", ExposedName: "note", CanBeEmpty: &yes}, // control: unchanged + } + var updated *model.PublishedODataService + mb.UpdatePublishedODataServiceFunc = func(s *model.PublishedODataService) error { updated = s; return nil } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + stmt := &ast.CreateODataServiceStmt{ + Name: ast.QualifiedName{Module: "MyModule", Name: "CatalogService"}, + CreateOrModify: true, + Entities: []*ast.PublishedEntityDef{ + {Entity: ast.QualifiedName{Module: "MyModule", Name: "Order"}, ExposedName: "Orders", + Members: []*ast.PublishedMemberDef{ + {Name: "Label", ExposedName: "label", IsPartOfKey: true}, + {Name: "Code", ExposedName: "code"}, + {Name: "Note", ExposedName: "note"}, + }}, + }, + } + assertNoError(t, createODataService(ctx, stmt)) + if updated == nil { + t.Fatal("the service was never updated") + } + + if m := findPublishedMember(t, updated, "Label"); m.CanBeEmpty != nil && *m.CanBeEmpty { + t.Errorf("Label is now part of the key but carried CanBeEmpty true (CE0309)") + } + if m := findPublishedMember(t, updated, "Code"); m.CanBeEmpty != nil && !*m.CanBeEmpty { + t.Errorf("Code is no longer part of the key but carried the key's CanBeEmpty false") + } + // Control: a member whose key status did not change still carries. + if m := findPublishedMember(t, updated, "Note"); m.CanBeEmpty == nil || !*m.CanBeEmpty { + t.Errorf("Note CanBeEmpty = %v, want the stored true", m.CanBeEmpty) + } +} diff --git a/mdl/executor/issue1006_emitter_quoting_test.go b/mdl/executor/issue1006_emitter_quoting_test.go index 0c759e1a21..af1f6b9cfe 100644 --- a/mdl/executor/issue1006_emitter_quoting_test.go +++ b/mdl/executor/issue1006_emitter_quoting_test.go @@ -127,6 +127,7 @@ func TestDescribers_HaveNoHandRolledStringLiterals(t *testing.T) { "cmd_published_rest.go", "cmd_rest_clients.go", "cmd_agenteditor_agents.go", + "cmd_navigation.go", // mendixlabs/mxcli#1343 } { src, err := os.ReadFile(f) if err != nil { diff --git a/mdl/executor/issue1343_menu_caption_quote_test.go b/mdl/executor/issue1343_menu_caption_quote_test.go new file mode 100644 index 0000000000..2f1e70a807 --- /dev/null +++ b/mdl/executor/issue1343_menu_caption_quote_test.go @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// mendixlabs/mxcli#1343: describe navigation wrote a menu caption holding an +// apostrophe unescaped — `menu item 'Customer's orders'` — so its output did +// not re-parse ("extraneous input 's' expecting {MENU_KW, '}'}") and a +// describe -> exec round trip lost the navigation statement. Both shapes a +// caption is written in, a sub-menu and an item, must read back unchanged. +func TestDescribeNavigation_CaptionWithApostropheRoundTrips(t *testing.T) { + mb := &mock.MockBackend{IsConnectedFunc: func() bool { return true }} + ctx, _ := newMockCtx(t, withBackend(mb)) + items := []*types.NavMenuItem{{ + Caption: "Customer's menu", + Items: []*types.NavMenuItem{{Caption: "Customer's orders", Page: "M.Orders"}}, + }} + var buf bytes.Buffer + printMenuMDL(ctx, &buf, items, 1, "CREATE NAVIGATION") + out := buf.String() + + prog, errs := visitor.Build("create or modify navigation Responsive {\n" + out + "};") + if len(errs) > 0 { + t.Fatalf("describe output does not re-parse: %v\n%s", errs[0], out) + } + menu := prog.Statements[0].(*ast.AlterNavigationStmt).MenuItems + if len(menu) != 1 || menu[0].Caption != "Customer's menu" { + t.Fatalf("sub-menu re-parsed as %+v\n%s", menu, out) + } + if len(menu[0].Items) != 1 || menu[0].Items[0].Caption != "Customer's orders" { + t.Fatalf("item re-parsed as %+v\n%s", menu[0].Items, out) + } +} + +// The notes that name a menu item quote its caption the way the item line +// does, so a note can be matched to its item by the same text. +func TestDescribeNavigation_ActionNoteQuotesCaption(t *testing.T) { + mb := &mock.MockBackend{IsConnectedFunc: func() bool { return true }} + ctx, _ := newMockCtx(t, withBackend(mb)) + var buf bytes.Buffer + printMenuMDL(ctx, &buf, []*types.NavMenuItem{{Caption: "Customer's orders", + StoredAction: []byte("x"), ActionType: "Forms$Unknown"}}, 1, "CREATE NAVIGATION") + if !strings.Contains(buf.String(), "-- menu item 'Customer''s orders':") { + t.Errorf("note does not quote the caption as the item line does:\n%s", buf.String()) + } +} diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index b3b3d60112..753a3a03ff 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -683,6 +683,14 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewNotFound("module", s.Name.Module) } } + // exec resolves a module role's module first (findModule) and stops there, + // after the statements above it are written (mendixlabs/mxcli#1354). + case *ast.CreateModuleRoleStmt: + if s.Name.Module != "" && !sc.modules[s.Name.Module] { + if _, err := findModule(ctx, s.Name.Module); err != nil { + return mdlerrors.NewNotFound("module", s.Name.Module) + } + } case *ast.CreateEnumerationStmt: if s.Name.Module != "" && !sc.modules[s.Name.Module] { if _, err := findModule(ctx, s.Name.Module); err != nil { diff --git a/mdl/executor/validate_module_role_module_test.go b/mdl/executor/validate_module_role_module_test.go new file mode 100644 index 0000000000..3acd56a077 --- /dev/null +++ b/mdl/executor/validate_module_role_module_test.go @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/security" +) + +// mendixlabs/mxcli#1354: `create module role NoSuchModule.Reader` passed +// `check -p`, and exec wrote the statements above it before stopping with +// "module not found: NoSuchModule". The module a module role lives in is +// resolved the way exec resolves it (findModule), or created above it. +func TestValidateModuleRole_ModuleMustExist(t *testing.T) { + mod := mkModule("MyModule") + ctx, _ := newMockCtx(t, withBackend(&mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + GetModuleSecurityFunc: func(model.ID) (*security.ModuleSecurity, error) { + return &security.ModuleSecurity{}, nil + }, + })) + + cases := []struct { + name, src string + want []string // substrings, one error each; nil = clean + }{ + {"missing module", "create module role MyModule.Writer;\ncreate module role NoSuchModule.Reader;", + []string{"statement 2: module not found: NoSuchModule"}}, + {"create or modify", "create or modify module role NoSuchModule.Reader;", + []string{"statement 1: module not found: NoSuchModule"}}, + // Controls. + {"stored module", "create module role MyModule.Writer;", nil}, + {"module the script creates", "create module NewMod;\ncreate module role NewMod.Reader;", nil}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + var got []string + for _, err := range validateProgram(ctx, parseMDL(t, c.src)) { + got = append(got, err.Error()) + } + if len(got) != len(c.want) { + t.Fatalf("want %d error(s) %q, got %q", len(c.want), c.want, got) + } + for i, w := range c.want { + if !strings.Contains(got[i], w) { + t.Errorf("error %d: want %q in %q", i, w, got[i]) + } + } + }) + } +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index b1042ba1b5..f4b4a9d329 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -745,8 +745,10 @@ func objectListMappingSet(def *WidgetDefinition) map[string]*ObjectListMapping { // self-warns). A property outside this set on a core widget is not consumed by // any builder — i.e. it is silently dropped — so it earns an MDL-WIDGET07 // warning. It is deliberately generous (a union across all widget types, not -// per-type) to avoid false positives; TestStaticWidgetKnownPropsCoverDescribe -// guards it against describe-vocabulary drift. +// per-type) to avoid false positives. TestStaticWidgetKnownProps_CoverSourceVocabulary +// derives the builder and describe vocabulary from the source and fails on any +// key missing here — the hand-typed sample in TestStaticWidgetKnownPropsCoverDescribe +// drifted with the list it guarded (mendixlabs/mxcli#1346). var staticWidgetKnownProps = func() map[string]bool { names := []string{ // grammar keyword properties (widgetPropertyV3) @@ -779,6 +781,11 @@ var staticWidgetKnownProps = func() map[string]bool { // dynamicimage's fallback image, and the two display flags it shares // with the pluggable image widget. Same reason: describe emits them. "DefaultImage", "OnClickType", + // read by a builder and emitted by describe, but missing here until + // mendixlabs/mxcli#1346: a dataview's explicit footer flag (#813), the + // menu source and orientation of navigationtree / menubar / + // simplemenubar, and a scroll-container region's size mode. + "ShowFooter", "Menu", "Profile", "Orientation", "SizeMode", // fragment / building-block sentinel-internal keys (USE_FRAGMENT / // USE_BUILDING_BLOCK), consumed by the expander, never serialized "Args", "DataSourceOverride", "ActionOverride", @@ -809,7 +816,8 @@ var staticWidgetKnownPropList = func() []string { "Attributes", "FilterType", "DesignProperties", "Width", "Height", "Visible", "Editable", "Tooltip", "DynamicClasses", "WidthUnit", "HeightUnit", "DesktopColumns", "TabletColumns", "PhoneColumns", "PageSize", "Pagination", - "Image", "DefaultImage", "DisplayAs", "OnClickType") + "Image", "DefaultImage", "DisplayAs", "OnClickType", "ShowFooter", "Menu", + "Profile", "Orientation", "SizeMode") return list }() diff --git a/mdl/executor/validate_widgets_known_props_1346_test.go b/mdl/executor/validate_widgets_known_props_1346_test.go new file mode 100644 index 0000000000..40d0f29930 --- /dev/null +++ b/mdl/executor/validate_widgets_known_props_1346_test.go @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + goast "go/ast" + "go/parser" + "go/token" + "os" + "path/filepath" + "regexp" + "sort" + "strconv" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mendixlabs/mxcli#1346. `dataview dv (DataSource: $O, ShowFooter: true)` drew +// MDL-WIDGET07 "not recognized and will be silently dropped on write" although +// the dataview builder reads it and `describe page` emits it. navigationtree / +// menubar's `Menu:`, `Profile:` and `Orientation:` were the same false warning: +// read by the builder, emitted by describe, absent from the allow-list. +func TestStaticWidgetUnknownProps_BuilderConsumedKeysDoNotWarn(t *testing.T) { + prog := parseMDL(t, `create or modify page Shop.Order_Edit ( + title: 'Order', + Layout: Atlas_Core.Atlas_Default, + params: ( $Order: Shop.Order ) +) +{ + dataview dvOrder (datasource: $Order, showFooter: true) { + textbox txtNumber (attribute: Number) + } + navigationtree nav1 (Profile: 'Responsive', Orientation: Vertical) + menubar mb1 (Menu: Shop.MainMenu) +};`) + var widgets []*ast.WidgetV3 + for _, s := range prog.Statements { + if p, ok := s.(*ast.CreatePageStmtV3); ok { + widgets = p.Widgets + } + } + if len(widgets) == 0 { + t.Fatal("no page widgets parsed") + } + for _, v := range validateWidgetTree(widgets, LoadWidgetRegistry(""), "page Shop.Order_Edit") { + if v.RuleID == "MDL-WIDGET07" { + t.Errorf("false MDL-WIDGET07: %s", v.Message) + } + } +} + +// The allow-list and TestStaticWidgetKnownPropsCoverDescribe were both typed by +// hand, so a property added to a builder and to describe (#813's ShowFooter) +// slipped past both. This derives the vocabulary from the source instead: every +// key a static-widget builder reads, and every `Key: …` describe emits, must be +// recognized — otherwise check warns that a written property is dropped, and +// re-executing describe's output warns about itself. +func TestStaticWidgetKnownProps_CoverSourceVocabulary(t *testing.T) { + // Emitted only for the pluggable `image` widget, whose `Params` + // companions the pluggable engine consumes (textTemplateParams); MDL-WIDGET01 + // validates that path, never MDL-WIDGET07. + pluggableOnly := map[string]bool{"ImageUrlParams": true, "AlternativeTextParams": true} + + builderFiles, err := filepath.Glob("cmd_pages_builder*.go") + if err != nil { + t.Fatal(err) + } + consumed := map[string]string{} + fset := token.NewFileSet() + for _, path := range builderFiles { + if strings.HasSuffix(path, "_test.go") { + continue + } + f, err := parser.ParseFile(fset, path, nil, 0) + if err != nil { + t.Fatal(err) + } + goast.Inspect(f, func(n goast.Node) bool { + var lit goast.Expr + switch x := n.(type) { + case *goast.CallExpr: + switch fn := x.Fun.(type) { + case *goast.SelectorExpr: // w.GetStringProp("Menu") + if strings.HasPrefix(fn.Sel.Name, "Get") && strings.HasSuffix(fn.Sel.Name, "Prop") && len(x.Args) == 1 { + lit = x.Args[0] + } + case *goast.Ident: // lookupPropCI(w, "ShowFooter") + if fn.Name == "lookupPropCI" && len(x.Args) == 2 { + lit = x.Args[1] + } + } + case *goast.IndexExpr: // w.Properties["Footer"] + if sel, ok := x.X.(*goast.SelectorExpr); ok && sel.Sel.Name == "Properties" { + lit = x.Index + } + } + if bl, ok := lit.(*goast.BasicLit); ok && bl.Kind == token.STRING { + if key, err := strconv.Unquote(bl.Value); err == nil { + consumed[key] = fset.Position(bl.Pos()).String() + } + } + return true + }) + } + if len(consumed) < 20 { + t.Fatalf("found only %d builder-consumed keys — the source scan no longer matches how builders read properties", len(consumed)) + } + + emitted := map[string]string{} + src, err := os.ReadFile("cmd_pages_describe_output.go") + if err != nil { + t.Fatal(err) + } + for _, m := range regexp.MustCompile(`"([A-Z][A-Za-z]+): `).FindAllStringSubmatch(string(src), -1) { + emitted[m[1]] = "cmd_pages_describe_output.go" + } + if len(emitted) < 20 { + t.Fatalf("found only %d describe-emitted keys — the source scan no longer matches how describe prints properties", len(emitted)) + } + + var missing []string + for _, set := range []map[string]string{consumed, emitted} { + for key, where := range set { + if !pluggableOnly[key] && !isKnownStaticWidgetProp(key) { + missing = append(missing, key+" ("+where+")") + } + } + } + sort.Strings(missing) + for _, m := range missing { + t.Errorf("%s is read or emitted for a built-in widget but is not in staticWidgetKnownProps — check would false-warn MDL-WIDGET07", m) + } +} diff --git a/mdl/exprcheck/parser.go b/mdl/exprcheck/parser.go index 3224be1d18..b75c855e25 100644 --- a/mdl/exprcheck/parser.go +++ b/mdl/exprcheck/parser.go @@ -407,24 +407,25 @@ func parsePrimary(s *Stream, ctx Context) (RobustExpr, []Hint) { if salvage == "" { salvage = s.Consume().Text } - return &RecoveredExpr{ - baseNode: baseNode{P: pos}, - SourceFragment: salvage, - Reason: "unrecognised tokens at primary expression position", - }, []Hint{{ - Code: "E007", - Slug: "unknown-token", - Severity: hints.SeverityWarning, - Where: hints.Location{ - Microflow: ctx.Microflow, - Context: SlotToContext(ctx.SlotPath), - Line: pos.Line, - Column: pos.Column, - }, - YouWrote: salvage, - Problem: "Unrecognised tokens at this position. The parser skipped to the next safe boundary so the rest of the expression could be parsed; additional hints below assume that recovery point.", - Fix: "Replace the highlighted fragment with a valid Mendix expression — a literal, variable, qualified name, or function call.", - }} + recovered := &RecoveredExpr{ + baseNode: baseNode{P: pos}, + SourceFragment: salvage, + Reason: "unrecognised tokens at primary expression position", + } + return recovered, []Hint{{ + Code: "E007", + Slug: "unknown-token", + Severity: hints.SeverityWarning, + Where: hints.Location{ + Microflow: ctx.Microflow, + Context: SlotToContext(ctx.SlotPath), + Line: pos.Line, + Column: pos.Column, + }, + YouWrote: salvage, + Problem: "Unrecognised tokens at this position. The parser skipped to the next safe boundary so the rest of the expression could be parsed; additional hints below assume that recovery point.", + Fix: "Replace the highlighted fragment with a valid Mendix expression — a literal, variable, qualified name, or function call.", + }} } func parseIdentLed(s *Stream, ctx Context) (RobustExpr, []Hint) { diff --git a/mdl/scriptdiff/scriptdiff.go b/mdl/scriptdiff/scriptdiff.go index 93b034b3e2..17d718cefb 100644 --- a/mdl/scriptdiff/scriptdiff.go +++ b/mdl/scriptdiff/scriptdiff.go @@ -83,23 +83,12 @@ func Run(mprPath string, prog *ast.Program, opts Options) (*Report, error) { if opts.NewBackend == nil { return nil, errors.New("scriptdiff: no backend to open the scratch copy with") } - src, err := filepath.Abs(mprPath) + scratch, err := NewScratch(mprPath) if err != nil { return nil, err } - if _, err := os.Stat(src); err != nil { - return nil, err - } - scratch, err := os.MkdirTemp("", "mxcli-diff-") - if err != nil { - return nil, fmt.Errorf("create scratch folder: %w", err) - } - defer os.RemoveAll(scratch) - copyDir := filepath.Join(scratch, "project") - if err := copyProject(filepath.Dir(src), copyDir); err != nil { - return nil, fmt.Errorf("copy the project to %s: %w", copyDir, err) - } - copyMpr := filepath.Join(copyDir, filepath.Base(src)) + defer scratch.Close() + src, copyMpr := scratch.src, scratch.Mpr before, err := TakeSnapshot(copyMpr) if err != nil { @@ -155,6 +144,68 @@ func Run(mprPath string, prog *ast.Program, opts Options) (*Report, error) { return rep, nil } +// Scratch is a scratch copy of a project that scripts can be executed against +// in turn, each seeing what the ones before it wrote — the state exec leaves +// for the next file of a script set. The project it was copied from is only +// read. +type Scratch struct { + dir string // the scratch folder, removed by Close + src string // the project's .mpr, absolute + // Mpr is the copy's .mpr. + Mpr string +} + +// NewScratch copies the project at mprPath into a new scratch folder. +func NewScratch(mprPath string) (*Scratch, error) { + src, err := filepath.Abs(mprPath) + if err != nil { + return nil, err + } + if _, err := os.Stat(src); err != nil { + return nil, err + } + dir, err := os.MkdirTemp("", "mxcli-diff-") + if err != nil { + return nil, fmt.Errorf("create scratch folder: %w", err) + } + copyDir := filepath.Join(dir, "project") + if err := copyProject(filepath.Dir(src), copyDir); err != nil { + os.RemoveAll(dir) + return nil, fmt.Errorf("copy the project to %s: %w", copyDir, err) + } + return &Scratch{dir: dir, src: src, Mpr: filepath.Join(copyDir, filepath.Base(src))}, nil +} + +// Close removes the scratch folder. +func (s *Scratch) Close() error { return os.RemoveAll(s.dir) } + +// Apply executes prog against the copy as exec would — stopping at the first +// error — and returns what exec printed. A statement that would act outside +// the copy (a CONNECT elsewhere, SQL, IMPORT) is refused, and the refusal +// returned, as in Run. Options.Preflight and ContinueOnError are not used. +func (s *Scratch) Apply(prog *ast.Program, opts Options) (string, error) { + if opts.NewBackend == nil { + return "", errors.New("scriptdiff: no backend to open the scratch copy with") + } + var out bytes.Buffer + x := newExecutor(&out, opts) + defer x.Close() + if err := x.Execute(&ast.ConnectStmt{Path: s.Mpr}); err != nil { + return out.String(), fmt.Errorf("connect to the scratch copy: %w", err) + } + g := &outsideGuard{project: s.src, copyMpr: s.Mpr} + x.SetStatementGuard(g.check) + err := x.ExecuteProgram(prog) + _ = x.Execute(&ast.DisconnectStmt{}) + if g.refused != nil { + return out.String(), g.refused + } + if errors.Is(err, executor.ErrExit) { + err = nil + } + return out.String(), err +} + func newExecutor(w io.Writer, opts Options) *executor.Executor { x := executor.New(w) x.SetBackendFactory(opts.NewBackend) diff --git a/modelsdk/mpr/security_patch.go b/modelsdk/mpr/security_patch.go deleted file mode 100644 index c6059ab874..0000000000 --- a/modelsdk/mpr/security_patch.go +++ /dev/null @@ -1,532 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package mpr - -import ( - "fmt" - "strings" - - "go.mongodb.org/mongo-driver/v2/bson" -) - -// ReconcileChange records a single member-access change made during -// reconciliation. Action is either "added" (new entry written) or "stripped" -// (stale entry removed, e.g. a CrossAssociation reference that Studio Pro -// rejects with CE0066). The String() format is the per-entity log line -// printed by UPDATE SECURITY. -type ReconcileChange struct { - Entity string // entity short name - Member string // association or attribute short name - Action string // "added" | "stripped" -} - -func (c ReconcileChange) String() string { - if c.Action == "stripped" { - return fmt.Sprintf("%s: stripped stale MemberAccess for %s", c.Entity, c.Member) - } - return fmt.Sprintf("%s: added MemberAccess for %s", c.Entity, c.Member) -} - -// PatchReconcileMemberAccesses reconciles entity member accesses in a raw -// DomainModel BSON document for the given module. Pure BSON manipulation — -// no database access required. -// Returns patched bytes, a list of every MemberAccess entry that was added, -// and error. A non-empty changes list means the caller must write back the -// patched bytes. -func PatchReconcileMemberAccesses(rawBytes []byte, moduleName string) ([]byte, []ReconcileChange, error) { - var doc bson.D - if err := bson.Unmarshal(rawBytes, &doc); err != nil { - return nil, nil, fmt.Errorf("unmarshal: %w", err) - } - doc, changes := secPatchReconcileMemberAccessesDoc(doc, moduleName) - out, err := bson.Marshal(doc) - if err != nil { - return nil, nil, fmt.Errorf("marshal: %w", err) - } - return out, changes, nil -} - -// secPatchReconcileMemberAccessesDoc performs reconciliation on a parsed BSON -// DomainModel document: ensures each entity's access rules have correct -// MemberAccess entries for all current attributes and associations. -func secPatchReconcileMemberAccessesDoc(doc bson.D, moduleName string) (bson.D, []ReconcileChange) { - var changes []ReconcileChange - - entitiesArr := secGetBsonArray(doc, "Entities") - if entitiesArr == nil { - return doc, nil - } - - assocNames := map[string]bool{} - assocArr := secGetBsonArray(doc, "Associations") - for _, item := range assocArr { - assocDoc, ok := item.(bson.D) - if !ok { - continue - } - for _, f := range assocDoc { - if f.Key == "Name" { - if name, ok := f.Value.(string); ok { - assocNames[name] = true - } - break - } - } - } - crossArr := secGetBsonArray(doc, "CrossAssociations") - for _, item := range crossArr { - crossDoc, ok := item.(bson.D) - if !ok { - continue - } - for _, f := range crossDoc { - if f.Key == "Name" { - if name, ok := f.Value.(string); ok { - assocNames[name] = true - } - break - } - } - } - - for i, item := range entitiesArr { - entityDoc, ok := item.(bson.D) - if !ok { - continue - } - - entityName := "" - for _, f := range entityDoc { - if f.Key == "Name" { - entityName = secBsonString(f.Value) - break - } - } - if entityName == "" { - continue - } - - attrNames := map[string]bool{} - calculatedAttrs := map[string]bool{} - attrsArr := secGetBsonArray(entityDoc, "Attributes") - for _, attrItem := range attrsArr { - attrDoc, ok := attrItem.(bson.D) - if !ok { - continue - } - attrName := "" - isCalculated := false - for _, f := range attrDoc { - if f.Key == "Name" { - attrName = secBsonString(f.Value) - } - if f.Key == "Value" { - if valueDoc, ok := f.Value.(bson.D); ok { - for _, vf := range valueDoc { - if vf.Key == "$Type" { - if vt, ok := vf.Value.(string); ok && vt == "DomainModels$CalculatedValue" { - isCalculated = true - } - } - } - } - } - } - if attrName != "" { - attrNames[attrName] = true - if isCalculated { - calculatedAttrs[attrName] = true - } - } - } - - entityID := "" - for _, f := range entityDoc { - if f.Key == "$ID" { - entityID = secExtractBsonIDValue(f.Value) - break - } - } - entityAssocNames := map[string]bool{} - - systemAssocRefs := map[string]bool{} - for _, f := range entityDoc { - if f.Key == "Generalization" || f.Key == "MaybeGeneralization" { - if genDoc, ok := f.Value.(bson.D); ok { - for _, gf := range genDoc { - if gf.Key == "$Type" { - if gt, ok := gf.Value.(string); ok && gt == "DomainModels$NoGeneralization" { - for _, ngf := range genDoc { - switch ngf.Key { - case "HasOwner": - if v, ok := ngf.Value.(bool); ok && v { - systemAssocRefs["System.owner"] = true - } - case "HasChangedBy": - if v, ok := ngf.Value.(bool); ok && v { - systemAssocRefs["System.changedBy"] = true - } - } - } - } - } - } - } - break - } - } - for _, aItem := range assocArr { - aDoc, ok := aItem.(bson.D) - if !ok { - continue - } - aParentID := "" - aName := "" - for _, f := range aDoc { - switch f.Key { - case "ParentPointer": - aParentID = secExtractBsonIDValue(f.Value) - case "Name": - aName = secBsonString(f.Value) - } - } - if aParentID == entityID && aName != "" { - entityAssocNames[aName] = true - } - } - for _, caItem := range crossArr { - caDoc, ok := caItem.(bson.D) - if !ok { - continue - } - parentID := "" - caName := "" - for _, f := range caDoc { - if f.Key == "ParentPointer" { - parentID = secExtractBsonIDValue(f.Value) - } - if f.Key == "Name" { - caName = secBsonString(f.Value) - } - } - if parentID == entityID && caName != "" { - entityAssocNames[caName] = true - } - } - - for j, f := range entityDoc { - if f.Key != "AccessRules" { - continue - } - rulesArr, ok := f.Value.(bson.A) - if !ok { - break - } - - for k, ruleItem := range rulesArr { - ruleDoc, ok := ruleItem.(bson.D) - if !ok { - continue - } - - ruleDoc, stripped := secStripInvalidAccessRuleProps(ruleDoc) - if stripped { - rulesArr[k] = ruleDoc - // stripped is counted implicitly via changed below - } - - for m, rf := range ruleDoc { - if rf.Key != "MemberAccesses" { - continue - } - maArr, ok := rf.Value.(bson.A) - if !ok { - break - } - // NOTE: do NOT skip when len(maArr) <= 1 (only version prefix). - // Entities with empty MemberAccesses still need association - // entries added — the previous guard caused silent CE0066. - - defaultRights := "ReadWrite" - for _, drf := range ruleDoc { - if drf.Key == "DefaultMemberAccessRights" { - if dr, ok := drf.Value.(string); ok { - defaultRights = dr - } - break - } - } - - coveredAttrs := map[string]bool{} - coveredAssocs := map[string]bool{} - changed := stripped // propagate stripped-props change - // Preserve version prefix; fall back to int32(3) if absent. - var filtered bson.A - if len(maArr) > 0 { - filtered = bson.A{maArr[0]} - } else { - filtered = bson.A{int32(3)} - } - - coveredSystemAssocs := map[string]bool{} - for _, maItem := range maArr[1:] { - maDoc, ok := maItem.(bson.D) - if !ok { - continue - } - attrRef := "" - assocRef := "" - for _, mf := range maDoc { - if mf.Key == "Attribute" { - attrRef = secBsonString(mf.Value) - } - if mf.Key == "Association" { - assocRef = secBsonString(mf.Value) - } - } - - if attrRef != "" { - parts := secSplitQualifiedRef(attrRef) - // An attribute reference is qualified against the entity that - // DECLARES it, so an inherited member carries an ancestor's - // name, not this entity's. Reconciliation used to compare only - // the bare member name against this entity's own attributes, - // so every inherited entry looked stale and was deleted — - // silently, and on any write that touched the module - // (mendixlabs/mxcli#758). The ancestor may live in another - // module or in System, neither of which is present in this - // document, so an inherited reference cannot be validated here - // at all: preserve it rather than delete what cannot be - // checked. - if !secRefBelongsToEntity(attrRef, moduleName, entityName) { - filtered = append(filtered, maDoc) - } else if parts != "" && attrNames[parts] { - coveredAttrs[parts] = true - if calculatedAttrs[parts] { - maDoc = secDowngradeCalculatedAttrRights(maDoc) - } - filtered = append(filtered, maDoc) - } else { - // Genuinely stale: the reference claims to be this entity's - // own attribute, and the entity no longer has it. - changes = append(changes, ReconcileChange{Entity: entityName, Member: parts, Action: "stripped"}) - changed = true - } - } else if assocRef != "" { - if systemAssocRefs[assocRef] { - coveredSystemAssocs[assocRef] = true - filtered = append(filtered, maItem) - } else { - parts := secSplitAssocRef(assocRef) - if parts != "" && entityAssocNames[parts] { - coveredAssocs[parts] = true - filtered = append(filtered, maItem) - } else { - // Stale or cross-module association ref — strip it. - // Critically: CrossAssociation names are not in entityAssocNames, - // so they fall here and are removed (CE0066 fix). - changes = append(changes, ReconcileChange{ - Entity: entityName, - Member: parts, - Action: "stripped", - }) - changed = true - } - } - } else { - filtered = append(filtered, maItem) - } - } - - for attrName := range attrNames { - if !coveredAttrs[attrName] { - rights := defaultRights - if calculatedAttrs[attrName] && (rights == "ReadWrite" || rights == "WriteOnly") { - rights = "ReadOnly" - } - newMA := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "DomainModels$MemberAccess"}, - {Key: "AccessRights", Value: rights}, - {Key: "Attribute", Value: moduleName + "." + entityName + "." + attrName}, - } - filtered = append(filtered, newMA) - changes = append(changes, ReconcileChange{Entity: entityName, Member: attrName}) - changed = true - } - } - - for aName := range entityAssocNames { - if !coveredAssocs[aName] { - newMA := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "DomainModels$MemberAccess"}, - {Key: "AccessRights", Value: defaultRights}, - {Key: "Association", Value: moduleName + "." + aName}, - } - filtered = append(filtered, newMA) - changes = append(changes, ReconcileChange{Entity: entityName, Member: aName}) - changed = true - } - } - - for sysRef := range systemAssocRefs { - if !coveredSystemAssocs[sysRef] { - newMA := bson.D{ - {Key: "$ID", Value: idToBsonBinary(generateUUID())}, - {Key: "$Type", Value: "DomainModels$MemberAccess"}, - {Key: "AccessRights", Value: defaultRights}, - {Key: "Association", Value: sysRef}, - } - filtered = append(filtered, newMA) - changes = append(changes, ReconcileChange{Entity: entityName, Member: sysRef}) - changed = true - } - } - - if changed { - ruleDoc[m].Value = filtered - rulesArr[k] = ruleDoc - } - - break - } - } - - entityDoc[j].Value = rulesArr - break - } - - entitiesArr[i] = entityDoc - } - - return secSetBsonField(doc, "Entities", entitiesArr), changes -} - -// ── Private helpers (sec* prefix to avoid collisions) ──────────────────────── - -// secSetBsonField sets a top-level field in a bson.D, adding it if not found. -func secSetBsonField(doc bson.D, key string, value any) bson.D { - for i, elem := range doc { - if elem.Key == key { - doc[i].Value = value - return doc - } - } - return append(doc, bson.E{Key: key, Value: value}) -} - -// secGetBsonArray returns the bson.A for a named field, or nil if absent. -func secGetBsonArray(doc bson.D, key string) bson.A { - for _, elem := range doc { - if elem.Key == key { - if arr, ok := elem.Value.(bson.A); ok { - return arr - } - } - } - return nil -} - -// secBsonString extracts a string value from a BSON any value (best-effort). -func secBsonString(v any) string { - if s, ok := v.(string); ok { - return s - } - return "" -} - -// secExtractBsonIDValue extracts a string ID from a BSON field value. -func secExtractBsonIDValue(v any) string { - switch val := v.(type) { - case string: - return val - case bson.Binary: - return blobToUUID(val.Data) - default: - return fmt.Sprintf("%v", v) - } -} - -// secSplitQualifiedRef extracts the last component from "Module.Entity.AttrName". -func secSplitQualifiedRef(ref string) string { - parts := secSplitByDot(ref) - if len(parts) >= 3 { - return parts[len(parts)-1] - } - return "" -} - -// secSplitAssocRef extracts the association name from "Module.AssocName". -func secSplitAssocRef(ref string) string { - parts := secSplitByDot(ref) - if len(parts) >= 2 { - return parts[len(parts)-1] - } - return "" -} - -// secSplitByDot splits a string by ".". -func secSplitByDot(s string) []string { - var parts []string - start := 0 - for i := 0; i < len(s); i++ { - if s[i] == '.' { - parts = append(parts, s[start:i]) - start = i + 1 - } - } - parts = append(parts, s[start:]) - return parts -} - -// secDowngradeCalculatedAttrRights downgrades ReadWrite/WriteOnly to ReadOnly -// for calculated attributes (which cannot be written). -func secDowngradeCalculatedAttrRights(doc bson.D) bson.D { - for i, f := range doc { - if f.Key == "AccessRights" { - if rights, ok := f.Value.(string); ok && (rights == "ReadWrite" || rights == "WriteOnly") { - doc[i].Value = "ReadOnly" - } - } - } - return doc -} - -// secInvalidAccessRuleProps lists BSON keys that are NOT valid Mendix metamodel -// properties on DomainModels$AccessRule. Old mxcli versions wrote these; -// Studio Pro crashes with "Sequence contains no matching element" if present. -var secInvalidAccessRuleProps = map[string]bool{ - "AllowRead": true, - "AllowWrite": true, -} - -// secStripInvalidAccessRuleProps removes invalid properties from an AccessRule -// BSON document. Returns the cleaned document and true if anything was removed. -func secStripInvalidAccessRuleProps(doc bson.D) (bson.D, bool) { - cleaned := make(bson.D, 0, len(doc)) - stripped := false - for _, f := range doc { - if secInvalidAccessRuleProps[f.Key] { - stripped = true - continue - } - cleaned = append(cleaned, f) - } - return cleaned, stripped -} - -// secRefBelongsToEntity reports whether a MemberAccess attribute reference -// ("Module.Entity.Attribute") is qualified against the given entity — i.e. names -// one of its OWN attributes rather than one inherited from an ancestor. -// -// Only an own reference can be validated from this document: an ancestor may live -// in another module or in System, neither of which is loaded here. -func secRefBelongsToEntity(attrRef, moduleName, entityName string) bool { - parts := secSplitByDot(attrRef) - if len(parts) < 3 { - return false - } - owner := strings.Join(parts[:len(parts)-1], ".") - return strings.EqualFold(owner, moduleName+"."+entityName) -} diff --git a/modelsdk/mpr/security_patch_test.go b/modelsdk/mpr/security_patch_test.go deleted file mode 100644 index dd6dbfe68d..0000000000 --- a/modelsdk/mpr/security_patch_test.go +++ /dev/null @@ -1,426 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package mpr - -import ( - "fmt" - "testing" - - "go.mongodb.org/mongo-driver/v2/bson" -) - -// buildTestDomainModel constructs a minimal DomainModel BSON document for -// testing PatchReconcileMemberAccesses. entityID is used as the $ID for the -// single entity; the entity owns every association whose ParentPointer equals -// entityID (from assocs) and every cross-module association whose ParentPointer -// equals entityID (from crossAssocs). -// -// attrNames lists attribute names already present in the entity's Attributes -// array. ruleMA is the raw MemberAccesses bson.A to embed in the AccessRule -// (pass nil or empty for "empty access rule"). -func buildTestDomainModel( - entityName string, - entityID string, - attrNames []string, - ruleMA bson.A, - assocs []bson.D, - crossAssocs []bson.D, -) []byte { - attrs := bson.A{int32(3)} - for _, name := range attrNames { - attrs = append(attrs, bson.D{ - {Key: "$ID", Value: entityID + "-attr-" + name}, - {Key: "$Type", Value: "DomainModels$StoredValue"}, - {Key: "Name", Value: name}, - }) - } - - maArr := bson.A{int32(3)} - if ruleMA != nil { - for _, item := range ruleMA { - maArr = append(maArr, item) - } - } - - rule := bson.D{ - {Key: "$ID", Value: entityID + "-rule"}, - {Key: "$Type", Value: "DomainModels$AccessRule"}, - {Key: "AllowCreate", Value: true}, - {Key: "AllowDelete", Value: false}, - {Key: "DefaultMemberAccessRights", Value: "ReadWrite"}, - {Key: "AllowedModuleRoles", Value: bson.A{int32(1), "TestModule.User"}}, - {Key: "MemberAccesses", Value: maArr}, - } - - entity := bson.D{ - {Key: "$ID", Value: entityID}, - {Key: "$Type", Value: "DomainModels$Entity"}, - {Key: "Name", Value: entityName}, - {Key: "Attributes", Value: attrs}, - {Key: "AccessRules", Value: bson.A{int32(3), rule}}, - } - - assocsArr := bson.A{int32(3)} - for _, a := range assocs { - assocsArr = append(assocsArr, a) - } - - crossArr := bson.A{int32(3)} - for _, ca := range crossAssocs { - crossArr = append(crossArr, ca) - } - - dm := bson.D{ - {Key: "$Type", Value: "DomainModels$DomainModel"}, - {Key: "Entities", Value: bson.A{int32(3), entity}}, - {Key: "Associations", Value: assocsArr}, - {Key: "CrossAssociations", Value: crossArr}, - } - - out, err := bson.Marshal(dm) - if err != nil { - panic(fmt.Sprintf("buildTestDomainModel marshal: %v", err)) - } - return out -} - -func assocDoc(name, parentID string) bson.D { - return bson.D{ - {Key: "$ID", Value: name + "-id"}, - {Key: "$Type", Value: "DomainModels$Association"}, - {Key: "Name", Value: name}, - {Key: "ParentPointer", Value: parentID}, - {Key: "ChildPointer", Value: "some-child-id"}, - } -} - -func crossAssocDoc(name, parentID string) bson.D { - return bson.D{ - {Key: "$ID", Value: name + "-id"}, - {Key: "$Type", Value: "DomainModels$CrossAssociation"}, - {Key: "Name", Value: name}, - {Key: "ParentPointer", Value: parentID}, - {Key: "Child", Value: "Other.Entity"}, - } -} - -// TestPatchReconcile_EmptyMemberAccesses_AddsAssociation is the core regression -// test for the len(maArr)<=1 silent-skip bug. -// -// When an entity has an AccessRule whose MemberAccesses contains ONLY the -// int32 version-prefix (len==1), the reconciler must still add MemberAccess -// entries for every owned association — it must NOT silently skip. -// -// Before the fix this test fails because the guard `if len(maArr) <= 1 { break }` -// exits before any work is done, returning 0 changes and leaving the BSON -// untouched. After the fix it passes. -func TestPatchReconcile_EmptyMemberAccesses_AddsAssociation(t *testing.T) { - const moduleName = "TestModule" - const entityName = "MySetting" - const entityID = "entity-mysetting" - const assocName = "MySetting_Account" - - // MemberAccesses has ONLY the version prefix — the silent-skip scenario. - // Use a regular (same-module) association so we test the add-assoc path - // without interference from the CrossAssociation exclusion. - raw := buildTestDomainModel( - entityName, entityID, - []string{"Theme", "Mode"}, // entity has attributes - nil, // MemberAccesses is empty (will be [v3]) - []bson.D{assocDoc(assocName, entityID)}, // regular assoc - nil, - ) - - patched, changes, err := PatchReconcileMemberAccesses(raw, moduleName) - if err != nil { - t.Fatalf("PatchReconcileMemberAccesses returned error: %v", err) - } - - if len(changes) == 0 { - t.Fatalf( - "PatchReconcileMemberAccesses returned 0 changes for entity %q with owned association %q and empty MemberAccesses.\n"+ - "Expected at least one ReconcileChange — the len(maArr)<=1 guard is silently skipping this entity.\n"+ - "Fix: remove the guard so reconciliation runs even when MemberAccesses contains only the version prefix.", - entityName, assocName, - ) - } - - found := false - for _, ch := range changes { - if ch.Entity == entityName && ch.Member == assocName { - found = true - break - } - } - if !found { - t.Errorf( - "changes does not contain entry {Entity:%q, Member:%q}; got: %v", - entityName, assocName, changes, - ) - } - - // Verify the association actually appears in the patched BSON. - assocRef := moduleName + "." + assocName - if !patchedBSONContainsMemberAccess(t, patched, entityName, assocRef) { - t.Errorf( - "patched BSON: entity %q MemberAccesses does not contain Association=%q.\n"+ - "The association was reported as a change but wasn't written to the BSON.", - entityName, assocRef, - ) - } -} - -// TestPatchReconcile_ExistingAttrs_AddsNewAssociation verifies that when an -// entity already has explicit attribute MemberAccesses the reconciler appends -// the missing association and reports it in the changes list. -func TestPatchReconcile_ExistingAttrs_AddsNewAssociation(t *testing.T) { - const moduleName = "TestModule" - const entityName = "Attachment" - const entityID = "entity-attachment" - const attrName = "FileType" - const assocName = "Attachment_FileMetadata" - - existingMA := bson.A{ - bson.D{ - {Key: "$ID", Value: "ma-001"}, - {Key: "$Type", Value: "DomainModels$MemberAccess"}, - {Key: "AccessRights", Value: "ReadWrite"}, - {Key: "Attribute", Value: moduleName + "." + entityName + "." + attrName}, - {Key: "Association", Value: ""}, - }, - } - - raw := buildTestDomainModel( - entityName, entityID, - []string{attrName}, - existingMA, - []bson.D{assocDoc(assocName, entityID)}, - nil, - ) - - patched, changes, err := PatchReconcileMemberAccesses(raw, moduleName) - if err != nil { - t.Fatalf("PatchReconcileMemberAccesses returned error: %v", err) - } - - // The association must be in the changes list. - found := false - for _, ch := range changes { - if ch.Entity == entityName && ch.Member == assocName { - found = true - break - } - } - if !found { - t.Errorf( - "changes missing {Entity:%q, Member:%q}; got: %v\n"+ - "Existing attribute MemberAccesses should not prevent reconciler from adding the association.", - entityName, assocName, changes, - ) - } - - assocRef := moduleName + "." + assocName - if !patchedBSONContainsMemberAccess(t, patched, entityName, assocRef) { - t.Errorf( - "patched BSON: entity %q MemberAccesses missing Association=%q", - entityName, assocRef, - ) - } - - // Existing attribute entry must be preserved. - attrRef := moduleName + "." + entityName + "." + attrName - if !patchedBSONContainsMemberAccess(t, patched, entityName, attrRef) { - t.Errorf( - "patched BSON: entity %q lost existing Attribute=%q after reconcile", - entityName, attrRef, - ) - } -} - -// TestPatchReconcile_NoAccessRules_NoChanges ensures entities without any -// AccessRules are not touched. -func TestPatchReconcile_NoAccessRules_NoChanges(t *testing.T) { - // Build a DomainModel BSON with no entities that have AccessRules. - dm := bson.D{ - {Key: "$Type", Value: "DomainModels$DomainModel"}, - {Key: "Entities", Value: bson.A{int32(3)}}, - {Key: "Associations", Value: bson.A{int32(3)}}, - {Key: "CrossAssociations", Value: bson.A{int32(3)}}, - } - raw, _ := bson.Marshal(dm) - - _, changes, err := PatchReconcileMemberAccesses(raw, "TestModule") - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if len(changes) != 0 { - t.Errorf("expected 0 changes for empty domain model, got %d: %v", len(changes), changes) - } -} - -// patchedBSONContainsMemberAccess is a test helper that walks the patched BSON -// and returns true when the named entity has a MemberAccess referencing ref -// (as either Attribute or Association). -func patchedBSONContainsMemberAccess(t *testing.T, patched []byte, entityName, ref string) bool { - t.Helper() - var doc bson.D - if err := bson.Unmarshal(patched, &doc); err != nil { - t.Fatalf("unmarshal patched bytes: %v", err) - } - for _, top := range doc { - if top.Key != "Entities" { - continue - } - entities, ok := top.Value.(bson.A) - if !ok { - continue - } - for _, entItem := range entities { - entDoc, ok := entItem.(bson.D) - if !ok { - continue - } - name := "" - for _, f := range entDoc { - if f.Key == "Name" { - name, _ = f.Value.(string) - break - } - } - if name != entityName { - continue - } - for _, f := range entDoc { - if f.Key != "AccessRules" { - continue - } - rules, ok := f.Value.(bson.A) - if !ok { - continue - } - for _, ruleItem := range rules { - ruleDoc, ok := ruleItem.(bson.D) - if !ok { - continue - } - for _, rf := range ruleDoc { - if rf.Key != "MemberAccesses" { - continue - } - mas, ok := rf.Value.(bson.A) - if !ok { - continue - } - for _, maItem := range mas { - maDoc, ok := maItem.(bson.D) - if !ok { - continue - } - for _, mf := range maDoc { - if (mf.Key == "Attribute" || mf.Key == "Association") && mf.Value == ref { - return true - } - } - } - } - } - } - } - } - return false -} - -// TestPatchReconcile_CrossAssoc_AddedToMemberAccesses verifies that -// PatchReconcileMemberAccesses correctly handles cross-module associations. -// -// Cross-module associations (CrossAssociation BSON type, stored in the -// CrossAssociations array of the DomainModel) owned by an entity ARE included -// in MemberAccesses — Studio Pro includes them when running "Update Security." -// -// NOTE: The original hypothesis that CrossAssociations should be excluded was -// wrong. PayerRegistration entities have CrossAssociation MAs and no CE0066. -// CE0066 in Common_Utils has a different root cause (internal Studio Pro -// security stamp that mxcli cannot replicate). -func TestPatchReconcile_CrossAssoc_AddedToMemberAccesses(t *testing.T) { - const moduleName = "Common_Utils" - const entityName = "MySetting" - const entityID = "entity-mysetting" - const crossAssocName = "MySetting_Account" // cross-module: FROM MySetting TO Administration.Account - - // Scenario A: entity owns a CrossAssociation with empty MemberAccesses — must ADD it. - t.Run("cross_assoc_added_when_missing", func(t *testing.T) { - raw := buildTestDomainModel( - entityName, entityID, - []string{"Theme"}, // one attribute - nil, // empty MemberAccesses - nil, // no regular associations - []bson.D{crossAssocDoc(crossAssocName, entityID)}, // one cross assoc - ) - - patched, changes, err := PatchReconcileMemberAccesses(raw, moduleName) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - - // Must report a change for the cross assoc. - found := false - for _, ch := range changes { - if ch.Member == crossAssocName { - found = true - break - } - } - if !found { - t.Errorf( - "expected a ReconcileChange for CrossAssociation %q but got none.\n"+ - "CrossAssociations owned by an entity ARE included in MemberAccesses "+ - "(Studio Pro includes them in 'Update Security').\n"+ - "changes: %v", - crossAssocName, changes, - ) - } - - // Must appear in patched BSON. - assocRef := moduleName + "." + crossAssocName - if !patchedBSONContainsMemberAccess(t, patched, entityName, assocRef) { - t.Errorf( - "patched BSON missing MemberAccess for CrossAssociation %q", - assocRef, - ) - } - }) - - // Scenario B: existing MemberAccess for CrossAssociation is preserved. - t.Run("cross_assoc_preserved_when_existing", func(t *testing.T) { - existingCrossMA := bson.A{ - bson.D{ - {Key: "$ID", Value: "existing-ma-id"}, - {Key: "$Type", Value: "DomainModels$MemberAccess"}, - {Key: "AccessRights", Value: "ReadWrite"}, - {Key: "Association", Value: moduleName + "." + crossAssocName}, - {Key: "Attribute", Value: ""}, - }, - } - - raw := buildTestDomainModel( - entityName, entityID, - []string{"Theme"}, - existingCrossMA, - nil, - []bson.D{crossAssocDoc(crossAssocName, entityID)}, - ) - - patched, _, err := PatchReconcileMemberAccesses(raw, moduleName) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - - assocRef := moduleName + "." + crossAssocName - if !patchedBSONContainsMemberAccess(t, patched, entityName, assocRef) { - t.Errorf( - "patched BSON lost existing MemberAccess for CrossAssociation %q — it should be preserved", - assocRef, - ) - } - }) -} diff --git a/scripts/check-skill-mdl-semantics.sh b/scripts/check-skill-mdl-semantics.sh new file mode 100755 index 0000000000..549e3d8dbc --- /dev/null +++ b/scripts/check-skill-mdl-semantics.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# check-skill-mdl-semantics.sh — run every complete ```mdl / ```sql block in the +# given docs through the full `mxcli check`, and fail on any error it reports. +# +# check-skill-mdl.sh is the syntax gate: it splits blocks into statements and +# fails only on parse errors, on purpose, because a single statement lifted out +# of its block has lost its context. That left a hole: a whole example that +# parses and that `mxcli check` still rejects — `$response` created twice +# (MDL063, CE0111 at build), a view entity declaring String(400) over an OQL +# expression that returns String(200) (MDL031) — shipped marked as correct, and +# was found by a user who ran the pack through `mxcli check` themselves +# (mendixlabs/mxcli#1357). This gate runs that same check. +# +# The unit is the whole block, never a statement, so a variable declared in one +# statement and used in the next keeps its context. No project is loaded, so +# reference checks do not run: this is what `mxcli check script.mdl` alone +# reports, which is what a reader copying the example sees first. +# +# Skipped, because they are not complete scripts: +# - a block that does not parse (MDL-SYNTAX) — a fragment, such as microflow +# activities or an ALTER PAGE operation; check-skill-mdl.sh owns syntax; +# - a ```sql block holding real SQL or raw OQL, which check refuses as +# "produced no statements"; +# - a block marked as deliberately wrong (❌, -- BAD, INCORRECT, WRONG) or +# opted out with `-- check-skip`. +# Ignored rules: +# - MDL-DUPDEF: a block showing alternatives of one statement side by side +# (the same demo user with and without `Entity:`) defines the name twice by +# design. Nobody runs such a block as one script. +# +# Usage: scripts/check-skill-mdl-semantics.sh [mxcli-binary] [docs-dir-or-file]... +set -uo pipefail + +MXCLI="${1:-bin/mxcli}" +shift || true +[ "$#" -gt 0 ] || set -- .claude/skills/mendix + +if [ ! -x "$MXCLI" ]; then + echo "error: mxcli binary not found or not executable: $MXCLI (run 'make build')" >&2 + exit 2 +fi +command -v jq >/dev/null || { echo "error: jq is required" >&2; exit 2; } + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +FAILED=0 +CHECKED=0 +SKIPPED=0 + +while IFS= read -r md; do + rm -f "$WORK"/block_* 2>/dev/null || true + # One file per block, named after the line its body starts on. + awk -v dir="$WORK" ' + /^```(mdl|sql)[ \t]*$/ { inb=1; fn=sprintf("%s/block_%06d.mdl", dir, NR+1); next } + inb && /^```/ { inb=0; next } + inb { print > fn } + ' "$md" + + for blk in "$WORK"/block_*.mdl; do + [ -e "$blk" ] || continue + line="$(basename "$blk" .mdl)"; line=$((10#${line#block_})) + + if grep -qE '❌|--[[:space:]]*BAD\b|\bINCORRECT\b|\bWRONG\b' "$blk" || + grep -qi 'check-skip' "$blk"; then + SKIPPED=$((SKIPPED + 1)); continue + fi + + json="$("$MXCLI" check --json "$blk" 2>"$WORK/stderr")" + if ! printf '%s' "$json" | jq -e . >/dev/null 2>&1; then + # Real SQL or raw OQL in a ```sql fence: not MDL at all, and check + # says so instead of reporting. Anything else without a report (a + # crash) is a failure. + if grep -q 'produced no statements' "$WORK/stderr"; then + SKIPPED=$((SKIPPED + 1)); continue + fi + FAILED=1 + echo "FAIL: $md:$line — mxcli check produced no JSON report" + continue + fi + if printf '%s' "$json" | jq -e '[.violations[]? | select(.ruleId == "MDL-SYNTAX")] | length > 0' >/dev/null; then + SKIPPED=$((SKIPPED + 1)); continue + fi + + CHECKED=$((CHECKED + 1)) + errs="$(printf '%s' "$json" | jq -r '.violations[]? + | select(.severity == "error" and .ruleId != "MDL-DUPDEF") + | " [\(.ruleId)] \(.message)"')" + if [ -n "$errs" ]; then + FAILED=1 + echo "FAIL: $md:$line" + printf '%s\n' "$errs" + fi + done +done < <(find "$@" -name '*.md' | sort) + +echo "---" +echo "Complete MDL blocks: checked $CHECKED, skipped $SKIPPED (fragments/illustrative)" +if [ "$FAILED" -ne 0 ]; then + echo "Some complete MDL examples fail 'mxcli check' (see above)." + exit 1 +fi +echo "All complete MDL examples pass 'mxcli check'." diff --git a/sdk/domainmodel/domainmodel.go b/sdk/domainmodel/domainmodel.go index ed61aefd99..de9a9529aa 100644 --- a/sdk/domainmodel/domainmodel.go +++ b/sdk/domainmodel/domainmodel.go @@ -456,6 +456,16 @@ type DeleteBehavior struct { // null message stops the RUNTIME STARTING, which is a worse failure than a // build error and names nothing about the model (CapTrackV2 §1). ErrorMessage string `json:"errorMessage,omitempty"` + // ErrorMessageLanguage is the LanguageCode a write stores ErrorMessage + // under. The executor sets the project's default language, as for every + // other text it authors (mendixlabs/mxcli#970, #1344); empty means + // model.AuthoringLanguage, which is en_US until a project's settings are read. + ErrorMessageLanguage string `json:"errorMessageLanguage,omitempty"` + // ErrorMessageTranslations holds every translation the stored message + // carries, keyed by LanguageCode. Read side only: ErrorMessage alone cannot + // say which language it came from, so DESCRIBE could not pick the project's + // default language out of a text that has several. + ErrorMessageTranslations map[string]string `json:"-"` } // DeleteBehaviorType represents the type of delete behavior.