diff --git a/CHANGELOG.md b/CHANGELOG.md index b7202542..fc077b6d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- fix(app-restart): **a secret with spaces in quotes is masked whole, and `Pwd` and `Credential` join the secret words.** Follow-up to #403: `--constant "Mod.ApiKey=two words"` and `--db-password 'two words'` printed everything after the first word (the quoted-flag case leaked before #403 too). `mask()` now masks each argument before quoting it for display, so the whole quoted value goes; the executed command is untouched, and a real-launch test proves the started app receives the raw argv; an env assignment at the start of a `--cmd` string keeps the whole command visible; the per-argument seds only run for arguments holding a secret word. The substring match still over-masks names like `Monkey` (Key) and `Turkey`; that is display only, kept, and pinned by a test. `tests/wave2/test-app-restart.sh` gains 22 cases, 8 of them failing on master. — review of #403 - fix(install-manifest): **a clean toolkit checkout passes its own `project-bin/` manifest self-check, so adopters no longer see the "named by NEITHER list" warning at the top of every sync.** `coverage-check-all.sh` (found by `review-module.sh` through its `_tool` lookup in the toolkit clone) and `ts-sync.sh` (UNPROVEN and SUPERSEDED per its header) join `MXTK_PROJECT_BIN_NOINSTALL` with reasons. New `tests/wave2/test-manifest-clean-checkout.sh` fails, rather than warns, when a `project-bin/` file is in neither list. — joshuamoesa (#381) - fix(sync-project): **the "Baseline-only skills cited" warning no longer fires on the toolkit's own wiring block, and no longer sizes a section that does not exist.** The check grepped all of `CLAUDE.md` for `skills/learned-mdl-preflight.md` / `skills/query-the-model.md`, and `wire-agents.sh`'s `mxtk:wiring` block cites the first, so every sync flagged the toolkit's own text; with no "mxcli-project-toolkit Integration" heading it then printed a whole-file word count as "heading to EOF". The wiring block is now stripped before the grep, and the warning is raised only when the heading exists. `tests/wave2/test-sync-dup-heading-exit.sh` T4 flips (no heading: no warning) and gains T7-T9. — joshuamoesa (#382) - learn(workflow, rest): **the v0.20-era workflow notes are marked superseded for mxcli v0.26.0:** `WAIT FOR NOTIFICATION` takes a name, `NOTIFY WORKFLOW` needs `TARGET Module.Workflow.Element` (CE0166 / MDL-WF16 without), mxcli writes the end-of-boundary-path marker itself so a boundary terminator is no longer a hand-add (the parallel-split terminator is a separate node and unchanged), and published REST `Authentication: (basic)` works (BUG-152 item 1: 401 / 401 / 200 proven, URL includes the resource segment). Sources: a five-app lab field run (Mendix 11.15, mxcli v0.26.0) and mxcli's public `write-workflows` skill and docs. `wf-add-path-terminators.py` already takes an `.mpr` since #294 (the reported crash was the older script). — TK-019 of a five-app lab report diff --git a/project-bin/app-restart.sh b/project-bin/app-restart.sh index dbf84ae9..46900b92 100755 --- a/project-bin/app-restart.sh +++ b/project-bin/app-restart.sh @@ -131,16 +131,55 @@ bundle_ok() { # $1 port — the script bundle index.html loads must answer 2xx case "$code" in 2??) return 0 ;; esac BUNDLE="$src → HTTP ${code:-none}"; return 1 } +mask_has_secret() { # pure bash glob: does $1 hold a secret word? (no fork) + case "$1" in + *[Pp][Aa][Ss][Ss]*|*[Pp][Ww][Dd]*|*[Ss][Ee][Cc][Rr][Ee][Tt]*|*[Tt][Oo][Kk][Ee][Nn]*|*[Kk][Ee][Yy]*|*[Cc][Rr][Ee][Dd][Ee][Nn][Tt][Ii][Aa][Ll]*) return 0 ;; + esac + return 1 +} mask() { # print argv joined, with the value after any password/secret/token/key flag masked - local a out="" - for a in "$@"; do case "$a" in *" "*) a="'$a'" ;; esac; out="$out $a"; done - # Two passes (TK-003): (1) a secret-named FLAG masks its value, as `--db-password v` or - # `--db-password=v`; (2) ANY `Name=value` whose Name holds a secret word masks the value, which - # is how `--constant Module.HubApiPassword=v` (flag name `--constant`) used to print in clear. - local _w='([Pp][Aa][Ss][Ss]|[Ss][Ee][Cc][Rr][Ee][Tt]|[Tt][Oo][Kk][Ee][Nn]|[Kk][Ee][Yy])' - printf '%s\n' "${out# }" | sed -E \ - -e "s/(-[-A-Za-z_]*${_w}[-A-Za-z_]*)(=| +)[^ ']+/\1\3****/g" \ - -e "s/([A-Za-z0-9_.-]*${_w}[A-Za-z0-9_.-]*)=[^ ']+/\1=****/g" + # Display only (TK-003): the executed ARGV is never touched. Each argument is masked on its own, + # BEFORE it is quoted for display, so a quoted secret with spaces is hidden whole. Passes: + # (0) the argument after a secret-named flag is a secret, whole; (1) the argument IS `Name=value` + # with a secret Name: everything after `=` is masked, spaces included (the argv form of + # `--constant "Mod.ApiKey=two words"`); (2) inside a shell string (`--cmd '...'`) a quoted + # `"Name=value with spaces"` / `'Name=...'`; (3) a secret-named FLAG masks its value, as + # `--db-password v`, `--db-password=v`, `--db-password "two words"`; (4) ANY `Name=value` whose Name + # holds a secret word masks the value, quoted or not. Known over-mask, kept: the word match is a + # substring, so `Monkey` (Key), `Turkey` (Key) and `Bypass` (Pass) mask too. Display only, harmless. + local _w='([Pp][Aa][Ss][Ss]|[Pp][Ww][Dd]|[Ss][Ee][Cc][Rr][Ee][Tt]|[Tt][Oo][Kk][Ee][Nn]|[Kk][Ee][Yy]|[Cc][Rr][Ee][Dd][Ee][Nn][Tt][Ii][Aa][Ll])' + local _v="\"[^\"]*\"|'[^']*'|[^ ]+" + # Speed: the sed forks only run for an argument that holds a secret word (a bash glob decides), + # so a typical argv costs no fork at all. The argument after `-c` is a shell string: pass (1) is + # skipped for it, so `bash -c 'DB_PASSWORD=x ./mxcli run ...'` keeps the whole command visible. + local a out="" prev_secret=0 prev_c=0 p1 + for a in "$@"; do + p1='s/^$//' + if [ "$prev_secret" = 1 ]; then a="****"; prev_secret=0; prev_c=0 + else + if [ "$prev_c" = 1 ]; then prev_c=0 + else + p1="s/^([A-Za-z0-9_.-]*${_w}[A-Za-z0-9_.-]*)=.*$/\\1=****/" + case "$a" in + -c) prev_c=1 ;; + -*" "*|-*=*) ;; # flag=value or a shell string: sed below + -*[!-A-Za-z_]*) ;; + -*) mask_has_secret "$a" && prev_secret=1 ;; + esac + fi + if mask_has_secret "$a"; then + a="$(printf '%s' "$a" | sed -E \ + -e "$p1" \ + -e "s/\"([A-Za-z0-9_.-]*${_w}[A-Za-z0-9_.-]*)=[^\"]*\"/\"\\1=****\"/g" \ + -e "s/'([A-Za-z0-9_.-]*${_w}[A-Za-z0-9_.-]*)=[^']*'/'\\1=****'/g" \ + -e "s/(-[-A-Za-z_]*${_w}[-A-Za-z_]*)(=| +)(${_v})/\\1\\3****/g" \ + -e "s/([A-Za-z0-9_.-]*${_w}[A-Za-z0-9_.-]*)=(${_v})/\\1=****/g")" + fi + fi + case "$a" in *" "*) a="'$a'" ;; esac + out="$out $a" + done + printf '%s\n' "${out# }" } # Port and process helpers (mxtk_port_holder, mxtk_holder_text, mxtk_pid_in_tree) live in diff --git a/tests/wave2/test-app-restart.sh b/tests/wave2/test-app-restart.sh index bfed6027..c1261b59 100755 --- a/tests/wave2/test-app-restart.sh +++ b/tests/wave2/test-app-restart.sh @@ -75,6 +75,7 @@ mkapp() { # $1 = project dir, $2 = port const fs = require('fs'), http = require('http'), path = require('path'), cp = require('child_process'); const dir = process.cwd(), hs = path.join(dir, '.mxcli', 'run-local.json'); const port = parseInt(fs.readFileSync(path.join(dir, '.fakeport'), 'utf8'), 10); +fs.writeFileSync(path.join(dir, '.fakeargv'), process.argv.slice(2).join('\n') + '\n'); // what the launch really passed const child = cp.spawn('sleep', ['300'], { stdio: 'ignore' }); fs.writeFileSync(path.join(dir, '.fakechild'), String(child.pid)); http.createServer((req, res) => { @@ -273,6 +274,41 @@ hasnt "a Token-named constant is masked" "$OUT" "tokB" hasnt "--db-password and --db-password= are masked" "$OUT" "hunter3" hasnt "the = form is masked" "$OUT" "hunter4" has "a non-secret constant stays readable" "$OUT" "Mod.Port=8080" +echo "-- secrets masked: quoted values with spaces, wider word list (follow-up to #403)" +OUT="$(restart "$P" --dry-run --cmd './mxcli run --local --constant "Mod.ApiKey=two words" --db-password '"'"'zqx pw9'"'"' --constant Mod.Pwd=pwdv --constant Mod.Credential=credv --constant Mod.Passphrase=ppv --constant Mod.ClientSecret=csv -p Fixture.mpr')" +hasnt "a double-quoted Name=value with spaces is masked whole (first word)" "$OUT" "two" +hasnt "a double-quoted Name=value with spaces is masked whole (tail)" "$OUT" "words" +hasnt "--db-password 'two words' is masked whole (first word)" "$OUT" "zqx" +hasnt "--db-password 'two words' is masked whole (tail)" "$OUT" "pw9" +hasnt "a Pwd-named constant is masked" "$OUT" "pwdv" +hasnt "a Credential-named constant is masked" "$OUT" "credv" +hasnt "a Passphrase-named constant is masked" "$OUT" "ppv" +hasnt "a ClientSecret-named constant is masked" "$OUT" "csv" +has "the rest of the command stays readable" "$OUT" "-p Fixture.mpr" +has "a Name is kept, only the value masked" "$OUT" "Mod.ApiKey=" +# Known over-mask, pinned so it does not get worse unnoticed: a substring match on Key. +OUT="$(restart "$P" --dry-run --cmd './mxcli run --local --constant Mod.Monkey=1 --constant Mod.Port=8080 -p Fixture.mpr')" +hasnt "known over-mask: Monkey matches Key" "$OUT" "Mod.Monkey=1" +has "a name with no secret word stays readable" "$OUT" "Mod.Port=8080" +echo "-- the masked display never changes the executed command (real launch path)" +CUR="$(hs_pid "$P")"; KILL_LIST="$KILL_LIST $CUR $(cat "$P/.fakechild" 2>/dev/null)" +OUT="$(restart "$P" --timeout 30 --cmd './mxcli run --local --constant "Mod.ApiKey=two words" --db-password '"'"'zqx pw9'"'"' --constant Mod.Pwd=pwdv -p Fixture.mpr')"; RC=$? +KILL_LIST="$KILL_LIST $(hs_pid "$P") $(cat "$P/.fakechild" 2>/dev/null)" +[ "$RC" -eq 0 ] && ok "the restart with secrets in --cmd succeeded" || bad "exit $RC: $(printf '%s' "$OUT" | tail -4)" +EXPECT_ARGV="$(printf '%s\n' run --local --constant 'Mod.ApiKey=two words' --db-password 'zqx pw9' --constant Mod.Pwd=pwdv -p Fixture.mpr)" +[ "$(cat "$P/.fakeargv" 2>/dev/null)" = "$EXPECT_ARGV" ] && ok "the started app received the raw argv, byte-identical" || bad "started app argv differs: $(cat "$P/.fakeargv" 2>/dev/null)" +hasnt "the printed output hides the spaced secret" "$OUT" "words" +hasnt "the printed output hides the --db-password value" "$OUT" "zqx" +case "$(cat "$P/.claude/loop/app-restart/start.cmd" 2>/dev/null)" in *"Mod.ApiKey=two words"*zqx\ pw9*) ok "start.cmd remembers the raw secrets" ;; *) bad "start.cmd lost the raw command" ;; esac +echo "-- env assignment at the start of a --cmd string keeps the whole command visible" +OUT="$(restart "$P" --dry-run --cmd 'DB_PASSWORD=envsecret ./mxcli run --local --watch -p Fixture.mpr')" +hasnt "the env secret is masked" "$OUT" "envsecret" +has "the whole command stays visible, quoted" "$OUT" "bash -c 'DB_PASSWORD=**** ./mxcli run --local --watch -p Fixture.mpr'" +OUT="$(restart "$P" --dry-run --cmd 'DB_PASSWORD=envsecret echo hi')" +hasnt "[refuse path] the env secret is masked" "$OUT" "envsecret" +has "[refuse path] the whole command stays visible" "$OUT" "bash -c 'DB_PASSWORD=**** echo hi'" +OUT="$(restart "$P" --dry-run --cmd 'echo "Mod.ApiKey=two words"')" +hasnt "[refuse path] a non-mxcli command masks a spaced secret too" "$OUT" "words" alive "$DECOY" && ok "decoy survived the whole run" || bad "decoy killed" echo "-- #160: a bin/_common.sh older than the moved helpers → exit 2 naming sync, nothing stopped"