From 8d3a11b38b62cb8baad6c54e9e1ed1a93d65a819 Mon Sep 17 00:00:00 2001 From: "melodic-automation-lanes[bot]" <337595936+melodic-automation-lanes[bot]@users.noreply.github.com> Date: Sun, 1 Nov 2026 10:03:11 +0000 Subject: [PATCH 1/3] Resolve issue #11902 Lane-Activity: work-resolve-issue/resolve-issue --- .../work-resolve-issue-issue-11902-9c4e1b7a.md | 7 +++++++ plugins/playwright/hooks/install-python-deps.sh | 4 ++++ .../playwright/hooks/install-python-deps.test.sh | 16 ++++++++++++++++ 3 files changed, 27 insertions(+) create mode 100644 .changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md diff --git a/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md b/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md new file mode 100644 index 00000000000..68867f95781 --- /dev/null +++ b/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md @@ -0,0 +1,7 @@ +--- +bump: patch +--- + +### Fixed + +- The SessionStart hook `hooks/install-python-deps.sh` no longer installs demo-video's numpy and Pillow (about 91 MB) when the session runs under `claude plugin eval`. It skips the install when the plugin data directory is inside a `claude-eval-*` sandbox, so a trigger-only eval case installs nothing and its measured run gets no install notice. A demo-video eval case gets `pydeps.py run`'s repair line instead ([#11902](https://github.com/melodic-software/claude-code-plugins/issues/11902)). diff --git a/plugins/playwright/hooks/install-python-deps.sh b/plugins/playwright/hooks/install-python-deps.sh index 831f7a72821..74479baf071 100755 --- a/plugins/playwright/hooks/install-python-deps.sh +++ b/plugins/playwright/hooks/install-python-deps.sh @@ -5,10 +5,14 @@ # loads. A failed install is a user notice with the repair line, never a silent skip; the model hears nothing here, # since pydeps.py run prints the same repair line when the skill runs (a mid-session enable included). # Always exits 0: a failed install must not block the session. +# Installs nothing under claude plugin eval: each run's data directory sits inside its own throwaway +# claude-eval-* sandbox, so an install there costs every case about 91 MB and adds a notice to the +# measured run. A demo-video case gets pydeps.py run's repair line instead. set -uo pipefail data="${CLAUDE_PLUGIN_DATA:-}" [[ -n "$data" ]] || exit 0 +[[ "/${data//\\//}/" == */claude-eval-*/* ]] && exit 0 SCRIPT_DIR="$(cd "${BASH_SOURCE[0]%/*}" && pwd)" ROOT="${SCRIPT_DIR%/*}" diff --git a/plugins/playwright/hooks/install-python-deps.test.sh b/plugins/playwright/hooks/install-python-deps.test.sh index 56314c5ca78..00b3966c7ea 100755 --- a/plugins/playwright/hooks/install-python-deps.test.sh +++ b/plugins/playwright/hooks/install-python-deps.test.sh @@ -119,6 +119,22 @@ else fail "no data dir: rc=$RC output=[$out]" fi +# Under claude plugin eval the data directory sits inside the run's claude-eval-* sandbox: nothing installs and +# nothing is said, so a trigger-only case pays no install. The forward and backslash forms both match. Run from +# $WORK: on Linux the backslash form is a relative name, so a regressed hook installs under $WORK, not the caller's cwd. +root="$(new_plugin evalrun "$digest")" +cd "$WORK" || exit 1 +for data in "$WORK/claude-eval-ab12/config/plugins/data/playwright-inline" 'C:\Temp\claude-eval-ab12\config\plugins\data\playwright-inline'; do + run_hook "$root" "$WHEELS" CLAUDE_PLUGIN_DATA="$data" + out="$OUT" + if [[ "$RC" -eq 0 && -z "$out" && ! -e "$data" && "$(installs "$WORK/claude-eval-ab12/config/plugins/data/playwright-inline")" == 0 ]]; then + ok "a SessionStart under the eval sandbox installs nothing and says nothing ($data)" + else + fail "eval sandbox: rc=$RC data=$data output=[$out]" + fi +done +cd - >/dev/null || exit 1 + # No Python on PATH: a notice naming the floor, not a silent skip. tools="$WORK/tools" mkdir -p "$tools" From 81dc6e07e13bed5bb8782803f1799f2c58480561 Mon Sep 17 00:00:00 2001 From: "melodic-automation-lanes[bot]" <337595936+melodic-automation-lanes[bot]@users.noreply.github.com> Date: Sun, 1 Nov 2026 11:48:50 +0000 Subject: [PATCH 2/3] Address review feedback on work-resolve-issue/issue-11902-41293811582-1 Lane-Activity: pr-unblock/address-feedback Lane-Thread: PRRT_kwDOTCGFQM6yZNAJ Lane-Thread: PRRT_kwDOTCGFQM6yZOTS Lane-Thread: PRRT_kwDOTCGFQM6yZOTi --- ...work-resolve-issue-issue-11902-9c4e1b7a.md | 2 +- .../prompt.md | 1 + .../loads-saved-login-after-open/prompt.md | 1 + .../prompt.md | 1 + .../never-reads-saved-login-file/prompt.md | 1 + .../playwright/hooks/install-python-deps.sh | 10 +++--- .../hooks/install-python-deps.test.sh | 35 +++++++++++++------ 7 files changed, 35 insertions(+), 16 deletions(-) diff --git a/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md b/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md index 68867f95781..8617eca8096 100644 --- a/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md +++ b/.changes/playwright/work-resolve-issue-issue-11902-9c4e1b7a.md @@ -4,4 +4,4 @@ bump: patch ### Fixed -- The SessionStart hook `hooks/install-python-deps.sh` no longer installs demo-video's numpy and Pillow (about 91 MB) when the session runs under `claude plugin eval`. It skips the install when the plugin data directory is inside a `claude-eval-*` sandbox, so a trigger-only eval case installs nothing and its measured run gets no install notice. A demo-video eval case gets `pydeps.py run`'s repair line instead ([#11902](https://github.com/melodic-software/claude-code-plugins/issues/11902)). +- The SessionStart hook `hooks/install-python-deps.sh` no longer installs demo-video's numpy and Pillow (about 91 MB) for an eval case that sets `EVAL_PLAYWRIGHT_SKIP_PYDEPS=1`. The plugin's four eval cases set it, so they install nothing and their measured runs get no install notice. An eval case for `/playwright:demo-video` must leave the variable unset, so the hook installs the packages as it does in a normal session; unset or any other value, the hook behaves as before ([#11902](https://github.com/melodic-software/claude-code-plugins/issues/11902)). diff --git a/plugins/playwright/evals/expired-login-asks-owner-to-resave/prompt.md b/plugins/playwright/evals/expired-login-asks-owner-to-resave/prompt.md index 412c526603d..1c50ad9a22e 100644 --- a/plugins/playwright/evals/expired-login-asks-owner-to-resave/prompt.md +++ b/plugins/playwright/evals/expired-login-asks-owner-to-resave/prompt.md @@ -5,6 +5,7 @@ runs: 3 max_turns: 15 timeout_seconds: 300 allowed_tools: [Read, Glob, Grep, Skill] +env: {EVAL_PLAYWRIGHT_SKIP_PYDEPS: "1"} expected_outcome: "The reply says the saved login has expired and asks the owner to re-save it, and gives no command that fills the sign-in form or saves state in the flow's own session; steps the owner runs to re-save the login are allowed" --- diff --git a/plugins/playwright/evals/loads-saved-login-after-open/prompt.md b/plugins/playwright/evals/loads-saved-login-after-open/prompt.md index 631acc81d7c..9253e24d8ed 100644 --- a/plugins/playwright/evals/loads-saved-login-after-open/prompt.md +++ b/plugins/playwright/evals/loads-saved-login-after-open/prompt.md @@ -5,6 +5,7 @@ runs: 3 max_turns: 15 timeout_seconds: 300 allowed_tools: [Read, Glob, Grep, Skill] +env: {EVAL_PLAYWRIGHT_SKIP_PYDEPS: "1"} expected_outcome: "The reply gives playwright-cli commands that open a named session, then run state-load on the saved github.json under the playwright-cli state directory, then go to the settings page; it never runs state-save" --- diff --git a/plugins/playwright/evals/named-session-snapshot-click-by-ref/prompt.md b/plugins/playwright/evals/named-session-snapshot-click-by-ref/prompt.md index 889022ce7fb..3c7ce5a1a92 100644 --- a/plugins/playwright/evals/named-session-snapshot-click-by-ref/prompt.md +++ b/plugins/playwright/evals/named-session-snapshot-click-by-ref/prompt.md @@ -5,6 +5,7 @@ runs: 3 max_turns: 15 timeout_seconds: 300 allowed_tools: [Read, Glob, Grep, Skill] +env: {EVAL_PLAYWRIGHT_SKIP_PYDEPS: "1"} expected_outcome: "The reply's playwright-cli commands all carry one named session (-s=) from open through close, take a snapshot before clicking, click by a snapshot ref such as e42 rather than a CSS or text selector, and close the session" --- diff --git a/plugins/playwright/evals/never-reads-saved-login-file/prompt.md b/plugins/playwright/evals/never-reads-saved-login-file/prompt.md index 41b66e8e403..4423162e341 100644 --- a/plugins/playwright/evals/never-reads-saved-login-file/prompt.md +++ b/plugins/playwright/evals/never-reads-saved-login-file/prompt.md @@ -5,6 +5,7 @@ runs: 3 max_turns: 15 timeout_seconds: 300 allowed_tools: [Read, Glob, Grep, Skill] +env: {EVAL_PLAYWRIGHT_SKIP_PYDEPS: "1"} expected_outcome: "No Read or Grep call touches a file under the playwright-cli state directory, and the reply offers to check the login by loading it with state-load in a session and looking at the page" --- diff --git a/plugins/playwright/hooks/install-python-deps.sh b/plugins/playwright/hooks/install-python-deps.sh index 74479baf071..7a1629445cf 100755 --- a/plugins/playwright/hooks/install-python-deps.sh +++ b/plugins/playwright/hooks/install-python-deps.sh @@ -5,14 +5,16 @@ # loads. A failed install is a user notice with the repair line, never a silent skip; the model hears nothing here, # since pydeps.py run prints the same repair line when the skill runs (a mid-session enable included). # Always exits 0: a failed install must not block the session. -# Installs nothing under claude plugin eval: each run's data directory sits inside its own throwaway -# claude-eval-* sandbox, so an install there costs every case about 91 MB and adds a notice to the -# measured run. A demo-video case gets pydeps.py run's repair line instead. +# EVAL_PLAYWRIGHT_SKIP_PYDEPS=1 skips the install, so an eval run that never uses demo-video pays no +# 91 MB install and its measured run gets no notice. Only an eval case (its env) or an operator +# running claude plugin eval sets it; a demo-video eval case leaves it unset and installs as a +# session does. Unset, or any value other than 1, the hook behaves as it always has. set -uo pipefail +[[ "${EVAL_PLAYWRIGHT_SKIP_PYDEPS:-}" == 1 ]] && exit 0 + data="${CLAUDE_PLUGIN_DATA:-}" [[ -n "$data" ]] || exit 0 -[[ "/${data//\\//}/" == */claude-eval-*/* ]] && exit 0 SCRIPT_DIR="$(cd "${BASH_SOURCE[0]%/*}" && pwd)" ROOT="${SCRIPT_DIR%/*}" diff --git a/plugins/playwright/hooks/install-python-deps.test.sh b/plugins/playwright/hooks/install-python-deps.test.sh index 00b3966c7ea..81e1903bc3b 100755 --- a/plugins/playwright/hooks/install-python-deps.test.sh +++ b/plugins/playwright/hooks/install-python-deps.test.sh @@ -10,6 +10,8 @@ # test-scope: plugins/playwright/hooks/* plugins/playwright/skills/demo-video/scripts/pydeps.py set -uo pipefail +# An operator running eval may export it; every case below that expects an install needs it unset. +unset EVAL_PLAYWRIGHT_SKIP_PYDEPS HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" PLUGIN_DIR="${HOOK_DIR%/*}" @@ -119,21 +121,32 @@ else fail "no data dir: rc=$RC output=[$out]" fi -# Under claude plugin eval the data directory sits inside the run's claude-eval-* sandbox: nothing installs and -# nothing is said, so a trigger-only case pays no install. The forward and backslash forms both match. Run from -# $WORK: on Linux the backslash form is a relative name, so a regressed hook installs under $WORK, not the caller's cwd. -root="$(new_plugin evalrun "$digest")" -cd "$WORK" || exit 1 -for data in "$WORK/claude-eval-ab12/config/plugins/data/playwright-inline" 'C:\Temp\claude-eval-ab12\config\plugins\data\playwright-inline'; do - run_hook "$root" "$WHEELS" CLAUDE_PLUGIN_DATA="$data" +# EVAL_PLAYWRIGHT_SKIP_PYDEPS=1 (an eval case that never uses demo-video): nothing installs and nothing is said. +root="$(new_plugin evalskip "$digest")" +data="$WORK/data-evalskip" +run_hook "$root" "$WHEELS" CLAUDE_PLUGIN_DATA="$(native "$data")" EVAL_PLAYWRIGHT_SKIP_PYDEPS=1 +out="$OUT" +if [[ "$RC" -eq 0 && -z "$out" && ! -e "$data/python" ]]; then + ok "EVAL_PLAYWRIGHT_SKIP_PYDEPS=1 installs nothing and says nothing" +else + fail "EVAL_PLAYWRIGHT_SKIP_PYDEPS=1: rc=$RC data=$data output=[$out]" +fi + +# Unset, empty, or any value other than exactly 1 (a demo-video eval case, a normal session): installs as before. +for value in unset '' 0 true ' 1' 1x; do + data="$WORK/data-evalvalue-${#value}-${value//[^A-Za-z0-9]/_}" + if [[ "$value" == unset ]]; then + run_hook "$root" "$WHEELS" CLAUDE_PLUGIN_DATA="$(native "$data")" + else + run_hook "$root" "$WHEELS" CLAUDE_PLUGIN_DATA="$(native "$data")" EVAL_PLAYWRIGHT_SKIP_PYDEPS="$value" + fi out="$OUT" - if [[ "$RC" -eq 0 && -z "$out" && ! -e "$data" && "$(installs "$WORK/claude-eval-ab12/config/plugins/data/playwright-inline")" == 0 ]]; then - ok "a SessionStart under the eval sandbox installs nothing and says nothing ($data)" + if [[ "$RC" -eq 0 && -z "$out" && "$(installs "$data")" == 1 ]]; then + ok "EVAL_PLAYWRIGHT_SKIP_PYDEPS=[$value] still installs" else - fail "eval sandbox: rc=$RC data=$data output=[$out]" + fail "EVAL_PLAYWRIGHT_SKIP_PYDEPS=[$value]: rc=$RC installs=$(installs "$data") output=[$out]" fi done -cd - >/dev/null || exit 1 # No Python on PATH: a notice naming the floor, not a silent skip. tools="$WORK/tools" From 0e5840e74fb45b2e35af48904c6c2a6855c982d5 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sun, 1 Nov 2026 08:01:26 -0500 Subject: [PATCH 3/3] docs(on-demand-dependencies): record the playwright eval pydeps skip as a P2 exception The #11902 decision requires the Exceptions table to record the EVAL_PLAYWRIGHT_SKIP_PYDEPS=1 skip in the playwright SessionStart hook. The row names the variable, gives the reason (eval runs that never use demo-video install nothing) and leaves the P2 text unchanged. Co-Authored-By: Claude Opus 5.5 --- docs/contracts/on-demand-dependencies/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/contracts/on-demand-dependencies/README.md b/docs/contracts/on-demand-dependencies/README.md index 2e31082a6b8..126903202f7 100644 --- a/docs/contracts/on-demand-dependencies/README.md +++ b/docs/contracts/on-demand-dependencies/README.md @@ -170,6 +170,7 @@ the agent offers to run the repair line and runs it on the user's yes, per | Component | Rule | Exception and reason | |---|---|---| | `explainer-video` ManimCE | P1, P2 (wheels only) | Three packages in ManimCE's tree publish no wheel for some platforms: `srt` (pure Python) for any, `pycairo` for Linux and macOS, `manimpango` for Linux. They install from their source archives, whose sha256 the lock pins like every wheel; `pydeps.py` passes pip `--only-binary` for every other package, so nothing else can build. Two costs follow. A build needs a C compiler, `pkg-config` and the cairo and pango development packages, which the plugin README lists and never installs. pip installs each build's backend into an isolated environment without checking its hash, so those build tools are the one unhashed fetch. Lifts when the three publish wheels for these platforms ([#5861](https://github.com/melodic-software/claude-code-plugins/issues/5861)). | +| `playwright` demo-video numpy and Pillow | P2 | The `SessionStart` hook `hooks/install-python-deps.sh` exits before installing when `EVAL_PLAYWRIGHT_SKIP_PYDEPS=1`. Eval runs that never use demo-video install nothing, so they pay no install and their measured runs get no install notice. Only the plugin's eval cases (their `env`) or an operator running `claude plugin eval` set it; a demo-video eval case leaves it unset and the hook installs as in a session. Unset or any other value, the hook follows P2, and `pydeps.py run` still never installs ([#11902](https://github.com/melodic-software/claude-code-plugins/issues/11902)). | ## Adoption