Skip to content

fix(user-interface): trigger design on single visual-value questions - #11895

Merged
kyle-sexton merged 2 commits into
mainfrom
fix/11423-round4
Nov 1, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
fix/11423-round4

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Closes #11423

Summary

The last three items on #11423. /user-interface:design now auto-invokes on a question about a single visual value, such as a nested element's corner radius or a layer's shadow, even when the question gives every number. Before this change the model answered those questions itself and loaded no skill, which is why shape-and-depth-nested-radius fired no skill on base or head. The probe sets are re-measured on current main with the four probes reworded after #11765. /user-interface:write-css gets one added trigger, chosen by a pass rule written before any result was read.

Fix

  • plugins/user-interface/skills/design/SKILL.md: "to decide how something should look" now reads "to decide how something should look, down to one corner radius or shadow, even with every number given".
  • plugins/user-interface/skills/write-css/SKILL.md: adds the quoted trigger 'fix this layout in the stylesheet'.
  • plugins/user-interface/probes/design.json: adds three shape probes (pos-train-shape-01, pos-train-shape-02, pos-validation-shape-01); one of them gives every number.
  • .changes/user-interface/fix-11423-shape-and-css-triggers-7d3a91c2.md.

Verification

Round 4 chose these two descriptions (design A3 and write-css B2) under a pass rule written before any result was read, but run-validity.py judged every round-4 probe side INVALID, so its numbers count for nothing. Round 5 re-measured the same two descriptions on runs the gate judges VALID, under a decision rule written before any round-5 run started.

Round 5 setup. claude plugin eval, 3 runs per case, opus judge, --ablation none, Claude Code 2.1.316, both sides concurrently. Base is plugins/user-interface at main c3a4498. Head is base with only the two description: lines replaced by this PR's. Probe cases: main's design.json and write-css.json, the three shape probes above, and a held-out set of 6 design and 6 write-css probes (written by a fresh agent from neutral purpose statements, not committed), emitted by measure-invocation.sh emit-plugin-eval --runs 3 and run with --disallow-tools Read Glob Grep Agent.

Gate. run-validity.py --runs 3: base probes VALID, head probes VALID (297 of 297 traces read, no errored runs, no permission denials).

group base head change
design positives, main's 60/78 (0.77) 67/78 (0.86) +9.0 points
shape positives (visible + held-out) 11/21 (0.52) 21/21 (1.00) +47.6 points
design positives, all 71/99 (0.72) 88/99 (0.89) +17.2 points
design negatives fired 9/60 9/60 0
write-css positives, main's 59/66 (0.89) 60/66 (0.91) +1.5 points
write-css positives, all 71/78 (0.91) 72/78 (0.92) +1.3 points
write-css negatives fired 0/60 0/60 0
CSS-only design near-misses fired 0/9 0/9 0

noise-report.py <head> --baseline <base> --margin 0.05: paired change +0.06 over 99 cases (95% interval +0.02 to +0.10), non-inferior.

shape-and-depth-nested-radius (the case the design change targets; 3 runs per side, no errored runs, no denials):

side design fired answer grader (border-subtracted) passed
base 0/3 3/3
head 3/3 3/3

The gate marks the named-four result files INVALID on both sides for reasons outside this measurement: its skill-fired check counts base not firing on this case, which is the behavior the change fixes, as an invalid run; base also had one denied plugin-root Glob in states-projects-list, and head one unfired accessibility-mobile-nav-drawer run. The owner decided to judge this case on its runs with no error or denial, counting unfired runs as data.

Outcome against the round-5 rule.

  • Design (A3): all five conditions pass. Nested-radius fires in 3/3 head runs with no answer drop; main's design positives +9.0 points; design negatives +0; write-css main positives +1.5 points; shape positives rise.
  • write-css (B2): all three conditions pass. write-css positives +1.3 points; design positives +17.2 points (with design conditions 1 and 5); write-css negatives +0.

measure-invocation.sh validate passes on the four probe files. check-skill.sh passes design and write-css.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits November 1, 2026 02:18
design's description now claims deciding one corner radius or shadow,
even when every number is given, so a nested-radius question routes to
it instead of being answered without the skill. write-css adds 'fix this
layout in the stylesheet' to its triggers. design.json adds shape probes.

Refs: #11423

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 1, 2026 07:26
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T07:28:20.337961Z 6798385 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 67983857dfc952f32b8f45ea5672d88981bfec0e: read 4 of 4 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #11895

No security issues found.

I read every hunk in all 4 changed files:

  • .changes/user-interface/fix-11423-shape-and-css-triggers-7d3a91c2.md: a new changelog fragment containing only prose.
  • plugins/user-interface/probes/design.json: an extended ranking_note string and three new should-trigger eval probes. These are test data for invocation measurement and are never executed.
  • plugins/user-interface/skills/design/SKILL.md: the frontmatter description gains "down to one corner radius or shadow, even with every number given". No other frontmatter key changes. user-invocable and disable-model-invocation stay the same, and no tool grant changes.
  • plugins/user-interface/skills/write-css/SKILL.md: the frontmatter description gains one quoted trigger, 'fix this layout in the stylesheet'. No other key changes.

What I checked:

  • Trusted paths: none of the changed files match a trusted-path glob.
  • Plugin manifests: no .claude-plugin/plugin.json or marketplace.json changes, so there are no manifest lines to report.
  • Instruction surface: the two description edits widen when each skill auto-triggers. They delete, narrow or soften no guardrail or standing rule, so the instruction-surface lens does not apply.
  • Trust controls and credentials: the diff touches no trust control, token scope, kill switch or merge hold. It touches no workflow, script or credential handling.
  • PR context file: it contains no embedded directives aimed at this lane.

Spend: none beyond normal trigger behaviour. The changed descriptions may make the design skill auto-load slightly more often, which only uses subscription usage.

critical-findings: 0

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67983857df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

claude-review reviewed 67983857dfc952f32b8f45ea5672d88981bfec0e: read 4 of 4 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #11895, no findings

I read every hunk of all 4 changed files:

  • the changelog fragment
  • probes/design.json
  • skills/design/SKILL.md
  • skills/write-css/SKILL.md

None of them matches a trusted-path glob.

What changed. The PR changes two skill descriptions. In design, "to decide how something should look" becomes "…down to one corner radius or shadow, even with every number given". write-css gains the trigger 'fix this layout in the stylesheet'. The PR also adds three shape probes to probes/design.json and a bump: patch changelog fragment.

Checks

  • Probes. The new ids (pos-train-shape-01, pos-train-shape-02, pos-validation-shape-01) are unique in the file. Their split and expect_trigger values match the file's convention. pos-train-shape-01 gives every number (12px corners, 8px inset), which the updated ranking_note says one shape probe does.
  • Description length. I estimate the design description at roughly 970 characters, still under the common 1024-character skill-description cap. The author says check-skill.sh passes. I could not run it, so that is unverified.
  • Path-scoped rules. .claude/rules/skill-bodies-state-current-rules.md and .claude/rules/cost-claims.md cover the SKILL.md files. Neither is broken: the new text states current triggers, and it makes no cost claims.
  • Plugin manifests. The PR changes no plugin.json or marketplace.json. The fragment's patch bump suits a trigger-wording fix.

Closing issue #11423. The reopen comment lists three open items:

  1. shape-and-depth-nested-radius should route. The design description now covers questions that give every number, and a committed probe (pos-train-shape-01) would show a regression. The author reports the case fired design 0/3 on base and 3/3 on head. I did not run this, so it is unverified.
  2. Re-measure the probe sets. This is measurement only, with no code surface. The PR body reports results; I did not run them.
  3. Decide write-css's description. The PR rewrites it, and the existing probes/write-css.json positives are where a drop past the 5-point bar would show. The author reports 57/66 on both base and head for main's positives. I did not run this either.

Embedded text. The PR context contains no instructions aimed at the review lane.

Spend. None. The change only alters model-invocation wording and probe data.

@kyle-sexton kyle-sexton added the do-not-merge Hard merge gate: do not merge while applied. label Nov 1, 2026
@kyle-sexton kyle-sexton removed the do-not-merge Hard merge gate: do not merge while applied. label Nov 1, 2026
@kyle-sexton

Copy link
Copy Markdown
Member Author

Arm request (Emil lane, for the hub): #11895 is ready to arm once ci-status re-runs green on head 6798385. do-not-merge is removed, the Codex P1 thread is answered with round-5 VALID probe runs and resolved (0 open threads), both review lanes posted for this head, and the body cites round 5. Owner decision (2026-11-01, in session): the nested-radius condition is judged on runs with no error or denial, counting unfired runs as data, because the gate's skill-fired check rejects base not firing on that case, which is the behavior this change fixes.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit 8eca216 Nov 1, 2026
122 of 138 checks passed
@kyle-sexton
kyle-sexton deleted the fix/11423-round4 branch November 1, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(user-interface): design and write-css rarely fire on build-a-component prompts

1 participant