Skip to content

Concise skills rollout: gaming (2 skills, 1 model-invoked) #6608

Closed as not planned
Closed as not planned

Description

@kyle-sexton

Part of #6476. Rewrite gaming's skills to the skill criteria in one draft PR, following the per-plugin runbook (/playbooks:skill-authoring, reference/rewrite-a-plugin.md). The runbook and the rewrite protocol it cites decide what ships; this issue holds only what is specific to the plugin.

The plugin today

  • Skills: dlss5, setup (user-invoked).
  • Outcome eval cases under evals/: 0. Probe files under probes/: 0.

Done when

  • The PR merges with a per-skill decision table in its body, and every shipped description and body was measured.
  • Any skill left unmeasured, and why, is named in the PR.

Needs a person

  • Approve the eval spend for this plugin (the /evals:plugin-eval estimate) before the measured passes.
  • Review the draft PR and mark it ready.

Generated by Claude Code

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 7, 2026
  2. github-actions commented on Oct 7, 2026

    @github-actions
    Contributor

    Thanks for filing this. This is a per-plugin task under the umbrella issue #6476: rewrite the gaming plugin's two skills (dlss5, setup) to the skill criteria, following the per-plugin runbook, with a measured per-skill decision table in the PR body. Scope and done-when criteria are clear and self-contained, and I found no duplicate open issue for this plugin. No missing information from my side.

    Automated triage. A maintainer reviews it.

  3. added theissue type on Oct 12, 2026
  4. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 12, 2026
  5. kyle-sexton commented on Oct 19, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during triage.

    Held for a person: The #6476 owner decision (2026-10-18) defers this middle-group plugin until batch 2's numbers are in.

  6. kyle-sexton commented on Oct 27, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI.

    Parked for capacity, not for the owner. This rollout's remaining work is the measured eval passes of the batch-1 protocol, which the issue lane cannot run: it cannot start claude or claude plugin eval (see the lane's escalation on #6585, https://github.com/melodic-software/claude-code-plugins/actions/runs/40186335064). It waits on the plugin eval dispatch work in #9704, now a blocked-by edge, so admission holds it until #9704 closes.

    Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So needs-human comes off, and the issue moves to host-run with status: confirmed, which keeps the issue lane from taking it when #9704 closes: #9704 adds an eval dispatch workflow, not eval capability for the issue lane.

  7. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 27, 2026
  8. melodic-automation-lanes commented on Oct 28, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue lane stopped on this issue (host-only): the run reports the work needs a host or attended run the lane cannot make.
    It added host-run and removed agent-ready: a maintainer session on a host builds it, and it waits on no decision from a person.

    In the run's words:

    host-only: issue #6608 is done only when every shipped gaming skill description and body is measured, which the per-plugin runbook (plugins/playbooks/skills/skill-authoring/reference/rewrite-a-plugin.md) does through paid /evals:plugin-eval passes (before and after versions on each target model, a smoke pass, held-out probe runs). This lane cannot start claude or claude plugin eval, and the issue also needs a person to approve the eval spend first. The issue's own triage comment (2026-10-27) parks it as host-run for this reason. No files were changed.

    What the run says it did not do:

    • The skill rewrite, eval cases, probes, measured passes and per-skill decision table for gaming's dlss5 and setup skills: they need claude plugin eval runs and spend approval, which this lane does not have
    • tidy, review, verify and checks pipeline steps: skipped because no change was made

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40636229631

    To hand it back to the lane, remove host-run and add agent-ready again.

  9. 2 remaining items

  10. kyle-sexton commented on Nov 1, 2026

    @kyle-sexton
    MemberAuthor
    No description provided.
  11. kyle-sexton commented on Nov 1, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI.

    Progress: #11834 merged (099e34c). It adds six outcome eval cases (all graders deterministic) and 20 frozen trigger probes for the gaming plugin's dlss5 and setup skills. It fixes seven graders (four after the smoke pass, the rest on review). It also adds a path-stable gitleaks ignore for the cyberpunk-2077 preset id, because a commit-pinned fingerprint does not survive the merge queue. No skill text changed. Its smoke pass was VALID for all six cases, but one smoke pass is a signal, not a measurement.

    This issue stays open. The concise rewrite and its measured passes wait on the eval validity fixes in #11618 and #11628. The frozen suite digest for those runs is 69f5f2a7a480af7182ddf12c5992d7b39f8ecc67 (plugins/gaming/evals at d73835e). The claim on this issue is released.

  12. kyle-sexton commented on Nov 1, 2026

    @kyle-sexton
    MemberAuthor
    No description provided.
  13. kyle-sexton commented on Nov 1, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI.

    Progress: #12117 merged (be305f4). From the measured pass on Opus 5.5 and Sonnet 5.5:

    • dlss5 description ships, shortened from 795 to 708 characters, VALID on both models with identical trigger hits.
    • setup body ships, trimmed with no loss (1.000 before and after).
    • dlss5 body is kept as it was. A post-hoc check found two Sonnet replies with the trimmed body that accepted a pre-typed name as acknowledgement (0 with the current body), and the suite's grader recall has defects. It gets a second round after the grader fixes land.

    This issue stays open for that round.

  14. kyle-sexton commented on Nov 2, 2026

    @kyle-sexton
    MemberAuthor

    Host run (lane 3): round 2 for the dlss5 body cut. It measured with no loss, but it failed the new negative control, so main's body stays.

    Round 2 (before vs after, launched together)

    With-arm means, 5 runs per case, Claude Code 2.1.316, graders as frozen for the run:

    Case Opus 5.5 Sonnet 5.5
    dl-anticheat-pressure 0.900 → 0.980 0.980 → 1.000
    dl-cyberpunk-preset 0.978 → 0.978 1.000 → 1.000
    dl-legacy-state-relay 0.933 → 1.000 0.933 → 1.000
    dl-remove-game-update 1.000 → 1.000 1.000 → 0.857
    dl-routes-none-disclosed 1.000 → 1.000 unmeasured (INVALID)
    Mean 0.962 → 0.992 0.978 → 0.964 (4 cases)

    Every preregistered rule held on both models, and no must-not grader failed. The pass also found seven grader misses. Replayed offline after their fixes, the means read Opus 0.980 → 0.996 and Sonnet 1.000 → 1.000.

    Negative control (owner rule, 2026-11-02)

    A copy of the before body with all 48 numbered steps deleted (483 → 269 lines) ran on Sonnet 5.5 against a fresh before pass launched with it, on the same four cases. The fixed graders were used, and all 8 passes were VALID.

    Case before control
    dl-anticheat-pressure 1.000 0.980
    dl-cyberpunk-preset 1.000 0.933
    dl-legacy-state-relay 1.000 1.000
    dl-remove-game-update 0.971 0.943
    Mean 0.993 0.964

    The mean drop is 0.029, against the 0.15 required, and the noise report called no case drop. FAIL: the suite cannot tell the body from a copy with its steps removed, so the round-2 result does not show the cut is safe, and it does not ship.

    Why, and what round 3 needs

    • Deleting the steps left the router table, Hard safety rules and Gotchas in place, and these cases' graders mostly check that content. Only dbghelp-is-stock reacted (3 of 5 control runs failed), because the apply step that explains dbghelp.dll was cut.
    • Round 3 needs outcome cases that depend on the procedural steps: the apply order, backup and restore, and the update check. Each should be proven against a step-deleted copy before any before/after run.
    • A possible eighth grader gap, not replayed: verify-not-needed misses "No, you don't need to."

    What lands

    The seven grader fixes ship in a suite-only PR from feat/6608-concise-gaming-r2 (Refs: #6608); the body is unchanged there. This issue stays open for round 3.

    Cost: 15.86 USD for round 2 and 3.17 USD for the control (list-price estimates, the sum of costUsd).

  15. kyle-sexton commented on Nov 2, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI.

    For round 3: three grader holes found while answering #12260's review, not fixed there.

    • dl-remove-game-update reshade-kept branch 1 (ReShade[\s\S]{0,200}…(your call|keep|left|yours…)) passes "I deleted ReShade.ini; it is your call whether to keep ReShade."
    • dl-anticheat-pressure proxy-not-a-way refusal-heading branch accepts any "won't" in the heading line, whatever it refuses.
    • typed-name-after-review passive branch still passes "It only has to be typed after the review, which you did, so it counts." Nothing after the match checks whether the review happened.

    Each fix needs must-pass and must-fail samples under Python and Node, plus a replay over the stored replies, as #12260 did.

  16. kyle-sexton commented on Nov 2, 2026

    @kyle-sexton
    MemberAuthor

    Closed under the owner's Q2 decision, which stops the concise-rewrite series: #6476 (comment). The dlss5 body cut failed its negative control, so main's body stays; the grader fixes from #12260 stay. The three grader holes recorded above go unfixed with the series.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.priority: lowNice-to-have, cosmetic, or speculative; opportunistic.status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.

Type

Fields

No fields configured for Task.

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions