Repository navigation
feat(eval): structured write allowances for the item skills (#67, step 2 of 3) - #99
Merged
Merged
Conversation
triage-github-issue, implement-github-issue and review-github-pr list what they may write as allow entries (ADR 0017, step 2). 12 tests add an effect each skill's text forbids to a golden scenario.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 2 of 3 for #67, on top of #97 (merged).
triage-github-issue,implement-github-issueandreview-github-prnow list what they may write asallowentries, so the scenario checker refuses a label, a comment marker or an op that the skill's text does not permit, even where its scope allows a write.How the lists were made
Each
allowlist comes from the skill's own text and the protocol references (labels.md,handoff.md,escalate.md,human-merge.md,issue-after-pr.md,dedupe.md), not from the goldens alone. So a label no scenario applies (bug,enhancement) is still listed. Every golden of the three skills passes unchanged, and the contracts' wording is unchanged: each sentence is kept and gains anallow.Decisions worth a look
agent:outcomeand no marker, so triage allows both. Pinning one is a skill change, not part of this PR.needs-triageand its own lease label. Eligibility excludes any issue that already carries a state label, andneversays only a human removesneeds-info, so no state-label removal is listed.review-github-prcan also removeready-for-agentandagent:pr-openfrom a linked issue, perissue-after-pr.md.Tests
12 new tests: for each skill, a golden scenario plus one effect the skill's text forbids (triage applying
wontfix, implementing merging a PR, review sending a linked issue back toready-for-agent, an unmarked comment on the claimed PR, and so on). Each asserts the golden is clean first, then that the added effect is refused by anallowentry. A scope-only checker accepted all 12.npm test: 548/548 (536 + 12).npm run validateandcheck-plugin-versionpass. ADR 0017 is updated with the migration notes, and there is a changeset.Not in this PR
Step 3:
triage-github-issues,implement-github-issues,review-github-prs,shepherd-github-prs,reconcile-github-issues,file-github-issues,explore-github-repo. Those are the group, sweep and file skills.Refs #67, #72.