Skip to content

test(skills): the vacuity guard takes the claims it judges - #72

Merged
github-actions[bot] merged 1 commit into
mainfrom
feat/vacuity-guard-once-71
Oct 28, 2026
Merged

github-actions[bot] merged 1 commit into
mainfrom
feat/vacuity-guard-once-71

Conversation

@aitrawog

Copy link
Copy Markdown
Contributor

Closes #71

Summary

Head ec98ba5f0cfc3c0b011320302ca5622351507b34 — the commit this PR pushes: 1 file changed, 13 insertions(+), 9 deletions(-).

The finding. #70's validator review raised one non-blocking observation: the control computes the body's usage claims into claims and then calls skillDocIsVacuous(body, command), which computes them AGAIN — the same claims, read twice, for one decision.

The RCA.

  • Mechanism — the guard took the BODY and the COMMAND, so it re-derived the claims its caller had already derived. The control needs those claims for the loop below, and the guard needs only their emptiness, so one body was read twice for one decision.
  • Missed control — no rule held the control to compute a derived value ONCE. The rule this control enforces is that the vacuity rule has ONE implementation, and a rule that re-derives its own input sits one step further from the thing it judges.
  • Blast radius — this control's subject is the corpus of this repo's verb skills, published VERBATIM into the opencharly/marketplace corpus. The redundancy itself changes no verdict: both reads see the same body and the same command, so they agree. The blast radius is therefore the CONTROL's own standing, not the corpus's content — a rule that re-derives its own input is one step away from judging something else.
  • Root fix — the guard takes the CLAIMS it judges, so the control computes them once and hands the same slice to both the guard and the loop. The rule stays one implementation, and the red arm still fires on a body that never invokes the command it documents.

This is the successor to #70, which landed the guard and its red arm. #70 merged before this fix could be pushed onto it, so the fix is its own change here, off a fresh origin/main — the successor's body names its predecessor, and #71 carries the finding.

What lands. One file:

file what it carries
candy/plugin-box/skill_doc_flags_test.go the vacuity guard taking the claims it judges, so the control reads a body's claims once

No verb, no flag, no published prose, and no grammar changes: this is a tidy-up of the control that guards the corpus, and the guard's behaviour is unchanged.

How tested

Everything below ran on the committed tree this PR pushes.

$ git rev-parse HEAD
ec98ba5f0cfc3c0b011320302ca5622351507b34
$ git diff --stat origin/main...HEAD
 candy/plugin-box/skill_doc_flags_test.go | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

1. The repo's own Go gate — the gate this repo's AGENTS.md names.

$ cd candy/plugin-box && gofmt -l . ; echo "GOFMT_CLEAN=$?"
GOFMT_CLEAN=0
$ cd candy/plugin-box && go build ./... && echo BUILD_OK=1
BUILD_OK=1
$ cd candy/plugin-box && go vet ./... && echo VET_OK=1
VET_OK=1
$ cd candy/plugin-box && go test -count=1 ./... ; echo "GOTEST_EXIT=$?"
ok  	github.com/opencharly/plugin-box/candy/plugin-box	0.025s
?   	github.com/opencharly/plugin-box/candy/plugin-box/cmd/serve	[no test files]
GOTEST_EXIT=0

2. R7 (a) — the guard's RED ARM still fires. The fix moves the guard's argument, so the arm that proves the guard can fire is re-proved on THIS tree: the rule's body is replaced by return false, so it never fires, and the suite must go RED on the arm that expects it to fire.

$ cd candy/plugin-box
$ cp skill_doc_flags_test.go /tmp/umb-stagepar/pb71-backup.go
$ sed -i 's|return len(claims) == 0|return false|' skill_doc_flags_test.go
$ grep -n 'return false' skill_doc_flags_test.go
308:	return false
$ go test -count=1 -run 'TestSkillDocVacuityIsEffective' .
--- FAIL: TestSkillDocVacuityIsEffective (0.00s)
    skill_doc_flags_test.go:454: the vacuity guard did not fire on a body that never invokes the command it documents
FAIL
FAIL	github.com/opencharly/plugin-box/candy/plugin-box	0.003s
FAIL
RED_ARM_VACUITY_EXIT=1
$ cp /tmp/umb-stagepar/pb71-backup.go skill_doc_flags_test.go
$ cd ../.. && git status --porcelain ; echo "RESTORED_CLEAN"
RESTORED_CLEAN

The file is restored and the tree is clean; the green run of the same test is below, and the whole-suite green is block 1.

$ cd candy/plugin-box && go test -count=1 -run 'TestSkillDocVacuityIsEffective' -v . ; echo "GREEN_VACUITY_EXIT=$?"
=== RUN   TestSkillDocVacuityIsEffective
--- PASS: TestSkillDocVacuityIsEffective (0.00s)
PASS
ok  	github.com/opencharly/plugin-box/candy/plugin-box	0.004s
GREEN_VACUITY_EXIT=0

3. R7 (b) — the guard is still live INSIDE the control. The unit test proves the function fires; this proves the CONTROL calls it, so the guard still stands where it does its work rather than only in a test of itself. One case's command is mutated to a spelling the body never invokes, so the body makes no usage claim for it:

$ cp candy/plugin-box/skill_doc_flags_test.go /tmp/umb-stagepar/pb71-backup2.go
$ sed -i 's|{skill: "build", command: "charly box build", grammar: \&buildGrammar{}},|{skill: "build", command: "charly box buildz", grammar: \&buildGrammar{}},|' candy/plugin-box/skill_doc_flags_test.go
$ grep -n 'charly box buildz' candy/plugin-box/skill_doc_flags_test.go
89:	{skill: "build", command: "charly box buildz", grammar: &buildGrammar{}},
$ (cd candy/plugin-box && go test -count=1 -run 'TestSkillBodiesDocumentOnlyRealFlags' . ) ; echo "RED_ARM_CONTROL_EXIT=$?"
--- FAIL: TestSkillBodiesDocumentOnlyRealFlags (0.00s)
    --- FAIL: TestSkillBodiesDocumentOnlyRealFlags/build (0.00s)
        skill_doc_flags_test.go:342: ../../.agents/skills/build/SKILL.md: the body makes no usage claim for charly box buildz, so the control compares nothing in it
FAIL
FAIL	github.com/opencharly/plugin-box/candy/plugin-box	0.004s
FAIL
RED_ARM_CONTROL_EXIT=1
$ cp /tmp/umb-stagepar/pb71-backup2.go candy/plugin-box/skill_doc_flags_test.go
$ git status --porcelain ; echo "RESTORED_CLEAN"
RESTORED_CLEAN

Both arms fail the suite when the rule is broken and pass when it is intact, and the tree is restored after each.

4. The pre-flight — the project's own manifest validates on the tree that carries the change.

$ charly -C plugin-box box validate
charly box validate: OK — checked 1 candy, 1 deploy, 7 distros, 7 builders; 0 warnings, 0 errors
BOX_VALIDATE_EXIT=0

5. R10 — the live bed. check-box-local is this repo's disposable witness (a kind:local deploy on host: local), and it ran green on the committed tree:

$ charly -C plugin-box check run check-box-local
charly check run check-box-local: [deploy-add] PASS after 1.133s (log: .check/check-box-local/2026.301.1953/deploy-add.log)
charly check run check-box-local: [bring-up-members] PASS after 0s
charly check run check-box-local: [check-live] PASS after 14.539s (log: .check/check-box-local/2026.301.1953/check-live.log)
charly check run check-box-local: [update] PASS after 1.169s (log: .check/check-box-local/2026.301.1953/update.log)
charly check run check-box-local: [cleanup] PASS after 1.136s (log: .check/check-box-local/2026.301.1953/cleanup.log)
charly check run check-box-local: [cleanup-members] PASS after 0s
charly check run check-box-local: PASS (steps=6)
BED_EXIT=0

The bed's own summary carries the verdicts, with zero diagnostics:

$ cat .check/check-box-local/2026.301.1953/summary.yml
bed: check-box-local
calver: 2026.301.1953
driver: /home/atrawog/Sync/Atrapub/a/claude/opencharly/.worktrees/umb-stagepar/charly/bin/charly
driver_version: 2026.301.1632
steps:
  - name: deploy-add
    status: PASS
  - name: bring-up-members
    status: PASS
  - name: check-live
    status: PASS
  - name: update
    status: PASS
  - name: cleanup
    status: PASS
  - name: cleanup-members
    status: PASS
diagnostics:
  errors: 0
  warnings: 0
  advisories: 0
ok: true
status: PASS

6. R5 — the sweep. The claim: the control reads a body's claims exactly ONCE, and the rule's text appears once in the repo. The block below is the sweep for both:

$ grep -rn "skillDocIsVacuous" --include="*.go" .
candy/plugin-box/skill_doc_flags_test.go:299:// skillDocIsVacuous reports whether a body's usage claims for the command it documents are
candy/plugin-box/skill_doc_flags_test.go:307:func skillDocIsVacuous(claims []docClaim) bool {
candy/plugin-box/skill_doc_flags_test.go:337:			if skillDocIsVacuous(claims) {
candy/plugin-box/skill_doc_flags_test.go:453:	if !skillDocIsVacuous(usageClaims(neverInvokes, "charly box build")) {
candy/plugin-box/skill_doc_flags_test.go:464:	if skillDocIsVacuous(usageClaims(invokes, "charly box build")) {
$ grep -rn "len(claims) == 0" --include="*.go" .
candy/plugin-box/skill_doc_flags_test.go:308:	return len(claims) == 0
$ grep -rn "usageClaims(" --include="*.go" .
candy/plugin-box/skill_doc_flags_test.go:227:func usageClaims(body []string, command string) []docClaim {
candy/plugin-box/skill_doc_flags_test.go:336:			claims := usageClaims(body, tc.command)
candy/plugin-box/skill_doc_flags_test.go:453:	if !skillDocIsVacuous(usageClaims(neverInvokes, "charly box build")) {
candy/plugin-box/skill_doc_flags_test.go:464:	if skillDocIsVacuous(usageClaims(invokes, "charly box build")) {

The control derives a body's claims at ONE place (line 336) and hands the same slice to the guard (line 337) and to the loop below; the rule's text is at ONE place (line 308), inside the function; and the only other callers of the rule are the test's two arms, each with its own body.

Rulebook compliance

  • R0 — skills first. /charly-image:image, /charly-build:build, /charly-build:validate, /charly-internals:plugin and /charly-internals:git-workflow are the skills this repo's AGENTS.md names for this area; they were read before the first edit, and the git-workflow mechanics this PR follows are the ones they state (branch off fresh origin/main, commit, gate the committed tree, write the whole body before the push, PR-only landing).
  • R1 — every anomaly gets an RCA. The finding this PR fixes is RCA'd above with mechanism, missed control, blast radius and root fix. It was raised on test(skills): the vacuity guard gets its own red arm #70's validator review, and the org's rule is that a finding raised on a PR is fixed or rebutted with evidence, never filed away: it is fixed here, under its own issue (the vacuity guard recomputes the claims it judges — the same claims, read twice for one decision #71), claimed before the branch was cut. Nothing else was filed by this change.
  • R2 — no duplication. The control derives a body's usage claims once and the rule's text appears once (block 6). The re-derivation the validator review raised is gone in the same cutover, which is what this change is.
  • R3 — no workaround. No sleep, no retry, no suppression, no magic number. The control fails with a message that names the body and the command, and the test fails with a message that names the arm that did not hold.
  • R4 — product before prose. The change is in the control. The comment this PR adds describes the function that exists in the same commit.
  • R5 — legacy deleted completely. The guard's old body-and-command signature has no occurrence left in the repo (block 6): the rule was moved, not copied.
  • R6 — preserve user work and Git safety. git status was run first in this worktree; the branch is feat/vacuity-guard-once-71, cut off a fresh origin/main (eec8a46); no force-push, no history rewrite, no hook bypass, no push to main. The two mutations in blocks 2 and 3 were reverted from a backup copy, and git status --porcelain printed nothing after each.
  • R7 — prove behavior, not compilation. Two red arms, both live and both pasted above: the unit test's failing branch goes RED when the rule is inverted (block 2), and the control itself goes RED with the guard's own message when a case names a command the body never invokes (block 3). Neither arm can pass vacuously, and the green runs are pasted beside them.
  • R8 — live or skip, never fake a live service. The bed ran live against a disposable kind:local deploy and removed it again (the cleanup and cleanup-members steps are in the summary above). Nothing is mocked.
  • R9 — binary equals source. The binary under test is this session's worktree-local CalVer-stamped build (2026.301.1632), named by the bed's own summary driver and driver_version lines. The only .go file in the diff is a TEST file, so no production code changed and the binary is unaffected by this commit; no new runtime OS dependency is introduced, so packaging/ is untouched.
  • R10 — fresh disposable proof. The change class is a test control in a plugin's Go package, so the gate is this repo's Go gate, the charly box validate pre-flight, and the disposable bed that composes the changed entity (check-box-local) — all pasted above, all on the committed tree, all with zero warnings and zero diagnostics.
  • Cutover. One change, one PR, one commit: the guard's argument moves from the body and the command to the claims it judges, and the re-derivation leaves with it. Nothing is deferred and no scope is hidden inside it.

Change classification

  • Class: a test control in a plugin's Go package — the Go class's gate (go build / go vet / go test) plus this repo's project-configuration pre-flight (charly box validate) and its disposable R10 bed (check-box-local). NOT documentation: what changes is the control's own code, and the two red arms above are what prove the guard still behaves.
  • Attribution tier: fully tested and validated — the change-class gate passed on the final tree (the repo's Go gate, the pre-flight, and the disposable R10 bed), the changed path was executed live, and both red arms demonstrate the coverage can fail.
  • Scope. This is the successor to test(skills): the vacuity guard gets its own red arm #70 (the vacuity guard and its red arm), not a second cutover of that surface: it changes no verb, no flag, and no published prose. It closes the one gap test(skills): the vacuity guard gets its own red arm #70's validator review recorded.

Filed issues

Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0

Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)

The control computed the body's usage claims into `claims` and then called
`skillDocIsVacuous(body, command)`, which computed them AGAIN — the same
claims, read twice, for one decision. The guard now takes the claims, so
the control computes them once and hands the same slice to both the guard
and the loop below. The rule stays one implementation and the red arm
still fires on a body that never invokes its command.

This is the non-blocking observation #70's validator
review raised, and it is fixed here rather than filed away.

Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)
@aitrawog

Copy link
Copy Markdown
Contributor Author

STATUS

STATUS

The successor PR is open: #72 — head ec98ba5f0cfc3c0b011320302ca5622351507b34, branched off a fresh origin/main (eec8a46). Its predecessor, #70, landed (squash, tag v2026.301.1952) and carried the guard and its red arm; this PR carries the one non-blocking observation #70's validator review raised.

What is done. The vacuity guard now takes the claims it judges, so the control derives a body's usage claims once and hands the same slice to both the guard and the loop below. The rule stays one implementation. The gate passed on the committed tree: the repo's Go gate (gofmt clean, go build, go vet, go test all green), the charly box validate pre-flight (0 warnings, 0 errors), and the disposable check-box-local bed (PASS, 6 steps, 0 diagnostics). Both red arms are pasted in the PR body and re-proved on THIS tree: the unit test's failing branch goes RED when the rule is inverted, and the control itself goes RED with the guard's own message when a case names a command the body never invokes.

What is next. Read the validator's full report on #72, fix every finding it raises in one commit, and land through the org's auto-merge. Then close this issue and record the resolution.

Next update. When the validator's verdict lands on #72.

Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0
Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)

@github-actions

Copy link
Copy Markdown

Review — PASS

What the change does

The diff is exactly what the body describes: skillDocIsVacuous changes signature from (body []string, command string) to (claims []docClaim), and its body from len(usageClaims(body, command)) == 0 to len(claims) == 0. The one production call site inside the control now derives claims once (claims := usageClaims(body, tc.command)) and passes the same slice to the guard (line 337), eliminating the double read the body's RCA targets. The two test arms in TestSkillDocVacuityIsEffective pass usageClaims(...) inline, which is correct for a test that exercises the rule directly.

Verification against the body

  • Counts. Removed lines: 6 (old comment + signature + return) + 1 + 1 + 1 = 9; added: 10 (new comment block + signature + return) + 1 + 1 + 1 = 13. Matches the stated 1 file changed, 13 insertions(+), 9 deletions(-).
  • Line references. Diff hunk headers (@@ -296,12 +296,16 @@, @@ -330,7 +334,7 @@, @@ -446,7 +450,7 @@, @@ -457,7 +461,7 @@) place the comment at 299, the function at 307, the return at 308, the control's claims := at 336 / guard call at 337, and the two arms at 453 / 464 — all exactly as the body's block-6 sweep prints them. The red-arm outputs (sed matching return len(claims) == 0 → return false at 308; the failure message at line 454; the charly box buildz mutation at line 89 failing at 342) are internally consistent with the post-change file, which would not be true of the pre-change signature.
  • Behaviour unchanged. len(nil) == 0 is true, so a body with no claims still trips the guard; the arms prove both directions (fires on a body that never invokes the command, does not fire on one that does). No verb, flag, grammar, or published prose is touched — the sole changed file is a _test.go.
  • Completeness. The sweep in block 6 lists every remaining reference to skillDocIsVacuous and usageClaims; all are accounted for in the diff, and no old body-and-command call site survives.

Disposition of comments

The <comment_thread> is empty (0 comments), so there is nothing to re-derive or disposition.

Checks

  • Correctness: refactor is faithful; no behavioural drift.
  • Honesty: every quantitative and line-level claim in the body is reproduced by the diff.
  • Tests: the guard keeps its red arm (rule inverted → suite RED) and the control keeps a live arm (mutated command → guard message at the control), both pasted with green runs beside them.
  • Security: test-only change; no secrets, input handling, or dependency surface.
  • Scope: single coherent commit, no dead path or duplication left behind.

No findings.

Verdict: PASS


ollama-cloud/deepseek-v4.1-flash — action-review.

View action run

@github-actions
github-actions Bot enabled auto-merge (squash) October 28, 2026 19:55
@github-actions
github-actions Bot merged commit 4dfb43a into main Oct 28, 2026
3 checks passed
charly-auto-merge Bot added a commit that referenced this pull request Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

the vacuity guard recomputes the claims it judges — the same claims, read twice for one decision

1 participant