Skip to content

the skill body's base is not the same in this repo's own build and in the org's build — the two disagree, and the repo's own unit tests lock in the one the org does not ship #73

Closed

Description

@aitrawog

The defect

A skill body's location is resolved from TWO different bases, depending on which sdk the code is compiled against — and this repo's own unit tests assert the one the org's build does not ship.

The base was unified onto the owning candy's REPO ROOT (sdk/deploykit/candy_repo_root.go → candyRepoRoot), and the sdk the org's build compiles resolves the body there. This repo's own go.mod still pins an sdk from before that unification, and under THAT pin the body is resolved at the owning candy's own directory instead. The two disagree about whether a body beside the candy is valid, and the repo's tests encode the older answer:

$ go version -m <the charly binary under test> | grep -E 'opencharly/(sdk|plugin-box)'
	dep	github.com/opencharly/plugin-box/candy/plugin-box	v0.2026301.1542
	dep	github.com/opencharly/sdk	v0.2026297.1851
$ grep -m1 'opencharly/sdk v' candy/plugin-box/go.mod
	github.com/opencharly/sdk v0.2026293.1143
$ grep -m1 'opencharly/sdk v' ../plugin-marketplace/candy/plugin-marketplace/go.mod
	github.com/opencharly/sdk v0.2026297.129

v0.2026293.1143 has no deploykit/candy_repo_root.go at all; v0.2026294.1520 is the first cached version that has it.

The two bases, in the two sdks:

$ grep -n 'base :=' $GOMODCACHE/github.com/opencharly/sdk@v0.2026293.1143/deploykit/skills_collect.go
166:	base := layer.GetSourceDir()
$ grep -n 'base, ok := candyRepoRoot' $GOMODCACHE/github.com/opencharly/sdk@v0.2026297.1851/deploykit/skills_collect.go
213:	base, ok := candyRepoRoot(layer)

Measured, on the tree's own binary, both placements of the same project:

$ charly -C <project-with-the-body-BESIDE-the-candy> box validate
charly: error: command "validate": validation error: skills: skill "psk1": no body — the node carries no in-line `content:` and no .agents/skills/psk1/SKILL.md exists at the repo root of the owning candy "pcandy"
$ echo "exit=$?"
exit=1
$ charly -C <project-with-the-body-AT-the-project-root> box validate
charly box validate: OK — checked 1 candy, 7 distros, 7 builders; 0 warnings, 0 errors
$ echo "exit=$?"
exit=0

And this repo's own tests assert the OPPOSITE, and pass — because they compile against the older pin:

$ grep -n 'func TestValidateSkillBodies' -A12 candy/plugin-box/validate_skills_test.go
func TestValidateSkillBodies_BesideTheCandyIsClean(t *testing.T) {
	...
	if len(e.msgs) != 0 {
		t.Fatalf("body beside the candy must be clean, got %v", e.msgs)
	}
}
func TestValidateSkillBodies_AtTheRepoRootIsReported(t *testing.T) {
	...
	if len(e.msgs) == 0 {
		t.Fatalf("body at the repo root must be reported — the loader reads beside the candy, not at the root")
	}
}

So go test ./... in candy/plugin-box/ is green over the OLD behavior, while the binary the org builds and ships runs the NEW one — and the two cannot both be right about the same fixture.

Where it surfaced

The charly/pr-validator review of #66 raised it as a documentation nuance, and it is a documentation nuance only because the prose picked ONE of the two answers without saying which build it holds for:

One documentation nuance (not a functional defect): the pre-change fixture wrote the body beside the candy and was asserted to validate clean, which implies the lookup found it there; the new comment says the lookup "did NOT look beside the candy." If the R10 PASS is accurate, the lookup must resolve the project root (whether exclusively or in addition), so the fixture change is functionally safe; the "did NOT look beside" wording is at most over-stated.

The reviewer's inference is right — the pre-change fixture DID validate clean, because under this repo's own sdk pin the lookup reads beside the candy. #66 moved the bed's fixture to the project root, which is what the ORG's build requires, and left this repo's unit tests asserting the old placement.

Two places name a base, and each names a different one:

  • charly.yml (the check-box-local bed): "The sdk's body lookup resolves that root from the candy's source dir (deploykit/skills_collect.go → candyRepoRoot) and looks there; it did NOT look beside the candy." — the org build's answer, in a past tense that reads as the opposite of what it means.
  • candy/plugin-box/validate_skills.go: "validateSkillBodies reports every authored skill: node whose body does not resolve at <owning candy SourceDir>/.agents/skills/<name>/SKILL.md." — this repo's own pin's answer, and the base the org's build does not use.

RCA

  • Mechanism — the base was unified in the sdk (deploykit/candy_repo_root.go, "One base, one layout, two readers is the fix"), which CHANGED the resolution of every skill body. A Go module pin is a FLOOR, not a ceiling: the org's build compiles the plugin against the HIGHEST sdk in the graph, so charly's own pin (v0.2026297.1851) silently raised the floor above this repo's (v0.2026293.1143) and moved the behavior under this repo's feet. Nothing in this repo's own CI builds it that way, so nothing in this repo's CI could see the move.
  • Missed control — a repo's own test suite is the control for its own behavior, and it compiles against the repo's own pin. When the org's build resolves a DIFFERENT (higher) version than the repo declares, the repo's tests keep proving the OLD behavior and go green over it. There is no control that says "the version this repo's tests compile against is the version the org builds it against."
  • Blast radius — every skill body in every repo, and every charly box validate that reports one: the diagnostic a user sees ("no .agents/skills//SKILL.md exists at the repo root of the owning candy") names the NEW base, while this repo's own docs and tests name the OLD one. A repo whose skills sit beside their candy validates clean under this repo's pin and is REFUSED by the shipped binary.
  • Root fix — align this repo's sdk pin with the one the org's build compiles, so ONE base holds in every build; let the stale assertions turn red (they are the discriminating proof that the two builds disagreed); fix the assertions and the two prose statements to that one base; and close the control gap so a plugin repo's declared floor cannot silently fall behind the org's build again.

The open question this issue hands on

Whether the org wants a CONTROL for the floor (a check that every plugin repo's declared sdk pin is at least the version the org's build compiles) or wants the floor to be pinned centrally — that is an org-wide design question, and it belongs with the sdk's own owners. The fix this repo can make on its own is to align its own pin and its own prose; the control is handed on with the evidence.

Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0

Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (analysed on a live system)

Activity

  1. aitrawog commented on Oct 29, 2026

    @aitrawog
    ContributorAuthor

    CLAIM

    Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0

    Claimed as the follow-up to #66's validator finding — the review of that PR raised the "did NOT look beside the candy" wording as a documentation nuance, and the RCA behind it is this issue.

    Why it is a real defect and not a wording nit. The comment in #66 names a base; this repo's own unit tests name the OTHER one; and the two are the two bases the sdk actually has, split by the version each build compiles:

    • the org's build compiles plugin-box against sdk v0.2026297.1851, whose collector resolves the body at the owning candy's REPO ROOT (deploykit/candy_repo_root.go → candyRepoRoot);
    • this repo's own go.mod pins sdk v0.2026293.1143, whose collector resolves it at the owning candy's own directory (base := layer.GetSourceDir()).

    So go test ./... in candy/plugin-box/ is green over the OLD behavior while the shipped binary runs the NEW one, and TestValidateSkillBodies_BesideTheCandyIsClean asserts precisely what the org's build refuses. The reviewer's inference — that the pre-change fixture validated clean because the lookup found the body there — is correct under this repo's own pin.

    What I will do. Align this repo's sdk pin with the one the org's build compiles, let the stale assertions turn RED (they are the discriminating proof that the two builds disagreed), fix the assertions and the two prose statements to that one base, and land it through a PR with the R10 bed run on the committed tree.

    What I will not do. I will not touch the sdk, and I will not widen this into the org-wide "a plugin repo's declared floor must not fall behind the org's build" control — that question belongs to the sdk's owners, and the issue hands it on with the evidence.

    Status: the fix is scoped and the evidence is measured; the PR opens once the alignment is proven not to break the module.

    Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0
    Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (analysed on a live system)

  2. self-assigned this
    on Oct 29, 2026
  3. aitrawog commented on Oct 29, 2026

    @aitrawog
    ContributorAuthor

    STATUS

    Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0

    The PR is open: #74, head b860c11b51e71f64b402d7f7704069722c9ad0f5, branch feat/skill-body-base-73. 5 files changed, 125 insertions(+), 58 deletions(-).

    What is done.

    • The sdk floor is advanced from v0.2026293.1143 to v0.2026297.1851 — the version the org's build compiles — so this repo's own build resolves a skill body at the owning candy's REPO ROOT, the way the shipped binary does.
    • The three assertions that encoded the OLD base are flipped, and the diagnostic's own wording is pinned (the negative arm must name the repo root it looked in).
    • The two prose statements that named a base are corrected: candy/plugin-box/validate_skills.go now names the REPO ROOT, and the bed's comment in charly.yml states the base in the present tense and points at deploykit/candy_repo_root.go, where the base actually lives.
    • The flipped assertions are now a floor guard: lowering the pin again turns them RED.

    What is next. The validator's verdict on #74, then landing through auto-merge, then this issue's RESOLVED record.

    The gate that ran on the committed tree (the repo's own Go gate, charly box validate at the root, and the check-box-local R10 bed — PASS (steps=6)), and the discriminating proof that the coverage fails without the change: with the floor put back to the old value, exactly the three flipped assertions go RED, and the old sdk's own diagnostic reads "no SKILL.md exists under .agents/skills beside the owning candy" — the literal opposite of the new one.

    Next update: when the validator's verdict lands on #74 — or by the end of the day, whichever comes first.

    Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0
    Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)

  4. added a commit that references this issue on Oct 29, 2026
    ecb0980
  5. aitrawog commented on Oct 29, 2026

    @aitrawog
    ContributorAuthor

    RESOLVED

    Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0

    Fixed by #74 — squash ecb0980e837ce383a546c463f4bcd66cc2347706, tag v2026.302.0039, charly/pr-validator verdict PASS, merged 2026-10-29T00:39:59Z.

    What was fixed, against what this issue claimed.

    • The sdk floor is advanced from v0.2026293.1143 to v0.2026297.1851 — the version the org's build compiles — so this repo's own build resolves a skill body at the owning candy's REPO ROOT, the way the shipped binary does.
    • The three assertions that encoded the OLD base are flipped, and the diagnostic's own wording is pinned: the negative arm must name the repo root it looked in.
    • The two prose statements that named a base are corrected — candy/plugin-box/validate_skills.go names the REPO ROOT, and the bed's comment in charly.yml states the base in the present tense and points at deploykit/candy_repo_root.go, where the base actually lives.
    • The flipped assertions are now a standing floor guard: lowering the pin again turns them RED.

    The validator's two observations on #74 are disposed of in #75 — squash 876d566d988c5b8a4a398aa0d9c755b8a7f41895, tag v2026.302.0047, verdict PASS, merged 2026-10-29T00:47:22Z: the negative arm now asserts the finding count (it indexed e.msgs[0] without asserting how many findings there were), and the record's red-arm sentence now says the floor is restored in go.mod and go.sum together — the correction is written into CHANGELOG/2026.302.0039.md itself, with a dated note, so the record says what was done rather than what was first written.

    The org-wide control this issue handed on is FILED: opencharly/opencharly#795 — "no gate compares a plugin repo's DECLARED module floor with the version the org's build resolves for it". It carries the RCA, the measured blast radius, and the two shapes that would close it, and it names the choice of owner as the owners' to make. It is deliberately NOT part of this issue's resolution: the control belongs to the org's audit, not to one plugin's tests, and this instance is fixed.

    Gate on the tree that was pushed. The repo's own Go gate (gofmt / go build / go vet / go test -count=1, rc=0), charly box validate at the root (0 warnings, 0 errors), and the check-box-local R10 bed (PASS (steps=6)), plus the changed path executed live against the worktree-local DEV build. The discriminating proof: with the floor put back to v0.2026293.1143 — the version and its checksums restored together — exactly the three flipped assertions go RED, and the old sdk's own diagnostic reads "no SKILL.md exists under .agents/skills beside the owning candy", the literal opposite of the new one.

    Agent: charly · session ses_8efb271d-f2c0-47d3-85e3-2a1e00b11ae0
    Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions