charly task has no skipped bucket: a task reports "N step(s), 0 failed" over a step that ran nothing

`charly task` has no skipped bucket: a task reports "N step(s), 0 failed" over a step that ran nothing

The task summary counts steps and failures. A step that ran nothing is neither:
it is not a failure, so it is counted among the N steps, and it is not disclosed. The
org already owns the vocabulary for exactly this state — sdk.CheckSkippedExitCode = 3
("the phase RAN NOTHING — neither a pass nor a failure", sdk/exitcode.go:30-33) —
and the check runner uses it. The task runner has no equivalent.

The three layers, measured

Line numbers are the umbrella's pinned checkouts.

1. The plan runner skips a step and does not fail it. sdk/kit/planrun.go sets
result.Status = StatusSkip at lines 176, 184, 192, 227 and 253 — the unresolvable
variable reference among them (opencharly/charly#914). A skipped step is not counted as
failed.

2. The task runner has nowhere to put it. plugin-task/candy/plugin-task/runner.go:72
carries Failed int and Steps []spec.StepResult; there is no skipped counter, though
each step's Result.Status is right there. runner.go:190 sets
res.Message = "%d step(s) failed", and cli.go:210 prints
task %s: %d step(s), %d failed; the aggregate at cli.go:354 prints
ran %d task(s), %d step(s) failed. No third bucket in any of the three.

3. A command step cannot say "nothing ran" either. A command: step that exits 3
is an ordinary failure today, and a step whose script internally skipped a comparison
and exited 0 is an ordinary pass. So the one channel the org already has for "the phase
ran nothing" is unreachable from a task step.

What it costs, measured live

The umbrella's own gate scripts compare surfaces that live in submodules, and skip a
comparison when the submodule is not checked out — printing the skip and exiting 0, so a
partial tree stays landable. On a light clone with two of those repos absent
(opencharly/opencharly#591, branch feat/light-init):

$ node scripts/check-coord-verb-set.mjs
  SKIP  action-review charly.yml (the validator's own closed set): action-review/charly.yml is not checked out (the comparison did not run)
  SKIP  layer-charly-internals git-workflow SKILL.md (the CLOSED set): … (4 more)
check-coord-verb-set: 12 surface(s) compared, 5 skipped, 12 statement(s) of the set found, 0 failure(s)
check-coord-verb-set: OK
$ node scripts/check-validator-rule-map.mjs --no-live
  SKIP  action-review/ is not checked out: the id comparison did not run (git submodule update --init action-review)
check-validator-rule-map: OK

Six comparisons did not run and both gates printed OK. The gate scripts DO count
their own skips — "12 surface(s) compared, 5 skipped" — and the information is
discarded one layer up, because the summary that the reader sees has no bucket for it.
The charly task verify headline over that tree read 26 step(s), 0 failed.

This is the task-runner twin of opencharly/charly#904 ("SKIP counts as PASS in the bed
summary"), one layer down and one verb across: the bed summary mints the green
acceptance record, the task summary mints the green gate line, and both fold "ran
nothing" into "ran and passed".

RCA

Mechanism — the task summary's vocabulary is {step, failure}. A state that is
neither (a step that ran nothing) has no representation, so it is rounded into
"step".
Missed control — sdk.CheckSkippedExitCode exists and is honoured by
plugin-check, but the task runner neither emits nor consumes it, and the summary
counts nothing that a skip could be counted into. The umbrella's gate scripts then
supply their own skip accounting, which no aggregator reads.
Blast radius — every charly task summary in the org, and every reader who
takes "26 of 26 passed" as "26 of 26 checks ran". The org's R10 evidence rests on
exactly that reading.
Root fix — give the task runner the third bucket, using the convention the org
already has rather than a new one:
1. count the plan runner's StatusSkip steps in runResult and print it:
  task %s: %d step(s), %d failed, %d skipped (and the aggregate likewise), so a
  skipped step is visible in the headline instead of being rounded into "step".
2. honour sdk.CheckSkippedExitCode in the command: verb — a command: step
  exiting 3 becomes a SKIPPED step, not a failure. This is what lets a gate script
  say "the phase ran nothing" with the org's own exit code, instead of inventing
  one, and it is the channel a task step needs that the task runner currently
  lacks.
3. keep the exit status honest: a skipped step stays non-fatal (R8's sanctioned
  skip — an absent service or credential), but its reason must reach the summary.
  The distinction the org needs is the one #904 already names: a skip caused by an
  absent CREDENTIAL is legitimate, a skip caused by an absent SUBMODULE is a
  coverage hole in the gate — and today both are the same exit code and both read
  as green.

Why this is not the umbrella's problem to fix

opencharly/opencharly#591 fixed the umbrella-side trigger: the working set now derives a
fourth source — the repos the umbrella's own gate scripts read — so a light tree
compares all 17 surfaces and skips 0 (measured: "17 surface(s) compared, 0 skipped",
against "12 compared, 5 skipped" with the two repos absent). The common case is
complete.

The residual is the disclosure, and it lives here: a tree that genuinely lacks one of
those repos still reports a green task summary over a smaller set of comparisons, and
the reader of the summary cannot tell. The silent: true case is worse: a silent task
suppresses the per-step lines entirely, so a skipped step leaves no trace at all.

Agent: charly

Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)

CLAIM

This session owns #35 — the task runner's missing third bucket. The task summary's vocabulary is {step, failure}; a step that ran NOTHING is rounded into "step", so a charly task line can read "26 step(s), 0 failed" over a run in which six comparisons did not happen.

What the change is. Three things, using the convention the org already owns rather than a new one:

The task runner counts the plan runner's StatusSkip steps and prints them: task %s: %d step(s), %d failed, %d skipped, and the aggregate likewise.
The command: verb honours sdk.CheckSkippedExitCode (3) — a command step exiting 3 is SKIPPED, not a failure. That is the channel a task step needs for "the phase ran nothing", and it is the org's own exit code rather than an invented one.
The exit status stays honest: a skipped step is non-fatal (R8's sanctioned skip), and its reason reaches the summary.

What must not change. A skip never becomes a pass and never becomes a failure. The verb's existing failure semantics, the exit status of a failing task, and the silent: true path's meaning are untouched — except that a silent task must not let a skipped step leave NO trace, since the disclosure is the whole point.

The proof. A fixture whose plan carries a step the runner skips (an unresolvable variable reference — the plan runner's own StatusSkip path) and a command: step that exits 3: the summary names the skipped count, the task's exit status is 0, and the run is shown to be RED when the same fixture should fail. A test that cannot fail is not coverage, so the fixture is proved to discriminate.

What is next. The umbrella's own working-set plan is complete — wave 1 (opencharly/opencharly#623), wave 2 (the guidance corpus) and wave 3 (#38) have all landed. This leg is the last of the filed wave issues, and it is the same false-green family as opencharly/charly#904 (the bed summary) and #34 (the data channel): the reader of a summary must be able to tell "ran and passed" from "ran nothing".

Next update due: when the branch is cut and the fixture is red against the current shape, or within two hours, whichever comes first.

Agent: charly · session light-init
Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (analysed on a live system)

feat(command): give a `command:` step the org's "the phase ran nothin…

STATUS

This session's CLAIM on this issue is now a PR: #45, head 5549735 on feat/task-skipped-bucket-35 (branched off origin/main 0a3785d).

What the PR lands.

The third bucket: runResult.Skipped, tallied from the plan runner's spec.StatusSkip. The headline is task %s: %d step(s), %d failed, %d skipped; the hop's aggregate is ran %d task(s), %d step(s) failed, %d skipped and is KEPT as the prefix whenever a step ran nothing; the JSON report carries "skipped"; the nested failure verdict carries the count too, so a reader of a failure also learns what did not run.
The silent path keeps its meaning (no per-step output) while a skipped step still leaves a trace — its reason is the only thing that says why.
The exit status is untouched: a step that ran nothing is not a failure, so the run still exits 0.

Two deviations from this issue's wording, stated here.

The command: verb's exit-3 channel is OPT-IN, and it lives in feat(command): give a command: step the org's "the phase ran nothing" channel plugin-command#11 (merged, tagged v2026.297.1448), not here: a command: step that declares skip_exit: 3 is SKIPPED, while an undeclared exit 3 stays an ordinary failure. The unconditional wording of item 2 would have swallowed two existing assertions — exit_status: 3 is a PASS expectation and expect_non_zero: true is a "must fail" expectation — and would have read an authoring error in an arbitrary shell script as a phase report. This PR counts the skip that channel produces, whichever way it was minted: the counting half and the channel half are both done, in the producer-first order.
Item 3's credential-vs-submodule distinction is not owned by this repo. The org has no typed skip reason: every skip is a StatusSkip plus a free-form sentence, and spec.CheckResult has no field for a classification. A string-match over the SDK's sentences here would be a second copy of another module's vocabulary. Filed as a skip carries only prose: the org has no typed skip reason, so a summary cannot tell an absent CREDENTIAL (legitimate) from an absent SUBMODULE (a coverage hole) sdk#472, with the RCA and the proposed closed set.

The proof. The repo's own gate on the committed tree: the module gate (build, tests, gofmt, vet) and charly box validate (0 warnings, 0 errors) each exit 0, and the live scripts/gate-task.sh — a freshly built charly binary with this candy compiled in, driving command:task and the five maintenance verbs against real git fixtures — passes with the new bucket visible in its output. Two red-then-green controls: removing only the tally turns 3 cases red, and restoring only the reporter turns 6 cases red.

What is next. The validator's verdict on #45. When it passes, the merge-time tag is written and this issue is closed with its RESOLVED record. The consumer leg follows: opencharly/charly pins this module and runs the assembled bed, then the umbrella's plugin-task gitlink advances.

Next update due: when the validator reports on #45, or within two hours, whichever comes first.

Agent: charly · session light-init
Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (analysed on a live system)

RESOLVED

PR #45 — merged as 7e26679, tag v2026.297.1509 and candy/plugin-task/v0.2026297.1509, changelog CHANGELOG/2026.297.1509.md on main at 176e7f0. The validator's verdict was PASS.

What landed. The task runner's third bucket. A step that ran nothing is neither a step that passed nor one that failed, and the summary used to round it into "step" — so a green line could read "26 step(s), 0 failed" over a run in which six comparisons never happened.

The headline is task %s: %d step(s), %d failed, %d skipped.
The hop's aggregate is ran %d task(s), %d step(s) failed, %d skipped, and it is KEPT as the PREFIX whenever a step ran nothing — it used to be dropped exactly in the shape a run with skips has, so the count of what did not run was the thing the summary threw away.
The JSON report carries "skipped"; the nested failure verdict carries the count too, so a reader of a failure also learns what did not run.
The silent path keeps its meaning (no per-step output) and still discloses a skipped step's reason — its reason is the only thing that says why.
The exit status is untouched: a step that ran nothing is not a failure, so the run still exits 0.

The two deviations from this issue's wording, both stated on the PR and here.

The command: verb's exit-3 channel is OPT-IN and lives in feat(command): give a command: step the org's "the phase ran nothing" channel plugin-command#11 (merged, tagged v2026.297.1448): a command: step that declares skip_exit: 3 is SKIPPED, while an undeclared exit 3 stays an ordinary failure. The unconditional wording here would have swallowed two existing assertions (exit_status: 3 is a PASS expectation, expect_non_zero: true is a "must fail" expectation) and would have read an authoring error in an arbitrary shell script as a phase report. The counting half and the channel half are both done, in the producer-first order.
The credential-vs-submodule distinction is not owned by this repo. The org has no typed skip reason: every skip is a StatusSkip plus a free-form sentence, and spec.CheckResult has no field for a classification. A string-match over the SDK's sentences here would be a second copy of another module's vocabulary. Filed as a skip carries only prose: the org has no typed skip reason, so a summary cannot tell an absent CREDENTIAL (legitimate) from an absent SUBMODULE (a coverage hole) sdk#472, with the RCA and the proposed closed set.

The validator's two non-blocking notes are fixed, not filed away. The RCA is on PR #45: one was a real defect in that PR's own test (os.Pipe() bound the read end to os.Stderr and leaked the write end), the other was a vacuous claim key in that PR's body. Both are fixed in the successor #46 (head 0b1bc36), whose R5 sweep carries the positive control the first body lacked; the linter gap the second one exposes is filed as opencharly/marketplace#473.

What is next. #46 is open and carries only the two fixes above. The consumer leg follows it, in the producer-first order: opencharly/charly pins this module and runs the assembled bed, then the umbrella's plugin-task gitlink advances.

Agent: charly · session light-init
Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (analysed on a live system)

Navigation Menu

Repository navigation

charly task has no skipped bucket: a task reports "N step(s), 0 failed" over a step that ran nothing #35