Skip to content

runCheckBed's report-FAIL propagation has no unit test: the branch that makes a verdict STRICTER is the one that can silently stop firing #137

Open

Description

@aitrawog

The gap

runCheckBed turns a child's report-FAIL into a FAIL of the step, and the unit tests stop
one layer above that wiring: TestCheckReportVerdictTreatsAFailedCheckAsAFail covers the
verdict function and TestStepVerdictFromReport covers the step record, but nothing
covers runCheckBed's own path
— the reportFail branch that clears ok, prints the
FAIL after … line and returns the error that fails the bed.

The validator raised it on the merged PR (#136, review of
b9546ce): "The bed_run.go report-FAIL propagation path is exercised only live, not by a
unit test of runCheckBed; the unit tests stop at stepVerdictFromReport. Acceptable
given the live gate, but a run-level test would harden it."

Why it is filed and not fixed in the PR that raised it: the PR is merged. The change it
carried is correct and complete without this test — the verdict logic is unit-covered at
both layers below the wiring, and the R10 beds ran live on the committed tree — so the gap
is genuinely separable from it. R1: a finding raised on a PR is fixed or filed, and a
finding that cannot be filed away carries its issue number on the thread.

What the test would prove

That a step whose child exits 0 while its own report says a declared check FAILED is
recorded as a failed step and fails the bed — the exact shape of the hole #134 closed:

  • the step's record carries ok: false and the failed check is named;
  • the run's headline reads FAIL, not PASS;
  • the bed's exit code is 2 (CheckFailExitCode), not 0.

The last one is the part that is unproven today: the reader's FAIL branch is exercised
through stepVerdictFromReport, and the bed runner's own failure path is exercised by
other failures (a non-zero child exit, a diagnostic error: line) — but not by this
one.

Why it matters beyond tidiness

The reader's whole direction of safety is "an unreadable report can withhold a credit,
never manufacture one", and the report-FAIL branch is the one place where a report makes
the verdict STRICTER. A wiring bug in that branch would not be a false PASS; it would be a
false PASS by omission — the branch silently stops firing and the step reads PASS again.
That is the same class of defect as #134 itself, one layer out.

Filed from the review of #136.

Activity

  1. aitrawog commented on Oct 25, 2026

    @aitrawog
    ContributorAuthor

    CLAIM — filed from the charly consumer leg (opencharly/charly#904); the hole is in the module this PR pins, and it is separable from the pin bump.

    Agent: charly

    Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated)

    Why this is filed and not fixed in the charly leg. The branch is runCheckBed's, inside candy/plugin-check. The charly leg carries the version; it does not carry the wiring.

    Why it is not blocking the charly leg. The verdict logic the branch guards is unit-covered one layer below, and the producer's R10 beds ran LIVE on the producer's committed tree — so the branch's behaviour is proven in its own repo, and the charly leg's own proof is a live differential of the verdict through charly check run (the same bed, two binaries: the before binary prints a bare PASS and exits 0 — the defect — and the after binary prints SKIP, names the check, and exits 3).

    Next update when the charly pin bump lands, naming the exact version this repo's release must carry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions