Skip to content

Test suite, first pass: a disabled list replaces the quarantine, sysret waits on its report, update_* reboot through the power connector - #542

Merged
Japabu merged 17 commits into
mainfrom
wt/toyos-testwins
Sep 27, 2026
Merged

Japabu merged 17 commits into
mainfrom
wt/toyos-testwins

Conversation

@Japabu

@Japabu Japabu commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

The first test-suite PR from the owner-approved test plan. The owner rules that a red is a red: a failing test, flaky or not, is fixed or disabled at once with its own issue, and nothing re-runs a red away. This branch carries that ruling through the harness.

1. The quarantine becomes a disabled list (src/redlist.rs, tests/toyos.rs)

  • A row is a registered test name and the issue file that owns its defect. There is no says, no rate and no text matching.
  • A disabled test does not run. The harness has one selection closure, keep, and every entry point takes it: the ordinary run, gate A and the metal loop. Every run prints [toyos] disabled: <test> — <issue> once per row, before anything compiles.
  • Bad rows are refused. redlist::check refuses a row whose test nothing registers, whose issue is not an issues/<area>/<slug>.md file opening with status: expected-red, or that names a test twice. check_redlist runs it on the ordinary path before --list, --debug and --audio-gate can return, and inside --metal against the metal registry. Each refusal is pinned by src/redlist.rs's lib tests.
  • One lookup. redlist::disabled(rows, test) is the whole-name match the harness's skip, the duplicate-row refusal and --known-red all use.
  • Re-enabling: the change that fixes a test deletes its row. A disabled test never runs, so every row's issue names an owner and an exit condition that ends in a fix.
  • --known-red keeps its name and answers YES, disabled — it does not run. with the issue, or NO, not disabled.

2. The ALONE re-run is deleted

src/alone.rs, alone_line and its test, retry_task, the wide-run re-run loop in main, and src/ci.rs's ALONE filter are gone. On main the re-run printed a line and never turned a red green, so no verdict changes: a red is reported once, with the wide run's own sentence. The comments that described the re-run are cut, and so are the two issue files about the classifier it no longer has.

3. sysret_ss_reload waits on its report

The probe's line lands before ===READY===, and drain_until reads only lines after it, so the test always spent its whole drain ceiling. It now asks boot_log() first and drains only for a report still owed. The drain ends on any of the probe's three outcomes, each a const (SYSRET_SS_RELOADED, SYSRET_SS_NOT_RELOADED, SYSRET_SS_UNARMED), and a probe that could not arm is a red by that name. The claim is pace only: mutating if !log.lines().any(sysret_ss_reported) to if true stays green, because the drain then spends its ceiling and appends nothing. One run each on the dev host, a record and not a gate: 16 s before, 2 s after (40dd2f5).

4. shipped_config_boots waits for init's lines

The test waited for four daemons' markers, then checked each init: started <program> against a capture it had not waited for. Nightly 36328646395's guest (12) red ended at netd: ready, before init's line about filepicker. Each line is now awaited with qemu::await_marker. It is fixed, not disabled, so it has no row and its issue file is gone.

5. The disabled list, and the nightly that judged this branch

17 rows. Four are added by the verdicts on nightly 36328646395 at 059c5de (merge base a637f5c). That diff touches no kernel, guest, driver or system.toml code. Nothing was re-run.

test job failure line verdict and evidence issue
handle_kill_policy guest (9) 16 more killed processes left more live objects behind: [("SharedMem", 9, 10)] red on main: same sentence on main's scheduled nightly at 16d2e64 (36306830048, guest 8), at a4f68c5 (36314576406) and 8df1a02 (36320607027), each green on its re-run issues/kernel/handle-kill-policy-census-grew-one-sharedmem-on-two-nightlies.md
xhci_flap guest (2) 0 slot(s) enabled and never disabled ([]) after 4 replugs red on main's lineage at a rate: a4f68c5 (36314576406), f231c43 (36280285913), e4317d3, a55d62c; green at 16d2e64 and 1ce7183 issues/hardware/a-collapsed-replug-is-enumerated-only-when-another-port-event-arrives.md
quiesce_dump_holds_the_stopped guest (7) QEMU never reported stopping: the guest asked for a reboot and stayed up (quiesce_writers: 4 of 6 writers reached their loop in 5s) flaky: green at a4f68c5 (36314576406, guest 7), whose kernel differs from the merge base only in kernel/src/rootfs.rs, and at 16d2e64, 1ce7183, 8df1a02, fd62f56 issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md
quiesce_wakes_on_the_last_exit — not red in 36328646395 red on main's lineage at f231c43, 67a430c, a55d62c with the stopped-boot drain carried no kernel output at all (38 bytes); the per-test file already held two QEMU died before ===READY=== sightings issues/build/quiesce-wakes-on-the-last-exit-lost-its-serial-ready-beside-other-guests.md

The same nightly's other two reds are not rows:

  • audio (2), gate A: audio_tone_load.smp1 wake lateness: median 5765 -> 6625 (Mann-Whitney z=3.73 > 3.09). Red on main too: 1ce7183 (36290616312), c271588 (36297455432), a4f68c5 (36314576406). tests/audio-baseline.toml declares the sample void for timing under QEMU 11.1.1. The list cannot hold it: with audio_tone_load disabled, AUDIO_TESTS has one config, and Shard::keep gives shard 2/2 none, which tests/toyos.rs's "owns no audio config" assert turns red. That needs the workflow's audio matrix, which is outside this branch.
  • portability-windows: continue-on-error, the declared frontier of issues/build/the-build-system-does-not-compile-on-windows.md, red on every nightly including main's.

6. Prose that described a deleted mechanism

Passages in issues/ that claimed a redlist row, rate, Instrument, EXPECTED_FAILURES entry, quarantine or ALONE protocol that no longer exists are cut, and so is tests/audio-baseline.toml's sentence about the quarantine being read. issues/README.md and the pull-request template name the disabled list where they named the quarantine.

CLAUDE.md and tests/CLAUDE.md carry only deletions of now-false rules and one sentence stating the disabled rule; the orchestrator judges them:

  • CLAUDE.md:69: "instruments and " deleted.
  • CLAUDE.md:122: replaced by the one sentence, "A red test is a defect unless src/redlist.rs disables it with its issue (cargo run -- --known-red <test>); a flaky test is disabled at once, never re-run."
  • tests/CLAUDE.md:3: the QUARANTINE clause deleted.
  • tests/CLAUDE.md: three bullets deleted, "green alone is not therefore the host", "ALONE: GREEN — its Sched::Parallel is wrong" and "may itself be quarantined".

Gates at 126a594

gate exit
cargo test --lib 0 (379 passed, 1 ignored)
cargo test --workspace --exclude toyos-build 0
cargo run -- --clippy 0 (clippy: 10 invocations clean)
cargo test --test toyos-build -- _a_verdict (runs check_redlist on the real registry, every row included, then the two guest-free harness tests) 0 (2 passed, 2 total)

Guest runs, one at a time on the dev host

run expected exit
cargo test --test toyos-build -- shipped_config_boots green 0, PASS shipped_config_boots (3s), logd, compositor, soundd, netd, filepicker started
control: start.push("a-program-init-never-starts"), a checked patch at 7e8cf34, --no-run BUILD_EXIT=0, reverted and the tree clean red naming the program at the guest wait 1, STALLED: waiting for init: started a-program-init-never-starts (303 s)
cargo test --test toyos-build -- quiesce_wakes_on_the_last_exit at 7e8cf34 (every match disabled) not run, printed as disabled 1, [toyos] disabled: quiesce_wakes_on_the_last_exit — …, then No enabled test matches filter
cargo test --test toyos-build -- quiesce_wakes_on_the_last_ at 7e8cf34 (_park enabled, _exit disabled) only _park runs 0, running 1 tests, PASS quiesce_wakes_on_the_last_park, 1 passed, 1 total

Negative controls

Each a checked patch, shown to build, run, and restored with the tree clean.

control expected exit
stall_is_not_a_verdict's dispatch → panic!, plus a DISABLED row for it (40dd2f5) skipped, the disabled: line printed 0 (1 passed, 1 total)
the same panic with the row removed (40dd2f5) red on the panic 1
a row naming issues/no-such-issue.md, in the harness (40dd2f5) refused before any boot 1
sysret-ss: lines renamed in the kernel (40dd2f5) red by name at the drain ceiling 1

The src/redlist.rs lib tests stage each refusal separately (unregistered, missing file, issues/README.md, a path outside issues/, a second row, every non-expected-red status).

Oracle. None independent of the harness: the arms are the harness's own output and the tracker's own frontmatter. The nightly verdicts rest on other runs' logs, each at a named commit.

What I am unsure of

  • redlist::check does not refuse the converse: an issue at status: expected-red that no row names passes. No gate is added for it.
  • The control's red is a STALL after the 300 s guard, not an answer: a line init never prints can only end the wait by the guard.
  • Seventeen tests do not run. A disabled test proves nothing until its fix deletes its row.
  • audio (2) stays red on every nightly until gate A has a runner baseline or the audio matrix changes.

Nightly jobs this needs green

host, build, tcg, portability-linux, portability-macos, audio (1), audio (2), and guest (1) to guest (12). portability-windows is continue-on-error and not among nightly-red's needs. audio (2) is red on main; see section 5.

Net lines against main (git diff --shortstat origin/main...HEAD): 40 files, +519 −1560. src/ +199 −506, tests/ +132 −771, issues/ +185 −280.

🤖 Generated with Claude Code

Japabu and others added 3 commits September 27, 2026 14:15
…t each wait spans

The four update_* reds on main (nightlies 36290616312 and 36306830048) are
one defect. #527 moved the machine's stop from a `power` syscap held by the
caller to init's `power` connector; #530's tests/updatecase/system.toml still
granted toybox `syscap = ["power"]`, so `ssh … reboot` ran toybox's reboot
applet with no connector, it was refused and exited 1
(`exit: reboot pid=8 code=1` on the console), and the machine never reset.
Every test that reboots over ssh then waited out the 300 s GUEST_WEDGED
backstop, because the kernel's own 10 s reporter kept the console from ever
going quiet: STALL 303-304 s, or "did not stop at a reset within 300 s" on
the QMP hold. The three update_* tests that never reboot over ssh were green.
Reproduced on the dev host (TCG): the same STALL at 303 s.

The config now hands toybox `receives = ["power"]`, as every other config
that reboots over ssh does. All four pass locally: update_floor 27 s,
update_boots 90 s, update_falls_back 44 s, update_refusals 51 s.

And the waits are bounded by what each one spans rather than by the 300 s
backstop: the guest's stop by the bounds it declares (init's FLUSH_BOUND 5 s,
quiesce::PARK about 3 s, the xHCI BARRIER 4 s), plus panic-reboot-fast's 5 s
where a kernel dies, plus each boot priced by the ceiling wait_for_ready
holds a boot to (now one function, boot_ceiling). Liveness scaling as
everywhere, and never past GUEST_WEDGED. A red now carries the console since
the reboot, which is where a refused reboot says so.

Negative control: the stale syscap line restored as a checked patch reds
update_floor_is_the_images_own in 57 s against its 54 s bound (was 303 s),
quoting `exit: reboot pid=8 code=1`; EXIT=1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
The owner's rule, as a gate: a registration red in RED_STREAK consecutive
nightly runs on main is quarantined with an issue or reverted. The
nightly-red job gains a step that reads main's nightly runs through the
Actions API (`gh api`: the run list, each run's jobs, and the raw log of each
job that ended neither green nor skipped), takes every `FAIL`/`STALL` verdict
line `report_line` prints (never an `XFAIL`, which a quarantine row excused),
and reds naming each registration red in all of the newest RED_STREAK runs
and those runs. A run that judged nothing red ends every streak, so older
runs are only read while the newest has reds.

Two, because that is the reproduction a one-wide run no longer makes in-job
(the next commit drops its ALONE rerun): red on two runners on two nights is
a defect reproduced, and the first red has already raised the alarm a night
earlier. The job needs `actions: read` for the history, and gh >= 2.97 for
`gh api --allow-escape-sequences`, without which gh refuses a raw job log.

Run once off a runner against the live history (a throwaway ignored test,
not committed): it read runs 36306830048 and 36290616312 and named eight
registrations red in both — the four update_* the previous commit fixes, and
screen_diag_boot, usb_flush_optional, usb_reset_hands_devices_back and
usb_transport_break, none of them quarantined.

Negative control: a_registration_red_two_nightlies_running_is_red stages a
history crossing the streak and asserts the red; the judge mutated to find
no streak (checked patch, built, restored) turns it red, EXIT=101.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…uns nothing

sysret_ss_reload's probe line lands before ===READY=== on every recorded
boot, and drain_until reads only lines after it, so the test always spent
its whole drain ceiling (10 s scaled by width and host; 123-227 s in the
fast tier on the dev host). It now asks the boot log first and drains only
for a report still owed, ending on any of the probe's three outcomes; a probe
that could not arm is now a red by that name. One run each at width 1 on the
dev host: 16 s before, 2 s after.

Negative control: the kernel's three `sysret-ss:` lines renamed (checked
patch, the mutated kernel built and booted — its renamed line is in the
capture, restored) reds the test by name at the drain's ceiling, EXIT=1.

The ALONE rerun is skipped when the run is one wide. Every red of such a
run already had the host to itself, so the rerun cannot reach the finding it
exists for (a red only beside other guests: a wrong Sched::Parallel) and only
samples the same host and binary twice — a full second ceiling for any red
that hit one, about 1.3 ks of nightly 36306830048's guest time. Whether a red
on the one-wide nightly lanes reproduces is now the previous commit's red
streak. Each red still carries an ALONE line saying it was not re-run; a
wider run's reds, serial tail included, are re-run as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
@Japabu
Japabu marked this pull request as ready for review September 27, 2026 12:17
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #542 at 40dd2f5.

Gate. CI run 36318468510 at 40dd2f5: abi-split success; host is skipped on PRs and runs only in merge_group. Items 1–3 each carry a green arm with EXIT=0 in the body (dev host, TCG, each test run alone). Reviewed.

Net lines. +324 −39 in total.

Item 4 (the red-streak gate) is rejected by the owner and was not reviewed. It is listed under REMOVE.

BLOCKER

  • tests/common/update.rs:75 (applied at :226, and inside A_REBOOT and A_DEATH) — A_STOP is a timing verdict, not a hang bound.
    • It leaves out most of the stop. kernel/src/syscall/machine.rs quiesce() runs writeback::drain_all, vfs::lock().sync_all() and xhci::flush_disks between PARK and BARRIER. That sync is bounded only by block::DEADMAN (120 s).
    • It counts PARK as a bound. kernel/src/quiesce.rs's own doc calls PARK "a budget, and not a bound the kernel can prove … a thread can therefore outlast this".
    • host_scale floors at 1 and budget() has no width floor, and the rig's guest is smp 2. So the reset wait at :226 is a flat 12 s on the one-wide KVM lane. A slow sync prints a STALL, and under "a red is a red" that disables update_boots_the_new_kernel.
    • The literals 5 + 3 + 4 (+ 5) copy init's FLUSH_BOUND, quiesce::PARK, the xHCI BARRIER and panic_reboot's FAST_BOUND. Nothing ties them to those constants.
    • After Main's nightly: a vanished disk is refused at ROOT's hold instead of panicking, and the reds #506 and #527 left #535, a refused reboot fails at ssh_fire in 4 s, so on the merged tree these bounds have no negative control.
    • Fix, one of two:
      • delete Spans, A_BOOT, A_STOP, A_REBOOT, A_DEATH, Spans::ceiling and QemuInstance::boot_ceiling/boot_ceiling, and keep qemu::GUEST_WEDGED as every other wait does; or
      • derive A_STOP from the stop's real bound, with DEADMAN in it, and measure each wait's elapsed time against its ceiling on a KVM nightly lane of the merged branch.

NOTE

  • tests/toyos.rs:17010 — "sysret-ss: reloaded" and "sysret-ss: NOT reloaded" are spelled twice, here and in sysret_ss. Only the unarmed line got a const; give the other two consts beside it.
  • tests/toyos.rs:11558 — the mutation that matters, if !log.lines().any(sysret_ss_reported) → if true, stays green. The claim is pace only, and it stands on the one 16 s → 2 s reading.
  • tests/toyos.rs:21289 — the mutation if width == 1 → if true skips the rerun on wide runs too, and no test goes red. "A wider run's reds are re-run as before" rests on reading the code.
  • tests/common/qemu.rs:3505 — take_pending is new public API with one caller, an error path. drain_serial, which await_machine already uses, collects the same evidence. Fold it in, or drop it with the bound change.

REMOVE

Overlap with #535 (head 877b8c9)

SEND BACK

Japabu and others added 2 commits September 27, 2026 14:41
Both are sent back by the review of #542, and the owner rejects the first.

- The red-streak gate (`src/ci.rs`'s `nightly-red` second step, `RED_STREAK`
  and everything that read the run history, and nightly.yml's `actions: read`)
  goes whole. A red is a red: a flaky test is disabled at once with its issue,
  so nothing waits two nights to find out that it reproduces.
- `tests/common/update.rs` waits on the machine under `qemu::GUEST_WEDGED`
  again, as every other wait does. `Spans`, `A_BOOT`, `A_STOP`, `A_REBOOT`,
  `A_DEATH`, `QemuInstance::boot_ceiling` and `take_pending` go. `A_STOP`
  was a timing verdict and not a hang bound. It left out the stop's sync,
  which only `block::DEADMAN` bounds (120 s), and it counted
  `quiesce::PARK`, which that module calls a budget and not a bound. On the
  one-wide KVM lane it came to a flat 12 s.
- `since_the_reboot` goes with them. It existed to show a refused `reboot`,
  and #535's `ssh_fire` now refuses one by name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
The owner's ruling: a red is a red, and a flaky test is disabled at once
with an issue filed. The quarantine ran a listed test every time and let a
failure through when its text held one of the row's quoted fragments. The
disabled list is simpler.

- A row is a registered test name and the issue file that owns it, nothing
  else.
- A disabled test does not run. The harness's one selection closure (`keep`)
  leaves it out of every entry point: the ordinary run, gate A and the metal
  loop. Every run prints one `[toyos] disabled: <test> — <issue>` line per
  row before anything is compiled.
- `redlist::check` refuses a row whose test nothing registers, whose issue is
  not a file under `issues/`, or that names a test twice. The harness runs it
  against the registry, and `cargo test --lib` runs it against the tree.
- The change that fixes a test deletes its row, which re-enables it.
- All thirteen quarantine rows become disabled rows. Each one's issue file
  exists, and the nine that were `open` are now `expected-red`.
- `cargo run -- --known-red <test>` keeps its name, because some forty issue
  files cite it. It answers "YES, disabled" or "NO".
- What this made dead is deleted: `says` and the fragment matching, the gate
  that refused a harness-framing quote, `Verdict::Quarantined` and the
  `Option` rows on `Pass`/`Fail`, the XFAIL and "quarantined for something
  else" lines, the tally's `fired`/`quiet` and its "ok, NOT clean" status,
  and ci.rs's XFAIL filter. The harness's `quarantine_verdicts` and
  `quarantine_entries` go. What `quarantine_exit_status` held that did not
  concern the quarantine (exit 1, 2 and 0) is now `run_exit_status`.
- The CLAUDE.md sentences on the quarantine are one sentence each now, as the
  orchestrator authorized.

Two review NOTEs from #542 ride along:

- `sysret-ss: reloaded` and `sysret-ss: NOT reloaded` are consts beside the
  unarmed one.
- The one-wide rerun skip is `toyos_build::alone::reruns`, which a host test
  reaches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
@Japabu Japabu changed the title Test suite, first pass: sysret waits on its report, update_* reboot through the power connector, no ALONE rerun one-wide, and a red-streak gate Test suite, first pass: a disabled list replaces the quarantine, sysret waits on its report, update_* reboot through the power connector, no ALONE rerun one-wide Sep 27, 2026
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #542, round 2, at cb008fa.

Gate. CI run 36320222303 at cb008fa: abi-split success. host is skipped on pull requests and runs only in merge_group. The body records these exits: cargo test --lib 0 (398 passed); the guest-free harness tests run_exit_status and _a_verdict 0; the four update_* tests alone with --nightly 0 each; sysret_ss_reload 0. No hardware target is involved. Reviewed.

Net lines. +244 −627 in total. src/ is +115 −165 and tests/ is +113 −446. The author puts production at −37, which I accept from the numstat.

Round-1 BLOCKER

  • tests/common/update.rs A_STOP timing verdict: CLOSED. git diff origin/main...HEAD no longer touches tests/common/update.rs. git grep finds none of Spans, A_STOP, A_REBOOT, A_DEATH, boot_ceiling, take_pending or since_the_reboot. At cb008fa all four update_* tests pass alone with --nightly (EXIT=0 each, body).

The round-1 NOTEs are closed:

  • the three SYSRET_SS_* consts are in place;
  • alone::reruns has a host test, and its width > 1 → true mutation gave EXIT=101;
  • take_pending is gone.

The round-1 REMOVEs are closed:

  • src/ci.rs differs from main only in the verdicts filter;
  • nightly.yml has no diff;
  • the red-streak comment and update.rs:269 are gone.

BLOCKER

  • Eight of the thirteen DISABLED rows point at an issue with no exit condition, or at a file that covers many tests. Under the owner's ruling, the issue is the only thing that brings a disabled test back.
    • These files carry no exit condition and no owner for the disabled test:
      • screen_fatal_halt → issues/boot-media/screen-fatal-halt-…md;
      • short_sleep_livelock → issues/kernel/short-sleep-livelock-…md;
      • so_cache_refusals → issues/kernel/so-cache-refusals-…md:16, which says "Owed: a mechanism. Nobody has one.";
      • latency_wake → issues/build/latency-wake-…md;
      • hda_tone → issues/audio/hda-tone-phase-check.md;
      • desktop_window_child → issues/kernel/desktop-window-child-freeze.md.
    • Two rows point at files that cover many tests, and neither file has an exit condition for the disabled one:
      • console_line_atomicity → issues/build/parallel-tests-red-under-other-suites.md:222, a bullet in a 565-line file about dozens of names;
      • usb_disk_index_stable → issues/hardware/eleven-names-red-on-ci.md, which covers eleven names.
    • Setting those two files to status: expected-red takes all their other open work out of rg -l '^status: open'.
    • Fix: give each of the six its exit condition and owner. File a per-test issue for console_line_atomicity and one for usb_disk_index_stable, point the two rows at them, and put the two shared files back to status: open.

NOTE

  • tests/toyos.rs:20719 — redlist::check runs only on the ordinary path. --metal exits at :20597 and --audio-gate returns at :20697, both before it runs, so those two paths never validate the list. The body's "before any boot" holds only for the ordinary run.
  • tests/toyos.rs:20719 — the mutation |name| runnable.contains(name) → |_| true stays green:
    • the lib test runs check with |_| true (src/redlist.rs:110), which is also what the merge queue's host job runs;
    • none of the four negative controls is an unregistered row in the harness.
    • So a renamed disabled test is caught first by a nightly after it lands. Add that control to the body's table.
  • src/redlist.rs:77 — issues/README.md passes as an issue file, and it is the lib test's own valid fixture (:107). So does any file under issues/. Requiring issues/<area>/<slug>.md whose frontmatter says status: expected-red would make the README's expected-red row the one tie between the list and the tracker.
  • src/alone.rs:4 — the header's reason ("a red that had the host to itself already ran alone … cannot find the one thing it exists for") also covers the serial tail, and tests/toyos.rs:20932 still re-runs serial-tail reds on a wide run. The deletion it points to:
    • re-run only reds whose shared_the_host is true;
    • delete alone::reruns and its test;
    • delete alone_line's Verdict::Pass "alone both times" arm and its shared_the_host parameter.
    • That gives one rule, fewer lines, and no second ceiling spent on a red that already ran alone.
  • tests/toyos.rs:11551 — carried from round 1: if !log.lines().any(sysret_ss_reported) → if true stays green. The claim is about pace only, and the body says so.
  • Other open branches meet this one:

REMOVE

Each item is a line that describes the quarantine or a row that no longer exists.

  • CLAUDE.md:122 — ", never re-run". The harness still re-runs a wide run's reds alone (tests/toyos.rs:20942); :124's "never re-run away" is the accurate wording.
  • issues/build/the-shard-split-prices-a-boot-and-not-the-image-behind-it.md:123-128 — the sysret mechanism paragraph. "The fix is the test's (… or check boot_log first)" is this branch's change.
  • issues/hardware/eleven-names-red-on-ci.md:9-14 — "transcribed into src/redlist.rs, one row per measurement".
  • issues/hardware/eleven-names-red-on-ci.md:64-66 — "src/redlist.rs carries that row at 1 of 3 … retired there".
  • issues/kernel/desktop-window-child-freeze.md:90-96 — the says list and the XFAIL pointer.
  • issues/kernel/deferred-release-outlives-its-syscall.md:146-149 — "its rows stay, so a landing gate that hits it has a rate to check the red against".
  • issues/kernel/so-cache-refusals-saw-the-kernel-refuse-nothing-once.md:16-17 — "src/redlist.rs quarantines the name until then."
  • issues/audio/doom-sound-flood-played-full-scale-once.md:40-42 — "src/redlist.rs carries the rate … the two rows … are retired".
  • issues/build/latency-wake-reds-on-the-dev-host-at-a-rate.md:31-33 — "The rate is on the list — two src/redlist.rs rows …".
  • issues/build/the-pass-cost-gates-ci-sample-is-eight-days-stale-twice.md:37-39 — "Until then, a CI red under this name is this file and the redlist row, and the landing it dequeues is re-queued once".
  • issues/build/the-pass-cost-gates-ci-sample-is-eight-days-stale-twice.md:41 — ", which this file's quarantine row does not cover".
  • issues/boot-media/screen-fatal-halt-…md:27-29 and issues/kernel/short-sleep-livelock-…md:28-30 — "the redlist row (src/redlist.rs, …, Instrument::Ci) records …".
  • issues/build/the-console-input-path-can-stop-after-a-ps2-overflow.md:61-62 — "src/redlist.rs carries it as this name's first DevHostLoaded row".
  • issues/build/parallel-tests-red-under-other-suites.md:223 — "(its standing rows are the loaded dev host's, 1 of 3 there)".
  • issues/kernel/toyos-runs-on-arm64.md:332 — "src/redlist.rs quarantines per arch".
  • Present-tense claims of redlist rows that do not exist, which predate this branch:
    • issues/boot-media/kernel-log-file-…md:26-27,53-54;
    • issues/panic-path/a-double-panic-at-boots-edge-says-nothing-but-its-name.md:11,51,56;
    • issues/boot-media/log-flush-retry-two-older-failure-modes-with-no-home.md:38;
    • issues/build/parallel-tests-red-under-other-suites.md:71-85,106,150-151,278,520.

Answers to the brief

  1. Removals. No dead code is left, and nothing dangles: git grep finds no says, XFAIL, Quarantined, QUARANTINE or check_quarantine. The quarantine wording that remains is prose in issues/, listed under REMOVE. The CLAUDE.md files and .claude/ are clean. i8042_quarantine* is the kernel's i8042 quarantine, not the list.

  2. Issues behind the rows. Every row names an existing file. Exit conditions exist only for:

    • doom_sound_flood;
    • console_locale_detect;
    • handle_transfer and kill_while_blocked;
    • sched_check_build.

    The other eight are the BLOCKER. None of the 13 is fixed on Main's nightly: a vanished disk is refused at ROOT's hold instead of panicking, and the reds #506 and #527 left #535: its body's table, its commits and its diff name none of them, and no other open PR does either.

  3. The check.

    • It refuses a missing issue: lib test, and harness EXIT=1.
    • It refuses a duplicate row.
    • It refuses an unregistered name, but only in a guest-run harness process on the ordinary path, and no measurement covers that arm (NOTE).
    • It accepts any file under issues/ (NOTE).
  4. CLAUDE.md. Each edit is one sentence, and all are true except ", never re-run" (REMOVE).

  5. tests/updatecase/system.toml and Main's nightly: a vanished disk is refused at ROOT's hold instead of panicking, and the reds #506 and #527 left #535. git merge-tree --write-tree origin/nightly-green2 cb008fac exits 1 with that file its only conflict. Both sides set receives = ["power"], and only the comment differs. Take Main's nightly: a vanished disk is refused at ROOT's hold instead of panicking, and the reds #506 and #527 left #535's side (blob b8cf0b10) whole, and this branch's hunk disappears. After the merge this branch changes nothing about the update_* path, so Main's nightly: a vanished disk is refused at ROOT's hold instead of panicking, and the reds #506 and #527 left #535's own update_ measurements cover it. In QEMU, the loader writes boot variables on the running system's request and a failed pass falls to the entry behind its own or powers off; toyos-metal drives a boot through a machine running ToyOS alone #539 carries the same one-line change as well.

SEND BACK

Japabu added a commit that referenced this pull request Sep 27, 2026
Nightly 36314576406 at a4f68c5 reddened lan_swap and xhci_flap in guest (1)
and handle_kill_policy in guest (8). None of them is this branch's. Each is
filed where its code lives, with the recommendation that it goes on #542's
disabled list when that lands.

- xhci_flap: a lost wake in main's driver. Slot_gone's Teardown arm leaves
  the port Settled with the device in it and CSC unacknowledged, and poll
  steps no port unless one is dirty or outstanding. The same sentence was red
  on wt/toyos-lld at a55d62c (run 36287592139). On the dev host, QEMU
  11.1.1 TCG, a printed serial shows every other collapse stuck about 700 ms
  until the next cycle's edges. The committed four-cycle gate is green by
  parity. At CYCLES = 3 it is red (EXIT=1), and adding
  `self.ports_dirty = true;` after torn_down() makes it green (EXIT=0). All
  measurement patches were applied checked and reverted, and the tree is clean.
- lan_swap: the redial ceiling that is already recorded
  (issues/diagnostics/a-swaps-redial-asks-again-with-no-event-to-wait-on.md),
  reached on the 82574 bench. The branch changes nothing on that path.
- handle_kill_policy: the same failure text, byte for byte, on main's nightly
  at 16d2e64. Consistent with the deferred release this branch does not touch.

Named runs, dev host, one at a time, branch 877b8c9 / main 16d2e64:
xhci_flap 0/0, lan_swap 0/0, handle_kill_policy 0/0,
log_reserve_window_negative 0/0. The shard-8 `log-gate: FAILED` line is that
negative control's owed refusal, printed by a passing test on both trees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
Japabu and others added 2 commits September 27, 2026 16:33
…n and

owner, checked on every path, and the ALONE re-run machinery is gone

The BLOCKER: six issues that named no exit condition or owner now do
(screen_fatal_halt, short_sleep_livelock, so_cache_refusals, latency_wake,
hda_tone, desktop_window_child). console_line_atomicity and
usb_disk_index_stable get their own per-test issue instead of pointing at a
shared file with dozens of other names, and the two shared files
(parallel-tests-red-under-other-suites.md, eleven-names-red-on-ci.md) go back
to `status: open` now that their own open work is no longer masked by
`expected-red`.

redlist::check now requires an `issues/<area>/<slug>.md` path whose
frontmatter says `status: expected-red` — issues/README.md, which has no
frontmatter and sits directly in `issues/`, no longer passes. It runs on
every harness entry point, `--metal` and `--audio-gate` included, factored
into one `check_redlist` both the ordinary path and `--metal` call (`--list`,
`--debug` and `--audio-gate` now see it too, hoisted before their early
returns). It stages a name nothing registers first and refuses to trust its
own verdict on the real list until that reds for the right reason — the
negative control the round-1 mutation review found missing: `|_| true` in
place of the real `registered` predicate now visibly breaks the harness
instead of passing silently.

The owner's ruling goes further than the review's NOTE: the ALONE re-run
machinery is deleted outright rather than trimmed. `src/alone.rs`,
`alone_line` and its test, `retry_task`, the wide-run rerun loop, and every
doc comment that described re-running a red alone are gone; a red on any
width is reported red, once. CLAUDE.md's ", never re-run" is therefore true
now and stays; the two tests/CLAUDE.md sentences that still described the
`ALONE:` line are corrected to describe manual investigation instead of a
harness feature that no longer exists. The two issue files about defects in
the classifier `alone_line` no longer has are deleted with it — there is
nothing left for either to be about.

Every REMOVE the review named is applied: the stale sysret-mechanism
paragraph, the `says`/XFAIL pointer, and every present-tense claim that
`src/redlist.rs` still carries a rate, a `Finding::Seen`, an `Instrument::Ci`
row or a quarantine — the schema those describe predates the disabled list
and none of it exists any more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tests/updatecase/system.toml
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #542, round 3, at 7befb95.

Gate. CI on the PR: abi-split SUCCESS (run 36326774153), host SKIPPED — correct per ci.yml, which runs host only in the merge queue. No new host gates are claimed this round beyond round 2's (cargo test --lib, cargo test --workspace --exclude toyos-build, cargo run -- --clippy, cargo build --tests post-merge), all already reviewed at round 2. This push is exactly one commit, the merge of origin/main (a637f5c, #535) onto 72d8db5, resolving one conflict. Reviewed.

Round-2 BLOCKER

CLOSED. All eight rows now carry an exit condition and an owner:

  • screen_fatal_halt, short_sleep_livelock, so_cache_refusals, latency_wake, hda_tone, desktop_window_child each gained an Exit condition paragraph naming a reproduction and an Owner naming a subsystem/path.
  • console_line_atomicity and usb_disk_index_stable each got a new per-test file (issues/build/console-line-atomicity-loses-five-of-a-thousand-lines-on-ci.md, issues/hardware/usb-disk-index-stable-nothing-enumerates-on-the-first-controller.md), each with its own exit condition and owner; src/redlist.rs's rows point at them.
  • issues/build/parallel-tests-red-under-other-suites.md and issues/hardware/eleven-names-red-on-ci.md are back to status: open and no row in DISABLED points at either any more.

Round-2 NOTE, checked

  • check_redlist now runs on --metal (its own registry, tests/toyos.rs:20408) and on the ordinary path before --list/--debug/--audio-gate can return (:20468, hoisted above every early return). Confirmed by reading control flow. Still unexecuted through the harness this round — cargo test --test toyos-build needs the QEMU tier the body defers. Naming it for the orchestrator's tier run, as asked: run cargo test --test toyos-build -- redlist_negative_control (or whatever selects check_redlist's callers) and confirm it prints nothing about "did not red on its reason", i.e. exercise the embedded staged-row control at least once for real, not just cargo build --tests.
  • is_per_test_issue_path now refuses issues/README.md (lib test a_row_is_refused_for_an_unregistered_test_a_missing_issue_or_a_second_row) — closed.
  • By reading, the registered → |_| true mutation (round-1's finding) is now caught: check_redlist's embedded control calls redlist::check with the real registered closure against a fabricated unregistered name; under the mutation registered always answers true, so the row survives the "nothing registers it" arm and instead fails is_per_test_issue_path("issues/nowhere.md") (no slug segment) — a different string, which does not contain "nothing registers it", so check_redlist's own match falls into other => Err(...) and the run reds. This is real by inspection, not measurement — flagging it, per the brief, for the orchestrator's QEMU-tier pass rather than accepting it on reading alone for a change this shaped.

Merge (item 4)

Correct. git merge-tree --write-tree 72d8db56 a637f5cb produces exactly one conflict, tests/updatecase/system.toml; the merge commit's blob for that path (git ls-tree 7befb952 -- tests/updatecase/system.toml) is b8cf0b10, byte-identical to origin/main's blob at that path (git diff a637f5cb 7befb952 -- tests/updatecase/system.toml is empty). Every other file in the merge's diffstat is inherited from main untouched (tests/toyos.rs auto-merged with no conflict, matching round 1's finding).

Net lines against origin/main (a637f5c, which is also the merge base)

git diff --shortstat origin/main...HEAD: 33 files, +499 −921 net (−921).

  • src/: +183 −506 (net −323) — src/alone.rs deleted whole (−346 including its module doc), src/redlist.rs net +/−, src/lib.rs/src/main.rs trivial.
  • tests/: +181 −737 (net −556) — tests/toyos.rs −863/+868 net −? (see below), tests/test-durations −4/+... trivial.
  • issues/: +132 −174 (net −42).
  • CLAUDE.md, tests/CLAUDE.md, .github/pull_request_template.md: net 0.

BLOCKER

  • src/ci.rs:670,681,1006-1010 — dead "ALONE " handling the branch itself left behind while editing this exact function this round. git diff origin/main...HEAD -- src/ci.rs shows this branch touched verdicts() this round (dropping the XFAIL/quarantine arm) and, in the same edit, kept the "ALONE " filter and rewrote its doc comment to "a failure, and whether it survived being run alone." But grep -rn '"ALONE' tests/ finds no line the harness still prints that way — alone_line, the only emitter, is deleted whole this round by this branch's own hand. The filter branch is now permanently unreachable, the doc comment is a false claim about current behaviour, and src/ci.rs's own test the_summary_keeps_the_count_and_the_verdicts still asserts on the dead literal ("ALONE lan_talk: GREEN"). This is exactly what the owner's "no zookeeping code: shorter, simpler, faster" ruling and the growth criterion ("a branch that could delete more than it adds and does not goes back with the deletion named") are for: the branch deleted the emitter and, in the same function, in the same round, left the consumer un-deleted.
    • Fix: delete the "ALONE " arm from the filter, drop "and whether it survived being run alone" from the doc comment, and cut "ALONE lan_talk: GREEN" and its assertions from the_summary_keeps_the_count_and_the_verdicts (or the whole test, if nothing is left for it to check beyond FAIL/STALL/INVL, which verdicts's other tests already cover).

REMOVE

Per the owner's ruling that stale prose is corrected nowhere and deleted everywhere — every passage below used to name alone_line, retry_task, or an ALONE:/XFAIL line and was rewritten in place this round instead of cut. None of them is a one-clause invariant the code below it needs — each just narrates why a comparison or a decision used to matter under the deleted mechanism.

  • tests/common/qemu.rs:539-544 — without_stamp's doc, rewritten from citing alone_line to "decides whether two boots of one deterministic panic read as one defect or two." Delete; the function's own name and body say what it does.
  • tests/common/qemu.rs:555-559 — kernel_died_here's doc, rewritten from "alone_line compares those two sentences" to "the sentence has to read as one defect on both."
  • tests/common/qemu.rs:641-647 — WaitVerdict::sentence()'s doc, rewritten to drop "alone_line compares two runs of one defect on."
  • tests/common/qemu.rs:776-779 — inline comment in ceiling_self_check, rewritten from "alone_line is what compares the two" to "two boots of one panic read as two defects."
  • tests/common/qemu.rs:924-927 — inline comment, rewritten to drop "alone_line compares two runs of one defect on it."
  • tests/toyos.rs:434-440 — the RUST_SKIP comment on cache_eviction, rewritten from naming retry_task's shared-registry lookup and the ALONE: line to "the shared registry answered for one of these under a machine test's name."
  • tests/toyos.rs:1605-1609 — the stall_is_not_a_verdict comment, rewritten to drop "and whether the ALONE: line under a red is about the run it claims to be about."
  • tests/toyos.rs:18657-18660 — Outcome::stalled()'s doc, rewritten from "the exit code and the alone re-run both have to treat it identically" to "the exit code has to treat it identically."
  • tests/toyos.rs:20214-20218 — check_no_collisions's doc, rewritten from naming retry_task's shared-registry-first lookup and its ALONE: verdict to "a verdict and a duration label both have to identify exactly one execution."
  • tests/CLAUDE.md:8 — "a red that reproduces when run by itself on a loaded host means nothing" is ALONE: red again rewritten in place.
  • tests/CLAUDE.md:11 — "A red that goes green run by itself does not make its Sched::Parallel wrong" is `ALONE: GREEN — its Sched::Parallel is wrong` is a hypothesis, not a finding rewritten in place.
  • issues/audio/hda-tone-phase-check.md:10,15 — "tests/toyos.rs's EXPECTED_FAILURES names for hda_tone" / "Declared in EXPECTED_FAILURES": EXPECTED_FAILURES has not existed since 3c84037d (already on main before this branch), and this branch edits this exact file this round to add the exit condition — it should have cut the reference it was already touching.
  • issues/kernel/desktop-window-child-freeze.md:75,83,163 — three more EXPECTED_FAILURES mentions, same defect, same file this branch edits this round.

NOTE

  • tests/test-durations:138 — alone_line_reports_the_alone_run 0 names a deleted test; harmless (an unqueried name costs nothing per load_durations's own doc), but it is dead data in a committed file this branch's own deletion orphaned.

Answers to the round-3 judge

  1. Every round-2 BLOCKER row: CLOSED, per above.
  2. Re-run machinery: the mechanism (src/alone.rs, alone_line, retry_task, the wide-run rerun loop) is gone; its prose is not — eleven rewritten comments plus the src/ci.rs dead-code/false-comment pair listed above.
  3. check_redlist is on every entry point by reading (--metal, --audio-gate, --list, --debug, the ordinary run all pass through it before their own logic). The |_| true mutation and the unregistered row read as caught by the embedded control, by inspection only — unexecuted this round, named above for the tier run.
  4. Merge: correct, verified by git merge-tree and a direct blob comparison against origin/main.
  5. Net lines against origin/main: −921 (+499 −1420 raw), src −323, tests −556 (dominated by tests/toyos.rs), issues −42.

SEND BACK

Japabu and others added 4 commits September 27, 2026 17:09
…n-in-place

sentence the round-2 alone-mechanism deletion left narrating a mechanism that
is no longer there, instead of deleting it with the code.

Deletes the unreachable "ALONE " filter arm and its now-false doc claim in
src/ci.rs::verdicts, plus the dead "ALONE lan_talk: GREEN" literal and its
assertion in the_summary_keeps_the_count_and_the_verdicts. Deletes eleven
comments/doc paragraphs across tests/common/qemu.rs, tests/toyos.rs and
tests/CLAUDE.md that were rewritten instead of cut when alone_line/retry_task
went, the three stale EXPECTED_FAILURES mentions in two issue files this
branch already edits this round, and the orphaned
alone_line_reports_the_alone_run row in tests/test-durations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tly 36328646395

Nightly 36328646395 ran this branch at 059c5de (merge base a637f5c). Its
diff touches no kernel, guest, driver or system.toml code, and on main the
ALONE re-run printed a line but never turned a red green, so no verdict
below is this branch's.

- handle_kill_policy (guest 9): `16 more killed processes left more live
  objects behind: [("SharedMem", 9, 10)]`. Red on main at 16d2e64 (run
  36306830048, guest 8), at a4f68c5 (36314576406) and 8df1a02
  (36320607027), each green on its own re-run. Red on main at a rate.
- xhci_flap (guest 2): `0 slot(s) enabled and never disabled ([]) after 4
  replugs`. Red at a4f68c5 (36314576406), f231c43 (36280285913), e4317d3
  and a55d62c, all ancestors of main; green at 16d2e64 and 1ce7183. Red
  on main at a rate.
- quiesce_dump_holds_the_stopped (guest 7): `QEMU never reported stopping`,
  with `quiesce_writers: 4 of 6 writers reached their loop in 5s`. Green at
  a4f68c5 (36314576406, guest 7), whose kernel differs from the merge base
  only in kernel/src/rootfs.rs, and at 16d2e64, 1ce7183, 8df1a02 and
  fd62f56. Flaky.
- shipped_config_boots (guest 12): `init never said "init: started
  filepicker"`. Green at a4f68c5, 16d2e64 and 1ce7183 (guest 3). Flaky.
- quiesce_wakes_on_the_last_exit: not red in 36328646395. Red wide and green
  alone in PR #536's review of 7e2e104, with a TLB shootdown panic that is
  main's issues/kernel/a-shootdown-panicked-on-a-cpu-the-host-starved.md,
  which becomes its issue. Red on main's lineage at f231c43, 67a430c and
  a55d62c with `the stopped-boot drain carried no kernel output at all (38
  bytes)`, not shown to be the same defect.

The first three already had a per-test issue; each goes to expected-red with
this run's evidence and an exit condition and owner, and the two that asked
to be put on this list stop asking. shipped_config_boots gets a new file.

Nothing was re-run. The same run's audio (2) red (gate A,
`audio_tone_load.smp1 wake lateness: median 5765 -> 6625`) is main's too
(1ce7183, c271588, a4f68c5) and is not disabled: with audio_tone_load
off, gate A's shard 2/2 owns no config and asserts. portability-windows is
continue-on-error and red on every run, main's included.

Also: `redlist::disabled` takes the rows, so the harness's skip, the
duplicate-row refusal and `--known-red` share one lookup. The merge brought
tests/audio-baseline.toml's sentence about the quarantine being read, and
two issue passages still described deleted redlist rows; all three are cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #542, round 4, at b1aeafe.

Gate. CI run 36339013300 at b1aeafe: host success, the only check ci.yml runs on a ready PR. run_exit_status is unchanged since cb008fa, where the body records it at EXIT=0. The five reds are real: I read the logs of nightly 36328646395 at 059c5de, jobs guest (2), (7), (9), (12) and audio (2), and each carries the sentence the body quotes. Every commit the body cites as main's (16d2e64, a4f68c5, 8df1a02, f231c43, e4317d3, a55d62c, 1ce7183, 67a430c, fd62f56, c271588) is an ancestor of origin/main (git merge-base --is-ancestor). Reviewed.

Net lines. git diff --shortstat origin/main...HEAD gives 39 files, +594 −1485, net −891:

  • src/: +203 −506;
  • tests/: +159 −757;
  • issues/: +229 −219.

src/redlist.rs production grows by 28 lines (141 → 169): check, is_per_test_issue_path and is_expected_red moved in from the harness. I accept it.

Round-3 BLOCKER

  • src/ci.rs dead "ALONE " arm: CLOSED. verdicts filters only FAIL, STALL and INVL, its doc says only "a failure", and the_summary_keeps_the_count_and_the_verdicts has no ALONE literal. host is green at b1aeafe.
  • Round-3 REMOVEs are closed except tests/CLAUDE.md:8, which is half cut (below).
  • Round-2 REMOVE deferred-release-outlives-its-syscall.md:146-149 is still open (below).

BLOCKER

  • tests/toyos.rs:14763-14770 — shipped_config_boots is a test bug, and the branch disables it instead of fixing it.
    • The test waits for the four daemons' markers. It then checks each init: started <program> against the log it has already read, without waiting for it.
    • The guest (12) capture shows the race. init: started netd came 34 ms after netd's spawn line (0.780 → 0.814). The capture ends at filepicker's spawn at 0.887, before init's line about it, because netd: ready (0.852) was the last marker the test waited on.
    • This is the only gate on the shipped system.toml, and a two-line fix beats disabling it: in the loop, let line = format!("init: started {program}"); qemu::await_marker(&mut qemu, &mut log, &line, &line)?;.
    • Then delete the row and issues/build/shipped-config-boots-ended-before-init-said-it-started-filepicker.md.
    • Negative control: a checked patch that pushes a name init never starts onto start must red by that name at the guest wait.
  • src/redlist.rs quiesce_wakes_on_the_last_exit row → issues/kernel/a-shootdown-panicked-on-a-cpu-the-host-starved.md — this is a sibling of an issue the tree already has, and it is filed on a defect the body itself says is not shown to be this test's.
  • Exit conditions and owners. Before this branch, a quarantined test still ran. Now a disabled test never runs, so its issue is its only way back. An exit that re-enables a test on a diagnosis re-enables a test that is still red. Each exit below must end in the defect fixed and shown against the red's own shape.
    • issues/kernel/handle-kill-policy-census-grew-one-sharedmem-on-two-nightlies.md:42 — "or the red is attributed" re-enables on a diagnosis. The fix is the one deferred-release-outlives-its-syscall.md "What to do" names.
    • issues/audio/doom-sound-flood-played-full-scale-once.md:39 — "one sentence naming which of the three it is": a diagnosis, and the file names no owner.
    • issues/build/the-console-input-path-can-stop-after-a-ps2-overflow.md:74 (console_locale_detect) — "read whether RX_BYTES is still rising": a diagnosis, and no owner.
    • issues/build/the-pass-cost-gates-ci-sample-is-eight-days-stale-twice.md:33 (sched_check_build) — the exit is a fix, but the file names no owner.
    • Diagnosis-only exits:
      • issues/audio/hda-tone-phase-check.md:104 — "pins the … breaks to one side";
      • issues/build/console-line-atomicity-loses-five-of-a-thousand-lines-on-ci.md:23 — "identifies which of the two sides";
      • issues/hardware/usb-disk-index-stable-nothing-enumerates-on-the-first-controller.md:19 — "pins whether";
      • issues/kernel/short-sleep-livelock-stalls-on-ci-with-one-sleeper-never-returning.md:32 — "shows the fifth sleeper's own exit";
      • issues/kernel/so-cache-refusals-saw-the-kernel-refuse-nothing-once.md:18 — "reproduces … with the byte counts".
    • issues/boot-media/screen-fatal-halt-reds-on-ci-with-a-usb-storage-transport-break-during-boot.md:31 — its second arm, "the runner is shown to make progress through a broken transport", is a diagnosis.

NOTE

  • issues/hardware/a-collapsed-replug-is-enumerated-only-when-another-port-event-arrives.md:48-50 records a measured one-line wake fix (EXIT=0). xhci_flap is disabled with its fix already in hand, and the driver is outside this fence, so it is the orchestrator's to dispatch.
  • src/redlist.rs:123 — check never refuses the converse. An issue at status: expected-red that no row names passes. The mutation that stays green: delete the xhci_flap row and leave its file untouched. The fix that deletes a row then leaves its issue claiming a disabled test.
  • tests/toyos.rs:20249 — check_redlist's staged row tests BTreeSet::contains on every harness run. The |_| true mutation was a one-off measurement, and the refusal arms are pinned by the lib tests. It is a deletion candidate: the staged match and its doc paragraph.
  • tests/toyos.rs:20497 — the list cannot disable a gate-A config: with audio_tone_load gone, shard 2/2 trips "owns no audio config". So audio (2) is a standing red that no row can hold. Record that hole where the orchestrator's audio investigation lives.
  • The skip arm, a row making the harness skip a test and print disabled:, was last measured at 40dd2f5, before keep moved above the metal and gate-A paths. The b1aeafe _a_verdict run in the body does not show the [toyos] disabled: lines.
  • CLAUDE.md:69,122 and tests/CLAUDE.md:3,8,20 are agent edits to a CLAUDE.md. CLAUDE.md reserves their placement for the orchestrator.
  • Nothing is left that re-runs a red, retries one, or treats ALONE specially:
    • git grep finds no alone, retry_task, XFAIL or Quarantined in src/, tests/*.rs, tests/common or .github/;
    • src/heartbeat.rs:295's SHARD_8_ALONE is a recorded capture used as a fixture;
    • INVL's "re-run" is about a host suspend, not a red.

REMOVE

  • tests/CLAUDE.md:8 — the whole bullet: "green alone" and "none of this re-runs" describe the deleted re-run.
  • tests/common/qemu.rs:760 — ", so a re-run reads as a second defect".
  • tests/toyos.rs:20244-20248 — check_redlist's --metal sentence and its "Returns the refusal rather than exiting" sentence.
  • issues/kernel/deferred-release-outlives-its-syscall.md:81-84 — "handle_kill_policy is on the redlist already … re-adjudicating that row".
  • issues/kernel/deferred-release-outlives-its-syscall.md:143-149 — "its rows stay, so a landing gate that hits it has a rate …" (open since round 2).
  • issues/audio/idle-suspend-reds-on-a-loaded-host-and-on-main.md:56-59 — "whether the row records a soundd defect or a Sched::Parallel misclassification".
  • issues/hardware/eleven-names-red-on-ci.md:49-51 — "No entry here is a candidate for EXPECTED_FAILURES …".
  • issues/build/parallel-tests-red-under-other-suites.md:92-96 — "Its EXPECTED_FAILURES entry covers … do not widen it."
  • issues/build/parallel-tests-red-under-other-suites.md:415-418 and :452-455 — the "read the ALONE line under it" protocol and its "Consequence for the protocol".

SEND BACK

Japabu and others added 3 commits September 27, 2026 20:19
…it is a fix

Round 4 of #542's review.

- shipped_config_boots read each `init: started <program>` from a capture
  it had not waited for: the guest (12) red ended at `netd: ready`, before
  init's line about filepicker. Each line is now awaited, and the row and
  its issue file go.
- quiesce_wakes_on_the_last_exit's row points at its own per-test issue,
  which now carries the three main-lineage runs; the shootdown issue is
  back to main's text and `status: open`.
- Every disabled test's issue ends in a fix and names an owner: a disabled
  test never runs, so a diagnosis-only exit could never be met.
- check_redlist's staged row only tested a set lookup; the refusals are
  pinned by src/redlist.rs's lib tests. Deleted with its doc paragraph.
- Prose about the ALONE re-run, EXPECTED_FAILURES and redlist rows cut
  from issue files, tests/CLAUDE.md and one error string.
- CLAUDE.md and tests/CLAUDE.md keep only deletions of now-false rules and
  the one sentence stating the disabled rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found as the shipped_config_boots negative control's own output: a STALL
exits 1 and its summary still says "Re-run". Filed, not fixed: outside
#542's round-5 brief.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stall summary told the reader to re-run a red. Under "a red is a
red" that sentence is false, so it goes, and the issue filed for it
goes with it. Two disabled tests' issues named a code site with nobody
holding the fix; the orchestrator holds them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #542, round 5, at 58f1c35.

Gate. CI run 36341043735 at 58f1c35: host success, the only check on this PR (gh pr checks 542 exit 0). The guest measurements in the body are at 7e8cf34. The shipped_config_boots arm is exit 0 (PASS). The control is exit 1 (STALLED: waiting for init: started a-program-init-never-starts). The skip arm is exit 1 No enabled test matches filter, and the _park-only arm is exit 0. 58f1c35 deletes one say and changes no test body. Reviewed.

Net lines. git diff --shortstat origin/main...HEAD gives 39 files, +520 −1543, net −1023:

  • src/: +199 −506;
  • tests/: +135 −770;
  • issues/: +183 −264.

Since b1aeafe, production shrinks: check_redlist loses its staged row, and a summary sentence is deleted.

Round-4 BLOCKERs

  • shipped_config_boots disabled instead of fixed: CLOSED.
    • tests/toyos.rs:14765 now awaits each init: started <program> with qemu::await_marker.
    • The row and the issue file are gone, and git grep finds no citation of either (exit 1).
    • The green arm is exit 0. The control is exit 1 and names the missing line.
  • quiesce_wakes_on_the_last_exit on a sibling issue: CLOSED.
    • The row points at issues/build/quiesce-wakes-on-the-last-exit-lost-its-serial-ready-beside-other-guests.md.
    • That file is status: expected-red, carries the three main-lineage runs, names the orchestrator as owner, and has a fix-shaped exit.
    • git diff origin/main...HEAD -- issues/kernel/a-shootdown-panicked-on-a-cpu-the-host-starved.md is empty.
  • Exit conditions and owners: CLOSED for every file round 4 named.
    • Now fix-shaped, and each names an owner: handle-kill-policy, doom-sound-flood, the PS/2 console-input file, pass-cost, hda-tone-phase-check, console-line-atomicity, usb-disk-index-stable, short-sleep-livelock and so-cache-refusals.
    • screen_fatal_halt's second arm is deleted.
    • Three files round 4 did not name still fall short of the body's claim; see the NOTEs.
  • Round-4 REMOVEs: CLOSED. All ten are deleted.
  • Round-4 NOTEs.
    • Closed:
      • the staged row is deleted;
      • the skip path was re-measured at 7e8cf34;
      • the orchestrator accepted the CLAUDE.md hunks;
      • xhci_flap is now held by the orchestrator.
    • Still open: the converse check and the audio (2) hole. Both are carried below.

The four questions

  1. All round-4 BLOCKERs are closed.
  2. The 303 s STALL is an acceptable negative control.
    • An absent line has no event to wait on. userland/init/src/main.rs prints nothing after its boot list: its only lines are init: started at :1638 and the panic!("init: cannot start …") at :369.
    • So the guard is the only thing that can end that wait. It fails loudly, it names the line, and only the red path pays for it.
    • A fast answer needs a guest line. That is outside this fence; see the NOTE.
  3. No. Three places still say a red is noise, and two of them were rewritten by this branch (REMOVE below).
  4. Not yet. The body carries one false sentence, round chronology, a stale count and a stale title clause (REMOVE below).

BLOCKER

None.

NOTE

  • issues/build/latency-wake-reds-on-the-dev-host-at-a-rate.md:36-38 — the exit's first arm, "the bucket ceiling is raised and a re-measured base rate stays under it on a loaded dev host", re-enables on a rate and not on a fix. This is the one row whose exit still does. Delete the span from "the histogram question above is settled — either" through "or " so the exit is the p99 fix.
  • issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md:46-47 — "nobody is holding it yet". It is a sibling of the two files 58f1c35 moved to "held by the orchestrator", so body line 11 ("every row's issue names an owner") is false for it. short-sleep-livelock…:34, screen-fatal-halt…:34 and desktop-window-child-freeze.md:177 name a code site or a track, and no holder.
  • issues/boot-media/screen-fatal-halt-reds-on-ci-with-a-usb-storage-transport-break-during-boot.md:31-34 — the fix clause only applies if the stdout was blocked behind the transport break. If the reproduction shows the two are independent, the exit names no fix.
  • issues/audio/hda-tone-red-beyond-its-exemption.md — this status: open defect, a mid-tone silence in hda_tone, can no longer be seen at all: the whole name is disabled under hda-tone-phase-check.md. Nothing in either file says so.
  • tests/toyos.rs:20233 — with the staged row gone, check_redlist's predicate mutated to |_| true stays green on every path. The lib tests pin check, not the registry fed to it. The deletion was accepted in round 4; this is the one mutation left.
  • src/redlist.rs:119 — an expected-red issue that no row names still passes (carried from round 4).
  • The audio (2) hole is still recorded only in the PR body (carried from round 4). Gate A's shard 2/2 asserts it owns an audio config, so no row can disable audio_tone_load. issues/audio/gate-a-has-no-runner-baseline.md records the red but not this hole.
  • shipped_config_boots — a real start failure, init panicking at main.rs:369, also costs the full 300 s guard. One init line when its boot list is done would turn it into an answer in about a second. That line is a guest change outside this fence; it is for the orchestrator.
  • src/CLAUDE.md:39 — "re-run in isolation before believing any single red" is about build reds, outside this fence. It reads against the new CLAUDE.md:122; the orchestrator's call.
  • Many open issues/ files record an enabled test as "red beside other guests and green alone" and ask for a rate before a row, for example usb-short-read-reds-beside-other-guests-and-is-green-alone.md:38-41 and partition-claim-gives-up-…:36. Under CLAUDE.md:122 each of these tests is flaky and is owed a row now. That is the orchestrator's sweep, not this branch's.

REMOVE

  • tests/toyos.rs:19709-19710 — ", so this says nothing about the tree" in the per-test STALL line. The branch rewrote this line. It contradicts tests/common/qemu.rs:487 ("Still red"), and src/ci.rs::verdicts copies it into every CI job summary.
  • tests/toyos.rs:18648-18650 — "What it changes is only what the reader is told … nobody should bisect it". The branch rewrote this doc: it cut "A stall is red on exactly the same terms as any other red" and kept the claim that 58f1c35 deleted from the summary.
  • tests/CLAUDE.md:10 — "stash and re-run before believing a red is yours; ". It contradicts the new CLAUDE.md:122; the orchestrator places it.
  • issues/build/latency-wake-reds-on-the-dev-host-at-a-rate.md:31 — "So this is a rate on this host, not a classification and not a regression." The branch rewrote it, and it reads a red as host noise.
  • issues/audio/hda-tone-red-beyond-its-exemption.md:18-22 and :29-33 — the EXPECTED_FAILURES paragraph and "src/redlist.rs carries rows for the name … told whose it is". Both are false now: no such entry or rows exist, and the one row hides this defect.
  • PR body, "What I am unsure of", bullet 2 — "Its summary still says "Re-run"; filed as issues/build/the-stall-summary-tells-the-reader-to-re-run-a-red.md." It is false at 58f1c35: both the sentence and the file are gone.
  • PR body — round chronology:
    • "--clippy, --workspace and _a_verdict ran at 7e8cf34; 22d6f42 adds only an issue file, and cargo test --lib was run again there." That issue file no longer exists.
    • "The green arm ran at a working tree that still had check_redlist's staged row; the test body is the same at 7e8cf34."
    • ", earlier heads" in the heading "Negative controls, earlier heads".
  • PR body, last line — the net-lines count. It is already stale, and a count in main's record rots.
  • PR title — ", no ALONE rerun one-wide". The ALONE re-run is deleted at every width, so the clause misstates what main gets.

LAND AFTER NAMED CHANGES

Japabu and others added 2 commits September 27, 2026 20:57
…rator as owner

Deletes clauses that read a red as noise or describe mechanisms already gone
(the STALL line's "so this says nothing about the tree", the stalled() doc's
"nobody should bisect it", the EXPECTED_FAILURES/redlist paragraph in
hda-tone-red-beyond-its-exemption.md, and the histogram-rate reading in
latency-wake-reds-on-the-dev-host-at-a-rate.md). Four issue files that named
a code site or track but no holder now say "held by the orchestrator",
matching the two files 58f1c35 already moved there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu Japabu changed the title Test suite, first pass: a disabled list replaces the quarantine, sysret waits on its report, update_* reboot through the power connector, no ALONE rerun one-wide Test suite, first pass: a disabled list replaces the quarantine, sysret waits on its report, update_* reboot through the power connector Sep 27, 2026
tests/CLAUDE.md told the reader to stash and re-run a gate-A red, and
src/CLAUDE.md to re-run a build red in isolation. A red is a red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu enabled auto-merge September 27, 2026 19:04
@Japabu
Japabu added this pull request to the merge queue Sep 27, 2026
Merged via the queue into main with commit c551894 Sep 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant