Disable quiesce_stops_the_machine and quiesce_wakes_on_the_last_park behind their filed defects

[Japabu]

What changed, and why

quiesce_stops_the_machine is disabled behind the slow first pass its capture shows.
In PR #566's Fast tier at 74f7d717, the test's job gave up before any stop began:

• writer 5's first pass ran from 2.170 s to 7.434 s;
• at 6.874 s the job printed quiesce_writers: 5 of 6 writers reached their loop in 5s;
• at 10.303 s the runner printed ===TEST_END test_rs_quiesce_writers exit=1===;
• the job never printed 6 writers are running; asking for the reset.

PR #524's capture at 235c5a5b shows the same with 3 of 6. So does nightly run 36351950439 on PR #555 at d2656765, with 4 of 6.

quiesce_dump_holds_the_stopped boots the same job. Its issue already tracked this pass: its exit names a writer's first write-and-fsync pass that takes over 5 s. That issue is renamed to
issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md
and gains these sightings. Both tests' rows point at it.

The stops finding is folded into it and deleted.
issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md was a kind: finding, and its title and exit rested on the harness's message. It had no durable line for a module header, so its sightings move to the writers issue. Its first sighting, at a58abf50, also had no stop: record. Whether that job printed its give-up line was not recorded.

The harness's misreport is filed as tooling:
issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md.
stopped_boot waits its whole QMP budget. It then calls returned_to_firmware, and that function reports any None as a guest that asked for a reboot and stayed up. In the #566 capture, every scheduler heartbeat from 10.750 s to 253.244 s was idle, and the test went red after 266 s. The owner is tests/common/power.rs, held by the orchestrator.

quiesce_wakes_on_the_last_park is disabled behind
issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md.
Three Fast tiers carry stop: 3 of 5 userland thread(s) stopped ... in 2010 ms of a 2010 ms budget over 2 sweep(s), 0 of N userland block operation(s) still open: PR #539 at 2d6d228f, PR #555 at d2656765 and PR #559 at ac948e6a. The earliest stop that gave up is PR #510's at 98e803cb.

One of the two threads is the held thread, which quiesce::last::hold keeps spinning while a sweep counts 2. The issue does three things:

• it labels the dispose_yield suspect and the logd-parked-in-fsync suspect as two labelled hypotheses;
• it records that woken_by_its_threads has no enabled caller;
• its exit asks for an instrument that names each thread still running when the stop gives up, and for that coverage back.

The build/ park issue loses its --known-red answers NO line, which is no longer true.

Round 2, after review. The park issue is renamed again to
issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md:
every sighting carries 2 thread(s), one of them the held thread by
construction, so the stop gave up on one thread beside it — not "threads,"
and "running" never held as a scheduler state. Deleted from it: the false
inference that 0 open block operations means neither thread was parked
(stop_if_blocked refuses a thread parked in SYS_FSYNC's OpenUpdate,
which adds 0 to in_flight, so a logd fsync park is a candidate the
records cannot exclude — added as a second labelled hypothesis beside
dispose_yield's); the false claim that none of four branches touch the
guest's stop path (PR #510 changes fat32_adapter's refused-write path,
where an fsync park happens); and the stale four-of-six disabled-test count.
Deleted from the harness issue: the false claim that returned_to_firmware
runs before the boot console is read, and the false claim that every
writers-issue sighting carries the same never-asked message (a58abf50's
give-up line was never recorded). Merged origin/main.

Gates

gate	exit
cargo test --lib	0 (384 passed, 1 ignored)
cargo test --test toyos-build -- --list	0; both rows print disabled with their issue
cargo run -- --known-red quiesce_stops_the_machine	0, YES, disabled, the writers issue
cargo run -- --known-red quiesce_wakes_on_the_last_park	0, YES, disabled, the park issue
cargo run -- --known-red quiesce_dump_holds_the_stopped	0, YES, disabled, the writers issue (renamed)

Unsure

• The writers issue belonged to quiesce_dump_holds_the_stopped. It is renamed and retitled here, because a row for quiesce_stops_the_machine pointing at a file named for the dump test and USB transport breaks would name the wrong failure.
• No capture of the a58abf50 sighting was found, so that sighting is attributed to the slow pass by its shape alone.

Not run here

This PR changes no code. Guest tests are the orchestrator's to run.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j

[Japabu]

Review of #574 at f691600a, round 1.

CI host concluded SUCCESS at f691600a. The branch adds no test. Host gates rerun at this head: cargo test --lib EXIT=0 (384 passed, 1 ignored); cargo test --test toyos-build -- --list EXIT=0, and both rows print disabled; cargo run -- --known-red quiesce_stops_the_machine and -- --known-red quiesce_wakes_on_the_last_park EXIT=0, each YES, disabled, each pointing at the issue in its row.

Net: +47 −1. Tooling data +8 (src/redlist.rs), issue prose +39 −1, production 0, tests 0.

Judgements:

• Is the new park issue a second record of the same defect? No. The old file's second sighting (a7f423b5: 7 of 7 ... in 1 ms, empty uart) shows the lost READY without any stop giving up. The three new sightings reached READY and read a stop that gave up. So there are two mechanisms. But the old file's first sighting (98e803cb, PR fat32: a refused write is re-driven to a consistent state before anything else touches the volume #510, fat32 only: stop: 4 of 7 ... in 2010 ms of a 2010 ms budget) is already a stop that gave up.
• Is the park red main's? Yes. In QEMU, the loader writes boot variables on the running system's request and a failed pass falls to the entry behind its own or powers off; toyos-metal drives a boot through a machine running ToyOS alone #539 changes one kernel log line in the slot record and adds a recovery-stick option to the harness. tests: a pty tethers QEMU and the judge's servers to the harness #555 tethers QEMU on the host side. netd: a listener's socket that left Listen is handed over or listens again, and a refused accept leaves its owner a wake #559 changes netd, which the quiescelastcase boot does not start. None of them touches the guest's stop path. The 98e803cb sighting shows the stop giving up before any of the three branches existed.
• Is the park red the kernel's or the harness's? The kernel's. The failing line is the kernel's own stop: record, which the harness only parses. Each red lasts 4–5 s, so no harness budget is involved. The shape was identical all three times (3 of 5 stopped, 2 sweeps, the whole 2010 ms budget spent), and the other ~28 park runs in the orchestrator's logs record 5 of 5 (one 6 of 6, four on a 14/15-thread boot). The owner, kernel/src/quiesce.rs, fits: the stop and its last actuator both live there.
• Is the stops red attributed right? No; see the BLOCKER.

BLOCKER

• src/redlist.rs:68, issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md:30 — The test is disabled behind a defect that its own new evidence refutes. In 566r1-fast.log the job refused before any stop began:

  • writer 5's first pass ran from 2.170 s to 7.434 s;
  • at 6.874 s the job printed quiesce_writers: 5 of 6 writers reached their loop in 5s;
  • at 10.303 s the runner printed ===TEST_END test_rs_quiesce_writers exit=1===;
  • the job never printed 6 writers are running; asking for the reset, so it never asked for a reboot;
  • the harness still waited 266 s for a QMP reset, then printed ASKED_AND_STAYED_UP (tests/common/power.rs:39 prints it on any None, whatever the job did).

  The defects this capture actually shows are recorded nowhere: a stopped boot that waits its whole scaled budget for a reset its job never asked for and then says the guest asked, and a 64 KiB write+fsync pass that took over 5 s. The added sighting's "the same" is false. Fix: record what the capture shows, file the harness defect, and point the row at the issue that describes the observed failure.

NOTE

• issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md:1 — The file's first two sightings also had no stop: record, which fits a job that never asked for a stop. Its title and its exit ("the stop's record, or its absence, explained") rest on the harness's message. Its kind: finding has reached the next review that issues/README.md says promotes it or folds it.

• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md:1 — The slug says "parked", but the record shows 0 open block operations. stop_if_blocked stops every parked thread that is outside an OpenUpdate, so both of these threads were running. One of them is the actuator's held thread by construction: last::hold yields until RUNNING == 1, and a sweep that counts 2 keeps it spinning. So the defect is one unnamed thread. The file is new, so renaming it costs nothing now.

• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md:31 — The stop: record carries only counts, so no sighting can name the stuck thread. The exit needs an instrument that names each thread still running when the stop gives up (name, tid, cpu, state). Suspect for the owner: the hold's yield loop keeps its CPU busy, so a Ready thread queued on that CPU runs only if dispose_yield (toyos-sched/src/cpu.rs:1608) re-inserts the spinner behind it.

• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md:9 — No sighting names its PR and head, so "a branch that touches no quiesce code" cannot be re-checked from the record. The 98e803cb stop that gave up, recorded in the build/ park issue, is the earliest evidence and is missing here.

• tests/common/power.rs:235 — After this branch, woken_by_its_threads has no enabled caller: its callers at :225 and :384 are the three disabled tests. As a result, no enabled guest test checks any of these:

  • that a band, a park or an exit wakes the stop, rather than its deadline;
  • in_flight == 0 with begun > 0;
  • the thread census;
  • the console-queue-at-the-stop drain.

  quiesce_refuses_a_second_shutdown stays green over a lost post, because it spends the budget and then finds everything stopped. Four of the six quiesce guest tests are now disabled. For the owner, kernel/src/quiesce.rs.

REMOVE

• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md:17-23 — the scratch log names, local worktree paths, test-binary hashes and block counts: they rot and identify nothing.
• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md:25-29 — "after a stop that reported every thread stopped": false for that file's first sighting (4 of 7 ... 2010 ms of a 2010 ms budget).
• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md:31 — "on a loaded host": no sighting measured load.
• issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md:17 — "cargo run -- --known-red answers NO.": false from this branch on.
• issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md:16 — "cargo run -- --known-red answers NO.": false from this branch on.
• PR body — "A third sighting of the shape ... already tracks" and "whose sighting is an empty uart after a stop that reported every thread stopped": both false. "A flaky test is disabled at once, never re-run." restates CLAUDE.md. The body becomes main's record.

SEND BACK

[Japabu]

Review of #574 at f27d77c9, round 2: CI host has no conclusion at this head (status QUEUED, run 36415065241).

NOT READY FOR REVIEW

[Japabu]

Review of #574 at f27d77c9, round 2. Covers what changed since f691600a.

CI host concluded SUCCESS at f27d77c9, and the PR is not a draft. The branch adds no test. I reran the host gates at this head:

• cargo test --lib: EXIT=0 (384 passed, 1 ignored).
• cargo test --test toyos-build -- --list: EXIT=0. The three quiesce rows print disabled with their issue.
• cargo run -- --known-red for quiesce_stops_the_machine, quiesce_wakes_on_the_last_park and quiesce_dump_holds_the_stopped: EXIT=0 each, each YES, disabled.
• Worktree clean afterwards.

Net since origin/main: +127 −34. Of that, src/redlist.rs is +9 −1 and issue prose +118 −33. Production 0, tests 0.

Round 1 BLOCKER

• CLOSED — src/redlist.rs:68, stops row disabled behind a refuted finding. The --known-red quiesce_stops_the_machine log now names issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md. That issue records the A swap's redial dials until logd admits one, bounded by the swap's window alone #566 capture: writer 5's pass ran 2.170→7.434 s, 5 of 6 at 6.874 s, exit=1 at 10.303 s, and no asking for the reset. The harness defect is filed. One cited sighting I checked independently: nightly 36351950439 at d2656765, job guest (3) (conclusion failure). Its log carries {6.158 ... } quiesce_writers: 4 of 6 writers reached their loop in 5s, {7.284} ===TEST_END test_rs_quiesce_writers exit=1=== and FAIL quiesce_stops_the_machine (47s).

Round 1 NOTEs and REMOVEs

• All acted on.
• Each REMOVE was deleted. The stops finding and the old park file are gone whole, and git grep at HEAD finds none of their lines.
• I retract one round-1 NOTE (see NOTE 2): my claim that 0 open block operations means neither thread was parked. The branch copied that claim into the park issue.

Judgements asked for

• Is the fold one defect? Yes. Both tests boot quiesce_writers, and every placed sighting is the same SPIN_UP give-up (tests/toyos-rust-tests/src/bin/quiesce_writers.rs:86-92). The dump issue's exit already named a first write-and-fsync pass over 5 s.
  • The rename is legitimate under issues/README.md's slug-is-a-claim rule. The old slug's "reds wide" was refuted by its own 059c5de7 one-guest sighting, and "with usb transport breaks" by its 396f5b4d and 059c5de7 sightings.
  • The bare slug and the path of the old dump issue, the stops finding and the round-1 park file: 0 hits in git grep HEAD. On origin/main the only hit is the redlist row this branch moves.
  • Open PRs still cite them; see NOTEs 3 and 4.
• Is a58abf50 honestly labelled? Adequately. issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md:66-68 says its give-up line was not recorded. The harness issue contradicts that; see REMOVE.
• Is the harness issue accurate? Its mechanism is right: stopped_boot never checks whether the job asked before returned_to_firmware (tests/common/power.rs:288-293). It has an owner and an exit. Two of its sentences are false; see REMOVE.
• The park issue:
  • Sightings are named by PR and head. All four heads are commits of the PRs named.
  • The hypothesis is labelled.
  • The exit requires the instrument and the coverage back.
  • The file name is not yet final; see NOTE 1.

BLOCKER
(none)

NOTE

1. issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md (slug and title) — "threads" is plural, but every record is 2 thread(s), and one of the two is the held thread. So one thread is beside it. "Running" holds only as the stop: record's counter, not as a scheduler state (NOTE 2). The name must be final before Kernel: a kill never waits on its victim — the last thread out tears its process down #549 adds its sighting, so rename it now to what the records show: the stop gave up on one thread beside the held one.
2. issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md:26-28 — The record's 0 of N userland block operation(s) still open counts block::begin_operation guards (OPEN_OPERATIONS, kernel/src/block.rs:121-132), not OpenUpdate.
   • begin_update's only caller is SYS_FSYNC (kernel/src/object/ops.rs:626). That update spans every retry, including the park in between_attempts.
   • A thread parked there is Blocked with MID_UPDATE. stop_if_blocked refuses it (toyos-sched/src/task.rs:450), the sweep counts it running, and it adds 0 to in_flight.
   • tests/quiescelastcase/system.toml starts logd, which fsyncs /log.
   • So "logd parked between refused fsync attempts past the 2010 ms budget" is a live candidate the records cannot exclude. It sits on the same /log fsync path the writers issue owns.
   • The exit's per-thread scheduler state decides it. The prose that excludes it is a REMOVE below.
3. PR tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 (open, 617e934f) conflicts with this branch:
   • It edits the old dump-issue path (sets status: open and rewords the exit).
   • It deletes quiesce_dump_holds_the_stopped and quiesce_wakes_on_the_last_exit and their rows.
   • It moves quiesce_wakes_on_the_last_park to Tier::Nightly.
   • Whichever of tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 and Disable quiesce_stops_the_machine and quiesce_wakes_on_the_last_park behind their filed defects #574 lands second must keep the writers issue expected-red, which src/redlist.rs:133-180 enforces. It must also drop the dump test from that issue's first line, and settle the park issue's exit "green in a Fast tier", which becomes unreachable. The sequencing is the orchestrator's.
4. PR Track: the supervisor is host-tested and owns the stop #575 (open, d346d9de) adds issues/isolation/the-supervisor-is-host-tested-and-owns-the-stop.md, which cites the old dump path and the deleted stops finding. No gate checks an issue path outside src/redlist.rs, so whichever lands second must point both at the writers issue.

REMOVE

• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md:26-28 — "In the three sightings above, neither of the two threads was parked. ... so the sweep counted both as running." — false inference (NOTE 2); round 1 planted it.
• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md:24 — "None of the four branches touches the guest's stop path." — misleading: fat32: a refused write is re-driven to a consistent state before anything else touches the volume #510 changes kernel/src/fat32_adapter.rs and toyos-fat32's refused-write path, where an fsync's OpenUpdate parks.
• issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md:38-40 — "Four of the six quiesce guest tests are disabled: ... and this one." — restates src/redlist.rs and goes stale when tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 deletes two of them.
• issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md:9-10 — "serves the five quiesce tests that end in a reset" — a count tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 moves.
• issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md:11 — "before it reads anything off the console" — false: power.rs:286 judges the boot console and :292 the drained tail before :293. What it never reads is whether the job asked.
• issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md:19-21 — "Every sighting in ... carries the same message over a job that never asked." — false for a58abf50, whose give-up line the writers issue says was not recorded.
• PR body — "It then calls returned_to_firmware before it reads the console" and "With 0 block operations open, neither thread was parked." — both false, as above; the body becomes main's record.

LAND AFTER NAMED CHANGES