From 63df9b18fa28eed2bc7631e1644f8a63a48061ac Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 12:49:44 +0200 Subject: [PATCH 1/3] Disable quiesce_stops_the_machine and quiesce_wakes_on_the_last_park behind their filed defects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit quiesce_stops_the_machine: a third sighting of the same shape already tracked in issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md, now on the orchestrator's Fast tier for PR #566 at 74f7d717 — a diff that is host-side only — after 266 s, with no `stop:` record in its capture. quiesce_wakes_on_the_last_park: a distinct shape from issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md's empty-uart-after-a-clean-stop failure. Three orchestrator Fast-tier runs on branches that touch no quiesce code (539r7-fast.log, 555r2-fast.log, 559-fast.log) carry the identical `the stop gave up on 2 thread(s) that never reached a safe point`, `stop: 3 of 5 userland thread(s) stopped ... over 2 sweep(s)`. Filed issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md for it. A flaky test is disabled at once, never re-run. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- ...e-machine-stayed-up-beside-other-guests.md | 7 +++- ...last-park-gave-up-on-two-parked-threads.md | 33 +++++++++++++++++++ src/redlist.rs | 8 +++++ 3 files changed, 47 insertions(+), 1 deletion(-) create mode 100644 issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md diff --git a/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md b/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md index 436dffd5fe..68192f2dff 100644 --- a/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md +++ b/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md @@ -1,5 +1,5 @@ --- -status: open +status: expected-red kind: finding opened: 2026-09-26 --- @@ -26,3 +26,8 @@ never reported stopping: the guest asked for a reboot and stayed up`, after 266 s, and the harness's re-run alone was green. The host was loaded throughout by another worktree's spinner at 397% CPU, with the load average between 20 and 28. + +Again in the fast tier on PR #566's branch at `74f7d717`, a diff that is +host-side only: the same `QEMU never reported stopping: the guest asked for a +reboot and stayed up`, after 266 s, with the writers' progress lines and no +`stop:` record in its capture. diff --git a/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md new file mode 100644 index 0000000000..9e405c9ddc --- /dev/null +++ b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md @@ -0,0 +1,33 @@ +--- +status: expected-red +kind: defect +opened: 2026-09-28 +--- + +# `quiesce_wakes_on_the_last_park` gave up on two threads that never reached a safe point + +Three orchestrator Fast-tier runs, each on a branch that touches no quiesce +code, carry the identical failure: + +``` +FAIL quiesce_wakes_on_the_last_park: the stop gave up on 2 thread(s) that never reached a safe point: + stop: 3 of 5 userland thread(s) stopped across 2 cpu(s) in 2010 ms of a 2010 ms budget over 2 sweep(s), 0 of N userland block operation(s) still open; this reset lands wherever the other 2 are +``` + +`539r7-fast.log` (`Compiling toyos-build v0.1.0 (/Users/jan/Dev/jan/toyos-install7)`, +`Running tests/toyos.rs (target/debug/deps/toyos_build-eac527f418717d9c)`), block +count 294. `555r2-fast.log` (`Compiling toyos-build v0.1.0 +(/Users/jan/Dev/jan/toyos-reap)`, `Running tests/toyos.rs +(target/debug/deps/toyos_build-2ff7bec9bcd9b25c)`), block count 263. +`559-fast.log` (`Running tests/toyos.rs +(target/debug/deps/toyos_build-2ff7bec9bcd9b25c)`), block count 331. + +This is a different shape than +`issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md`, +whose sighting is `QEMU died before ===READY===` with an empty uart after a +stop that reported every thread stopped; here the stop itself reports two +threads still parked and gives up. + +**Exit**: the two threads' safe-point wait explained on a loaded host, and +the test green in a Fast tier beside other guests. Owner: the stop path, +`kernel/src/quiesce.rs`; held by the orchestrator. diff --git a/src/redlist.rs b/src/redlist.rs index 8fb71e1968..bfbd6a256e 100644 --- a/src/redlist.rs +++ b/src/redlist.rs @@ -63,10 +63,18 @@ pub const DISABLED: &[Disabled] = &[ test: "quiesce_dump_holds_the_stopped", issue: "issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md", }, + Disabled { + test: "quiesce_stops_the_machine", + issue: "issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md", + }, Disabled { test: "quiesce_wakes_on_the_last_exit", issue: "issues/build/quiesce-wakes-on-the-last-exit-lost-its-serial-ready-beside-other-guests.md", }, + Disabled { + test: "quiesce_wakes_on_the_last_park", + issue: "issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md", + }, Disabled { test: "sched_check_build", issue: "issues/build/the-pass-cost-gates-ci-sample-is-eight-days-stale-twice.md", From f27d77c94b5f869005112b2831bf6135a80790c7 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 13:19:28 +0200 Subject: [PATCH 2/3] Point quiesce_stops_the_machine at the slow first pass its capture shows, and file the harness's misreport quiesce_stops_the_machine was disabled behind a finding whose title and exit rested on the harness's message that the guest asked for a reboot and stayed up. PR #566's capture at 74f7d717 refutes that: writer 5's first pass ran from 2.170 s to 7.434 s, the job printed "5 of 6 writers reached their loop in 5s" at 6.874 s and exited 1, and it never printed "asking for the reset". No stop began. PR #524's capture at 235c5a5b shows the same with "3 of 6", and nightly run 36351950439 on PR #555 at d2656765 with "4 of 6". - The slow pass is the defect quiesce_dump_holds_the_stopped's issue already tracks, whose exit names a first write-and-fsync pass over 5 s. That issue is renamed to what both tests show, and gains these sightings. Both rows point at it. - The stops finding is folded into it and deleted. It carried no durable line for a module header; its three sightings move with their evidence. Its a58abf50 sighting also had no stop: record, and whether that job printed its give-up line was not recorded. - stopped_boot waits its whole QMP budget and then calls returned_to_firmware before it reads the console, so a job that never asked is reported as a guest that asked. In the #566 capture every scheduler heartbeat from 10.750 s to 253.244 s was idle, and the test went red after 266 s. Filed as tooling, held by the orchestrator. - The park issue is renamed: its records show 0 block operations open, so both threads were running, and one was the held thread, which last::hold keeps spinning while a sweep counts 2. It now names each sighting's PR and head, adds the 98e803cb stop that gave up, labels the dispose_yield suspect as a hypothesis, and records that woken_by_its_threads has no enabled caller. Its exit asks for an instrument that names each thread still running, and for that coverage back. - Deleted: the scratch log names and paths, "after a stop that reported every thread stopped", "on a loaded host", and the build/ park issue's "--known-red answers NO". Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- ...-out-the-reset-budget-and-says-it-asked.md | 26 ++++++++ ...st-its-serial-ready-beside-other-guests.md | 1 - ...-outlasts-the-jobs-five-second-spin-up.md} | 35 +++++++++-- ...e-machine-stayed-up-beside-other-guests.md | 33 ---------- ...-on-threads-running-beside-the-held-one.md | 61 +++++++++++++++++++ ...last-park-gave-up-on-two-parked-threads.md | 33 ---------- src/redlist.rs | 6 +- 7 files changed, 121 insertions(+), 74 deletions(-) create mode 100644 issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md rename issues/kernel/{quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md => a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md} (54%) delete mode 100644 issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md create mode 100644 issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md delete mode 100644 issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md diff --git a/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md b/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md new file mode 100644 index 0000000000..d1c5d345a1 --- /dev/null +++ b/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md @@ -0,0 +1,26 @@ +--- +status: assigned +kind: tooling +opened: 2026-09-28 +--- + +# A stopped boot whose job never asked waits out the reset budget and says it asked + +`stopped_boot` (`tests/common/power.rs`) serves the five quiesce tests that end +in a reset. It waits `qemu.budget(WAIT)` for QEMU's `SHUTDOWN` event and then +calls `returned_to_firmware` before it reads anything off the console. That +function turns every `None` into `QEMU never reported stopping: the guest asked +for a reboot and stayed up`. So a boot whose job exited without asking for the +reset waits out the whole budget and then reports that the guest asked. + +In PR #566's fast tier at `74f7d717`, `quiesce_stops_the_machine`'s job exited 1 +at 10.303 s without asking. The guest's scheduler then reported both CPUs idle +(`current=None`, `parked=2`) from 10.750 s through its last heartbeat at +253.244 s. The test went red after 266 s with that message. Every sighting in +`issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md` +carries the same message over a job that never asked. + +**Exit**: `stopped_boot` stops waiting when the guest's job ends without +asking, and its red names that job's exit and last error line. A job that exits +before asking is then red in seconds, for its own reason. Owner: +`tests/common/power.rs`; held by the orchestrator. diff --git a/issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md b/issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md index 8af3bd54ce..54d8ef7f73 100644 --- a/issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md +++ b/issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md @@ -13,7 +13,6 @@ until the stop waits on it alone`, `stop: 4 of 7 userland thread(s) stopped ... in 2010 ms of a 2010 ms budget`, `usb-quiesce: disk 0 SYNCHRONIZE CACHE ok` and `Rebooting.`, then `shutdown: /log did not answer in 2000ms`; the uart captured `nothing at all`. The harness's re-run alone was green in 2 s. -`cargo run -- --known-red` answers NO. The same shape as `issues/build/quiesce-wakes-on-the-last-exit-lost-its-serial-ready-beside-other-guests.md`, diff --git a/issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md b/issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md similarity index 54% rename from issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md rename to issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md index 25c97e6868..af7028ec9a 100644 --- a/issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md +++ b/issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md @@ -4,11 +4,20 @@ kind: defect opened: 2026-09-25 --- -# quiesce_dump_holds_the_stopped reds wide, with two USB transport breaks, and is green alone +# A `quiesce_writers` writer's first write-and-fsync pass outlasts the job's 5 s spin-up -Seen once, in the fast tier on the logd branch (PR #492), dev host: red wide -with `QEMU never reported stopping: the guest asked for a reboot and stayed -up`, then `ALONE ... GREEN`. +`quiesce_dump_holds_the_stopped` and `quiesce_stops_the_machine` both boot +`quiesce_writers`. It asks for the reset only once each of its six writers has +finished one pass: a create, 64 KiB of writes and an fsync. If a writer is +still in its first pass after 5 s, the job prints `quiesce_writers: of 6 +writers reached their loop in 5s` and exits 1 without asking, so no stop +begins. Every sighting below then reads `QEMU never reported stopping: the +guest asked for a reboot and stayed up`; that misreport is +`issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md`. + +`quiesce_dump_holds_the_stopped`, in the fast tier on the logd branch +(PR #492), dev host: red wide with `QEMU never reported stopping: the guest +asked for a reboot and stayed up`, then `ALONE ... GREEN`. The boot's console shows the stick's transport breaking twice on `SCSI 0x2a` (`no answer in the status phase in 2000 ms`, recovered each time), then @@ -40,6 +49,24 @@ after their ` 0` lines. Writer 1 printed pass 33 at 6.905 s, its passes 93 to 306 ms apart. The branch changes no kernel or guest code. A red on a one-guest lane is not the other suites' load. +**`quiesce_stops_the_machine`, PR #566's fast tier at `74f7d717`.** Writer 5 +began its first pass at 2.170 s (`quiesce-writer: 5 0`) and ended it at +7.434 s (`5 1`), 5.264 s later. The other five had ended theirs by 3.738 s. At +6.874 s the job printed `quiesce_writers: 5 of 6 writers reached their loop in +5s`, and at 10.303 s the runner printed `===TEST_END test_rs_quiesce_writers +exit=1===`. The job never printed `6 writers are running; asking for the +reset`, and the capture has no `stop:` record. Red after 266 s. + +The same test, with the same harness message and no `stop:` record: + +- PR #524's fast tier at `235c5a5b`, load average 20 to 28: after 266 s, with + `quiesce_writers: 3 of 6 writers reached their loop in 5s`. +- PR #555's nightly at `d2656765` (run 36351950439, `guest (3)`): after 47 s, + with `4 of 6`. +- PR #511's merged head `a58abf50`: after 306 s, with a `usb-storage` transport + break on `SCSI 0x2a` that recovered. Whether its job printed the give-up + line was not recorded. + **Exit condition.** Re-enabled when a reproduction names what holds a writer's first write-and-fsync pass for over 5 s while another writer passes in under a third of a second, and the fix is shown against it. Owner: the `/log` write and diff --git a/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md b/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md deleted file mode 100644 index 68192f2dff..0000000000 --- a/issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -status: expected-red -kind: finding -opened: 2026-09-26 ---- - -# `quiesce_stops_the_machine` stayed up after asking for a reboot, beside other guests - -Fast tier at `a58abf50` (PR #511's merged head; another worktree's FAT suite -ran on the host at the same time): `QEMU never reported stopping: the guest -asked for a reboot and stayed up`, after 306 s. Its capture holds the -writers' progress lines and a `usb-storage ... transport broke on SCSI 0x2a: -no answer in the status phase in 2000 ms` that recovered after one break. The -harness's re-run alone was green in 3 s (`stop: 9 of 9 userland thread(s) -stopped across 2 cpu(s) in 29 ms of a 2010 ms budget over 2 sweep(s)`), and so -was `cargo test --test toyos-build -- quiesce_stops_the_machine` alone -afterwards (EXIT=0). `cargo run -- --known-red` answers NO. - -Not shown: where the loaded run's stop went, since its capture has no `stop:` -record. - -**Exit**: the stop's record, or its absence, explained on a loaded run. - -Again in the fast tier on PR #524's branch at `235c5a5b`: the same `QEMU -never reported stopping: the guest asked for a reboot and stayed up`, after -266 s, and the harness's re-run alone was green. The host was loaded -throughout by another worktree's spinner at 397% CPU, with the load average -between 20 and 28. - -Again in the fast tier on PR #566's branch at `74f7d717`, a diff that is -host-side only: the same `QEMU never reported stopping: the guest asked for a -reboot and stayed up`, after 266 s, with the writers' progress lines and no -`stop:` record in its capture. diff --git a/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md new file mode 100644 index 0000000000..a6b4028856 --- /dev/null +++ b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md @@ -0,0 +1,61 @@ +--- +status: expected-red +kind: defect +opened: 2026-09-28 +--- + +# `quiesce_wakes_on_the_last_park`'s stop gave up on threads still running beside the held one + +Three Fast tiers carry the identical failure: + +``` +FAIL quiesce_wakes_on_the_last_park: the stop gave up on 2 thread(s) that never reached a safe point: + stop: 3 of 5 userland thread(s) stopped across 2 cpu(s) in 2010 ms of a 2010 ms budget over 2 sweep(s), 0 of N userland block operation(s) still open; this reset lands wherever the other 2 are +``` + +- PR #539 at `2d6d228f`. +- PR #555 at `d2656765`. +- PR #559 at `ac948e6a`. + +The earliest is PR #510 at `98e803cb`, recorded in +`issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md`: +`stop: 4 of 7 userland thread(s) stopped ... in 2010 ms of a 2010 ms budget`. +That boot then lost its READY, so the harness reported the READY and not the +stop. None of the four branches touches the guest's stop path. + +In the three sightings above, neither of the two threads was parked. Each +record has 0 block operations open, and `stop_if_blocked` stops every parked +thread outside a `block::OpenUpdate`, so the sweep counted both as running. One +of them is the held thread by construction: `quiesce::last::hold` yields until +the latest sweep counts 1 running, so a sweep that counts 2 keeps it spinning. +The defect is the other thread. No sighting can name it, because the `stop:` +record carries only counts. + +**Hypothesis, untested.** The hold's yield loop keeps its CPU busy. A Ready +thread queued on that CPU then runs only if `dispose_yield` +(`toyos-sched/src/cpu.rs`) re-inserts the spinner behind it. + +**What no enabled guest test checks while this is disabled.** Four of the six +quiesce guest tests are disabled: `quiesce_stops_the_machine`, +`quiesce_dump_holds_the_stopped`, `quiesce_wakes_on_the_last_exit` and this one. +`woken_by_its_threads` (`tests/common/power.rs`) has no enabled caller. So no +enabled test checks any of these: + +- that a band, a park or an exit wakes the stop, rather than its deadline; +- `in_flight == 0` with `begun > 0`; +- the thread census; +- the `console-queue-at-the-stop` drain. + +`quiesce_refuses_a_second_shutdown` stays green over a lost post. It judges the +stop only by `stopped_the_machine`, so a stop that spends its budget and then +finds everything stopped passes it. + +**Exit**: + +- an instrument that names each thread still running when the stop gives up, + with its name, tid, cpu and scheduler state; +- the mechanism it names fixed; +- this test green in a Fast tier beside other guests; +- an enabled guest test checking each claim listed above. + +Owner: the stop path, `kernel/src/quiesce.rs`; held by the orchestrator. diff --git a/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md deleted file mode 100644 index 9e405c9ddc..0000000000 --- a/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -status: expected-red -kind: defect -opened: 2026-09-28 ---- - -# `quiesce_wakes_on_the_last_park` gave up on two threads that never reached a safe point - -Three orchestrator Fast-tier runs, each on a branch that touches no quiesce -code, carry the identical failure: - -``` -FAIL quiesce_wakes_on_the_last_park: the stop gave up on 2 thread(s) that never reached a safe point: - stop: 3 of 5 userland thread(s) stopped across 2 cpu(s) in 2010 ms of a 2010 ms budget over 2 sweep(s), 0 of N userland block operation(s) still open; this reset lands wherever the other 2 are -``` - -`539r7-fast.log` (`Compiling toyos-build v0.1.0 (/Users/jan/Dev/jan/toyos-install7)`, -`Running tests/toyos.rs (target/debug/deps/toyos_build-eac527f418717d9c)`), block -count 294. `555r2-fast.log` (`Compiling toyos-build v0.1.0 -(/Users/jan/Dev/jan/toyos-reap)`, `Running tests/toyos.rs -(target/debug/deps/toyos_build-2ff7bec9bcd9b25c)`), block count 263. -`559-fast.log` (`Running tests/toyos.rs -(target/debug/deps/toyos_build-2ff7bec9bcd9b25c)`), block count 331. - -This is a different shape than -`issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md`, -whose sighting is `QEMU died before ===READY===` with an empty uart after a -stop that reported every thread stopped; here the stop itself reports two -threads still parked and gives up. - -**Exit**: the two threads' safe-point wait explained on a loaded host, and -the test green in a Fast tier beside other guests. Owner: the stop path, -`kernel/src/quiesce.rs`; held by the orchestrator. diff --git a/src/redlist.rs b/src/redlist.rs index 80bc753f4e..c59fb3b2b8 100644 --- a/src/redlist.rs +++ b/src/redlist.rs @@ -62,11 +62,11 @@ pub const DISABLED: &[Disabled] = &[ }, Disabled { test: "quiesce_dump_holds_the_stopped", - issue: "issues/kernel/quiesce-dump-holds-the-stopped-reds-wide-with-usb-transport-breaks.md", + issue: "issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md", }, Disabled { test: "quiesce_stops_the_machine", - issue: "issues/kernel/quiesce-stops-the-machine-stayed-up-beside-other-guests.md", + issue: "issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md", }, Disabled { test: "quiesce_wakes_on_the_last_exit", @@ -74,7 +74,7 @@ pub const DISABLED: &[Disabled] = &[ }, Disabled { test: "quiesce_wakes_on_the_last_park", - issue: "issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-two-parked-threads.md", + issue: "issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md", }, Disabled { test: "sched_check_build", From 44f7d79c5b11bed4afeb7fef8bbb08f430788edf Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 13:49:01 +0200 Subject: [PATCH 3/3] Rename the park issue to what its records show, and delete false inferences from it and the harness issue MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The park issue's slug and title said "threads running" beside the held one, but every record carries 2 thread(s) and one of the two is the held thread by construction — so the stop gave up on one thread beside it, and "running" never held as a scheduler state. Renamed to issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md and the redlist row moved with it. Deleted the park issue's false inference that 0 open block operations means neither thread was parked: stop_if_blocked refuses a thread parked in SYS_FSYNC's OpenUpdate, which adds 0 to in_flight, so logd parked between refused fsync attempts is a candidate the records cannot exclude. Added it as a second labelled hypothesis beside the dispose_yield one. Deleted the false claim that none of four branches touch the guest's stop path (PR #510 changes fat32_adapter's refused-write path, where an fsync park happens), and the stale count of disabled guest tests that PR #564 moves. Deleted the harness issue's false claim that returned_to_firmware runs before the boot console is read (power.rs judges the boot console and the drained tail first) and its false claim that every writers-issue sighting carries the same never-asked message (a58abf50's give-up line was never recorded). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- ...-out-the-reset-budget-and-says-it-asked.md | 15 ++++----- ...e-up-on-one-thread-beside-the-held-one.md} | 31 +++++++++++-------- src/redlist.rs | 2 +- 3 files changed, 25 insertions(+), 23 deletions(-) rename issues/kernel/{quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md => quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md} (60%) diff --git a/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md b/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md index d1c5d345a1..ef355adc12 100644 --- a/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md +++ b/issues/build/a-stopped-boot-whose-job-never-asked-waits-out-the-reset-budget-and-says-it-asked.md @@ -6,19 +6,16 @@ opened: 2026-09-28 # A stopped boot whose job never asked waits out the reset budget and says it asked -`stopped_boot` (`tests/common/power.rs`) serves the five quiesce tests that end -in a reset. It waits `qemu.budget(WAIT)` for QEMU's `SHUTDOWN` event and then -calls `returned_to_firmware` before it reads anything off the console. That -function turns every `None` into `QEMU never reported stopping: the guest asked -for a reboot and stayed up`. So a boot whose job exited without asking for the -reset waits out the whole budget and then reports that the guest asked. +`stopped_boot` (`tests/common/power.rs`) waits `qemu.budget(WAIT)` for QEMU's +`SHUTDOWN` event and then calls `returned_to_firmware`. That function turns +every `None` into `QEMU never reported stopping: the guest asked for a reboot +and stayed up`. So a boot whose job exited without asking for the reset waits +out the whole budget and then reports that the guest asked. In PR #566's fast tier at `74f7d717`, `quiesce_stops_the_machine`'s job exited 1 at 10.303 s without asking. The guest's scheduler then reported both CPUs idle (`current=None`, `parked=2`) from 10.750 s through its last heartbeat at -253.244 s. The test went red after 266 s with that message. Every sighting in -`issues/kernel/a-quiesce-writers-first-pass-outlasts-the-jobs-five-second-spin-up.md` -carries the same message over a job that never asked. +253.244 s. The test went red after 266 s with that message. **Exit**: `stopped_boot` stops waiting when the guest's job ends without asking, and its red names that job's exit and last error line. A job that exits diff --git a/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md similarity index 60% rename from issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md rename to issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md index a6b4028856..5a9246a9d9 100644 --- a/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md +++ b/issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md @@ -4,7 +4,7 @@ kind: defect opened: 2026-09-28 --- -# `quiesce_wakes_on_the_last_park`'s stop gave up on threads still running beside the held one +# `quiesce_wakes_on_the_last_park`'s stop gave up on one thread beside the held one Three Fast tiers carry the identical failure: @@ -21,23 +21,28 @@ The earliest is PR #510 at `98e803cb`, recorded in `issues/build/quiesce-wakes-on-the-last-park-lost-its-serial-ready-beside-other-guests.md`: `stop: 4 of 7 userland thread(s) stopped ... in 2010 ms of a 2010 ms budget`. That boot then lost its READY, so the harness reported the READY and not the -stop. None of the four branches touches the guest's stop path. +stop. -In the three sightings above, neither of the two threads was parked. Each -record has 0 block operations open, and `stop_if_blocked` stops every parked -thread outside a `block::OpenUpdate`, so the sweep counted both as running. One -of them is the held thread by construction: `quiesce::last::hold` yields until -the latest sweep counts 1 running, so a sweep that counts 2 keeps it spinning. -The defect is the other thread. No sighting can name it, because the `stop:` -record carries only counts. +One of the two threads is the held thread by construction: `quiesce::last::hold` +yields until the latest sweep counts 1 running, so a sweep that counts 2 keeps +it spinning. The defect is the other thread. No sighting can name it, because +the `stop:` record carries only counts. -**Hypothesis, untested.** The hold's yield loop keeps its CPU busy. A Ready +**Hypothesis A, untested.** The hold's yield loop keeps its CPU busy. A Ready thread queued on that CPU then runs only if `dispose_yield` (`toyos-sched/src/cpu.rs`) re-inserts the spinner behind it. -**What no enabled guest test checks while this is disabled.** Four of the six -quiesce guest tests are disabled: `quiesce_stops_the_machine`, -`quiesce_dump_holds_the_stopped`, `quiesce_wakes_on_the_last_exit` and this one. +**Hypothesis B, untested.** `tests/quiescelastcase/system.toml` starts `logd`, +which fsyncs `/log`. `begin_update`'s only caller is `SYS_FSYNC` +(`kernel/src/object/ops.rs:626`), and that update spans every retry, including +the park in `between_attempts`. A thread parked there is `Blocked` with +`MID_UPDATE`; `stop_if_blocked` refuses it (`toyos-sched/src/task.rs:450`), the +sweep counts it running, and it adds 0 to `in_flight` — so `logd` parked +between refused fsync attempts past the 2010 ms budget is a candidate the +records cannot exclude, and it sits on the same `/log` fsync path the writers +issue owns. + +**What no enabled guest test checks while this is disabled.** `woken_by_its_threads` (`tests/common/power.rs`) has no enabled caller. So no enabled test checks any of these: diff --git a/src/redlist.rs b/src/redlist.rs index c59fb3b2b8..d3bd7c673e 100644 --- a/src/redlist.rs +++ b/src/redlist.rs @@ -74,7 +74,7 @@ pub const DISABLED: &[Disabled] = &[ }, Disabled { test: "quiesce_wakes_on_the_last_park", - issue: "issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-threads-running-beside-the-held-one.md", + issue: "issues/kernel/quiesce-wakes-on-the-last-park-gave-up-on-one-thread-beside-the-held-one.md", }, Disabled { test: "sched_check_build",