No QEMU test measures time, and audio is judged on metal only

[Japabu]

The owner's rulings, applied: a QEMU test measures no time, and the harness's hang ceiling is its only clock. An in-guest deadline becomes a wait on the event it stood for. A property that cannot be kept without time moves to a METAL row if it matters on the T14 and is deleted otherwise. Audio is judged on metal and nowhere else. The QEMU machines keep their audio devices, with -audiodev none, for the device-isolation tests that use them as DMA masters. Doom's sound and music tests are gone.

The ceiling

A hang is red like any other red. "N of those reds are the ceiling" counts both of the ceiling's reds: a wait's STALLED:, and the backstop's timed out after (qemu::TIMED_OUT) for a guest still talking. a_stall_stays_red holds each.

In-guest waits

Each wait below now blocks until its event arrives: Poller::wait with no timeout, recv in place of recv_timeout, or a retry loop with no bound. Where a stimulus needs a thread parked first, the test asks the kernel's roster (tests/toyos-rust-tests/src/roster.rs) instead of sleeping. Census readings wait until the census settles (census_wait.rs).

test	what it waited on	now
kill_while_blocked	a killer's go byte, GONE_AFTER_NS, ENDS_WITHIN	arm 4 kills the spinner itself; the kill is the event
handle_kill_policy	POLL_ANSWER, SETTLE_SAMPLES	answers awaited; census settled, then released
handle_lifetime, shm_release_reclaims	census read after sleeps	census_wait
inbox_cancel_wakes	a 30 s harness override, and a sleep before the close	the close follows the roster showing the waiter blocked
copy_out_races_munmap, tls_dtv_race	bounded spins	unbounded
compositor_stall	HANDSHAKE_WAIT, a timed stream	waits for the hang-up; the stream runs until a second window gets its frame
blockd_io	CLAIM_RETURN, AIMED, SILENCE_ENDS, the bench's MB/s	unbounded claim retry, waits on replies; throughput deleted
fat_backing_revoked, gpu_set_resolution	5 s retry bounds	unbounded
i8042_mouse, i8042_keyboard, input_events, window_drag, window_child, window_wake	run ceilings and poll timeouts	waits on the event
winit_loop	CEILING; idle wakes counted over IDLE_WINDOW	ControlFlow::Wait; the idle-wake stage is deleted
locale_gate, launcher_refusals, mmap_prot, swap_claim_astray, the quiesce binaries	answer and stop budgets	waits with no deadline; quiesce_twice's threads sleep for Duration::MAX, because nanosleep is the syscall quiesce-last-park holds the named thread in
every netd binary, netd_stream.rs	WITHIN, await_until deadlines, connect timeouts	await_until waits for its check and nothing else; connects pass NO_DEADLINE
netd_lookup_let_go	WITHIN, the schedule's lower bound	the lookup waits for its answer or its close
netd_hostile_peer	2 s and 10 s answers; a timed burst	each ruling awaited; the burst half is deleted
netd_caps	a burst to a black hole, held by a 4 s connect timeout	the cap is filled with established connections to the host's server
process_lifecycle, process_stats	200 ms sleeps before the stimulus; a per-call ns floor	roster waits; the floor is deleted
futex_wake_counts	PARK_MARGIN; 2 s, 1 s and 5 s bounded loops; 120 ms settles	counts, claim_semantics and orphaned_by_unmap proceed once each waiter has counted itself in before its futex_wait and the roster shows them blocked; the spinners, once the roster shows one running on every other CPU; the rest wait for their returns with no bound
std_threading	a child lingering by the clock; a 5 s poll bound	the child is held by its stdin
sched_stress	500 ms and 200 ms poll timeouts; burners timed by Instant	port b is asked once port a's poll completed; burners run until killed; the lag is read once min_vruntime moved
munmap_reissues_read_window	a 30 ms sleep over 12 attempts	one attempt, unmapped once the roster shows the reader blocked
home_overwrite_zero	a 300 ms sampled window	one reading; the host judges the device
panic_halts_first	STARTED_WITHIN	restored without it
test-runner's log_gate	CEILING	deleted
ftruncate_flush_race	a truncate that waited 150 ms for the stalled flush	deleted; the host still requires HELD, refuses BROKEN and reads the settled size
spawn_cwd, log_flood, demand_window_race, nmi_window_spin, wall_clock_now	timing prints; MAX_CLOCK_SKEW_SECS	deleted; the clock is only ordered
compositor_stall	its presenter spun yield_now beside the writer until the ring filled	parked until the writer's first fill unparks it

Host verdicts

test	what it read over a clock	now
usb_boot_stick_pulled	probes answered, and frame batches, per 120 ms drain	every probe's answer awaited and read, then two frame batches, before the pull, after it and after the replug
lan_dhcp_lease	a 500 ms drain after ready	awaits the lease line
lan_no_lease	a LEASE_BOUND + 10 s drain	drains until netd's ready after the give-up, under the harness ceiling
the swap rig	the first ask retried for 30 s, 1 s apart	awaits sshd's listening on port 22, then asks once
pci_function_is_exclusive, bar_placement_is_proven	500 ms drains	await init's refusal to test-runner, and netd's ready
iommu fault boots	a 2 s must-not window	qemu::await_reset under panic-reboot-fast: the capture closes when the fatal path's reset ends QEMU
iommu foreign and scanout arms, gpt, log_partition_identity	500 ms and 2 s drains	deleted
the nested fault	drains	await_reset, as above
pre_idle_wedge_speaks	a 1.5 s must-not window	deleted: the wedge never returns
screen_diag_boot	a flat 5 s sleep	awaits exit: toybox
screen_pager_keys	a quiet window	PageUp, and every page after the first is exactly one back
screen_recoverable_untouched	AFTER_END 1.5 s	through one dump taken after the child's end
screen_blocked_dump	the dump surviving a repaint	that half is deleted
desktop, kernel_heartbeat, nvme wide sector	flat drains	await_guest on the frame or heartbeat lines
metal_sim_pointer_churn	more than two frames in a 2 s interval	a frame that drew the cursor
kernel_log_file, the usb stick with no write cache	"promptly", within 10 s	polled under the harness ceiling
log_stream_stalled_reader	logd's STALLED judged as a deadline	the flood's bound is the harness ceiling
usb moved stick	a 5 s drain after the end line	ends on each shape's last line and the held call's end
usb late disk, refused replacement	1.2 s sleeps	await the disk's line, the port's disconnect and the bind
usb flush fails	500 ms drains per probe	the give-up, then each probe's answer, read
usb refused INQUIRY	a 1.2 s drain for the slot's return	awaits the slot disabled after the INQUIRY line
xhci_deaf_registers	the refusal ≥ 1.5 s after the ask	deleted
usb ladder	unverified − broke ≥ 1.4 s	deleted
xhci_slow_connect	the floor and a timing print	deleted
quiesce_wakes_on_the_last_park	the stop completed, and woken before its budget	not judged in QEMU: the kernel's 2010 ms budget is its own clock; metal::Readback::stop_completed holds every metal boot to it
i8042_health_cadence, i8042_quarantine_verdict, the idle-trip check	counts over clock windows	deleted
i8042_quarantine	a program's timed keys	a QMP key, then the quarantine line
panic_key_holds	the panel held for a span	deleted
boot_from_power_on	its TSC total against the host's	deleted; the loader's and the kernel's accounts are still held to each other
wall_clock_*	drift ≤ 300 s	the guest's reading is at or after the base and no later than the base plus the QEMU process's life
log_ring_keeps_the_owners_slots	"answered" inferred from the stop line	unanswered when init says it waited the flush out, on the console or in /log
mdns, update, virt panel, devices, lan	timing prints	deleted

A usb probe names a binary no image carries, on purpose: the spawn's refusal is a kernel record (spawn: /system/bin/<probe>: not found), and that record is the load. absent_probe ends its wait on test-runner's answer for the probe, whatever it is, and reds naming it unless it is error=entity not found.

A fatal path's CPU never halts: halt_all_cpus claims the panel whether or not a framebuffer exists, then holds it (hold_the_panel, page_forever) until the reboot bound resets the machine. So await_reset ends when -no-reboot turns that reset into QEMU's exit, or at once on a refused line. The exit counts only with a success status and the raw panic: no key inside the bound line on the console or in the 16550's log.

metal_sim_compositor_stall: the test was wrong

The nightly red it: the watcher's Frame never came. Three diagnostic arms, run by the orchestrator (notiming-r4/mutations/c-diag*.patch), decided which side was wrong:

• c-diag adds prints only. No diag ring full appeared, and diag N writes, 0 fills ran to 19922944 writes. The ring never filled, so the watcher never presented.
• c-diag-blocked blocks the presenter on the writer's first fill. It was green. It presents only after that fill, so the ring filled.
• c-diag-nostream streams nothing. It was green.

That is the measurement's row "diag ring full appears and the run is green". The presenter spinning yield_now beside the writer held the writer below the compositor's drain rate. The presenter now parks until the writer's first fill unparks it. It prints compositor stall: the ring is full; a second window presents under it before it presents, so a hang after that point is named by the test's last line. The whole-change control is 925e1a66, red as above. r5/c-drain-never-ends shows the fixed test still reds on a real stall (see the arm table).

A ceiling red prints the test's lines

check_rust_result printed the test's stdout for an exit code and for no exit code, but printed only the kernel's last 60 lines for a ceiling red. On a hang those 60 lines are heartbeats. So m-revoke-no-wake red at 300 s with nothing to say which wait it was in. One print now serves every red, and each carries stdout:.

userdev_dma_fault awaits netd's end

On main, foreign_fault drained 2 s into a capture it dropped at return, and that drain swallowed netd's lines after the staged fault. This branch deleted the drain, so netd's panic on its refused claim (Card::begin_pass: netd: this NIC's claim refused an interrupt read: Io) reached run_test's window after log_origin's spawn record, and must_be_clean_apart_from red on it (562r4 nightly, 562r6 Fast). The test now awaits that line and exit: netd pid=… code=101 with await_guest, before run_test.

• Negative control: r7/take-record-no-refusal deletes the faulted() refusal in pcidev::take_record. netd then never ends, and the wait reds. The mutated tree builds (cargo run -- --build-only --kernel-feature boot-actuators --kernel-feature test-actuators: EXIT=0, restored clean), and the orchestrator's run of it at 6079c8c1 is EXIT=1, FAIL userdev_dma_fault: STALLED: waiting for netd's end on its refused claim — it never stopped talking and never got there (562r7, below).
• Oracle: a recorded real failure. 562r6-fast.log at ba14967e carries both lines, after the DMA FAULT that ends the boot log.

Disabled on main, changed here

main disabled these tests, so this branch's changes to them run nowhere. Each disabling issue records its change, so the change returns with its test. The merge deletes the sentences in those issues that the merged tree makes false.

• log_ring_keeps_the_owners_slots reads an unanswered flush off init's FLUSH_WAITED_OUT, or off a stop line missing from /log. It no longer reads it off the kernel's sync time.
• usb_transport_break loses its 2 s staged-break check.
• quiesce_wakes_on_the_last_park: stopped_the_machine and woken_by_its_threads are no QEMU verdict here, so the failure its issue quotes no longer reds it.
• quiesce_stops_the_machine: quiesce_writers' 5 s spin-up is deleted. Its issue's sightings are all that give-up.
• ftruncate_flush_race: the guest's 150 ms verdict is deleted. Its issue's reds are all that verdict.

For the last three, the premise the disabling issue records is gone at this head. Whether each test is green is a guest run's to say.

To metal, off QEMU

watchdog_fed, dump_nmi_probe, blocking_read_window's held window, tlb_shootdown_waits and syscall_cost. Each has a METAL row and a METAL_ONLY reason. The new boots are priced in tests/metal-profile.toml, and each parameter is FLASHABLE. The QEMU registrations of latency_wake and tlb_shootdown_cost are deleted; their metal rows already existed.

Restored

• panic_halts_the_others_first: QEMU is the verdict. After the fatal CPU's line past stop_other_cpus (panic_reboot::arm's), info registers -a must show every vCPU but one halted with IF clear. The wait is guarded by GUEST_QUIET. The fatal CPU holds its panel under the shipped 60 s bound, so the machine is still up to be asked. Non-vacuity: a sibling made records before the fatal one. There is no record-order check: with stop_other_cpus emptied the console also goes quiet, so no sibling record ever arrives to be counted.
• hda_two_live_refused, under -audiodev none. The wait ends on whichever sink soundd took.

Metal audio judges

The host test metal_audio_judges runs every judge against a log crafted to pass it and logs crafted to fail it.

• job_window ends at soundd's flush on the last client leaving, not at the next spawn.
• hda_tone also holds underruns to 0.
• audio_idle_suspend reds on soundd: resumed with no client.
• null_sink_shipped_client is renamed shipped_client_departures, because it shares its boot with hda_tone.

Kept

sched_check_build asserts that the CPUs whose reports PassCostReport::parse reads are exactly the default SMP.

Deleted code

• Doom's --sound-stress and --music-check, the stalled-consumer actuator and MIXED_PERIODS, with tests/doomcase, tests/doommusiccase, their rows and the binaries.
• usb-slow-device: SLOW_TRANSFER_NS, held_event, slow_device_would_have_answered and the slot_id they alone called.
• sched_fast_health, and toyos-quiesce's spent_its_budget.

Issues

• issues/build/timing-verdicts-ruled-off-qemu-have-no-metal-arm.md lists every property deleted or moved without a metal row, including desktop_audio_client's three verdicts, inspect_plays' period sum and boot_from_power_on's host comparison. It also lists every clock a QEMU test still waits on without deciding a verdict (paces, drains of another thread's work, and product clocks a boot races), and the one that does: census_wait::settled's 10 ms window.
• a-megabyte-written-to-the-stick-starves-a-tone-beside-it exits on a METAL row, and so do gate-a-suspend-structure-verdict-unread, desktop-session-put-26ms-of-silence and logging-records-from-every-producer…'s preallocation item.
• Closed:
  • hda-tone-phase-check: its capture is gone.
  • idle-suspend-reds-on-a-loaded-host-and-on-main: its subject was the QEMU arm.
  • doom-sound-flood-played-full-scale-once.
  • i8042-health-cadence-counted-three-lines-once-on-mains-nightly.
  • harness-reads-after-a-flat-half-second-drain: every flat drain before a judgement now waits on its line. What rg 'drain_serial\(Duration::from_millis' tests/common still finds is the pace inside an event wait: await_guest, pkg's window_seen and three usb polls.
• Citations of the deleted tests, of gate A and of usb-slow-device are deleted with them.
• Each disabling issue in "Disabled on main, changed here" says what this branch changed.

No gate enforces the timing rule, and none is added.

Merged with main at 807f4561 (#564, the measured schedule)

• A test tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 deleted stays deleted where this branch had edited it: home_budget_refusal_retried, usb_disk_index_stable, wall_clock_file, quiesce_wakes_on_the_last_exit and quiesce_dump_holds_the_stopped. This branch's fsync-budget-spent: staged a spent budget on kernel record went too, since only home_budget_refusal_retried read it; kernel/src/object/ops.rs is main's again. wallclock::after_the_base stays, because wall_clock_zone reads it.
• A row both branches changed takes this branch's Sched and comment and tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564's tier: netd_hostile_peer is Parallel at Weekly, hda_two_live_refused Weekly, panic_halts_the_others_first Nightly, userdev_dma_fault Nightly. The rows tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 left to this branch keep this branch's form.
• The harness's own checks are toyos-checks libtest tests in tests/checks.rs. a_stall_stays_red replaces stall_is_not_a_verdict there. i8042_quarantine_verdict went with the idle-trip check. metal_audio_judges is one of those tests now, not a guest row: cargo test --test toyos-checks metal_audio_judges. The audio-judge arms below went through the guest harness's dispatch at f260f3e7.
• --audio-gate is gone and --weekly stays. A reach is refused beside --metal only. schedule and --list hold no AUDIO_TESTS.
• Two issue sentences that the merged tree made false are deleted.
• No in-guest deadline came back. Against 6079c8c1, the merge adds no Instant, Duration, recv_timeout, elapsed or deadline line to tests/toyos-rust-tests or tests/common. The Duration lines it adds are the synthetic values of the host checks in tests/checks.rs.
• No harness helper was left dead. With tests/common/mod.rs's allow(dead_code) taken off, cargo check of both test targets names only the three that issues/build/the-harness-carries-three-helpers-nothing-calls.md already names (EXIT=0 each, attributes restored).

Merged with main at bc9ccad8 (#572, #580, #579, #549, #566)

Merge commit 6c8ff3f4.

• src/redlist.rs: user_copy_races_munmap and quiesce_leaves_the_volume_whole were added on both sides. Each keeps one row, pointing at the issue file on main. main's new rows stay. The rows for tests or issues this branch deleted go: hda_tone, doom_sound_flood, latency_wake and sched_check_build. lan_swap goes too, because main deleted its issue.
• The two issue files both sides added take main's text.
• tests/common/power.rs: main's woken_by_the_held_thread, which the new quiesce_wakes_on_the_last_teardown shares, without the two clock verdicts this branch took off QEMU (stopped_the_machine and woken_by_its_threads). Both tests are disabled on main.
• tests/common/qemu.rs: qemu_command takes main's firmware_vars and has no audio_wav. Its too_many_arguments allow goes: at seven parameters the lint does not fire, and --clippy with --all-targets -D warnings is clean without it.
• kill_while_blocked.rs is main's. After Kernel: a kill never waits on its victim — the last thread out tears its process down #549 a kill no longer parks in retire_task, so this branch's arm-4 doc was false. main's arm has no clock either.
• check_rust_result keeps this branch's form. It already prints the stdout that main added.
• The merge brought in no in-guest deadline. What it adds under tests/toyos-rust-tests and tests/common:
  • kill_ends_every_wait: a 10 ms pace while it polls the roster for a parked thread, with no deadline;
  • std::thread::sleep(3600 s) as the thread-join child's park;
  • the ceiling_self_check case from Disable flaky netd_refused_accept (hung 2341s waiting on a wake) #579.

r7/take-record-no-refusal still applies at 6c8ff3f4 (git apply --check: EXIT=0).

Merged with origin/main at e3a1cdc8 (#578, #581) in 0ca1fdb2; no conflicts, and the merge does not touch tests/toyos.rs.

At 0ca1fdb2, each gate EXIT=0:

• cargo test -p toyos-build --lib: 394 passed, 3 ignored.
• cargo test --test toyos-build -- --list: EXIT=0, 235 Fast, 142 Nightly, 115 Weekly, 3 Local, 21 disabled, no i8042_health_cadence; every run_machine_test/run_screen_test registration and dispatch arm agree both ways.
• cargo test --test toyos-checks: 11 passed.
• cargo run -- --clippy: EXIT=0.
• cargo run -- --build-only: EXIT=0.

Metal audio judges, red arms (run here at f260f3e7; tests/common/audio.rs has not changed since): 15 mutations of tests/common/audio.rs. Each was applied as a checked patch, built, run as cargo test --test toyos-build -- metal_audio_judges, and restored byte-identical. The unmutated run is EXIT=0. Every arm is EXIT=1, naming the crafted log that passed:

arm	named case
the hda-path line dropped	a tone off no hda path
underruns != 0 off	a tone short of periods
the null-sink must-not dropped	a tone on the null sink
resumes < 2 → < 1	a stalled client whose second stream never resumed soundd
the staging check off	a stalled client soundd filled no period for
the deferred check off	a stalled client soundd deferred for after the next job began
the repeated-completion must-not dropped	a stalled client over a repeated completion
job_window ends at the next spawn	a stalled client soundd deferred for after the next job began
the removal count off	a departure soundd never reported
the departure vocabulary off	a departure soundd did not establish
the death word off	a death soundd claimed beside a departure it established
the start-line must-not off	an idle soundd that started the device
the full-ring check off	a log stall that never filled the ring
the ledger balance off	a log stall that lost a line silently
unsaid == 0 off	a log stall that counted nothing unwritten

Run by the orchestrator

Agents never boot QEMU.

• Fast: cargo test --test toyos-build.
• Nightly: cargo test --test toyos-build -- --nightly.
• Weekly: cargo test --test toyos-build -- --weekly.

Measured at 925e1a66 by the orchestrator:

• EXIT=0: lan_no_lease, usb_flush_optional and usb_boot_stick_pulled.
• The Fast tier: green.
• EXIT=1, each red as its row below names: m-fsync-staged-every-flush (its test is gone with tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564), m-counts-probe-for-park, m-runner-answers-otherwise (both tests) and m-revoke-no-wake. m-revoke-no-wake red at the 300 s ceiling. Its guest lines were not in the report, which the ceiling red now prints.
• The full nightly: 494 passed, 2 failed. The failures were metal_sim_compositor_stall (timed out; fixed above) and userdev_dma_fault.

Measured by the orchestrator at 2a9c77ee (562r5): the nightly EXIT=0 (493/493); metal_sim_compositor_stall EXIT=0; r5/c-drain-never-ends and r4/m-revoke-no-wake EXIT=1. At ba14967e (562r6): the Fast tier EXIT=1, 386/387, the one red userdev_dma_fault (fixed above).

At 6079c8c1 (562r7), before the #564 merge: userdev_dma_fault EXIT=0 (test result: ok. 1 passed, 1 total (9.7s)); r7/take-record-no-refusal EXIT=1, FAIL userdev_dma_fault: STALLED: waiting for netd's end on its refused claim — it never stopped talking and never got there (0 passed, 1 failed, 0 invalidated, 1 total (304.4s)); the Fast tier EXIT=0, 388 passed, 388 total (190.6s), 104 held back for the nightly tier. Nothing has booted the merged head fe1c4d99.

Measured at ee646399 by the orchestrator, each EXIT=1: m-stop-other-cpus-empty, m-dma-fault-no-halt (both tests), m-recursive-fault-reports, m-quiesce-twice-parks, m-counts-probe-for-park and m-unmap-revokes-no-futex, with the reds below.

The patches are in the scratchpad and are not committed: notiming-r4/mutations/, notiming-r3/mutations/ and notiming-r2/mutations/. Each applies to 2a9c77ee. Each mutated tree was built there and restored clean (notiming-r5/arms-build.txt, logs notiming-r5/build-<patch>-<kind>.log):

• kernel patches: cargo run -- --build-only --kernel-feature boot-actuators --kernel-feature test-actuators;
• guest-binary patches: cargo test --test toyos-build -- --list;
• userland patches: cargo run -- --build-only.

To run an arm: git apply <patch>, run the command, then git apply -R <patch>.

patch	command	expected red
r3/m-stop-other-cpus-empty	-- panic_halts_the_others_first	STALLED: waiting for the other CPUs to halt after the fatal path on cpuN stopped them — QEMU shows each vCPU in \cli; hlt` as [false, false, false, false](measured atee64639`)
r2/g-hda-binds-the-first	-- hda_two_live_refused	"has a live link (statests=" never reached the boot console
r3/m-dma-fault-no-halt	-- iommu_context_absent, -- iommu_empty_domain	"Boot: complete" on a boot console that should not have it (measured)
r3/m-recursive-fault-reports	-- nested_fault_is_recursive	"KERNEL PANIC:" … that should not have it (measured)
r2/g-sched-stress-lag-fails	-- sched_check_build	sched_stress failed on the check build
r2/g-close-cancels-no-poll	-- inbox_cancel_wakes	timed out after 300s, with the guest still talking … — it was working and did not finish, counted in "1 of those reds are the ceiling"
r3/m-unmap-revokes-no-futex	-- futex_wake_counts	the sweeper's futex_wake on this process's own freshly mapped memory answered [...] … another process's waiters, then the sweeper found a wake of its own fresh memory answered for by somebody else's parked thread (measured)
r4/m-revoke-no-wake	-- futex_wake_counts	timed out after 300s, with the guest still talking … — it was working and did not finish, counted in "1 of those reds are the ceiling" (measured at 925e1a66). The FAIL rs::futex_wake_counts line's stdout: ends with claims: a claim already taken is neither counted nor charged and the sweeper's sweeper: 12 fresh frames, and no wake belonged to anybody else, with no unmap: …
r5/c-drain-never-ends	-- --nightly metal_sim_compositor_stall	the ceiling (timed out after or STALLED:), counted among the ceiling's reds. Its stdout ends with compositor stall: the ring is full; a second window presents under it, with no compositor stall: 6 stalls survived. If the unbounded drain keeps the ring from filling, that line is absent too, and the red is the same ceiling
r4/m-counts-probe-for-park	-- futex_wake_counts	futex_wake(count=1) with two waiters answered 0 (measured on the patch's form at ee646399)
r3/m-quiesce-twice-parks	-- quiesce_refuses_a_second_shutdown	"PANIC:" on a stopped-boot drain that should not have it: … quiesce.rs (measured)
r4/m-runner-answers-otherwise	-- --nightly usb_flush_optional, -- --nightly usb_boot_stick_pulled	test-runner answered flush-probe-0 after the give-up with Some("… error=…"), and a name no image carries is answered "===TEST_END flush-probe-0 error=entity not found===", and the same for pull-probe-0 before the pull, at once and not at the ceiling
r2/r1-exit_wait_storm-never-releases	-- exit_wait_storm	the ceiling
r2/r1-blocking_read_stress-echo-writes-nothing	-- blocking_read	the ceiling, for blocking_read_stress and blocking_read_window
r2/r1-poll_wake_pipe-writer-waits-a-round-ahead	-- poll_wake_pipe	the ceiling
r2/r1-heartbeat-no-pin-line	-- --nightly kernel_heartbeat	… heartbeats carry no \i8042: line` of their own`
r2/r1-i8042-no-floating-bus-exit	-- --nightly i8042_absent	no `i8042: absent — port 0x64 reads 0xff` line on a machine with no i8042
r2/r1-netd-keeps-a-chatty-client	-- netd_lookup_let_go	the guest's a client that spoke again while its lookup ran was answered
r7/take-record-no-refusal	-- userdev_dma_fault	STALLED: waiting for netd's end on its refused claim

The prefix names the directory: r2 is notiming-r2/mutations/, r3 notiming-r3/mutations/, r4 notiming-r4/mutations/, r5 notiming-r5/mutations/, r7 notiming-r7/.

Each converted kind has an arm that stages its park:

• an in-guest poll wait: g-close-cancels-no-poll;
• an in-guest wait with no bound: m-revoke-no-wake;
• a host flat drain before a verdict: m-dma-fault-no-halt;
• a host wait on a guest's answer: m-runner-answers-otherwise.

Negative controls are the patches above. Each reverts the one mechanism its test is about, onto this head. c-drain-never-ends takes out the compositor's drain budget, which is the defect compositor_stall's streaming case exists for: the drain never ends while a client has something to send, so nothing is composited and the watcher's Frame never comes. The whole-change control for the presenter's park is 925e1a66, where the test red. m-counts-probe-for-park and m-quiesce-twice-parks each revert their test's whole change. The whole-change control for await_reset is f260f3e7 itself: iommu_context_absent, iommu_empty_domain, nested_fault_is_recursive and panic_halts_the_others_first stalled there, waiting for every vCPU halted. The whole-change control for lan_no_lease is ee646399: it red on the nightly there, "netd: ready, at most " never reached … after "netd: DHCP: no lease as toyos-t14 in ". For the usb probes it is ee646399 too: both tests hung to the ceiling, STALLED: waiting for probe 0 ….

Independent oracles:

• QEMU's own monitor: info registers -a gives HLT=1 with RFL bit 9 clear for a vCPU in the stop's cli; hlt.
• QEMU's own exit: under -no-reboot a guest reset ends the process, so the fatal path's reset is seen by the host, not reported by the guest.
• The host's bcachefs reader of the NVMe image, for the storage verdicts.
• The kernel's roster, for "this thread is parked". It is the scheduler's own record, and not the test's clock.
• test-runner's own answer to a probe, and the kernel's spawn: … not found record beside it, for the usb probes.

🤖 Generated with Claude Code

[Japabu]

Review of #562 at ba636828, round 1.

Readiness. CI host is pending at ba636828 and no guest arm has run. None of the behaviour below has been measured yet. The runs listed at the end are required before any LAND.

Net lines (git diff --numstat origin/main...HEAD):

• tests +582 −5394
• production +61 −899
• issues +68 −892

BLOCKER

1. tests/toyos.rs:18351 (with :17521-17523): the STALL line still says "the guard expired, so this says nothing about the tree". This branch makes that ceiling the only verdict for a lost wake or publish in blocking_read_stress, blocking_read_window, exit_wait_storm and poll_wake_pipe. The PR's own red arms name exactly that line, so every real red it catches tells its reader to ignore it. Delete the clause, and the Tally doc's "measured the host and not the tree", in this branch.
2. In-guest deadlines survive. They are timing: a starved vCPU blows them on a correct kernel. They are the same kind this branch removed from three binaries. Wait on the event with no deadline and let the ceiling red.
   • tests/toyos-rust-tests/src/bin/kill_while_blocked.rs:89 (2 s)
   • handle_kill_policy.rs:81 (2 s)
   • inbox_cancel_wakes.rs:20 (10 s), plus the flat PARK_MARGIN sleep at :27
   • netd_hostile_peer.rs:32, :34 and :37, plus the flat SETTLE sleep at :56. That sleep feeds the kept < opened / kept >= 2 counts at :153-160, which are a count over a clock window.
   • netd_lookup_let_go.rs:40 (60 s)
   • tests/toyos.rs:3573: settle_exit_wait_storm's 5 s host drain decides whether the checked line arrived. Use await_guest.
3. Host-timed verdicts that a correct guest can fail on a slow host. Each needs a metal row or an anchor on a guest event.
   • tests/common/power.rs:640 watchdog_fed: 20 s of host clock against a 2.4 s TCO. A vCPU starved for 2.4 s resets a correct guest; the deleted heartbeat fixture records a dev-host guest left unscheduled for 7 s.
   • power.rs:985 panic_key_holds: the QMP key must land inside the 5 s fast bound.
   • tests/toyos.rs:13003-13013 i8042_health_cadence: flat host sleeps of 3000/1000 ms against a 5 s guest claim, asserting exactly two lines.
   • tests/toyos.rs:13966 i8042_quarantine: an idle-trip delta over a capture whose length the host picks.
4. panic_halts_the_others_first was deleted as "the bound was all it proved". That is false. Its non-vacuity plus its bound were the only QEMU witness that stop_other_cpus stops anything.
   • Mutation: empty the body of kernel/src/arch/x86_64/apic.rs:190-194. I find nothing left that would red on it.
   • Fix, either way: keep it as an order assertion, or record the whole property ("a fatal path stops the other CPUs at all") in the issue with its metal row.
   • The order assertion: a kernel record emitted after stop_other_cpus, and no sibling record after it beyond one in flight per sibling. Show that mutation red on it.
5. hda_two_live_refused was deleted, with Profile::HdaTwoLive. It plays no audio and has no clock in it.
   • It is the only test of the kernel refusal at kernel/src/drivers/hda.rs:358, and the only negative control on the bind path.
   • "The T14 has one controller" is not a reason to ship a refusal nothing tests.
   • Restore it with -audiodev none. Mutation: bind the first live controller instead of refusing. It must red.
6. sched_check_build lost a non-timing half along with its timing half.
   • The lost check: "the check build published no pass-cost report at all". That is a content check.
   • PassCostReport::parse (toyos-sched/src/cpu.rs:1439, tests at :3844-3875) now has no caller.
   • src/build.rs:1689 still says a guest proves the report was published.
   • Either keep the check (at least one parsed report per CPU, no quantile), or delete the emission, parse, its tests and the SCHED_CHECK_LITERALS entry.
7. desktop_audio_client was deleted with no record.
   • issues/audio/null-sink-applies-one-connect.md:62-65 names its three verdicts as "what stands between the milestone and a silent recurrence".
   • The new issue claims to list every property no metal row judges, and omits them.
   • a-megabyte-written-to-the-stick-starves-a-tone-beside-it.md:28 exits on the deleted audio_tone_load, so it can never close.
   • Record both, or move them to metal.
8. Production code this branch orphaned. The only caller was the deleted --slow-usb. Delete it, with its recipe lines in issues/audio/disk-wait-pins-a-cpu.md:24-27.
   • kernel/src/actuator.rs:279 usb-slow-device
   • kernel/src/drivers/xhci/mod.rs:893 and :906-927 (slow_device_would_have_answered)
   • SLOW_TRANSFER_NS at :347
   • held_event at :767
9. The new metal judges have no red arm. tone_on_metal, client_stall_on_metal, departures_on_metal, log_stall_on_metal and job_window in tests/common/audio.rs are pure text functions with no host test.
   • Example: changing resumes < 2 to resumes < 1 passes everything in the tree.
   • Add host tests over crafted logs, as idle_trip_verdict does, each red on its mutation.

NOTE

• tests/common/audio.rs:13 job_window ends at the next test_rs_ spawn. soundd's last stats line races the client's exit, which is why the deleted settle_null_sink_client_exits existed. So deferred == 0 in client_stall_on_metal can false-green. End the window on soundd's removal of that client.
• tests/common/audio.rs:17-21: the hda_tone row judges the bind only. underruns == 0 over the tone's window is the audio verdict the issue says metal owes, and it is one line.
• The audio_idle_suspend metal row lost the old check's discriminator: the device starting with no client connected. Add HDA's start line as a must-not in the job window.
• null_sink_shipped_client shares a boot with hda_tone, which asserts there is no null sink. The row name is false on the T14.
• tests/common/origin.rs:468-473: Some(_) => None reads "the stop line is in /log" as "the flush was answered".
  • That is the converse of the comment above it.
  • The unanswered case now rests on the console carrying FLUSH_WAITED_OUT, which the deleted comment says it rarely does.
  • Also look for it in /log: a content check, no clock.
• Host-measured must-not windows: screen_diag_boot (tests/toyos.rs:4105, a flat 5 s sleep) and screen_pager_keys (:5652). They cannot false-red, but how much they can catch depends on the host. That makes them timing in the owner's words; the owner decides.
• tests/toyos.rs:15628 blocking_read_window's floor counts posts landing in held windows between two vCPUs, so it depends on interleaving. Measure it beside eleven other guests before calling it a count.
• The placed CLAUDE.md sentence says "never how long something took", and this branch keeps lower bounds on spans the guest stages and times on its own clock. I rule those not timing (see Q1). The sentence and the tree disagree on landing day.
• issues/audio/hda-tone-phase-check.md: the evidence is a load-keyed QEMU capture (0 of 8 alone, 3 of 11 loaded), and the guest-side cause is fixed. No capture exists anywhere now, so its exit cannot be measured. Close it or give it a metal exit.
• These issues have also lost what their exit reads:
  • idle-suspend-reds-on-a-loaded-host-and-on-main.md: its subject is the deleted QEMU arm.
  • gate-a-suspend-structure-verdict-unread.md: check_suspend_structure is gone, and no metal judge reads soundd: suspended after the last removal.
• No gate enforces the timing rule. -audiodev none enforces the audio rule in practice; the timing rule is prose only.

REMOVE

• By the owner's ruling, remove the doom rows and everything that exists only for them:
  • the metal rows doom_sound_flood and doom_music, with DOOMCASE, DOOMMUSICCASE, their METAL_ONLY and RUST_SKIP entries
  • the 12 boot.doom* and list.doom* rows in tests/metal-profile.toml
  • sound_flood_on_metal, StressCounters, parse_stress_line, check_playback and music_on_metal
  • tests/toyos-rust-tests/src/bin/doom_sound_flood.rs and doom_music.rs
  • tests/doomcase/, tests/doommusiccase/ and their src/build.rs:3074-3075 entries
  • userland/doom/src/main.rs:204-213 and sound.rs's sound_stress and music_check, with whatever only they reach
  • their citations in doom-audio-callback-stalled-on-the-t14.md:10,33, eleven-names-red-on-ci.md:67 and harness-reads-after-a-flat-half-second-drain.md:12
• issues/audio/doom-sound-flood-played-full-scale-once.md: it records a defect only in Doom.
  • The volume path it names is doom's own: pack_volume, CHANNEL_VOLUME, and the channel mix clamped at userland/doom/src/sound.rs:269.
  • Nothing in it implicates soundd, toyos-mixer or the HDA driver, and its instrument is gone.
• kernel/src/drivers/hda.rs:80-81: rewritten, not deleted, and now split //////, so the doc reads "against it."
• toyos-abi/src/virtio_sound.rs:17-19: cites the deleted gate A. Deleting it is a comment change, not an ABI change.
• toyos-sched/src/cpu.rs:1274 "a verdict on it is metal's": no metal arm boots the check kernel.
• userland/soundd/src/mix.rs:254-260: check_suspend_structure is deleted, and soundd: resumed is now read.
• tests/toyos.rs:18340-18342 "the audio configs go through here too", :18884 "and the audio configs", :18261 audio_tone (smp=1), :10587-10588 "are the T14's to judge" (there is no metal row).
• tests/common/qemu.rs:961-964 and :3568: cite deleted tests.
• tests/toyos-rust-tests/src/bin/null_sink_client_exits.rs:5: cites metal_sim_null_audio.
• Citations of deleted files or of gate A:
  • issues/kernel/scheduler-pass-blocks-in-xhci.md:62,68
  • issues/diagnostics/no-cyclictest.md:29-30
  • issues/audio/desktop-session-put-26ms-of-silence.md:24,41
  • issues/kernel/soundd-past-due-wake-max-1.md:19
  • issues/kernel/syscall-preemption-is-incidental.md:39
  • issues/audio/client-ring-depth-is-the-devices-pipeline-depth.md:26
  • issues/build/there-is-no-network-gate.md:14,31
• issues/build/timing-verdicts-ruled-off-qemu-have-no-metal-arm.md:12-14: the "already have one" list will rot. :43 "which is QemuOnly": that row is deleted.
• PR body: the line-number inventory "at c5518949", "Expected green" and "expected red" (not measured), and "What I am unsure of". The body becomes main's record: keep what was done and the measured exit codes.

Rulings

• Q1: timing verdicts that survived in QEMU.
  • Lower bounds on spans the guest stages and times on its own clock: not timing, keep. No host can shorten them. tlb_shootdown_waits.rs:126,158, tests/common/usb.rs:1763,2122,2289, the xhci_slow_connect floor, netd_lookup_let_go.rs:60 and process_stats.rs:92.
  • Must-not windows: timing where a correct guest can fail (power.rs:640, power.rs:985; BLOCKER 3). Where only a false green is possible (tests/toyos.rs:4105, :5652, the refused-flush window), the host decides how much they can catch; NOTE.
  • Counts over a clock window: timing (tests/toyos.rs:12986, :13966, netd_hostile_peer.rs:153-160). :15628 must be measured first.
  • In-guest liveness guards: timing (BLOCKER 2).
  • Keep: boot_from_power_on (tests/toyos.rs:12920) is a consistency check, and a slower host only widens it. wall_clock_* (wallclock.rs:60) is a content check with a ceiling-sized allowance.
• Q2: non-timing properties deleted with their timing half.
  • panic_halts_the_others_first: yes (BLOCKER 4).
  • netd_stalled_peer: no. Its only verdict was a busy fraction, and the stall was its setup.
  • Found besides: BLOCKER 5, 6 and 7.
• Q3: audio devices left in QEMU machines. They are device-isolation tests that happen to use an audio device. Keep iommu_hda_foreign_bdl, iommu_sound_foreign_dma, iommu_domain_isolation and inspect_reads_its_owners' sound.* keys: no sample is played or judged.
• Q4: the metal rows.
  • All are registered in METAL_ONLY, staged offline (exit 2), and none has run.
  • wake_storm_cost, null_sink_shipped_client, hda_client_stall and soundd_log_stall carry real verdicts.
  • hda_tone checks the bind, not the tone. audio_idle_suspend is weaker than its QEMU form.
  • No judge has a red arm (BLOCKER 9). The doom rows are REMOVE.
• Q5: more to delete. The doom rows, the usb-slow-device actuator, and either PassCostReport::parse or the kept check.
• Q6: checklist. Deleted flags are refused by the generic unknown-flag refusal, which names the word: satisfied. A gate for the timing rule is missing (NOTE). The new judges have no red arms (BLOCKER 9). The PR body carries unmeasured expectations (REMOVE).

Guest runs required (orchestrator)

1. CI green at the final head.
2. cargo test --test toyos-build and cargo test --test toyos-build -- --nightly at the final head, with exit codes.
3. The author's eight arms, each red with its named line, at the final head. After BLOCKER 1, no STALL line may disclaim itself.
4. stop_other_cpus emptied (kernel/src/arch/x86_64/apic.rs:190-194):
   • once on ba636828, to record that nothing reds there
   • once on the fixed head, where it must red
5. The restored hda_two_live_refused with the refusal at kernel/src/drivers/hda.rs:358 replaced by binding the first live controller: must red.
6. For each converted in-guest deadline, its park staged (for example, the wake of inbox_cancel_wakes' cancellation dropped): red by the ceiling.
7. The metal judges' host tests: no guest needed.

SEND BACK

[Japabu]

Review of #562 at ee646399, the round after the round-1 review at ba636828.

Readiness. CI host is SUCCESS at ee646399. The orchestrator's guest runs are at this head (562r3-*.log). Fast tier: 396 passed. Nightly: 497 passed, 4 failed. cargo run -- --known-red on main says NO for all four reds. All four PASS on main's nightly at d30dfff3 (run 36388022685): lan_no_lease 32s, usb_flush_optional 11s, usb_boot_stick_pulled 16s, metal_sim_compositor_stall 11s. Three of them also red on the r2 nightly. All four come from this branch.

The seven arms, each EXIT=1:

• stopcpus: … QEMU shows each vCPU in cli; hlt as [false, false, false, false]
• dma ctx and dma empty: "Boot: complete" on a boot console that should not have it
• recursive: "KERNEL PANIC:" … read unmapped address at 0xffff8ffffffff000
• quiesce: PANIC: panicked at src/quiesce.rs:355:9
• counts: exit 101, futex_wake_counts.rs:191 futex_wake(count=1) with two waiters answered 0, as named.
• unmap: exit 101 at futex_wake_counts.rs:558, futex_wake on this process's own freshly mapped memory answered [(0, 1), (1, 1), (2, 1)] … another process's waiters, then :445, the sweeper's red. That is a content red and not the hang ceiling the PR's table promises.

Net lines (git diff --numstat origin/main...HEAD): tests +2410 −8802, production +72 −1405, issues +117 −1220. Total −8828.

Round-1 BLOCKERs

1. CLOSED. No STALL line disclaims itself (rg finds none). a_stall_stays_red holds both halves.
2. CLOSED. None of the named in-guest deadlines remain. inbox_cancel_wakes, poll_wake_pipe and blocking_read_stress red by the 300s ceiling (562r2-arm-close/pollwake/blockingread). exit_wait_storm reds STALL (562r2-arm-exitwait).
3. CLOSED. watchdog_fed is a metal row. panic_key_holds, i8042_health_cadence and the idle-trip delta are deleted and recorded. i8042_quarantine is driven by a QMP key.
4. CLOSED. 562r3-arm-stopcpus is EXIT=1 with [false, false, false, false]. 562r3-green-panic_halts_the_others_first is EXIT=0.
5. CLOSED. 562r2-arm-hda reds "has a live link (statests=" never reached. The green run is ok.
6. CLOSED. sched_check_build reads PassCostReport::parse for every default CPU (tests/toyos.rs:11734). 562r2-arm-sched is red.
7. CLOSED. desktop_audio_client is recorded in the new issue, and a-megabyte… exits on a METAL row.
8. CLOSED. usb-slow-device, SLOW_TRANSFER_NS, held_event and slow_device_would_have_answered are gone.
9. CLOSED. The PR body has 15 metal_audio_judges arms, each EXIT=1, at f260f3e7. tests/common/audio.rs has not changed between f260f3e7 and ee646399.

BLOCKER

• tests/common/lan.rs:525 — drain_until returns on the no-lease line itself, and :533 then requires READY after that line. READY is printed after it and is never read, so lan_no_lease reds on every run (r2 and r3 nightlies). Fix: end the drain on READY seen after gave_up, a stateful predicate as dump_nmi_probe had.
• tests/common/usb.rs:1077 and :4294 — both wait for ===TEST_END <probe> exit=. A probe names a binary that does not exist, and test-runner answers ===TEST_END pull-probe-0 error=entity not found=== (userland/test-runner/src/main.rs:346; nightly log line 3496). So usb_flush_optional and usb_boot_stick_pulled hang to the 305s ceiling on every run. This is the unbounded wait the brief feared. Fix: wait for the exact expected answer, ===TEST_END {probe} error=entity not found===, which makes it a content check too.
• tests/toyos-rust-tests/src/bin/compositor_stall.rs:122-125 — the watcher's Frame never came, and the run reached the 2880s ceiling. The compositor drew 2 frames of 9472 px per interval for 48 minutes, and main's own metal-sim line calls 9472 px the idle frame. So the watcher's present was never composited. Either the compositor starves a second client under a stream (a real defect, which the deleted frames(stream) > 0 could not see because the idle clock frame satisfied it), or the test is wrong. Measure which before changing anything. A defect is filed and redlisted; a test error is fixed.
• tests/common/storage.rs:618-629 — the whole-boot, per-path retry count reds on two genuine budget expiries of one path. A starved host produces those on a correct kernel, and the PR body says so. That is a timing verdict in QEMU. Fix: have fsync-budget-spent name each refusal it stages (kernel/src/object/ops.rs:706) and hold that count to one per path. Genuine expiries then count for nothing.
• tests/common/swap.rs:63 and :824 — asked.elapsed() < 30s, with flat 1s sleeps between tries, decides red when sshd is slow to answer. It is a host deadline in a file this branch edited (KNOCKS_WITHIN went beside it). Fix: await sshd's own listening line, or retry on the refusal with no deadline under the harness ceiling.

NOTE

• tests/toyos.rs:16467 — stalled() matches only STALLED:. So run_test's timed out after Ns … did not finish is left out of "N of those reds are the ceiling", and that is the red every converted in-guest wait produces (562r2-arm-close, 562r3 metal_sim_compositor_stall). Count it there or drop the claim.
• futex_wake_counts.rs:445-458,505-508 — the unbounded joins that replaced the bounded loops have no staged park: m-unmap-revokes-no-futex reds on the sweeper first. Stage one: kernel/src/watch.rs:78 revoke unregisters without posting the wake. It must red by the ceiling.
• futex_wake_counts.rs:292 and :434 — sleep(SETTLE) right after wait_until_parked already proved the park. Delete both. :297 (spinners in their loops) is readable on the roster this branch added, so the issue's premise that there is "no word a test can read" is false for it.
• tests/toyos-rust-tests/src/census_wait.rs:22 — settled() is a 10ms stability window. On a starved host before can be read mid-release, which is a false green. It is not in the issue's premises: record it or remove it.
• issues/build/timing-verdicts-ruled-off-qemu-have-no-metal-arm.md — it lacks two removed verdicts. One is inspect_plays: sound.periods.submitted × period_frames ≥ the frames soundd took. The other is boot_from_power_on's TSC total checked against the host.
• issues/kernel/logging-records-from-every-producer-and-a-kernel-that-waits-on-nobody.md:61 and issues/audio/desktop-session-put-26ms-of-silence.md:116 — the first exit cites the deleted audio_tone_load and the second the deleted gate A. Neither exit can be met now. Give each a METAL exit, as a-megabyte… got.
• Merge: origin/main is 25 commits ahead. git merge-tree finds one conflict, src/redlist.rs: main adds lan_swap, log_ring_keeps_the_owners_slots, swap_crash_rolls_back, swap_netd and usb_transport_break beside the rows this branch deletes. Keep main's five and drop this branch's four. CLAUDE.md auto-merges. The rust pin is 1b236638 on HEAD, on main and on the merge base, so there is nothing to reconcile. After the merge, log_ring_keeps_the_owners_slots and usb_transport_break are disabled on main, so what this branch changed in them runs nowhere.

REMOVE

• PR body, "The ceiling": "the summary names it apart". False for run_test timeouts.
• PR body patch table, "expected red": m-unmap "the hang ceiling" (measured exit 101 at :558); g-close-cancels-no-poll "STALL, and counted among the ceiling's reds" (measured timed out after 300s, not counted); "an in-guest bounded loop: m-unmap-revokes-no-futex".
• PR body, "What I am unsure of". This was already REMOVE in round 1.
• tests/toyos-rust-tests/src/roster.rs:45-47: "The roster call inside cond is the loop's preemption point". User threads are preempted on the quantum, and futex_wake_counts.rs:258-263 depends on that.
• issues/build/no-device-class-answers-for-a-block-device.md:53: it cites the deleted tests/common/hda.rs.
• kernel/src/scheduler.rs:620-621: "counted on every trip rather than only the ones that print."
• toyos-mixer/src/stats.rs:18: "Each has to mean exactly one thing." It repeats the module doc.
• toyos-hda/src/config.rs:147, toyos-mixer/src/lib.rs:12, toyos-mixer/Cargo.toml:7: these were rewritten instead of deleted, and are now overlong lines.

SEND BACK

[Japabu]

Review, round 6, at ba14967

Readiness. CI host is COMPLETED SUCCESS at ba14967e. Orchestrator runs: 562r5 at 2a9c77ee gives nightly EXIT=0 (493/493), compositor-stall EXIT=0, compositor-drain-never-ends EXIT=1 and revoke-no-wake EXIT=1. 562r6 Fast at ba14967e is EXIT=1, 386/387. The one red is userdev_dma_fault.

Net lines (git diff --numstat origin/main...HEAD): tests +2522 −8890, production +85 −1417, issues +323 −1236.

Round-2 BLOCKERs

1. lan.rs drain — CLOSED. 562r4 nightly-lan_no_lease EXIT=0 at 925e1a66, and PASS in 562r5-nightly.
2. usb probes — CLOSED. usb_flush_optional and usb_boot_stick_pulled are EXIT=0 at 925e1a66 and PASS in 562r5-nightly. arm-runner-flush and arm-runner-pull are EXIT=1.
3. compositor_stall — CLOSED. The c-diag* arms decided it. The whole-change control is 925e1a66, EXIT=1. 562r5 compositor-stall is EXIT=0. compositor-drain-never-ends reds at the ceiling, and its last line is the ring is full.
4. storage.rs retry count — CLOSED. 562r4 arm-fsync is EXIT=1. home_budget_refusal_retried is PASS in 562r5-nightly.
5. swap.rs 30 s deadline — CLOSED. Both loops are gone and the rig awaits sshd: listening on port 22. All eight swap_* pass in 562r5-fast and 562r6-fast.

Round-2 NOTEs and REMOVEs: all closed.

• stalled() counts TIMED_OUT: the drain arm prints "1 of those reds are the ceiling".
• m-revoke-no-wake is EXIT=1 in 562r5.
• Both SETTLE sleeps are gone.
• The census window and the two verdicts are recorded.
• Both issues have METAL exits.
• roster.rs, scheduler.rs, stats.rs, config.rs, toyos-mixer and the PR-body lines were cut.

Attribution of the round-5 Fast reds

• user_copy_races_munmap: not this branch's.
  • Instant::now reads the clock page (toyos-abi/src/clock.rs:49) and makes no syscall, so the deleted assert moved no scheduling.
  • It could fire only while HELD was unset. Without it, that case hangs to the ceiling; it cannot become the kernel's remap_race panic.
  • Disable three flaky main-level reds behind their issues #580 records the same panic at Storage: file servers for DATA, the log and the boot volume; the kernel's NVMe and FAT go #536's 069722c3.
• quiesce_leaves_the_volume_whole: not this branch's.
  • The deleted deadline bounded a wait the program had left at 0.649 s, before the stop.
  • The host verdict in volumes.rs is unchanged.
  • Two kernel changes are on this boot's path, and neither changes its behaviour. The ops.rs change is inert unless fsync-budget-spent is armed. The scheduler.rs change deletes an actuator-gated cadence.
  • Without the deadline, "never refused twice" is a ceiling red, not a panic.
• netd_refused_accept: not on this branch's path.
  • Its binary uses only ask, Ask, HOST and FORWARDED_PORT from netd_stream.rs.
  • netcase_against_host, PatternServer and tests/netcase are unchanged.

BLOCKER

• tests/common/iommu.rs:1199 (foreign_fault) — the deleted log.push(&qemu.drain_serial(Duration::from_secs(2))) went into a local log that is dropped at return, so on main that drain swallows netd's post-fault lines. Here they arrive after log_origin's spawn record, run_test moves them into result.serial, and userdev_dma_fault holds that to must_be_clean_apart_from (:2112) — why: this red is the branch's, not main's.
  • Evidence: in 562r6-fast.log the DMA FAULT at 0.403 is the boot log's last line (1244+217). spawn: … test_rs_log_origin follows at 0.493, then netd's 0.423 panic (1244+218..228).
  • The test's only two unmutated reds in the orchestrator's logs are both on this branch (562r4-nightly, 562r6-fast). About 50 runs of other heads have no red.
  • So these rest on a false premise: the PR body's "can only see fewer lines", the issue's "It is not PR No QEMU test measures time, and audio is judged on metal only #562's", and Disable three flaky main-level reds behind their issues #580's row disabling a test that is green on main.
  • Fix, in this branch: after the fault, await netd's own end by event (exit: netd pid=… code=101 and its refused an interrupt read: Io line) before run_test, then delete the issue.
  • Mutation that must red: delete the faulted() refusal in kernel/src/pcidev/mod.rs:1725 (take_record). netd then never ends, and the new wait must stall by name.

NOTE

• src/redlist.rs — the two rows and their issues were cherry-picked unchanged into Disable three flaky main-level reds behind their issues #580, which adds a sighting to the copy-meets-a-remap issue — when Disable three flaky main-level reds behind their issues #580 lands, the merge takes main's two files and one row each. If the fix above lands, Disable three flaky main-level reds behind their issues #580's userdev_dma_fault row and issue go in the same merge.

REMOVE

• PR body, "Found, not this branch's: userdev_dma_fault" — false (BLOCKER).
• issues/isolation/userdev-dma-fault-forbids-the-panic-netd-answers-a-refused-claim-with.md:40-43 — "It is not PR No QEMU test measures time, and audio is judged on metal only #562's…" is false.
• issues/kernel/copy-meets-a-remap-holds-a-cpu-the-thread-it-waits-on-may-be-queued-behind.md:34-38 and issues/build/quiesce-leaves-the-volume-whole-needs-its-flush-to-close-inside-the-stops-budget.md:34-37 — a PR's attribution argument is not the issue's record, and it rots.
• PR body, the "Owed at 2a9c77ee" list — predictions ("EXIT=0, unless …"), and measurements now exist.
• PR body, net lines "+2821" — false at ba14967e (+2930).

SEND BACK

[Japabu]

Review, round 7, at 0ca1fdb

Readiness.

• CI host is COMPLETED SUCCESS at 0ca1fdb2 (run 36476148205).
• 562r7 at 6079c8c1: userdev_dma_fault EXIT=0, take-record-no-refusal EXIT=1, Fast EXIT=0.
• 562m at 6c8ff3f4: Fast EXIT=0 (228/228); userdev_dma_fault EXIT=0; take-record EXIT=1; the other 12 named tests EXIT=0.
• 562m nightly at 6c8ff3f4: EXIT=1, 361/362. The only FAIL is i8042_health_cadence: unknown input test, which 65511a24 deletes.
• No 562f line exists yet. The Fast and nightly runs at 0ca1fdb2 are still owed, and this verdict depends on them.

Net lines (git diff --numstat origin/main...0ca1fdb2):

• tests +2469 −8643
• production +75 −1410
• issues +172 −1240
• total +2716 −11293

Round-6 findings

• BLOCKER userdev_dma_fault (the deleted 2 s drain) — CLOSED.
  • Green: 562r7 userdev_dma_fault EXIT=0, 562m userdev_dma_fault EXIT=0, and PASS in 562m-nightly.
  • Control: 562m-take-record.log:34 reads STALLED: waiting for netd's end on its refused claim — it never stopped talking and never got there, counted in "1 of those reds are the ceiling" (306.1 s).
  • Whole-change control: ba14967e, red in 562r6-fast.
• NOTE, Disable three flaky main-level reds behind their issues #580's rows — CLOSED. src/redlist.rs has one row each for user_copy_races_munmap and quiesce_leaves_the_volume_whole, and each points at an issue that exists. Neither origin/main nor the head has a userdev_dma_fault row. Every test: is unique and every issue: exists (fixtures in the module's own tests aside).
• REMOVE, PR body "Found, not this branch's: userdev_dma_fault" — CLOSED.
• REMOVE, the userdev issue's "It is not PR No QEMU test measures time, and audio is judged on metal only #562's" — CLOSED. The file is deleted, and neither its path nor its slug is cited anywhere at the head.
• REMOVE, the copy-meets-a-remap issue's attribution paragraph — CLOSED. The file is main's text, and that text has no such paragraph.
• REMOVE, the quiesce-leaves-the-volume-whole issue's attribution paragraph — OPEN. 845052d1 deleted it. 6c8ff3f4 took main's text for the file, and main's text brings it back (see REMOVE).
• REMOVE, the "Owed at 2a9c77ee" list — CLOSED.
• REMOVE, the net-line count — OPEN in a new form (see REMOVE).

Merges

• fe1c4d99. The one hunk lost from the conflicts was the i8042_health_cadence registration, and 65511a24 deletes it.
  • Registration ↔ dispatch at the head: 291 = 291, and they agree both ways across MACHINE_TESTS/SCREEN_TESTS and run_machine_test/run_screen_test.
  • Nothing is resurrected. The names the head has and ba14967e lacks are exactly the ones main added: metal_sim_hostile_clipboard, quiesce_wakes_on_the_last_teardown and root_chunk_refused_on_a_usb_stick.
  • Nothing main added is lost. Nothing either side deleted is present.
  • The same holds for METAL, CARRIES and RUST_SKIP.
  • a_stall_stays_red and metal_audio_judges move to tests/checks.rs libtest tests.
• 6c8ff3f4. Every hunk is accounted for except the quiesce issue's paragraph.
• 0ca1fdb2. --remerge-diff is empty. Of main's delta, only CLAUDE.md is also a branch file, and main's edit there is not in the branch's hunk.
• git merge-tree --write-tree origin/main 0ca1fdb2. EXIT=0, and the result is 86e050c6 = the head's tree.
• No in-guest deadline came back. Every clock line the three merges add under tests/toyos-rust-tests, tests/common, tests/toyos.rs and tests/checks.rs is one of three kinds: a synthetic Duration in the host checks, main's kill_ends_every_wait roster pace (10 ms, no deadline) and its 3600 s park, or ceiling_self_check.

BLOCKER

None open.

NOTE

• tests/common/iommu.rs:2087 — the end capture (the fault until netd's exit) is never judged, because after (:2115) is only the boot log plus result.serial. Another process's panic or a second iommu: DMA FAULT owner=slot0 line in that window passes. That window is where "the staged fault happened once" (:2111) would show up. main also dropped this window, so this is not a regression, but the branch now holds the capture. Mutation that is green today: in kernel/src/arch/x86_64/vtd/fault.rs:298, drain's log!, emit the record twice when owner.is_some(). It must turn red once after.push(&end) is judged, with netd's own panic line excepted.
• issues/kernel/netd-never-reaches-its-loop-under-iommu-userdev-foreign-dma.md — its exit clause "or userdev_dma_fault asserting what netd does after the fault" is met by 845052d1. netd's interrupt read and its exit are in 562r6-fast, 562r7 and 562m. Close the issue, or give the reason it stays open.
• tests/toyos.rs MACHINE_TESTS — no host gate ties a row to a dispatch arm. 65511a24's orphan passed CI host, --list and --clippy at 6c8ff3f4, and only a nightly boot found it. File it; it is outside this branch's fence.
• Weekly tier — it has not run at any head since the tests: the measured schedule — Fast is every PR, then Nightly, then Weekly; 15 never-caught tests and the kernel code only they armed deleted #564 merge. hda_two_live_refused, which this branch restores, and netd_hostile_peer, which it rewrites, are Weekly now, and their last green is 562r5-nightly at 2a9c77ee. Run those two by name at 0ca1fdb2.

REMOVE

• PR body, "Net lines against main" line — false at 0ca1fdb2 (+2716 −11293; tests +2469 −8643, production +75 −1410, issues +172 −1240).
• PR body, the table row compositor_client_death — main's 2a5c94c4 made that change, and git diff origin/main...0ca1fdb2 does not touch the file.
• PR body, "## Found, not this branch's: the round-5 Fast reds" — both rows are now main's (Disable three flaky main-level reds behind their issues #580), so "disabled here" is false, and the section is an attribution argument.
• PR body, "Nothing has booted 6c8ff3f4" — false (562m).
• PR body, "## The merge defect at 6c8ff3f4, and the fix" — this is branch chronology, and 65511a24's message already holds it.
• PR body, the gate lists "At fe1c4d99" and "At 6c8ff3f4", the "## Gates, at 2a9c77ee" section (its heading is false, and it carries "Free disk before the run: 100 GiB"), and "The test is deleted, not kept … Queued for the orchestrator" — superseded, and they rot.
• PR body, the trailing https://claude.ai/code/session_… line — not load-bearing in main's merge record.
• issues/build/quiesce-leaves-the-volume-whole-needs-its-flush-to-close-inside-the-stops-budget.md:33-36 — "Nothing here is PR No QEMU test measures time, and audio is judged on metal only #562's doing either … only reached on a hang." This is round 6's REMOVE, brought back by 6c8ff3f4.
• issues/kernel/netd-never-reaches-its-loop-under-iommu-userdev-foreign-dma.md:18-19 — "userdev_dma_fault passes without asking where: its verdict is the machine, not the driver." False at this head.
• tests/common/iommu.rs:2111 — "Nothing panicked on the way" is false. The test now waits for netd's panic on the way.

Conditional on 562f Fast and nightly at 0ca1fdb2:
LAND AFTER NAMED CHANGES