Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ the event it stands for has no word a test can read:
`INTO_WINDOW`, and `redirty_mid_flush`'s swept delay.
- Drains of work handed to another thread: the 200 ms `iod` drains in
`writeback_durability` and `home_backing_revoked`, `fat_backing_revoked`'s
drain, and `log_gate`'s `STORM_SETTLE` and `QUIET_READS`.
drain, and `log_gate`'s `QUIET_READS`.
- Product clocks a QEMU boot races: `boot_deadline_ends_a_wedge`'s 15 s
deadline against the job list reaching its shutdown,
`hard_lockup_ends_a_deaf_cpu`'s 30 s, `loader_watchdog_arms` reading
Expand Down
49 changes: 0 additions & 49 deletions issues/diagnostics/a-shards-timestamps-run-backwards-at-seq-517.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
status: open
kind: defect
opened: 2026-09-28
---

# test-runner holds `logread` where no log builtin runs

`logread` in a `[programs.test-runner]` row is authority for the builtins that
read the log inside test-runner (`log-gate`, `log-storm`, `log-close`,
`userland/test-runner/src/main.rs`'s `BUILTINS`). Every boot that runs one is a
`tests/testcases` boot (`tests/common/logread.rs` boots the machine tests' config;
`tests/toyos.rs`'s one job list naming `log-close` is `tests/testcases`). Seven
other manifests grant it anyway: `tests/partclaimcase`, `tests/blockdcase`,
`tests/doommusiccase`, `tests/logrotatecase`, `tests/metalcase`, `tests/netcase`
and `tests/sshdcase`.

`partclaimcase` and `blockdcase` also grant `dup`, so there every child
test-runner spawns receives a `SysCap` duplicate carrying `LOG` as well.

**Evidence:** `rg -n '"log-gate"|"log-storm"|"log-close"' tests userland`, and
`rg -n logread -g system.toml tests`.

**Exit:** every `logread` in a test manifest is one a program on that boot
uses, or the row says which use it is for.
4 changes: 1 addition & 3 deletions issues/kernel/the-kernel-still-creates-threads.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,8 @@ code creates a schedulable task other than the per-CPU idle loop.
last thread tears down its own process on its way out of the kernel, and the
scheduler frees that thread's kernel stack after switching away. Blocked on
#549 landing.
- **K3:** the test-only `logstorm`/`lognest` producers are deleted if the log
gate does not need kernel-context producers. Blocked on nothing.
- **K4:** `klogd` goes: the owner-approved driver-model design moves the
console to logd, and this track owns that move.
- **K5:** `iod` goes with the kernel's write-back queue; met only when #536
lands with no new `kthread::spawn`.
- **K6:** delete the machinery named above. Blocked on K2–K5.
- **K6:** delete the machinery named above. Blocked on K2, K4 and K5.
6 changes: 0 additions & 6 deletions kernel/src/actuator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -423,9 +423,6 @@ actuators! {
/// Log the monotonic time and which CPUs are alive every 250ms.
heartbeat = "heartbeat";

/// Have every CPU emit patterned log records at once from spawned kernel threads.
log_storm = "log-storm";

/// Remove the IF/TF bracket around shard selection through publication — the negative control on the log's interrupt-atomicity claim.
log_unbracketed_reserve = "log-unbracketed-reserve";

Expand All @@ -435,9 +432,6 @@ actuators! {
/// The same IPI, sent between the shard-pointer read and the unlocked `xadd` — stages order damage the log gate detects, unlike the row above's invisible corruption.
log_nested_reserve = "log-nested-reserve";

/// Turn the reservation's `xadd` into a load, an open interrupt window, and a store.
log_shared_reservation = "log-shared-reservation";

/// Let a handle close cancel every poll on the log's watch in the machine.
log_close_cancels_any_syscap = "log-close-cancels-any-syscap";

Expand Down
12 changes: 0 additions & 12 deletions kernel/src/arch/aarch64/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,18 +109,6 @@ pub unsafe fn percpu_fetch_add(
_guard: &IrqGuard,
) -> u64 {
let previous = counter.load(core::sync::atomic::Ordering::Relaxed);
// Under `log-shared-reservation`, open the window the guard closes, so a
// nested record can land between the load and the store.
if crate::actuator::log_shared_reservation() && crate::log::nested::inject() {
// SAFETY: each writes `DAIF.I` and touches no memory.
unsafe {
core::arch::asm!("msr daifclr, #2");
for _ in 0..256 {
core::hint::spin_loop();
}
core::arch::asm!("msr daifset, #2");
}
}
// A load and a store, not an atomic add: the guard masks the only other
// writer this CPU has, and no other CPU writes the counter.
counter.store(previous + 1, core::sync::atomic::Ordering::Relaxed);
Expand Down
17 changes: 0 additions & 17 deletions kernel/src/arch/x86_64/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,23 +107,6 @@ pub unsafe fn percpu_fetch_add(
counter: &core::sync::atomic::AtomicU64,
_guard: &IrqGuard,
) -> u64 {
// Under `log-shared-reservation`, stage a load/store race instead of the `xadd` below.
if crate::actuator::log_shared_reservation() {
let previous = counter.load(core::sync::atomic::Ordering::Relaxed);
if crate::log::nested::inject() {
// SAFETY: `sti`/`cli` each write one `RFLAGS` bit and touch no memory.
unsafe {
core::arch::asm!("sti");
for _ in 0..256 {
core::hint::spin_loop();
}
core::arch::asm!("cli");
}
}
counter.store(previous + 1, core::sync::atomic::Ordering::Relaxed);
return previous;
}

let previous: u64;
// Not `AtomicU64::fetch_add`: its locked xadd is costly under QEMU TCG emulation.
// SAFETY: `counter.as_ptr()` is live; unlocked `xadd` retires whole, atomic against an interrupt here.
Expand Down
2 changes: 1 addition & 1 deletion kernel/src/log/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ pub mod read;
pub mod recovery;
pub mod registry;
pub mod shard;
#[cfg(feature = "boot-actuators")]
#[cfg(any(feature = "boot-actuators", feature = "test-actuators"))]
pub mod storm;
pub mod user;

Expand Down
30 changes: 10 additions & 20 deletions kernel/src/log/nested.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
//! produces a shard whose `at_ns` descends, which `Descent::advance` and the
//! log gate assume cannot happen.

/// Producer id the burst's records use — outside the range any real storm thread can have.
/// Producer id the burst's records use — one the log gate's own producer never takes.
#[cfg(feature = "boot-actuators")]
pub const NEST_PRODUCER: u64 = u64::MAX;

Expand All @@ -16,9 +16,8 @@ mod armed {
use core::sync::atomic::{AtomicBool, Ordering};

use crate::log::shard::SHARD_RECORDS;
use crate::sched::kthread;

/// One-shot for the body-copy injection point, consumed by `mid_body` or, under `log-shared-reservation`, by the outer `inject`.
/// One-shot for the body-copy injection point, consumed by `mid_body`.
static ARMED: AtomicBool = AtomicBool::new(false);

/// One-shot for the reservation-window injection point, consumed by [`reserve_window`]; kept separate from `ARMED` so it can't starve the body window.
Expand All @@ -36,17 +35,19 @@ mod armed {
// Both actuators name the same injection; arming both would inject into one record twice.
assert!(
!(crate::actuator::log_nested_emit() && crate::actuator::log_nested_reserve()),
"log-nested-emit and log-nested-reserve both name the one injection this thread arms"
"log-nested-emit and log-nested-reserve both name the one injection this read arms"
);
if STARTED.swap(true, Ordering::Relaxed) {
return;
}
crate::log!("lognest start records={SHARD_RECORDS}");
// A kernel thread, not the syscall that arms it: `IF` is clear for a whole syscall, so injecting there would never test the guard.
kthread::spawn("lognest", body, 0);
// `IF` is clear for a whole syscall, so injecting with it clear would never test the guard; Ring 0 is not preempted, so the body runs whole on this CPU.
crate::arch::cpu::enable_interrupts();
body();
crate::arch::cpu::disable_interrupts();
}

extern "C" fn body(_arg: u64) -> ! {
fn body() {
if crate::actuator::log_nested_reserve() {
ARMED_RESERVE.store(true, Ordering::Relaxed);
crate::log!(
Expand All @@ -61,12 +62,10 @@ mod armed {
ARMED.store(false, Ordering::Relaxed);
}
crate::log!("lognest done emitted={SHARD_RECORDS}");

crate::watch::park_forever();
}

/// Consumes the one-shot and sends this CPU its own IPI; `true` if this call sent it.
pub fn inject() -> bool {
fn inject() -> bool {
if !ARMED.swap(false, Ordering::Relaxed) {
return false;
}
Expand Down Expand Up @@ -108,21 +107,12 @@ mod armed {
}
}

/// Arms the injection on a dedicated kernel thread, once; compiled only under `boot-actuators`.
/// Arms the injection and emits the record it lands in, inline in the calling `SYS_LOG_READ`, once; compiled only under `boot-actuators`.
#[cfg(feature = "boot-actuators")]
pub fn start_once() {
#[cfg(feature = "boot-actuators")]
armed::start_once();
}

/// Consumes the one-shot at the reservation, for `log-shared-reservation`; `true` if an IPI went out.
pub fn inject() -> bool {
#[cfg(feature = "boot-actuators")]
return armed::inject();
#[cfg(not(feature = "boot-actuators"))]
false
}

/// Injection point halfway through a record's body copy; always compiled so `kernel-loom`'s separate copy of `log::shard` names one path.
pub fn mid_body() {
#[cfg(feature = "boot-actuators")]
Expand Down
36 changes: 1 addition & 35 deletions kernel/src/log/storm.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
//! Generates patterned records so the log gate's reader can check a conservation law over them.

use core::sync::atomic::{AtomicBool, Ordering};

use crate::sched::kthread;

// Exceeds a shard's capacity, so the drop path under test is reached at every `--smp` count.
const STORM_RECORDS: u64 = 1024;

// Must exceed one machine word: a single-store payload couldn't reveal a torn write.
const PAYLOAD: usize = 96;

Expand All @@ -21,7 +14,7 @@ pub fn payload_byte(checksum: u64, offset: usize) -> u8 {
b'a' + (checksum.wrapping_add(offset as u64) % 26) as u8
}

/// One patterned record for `thread`/`index`; also called by `log-nested-reserve` from an interrupt handler.
/// One patterned record for `thread`/`index`; called by `SYS_DEBUG`'s `LOG_PATTERNED` and by `log-nested-reserve` from an interrupt handler.
/// The reader regenerates this text independently from `t=`/`i=`, so the format here must stay in sync with it.
pub fn emit_patterned(thread: u64, index: u64) {
let checksum = checksum(thread, index);
Expand All @@ -33,30 +26,3 @@ pub fn emit_patterned(thread: u64, index: u64) {
let payload = core::str::from_utf8(&payload).unwrap_or("");
crate::log!("logstorm t={thread} i={index} k={checksum:016x} {payload}");
}

static STARTED: AtomicBool = AtomicBool::new(false);

/// Spawns one storm thread per shard, once for the life of the machine.
/// Called from `SYS_LOG_READ`, which is what makes the storm concurrent with a reader by construction.
pub fn start_once() {
if STARTED.swap(true, Ordering::Relaxed) {
return;
}
let threads = super::shard_count();
// The reader parses this line to learn the storm's shape.
crate::log!("logstorm start threads={threads} records={STORM_RECORDS}");
for thread in 0..threads {
kthread::spawn("logstorm", body, thread as u64);
}
}

extern "C" fn body(thread: u64) -> ! {
for index in 0..STORM_RECORDS {
emit_patterned(thread, index);
}
// The reader decides from its own cursor rather than waiting on this record: a barrier here was tried and hung at scale.
crate::log!("logstorm done t={thread} emitted={STORM_RECORDS}");

// Parks rather than exits: kthread rows are never removed, and spinning here would compete with the reader for the rest of the boot.
crate::watch::park_forever();
}
7 changes: 1 addition & 6 deletions kernel/src/log/user.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,7 @@ pub fn read(
out: &mut UserBytesMut,
capacity: usize,
) -> Result<usize, SyscallError> {
// Started on first read, not at boot: an unread storm has already spent itself before a cursor exists to notice it.
#[cfg(feature = "boot-actuators")]
if crate::actuator::log_storm() {
super::storm::start_once();
}
// Armed here too, once: one thread serves both injection windows; `log::nested` picks the target from whichever actuators are armed.
// Run once, inside the first read's own syscall; `log::nested` picks the window from whichever actuator is armed.
#[cfg(feature = "boot-actuators")]
if crate::actuator::log_nested_emit() || crate::actuator::log_nested_reserve() {
super::nested::start_once();
Expand Down
5 changes: 1 addition & 4 deletions kernel/src/sched/kthread.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,8 @@ use crate::sync::Lock;

use super::payload::ThreadSched;

/// `klogd` and `iod`, plus one `log-storm` thread per shard in the actuator build.
#[cfg(not(feature = "boot-actuators"))]
/// `klogd` and `iod`.
const MAX_KERNEL_TASKS: usize = 2;
#[cfg(feature = "boot-actuators")]
const MAX_KERNEL_TASKS: usize = 2 + toyos_abi::log::MAX_LOG_SHARDS;

/// Collides with no packed id: neither id map issues `u32::MAX`.
const NO_TASK: u64 = u64::MAX;
Expand Down
4 changes: 4 additions & 0 deletions kernel/src/syscall/dispatch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -602,6 +602,10 @@ pub(crate) fn syscall_dispatch(num: u64, a1: u64, a2: u64, a3: u64, a4: u64) ->
None => SyscallError::InvalidArgument.to_u64(),
}
}
DA::LOG_PATTERNED => {
crate::log::storm::emit_patterned(0, a2);
0
}
_ => SyscallError::InvalidArgument.to_u64(),
},
SYS_SCHED_INFO => match ctx.copy_out(UserAddr::new(a1), &sys_sched_info()) {
Expand Down
12 changes: 0 additions & 12 deletions kernel/src/watch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -266,18 +266,6 @@ pub fn wait_uncancellable_until(p: &Parkable, watch: &Watch, token: u64, ready:
}
}

/// Parks forever rather than exiting: exiting frees a stack a producer may still write to.
#[cfg(feature = "boot-actuators")]
#[track_caller]
pub fn park_forever() -> ! {
let parkable = crate::scheduler::Parkable::at_entry();
let handle = crate::sched::driver::current_handle().expect("a kernel thread is a task");
let armed = arm(handle.watch(), 0, WaitClass::Other).expect("a task can arm");
loop {
let _ = wait(&parkable, &armed, Deadline::never());
}
}

#[track_caller]
fn wait_inner(
_p: &Parkable,
Expand Down
2 changes: 1 addition & 1 deletion src/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2533,7 +2533,7 @@ mod tests {
fn the_pre_flash_gate_clears_a_valued_parameter_and_refuses_an_actuator() {
let root = Path::new(env!("CARGO_MANIFEST_DIR"));
assert_eq!(flashable_params(root, &[format!("{}0x1000", toyos_blackbox::PARAM)]), Ok(()));
assert!(flashable_params(root, &["log-storm".to_string()]).is_err());
assert!(flashable_params(root, &["wedge-before-reset".to_string()]).is_err());
}

#[test]
Expand Down
3 changes: 0 additions & 3 deletions tests/blockdcase/system.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ syscap = ["logread"]
# `device` because five of the guest binaries claim the keyboard or the mouse
# and no manifest row can name them — they are not `[programs]` keys — and `dup`
# because a claim moves and one boot runs several of them.
# `logread` because the log gate reads the kernel's own records and cannot be
# a spawned binary: a `SysCap` dup is not part of the namespace test-runner
# hands down.
# `power` because `run shutdown` is how a dozen host-side gates end their guest
# and read what reached the volume, and test-runner spawns `/system/bin/shutdown`
# directly — it holds no `launcher` connector, so the applet's authority is the
Expand Down
Loading
Loading