diff --git a/issues/build/a-sysroot-cloned-during-a-toolchain-rebuild-never-gets-its-cargo.md b/issues/build/a-sysroot-cloned-during-a-toolchain-rebuild-never-gets-its-cargo.md deleted file mode 100644 index 9594732d9ff..00000000000 --- a/issues/build/a-sysroot-cloned-during-a-toolchain-rebuild-never-gets-its-cargo.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-27 ---- - -# A sysroot cloned during a toolchain rebuild never gets its cargo - -The primary checkout's `rust/build/aarch64-apple-darwin/stage2/bin` was -recreated at 22:00 on 2026-09-27 and, read while a `toyos-build --build-only` -ran in the primary, held `rustc` and `rustdoc` and no `cargo`; the `cargo` link -appeared at 22:08. At 22:03:59 a `cargo test --test toyos-build` in the -`wt/toyos-nokthread` worktree rebuilt std and published -`rust/build/sysroots/5dc157f7fac727be` cloned from that `bin/`, so it has -`rustc` and `rustdoc` and no `cargo`. The key is found again on every later -run and nothing re-provisions it: every harness run from that worktree since -panics at `tests/toyos.rs:2970` with `the toyos toolchain at -.../sysroots/5dc157f7fac727be/bin is missing cargo` on the C corpus, before any -test runs. - -**Evidence:** the two directory listings and the harness log above, read on -the dev host; not reproduced on purpose. - -**Exit condition:** a sysroot is never published without the provisioned -`cargo`, and a run that finds a published one without it provisions it or -refuses by name at the toolchain step rather than inside the corpus. diff --git a/issues/build/record-reads-compiler-source-after-the-bootstrap-it-records.md b/issues/build/record-reads-compiler-source-after-the-bootstrap-it-records.md new file mode 100644 index 00000000000..9300b15baea --- /dev/null +++ b/issues/build/record-reads-compiler-source-after-the-bootstrap-it-records.md @@ -0,0 +1,23 @@ +--- +status: open +kind: tooling +opened: 2026-09-28 +--- + +# `record` reads `compiler/` as it stands after the bootstrap it records + +`compiler::record` (`src/compiler.rs`) is called once `reassemble` has finished +building `stage2`, and it computes `source(rust_dir)` at that point — the +`compiler/` tree, working diff and untracked files as they read *then*, not as +they read when the bootstrap it is recording began. A `compiler/` edit made +while that bootstrap was running (`x.py build` takes minutes) is folded into +the record even though `stage2` was built without it, so the next build sees +`primary_is_current` return true for a `stage2` that does not contain the edit, +and skips a bootstrap that is actually owed. + +Exit condition: `record` (or whoever calls it) captures `source(rust_dir)` +before the bootstrap starts, not after, and a test edits `compiler/` mid-build +(a `bootstrap` closure that writes a file before returning) and asserts the +next `primary_is_current` is false. + +Owner: whoever next touches `compiler::record` or `rebuild_compiler`. diff --git a/src/CLAUDE.md b/src/CLAUDE.md index c80554a29db..2d21ce98bf4 100644 --- a/src/CLAUDE.md +++ b/src/CLAUDE.md @@ -19,7 +19,7 @@ Loads when you read a file under `src/` — the root cargo project, package name ## The host's locks -- **Sysroots are content-addressed** (`src/sysroot.rs`): one per key — the identity (`src/identity.rs`, so a comment is no change) of `toyos-abi/src`, `toyos/src`, `userland/libc/src` and their manifests, the std fork's `rust/library/` and `rust/src/bootstrap/`, and the compiler — at `rust/build/sysroots//`, made by whichever worktree first needs it and never written again. Every build compiles against its own key's, so two worktrees with different ABIs never refuse or wait for each other; the only shared step is the primary's compiler, which a sysroot build reads under the global lock in shared mode. A new key costs one std build of the three guest targets; `--worktree remove` sweeps the keys no worktree records. +- **Sysroots are content-addressed** (`src/sysroot.rs`): one per key — the identity (`src/identity.rs`, so a comment is no change) of `toyos-abi/src`, `toyos/src`, `userland/libc/src` and their manifests, the std fork's `rust/library/` and `rust/src/bootstrap/`, and the compiler — at `rust/build/sysroots//`, made by whichever worktree first needs it. Every build compiles against its own key's, so two worktrees with different ABIs never refuse or wait for each other; the only shared step is the primary's compiler, which a sysroot build reads under the global lock in shared mode. A new key costs one std build of the three guest targets; `--worktree remove` sweeps the keys no worktree records. - **The std fork is built per worktree, and nothing but the primary's own sync moves the primary's `rust/`.** A linked worktree's `rust/` becomes, on its first build, a git worktree of the primary's fork repository at the commit its tree pins — that is where the fork is edited, committed and pinned. A worktree whose fork `compiler/` differs from the one the primary built builds its own compiler, keyed by that source and placed beside the primary's without touching it. If that checkout later falls behind the commit its tree pins (a merge moved the pin), the build moves the checkout to it itself, fetching from the primary's repository first if it holds the commit, unless the checkout has local changes, which it refuses to move out from under. - `src/buildlock.rs` serialises the stateful phases in two scopes: `Global` (the primary's compiler and the rustup link — one directory in `.git/`, shared by every worktree) and `Worktree` (the crate-target cleans, and this worktree's std build). Only `./x.py` typed by hand in `rust/` escapes it. - **Never kill a build that has taken the global lock** — the kill removes the shell wrappers, not the bootstrap, which inherits the file descriptor and runs on regardless; a toolchain rebuild interrupted or unobserved this way can leave `stage2/bin` without a `cargo`. diff --git a/src/buildlock.rs b/src/buildlock.rs index 1e24eed06df..2b63a4c296a 100644 --- a/src/buildlock.rs +++ b/src/buildlock.rs @@ -265,14 +265,14 @@ impl Keyed { /// Make what `key` names: exclusive, and waited for by every other process that /// wants the same key, which then finds it made. -pub fn keyed_building(root: &Path, kind: Keyed, key: &str) -> Guard { +fn keyed_building(root: &Path, kind: Keyed, key: &str) -> Guard { let lock = format!("{} lock", kind.name()); exclusive(&keyed_lock_path(root, kind, key), &lock, &format!("building {} {key}", kind.name())) } /// Use what `key` names: shared, so any number of builds use it at once, a /// builder of it is waited for, and a sweep cannot remove it. -pub fn keyed_using(root: &Path, kind: Keyed, key: &str) -> Guard { +fn keyed_using(root: &Path, kind: Keyed, key: &str) -> Guard { let path = keyed_lock_path(root, kind, key); let file = open_lock_file(&path); if !try_lock(&file, LOCK_SH) { @@ -286,6 +286,33 @@ pub fn keyed_using(root: &Path, kind: Keyed, key: &str) -> Guard { Guard { file, records_holder: false } } +/// What `key` names, held in use and whole: made by `make` under +/// [`keyed_building`] while `defect`, which says why it is not whole, says it is +/// not. A `make` that leaves it not whole is refused by that defect rather than +/// run again. +pub fn keyed_made( + root: &Path, + kind: Keyed, + key: &str, + defect: impl Fn() -> Option, + mut make: impl FnMut(), +) -> Guard { + loop { + let using = keyed_using(root, kind, key); + if defect().is_none() { + return using; + } + drop(using); + let _building = keyed_building(root, kind, key); + if defect().is_some() { + make(); + if let Some(defect) = defect() { + panic!("{} {key} was made, and is not whole: {defect}", kind.name()); + } + } + } +} + /// What `key` names, exclusively and only if nobody is making or using it: what /// a sweep holds while it removes one. pub fn keyed_idle(root: &Path, kind: Keyed, key: &str) -> Option { diff --git a/src/clang.rs b/src/clang.rs index a682d2619b1..188d0938a3e 100644 --- a/src/clang.rs +++ b/src/clang.rs @@ -115,21 +115,24 @@ fn absent(toolchain: &Path) -> Vec { gone } -/// Whether `toolchain` lacks any of the C toolchain. -pub(crate) fn missing(toolchain: &Path) -> bool { - !absent(toolchain).is_empty() +/// Why `toolchain` cannot compile C, if it cannot. +pub(crate) fn defect(toolchain: &Path) -> Option { + let gone = absent(toolchain); + (!gone.is_empty()).then(|| { + format!( + "the toyos toolchain at {} carries no {}, and every C compile needs it: \ + `clang::provision` puts it there after each build that makes a stage2, and it did not", + toolchain.display(), + gone.join(", "), + ) + }) } /// Refuse a toolchain directory without its C toolchain. pub(crate) fn assert_present(toolchain: &Path) { - let gone = absent(toolchain); - assert!( - gone.is_empty(), - "the toyos toolchain at {} carries no {}, and every C compile needs it: \ - `clang::provision` puts it there after each build that makes a stage2, and it did not", - toolchain.display(), - gone.join(", "), - ); + if let Some(defect) = defect(toolchain) { + panic!("{defect}"); + } } /// The one version directory under an LLVM build's `lib/clang`. @@ -200,13 +203,13 @@ mod tests { write(&bin(&stage2).join("rust-lld"), "lld"); write(&bin(&stage2).join("llvm-ar"), "the archiver"); - assert!(missing(&stage2)); + assert!(defect(&stage2).is_some()); let refused = std::panic::catch_unwind(|| assert_present(&stage2)).expect_err("no clang, and not refused"); let said = refused.downcast_ref::().expect("a formatted refusal"); assert!(said.contains("clang") && said.contains("ld.lld") && said.contains("include"), "{said}"); provision(&stage2); - assert!(!missing(&stage2)); + assert_eq!(defect(&stage2), None); assert_eq!(fs::read_to_string(bin(&stage2).join("clang")).unwrap(), "the clang"); assert!(!fs::symlink_metadata(bin(&stage2).join("clang")).unwrap().file_type().is_symlink(), "clang is a copy, not the link"); assert_eq!(fs::read_link(bin(&stage2).join("ld.lld")).unwrap(), Path::new("rust-lld")); @@ -223,6 +226,6 @@ mod tests { assert_eq!(fs::read_to_string(resource_parent(&stage2).join("23/include/stddef.h")).unwrap(), "v23"); fs::remove_file(bin(&stage2).join("llvm-ar")).unwrap(); - assert!(missing(&stage2), "a missing llvm-ar went unnoticed"); + assert!(defect(&stage2).is_some(), "a missing llvm-ar went unnoticed"); } } diff --git a/src/compiler.rs b/src/compiler.rs index 15e94f2e0d4..e9a53c5c41e 100644 --- a/src/compiler.rs +++ b/src/compiler.rs @@ -111,7 +111,7 @@ impl Compiler { } /// The primary's record of which `compiler/` its `stage2` was built from. -fn primary_record(rust_dir: &Path) -> PathBuf { +pub(crate) fn primary_record(rust_dir: &Path) -> PathBuf { rust_dir.join("build/toyos-compiler") } @@ -147,8 +147,7 @@ fn compiler_source(checkout: &Path) -> String { } /// Record which compiler the primary's `stage2` is. The primary calls this -/// after a toolchain build, and when the record is missing — its compiler stamp -/// has just said `stage2` is built from what its `rust/` holds. +/// after a toolchain build. pub fn record(rust_dir: &Path) { let at = primary_record(rust_dir); let want = source(rust_dir); @@ -157,6 +156,36 @@ pub fn record(rust_dir: &Path) { } } +/// Remove the record of which compiler the primary's `stage2` is. The primary +/// calls this before a toolchain build. +pub fn forget(rust_dir: &Path) { + let at = primary_record(rust_dir); + match fs::remove_file(&at) { + Err(e) if e.kind() != std::io::ErrorKind::NotFound => panic!("remove {}: {e}", at.display()), + _ => {} + } +} + +/// Whether the primary's `stage2` is the compiler `checkout`'s `compiler/` +/// names — `Err` when nothing records which compiler that is. +/// +/// **The source's content, never its files' times**: a checkout that rewrites a +/// file with the bytes it had is no new compiler. +fn primary_is(rust_dir: &Path, checkout: &Path) -> Result { + let names = source(checkout); + Ok(fs::read_to_string(primary_record(rust_dir))?.trim() == names) +} + +/// Whether the primary's `stage2` is built from what its own `rust/compiler/` +/// holds: false until a bootstrap has finished and [`record`]ed it. +pub fn primary_is_current(rust_dir: &Path) -> bool { + match primary_is(rust_dir, rust_dir) { + Ok(current) => current, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => false, + Err(e) => panic!("read {}: {e}", primary_record(rust_dir).display()), + } +} + /// The key of the compiler `fork`'s sources name: their content, so committing /// what was built as local changes names the same compiler. pub fn key(fork: &Path) -> String { @@ -204,16 +233,15 @@ fn choose(root: &Path, rust_dir: &Path, fork: &Path, build: impl Fn(&Path) -> Pa if fork == rust_dir { return Compiler::primary(rust_dir); } - let record = primary_record(rust_dir); - let built_from = fs::read_to_string(&record).unwrap_or_else(|e| { + let names_primary = primary_is(rust_dir, fork).unwrap_or_else(|e| { panic!( "{} cannot be read ({e}), so nothing says which compiler the primary's stage2 is, \ and no worktree can know whether it names that one.\n\ The primary checkout writes it: run `cargo run -- --build-only` there once.", - record.display(), + primary_record(rust_dir).display(), ) }); - if built_from.trim() == source(fork) { + if names_primary { let _ = fs::remove_file(&recorded); return Compiler::primary(rust_dir); } @@ -222,18 +250,16 @@ fn choose(root: &Path, rust_dir: &Path, fork: &Path, build: impl Fn(&Path) -> Pa fs::create_dir_all(recorded.parent().expect("a file under target/")).ok(); fs::write(&recorded, &key).unwrap_or_else(|e| panic!("write {}: {e}", recorded.display())); let mut placed = false; - let using = loop { - let using = buildlock::keyed_using(root, Keyed::Compiler, &key); - if dir.join(SOURCE).is_file() { - break using; - } - drop(using); - let _building = buildlock::keyed_building(root, Keyed::Compiler, &key); - if !dir.join(SOURCE).is_file() { + let using = buildlock::keyed_made( + root, + Keyed::Compiler, + &key, + || (!dir.join(SOURCE).is_file()).then(|| format!("{} carries no {SOURCE}", dir.display())), + || { place(root, fork, &key, &dir, &build); placed = true; - } - }; + }, + ); // A compiler edit loop places one per edit, and the one this replaced is // named by nobody now; the one in use is held, so the sweep leaves it. if placed { @@ -594,6 +620,43 @@ mod tests { assert_eq!(builds.get(), 0, "a missing record built a compiler"); } + /// **The primary bootstraps when its `compiler/` holds other content, and + /// never because its files' times moved**: every file rewritten with its own + /// bytes is the compiler just recorded. + #[test] + fn only_the_compiler_s_content_makes_the_primary_bootstrap() { + let scratch = TempDir::new("compiler-current"); + let (_primary, rust_dir, _) = estate(&scratch); + assert!(primary_is_current(&rust_dir), "the compiler just recorded is not current"); + + let files = snapshot(&rust_dir.join("compiler")); + let later = std::time::SystemTime::now() + std::time::Duration::from_secs(3600); + for (file, bytes) in &files { + fs::write(file, bytes).unwrap(); + fs::File::options().write(true).open(file).unwrap().set_modified(later).unwrap(); + assert_eq!(fs::metadata(file).unwrap().modified().unwrap(), later); + } + assert!(!files.is_empty()); + assert!( + primary_is_current(&rust_dir), + "every file of compiler/ was rewritten with its own bytes, and the primary would bootstrap" + ); + + let spec = rust_dir.join("compiler/rustc_target/src/lib.rs"); + let held = fs::read(&spec).unwrap(); + write(&spec, "pub fn targets() { riscv() }\n"); + assert!(!primary_is_current(&rust_dir), "a change to compiler/ kept the old stage2"); + fs::write(&spec, held).unwrap(); + assert!(primary_is_current(&rust_dir), "compiler/ put back is not the compiler recorded"); + + write(&rust_dir.join("compiler/rustc_target/src/new_target.rs"), "pub fn t() {}\n"); + assert!(!primary_is_current(&rust_dir), "an untracked file in compiler/ kept the old stage2"); + fs::remove_file(rust_dir.join("compiler/rustc_target/src/new_target.rs")).unwrap(); + + fs::remove_file(primary_record(&rust_dir)).unwrap(); + assert!(!primary_is_current(&rust_dir), "a stage2 nothing recorded was taken for current"); + } + /// `fork`'s LLVM checked out at a commit of its own. fn llvm_checkout(fork: &Path) -> PathBuf { let llvm = fork.join(LLVM); diff --git a/src/lib.rs b/src/lib.rs index b10a2a5f703..625dc9dd334 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -44,7 +44,6 @@ pub mod soundfont; /// build system at all. #[cfg(test)] pub mod sourcegate; -pub mod stamps; pub mod sysroot; pub mod testargs; pub mod tiers; diff --git a/src/stamps.rs b/src/stamps.rs deleted file mode 100644 index 1f441bbd9e2..00000000000 --- a/src/stamps.rs +++ /dev/null @@ -1,55 +0,0 @@ -use std::fs; -use std::path::Path; - -/// Compare a directory's fingerprint against a stored stamp. Returns true if changed. -/// Fingerprint is based on file mtimes and sizes for speed. -pub fn dir_changed(dir: &Path, stamp_path: &Path) -> bool { - if !stamp_path.exists() { - return true; - } - let current = fingerprint_dir(dir); - let stored = fs::read_to_string(stamp_path).unwrap_or_default(); - current != stored -} - -/// Write a directory's fingerprint to a stamp file. -pub fn write_dir_stamp(dir: &Path, stamp_path: &Path) { - if let Some(parent) = stamp_path.parent() { - fs::create_dir_all(parent).ok(); - } - let fp = fingerprint_dir(dir); - fs::write(stamp_path, fp).ok(); -} - - -fn fingerprint_dir(dir: &Path) -> String { - let mut entries: Vec = Vec::new(); - collect_entries(dir, &mut entries); - entries.sort(); - entries.join("\n") -} - -fn collect_entries(dir: &Path, entries: &mut Vec) { - let Ok(read_dir) = fs::read_dir(dir) else { return }; - for entry in read_dir.flatten() { - let path = entry.path(); - if path.is_dir() { - let name = path.file_name().unwrap().to_string_lossy().to_string(); - if name.starts_with('.') || name == "target" { - continue; - } - collect_entries(&path, entries); - } else if path.extension().is_some_and(|e| e == "rs" || e == "toml" || e == "h") { - if let Ok(meta) = fs::metadata(&path) { - let size = meta.len(); - let mtime = meta - .modified() - .ok() - .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) - .map(|d| d.as_nanos()) - .unwrap_or(0); - entries.push(format!("{}:{}:{}", path.display(), size, mtime)); - } - } - } -} diff --git a/src/sysroot.rs b/src/sysroot.rs index 3cc6302e15a..e6220f123fa 100644 --- a/src/sysroot.rs +++ b/src/sysroot.rs @@ -7,11 +7,10 @@ //! ([`SYSROOT_SOURCES`]), the std fork's `library/` and `src/bootstrap/` in the //! checkout that builds it, and the compiler that builds it. `rust/build/ //! sysroots//` is a whole toolchain — the compiler's files cloned from its -//! `stage2`, the guest targets' libraries built from this key's sources — and -//! nothing writes it after its [`SOURCES`] file exists. A build compiles against -//! the directory its own key names, so two worktrees with different ABIs or -//! different compilers never refuse or wait for each other, and main and every -//! branch matching it share one copy. +//! `stage2`, the guest targets' libraries built from this key's sources. A build +//! compiles against the directory its own key names, so two worktrees with +//! different ABIs or different compilers never refuse or wait for each other, +//! and main and every branch matching it share one copy. //! //! **Each worktree builds std in its own fork checkout, and nothing but the //! primary's own sync moves the primary's.** The primary builds in its `rust/`; @@ -300,9 +299,22 @@ pub fn recorded_key(root: &Path) -> Option { fs::read_to_string(root.join(RECORD)).ok().map(|k| k.trim().to_string()) } -/// Whether `dir` is a finished sysroot. -fn finished(dir: &Path) -> bool { - dir.join(SOURCES).is_file() +/// Why `dir` is not a finished sysroot, if it is not: no [`SOURCES`], or not a +/// whole toolchain (`toolchain::toolchain_defect`). One found with the first and +/// not the second is made again rather than trusted — all of it even when only +/// its `bin/cargo` link dangles, because that is rare and a sysroot has no +/// repair path. +fn unfinished(dir: &Path) -> Option { + if !dir.join(SOURCES).is_file() { + return Some(format!("{} carries no {SOURCES}", dir.display())); + } + toolchain::toolchain_defect(dir) +} + +/// The sysroot `key` names at `dir`, made by `make` if nobody has made it, and +/// held in use for as long as the returned guard lives. +fn held(root: &Path, key: &str, dir: &Path, make: impl FnMut()) -> Guard { + buildlock::keyed_made(root, Keyed::Sysroot, key, || unfinished(dir), make) } /// The sysroot this worktree's sources name, made if nobody has made it, and @@ -316,18 +328,7 @@ pub fn ensure(root: &Path, rust_dir: &Path, lock: &mut Held) -> Sysroot { fs::create_dir_all(record.parent().expect("a file under target/")).ok(); fs::write(&record, &key).unwrap_or_else(|e| panic!("write {}: {e}", record.display())); - let using = lock.without_shared(|| loop { - let using = buildlock::keyed_using(root, Keyed::Sysroot, &key); - if finished(&dir) { - break using; - } - drop(using); - let _building = buildlock::keyed_building(root, Keyed::Sysroot, &key); - if !finished(&dir) { - build(root, &compiler, &fork, &key, &dir); - } - }); - toolchain::assert_toolchain_is_honest(&dir); + let using = lock.without_shared(|| held(root, &key, &dir, || build(root, &compiler, &fork, &key, &dir))); Sysroot { dir, primary_compiler: compiler.primary, _using: Some(using) } } @@ -341,34 +342,54 @@ fn build(root: &Path, compiler: &Compiler, fork: &Path, key: &str, dir: &Path) { let _compiler = compiler.primary.then(|| buildlock::compiler_shared(root, &what)); eprintln!("Building sysroot {key}: std from {}, the compiler {}", fork.display(), compiler.stage2.display()); - let built = build_std(root, compiler, fork); + publish(compiler, dir, |partial| { + let built = build_std(root, compiler, fork); + for target in GUEST_TARGETS { + place_std(&stamp(&built, target), &partial.join("lib/rustlib").join(target).join("lib")); + } + let libc_target = dir.with_extension("libc-target"); + for arch in Arch::ALL { + crate::libc::build(root, partial, &libc_target, arch); + crate::libc::build_c(root, partial, &libc_target, arch); + } + let _ = fs::remove_dir_all(&libc_target); + + // The sources the key named are the ones built, or this is not that key's. + let again = self::key(root, compiler, fork); + assert!( + again == key, + "the sources moved while sysroot {key} was being built (they are now {again}); \ + nothing was kept, and the next build makes the one they name" + ); + format!("{key}\nfork {}\n{}\n", fork.display(), witness(root)) + }); +} + +/// Put at `dir` a whole toolchain: `compiler`'s files and what `fill` adds to +/// them, then the [`SOURCES`] `fill` returns, last. A `dir` already there is one +/// [`unfinished`] refused, and it is replaced. A compiler that is not whole is +/// refused before `fill` runs, and nothing is published. +fn publish(compiler: &Compiler, dir: &Path, fill: impl FnOnce(&Path) -> String) { + if let Some(defect) = toolchain::toolchain_defect(&compiler.stage2) { + let fix = if compiler.primary { + "\nA bootstrap in the primary checkout was stopped before it finished: \ + `cargo run -- --build-only` there completes it." + } else { + "" + }; + panic!("no sysroot is made from {}, and no std was built for one: {defect}{fix}", compiler.stage2.display()); + } let partial = dir.with_extension("partial"); if partial.exists() { fs::remove_dir_all(&partial).unwrap_or_else(|e| panic!("remove {}: {e}", partial.display())); } clone_tree(&compiler.stage2, &partial); - for target in GUEST_TARGETS { - place_std(&stamp(&built, target), &partial.join("lib/rustlib").join(target).join("lib")); - } - let libc_target = dir.with_extension("libc-target"); - for arch in Arch::ALL { - crate::libc::build(root, &partial, &libc_target, arch); - crate::libc::build_c(root, &partial, &libc_target, arch); + let sources = fill(&partial); + fs::write(partial.join(SOURCES), sources) + .unwrap_or_else(|e| panic!("write {}: {e}", partial.join(SOURCES).display())); + if dir.exists() { + fs::remove_dir_all(dir).unwrap_or_else(|e| panic!("remove {}: {e}", dir.display())); } - let _ = fs::remove_dir_all(&libc_target); - - // The sources the key named are the ones built, or this is not that key's. - let again = self::key(root, compiler, fork); - assert!( - again == key, - "the sources moved while sysroot {key} was being built (they are now {again}); \ - nothing was kept, and the next build makes the one they name" - ); - fs::write( - partial.join(SOURCES), - format!("{key}\nfork {}\n{}\n", fork.display(), witness(root)), - ) - .unwrap_or_else(|e| panic!("write {}: {e}", partial.join(SOURCES).display())); fs::rename(&partial, dir) .unwrap_or_else(|e| panic!("rename {} -> {}: {e}", partial.display(), dir.display())); } @@ -912,6 +933,114 @@ mod tests { assert!(message.contains(&c1) && message.contains(&c2), "{message}"); } + /// The primary's compiler under `base`: `rustc` and `rust-lld`, and the C + /// toolchain `src/clang.rs` provisions beside them if `clang`; no cargo. + fn primary_compiler(base: &Path, clang: bool) -> Compiler { + let compiler = Compiler::primary(&base.join("rust")); + write(&compiler.stage2.join("bin/rustc"), "rustc"); + let lld = toolchain::rust_lld(&compiler.stage2); + write(&lld, "lld"); + write(&lld.with_file_name("llvm-ar"), "llvm-ar"); + if clang { + for tool in ["clang", "ld.lld"] { + write(&lld.with_file_name(tool), tool); + } + write(&lld.parent().unwrap().parent().unwrap().join("lib/clang/22/include/stddef.h"), "stddef"); + } + compiler + } + + /// What a panic in `f` said. + fn refusal(f: impl FnOnce()) -> String { + let refused = std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)).expect_err("nothing was refused"); + *refused.downcast::().expect("a formatted refusal") + } + + /// **A sysroot is whole, or it is made again**: a `stage2` without cargo — + /// what bootstrap leaves until the primary completes it — is refused by + /// name and nothing is published, and one found with its `SOURCES` and + /// without its cargo is rebuilt rather than trusted, once. + #[test] + fn a_sysroot_is_whole_or_it_is_made_again() { + let base = TempDir::new("whole"); + git(&base, &["init", "-q"]); + let compiler = primary_compiler(&base, true); + let made = std::cell::Cell::new(0); + // `most` bounds the makes so far, so a make that loops fails rather than hangs. + let make = |dir: &Path, most: usize| { + made.set(made.get() + 1); + assert!(made.get() <= most, "a sysroot that was not whole was made again: make {}", made.get()); + publish(&compiler, dir, |partial| { + write(&partial.join("lib/rustlib/x86_64-unknown-toyos/lib/libstd.rlib"), "std"); + "found\n".to_string() + }) + }; + + let fresh = sysroots_dir(&base.join("rust")).join("fresh"); + let said = refusal(|| drop(held(&base, "fresh", &fresh, || make(&fresh, 1)))); + assert!(said.contains("is missing cargo") && said.contains("`cargo run -- --build-only`"), "{said}"); + assert!(!fresh.exists() && !fresh.with_extension("partial").exists(), "a sysroot was published from a stage2 without cargo"); + assert_eq!(made.get(), 1); + + let dir = sysroots_dir(&base.join("rust")).join("found"); + clone_tree(&compiler.stage2, &dir); + write(&dir.join(SOURCES), "found\n"); + toolchain::provision_toolchain_cargo(&compiler.stage2); + let using = held(&base, "found", &dir, || make(&dir, 2)); + assert_eq!(made.get(), 2, "a sysroot without its cargo was trusted because it has SOURCES"); + assert_eq!(toolchain::toolchain_defect(&dir), None); + assert!(dir.join("lib/rustlib/x86_64-unknown-toyos/lib/libstd.rlib").is_file()); + drop(using); + drop(held(&base, "found", &dir, || make(&dir, 2))); + assert_eq!(made.get(), 2, "a whole sysroot was made again"); + } + + /// **A sysroot that cannot be made whole is refused after one make, never + /// made again**: a `stage2` without clang — what a stopped bootstrap leaves — + /// is refused before any std is built for it, with nothing published, and a + /// make that leaves its sysroot not whole is refused by what it lacks. + #[test] + fn a_sysroot_that_cannot_be_made_whole_is_made_once_and_refused() { + let base = TempDir::new("no-clang"); + git(&base, &["init", "-q"]); + let compiler = primary_compiler(&base, false); + toolchain::provision_toolchain_cargo(&compiler.stage2); + let made = std::cell::Cell::new(0); + let once = || { + made.set(made.get() + 1); + assert_eq!(made.get(), 1, "a sysroot that was not whole was made again"); + }; + + let dir = sysroots_dir(&base.join("rust")).join("cloned"); + let filled = std::cell::Cell::new(false); + let said = refusal(|| { + drop(held(&base, "cloned", &dir, || { + once(); + publish(&compiler, &dir, |_| { + filled.set(true); + "cloned\n".to_string() + }) + })) + }); + assert!(said.contains("carries no") && said.contains("/clang"), "{said}"); + assert!(said.contains(&compiler.stage2.display().to_string()) && said.contains("`cargo run -- --build-only`"), "{said}"); + assert!(!filled.get(), "a std was built for a sysroot of a compiler without clang"); + assert!(!dir.exists() && !dir.with_extension("partial").exists(), "a sysroot was published from a stage2 without clang"); + assert_eq!(made.get(), 1); + + made.set(0); + let dir = sysroots_dir(&base.join("rust")).join("made"); + let said = refusal(|| { + drop(held(&base, "made", &dir, || { + once(); + clone_tree(&compiler.stage2, &dir); + write(&dir.join(SOURCES), "made\n"); + })) + }); + assert!(said.starts_with("sysroot made was made, and is not whole") && said.contains("/clang"), "{said}"); + assert_eq!(made.get(), 1); + } + /// A key no registered worktree records goes, and so does a half-built one; /// a key a worktree records stays, and so does one somebody is using. #[test] diff --git a/src/toolchain.rs b/src/toolchain.rs index 7a6e64ef86b..d274c54787f 100644 --- a/src/toolchain.rs +++ b/src/toolchain.rs @@ -6,14 +6,13 @@ use std::sync::OnceLock; use crate::arch::Arch; use crate::buildlock; use crate::buildlock::Scope; -use crate::stamps; use crate::sysroot::{self, Sysroot, SYSROOT_SOURCES}; /// Whether the primary's compiler needs a bootstrap. `invalidate_hosted` /// separates "the compiler changed" from "the rustup link is missing": only the /// first makes the ToyOS-hosted rustc stale, and rebuilding that one costs /// minutes. -#[derive(Clone, Copy, PartialEq)] +#[derive(Clone, Copy, PartialEq, Debug)] struct Bootstrap { invalidate_hosted: bool, } @@ -286,15 +285,6 @@ fn cargo_link_stale(stage2: &Path) -> bool { /// Put a `cargo` beside the toolchain's `rustc`. /// -/// **The one step that provisions it, and every path that can produce a -/// toolchain directory goes through it**: the primary's bootstrap and its -/// staleness rebuild (both upstream of [`ensure`]'s call), a linked worktree -/// adopting the shared one, and a runner that unpacked the published artifact. -/// The fix this replaces was made once, by hand, in a step the rebuild path does -/// not run — so the 2026-08-14 sysroot rebuild recreated `bin/` without it and -/// nothing noticed, and CI, which links its toolchain fresh from the artifact -/// every run, never had it at all. -/// /// **A symlink, and what survives the artifact round-trip is this step rather /// than the link.** `src/release.rs` excludes it from the tarball for the reason /// it excludes `lib/rustlib/`: it names a path only the publishing runner @@ -309,35 +299,100 @@ pub(crate) fn provision_toolchain_cargo(stage2: &Path) { }); } -/// Refuse a toolchain layout that would make rustup narrate, or that has no -/// linker for the guest targets. +/// Why the toolchain at `stage2` is not whole, if it is not: a binary rustup +/// would narrate a fallback for, no linker for the guest targets, or no C +/// toolchain (`src/clang.rs`). /// -/// Unconditional and after the step that provisions, because the defect being -/// gated is a provisioning step that silently stopped running: a check that only -/// runs when the step runs asserts nothing about the build that skipped it. -pub(crate) fn assert_toolchain_is_honest(stage2: &Path) { +/// The one definition of whole: [`assert_toolchain_is_honest`] refuses by it, +/// and a sysroot is finished only by it (`src/sysroot.rs`). +pub(crate) fn toolchain_defect(stage2: &Path) -> Option { let bin = stage2.join("bin"); let narrated = narrated_binaries(&bin); - assert!( - narrated.is_empty(), - "the toyos toolchain at {} is missing {}, so rustup answers for {} by falling back to \ - another toolchain and narrating it on every invocation.\n\ - provision_toolchain_cargo is the step that puts them there, and it did not.", - bin.display(), - narrated.join(" and "), - if narrated.len() == 1 { "it" } else { "them" }, - ); + if !narrated.is_empty() { + return Some(format!( + "the toyos toolchain at {} is missing {}, so rustup answers for {} by falling back to \ + another toolchain and narrating it on every invocation.\n\ + provision_toolchain_cargo is the step that puts them there, and it did not.", + bin.display(), + narrated.join(" and "), + if narrated.len() == 1 { "it" } else { "them" }, + )); + } // Every guest target names `rust-lld` and rustc looks for it here, so a // toolchain without it is refused here, by name, rather than at the first link. let lld = rust_lld(stage2); - assert!( - lld.is_file(), - "the toyos toolchain at {} carries no {}, the linker every guest target names: \ - bootstrap puts it there when `write_config` says `lld = true`, and it did not", - stage2.display(), - lld.display(), - ); - crate::clang::assert_present(stage2); + if !lld.is_file() { + return Some(format!( + "the toyos toolchain at {} carries no {}, the linker every guest target names: \ + bootstrap puts it there when `write_config` says `lld = true`, and it did not", + stage2.display(), + lld.display(), + )); + } + crate::clang::defect(stage2) +} + +/// Refuse a toolchain that is not whole ([`toolchain_defect`]). +/// +/// Unconditional and after the step that provisions, because the defect being +/// gated is a provisioning step that silently stopped running: a check that only +/// runs when the step runs asserts nothing about the build that skipped it. +pub(crate) fn assert_toolchain_is_honest(stage2: &Path) { + if let Some(defect) = toolchain_defect(stage2) { + panic!("{defect}"); + } +} + +/// Whether the primary's toolchain lacks what bootstrap does not put there: +/// `stage2`'s cargo and clang, or the host target in the hosted rustc's sysroot. +fn incomplete(rust_dir: &Path) -> bool { + let stage2 = stage2(rust_dir); + cargo_link_stale(&stage2) || crate::clang::defect(&stage2).is_some() || host_target_missing(rust_dir) +} + +/// Give the primary's toolchain what [`incomplete`] finds missing. +fn complete(rust_dir: &Path) { + let stage2 = stage2(rust_dir); + if cargo_link_stale(&stage2) { + provision_toolchain_cargo(&stage2); + } + if crate::clang::defect(&stage2).is_some() { + crate::clang::provision(&stage2); + } + if host_target_missing(rust_dir) { + link_host_target(rust_dir); + } +} + +/// Run `bootstrap` in the primary's `rust/`, then [`complete`] what it +/// reassembled. Called inside the act that holds the global lock exclusively. +/// +/// **In the same hold, because bootstrap recreates `stage2` without its cargo +/// and clang**: a completion under a hold of its own queues behind every sysroot +/// build that takes the lock shared in between, and those last minutes. +fn reassemble(rust_dir: &Path, bootstrap: impl FnOnce()) { + bootstrap(); + complete(rust_dir); +} + +/// [`reassemble`] the primary's compiler with `bootstrap`, with nothing recording +/// which compiler `stage2` is until it is whole: a bootstrap that is stopped is +/// run again by the primary, and refused by name in every linked worktree. +fn rebuild_compiler(rust_dir: &Path, bootstrap: impl FnOnce()) { + crate::compiler::forget(rust_dir); + reassemble(rust_dir, bootstrap); + crate::compiler::record(rust_dir); +} + +/// What the primary bootstraps: a new compiler when asked to or when `stage2` +/// is not the one its `compiler/` names, and the same one again when rustup has +/// no `toyos` toolchain to run. +fn bootstrap(force_rebuild: bool, current: bool, toolchain_exists: bool) -> Option { + if force_rebuild || !current { + Some(Bootstrap { invalidate_hosted: true }) + } else { + (!toolchain_exists).then_some(Bootstrap { invalidate_hosted: false }) + } } /// Ensure the toolchain is up to date, and return the sysroot this checkout's @@ -395,12 +450,12 @@ pub fn ensure(root: &Path, force_rebuild: bool, lock: &mut buildlock::Held) -> S Owner::Us => {} } - let compiler_stamp = stamps_dir.join("compiler.stamp"); let hosted_stamp = stamps_dir.join("hosted-rustc.stamp"); lock.act_if( Scope::Global, "build the rust toolchain", || { + let current = crate::compiler::primary_is_current(&rust_dir); let toolchain_exists = Command::new("rustup") .args(["run", "toyos", "rustc", "--version"]) .stdout(std::process::Stdio::null()) @@ -408,39 +463,16 @@ pub fn ensure(root: &Path, force_rebuild: bool, lock: &mut buildlock::Held) -> S .status() .map(|s| s.success()) .unwrap_or(false); - // A `stage2` that links another LLVM than the one `rust/` names — - // or one no record says was built from `src/llvm-project` at all — - // is a compiler this checkout no longer describes. Named before any - // build, so an LLVM edit no commit holds is refused before one. - let names = crate::compiler::source(&rust_dir); - let moved = - fs::read_to_string(rust_dir.join("build/toyos-compiler")).is_ok_and(|built| built.trim() != names); - if stamps::dir_changed(&rust_dir.join("compiler"), &compiler_stamp) || moved || force_rebuild { - Some(Bootstrap { invalidate_hosted: true }) - } else if !toolchain_exists { - Some(Bootstrap { invalidate_hosted: false }) - } else { - None - } + bootstrap(force_rebuild, current, toolchain_exists) }, |kind| { eprintln!("Building full toolchain (this takes a while on first run)..."); - full_bootstrap(root, &rust_dir); - stamps::write_dir_stamp(&rust_dir.join("compiler"), &compiler_stamp); - crate::compiler::record(&rust_dir); + rebuild_compiler(&rust_dir, || full_bootstrap(root, &rust_dir)); if kind.invalidate_hosted { let _ = fs::remove_file(&hosted_stamp); } }, ); - // The compiler stamp above has just said `stage2` is built from what `rust/` - // holds, so a missing record is written from it. - lock.act_if( - Scope::Global, - "record which compiler the toolchain is", - || (!rust_dir.join("build/toyos-compiler").exists()).then_some(()), - |()| crate::compiler::record(&rust_dir), - ); let hosted_rustc = rust_dir.join(format!("build/{}/stage2/bin/rustc", HOSTED_ARCH.userland())); lock.act_if( @@ -448,7 +480,7 @@ pub fn ensure(root: &Path, force_rebuild: bool, lock: &mut buildlock::Held) -> S "build the ToyOS-hosted rustc", || (!hosted_stamp.exists() || !hosted_rustc.exists()).then_some(()), |()| { - build_hosted_rustc(&rust_dir); + reassemble(&rust_dir, || build_hosted_rustc(&rust_dir)); assert!(hosted_rustc.exists(), "Failed to build hosted rustc"); fs::write(&hosted_stamp, "").unwrap(); }, @@ -468,32 +500,14 @@ pub fn ensure(root: &Path, force_rebuild: bool, lock: &mut buildlock::Held) -> S }, ); - // After both bootstrap steps above, because either of them recreates `bin/` - // and a fix that lives upstream of a rebuild is a fix that rots. Every - // sysroot clones this `bin/`, so it is where the cargo is provisioned. lock.act_if( Scope::Global, - "give the toyos toolchain its own cargo", - || cargo_link_stale(&stage2).then_some(()), - |()| provision_toolchain_cargo(&stage2), - ); - lock.act_if( - Scope::Global, - "give the toyos toolchain the clang of its LLVM", - || crate::clang::missing(&stage2).then_some(()), - |()| crate::clang::provision(&stage2), + "complete the toyos toolchain", + || incomplete(&rust_dir).then_some(()), + |()| complete(&rust_dir), ); assert_toolchain_is_honest(&stage2); - // The hosted rustc's own sysroot needs the host target proc-macros compile - // against. - lock.act_if( - Scope::Global, - "add the host target to the ToyOS sysroot", - || host_target_missing(&rust_dir).then_some(()), - |()| link_host_target(&rust_dir), - ); - sysroot::ensure(root, &rust_dir, lock) } @@ -527,7 +541,8 @@ fn check_installed_toolchain(root: &Path, rust_dir: &Path, force_rebuild: bool) // toolchain this machine has, which is not a path any artifact can know. // This is CI's whole share of the cargo provisioning — it links its // toolchain fresh from the published artifact on every run, so nothing - // upstream of the download can have put one there. + // upstream of the download can have put one there. Its clang is the + // artifact's own, so this is not `complete`. if host_target_missing(rust_dir) { link_host_target(rust_dir); } @@ -1060,6 +1075,81 @@ mod tests { } } + /// **A bootstrap leaves the primary nothing that waits on another + /// worktree's sysroot build**: the act that reassembles `stage2` completes it + /// before its exclusive hold ends, so the step after it — run while a + /// sysroot build holds the lock shared — decides it has nothing to do, and + /// takes no lock. + #[test] + fn a_bootstrap_leaves_nothing_to_wait_on_a_sysroot_build_for() { + let rust_dir = TempDir::new("no-wait"); + let stage2 = stage2(&rust_dir); + // The LLVM bootstrap builds beside `stage2`, which `clang::provision` reads. + let llvm = stage2.with_file_name("llvm"); + for (file, text) in [("bin/clang", "clang"), ("lib/clang/22/include/stddef.h", "stddef")] { + fs::create_dir_all(llvm.join(file).parent().unwrap()).unwrap(); + fs::write(llvm.join(file), text).unwrap(); + } + + reassemble(&rust_dir, || { + // What bootstrap leaves: `stage2` made again, with `rustc` and the + // LLVM tools it assembles, and neither cargo nor clang; and the + // hosted rustc's sysroot without the host target. + let _ = fs::remove_dir_all(&stage2); + let lld = rust_lld(&stage2); + for file in [stage2.join("bin/rustc"), lld.clone(), lld.with_file_name("llvm-ar")] { + fs::create_dir_all(file.parent().unwrap()).unwrap(); + fs::write(file, b"").unwrap(); + } + let hosted = rust_dir.join(format!("build/{}/stage2/lib/rustlib", HOSTED_ARCH.userland())); + fs::create_dir_all(&hosted).unwrap(); + }); + assert!( + !incomplete(&rust_dir), + "a bootstrap let its exclusive hold go with a global step left, which the primary's \ + build then queues for behind every sysroot build" + ); + assert_eq!(toolchain_defect(&stage2), None, "a bootstrap let its exclusive hold go with stage2 not whole"); + } + + /// **A stopped bootstrap is run again**: nothing records which compiler + /// `stage2` is while one runs, so the primary's next build is not told the + /// old one is current. + #[test] + fn a_stopped_bootstrap_leaves_no_record() { + let rust_dir = TempDir::new("stopped"); + let record = crate::compiler::primary_record(&rust_dir); + fs::create_dir_all(record.parent().unwrap()).unwrap(); + fs::write(&record, "the compiler before").unwrap(); + let stopped = std::panic::catch_unwind(|| rebuild_compiler(&rust_dir, || panic!("stopped"))); + assert!(stopped.is_err()); + assert!(!record.exists(), "a stopped bootstrap left the record of the compiler before it"); + } + + /// **The primary bootstraps a new compiler exactly when asked to or when its + /// `stage2` is not current, and otherwise only when rustup has none.** + #[test] + fn the_primary_bootstraps_when_asked_stale_or_missing() { + let new = Some(Bootstrap { invalidate_hosted: true }); + let again = Some(Bootstrap { invalidate_hosted: false }); + for (force_rebuild, current, toolchain_exists, want) in [ + (false, true, true, None), + (false, true, false, again), + (false, false, true, new), + (false, false, false, new), + (true, true, true, new), + (true, true, false, new), + (true, false, true, new), + (true, false, false, new), + ] { + assert_eq!( + bootstrap(force_rebuild, current, toolchain_exists), + want, + "force_rebuild {force_rebuild}, current {current}, toolchain_exists {toolchain_exists}" + ); + } + } + /// The negative control is the defect itself: this is verbatim what cargo /// wrote for a worktree build before the override existed. #[test]