User copies are scatter-gather: a window is its physical runs, each pinned

issues/kernel/a-user-copy-pin-that-is-never-released-reds-no-test.md

@@ -0,0 +1,26 @@
+---
+status: open
+kind: defect
+opened: 2026-09-29
+---
+
+# A user-copy pin that is never released reds no test
+
+`kernel/src/user_ptr.rs`'s `impl Pins for Pmm` is the one pin path no host
+test reaches: the host tests of `toyos_userbound::Pinned` drive a counting
+fake. With its `unpin` emptied (`fn unpin(&mut self, _run: Segment) {}`),
+every frame a syscall ever copied through keeps a pin. `pmm::free_page` still
+counts such a frame free, and `alloc_page` and `alloc_contiguous` skip it for
+the rest of the boot, so the machine loses memory that no free count shows and
+no known guest test reads.
+
+## Exit condition
+
+A test that goes red under that mutation: for example, a guest job that copies
+into and unmaps more frames than the guest has, or a census that counts free
+frames still holding a pin and a test that asserts it returns to zero once no
+copy is in flight. Then this file is deleted.
+
+## Owner
+
+`kernel/src/user_ptr.rs`, `kernel/src/mm/pmm.rs`. Nobody holds it.

issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md

@@ -18,9 +18,6 @@ IOMMU's superpages, the kernel's own mappings. One paging design serves both,
 on x86-64 and on the ARM64 the tree is kept portable for
 (`issues/kernel/toyos-runs-on-arm64.md`).
 
-**Blocked on:** the owner's word to start. Recorded, not started, at the
-owner's instruction.
-
 **The constraint that makes this a track, measured on the T14** (run 29
 readback, `PMM: 100/16038MB used` at 11 s with four user processes and three
 kernel threads live; the same record's rows: stack 16 pages held, init-tls 5,
@@ -38,3 +35,51 @@ longer shares a 2 MiB page with a neighbour's registers, so the relocation
 that `kernel/src/pcidev/` does before a hand-over is no longer needed for a
 window firmware placed apart from its neighbours. Placing a window inside the
 host bridge's firmware-reported apertures stays correct under either page size.
+
+## Stages
+
+The owner approved starting this work ("Start it") and delegated every
+kernel-interface decision to the orchestrator, whose stages and rulings
+follow. **Commit is strict**, by the orchestrator's ruling: a mapping
+reserves its frame count when it is made and is refused by name then, never
+killed at a fault. No promotion (the kernel has no thread to do it) and no
+live split of a user leaf.
+
+1. **Scatter-gather user copies** (#593) owe one measurement: what the bulk
+   window's `Vec<Segment>` costs under the kernel heap's lock. It is one
+   heap allocation per bulk copy, taken before the address-space lock; the
+   host A/B in #593's review put a 64 MiB `read` at 1.28–1.31× main's, 32
+   to 40 ns more. No job measures it in-guest yet: the stage owes
+   `copy_cost`, a job beside `syscall_cost` that times `read` into a 64 KiB
+   and a 64 MiB window, and its metal row, so that
+   `cargo test -- --metal copy_cost` at the stage's merge and at its base is
+   the A/B.
+2. **Superframe frame allocator.** 4 KiB frames carved from 2 MiB
+   superframes; the pin rule held per superframe, so no pinned frame is
+   reissued and no superframe holding one is handed out whole; a watermark
+   refuses an allocation before the machine runs dry. Exit: host tests of
+   carve, free, pin and the refusal; every 2 MiB caller of the PMM unchanged;
+   `user_copy_spans_windows` and `munmap_reissues_second_read_window` red
+   again under their negative controls, because each is two physical runs
+   only while the allocator hands out the lowest free frame first.
+3. **4 KiB user leaves.** A software `OWNED` bit in the leaf is the ledger of
+   the frames a process owns, with per-process counts; stacks are demand-zero
+   and TLS is 4 KiB. A typed syscall value crossing a page is served in
+   segments, so `is_user_object` stops refusing a straddle and
+   `abuse_page_straddle`'s refusals become delivery verdicts. A 2 MiB window
+   of 4 KiB leaves from arbitrary frames is up to 512 runs, so the one-run
+   bound `user_ptr::window_split` panics on becomes reachable from userland
+   and goes with the stage. Exit: a
+   process's floor on the T14 against the 10 to 14 MB above.
+4. **Demand-zero anonymous `mmap`.** A fault installs a 2 MiB leaf only for a
+   whole aligned 2 MiB span of the mapping; unmapping a range no thread touched
+   sends no IPI; `munmap` refuses a size that is not the whole mapping, by the
+   orchestrator's ruling. Exit: `mmap` of more than the watermark allows
+   refused at `mmap`, and a guest that touches one page of a large mapping
+   holds one frame.
+5. **File-backed faults at 4 KiB.** ROOT's read-only text is shared
+   zero-copy. Exit: two processes of one binary hold its text once.
+6. **`/apps` image sharing.** The design is the owner's decision, owed when
+   this stage is reached.
+7. **Retire 2 MiB where it no longer pays.** Exit: every remaining 2 MiB user
+   leaf is a stage-4 whole span or a device or DMA grant.

kernel/TECHNICAL_DEBT.md

@@ -8,22 +8,6 @@ Removed. Kernel PML4 has no PML4[0..255] entries. Process page tables start
 empty and build user mappings on demand. SMP trampoline uses the bootloader's
 PML4 for AP transition, then switches to kernel PML4 in ap_entry.
 
-## 2. user_ptr assumes physically contiguous user buffers
-
-`user_ptr::window` walks every 2 MiB boundary a buffer crosses and refuses one
-whose pages are not physically adjacent, so a non-contiguous buffer is a
-`BadAddress` and not a silent misread. What remains is that it *only* refuses:
-ToyOS allocates user memory in contiguous 2 MiB blocks, so nothing produces one
-today, and swap, COW fork or non-contiguous VMAs would turn ordinary reads and
-writes into refusals rather than into corruption.
-
-**Fix, when one of those arrives:** `UserBytes`/`UserBytesMut` already hand out
-no reference, so the change is confined to how a window addresses its pages —
-a run list instead of one base pointer, with `read_at`/`write_at` splitting at
-the boundaries. No caller's signature moves.
-
-**Not urgent while all user allocations are 2MB-aligned contiguous blocks.**
-
 ## ~~3. No DmaAddr newtype~~ DONE
 
 Added `DmaAddr` newtype in addr.rs. `DmaPool::page_phys()` returns `DmaAddr`.

kernel/src/arch/aarch64/paging.rs

@@ -68,7 +68,7 @@ impl AddressSpace {
         match self.never {}
     }
 
-    pub fn translate_writable(&self, _vaddr: UserAddr) -> Option<crate::mm::DirectMap> {
+    pub fn leaf(&self, _vaddr: UserAddr, _access: toyos_userbound::Access) -> Option<(u64, u64)> {
         match self.never {}
     }
 

kernel/src/arch/x86_64/paging.rs

@@ -556,21 +556,28 @@ impl AddressSpace {
     /// Checked here, not at the callers: a user space shallow-copies the
     /// kernel PML4 half, so a kernel address would otherwise walk to a writable kernel page.
     pub fn translate(&self, vaddr: UserAddr) -> Option<crate::mm::DirectMap> {
-        self.walk(vaddr).map(|(dm, _)| dm)
+        self.walk(vaddr).map(|(dm, _, _)| dm)
     }
 
-    /// As [`translate`](Self::translate), but only where a user store would
-    /// land: every level of the walk grants `USER` and `WRITE`, as the MMU
-    /// demands of a ring 3 store under `CR0.WP`. A kernel copy into user
-    /// memory goes through this, so a syscall cannot write a page the process
-    /// itself may not — the clock page, a shared library's `.text`.
-    pub fn translate_writable(&self, vaddr: UserAddr) -> Option<crate::mm::DirectMap> {
+    /// `vaddr`'s physical address and the bytes from it to the end of the leaf
+    /// that maps it, which that one walk answers for. A `Write` is answered
+    /// only where a user store would land: every level of the walk grants
+    /// `USER` and `WRITE`, as the MMU demands of a ring 3 store under
+    /// `CR0.WP`. A kernel copy into user memory goes through this, so a syscall
+    /// cannot write a page the process itself may not — the clock page, a
+    /// shared library's `.text`.
+    pub fn leaf(&self, vaddr: UserAddr, access: toyos_userbound::Access) -> Option<(u64, u64)> {
         const STORE: u64 = PAGE_USER | PAGE_WRITE;
-        self.walk(vaddr).and_then(|(dm, rights)| (rights & STORE == STORE).then_some(dm))
+        let (dm, rights, size) = self.walk(vaddr)?;
+        let granted = match access {
+            toyos_userbound::Access::Read => true,
+            toyos_userbound::Access::Write => rights & STORE == STORE,
+        };
+        granted.then(|| (dm.phys(), size - (vaddr.raw() & (size - 1))))
     }
 
-    /// The direct-map address of `vaddr` and the rights every level of its walk grants in common.
-    fn walk(&self, vaddr: UserAddr) -> Option<(crate::mm::DirectMap, u64)> {
+    /// The direct-map address of `vaddr`, the rights every level of its walk grants in common, and the size of the leaf that maps it.
+    fn walk(&self, vaddr: UserAddr) -> Option<(crate::mm::DirectMap, u64, u64)> {
         let va = vaddr.raw();
         if !toyos_userbound::is_user_addr(va) {
             return None;
@@ -593,11 +600,11 @@ impl AddressSpace {
                 return None;
             }
             let dm = crate::mm::DirectMap::from_phys((pte & ADDR_MASK) + (va & 0xFFF));
-            return Some((dm, rights & pte));
+            return Some((dm, rights & pte, 4096));
         }
         let page_phys = pde & ADDR_MASK_2M;
         let offset = va & (PAGE_2M - 1);
-        Some((crate::mm::DirectMap::from_phys(page_phys + offset), rights))
+        Some((crate::mm::DirectMap::from_phys(page_phys + offset), rights, PAGE_2M))
     }
 
     /// Where `span` goes, top-down and never below the floor: a region the