Skip to content

The kernel boots on QEMU's stock edk2, network included: the direct map ends at the last memory, every kernel root slot exists before the first user space, and a BAR's free run is a gap inside its width's space - #563

Merged
Japabu merged 11 commits into
mainfrom
wt/toyos-edk2
Sep 28, 2026

Conversation

@Japabu

@Japabu Japabu commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Under the edk2 firmware that Homebrew's QEMU 11.1.1 ships, the base kernel dies right after pmm: with memory allocation of 4096 bytes failed. With the repository's ovmf/ the same image boots. The boot hits three defects, and this PR fixes all three:

  1. The direct map reached every range in the map. paging::init mapped physical memory up to the highest end of every descriptor, whatever its type, and took its page tables from the 512 KiB early bump heap (128 pages). edk2 hands an AMD vCPU (-cpu qemu64, 40 address bits) QEMU's HyperTransport reservation as EfiReservedMemoryType. The captured map's last descriptor is type 0 0xfd00000000..0x10000000000. Mapping up to 1 TiB takes 1024 page directories, so the heap runs out.
  2. A kernel root slot made after a user space existed was missing from that space. new_user copied the kernel's present root slots once. With the extent fixed, the direct map ends at 4 GiB (slot 256). edk2 puts the xHCI BAR at 0xc000004000 (slot 257). The kernel mapped it at 0.662 s, after init's space was made at 0.654 s. init's device_claim then read the virtio-net BAR at 0xc000000000 on init's CR3. The read faulted: KERNEL PANIC: read unmapped address at 0xffff80c000000004 in kernel::pcidev::place_bars+0x800, with the page walk showing PML4[257] … PML4E: 0x0 P=0 on CR3 0x1407000. The 60 s panic hold then reset the machine through acpi → xhci::stop::before_reset, which faulted again under the same CR3: FAULT rip=0xffff80007bfbae6b cr2=0xffff80c000004440 … RECURSIVE. That rip is kernel::drivers::xhci::stop::stop_all+0x6db, with Live::stop inlined. cr2 is the BAR plus 0x440, which is port 1's PORTSC (CAPLENGTH 0x40 + OP_PORT_BASE 0x400): the reset path's first register read.
  3. A 64-bit BAR was offered only the space above the highest extent firmware described. That extent is the same reservation, ending at 1 TiB, and it lies outside every window firmware declared: mem 0x80000000..0x81100000, mem 0xc000000000..0xc000100000, mem 0x81100000..0xe0000000, mem 0xc000100000..0xe000000000. So pcidev's 64-bit run was 0x10000000000..0x10003000000, inside no window, and the virtio NIC's BAR 4 (firmware's 0xc000000000, 0x4000 bytes) was refused: pcidev: PCI 00:03.0 NOT HANDED OVER — this machine has no 2 MiB-aligned 64-bit address space both above what firmware assigned and inside a window firmware declared to offer its BAR, then netd: no NIC on this machine, exiting. The rule is on main unchanged; main's kernel dies at pmm: on this firmware before it reaches it. Neither the direct map's extent nor the 2 MiB alignment enters the survey.

What changed, per decision

  • One rule for the direct map's extent: toyos_bootmap::x86_64::direct_map_end. It lives beside the boot map because the kernel may never map less than the boot map did. The kernel keeps using addresses it took through the boot map: the black box page, the parameter buffer, and serial::init's ACPI reads. So the floor is BOOT_MAP_BYTES, not a second constant. Above the floor, the extent is the end of the highest range the kernel reads as memory, rounded up to 2 MiB, wherever that range sits in the unordered map. "Memory" is the pmm's usable types plus ACPI reclaim and NVS. The floor sits behind the x86-64 boundary because mapping the low 4 GiB whole is safe only where the MTRRs type it. AArch64 types by the map, and its kernel page tables are still owed (stage 4).
  • The extent is a sealed type. direct_map_end answers a toyos_bootmap::DirectMapEnd, whose constructor is private. Only direct_map_end makes one, and get() reads it. The kernel keeps it in a DirectMapEndCell, which holds no other number. A crate that writes DirectMapEnd(1 << 52) does not compile (E0603), and a compile_fail,E0603 doctest pins that.
  • Memory past the direct map's window is refused by name. DIRECT_MAP_WINDOW is root slots 256..511 at PHYS_OFFSET: 0x800000000000, 128 TiB. A const assert beside the kernel's PHYS_OFFSET pins it to 0 - PHYS_OFFSET. A memory range ending past it is Refusal::PastWindow(end), and paging::init panics with it. This replaces a saturating_add that answered an end below the range's own end.
  • The usable-type rule moved too. is_usable_type and EFI_LOADER_DATA now live in the same crate, so one list of UEFI type numbers answers what the pmm hands out and what the map reaches.
  • The ACPI reader's bound is toyos_acpi::Mapped. The kernel supplies only the byte read (toyos_acpi::Memory for DirectMemory) and wraps it in Mapped::new(DirectMemory, mm::direct_map_end()). Mapped::readable refuses address zero and any range with a byte at or past the DirectMapEnd it was made with. The kernel no longer implements readable, so it has no bound of its own to put back. Before paging::init the extent is the boot map's, and afterwards the kernel's own. This holds on both architectures, since AArch64 keeps the boot map's. MAX_PHYS is gone from the reader, and issues/kernel/the-direct-map-bound-on-a-firmware-address-is-52-bits.md is closed. The bootloader's watchdog.rs comment that cited the kernel's bound is cut.
  • Every kernel root slot exists before the first user space. paging::init installs the slots its direct map needs, from the early heap. seal_kernel_half installs the rest once the heap is the pmm's, in mm::init right after alloc::init. new_user asserts every kernel slot is present when it copies them. The second-level tables are shared, so every later kernel mapping reaches every space, the reset path's MMIO included. The cost is at most 255 more 4 KiB tables from the kernel heap (≈1 MiB, by arithmetic). The filed issue this replaces is gone.
  • paging::init logs the extent it built: paging: the direct map covers 0x0..0x100000000.
  • One rule makes both lists of free runs: toyos_pci::placement::free_runs. A run is a gap between the firmware map, the BARs firmware assigned and the ranges bridges forward: below PLATFORM_MMIO for a 32-bit BAR, as before, and from 4 GiB (WIDE_FLOOR) for a 64-bit one, where it was the 48 MiB above the highest of them. An extent inside another opens no run inside the one that holds it: a bridge's forwarded range holds the BARs behind it. The floor keeps the two lists disjoint, so no address is handed out from both. placement::reserve still offers only an address inside a declared window, and place_bar still settles a candidate on the function answering its own dword there. The kernel's free_runs_below_4g, window and WINDOW_SPAN are deleted, and PLATFORM_MMIO moved into the crate with the rule. On this machine the NIC's first candidate is 0xc000200000, inside mem 0xc000100000.
  • A bridge's 64-bit prefetchable window is decoded whole. A gap rule above 4 GiB has to know what bridges forward there. bridge::prefetch reads the upper base and limit dwords into the range, and decides "disabled" on the whole address. It replaces prefetch_below_4g, which dropped a window above 4 GiB. PciDevice::forwarded returns every range, and prefetch_is_64_bit is private.
  • A false paragraph on pcidev::probe_dword ("the boot map already covers every physical address …") is deleted.

Net lines against main: production Rust +345/−216 (net +129; the kernel net −51), tests +482/−12 (127 of them the captured map), manifests and lockfiles +16/−6, issues +109/−26.

Gates

At d6716fc7 (this head, after merging origin/main):

  • cargo run -- --ci host: EXIT=0 ([ci] Host: 49 step(s), all green). This includes the host workspace's tests, the compile_fail doctest, and clippy with warnings denied over the kernel on both architectures and over the bootloader.
  • cargo test -p toyos-bootmap: EXIT=0 (direct_map.rs 11, plan.rs 22, 1 doctest).
  • cargo test -p toyos-acpi: EXIT=0 (mapped.rs 3, corpus.rs 24, fixtures.rs 10, resource.rs 12).
  • cargo test -p toyos-pci: EXIT=0 (75 tests).
  • cargo run -- --build-only: EXIT=0 for the green, no-seal and red trees (build-arm.sh).

High-risk checks (memory management, a firmware trust boundary)

Independent oracle for the placement: the function itself, on the machine. place_bar settles a candidate only when the NIC answers at it the dword it answered where firmware put it, and netd: DHCP: lease needs the device to move packets through that BAR. The fixture is checked against the boot it came from: on its extents the rule gives exactly the two 32-bit runs of 2 MiB or more and the three 64-bit runs that boot printed (0x100000000..0xc000000000, 0xc000008000..0xfd00000000, 0x10000000000..), in edk2s_32_bit_runs_are_the_ones_its_boot_printed and edk2s_64_bit_runs_are_the_ones_its_boot_printed.

Independent oracle for the extent: UEFI §7.2, the table of memory-type usage after ExitBootServices under EFI_BOOT_SERVICES.AllocatePages(). It gives the OS loader code and data, boot-services code and data, and conventional memory, and nothing else. It puts no bound on where a reserved or MMIO range may sit, and the specification does not order the map. the_pmm_hands_out_exactly_the_types_uefi_gives_the_os pins {1, 2, 3, 4, 7} against it. The second oracle is the recorded real failure: edk2's own map, captured descriptor by descriptor (127 entries) from a boot of this kernel, is the fixture. Its usable bytes and entry count equal that boot's pmm: line (2140844032 in 112), and the extent it gives is the boot map's.

Negative controls, host, at 58660e78. Each is a checked patch, shown to build (the tests and doctests compile, EXIT=0), run, and reversed with the tree shown clean (mutate.sh).

The ACPI reader's bound:

mutation build test red on
a1-acpi-max-phys: the same revert where the bound now lives, Mapped::readable bounded by 1 << 52 0 101 a_table_past_the_direct_map_is_refused_before_a_byte_is_read ("a table at 0xfd00000000", left: None) and an_xsdt_entry_past_the_direct_map_is_skipped (left: None, right: Some(Absent))
a2-acpi-no-zero: phys != 0 dropped 0 101 address_zero_and_a_range_past_every_address_are_refused ("address zero")
s1-seal-open: DirectMapEnd(pub u64) 0 101 the doctest DirectMapEnd (line 66) - compile fail ("Test compiled successfully, but it's marked compile_fail")
a crate outside toyos-bootmap forging DirectMapEnd(1 << 52) 101 — error[E0603]: tuple struct constructor DirectMapEnd is private, by a standalone cargo check

The placement's:

mutation build test red on
pci-ma-at-next-end: at = at.max(next.end) → at = next.end 0 101 an_extent_inside_another_opens_no_run (a run from 6 GiB + 0x4000), both edk2 64-bit tests, a_run_is_free_and_in_one_list_once
pci-mb-sort-by-end: taken sorted by end 0 101 an_extent_inside_another_opens_no_run (4 GiB..6 GiB against 4 GiB..5 GiB)
pci-m1-base-rule: a 64-bit BAR's runs put back to the base's rule, the 48 MiB above the highest extent, the whole rule change reverted 0 101 edk2_offers_a_64_bit_bar_the_window_it_declared, edk2s_64_bit_runs_are_the_ones_its_boot_printed, an_extent_inside_another_opens_no_run, taken_is_read_in_any_order_and_overlapping
pci-m2-floor-zero: WIDE_FLOOR = 0 0 101 a_run_is_free_and_in_one_list_once (a run in both lists), the edk2 64-bit tests, the nested test
pci-m3-unsorted: taken not sorted 0 101 taken_is_read_in_any_order_and_overlapping, all three edk2 tests, a_run_is_free_and_in_one_list_once
pci-m4-prefetch-low-half: a 64-bit prefetchable window read by its low half 0 101 a_64_bit_prefetchable_window_is_its_upper_dwords_too

The extent's:

mutation build test red on
bootmap-m6-base-rule: direct_map_end replaced by the base's arithmetic (every descriptor, 4 GiB floor), the whole extent change reverted 0 101 the_reserved_hole_below_1_tib_is_not_mapped and four more
bootmap-m1-take-while: .filter → .take_while 0 101 an_unsorted_map_is_read_whole
bootmap-m2-last-in-order: the last memory entry in map order 0 101 an_unsorted_map_is_read_whole
bootmap-m3-no-window: the window check removed 0 101 memory_past_the_window_is_refused_by_name
bootmap-m3b-window-boundary: > → >= 0 101 memory_past_the_window_is_refused_by_name
bootmap-m4-no-boot-services-code: type 3 dropped from the usable set 0 101 the_pmm_hands_out_exactly_the_types_uefi_gives_the_os and the_captured_map_is_usable_where_its_boot_said

Guest arms, run by the orchestrator at 6e227596 on the stock-edk2 line (build-arm.sh, then run-arm.sh, which stops QEMU by PID):

  • Green: all five owed lines — paging: the direct map covers 0x0..0x100000000, mmio: 0xc000004000+0x10000 PAT Uncacheable, pcidev: PCI 00:03.0 BAR 4 (0x4000 bytes) placed at 0xc000200000 — inside firmware's mem 0xc000100000, netd: DHCP: lease, compositor: ready — and no PANIC, FAULT rip or RECURSIVE.
  • Red (main at 1ec6daa9): pmm:, then EARLY PANIC: panicked at library/alloc/src/alloc.rs:659:9: with memory allocation of 4096 bytes failed under it.
  • No seal (this head with seal_kernel_half and its call deleted): PANIC: panicked at src/arch/x86_64/paging.rs:387:13: with new_user: kernel root slot 257 is absent, and this space would never see what is mapped there under it, when init's space is made.
  • On ovmf/: bar_placement_is_proven EXIT=0.
  • The Fast tier at this landing head is run by the orchestrator.

What I am unsure of

  • No machine in reach puts a 64-bit BAR it hands over behind a bridge. So the kernel line that feeds bridge::prefetch's ranges into the survey has no arm that reds when it is deleted, and neither has the line that feeds the assigned BARs. The decode itself is host-tested (pci-m4). Filed.
  • The T14 has not been booted under the 64-bit rule. The base offered no 64-bit run there below 0x60_3dc0_0000, the end of its 64-bit window. This rule offers the gaps of 0x40_0000_0000..0x60_3dc0_0000, a change in real-hardware behaviour with no reading.

Filed

  • issues/kernel/the-direct-maps-page-directories-come-from-a-512-kib-heap.md: the early heap still limits the direct map to about 120 GiB of memory. That figure is an estimate: 512 KiB / 4 KiB = 128 pages, before the root, one second-level table and earlier allocations.
  • issues/kernel/map-mmio-takes-an-address-past-the-direct-map-window.md: map_mmio does not bound phys by the 128 TiB window, and vtd still bounds by 1 << 52.
  • issues/build/no-harness-test-boots-qemus-own-edk2.md: filed, not added to the release PR's tests. The host tests here are the only standing guard, and its exit condition includes netd: DHCP: lease.
  • issues/panic-path/a-panic-inside-mm-init-may-not-reach-the-panel.md: from paging::init's CR3 load to panic_console::remap, a scanout past the direct map's end is unmapped, so a panic in that span faults instead of painting.
  • issues/kernel/nothing-reds-when-the-bar-survey-drops-an-assigned-bar.md: the two kernel pushes into the survey's taken have no arm.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j

Under the edk2 firmware Homebrew's QEMU 11.1.1 ships, the kernel died
right after `pmm:` with `memory allocation of 4096 bytes failed`.
`paging::init` mapped physical memory up to the highest `end` of every
descriptor in the UEFI map, whatever its type, and took the page
tables from the 512 KiB early bump heap (128 pages). QEMU gives an AMD
vCPU (`-cpu qemu64`) with 40 physical address bits a reserved e820
range for the HyperTransport hole, 0xfd00000000..0x10000000000, and
edk2 carries it into the UEFI map as EfiReservedMemoryType. The
loader's `GCD:` line shows edk2 saw it: its 64-bit PCI window sits at
0xc000000000..0xe000000000, below that reservation, where the repo's
`ovmf/` puts its own at 0x800000000. Mapping to 1 TiB takes 1024 page
directories, eight times the early heap, so a 4096-byte table is the
allocation that fails. The repo's `ovmf/` never names the range, so it
boots.

The UEFI specification's usage table after ExitBootServices says a
reserved range is not usable and an MMIO range is not used by the OS;
a conforming map may still put either anywhere. So the extent is now
one rule in a new pure crate, `toyos-memmap::direct_map_end`: the low
4 GiB, and above it the end of the highest range the kernel reads as
memory (what the pmm hands out, and ACPI reclaim and NVS), rounded up
to 2 MiB. MMIO past it is mapped on demand by `map_mmio`, as every
64-bit BAR already was.

The pmm's usable-type rule moves into the same crate, so the one list
of UEFI types holds both answers and a host test holds the containment
the pmm rests on: every type it hands out is inside the direct map.
`paging::init` logs the extent it built.

Filed: issues/kernel/a-kernel-mapping-made-after-a-user-space-exists-is-missing-from-it.md,
the kernel root entries `map_mmio` can add after a user space copied
them. This change leaves it as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu marked this pull request as ready for review September 27, 2026 21:38
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #563 at 492c2cb, round 1.

Readiness. CI host (run 36352441628) is QUEUED, not green. orch-runs/summary.txt has no 563-edk2 line, so the green, red and diag guest arms have not run. I checked the host arms from their logs: m1, m2 and m3 each build (EXIT=0) and are red on the named tests, and the green run passes all 6 tests. The findings below come from reading the code and hold whatever the guest runs show.

BLOCKER

  • toyos-memmap/ (the whole crate) — a sibling of toyos-bootmap. That crate is pure, architecture-neutral and host-tested, the kernel already depends on it (kernel/Cargo.toml:402), and it decides how far a page table maps physical memory. PAGE_2M (lib.rs:24) and DIRECT_MAP_FLOOR (lib.rs:28) re-declare its PAGE_2M and BOOT_MAP_BYTES. The floor must never be smaller than the boot map: the kernel keeps using addresses it took through the boot map before mm::init (the black box PAGE, the parameter buffer, serial::init's ACPI reads), and two constants cannot hold that. Fix: fold direct_map_end and is_usable_type into toyos-bootmap, with the floor as BOOT_MAP_BYTES. The new crate, its manifest, its workspace entry and both lockfile hunks are then deleted.

  • kernel/src/drivers/acpi.rs:42,49 — the ACPI reader is not bounded by the narrower map. DirectPhys::readable still bounds a firmware-named address by 1 << 52, while this change narrows what the direct map covers. A table in a non-memory range above max(4 GiB, the last memory end) is read on the base and faults in Ring 0 at this head, on untrusted firmware input. The exit condition of issues/kernel/the-direct-map-bound-on-a-firmware-address-is-52-bits.md says the missing piece is a reader for the map's ceiling, and this branch writes that reader. Fix: readable refuses by name any address past the extent paging::init built, and past the boot map before paging::init, on both architectures. MAX_PHYS and that issue are deleted. Add a test that turns red when readable is put back to MAX_PHYS.

  • toyos-memmap/src/lib.rs:58-61 — mutations survive on high-risk code. Two of them pass all six tests, because every fixture is sorted and no non-memory entry comes before the highest memory:

    • (a) .filter( → .take_while(.
    • (b) map.iter().rev().find(|e| is_read_as_memory(e.uefi_type)).map(round).map_or(DIRECT_MAP_FLOOR, |x| x.max(DIRECT_MAP_FLOOR)).

    UEFI does not order the map GetMemoryMap returns. Required test: [e(RESERVED, 0xfd_0000_0000, 0x100_0000_0000), e(CONVENTIONAL, 4*GIB, 8*GIB), e(CONVENTIONAL, 0x10_0000, 2*GIB)] gives 8 GiB. It is red under both (a) and (b).

    • (c) saturating_add → wrapping_add also passes. A usable range ending at u64::MAX gives 0xFFFF_FFFF_FFE0_0000 under saturating_add, which is below the range's own end, so the containment this PR claims is already false there. Under wrapping_add it collapses to the floor. Either refuse by name a memory range past what the direct-map window holds, or pin this result with a test that is red under (c).

Guest runs required (to be ready, not findings)

  • Green: an image built from a clean 492c2cb, with its head and porcelain recorded. green.img was built at 23:27, before the 23:37 commit, and nothing records what it was built from. Use the stock edk2 command line. The log must show:

    • paging: the direct map covers 0x0..0x100000000
    • the mmio: line for the xHCI BAR, which says whether root entry 257 exists before init
    • the pcidev: … placed at line for 00:03.0
    • netd: DHCP: lease
    • compositor: ready

    queue4.sh stops at the first compositor: ready, which can print before netd's claim. The arm must wait for both.

  • Red, the negative control: the merge base 1808fb8, built and booted on the same command line. Expected: EARLY PANIC … memory allocation of 4096 bytes failed after pmm:. release-r1/probe.log was built from wt/toyos-release (c551894 plus that branch's work), not this base. g-red-base-extent.patch reverts only the extent, onto this head.

  • Diag: the printed map. EDK2_Q35_AMD becomes the captured descriptors, or its claim goes (see REMOVE).

NOTE

  • issues/kernel/a-kernel-mapping-made-after-a-user-space-exists-is-missing-from-it.md — pre-existing, not caused by this PR.
    • The base never booted with a kernel root entry of 257 or higher made at init. Reaching 512 GiB takes 512 page directories, against the 512 KiB early heap.
    • On the base, the same fault is reachable wherever firmware puts a 64-bit window at or above 512 GiB with no descriptor there.
    • Here, probe-ovmf.log:261-266 shows the claim is made by init, on init's CR3: handed over at 0.449, netd spawned at 0.456. place_bar moves the BAR into the 64-bit run at 0xc000100000, which is in root entry 257. That is hidden only if the kernel's xHCI BAR is in 0xc000000000..0xc000100000 and mapped before init.
    • The green arm decides it. If the claim faults, the title's claim is false and this becomes a BLOCKER.
    • Cheapest exit: install every kernel root entry's PDPT once, after alloc::init and before the first new_user (256 pages). After that, a kernel-root install in ensure_table is a named panic.
  • kernel/src/arch/x86_64/paging.rs:912-927 — the root cause remains for RAM. Page tables still come from the 512 KiB early bump heap, one page directory per GiB. A machine with more than about 120 GiB of memory dies with this same panic. That figure is an estimate, not a measurement: the pmm draws on the same heap first. File it, with the arithmetic taken from a command.
  • toyos-memmap/src/lib.rs:26-28 — the 4 GiB floor is x86 policy. On AArch64 a Normal mapping over device registers is not permitted. Once the rule lives in toyos-bootmap, it takes the per-architecture split that crate already has.
  • toyos-memmap/tests/direct_map.rs:65 — the usable-type set is not pinned. is_usable_type's exact set {1,2,3,4,7} has no test: dropping EFI_BOOT_SERVICES_CODE passes every test, because the pmm and the map lose the type together. Add one assert against the §7.2 table.
  • CI cannot see this regression. No harness test boots QEMU's own edk2, and ovmf/ never names the reserved hole. The host test is the only guard that stays.

REMOVE

  • toyos-memmap/src/lib.rs:46 "reserved and I/O ranges are nothing a write-back mapping may cover" — false: the floor and every hole below end are mapped write-back.
  • toyos-memmap/src/lib.rs:26-27 "and neither is described as memory" — false: ACPI's tables are types 9 and 10, which this function counts as memory.
  • toyos-memmap/tests/direct_map.rs:20-23 — says this is the map edk2 hands over, but the map is reconstructed. Remove it unless the diag capture replaces the fixture.
  • toyos-memmap/Cargo.toml:1-6 — narration.
  • kernel/src/pcidev/mod.rs:1289-1291 "the boot map already covers every physical address, so this takes no address space …" — false, and the sibling of the map_2m clause this branch deleted.
  • Issue file, lines 24-27, "Unmeasured: no boot here has printed edk2's BARs" and "probe_dword's comment says … which is false" — the green run and the removal above make them stale.
  • PR body:
    • the "Expected:" lines under "Guest arms" and the first two bullets of "What I am unsure of" — the measured lines take their place; they are not reworded.
    • "on netd's CR3" — the claim runs on init's CR3.
    • "The guest arm checks this line" — queue4.sh does not check it.

Growth: git diff --shortstat gives 12 files, +220/−31. From --numstat: production Rust +72/−31, tests +84, manifests and lockfiles +33, issue +31. The first BLOCKER deletes the crate manifest, the workspace entry, both lockfile hunks and two constants.

SEND BACK

Japabu and others added 5 commits September 28, 2026 00:02
…ry kernel root slot follow it

toyos-memmap is folded into toyos-bootmap: the kernel's direct map may never
reach less than the boot map, and two crates with two constants could not
hold that. `x86_64::direct_map_end` floors at `BOOT_MAP_BYTES`, sits behind
the architecture boundary because mapping the low 4 GiB whole is safe only
where the MTRRs type it, and refuses by name memory past
`DIRECT_MAP_WINDOW` (root slots 256..511, 128 TiB) rather than saturating to
an end below the range it was asked to cover.

`DirectPhys::readable` is bounded by the direct map's extent through
`toyos_bootmap::reaches`: the boot map's until `paging::init` stores its
own, instead of x86-64's 52-bit width. `MAX_PHYS` goes from the ACPI reader
and `the-direct-map-bound-on-a-firmware-address-is-52-bits` is closed.

`paging::init` installs the root slots its direct map needs, and
`seal_kernel_half` installs the rest once the heap is the pmm's; a user
space asserts every kernel slot is present when it copies them, and
`ensure_table` never creates one. That closes
`a-kernel-mapping-made-after-a-user-space-exists-is-missing-from-it`.

Tests: an unsorted map with a reserved entry first gives 8 GiB, memory
ending at u64::MAX is refused by name, the usable set is exactly UEFI
§7.2's {1,2,3,4,7}, and an address past the direct map is not read. The
fixture no longer claims to be edk2's map.

Filed: the early heap's ceiling on the direct map, map_mmio's unbounded
address, and the missing edk2 boot in the harness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…flow

The case one byte past the window runs first, so a direct_map_end with no
window check fails on the assertion that names it, whatever the profile's
overflow checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… under

`DirectPhys` holds the end `mm::direct_map_end` answered when it was made,
and `readable` bounds by it through `toyos_bootmap::reaches`. A `readable`
that stops consulting the extent leaves the field unread, which the kernel's
clippy gate denies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 127 descriptors QEMU 11.1.1's edk2 handed a 2 GiB q35 guest with
`-cpu qemu64`, as the kernel printed them (diag boot, edk2-r1/diag.log).
The last is type 0 at 0xfd00000000..0x10000000000, the HyperTransport
reservation the reconstructed fixture assumed. The fixture's usable bytes
and entries are what that boot's pmm counted, and the RSDP and the five
tables the boot read are inside the direct map the fixture gives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu Japabu changed the title The direct map reaches memory, not every range the firmware map names The kernel boots on QEMU's stock edk2: the direct map ends at the last memory, and every kernel root slot exists before the first user space Sep 27, 2026
… lies above the highest extent

On QEMU's stock edk2 the virtio NIC was not handed over: the 64-bit run
was the 48 MiB above the highest extent firmware described, and that
extent is the reserved range 0xfd00000000..0x10000000000, outside every
window firmware declared, so the run at 0x10000000000 was inside none and
BAR 4 had no address to be placed at. The window firmware declared for
64-bit BARs, 0xc000100000..0xe000000000, was never looked at.

The rule was in pcidev on main and unchanged by this branch; main's kernel
died at pmm: on this firmware before reaching it.

One rule now makes both lists: toyos_pci::placement::free_runs, the gaps
between the firmware map, the assigned BARs and the bridges' forwarded
ranges, below PLATFORM_MMIO for a 32-bit BAR and from 4 GiB for a 64-bit
one. A gap rule above 4 GiB has to know what bridges forward there, so
bridge::prefetch decodes a 64-bit prefetchable window whole, upper dwords
included, where prefetch_below_4g dropped it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu Japabu changed the title The kernel boots on QEMU's stock edk2: the direct map ends at the last memory, and every kernel root slot exists before the first user space The kernel boots on QEMU's stock edk2, network included: the direct map ends at the last memory, every kernel root slot exists before the first user space, and a BAR's free run is a gap inside its width's space Sep 27, 2026
@Japabu

Japabu commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #563 at 3f1edf1, round 3 (last reviewed head: 492c2cb).

Readiness.

  • CI host is SUCCESS at 3f1edf1 (run 36356066591).
  • cargo test -p toyos-pci m1–m4 each build (EXIT=0) and each is red (EXIT=101) on the named tests (edk2-r3/mutations.out).
  • Stock edk2, from orch-runs/summary.txt and edk2-r3/*.log:
    • green 3f1edf1: owed=[ Y Y Y Y Y ], no PANIC.
    • red e5ffe95: pmm: then EARLY PANIC … alloc.rs:659:9 / memory allocation of 4096 bytes failed.
    • noseal: noseal.log:139-140, new_user: kernel root slot 257 is absent.
  • ovmf: bar_placement_is_proven EXIT=0. Fast tier: 392/4.

Round-1 BLOCKERs

  • toyos-memmap sibling: CLOSED. The crate is deleted, and git grep toyos-memmap over tracked files is empty. direct_map_end is in toyos_bootmap::x86_64 with BOOT_MAP_BYTES as the floor.
  • ACPI reader unbounded by the narrower map: OPEN, on its test only.
    • Behaviour is fixed: readable goes through reaches(self.end, …), MAX_PHYS is gone and the issue is deleted.
    • The test I required, red when readable goes back to 1 << 52, does not exist. k1 is red only because a field goes unread. The PR's own "unsure" bullet says the whole revert passes every gate. See BLOCKER 1.
  • Mutations (a) take_while, (b) last-in-order, (c) wrapping: CLOSED.
    • m1 and m2 are red on an_unsorted_map_is_read_whole.
    • (c) is replaced by Refusal::PastWindow. m3 and m3b are red on memory_past_the_window_is_refused_by_name.
    • m4 (type 3) is red on the §7.2 pin.

The orchestrator's four reds

The judgment holds. The new placement cannot plausibly make a netd swap's re-claim fail.

  • A re-claim does not run the rule. place_bar → cut_already returns the window already cut for that BAR, and free_runs/reserve are never reached. Both failing consoles show it: "released … where it may still be aimed is kept for its next claim", then "a grant of a range's size is placed there … handed over on slot 0".
  • Every re-claimed netd works. Each takes a DHCP lease, init prints swap netd: in service (swap_netd) or restored (swap_crash_rolls_back), and logd serves on 41337. The finding is only on the host side: "redial was turned away 64 times".
  • On ovmf the placement did not move. This head places 00:03.0 BAR 4 at 0x800200000, which is byte-identical to 557r2, 559r3 and 554r5 on the old rule. It is the same address because the 1 MiB window 0x800000000..0x800100000 cannot hold a 2 MiB span under either rule.
  • The same failure shows without this diff. 557r2's lan_swap has the identical two-line signature.
  • launcher_refusals is a flake already on record. Its [("PipeWrite", …), ("Connection", …)] leak is the "second red mode" of issues/build/parallel-tests-red-under-other-suites.md, recorded before this branch.

BLOCKER

  1. kernel/src/drivers/acpi.rs:43-56 — the ACPI reader's claim has no test that can fail. This is a firmware trust boundary. The patch below passes every gate:

    -pub struct DirectPhys { end: u64 }
    +pub struct DirectPhys;
    -Self { end: crate::mm::direct_map_end() }
    +Self
    -phys != 0 && toyos_bootmap::reaches(self.end, phys, len as u64)
    +phys != 0 && phys.checked_add(len as u64).is_some_and(|end| end <= 1 << 52)
    

    Something must turn red on it. Two ways:

    • a boot actuator that asks find_table for a table at mm::direct_map_end(), plus a harness test that reads the refusal;
    • or the bound moved into toyos_acpi::Phys, so the kernel cannot express the revert and toyos-acpi's host tests pin it.
  2. toyos-pci/src/placement.rs:35,46 — two mutations of free_runs pass every test. No fixture holds one extent inside another. Every taken in the tests has ends in the same order as starts. The mutations are:

    • (a) at = at.max(next.end) → at = next.end;
    • (b) sort_unstable_by_key(|extent| extent.start) → |extent| extent.end.

    Nesting is the normal case on real hardware: a bridge's forwarded window holds the BARs behind it, and an MMIO descriptor holds BARs. Under either mutation a run opens inside the bridge's forwarded range, and alone_in_its_page checks only BARs, so it does not catch this. Required test: free_runs(&mut [W(5 GiB, 8 GiB), W(6 GiB, 6 GiB + 0x4000)], true) yields exactly [W(4 GiB, 5 GiB), W(8 GiB, u64::MAX)], and it is red under (a) and under (b).

NOTE

  • toyos-pci/src/placement.rs:143-158 — the fixture's xHCI extent is wrong.
    • The fixture has xHCI ending at 0xc000014000. That boot's survey took BAR 0's decoded 16 KiB: the high run it printed starts at 0xc000008000 (edk2-r3/green.log:102).
    • Set the end to 0xc0_0000_8000 and assert the three high runs that boot printed (green.log:101-103). That makes the 64-bit list a recorded-failure oracle, which today only the 32-bit list is.
  • kernel/src/pcidev/mod.rs:479,494 — the kernel inputs to taken are not exercised by any arm.
    • Deleting taken.push(Window { start: memory.address(), end }) still places the NIC at 0xc000200000 on edk2 and 0x800200000 on ovmf, because the 2 MiB alignment steps over firmware's BARs. alone_in_its_page would panic before an overlap.
    • The forwarded push is untested too; the PR already says so.
  • kernel/src/arch/x86_64/paging.rs:818-822 — the ensure_table assert repeats new_user's (389). After seal_kernel_half no kernel root slot is ever absent. Only one guard is needed. new_user's is the measured red, so delete the other.
  • toyos-bootmap/tests/direct_map.rs:238 — the test pins a copy. It checks DIRECT_MAP_WINDOW against a PHYS_OFFSET copied into the test, not the kernel's. Replace it with const _: () = assert!(toyos_bootmap::DIRECT_MAP_WINDOW == 0u64.wrapping_sub(PHYS_OFFSET)); at kernel/src/mm/mod.rs:33, and delete the test.
  • kernel/src/main.rs:384-385 — a panic inside mm::init may not reach the panel. Between paging::init's CR3 load and panic_console::remap, the panel writes through the kernel's own map. That window now includes seal_kernel_half's 255 allocations. A scanout past direct_map_end that the base's every-descriptor extent covered is unmapped there, so such a panic faults instead of painting.
  • The T14 is unmeasured under the new 64-bit rule.
    • The base's high run started above every BAR, at or past 0x60_3dc0_0000, which is the end of the T14's 64-bit window. So the base refused every 64-bit placement there.
    • This rule offers the gaps of 0x40_0000_0000..0x60_3dc0_0000: a change in real-hardware behaviour with no reading.
  • The PR body does not carry the measurements the claims rest on.
    • Missing: the green, red and noseal arms at 3f1edf1/e5ffe950 (command, EXIT, log), bar_placement_is_proven EXIT=0, and the fast tier's 392/4 with its four reds adjudicated.
    • The noseal owed line names kernel/src/…, but the panic prints src/arch/x86_64/paging.rs:387:13.

REMOVE

  • toyos-pci/src/bar.rs:433-434 "moved to the first 2 MiB window above what firmware assigned": false under this rule.
  • toyos-pci/src/placement.rs:143 "as that boot printed it", and 145 "(xHCI's to the end its mmio: line names)": false, see NOTE.
  • toyos-pci/src/placement.rs:30-31 "(q35's edk2 reserves 0xfd00000000..0x10000000000)": provenance in source.
  • kernel/src/pcidev/mod.rs:454-458: rewritten; it restates free_runs's contract and the Machine::low doc.
  • toyos-bootmap/Cargo.toml:1-7: rewritten narration of who depends on the crate.
  • PR body, all of these:
    • the "Owed at this head" block;
    • the third "unsure" bullet (heap budget; the fast tier has run);
    • "and that is the Fast tier's, which is the orchestrator's";
    • "The fixture is that boot's own record: … its BAR addresses".

Growth: 25 files, +764/−247.

  • Production Rust: +286/−209 (net +77). The kernel is net −36; toyos-bootmap/src and toyos-pci/src are net +114.
  • Tests: +396/−9.
  • Manifests and lockfiles: +17/−3.
  • Issues: +65/−26.

The kernel shrinks. The NOTE deletions (the ensure_table assert, the copied-constant test) are the ones left.

SEND BACK

Japabu and others added 2 commits September 28, 2026 07:41
… firmware's map can make

The kernel's ACPI reader implemented `Phys::readable` itself, so putting it
back to the architecture's 52-bit ceiling passed every gate. The bound now
lives in `toyos_acpi::Mapped`, which the kernel wraps its byte read in
(`toyos_acpi::Memory`), and `tests/mapped.rs` pins it: a table in edk2's
reserved hole at 0xfd00000000, at the map's end, or one byte over it is
`Unmapped`; an XSDT entry pointing there is skipped; address zero and a range
past every address are refused. `toyos_bootmap::reaches` goes with it.

The end it bounds by is `toyos_bootmap::DirectMapEnd`, whose constructor is
private (E0603, pinned by a compile_fail doctest): only `x86_64::direct_map_end`
and `DirectMapEnd::BOOT` make one, and the kernel keeps it in a
`DirectMapEndCell`, which holds no other number.

`DIRECT_MAP_WINDOW` is asserted against the kernel's own `PHYS_OFFSET` at
compile time instead of against a copy in a test. `ensure_table`'s assert
repeated `new_user`'s and goes.

pci: a nested extent opens no run inside the one that holds it
(`an_extent_inside_another_opens_no_run`), and the edk2 fixture's xHCI
extent is its decoded 16 KiB, so the 64-bit runs are asserted as the three
that boot printed.

Filed: a panic inside `mm::init` may not reach the panel; nothing reds when
the BAR survey drops an assigned BAR or a forwarded range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
@Japabu

Japabu commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #563 at 6e22759, round 4 (last reviewed head: 3f1edf1).

Readiness.

  • CI host is SUCCESS at 6e22759 (run 36384257025).
  • Host arms, from edk2-r4/mutations.out: all 15 build (EXIT=0) and all 15 are red (EXIT=101), each restored clean. The final toyos-acpi and toyos-bootmap runs are green, including the compile_fail doctest.
  • Stock edk2 at 6e22759, from orch-runs/summary.txt and edk2-r4/*.log:
    • green: owed=[ Y Y Y Y Y ], no PANIC, FAULT rip or RECURSIVE. The 64-bit runs printed at green.log:101-103 equal the fixture's three.
    • red, main at 1ec6daa: red.log:55-57, pmm: then EARLY PANIC … alloc.rs:659:9 / memory allocation of 4096 bytes failed.
    • noseal: noseal.log:139-140, paging.rs:387:13 / new_user: kernel root slot 257 is absent.
  • ovmf at 6e22759: bar_placement_is_proven EXIT=0, NIC placed at 0x800200000.
  • Owed: the Fast tier on ovmf at 6e22759. queue24 has written no 563r4 fast line. The only Fast tier on record is 3f1edf1's, 392/4.

Round-3 BLOCKERs

  1. The ACPI reader's claim has no test that can fail: CLOSED.
    • The bound now lives in toyos_acpi::Mapped::readable. The kernel supplies only Memory::byte, so my round-3 revert has no line left to apply to (review-revert-at-head.log: patch does not apply).
    • The same revert where the bound now lives is a1 (<= 1 << 52). It builds, and it is red on a_table_past_the_direct_map_is_refused_before_a_byte_is_read and an_xsdt_entry_past_the_direct_map_is_skipped.
    • a2 (dropping phys != 0) is red on address_zero_and_a_range_past_every_address_are_refused.
    • The end cannot be forged as DirectMapEnd(1 << 52). s1 (pub u64) turns the compile_fail,E0603 doctest red.
  2. free_runs nesting mutations (a) and (b): CLOSED.
    • an_extent_inside_another_opens_no_run is the test I required, verbatim.
    • pci-ma (at = next.end) is red on it and three more.
    • pci-mb (sort by end) is red on it.

Round-3 NOTEs, all closed:

  • The ensure_table assert is deleted.
  • The const assert sits at kernel/src/mm/mod.rs:34 and the copied-constant test is gone.
  • The xHCI extent ends at 0xc0_0000_8000, and edk2s_64_bit_runs_are_the_ones_its_boot_printed asserts the three runs this head's own boot printed.
  • Two issues are filed, and the T14 is in "unsure".
  • Every round-3 source REMOVE is done.

BLOCKER

None.

NOTE

  • PR body: it is not yet main's record for this head.
    • "Guest arms, run by the orchestrator at 3f1edf10" and the Fast-tier bullet cite the previous head.
    • Replace them with the 6e22759 runs: green owed=[ Y Y Y Y Y ]; red on main 1ec6daa (pmm:, then EARLY PANIC … alloc.rs:659:9); noseal (paging.rs:387:13, slot 257 absent); bar_placement_is_proven EXIT=0.
    • Then add the 6e22759 Fast tier, with every red adjudicated. That is the one run still owed.
  • toyos-bootmap/src/lib.rs:74 — DirectMapEnd::BOOT is production API with no production caller.
    • DirectMapEndCell::boot() builds from BOOT_MAP_BYTES, and only toyos-acpi/tests/mapped.rs names BOOT.
    • Delete it, and have the tests take toyos_bootmap::x86_64::direct_map_end(&[]).unwrap().
    • Delete PartialEq, Eq, Debug on DirectMapEnd too (line 69): nothing compares or prints one.
  • toyos-acpi/tests/mapped.rs:25-38 — every_table_edk2_published_is_inside_the_map tests nothing.
    • Against a constant 4 GiB end, it asserts that six addresses below 2 GiB are nonzero and below 4 GiB.
    • No mutation of Mapped::readable reds it without also redding the is_ok arm at line 54 or readable(1, 1) at line 76.
    • In toyos-bootmap the same check read the captured map's extent. Here it reads a constant. Delete it and its doc at 11-13.

REMOVE

  • toyos-bootmap/src/lib.rs:63-64 "so no reader can be handed a wider map than one that was built": false. x86_64::direct_map_end is pub over any slice a caller builds.
  • PR body, the first "What I am unsure of" bullet ("No guest has booted this head…"): false now.
  • PR body, the table row "review's revert, verbatim … git apply --check EXIT=1": a patch that does not apply measures nothing.
  • PR body, "and the RSDP and all five tables that boot read lie inside that extent (every_table_edk2_published_is_inside_the_map)": it goes with the test, see NOTE.

Growth: 32 files, +952/−260.

  • Production Rust: +345/−216 (net +129). The kernel is net −51 (+129/−180). toyos-bootmap/src, toyos-acpi/src and toyos-pci/src are net +180.
  • Tests: +482/−12.
  • Manifests and lockfiles: +16/−6.
  • Issues: +109/−26.

About 75 lines of that are this round's Mapped/Memory/DirectMapEnd/DirectMapEndCell. They earn their place: they are what moved the bound where a test reds on its revert, and made the naive forgery refuse to compile. The deletions left are the NOTE's BOOT, the unused derives and the test that tests nothing.

LAND AFTER NAMED CHANGES

Japabu and others added 2 commits September 28, 2026 08:39
… test that pinned constants instead of the map goes too

`DirectMapEndCell::boot()` already built its own `BOOT_MAP_BYTES`; `DirectMapEnd::BOOT` had no production caller, only `toyos-acpi/tests/mapped.rs`. Its tests now take `x86_64::direct_map_end(&[]).unwrap()`, the same value, from the function that is the type's only real constructor. The unused `PartialEq, Eq, Debug` derives on `DirectMapEnd` go with it.

`every_table_edk2_published_is_inside_the_map` asserted six addresses were nonzero and below a constant 4 GiB end; no mutation of `Mapped::readable` reds it without also redding another test in the file, so it tested nothing and is deleted with its doc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
@Japabu
Japabu enabled auto-merge September 28, 2026 07:31
@Japabu
Japabu added this pull request to the merge queue Sep 28, 2026
Merged via the queue into main with commit 51fe14c Sep 28, 2026
1 check passed
@Japabu
Japabu deleted the wt/toyos-edk2 branch September 28, 2026 09:46
Japabu added a commit that referenced this pull request Sep 28, 2026
Takes #560, #541, #565, #563, #569 and #570. `src/ci.rs` and
`tests/toyos.rs` merge without conflict; `rust` takes main's pin, 1b236638,
since this branch carries no fork commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant