The kernel boots on QEMU's stock edk2, network included: the direct map ends at the last memory, every kernel root slot exists before the first user space, and a BAR's free run is a gap inside its width's space - #563
Conversation
Under the edk2 firmware Homebrew's QEMU 11.1.1 ships, the kernel died right after `pmm:` with `memory allocation of 4096 bytes failed`. `paging::init` mapped physical memory up to the highest `end` of every descriptor in the UEFI map, whatever its type, and took the page tables from the 512 KiB early bump heap (128 pages). QEMU gives an AMD vCPU (`-cpu qemu64`) with 40 physical address bits a reserved e820 range for the HyperTransport hole, 0xfd00000000..0x10000000000, and edk2 carries it into the UEFI map as EfiReservedMemoryType. The loader's `GCD:` line shows edk2 saw it: its 64-bit PCI window sits at 0xc000000000..0xe000000000, below that reservation, where the repo's `ovmf/` puts its own at 0x800000000. Mapping to 1 TiB takes 1024 page directories, eight times the early heap, so a 4096-byte table is the allocation that fails. The repo's `ovmf/` never names the range, so it boots. The UEFI specification's usage table after ExitBootServices says a reserved range is not usable and an MMIO range is not used by the OS; a conforming map may still put either anywhere. So the extent is now one rule in a new pure crate, `toyos-memmap::direct_map_end`: the low 4 GiB, and above it the end of the highest range the kernel reads as memory (what the pmm hands out, and ACPI reclaim and NVS), rounded up to 2 MiB. MMIO past it is mapped on demand by `map_mmio`, as every 64-bit BAR already was. The pmm's usable-type rule moves into the same crate, so the one list of UEFI types holds both answers and a host test holds the containment the pmm rests on: every type it hands out is inside the direct map. `paging::init` logs the extent it built. Filed: issues/kernel/a-kernel-mapping-made-after-a-user-space-exists-is-missing-from-it.md, the kernel root entries `map_mmio` can add after a user space copied them. This change leaves it as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review of #563 at 492c2cb, round 1. Readiness. CI BLOCKER
Guest runs required (to be ready, not findings)
NOTE
REMOVE
Growth: SEND BACK |
…ry kernel root slot follow it
toyos-memmap is folded into toyos-bootmap: the kernel's direct map may never
reach less than the boot map, and two crates with two constants could not
hold that. `x86_64::direct_map_end` floors at `BOOT_MAP_BYTES`, sits behind
the architecture boundary because mapping the low 4 GiB whole is safe only
where the MTRRs type it, and refuses by name memory past
`DIRECT_MAP_WINDOW` (root slots 256..511, 128 TiB) rather than saturating to
an end below the range it was asked to cover.
`DirectPhys::readable` is bounded by the direct map's extent through
`toyos_bootmap::reaches`: the boot map's until `paging::init` stores its
own, instead of x86-64's 52-bit width. `MAX_PHYS` goes from the ACPI reader
and `the-direct-map-bound-on-a-firmware-address-is-52-bits` is closed.
`paging::init` installs the root slots its direct map needs, and
`seal_kernel_half` installs the rest once the heap is the pmm's; a user
space asserts every kernel slot is present when it copies them, and
`ensure_table` never creates one. That closes
`a-kernel-mapping-made-after-a-user-space-exists-is-missing-from-it`.
Tests: an unsorted map with a reserved entry first gives 8 GiB, memory
ending at u64::MAX is refused by name, the usable set is exactly UEFI
§7.2's {1,2,3,4,7}, and an address past the direct map is not read. The
fixture no longer claims to be edk2's map.
Filed: the early heap's ceiling on the direct map, map_mmio's unbounded
address, and the missing edk2 boot in the harness.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…flow The case one byte past the window runs first, so a direct_map_end with no window check fails on the assertion that names it, whatever the profile's overflow checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… under `DirectPhys` holds the end `mm::direct_map_end` answered when it was made, and `readable` bounds by it through `toyos_bootmap::reaches`. A `readable` that stops consulting the extent leaves the field unread, which the kernel's clippy gate denies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 127 descriptors QEMU 11.1.1's edk2 handed a 2 GiB q35 guest with `-cpu qemu64`, as the kernel printed them (diag boot, edk2-r1/diag.log). The last is type 0 at 0xfd00000000..0x10000000000, the HyperTransport reservation the reconstructed fixture assumed. The fixture's usable bytes and entries are what that boot's pmm counted, and the RSDP and the five tables the boot read are inside the direct map the fixture gives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… lies above the highest extent On QEMU's stock edk2 the virtio NIC was not handed over: the 64-bit run was the 48 MiB above the highest extent firmware described, and that extent is the reserved range 0xfd00000000..0x10000000000, outside every window firmware declared, so the run at 0x10000000000 was inside none and BAR 4 had no address to be placed at. The window firmware declared for 64-bit BARs, 0xc000100000..0xe000000000, was never looked at. The rule was in pcidev on main and unchanged by this branch; main's kernel died at pmm: on this firmware before reaching it. One rule now makes both lists: toyos_pci::placement::free_runs, the gaps between the firmware map, the assigned BARs and the bridges' forwarded ranges, below PLATFORM_MMIO for a 32-bit BAR and from 4 GiB for a 64-bit one. A gap rule above 4 GiB has to know what bridges forward there, so bridge::prefetch decodes a 64-bit prefetchable window whole, upper dwords included, where prefetch_below_4g dropped it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review of #563 at 3f1edf1, round 3 (last reviewed head: 492c2cb). Readiness.
Round-1 BLOCKERs
The orchestrator's four redsThe judgment holds. The new placement cannot plausibly make a netd swap's re-claim fail.
BLOCKER
NOTE
REMOVE
Growth: 25 files, +764/−247.
The kernel shrinks. The NOTE deletions (the SEND BACK |
… firmware's map can make The kernel's ACPI reader implemented `Phys::readable` itself, so putting it back to the architecture's 52-bit ceiling passed every gate. The bound now lives in `toyos_acpi::Mapped`, which the kernel wraps its byte read in (`toyos_acpi::Memory`), and `tests/mapped.rs` pins it: a table in edk2's reserved hole at 0xfd00000000, at the map's end, or one byte over it is `Unmapped`; an XSDT entry pointing there is skipped; address zero and a range past every address are refused. `toyos_bootmap::reaches` goes with it. The end it bounds by is `toyos_bootmap::DirectMapEnd`, whose constructor is private (E0603, pinned by a compile_fail doctest): only `x86_64::direct_map_end` and `DirectMapEnd::BOOT` make one, and the kernel keeps it in a `DirectMapEndCell`, which holds no other number. `DIRECT_MAP_WINDOW` is asserted against the kernel's own `PHYS_OFFSET` at compile time instead of against a copy in a test. `ensure_table`'s assert repeated `new_user`'s and goes. pci: a nested extent opens no run inside the one that holds it (`an_extent_inside_another_opens_no_run`), and the edk2 fixture's xHCI extent is its decoded 16 KiB, so the 64-bit runs are asserted as the three that boot printed. Filed: a panic inside `mm::init` may not reach the panel; nothing reds when the BAR survey drops an assigned BAR or a forwarded range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
|
Review of #563 at 6e22759, round 4 (last reviewed head: 3f1edf1). Readiness.
Round-3 BLOCKERs
Round-3 NOTEs, all closed:
BLOCKERNone. NOTE
REMOVE
Growth: 32 files, +952/−260.
About 75 lines of that are this round's LAND AFTER NAMED CHANGES |
… test that pinned constants instead of the map goes too `DirectMapEndCell::boot()` already built its own `BOOT_MAP_BYTES`; `DirectMapEnd::BOOT` had no production caller, only `toyos-acpi/tests/mapped.rs`. Its tests now take `x86_64::direct_map_end(&[]).unwrap()`, the same value, from the function that is the type's only real constructor. The unused `PartialEq, Eq, Debug` derives on `DirectMapEnd` go with it. `every_table_edk2_published_is_inside_the_map` asserted six addresses were nonzero and below a constant 4 GiB end; no mutation of `Mapped::readable` reds it without also redding another test in the file, so it tested nothing and is deleted with its doc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
Takes #560, #541, #565, #563, #569 and #570. `src/ci.rs` and `tests/toyos.rs` merge without conflict; `rust` takes main's pin, 1b236638, since this branch carries no fork commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j
Under the edk2 firmware that Homebrew's QEMU 11.1.1 ships, the base kernel dies right after
pmm:withmemory allocation of 4096 bytes failed. With the repository'sovmf/the same image boots. The boot hits three defects, and this PR fixes all three:paging::initmapped physical memory up to the highestendof every descriptor, whatever its type, and took its page tables from the 512 KiB early bump heap (128 pages). edk2 hands an AMD vCPU (-cpu qemu64, 40 address bits) QEMU's HyperTransport reservation asEfiReservedMemoryType. The captured map's last descriptor istype 0 0xfd00000000..0x10000000000. Mapping up to 1 TiB takes 1024 page directories, so the heap runs out.new_usercopied the kernel's present root slots once. With the extent fixed, the direct map ends at 4 GiB (slot 256). edk2 puts the xHCI BAR at0xc000004000(slot 257). The kernel mapped it at 0.662 s, after init's space was made at 0.654 s. init'sdevice_claimthen read the virtio-net BAR at0xc000000000on init's CR3. The read faulted:KERNEL PANIC: read unmapped address at 0xffff80c000000004inkernel::pcidev::place_bars+0x800, with the page walk showingPML4[257] … PML4E: 0x0 P=0on CR30x1407000. The 60 s panic hold then reset the machine throughacpi→xhci::stop::before_reset, which faulted again under the same CR3:FAULT rip=0xffff80007bfbae6b cr2=0xffff80c000004440 … RECURSIVE. Thatripiskernel::drivers::xhci::stop::stop_all+0x6db, withLive::stopinlined.cr2is the BAR plus0x440, which is port 1's PORTSC (CAPLENGTH0x40+OP_PORT_BASE0x400): the reset path's first register read.mem 0x80000000..0x81100000, mem 0xc000000000..0xc000100000, mem 0x81100000..0xe0000000, mem 0xc000100000..0xe000000000. So pcidev's 64-bit run was0x10000000000..0x10003000000, inside no window, and the virtio NIC's BAR 4 (firmware's0xc000000000,0x4000bytes) was refused:pcidev: PCI 00:03.0 NOT HANDED OVER — this machine has no 2 MiB-aligned 64-bit address space both above what firmware assigned and inside a window firmware declared to offer its BAR, thennetd: no NIC on this machine, exiting. The rule is onmainunchanged;main's kernel dies atpmm:on this firmware before it reaches it. Neither the direct map's extent nor the 2 MiB alignment enters the survey.What changed, per decision
toyos_bootmap::x86_64::direct_map_end. It lives beside the boot map because the kernel may never map less than the boot map did. The kernel keeps using addresses it took through the boot map: the black box page, the parameter buffer, andserial::init's ACPI reads. So the floor isBOOT_MAP_BYTES, not a second constant. Above the floor, the extent is the end of the highest range the kernel reads as memory, rounded up to 2 MiB, wherever that range sits in the unordered map. "Memory" is the pmm's usable types plus ACPI reclaim and NVS. The floor sits behind the x86-64 boundary because mapping the low 4 GiB whole is safe only where the MTRRs type it. AArch64 types by the map, and its kernel page tables are still owed (stage 4).direct_map_endanswers atoyos_bootmap::DirectMapEnd, whose constructor is private. Onlydirect_map_endmakes one, andget()reads it. The kernel keeps it in aDirectMapEndCell, which holds no other number. A crate that writesDirectMapEnd(1 << 52)does not compile (E0603), and acompile_fail,E0603doctest pins that.DIRECT_MAP_WINDOWis root slots 256..511 atPHYS_OFFSET:0x800000000000, 128 TiB. Aconstassert beside the kernel'sPHYS_OFFSETpins it to0 - PHYS_OFFSET. A memory range ending past it isRefusal::PastWindow(end), andpaging::initpanics with it. This replaces asaturating_addthat answered an end below the range's own end.is_usable_typeandEFI_LOADER_DATAnow live in the same crate, so one list of UEFI type numbers answers what the pmm hands out and what the map reaches.toyos_acpi::Mapped. The kernel supplies only the byte read (toyos_acpi::MemoryforDirectMemory) and wraps it inMapped::new(DirectMemory, mm::direct_map_end()).Mapped::readablerefuses address zero and any range with a byte at or past theDirectMapEndit was made with. The kernel no longer implementsreadable, so it has no bound of its own to put back. Beforepaging::initthe extent is the boot map's, and afterwards the kernel's own. This holds on both architectures, since AArch64 keeps the boot map's.MAX_PHYSis gone from the reader, andissues/kernel/the-direct-map-bound-on-a-firmware-address-is-52-bits.mdis closed. The bootloader'swatchdog.rscomment that cited the kernel's bound is cut.paging::initinstalls the slots its direct map needs, from the early heap.seal_kernel_halfinstalls the rest once the heap is the pmm's, inmm::initright afteralloc::init.new_userasserts every kernel slot is present when it copies them. The second-level tables are shared, so every later kernel mapping reaches every space, the reset path's MMIO included. The cost is at most 255 more 4 KiB tables from the kernel heap (≈1 MiB, by arithmetic). The filed issue this replaces is gone.paging::initlogs the extent it built:paging: the direct map covers 0x0..0x100000000.toyos_pci::placement::free_runs. A run is a gap between the firmware map, the BARs firmware assigned and the ranges bridges forward: belowPLATFORM_MMIOfor a 32-bit BAR, as before, and from 4 GiB (WIDE_FLOOR) for a 64-bit one, where it was the 48 MiB above the highest of them. An extent inside another opens no run inside the one that holds it: a bridge's forwarded range holds the BARs behind it. The floor keeps the two lists disjoint, so no address is handed out from both.placement::reservestill offers only an address inside a declared window, andplace_barstill settles a candidate on the function answering its own dword there. The kernel'sfree_runs_below_4g,windowandWINDOW_SPANare deleted, andPLATFORM_MMIOmoved into the crate with the rule. On this machine the NIC's first candidate is0xc000200000, insidemem 0xc000100000.bridge::prefetchreads the upper base and limit dwords into the range, and decides "disabled" on the whole address. It replacesprefetch_below_4g, which dropped a window above 4 GiB.PciDevice::forwardedreturns every range, andprefetch_is_64_bitis private.pcidev::probe_dword("the boot map already covers every physical address …") is deleted.Net lines against
main: production Rust +345/−216 (net +129; the kernel net −51), tests +482/−12 (127 of them the captured map), manifests and lockfiles +16/−6, issues +109/−26.Gates
At
d6716fc7(this head, after mergingorigin/main):cargo run -- --ci host: EXIT=0 ([ci] Host: 49 step(s), all green). This includes the host workspace's tests, thecompile_faildoctest, and clippy with warnings denied over the kernel on both architectures and over the bootloader.cargo test -p toyos-bootmap: EXIT=0 (direct_map.rs11,plan.rs22, 1 doctest).cargo test -p toyos-acpi: EXIT=0 (mapped.rs3,corpus.rs24,fixtures.rs10,resource.rs12).cargo test -p toyos-pci: EXIT=0 (75 tests).cargo run -- --build-only: EXIT=0 for the green, no-seal and red trees (build-arm.sh).High-risk checks (memory management, a firmware trust boundary)
Independent oracle for the placement: the function itself, on the machine.
place_barsettles a candidate only when the NIC answers at it the dword it answered where firmware put it, andnetd: DHCP: leaseneeds the device to move packets through that BAR. The fixture is checked against the boot it came from: on its extents the rule gives exactly the two 32-bit runs of 2 MiB or more and the three 64-bit runs that boot printed (0x100000000..0xc000000000,0xc000008000..0xfd00000000,0x10000000000..), inedk2s_32_bit_runs_are_the_ones_its_boot_printedandedk2s_64_bit_runs_are_the_ones_its_boot_printed.Independent oracle for the extent: UEFI §7.2, the table of memory-type usage after
ExitBootServicesunderEFI_BOOT_SERVICES.AllocatePages(). It gives the OS loader code and data, boot-services code and data, and conventional memory, and nothing else. It puts no bound on where a reserved or MMIO range may sit, and the specification does not order the map.the_pmm_hands_out_exactly_the_types_uefi_gives_the_ospins{1, 2, 3, 4, 7}against it. The second oracle is the recorded real failure: edk2's own map, captured descriptor by descriptor (127 entries) from a boot of this kernel, is the fixture. Its usable bytes and entry count equal that boot'spmm:line (2140844032 in 112), and the extent it gives is the boot map's.Negative controls, host, at
58660e78. Each is a checked patch, shown to build (the tests and doctests compile, EXIT=0), run, and reversed with the tree shown clean (mutate.sh).The ACPI reader's bound:
a1-acpi-max-phys: the same revert where the bound now lives,Mapped::readablebounded by1 << 52a_table_past_the_direct_map_is_refused_before_a_byte_is_read("a table at 0xfd00000000",left: None) andan_xsdt_entry_past_the_direct_map_is_skipped(left: None,right: Some(Absent))a2-acpi-no-zero:phys != 0droppedaddress_zero_and_a_range_past_every_address_are_refused("address zero")s1-seal-open:DirectMapEnd(pub u64)DirectMapEnd (line 66) - compile fail("Test compiled successfully, but it's markedcompile_fail")toyos-bootmapforgingDirectMapEnd(1 << 52)error[E0603]: tuple struct constructor DirectMapEnd is private, by a standalonecargo checkThe placement's:
pci-ma-at-next-end:at = at.max(next.end)→at = next.endan_extent_inside_another_opens_no_run(a run from6 GiB + 0x4000), both edk2 64-bit tests,a_run_is_free_and_in_one_list_oncepci-mb-sort-by-end:takensorted byendan_extent_inside_another_opens_no_run(4 GiB..6 GiBagainst4 GiB..5 GiB)pci-m1-base-rule: a 64-bit BAR's runs put back to the base's rule, the 48 MiB above the highest extent, the whole rule change revertededk2_offers_a_64_bit_bar_the_window_it_declared,edk2s_64_bit_runs_are_the_ones_its_boot_printed,an_extent_inside_another_opens_no_run,taken_is_read_in_any_order_and_overlappingpci-m2-floor-zero:WIDE_FLOOR= 0a_run_is_free_and_in_one_list_once(a run in both lists), the edk2 64-bit tests, the nested testpci-m3-unsorted:takennot sortedtaken_is_read_in_any_order_and_overlapping, all three edk2 tests,a_run_is_free_and_in_one_list_oncepci-m4-prefetch-low-half: a 64-bit prefetchable window read by its low halfa_64_bit_prefetchable_window_is_its_upper_dwords_tooThe extent's:
bootmap-m6-base-rule:direct_map_endreplaced by the base's arithmetic (every descriptor, 4 GiB floor), the whole extent change revertedthe_reserved_hole_below_1_tib_is_not_mappedand four morebootmap-m1-take-while:.filter→.take_whilean_unsorted_map_is_read_wholebootmap-m2-last-in-order: the last memory entry in map orderan_unsorted_map_is_read_wholebootmap-m3-no-window: the window check removedmemory_past_the_window_is_refused_by_namebootmap-m3b-window-boundary:>→>=memory_past_the_window_is_refused_by_namebootmap-m4-no-boot-services-code: type 3 dropped from the usable setthe_pmm_hands_out_exactly_the_types_uefi_gives_the_osandthe_captured_map_is_usable_where_its_boot_saidGuest arms, run by the orchestrator at
6e227596on the stock-edk2 line (build-arm.sh, thenrun-arm.sh, which stops QEMU by PID):paging: the direct map covers 0x0..0x100000000,mmio: 0xc000004000+0x10000 PAT Uncacheable,pcidev: PCI 00:03.0 BAR 4 (0x4000 bytes) placed at 0xc000200000 — inside firmware's mem 0xc000100000,netd: DHCP: lease,compositor: ready— and noPANIC,FAULT riporRECURSIVE.mainat1ec6daa9):pmm:, thenEARLY PANIC: panicked at library/alloc/src/alloc.rs:659:9:withmemory allocation of 4096 bytes failedunder it.seal_kernel_halfand its call deleted):PANIC: panicked at src/arch/x86_64/paging.rs:387:13:withnew_user: kernel root slot 257 is absent, and this space would never see what is mapped thereunder it, when init's space is made.ovmf/:bar_placement_is_provenEXIT=0.What I am unsure of
bridge::prefetch's ranges into the survey has no arm that reds when it is deleted, and neither has the line that feeds the assigned BARs. The decode itself is host-tested (pci-m4). Filed.0x60_3dc0_0000, the end of its 64-bit window. This rule offers the gaps of0x40_0000_0000..0x60_3dc0_0000, a change in real-hardware behaviour with no reading.Filed
issues/kernel/the-direct-maps-page-directories-come-from-a-512-kib-heap.md: the early heap still limits the direct map to about 120 GiB of memory. That figure is an estimate: 512 KiB / 4 KiB = 128 pages, before the root, one second-level table and earlier allocations.issues/kernel/map-mmio-takes-an-address-past-the-direct-map-window.md:map_mmiodoes not boundphysby the 128 TiB window, andvtdstill bounds by1 << 52.issues/build/no-harness-test-boots-qemus-own-edk2.md: filed, not added to the release PR's tests. The host tests here are the only standing guard, and its exit condition includesnetd: DHCP: lease.issues/panic-path/a-panic-inside-mm-init-may-not-reach-the-panel.md: frompaging::init's CR3 load topanic_console::remap, a scanout past the direct map's end is unmapped, so a panic in that span faults instead of painting.issues/kernel/nothing-reds-when-the-bar-survey-drops-an-assigned-bar.md: the two kernel pushes into the survey'stakenhave no arm.🤖 Generated with Claude Code
https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j