Repository navigation
Build the guest binaries from nvx-rs - #561
Open
Pedro Henrique Penna (ppenna) wants to merge 20 commits into
Open
Pedro Henrique Penna (ppenna) wants to merge 20 commits into
Pedro Henrique Penna (ppenna) wants to merge 20 commits into
Conversation
Document the private fields, constants, helpers, and platform modules that predate the guest ports. The rustdoc records ownership, error behavior, wire contracts, platform invariants, and the reasons for compatibility helpers instead of merely restating item names. Enable Clippy's missing_docs_in_private_items lint for the whole crate so future public and private code must provide the same detail on every target. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the Linux guest feature and size-optimized musl profile, then port nvx-console-pending, nvx-mmio-write, nvx-port-io, and nvx-reseed with their C command lines, statuses, output, retry rules, and cleanup diagnostics. Golden unit and integration tests pin those contracts. Use nix 0.31.3, the latest MIT-licensed release compatible with Rust 1.89, for typed, owned syscall facilities with default features disabled. Confine all remaining libc, ioctl, mmap, and x86 unsafe operations to a documented safe guest::sys interface. OwnedFd-backed descriptors and owned mappings provide Drop cleanup plus consuming close and unmap operations where the C programs report failures; raw descriptors never leave the system layer. Give every tool a dedicated exit-status enum and typed internal errors. Call the statically linked libc strtoull with reset errno and a checked end pointer instead of reimplementing it, and deny unsafe code outside the system layer. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a no_std workspace crate as the shared home of nvx-exit and nvx-device-io. The main nvx library links std, so linking it into these binaries would pull in the std runtime and conflict with their panic and startup language items; a package dependency replaces every path inclusion while workspace tests still exercise the pure logic on every host. Confine initial-stack decoding, raw syscalls, entry and panic handling, and compiler memory intrinsics to the documented runtime module. Safe RAII wrappers own every descriptor above it, and the command logic and binary roots deny unsafe code. Keep nvx-device-io libc-free and keep nvx-exit freestanding for the restored guest's cold first exec. Preserve device-io's wrapping parser and exact statuses 0, 2, 3, 4, and 255, and give both tools typed statuses and internal errors. The resulting nvx-exit remains smaller than the shipped C executable. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add build-guest-binaries to validate the exact Rust pin, build the six PR 1 executables in a fresh remapped target, suppress build IDs, check every static ELF, and atomically write source and binary SHA-256 values to a manifest. Add test-guest-binaries to run the musl integration suite against that staged inventory. Make check-nvx-rs call those commands instead of duplicating Cargo and linker arguments in YAML. Run the no_std workspace crate's tests, lints, docs, MSRV check, macOS cross-check, and musl checks without dropping any existing gate. Register focused Python tests in validate-nvx and document the commands and guest conventions. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the guest attachment, acknowledgement, and credit records and window payloads. Parse the current nvx_maps count contract and expose the feature sets announced by each agent mode. Keep byte-oriented EXEC and MAPS decoders for the Linux guest beside the existing UTF-8 host decoders, including the C guest's 4096-byte cwd bound. Pin the new records and parser with golden and round-trip tests. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port the managed agent as a static Rust guest binary while preserving its control session, execution, cgroup, host-mapping, sealed configuration, helper, and diagnostic contracts. Put every OS-facing unsafe operation behind documented safe nvx::guest::sys APIs. Use nix where its wrappers preserve the syscall contract, own descriptors, pipes, and memfds through OwnedFd, and use typed agent, helper, process, mapping, and launch outcomes. Prepare argv, envp, physical-PWD storage, and descriptor actions before fork. The child allocates nothing, runs only direct async-signal-safe Linux/musl operations, and terminates through exec or _exit. Keep the privileged seam for production wrappers and failure-injecting tests, and cover the complete path with the host-to-agent loopback. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exercise the managed agent helper as a real process, including exact and layered environments, cwd handling, malformed configuration, the internal descriptor lifetime, and command-line failures. The tests can use either the Cargo test binary or the prebuilt guest-profile binary. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add nvx-managed-agent to the hosted Rust guest-binary inventory. The existing nvx.py build-guest-binaries and test-guest-binaries commands now build its static musl executable in a fresh reproducible target, check its ELF form, include its digest in the manifest, and run its prebuilt helper integration tests. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the listener announcer with a byte-compatible Rust implementation. Split socket-table and address parsing, byte-level socket diagnostics, UDP settling, announcement encoding, the live sender, and snapshot test mode by concern. Confine socket, netlink, signal, stdio, clock, sleep, and libc parsing to documented safe guest system wrappers. Own every socket with RAII, preserve the injectable sender boundary, and use typed errors and exit statuses. Port the eight C-driven ingress cases, pin the golden wire vector, and cover malformed input, bounded encoding, sender failures, refreshes, and the real binary through NVX_GUEST_BINARIES. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add nvx-ingress to GuestRustBuildConstants.HOSTED_BINARIES after recording the preceding seven-binary inventory. Keep the earlier six-binary inventory recognized, and test that both owned outputs validate while the immediate predecessor upgrades in place. Document the command line. The guest-image build remains unchanged until the final switch. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the byte-oriented restore packet, sample, state, kernel-log, timer, CPU, command-line, event, correction, and frequency logic for nvx-time. Expose every C test subcommand with injectable output and add golden unit and binary coverage. Introduce the documented safe guest system interface needed by the time series. Confine libc, descriptors, process operations, scheduling, clocks, x86 instructions, and port I/O to OwnedFd-backed or typed wrappers; the test command itself contains no unsafe code. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implement every production command and conformance check, the exact scalar port restore and sample transactions, atomic runtime state, detector suppression, and exhaustive CPUID and MSR validation. Run the boot checker, grace-period release, and restore step 13 through the documented fork boundary with the original scheduling bounds, async-signal-safe child rules, and cross-fork CPU accounting. Use RAII timer, signal, pipe, kernel-log, lock, and MSR resources throughout the daemon. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port the latency, pairwise warp, and clock diagnostics with safely owned pinned threads, 64-byte-separated atomics, the original acquire/release orderings, kernel-log fallback, CPUID reporting, and optional RAII MSR access. Keep every architecture and libc operation in guest::sys, render measured floating-point values through libc snprintf, and preserve the original command, marker, and exit-status formats. The shared size profile remains equivalent to the C -O2 probe without a per-binary override. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record PR 3's exact eight-binary inventory, then add nvx-time and nvx-time-probe to the authoritative size-optimized static-musl build and prebuilt-binary tests. Preserve upgrades from each earlier recognized inventory. Document both programs, their libc-compatible formatting, and the measured decision to keep the probe on the shared guest profile rather than force opt-level 2. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace native C helper compilation with the shared Rust guest-binary builder introduced by #528. Build one checked set of all ten executables per initramfs, install it at the established /sbin/nvx-* paths, and record the aggregate source and individual binary digests. Remove the obsolete C compiler flags, helper inventories, and build functions. Keep C test-only flags local until the C-driven suites are deleted later in this stack. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build one checked set of Rust guest executables in the digest-pinned Rust 1.95.0 image and reuse its exact bytes in Alpine, Ubuntu, and Azure Linux. Keep the toolchain available only for Ubuntu's two fresh determinism attempts, and remove the guest C compilers from the base. Bind reusable binary sets to normalized Rust-source, toolchain, target, profile, and executable digests. Record all ten helpers in the Azure Linux package manifest. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The privileged runtime and packaging migration needs human review and exact-head CI confirmation, with a cross-platform provenance issue still unresolved.
1 open finding
What changed in this PR
Completes the guest-build migration for #358, using nvx-rs binaries across Alpine, Ubuntu, and Azure Linux while preserving existing executable paths.
Changes:
- Integrates pinned Rust guest builds and static-executable validation.
- Extends artifact provenance, release metadata, caches, and setup for Rust inputs.
- Replaces C implementations and updates tests and documentation.
| File | Description |
|---|---|
| THIRD_PARTY_NOTICES.md | Adds Rust runtime and dependency notices. |
| SOURCE-MANIFEST.json | Records Rust toolchain and guest sources. |
| scripts/test_tool_versions.py | Checks musl target setup. |
| scripts/setup/setup-linux-runner.sh | Explicitly selects the target toolchain. |
| scripts/setup/setup-linux-mshv.sh | Installs and verifies the musl target. |
| scripts/nvx.py | Exposes guest-build, test, and ELF commands. |
| scripts/nvx_tools/ubuntu.py | Includes Rust customization inputs. |
| scripts/nvx_tools/release.py | Validates Rust source metadata. |
| scripts/nvx_tools/elf.py | Checks static ELF linkage. |
| scripts/nvx_tools/ci.py | Tracks Rust guest build dependencies. |
| scripts/nvx_tools/build.py | Installs Rust binaries and records provenance. |
| scripts/nvx_tools/benchmark.py | Records Rust device-I/O provenance. |
| scripts/nvx_tools/azurelinux.py | Includes Rust build inputs. |
| scripts/nvx_tools/archive.py | Excludes generated source-archive content. |
| README.md | Describes shared Rust guest binaries. |
| nvx-rs/src/protocol/outer.rs | Adds guest records and flow-control codecs. |
| nvx-rs/src/protocol/mod.rs | Exports shared guest protocol APIs. |
| nvx-rs/src/protocol/maps.rs | Adds byte-oriented guest mapping decoding. |
| nvx-rs/src/protocol/features.rs | Defines agent feature sets. |
| nvx-rs/src/protocol/boot.rs | Parses mapping-count boot parameters. |
| nvx-rs/src/net/cidr.rs | Documents existing internals. |
| nvx-rs/src/lib.rs | Exposes the Linux guest module. |
| nvx-rs/src/host/session.rs | Documents existing session internals. |
| nvx-rs/src/host/process.rs | Documents existing process internals. |
| nvx-rs/src/host/platform/windows.rs | Documents existing Windows primitives. |
| nvx-rs/src/host/platform/mod.rs | Documents platform modules. |
| nvx-rs/src/host/platform/linux.rs | Documents existing Linux primitives. |
| nvx-rs/src/host/instance.rs | Documents existing instance internals. |
| nvx-rs/src/host/filesystem.rs | Documents existing filesystem internals. |
| nvx-rs/src/host/artifacts.rs | Documents existing artifact helpers. |
| nvx-rs/src/guest/time/mod.rs | Adds time-tool dispatch. |
| nvx-rs/src/guest/time/events.rs | Formats bounded time-ABI events. |
| nvx-rs/src/guest/time_probe/mod.rs | Adds diagnostic command dispatch. |
| nvx-rs/src/guest/time_probe/format.rs | Preserves libc numeric formatting. |
| nvx-rs/src/guest/sys/x86.rs | Wraps x86 instructions. |
| nvx-rs/src/guest/sys/terminal.rs | Wraps terminal configuration and draining. |
| nvx-rs/src/guest/sys/stdio.rs | Wraps libc formatting and scanning. |
| nvx-rs/src/guest/sys/scheduler.rs | Wraps affinity and scheduling operations. |
| nvx-rs/src/guest/sys/port.rs | Provides time and shutdown port access. |
| nvx-rs/src/guest/sys/poll.rs | Wraps descriptor polling. |
| nvx-rs/src/guest/sys/msr.rs | Provides owned MSR-device access. |
| nvx-rs/src/guest/sys/mapping.rs | Owns mappings and device writes. |
| nvx-rs/src/guest/sys/kernel.rs | Wraps kernel-log and auxiliary-vector queries. |
| nvx-rs/src/guest/sys/ioctl.rs | Wraps console and entropy ioctls. |
| nvx-rs/src/guest/sys/filesystem.rs | Wraps filesystem and mount operations. |
| nvx-rs/src/guest/sys/errno.rs | Provides typed errno and libc diagnostics. |
| nvx-rs/src/guest/parse.rs | Preserves libc integer parsing. |
| nvx-rs/src/guest/mod.rs | Organizes guest tools and unsafe boundary. |
| nvx-rs/src/guest/ingress/test_mode.rs | Processes ingress snapshot fixtures. |
| nvx-rs/src/guest/ingress/mod.rs | Adds ingress entry and protocol definitions. |
| nvx-rs/src/guest/ingress/encode.rs | Encodes bounded listener announcements. |
| nvx-rs/src/guest/agent/tty.rs | Handles agent terminal setup. |
| nvx-rs/src/guest/agent/diagnostics.rs | Provides agent diagnostics and I/O helpers. |
| nvx-rs/src/error.rs | Documents existing error fields. |
| nvx-rs/src/bin/nvx-time.rs | Adds time-tool entry point. |
| nvx-rs/src/bin/nvx-time-probe.rs | Adds time-probe entry point. |
| nvx-rs/src/bin/nvx-reseed.rs | Adds reseed entry point. |
| nvx-rs/src/bin/nvx-port-io.rs | Adds port-I/O entry point. |
| nvx-rs/src/bin/nvx-mmio-write.rs | Adds MMIO-write entry point. |
| nvx-rs/src/bin/nvx-managed-agent.rs | Initializes the managed agent. |
| nvx-rs/src/bin/nvx-ingress.rs | Adds ingress entry point. |
| nvx-rs/src/bin/nvx-exit.rs | Selects freestanding or hosted exit runtime. |
| nvx-rs/src/bin/nvx-device-io.rs | Selects device-I/O runtime. |
| nvx-rs/src/bin/nvx-console-pending.rs | Adds console-query entry point. |
| nvx-rs/freestanding/src/runtime/syscall.rs | Implements raw Linux syscall boundary. |
| nvx-rs/freestanding/src/runtime/mod.rs | Organizes freestanding runtime modules. |
| nvx-rs/freestanding/src/runtime/hosted.rs | Provides hosted execution adapters. |
| nvx-rs/freestanding/src/runtime/entry.rs | Provides freestanding startup and buffers. |
| nvx-rs/freestanding/src/runtime/args.rs | Wraps initial-stack arguments. |
| nvx-rs/freestanding/src/lib.rs | Defines the no-std runtime crate. |
| nvx-rs/freestanding/src/exit.rs | Implements VM exit-code handling. |
| nvx-rs/freestanding/Cargo.toml | Configures the freestanding crate. |
| nvx-rs/Cargo.toml | Adds guest binaries, dependencies, and profile. |
| nvx-rs/Cargo.lock | Locks guest dependencies. |
| nvx-rs/build.rs | Documents existing bundle-build internals. |
| guest/common/nvx-reseed.c | Removes the C reseed implementation. |
| guest/common/nvx-mmio-write.c | Removes the C MMIO implementation. |
| guest/common/nvx-exit.c | Removes the C exit implementation. |
| guest/common/nvx-console-pending.c | Removes the C console implementation. |
| doc/usage.md | Documents new CLI commands. |
| doc/setup.md | Documents Rust guest prerequisites. |
| doc/project-structure.md | Describes the Rust guest layout. |
| doc/guests/ubuntu-guest.md | Documents Rust binary installation. |
| doc/guests/azurelinux-guest.md | Documents Rust binaries and provenance. |
| doc/contribute.md | Updates guest testing guidance. |
| doc/ci.md | Describes Rust checks and cache inputs. |
| doc/build.md | Documents Rust builds and packaging. |
| doc/benchmarks.md | Documents Rust benchmark provenance. |
| .github/workflows/copilot-setup-steps.yml | Triggers setup validation on Rust changes. |
| .github/copilot-instructions.md | Updates setup documentation. |
| .github/actions/validate-nvx/action.yml | Replaces retired C-driven test suites. |
| .github/actions/check-nvx-rs/action.yml | Adds workspace and musl validation. |
| .github/actions/build-guest-artifacts/action.yml | Updates guest cache identities. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 15, 2026 08:31
4185f68 to
b5ff179
Compare
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 15, 2026 08:41
b5ff179 to
e24b046
Compare
Cover the nvx-rs guest inputs in every image identity, package the crate without its target directory, and declare the exact Rust toolchain and musl target in SOURCE-MANIFEST.json. Verify those pins and inputs from nvx.py verify and release packaging. Make device-I/O benchmark provenance validate the shared Rust source digest, and bind the Azure Linux cache input to the newly tracked files. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Hash the complete nvx-rs tree except target, the exact toolchain pin, and the shared build modules into every guest cache, and advance the guest cache schemas. Treat nvx-rs changes as OpenVMM VMM-test inputs so they rebuild and boot the affected images. Keep Copilot setup and both Linux setup paths on the pinned musl target, and revalidate Copilot setup for every nvx-rs change. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove all ten guest C implementations and the C-compiled Python suites whose behavior the nvx-rs unit, guest-binary, loopback, and real-VM tests now cover. Drop the deleted modules from both CLI validation legs. Add a repository regression that rejects C, header, or assembler sources outside the OpenVMM submodule. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the one pinned build shared by native builds, Docker, and CI, the ten binaries shipped by every guest, their provenance, cache inputs, test ownership, and release source. Remove obsolete source comparisons and C-era wording now that the Rust definitions are authoritative. Extend the third-party notices for the Rust standard libraries, compiler_builtins and its compiler-rt-derived routines, libc, nix and its linked transitive crates, and musl's complete hosted-binary scope. Closes #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-358-build-switch
branch
from
October 15, 2026 08:54
e24b046 to
cc31fe2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #358 (PR 5 of 5).
Stacked on #550, #533, #531, and #528: the first fourteen commits belong to those PRs. Review from
4f0be8bf7on.Summary
nvx.py build-guest-binariesroutine from Port the small guest tools to Rust #528./sbin/nvx-*paths.SOURCE-MANIFEST.jsonto the complete Rust source/toolchain identity.openvmm/, and update the current documentation.libc,nix,bitflags,cfg-if, andmemoffset) inTHIRD_PARTY_NOTICES.md.The issue originally named six programs. The stacked implementation deliberately grew to ten so every compiled
guest/commonprogram switches together: the original six plusnvx-ingress,nvx-time,nvx-time-probe, andnvx-exit.Acceptance criteria
openvmm/.git ls-filesreports zero.c,.h,.S, or.sfiles outside the submodule, andBuildConstantsTests.test_no_c_or_assembler_sources_remain_outside_openvmmenforces it./sbin/nvx-*byte with the pinned builder output; all three images and their manifests match all ten digests.verify-guest-determinismpasses.ubuntu-determinismDocker target rebuilt both attempts from fresh Cargo targets and passed.test-aci-edge-sandboxescases; and the zero-warmup/one-run e2e plus 1-second, 64-MiB device-I/O smoke profile.Size comparison
Fresh Docker artifacts use the same guest inputs. Positive compressed deltas are under 0.9%; every unpacked initramfs is 228,352 bytes smaller.
Cold boot
prometheus32 used one exact kernel and OpenVMM binary, zero warmups, and 20 retained runs per build. Runs were interleaved in pairs with the first build alternating.
dev2735f2ace)cc31fe2e0)For context, #550 measured
nvx-time's own baremetal phase p50s (µs elapsed):Boot is about 0.3 ms slower; restore is mixed.
Validation
check-nvx-rsandcheck-aci-edge-sandboxescommand sets, Rust 1.89 MSRV, macOS cross-checks, all three musl Clippy modes,nvx.py verify, 980 validate-nvx tests (32 expected skips), 70 adversarial tests (6 expected skips), host-profile/help smokes, Ruff, Linux/Windows Pyright, PowerShell syntax, and pinned ShellCheck/shfmt images.7f225e2cdOpenVMM input.Merge order
#528, then #531, then #533, then #550, then this PR.