Skip to content

Implement MSHV aarch64 backend - #1626

Open
cshung wants to merge 1 commit into
hyperlight-dev:mainfrom
cshung:cshung/mshv-aarch64
Open

cshung wants to merge 1 commit into
hyperlight-dev:mainfrom
cshung:cshung/mshv-aarch64

Conversation

@cshung

@cshung cshung commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

ARM64 Linux MSHV

The backend provides partition creation, memory mappings, MMIO dispatch,
general and FPU registers, batched special-register access, and replacement-VM
reset. MMIO PC advancement wraps at the address boundary.

Native ARM64 MSHV CI uses hierarchical-virtualization runners and bounded test
scheduling for the platform's partition limit. Stress tests retain their
iteration counts.

Commit layers

This PR is D, one signed commit directly above main. It is independent of
GDB support. The MSHV debugger branch has a cherry-pick of shared support A as
E, followed by one MSHV-specific debugger commit F in #2052.

The complete graph and update rules are in
docs/gdb-commit-layers.md.

Validation

  • Formatting and ARM64 MSHV-only cross-Clippy pass after consolidation.
  • Backend Rust source matches the natively validated implementation.
  • Native ARM64 MSHV CI before consolidation passed debug and release, with
    1,076 passed execution records per profile.
  • The dedicated ARM64 MSHV debugger VM also passed the five backend tests.

November 16 rebase validation

Base: 5fe5c8f8. Each layer is one signed commit. A and E are patch-equivalent.
The shared ARM64 test helpers use upstream's guest startup and memory-layout APIs.

  • All six layers pass formatting and Linux ARM64 cross-Clippy.
  • KVM-only and MSHV-only ARM64 GDB configurations pass cross-Clippy.
  • WHP passes Windows ARM64 cross-Clippy and Linux just clippyw.
  • The final MSHV GDB tip passes Windows x64 formatting, Clippy, build, the full
    just test recipe and three focused debugger tests. Rust and C guests are
    rebuilt in debug and release for the guest startup ABI.

Native ARM64 runtime results above predate this rebase. They were not repeated
against the guest startup ABI in this base.

@cshung cshung added the C-enhancement For PRs adding features, improving functionality, docs, tests, etc. label Jul 7, 2026
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from b34cc5d to fb92ede Compare July 31, 2026 16:13
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from fb92ede to 3736e16 Compare August 12, 2026 14:55
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch 2 times, most recently from b440e24 to 4ada969 Compare August 20, 2026 17:42
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from 4ada969 to 7f2289d Compare September 2, 2026 22:47
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from a70dcfa to 1b275f2 Compare September 29, 2026 05:30
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from 1b275f2 to 2f42d32 Compare November 2, 2026 17:10
@hyperlight-gh-bot

This comment has been minimized.

@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from 2f42d32 to 0e44f0a Compare November 5, 2026 17:24
@hyperlight-gh-bot

This comment has been minimized.

@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from e039b60 to 747ee00 Compare November 10, 2026 16:59
@hyperlight-gh-bot

This comment has been minimized.

@cshung
cshung marked this pull request as ready for review November 10, 2026 18:22
Copilot AI balanced review requested due to automatic review settings November 10, 2026 18:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A guest-controlled MMIO store can overflow the PC increment and panic the host, while snapshot capture and restore also issue unnecessary separate register requests.

3 open issues
What changed in this PR

This PR implements the Linux aarch64 MSHV backend, allowing Hyperlight to run ARM64 sandboxes through MSHV.

Changes:

  • Implements VM creation, memory mapping, MMIO dispatch, register access, and vCPU reset.
  • Connects MSHV to ARM64 backend selection and memory-access decoding.
  • Adds ARM64 MSHV CI coverage and adjusts test concurrency and platform expectations.
File Description
typos.toml Allows an identifier spelling required by MSHV bindings.
src/​hyperlight_host/​tests/​sandbox_host_tests.rs Limits parallel callbacks for ARM64 MSHV.
src/​hyperlight_host/​tests/​integration_test.rs Limits ARM64 MSHV stress-test concurrency.
src/​hyperlight_host/​src/​sandbox/​snapshot/​file/​config.rs Accepts Arm CPU implementers in Linux snapshot tests.
src/​hyperlight_host/​src/​mem/​memory_region.rs Decodes ARM64 MSHV memory-access flags.
src/​hyperlight_host/​src/​hypervisor/​virtual_machine/​mshv/​aarch64.rs Implements the ARM64 MSHV virtual machine backend.
src/​hyperlight_host/​src/​hypervisor/​hyperlight_vm/​aarch64.rs Instantiates MSHV when selected on ARM64.
.github/​workflows/​ValidatePullRequest.yml Adds debug and release ARM64 MSHV jobs.
.github/​workflows/​dep_build_test.yml Configures runners, test limits, and snapshot-golden exclusions for ARM64 MSHV.

Review effort: Balanced

@hyperlight-gh-bot

This comment has been minimized.

@hyperlight-gh-bot

This comment has been minimized.

@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from bcd0df1 to 938f7e1 Compare November 11, 2026 19:52
@hyperlight-gh-bot

This comment has been minimized.

@simongdavies simongdavies left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from fb09873 to 93e0ffa Compare November 12, 2026 18:15
@cshung

cshung commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

The README platform-support table lists ARM64 Linux/MSHV general support in 93e0ffa. Guest debugging remains in the separate PR #2052.

@hyperlight-gh-bot

This comment has been minimized.

MSHV provides partition creation, memory mappings, MMIO dispatch, batched register access and replacement-VM reset on ARM64. Native CI uses tagged hierarchical-virtualization runners and bounded scheduling for the partition limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
@cshung
cshung force-pushed the cshung/mshv-aarch64 branch from 93e0ffa to 2b625b6 Compare November 16, 2026 18:16
@hyperlight-gh-bot

Copy link
Copy Markdown

Benchmark Results

Measured commit: 2b625b64291c
Baseline commit: 5fe5c8f8c44f

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 761.28 ns (➖ 1.00x slower)
vec_bytes 598.59 ns (➖ 1.02x slower)
372.57 µs (➖ 1.01x faster)

payload_allocation

slot_pool_segmented
262144 557.25 ns (➖ 1.05x slower)
65536 141.21 ns (➖ 1.01x faster)

sandboxes

create_initialized_and_drop
medium 13.12 ms (➖ 1.01x faster)

slot_pool

alloc_dealloc_128 alloc_dealloc_4096 alloc_dealloc_1500
7.68 ns (➖ 1.00x slower) 7.68 ns (➖ 1.00x faster) 7.68 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 98.97 µs (➖ 1.01x slower)

virtq_readonly

slot_pool_segmented slot_pool_segmented_fragmented
262144 6.97 µs (➖ 1.20x faster) 7.19 µs (➖ 1.00x faster)
65536 1.97 µs (➖ 1.05x faster) 2.05 µs (➖ 1.07x slower)

virtq_readwrite

slot_pool_segmented slot_pool_segmented_fragmented
262144 27.12 µs (➖ 1.01x faster)
65536 6.34 µs (➖ 1.01x slower) 6.15 µs (➖ 1.04x faster)
8192 1.09 µs (➖ 1.09x slower) 1.02 µs (➖ 1.00x slower)
kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 777.44 ns (➖ 1.02x slower)
vec_bytes 578.32 ns (➖ 1.01x faster)
683.80 µs (➖ 1.06x faster)

payload_allocation

slot_pool_segmented
262144 564.67 ns (➖ 1.07x slower)
65536 193.38 ns (➖ 1.06x slower)

sandboxes

create_initialized_and_drop
medium 15.34 ms (➖ 1.02x slower)

slot_pool

alloc_dealloc_128 alloc_dealloc_4096 alloc_dealloc_1500
7.61 ns (➖ 1.07x slower) 7.61 ns (➖ 1.07x slower) 7.61 ns (➖ 1.07x slower)

snapshot_files

load_snapshot_unverified
small 55.14 µs (➖ 1.06x slower)

virtq_readonly

slot_pool_segmented slot_pool_segmented_fragmented
262144 8.22 µs (➖ 1.04x slower) 8.32 µs (➖ 1.06x slower)
65536 2.37 µs (➖ 1.07x slower) 2.33 µs (➖ 1.04x slower)

virtq_readwrite

slot_pool_segmented slot_pool_segmented_fragmented
262144 33.02 µs (➖ 1.04x slower)
65536 7.99 µs (➖ 1.03x slower) 7.79 µs (➖ 1.02x slower)
8192 793.70 ns (➖ 1.03x slower) 796.28 ns (➖ 1.04x slower)
mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 947.43 ns (➖ 1.02x faster)
vec_bytes 720.14 ns (➖ 1.02x faster)
269.53 µs (➖ 1.01x slower)

payload_allocation

slot_pool_segmented
262144 729.43 ns (➖ 1.03x faster)
65536 192.85 ns (➖ 1.01x faster)

sandboxes

create_initialized_and_drop
medium 64.08 ms (➖ 1.01x faster)

slot_pool

alloc_dealloc_128 alloc_dealloc_4096 alloc_dealloc_1500
9.69 ns (➖ 1.02x slower) 10.34 ns (➖ 1.08x slower) 9.64 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 104.27 µs (➖ 1.01x faster)

virtq_readonly

slot_pool_segmented slot_pool_segmented_fragmented
262144 9.76 µs (➖ 1.01x slower) 9.83 µs (➖ 1.02x faster)
65536 2.34 µs (➖ 1.04x slower) 2.40 µs (➖ 1.02x faster)

virtq_readwrite

slot_pool_segmented slot_pool_segmented_fragmented
262144 36.36 µs (➖ 1.03x faster)
65536 8.34 µs (➖ 1.16x faster) 8.31 µs (➖ 1.18x faster)
8192 1.27 µs (➖ 1.05x slower) 1.23 µs (➖ 1.04x faster)
mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 934.74 ns (➖ 1.01x slower)
vec_bytes 654.76 ns (➖ 1.01x faster)
640.08 µs (➖ 1.01x faster)

payload_allocation

slot_pool_segmented
262144 674.31 ns (➖ 1.01x slower)
65536 255.10 ns (➖ 1.02x faster)

sandboxes

create_initialized_and_drop
medium 80.76 ms (➖ 1.57x slower)

slot_pool

alloc_dealloc_128 alloc_dealloc_4096 alloc_dealloc_1500
8.97 ns (➖ 1.03x slower) 8.67 ns (➖ 1.01x faster) 8.69 ns (➖ 1.01x faster)

snapshot_files

load_snapshot_unverified
small 56.05 µs (➖ 1.02x slower)

virtq_readonly

slot_pool_segmented slot_pool_segmented_fragmented
262144 7.92 µs (➖ 1.01x slower) 7.90 µs (➖ 1.00x faster)
65536 2.28 µs (➖ 1.01x slower) 2.28 µs (➖ 1.00x faster)

virtq_readwrite

slot_pool_segmented slot_pool_segmented_fragmented
262144 38.22 µs (➖ 1.08x faster)
65536 7.52 µs (➖ 1.01x faster) 7.56 µs (➖ 1.02x faster)
8192 909.85 ns (➖ 1.02x faster) 908.10 ns (➖ 1.01x faster)
hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.25 µs (➖ 1.01x slower)
vec_bytes 820.29 ns (➖ 1.07x slower)
1.96 ms (➖ 1.06x faster)

payload_allocation

slot_pool_segmented
262144 785.54 ns (➖ 1.00x faster)
65536 233.37 ns (➖ 1.02x slower)

sandboxes

create_initialized_and_drop
medium 12.33 ms (➖ 1.12x slower)

slot_pool

alloc_dealloc_128 alloc_dealloc_4096 alloc_dealloc_1500
9.98 ns (➖ 1.08x faster) 11.42 ns (➖ 1.14x slower) 9.92 ns (➖ 1.00x slower)

snapshot_files

load_snapshot_unverified
small 796.74 µs (➖ 1.02x slower)

virtq_readonly

slot_pool_segmented slot_pool_segmented_fragmented
262144 9.58 µs (➖ 1.04x slower) 9.41 µs (➖ 1.00x slower)
65536 2.34 µs (➖ 1.05x faster) 2.34 µs (➖ 1.03x faster)

virtq_readwrite

slot_pool_segmented slot_pool_segmented_fragmented
262144 39.01 µs (➖ 1.14x faster)
65536 8.87 µs (➖ 1.03x faster) 9.16 µs (➖ 1.02x faster)
8192 1.25 µs (➖ 1.04x faster) 1.26 µs (➖ 1.03x faster)
hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.22 µs (➖ 1.01x slower)
vec_bytes 804.50 ns (➖ 1.04x slower)
2.98 ms (➖ 1.00x slower)

payload_allocation

slot_pool_segmented
262144 734.65 ns (➖ 1.04x faster)
65536 210.03 ns (➖ 1.12x faster)

sandboxes

create_initialized_and_drop
medium 13.93 ms (➖ 1.00x slower)

slot_pool

alloc_dealloc_128 alloc_dealloc_4096 alloc_dealloc_1500
9.75 ns (➖ 1.03x faster) 9.80 ns (➖ 1.05x faster) 9.63 ns (➖ 1.03x faster)

snapshot_files

load_snapshot_unverified
small 663.61 µs (➖ 1.25x faster)

virtq_readonly

slot_pool_segmented slot_pool_segmented_fragmented
262144 7.75 µs (➖ 1.01x slower) 7.80 µs (➖ 1.02x slower)
65536 2.44 µs (➖ 1.06x slower) 2.30 µs (➖ 1.01x faster)

virtq_readwrite

slot_pool_segmented slot_pool_segmented_fragmented
262144 42.17 µs (➖ 1.02x faster)
65536 8.54 µs (➖ 1.02x slower) 8.29 µs (➖ 1.00x faster)
8192 1.06 µs (➖ 1.11x faster) 1.07 µs (➖ 1.01x faster)

Reported by cargo ci bench-report --candidate run:43927111852 --config-file bench_report.toml.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants