Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
71 changes: 45 additions & 26 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:

env:
CARGO_TERM_COLOR: always
RUSTC_WRAPPER: sccache

jobs:
ci:
Expand All @@ -22,9 +23,7 @@ jobs:
- uses: taiki-e/install-action@cargo-make
- uses: taiki-e/install-action@nextest

- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- uses: mozilla-actions/sccache-action@v0.0.9

- name: Install iperf3
run: sudo apt-get update && sudo apt-get install -y iperf3
Expand All @@ -44,16 +43,53 @@ jobs:
- name: Format
run: cargo make format-check

e2e:
runs-on: ubuntu-latest
macos-check:
runs-on: [self-hosted, macOS, arm64]
steps:
- uses: actions/checkout@v5

- uses: dtolnay/rust-toolchain@stable
with:
components: clippy

- uses: Swatinem/rust-cache@v2
- uses: actions/setup-node@v5
with:
cache-on-failure: true
node-version: "20"

- uses: mozilla-actions/sccache-action@v0.0.9

- name: Build patchbay-vm
run: cargo build -p patchbay-vm

- name: Clippy patchbay-vm
run: cargo clippy -p patchbay-vm -- -D warnings

- name: Container backend smoke test
run: |
if ! command -v container >/dev/null 2>&1; then
echo "container CLI not found, skipping smoke test"
exit 0
fi
if ! command -v aarch64-linux-musl-gcc >/dev/null 2>&1; then
echo "musl cross-compiler not found, skipping smoke test"
echo "install with: brew install filosottile/musl-cross/musl-cross"
exit 0
fi
rustup target add aarch64-unknown-linux-musl
cargo build --release -p patchbay-vm -p patchbay-runner --bin patchbay --target aarch64-unknown-linux-musl
./target/release/patchbay-vm --backend container run \
--patchbay-version "path:target/aarch64-unknown-linux-musl/release/patchbay" \
./iroh-integration/patchbay/sims/iperf-1to1-public.toml
./target/release/patchbay-vm --backend container down

e2e:
runs-on: [self-hosted, linux, x64]
steps:
- uses: actions/checkout@v5

- uses: dtolnay/rust-toolchain@stable

- uses: mozilla-actions/sccache-action@v0.0.9

- uses: actions/setup-node@v5
with:
Expand All @@ -69,32 +105,15 @@ jobs:
working-directory: ui
run: npm run build

- name: Get Playwright version
id: pw-version
working-directory: ui
run: echo "version=$(node -e "console.log(require('./node_modules/@playwright/test/package.json').version)")" >> "$GITHUB_OUTPUT"

- uses: actions/cache@v5
id: pw-cache
with:
path: ~/.cache/ms-playwright
key: playwright-${{ steps.pw-version.outputs.version }}-chromium

- name: Install Playwright browser
- name: Install Playwright browser and system deps
working-directory: ui
run: npx playwright install --with-deps chromium
if: steps.pw-cache.outputs.cache-hit != 'true'

- name: Install Playwright system deps
working-directory: ui
run: npx playwright install-deps chromium
if: steps.pw-cache.outputs.cache-hit == 'true'

- name: Install iperf3
run: sudo apt-get update && sudo apt-get install -y iperf3

- name: Enable unprivileged user namespaces
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true

- name: Build Rust (bins + test targets)
run: cargo build --workspace --all-targets
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
/target
/log-*
/.qemu-vm
/.container-vm
/.tmp
/.netsim-work
/resources
/docs/book
Expand Down
181 changes: 171 additions & 10 deletions docs/guide/vm.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,25 @@

patchbay requires Linux network namespaces, which means it cannot run
natively on macOS or Windows. The `patchbay-vm` crate solves this by
wrapping your simulations and tests in a QEMU Linux VM, giving you the
same experience on any development machine.
wrapping your simulations and tests in a Linux VM, giving you the same
experience on any development machine.

Two VM backends are available:

| Backend | Platform | Boot time | How it works |
|---------|----------|-----------|--------------|
| **QEMU** | Linux, macOS (Intel and Apple Silicon) | 30-60s | Full Debian cloud image with SSH access |
| **Apple container** | macOS 26+ Apple Silicon only | Sub-second | Lightweight VM via Apple's [Containerization](https://github.com/apple/containerization) framework |

By default, `patchbay-vm` auto-detects the best backend. On macOS 26
with Apple Silicon and the `container` CLI installed it picks the
container backend; everywhere else it falls back to QEMU. You can force
a backend with `--backend`:

```bash
patchbay-vm --backend container run sim.toml
patchbay-vm --backend qemu run sim.toml
```

## Installing patchbay-vm

Expand Down Expand Up @@ -85,35 +102,179 @@ patchbay-vm ssh -- nft list ruleset

## How it works

`patchbay-vm` downloads a Debian cloud image (cached in
`~/.local/share/patchbay/qemu-images/`), creates a COW disk backed by
it, and boots QEMU with cloud-init for initial provisioning. The guest
gets SSH access via a host-forwarded port (default 2222) and three shared
mount points:
Both backends share the same three mount points inside the guest:

| Guest path | Host path | Access | Purpose |
|------------|-----------|--------|---------|
| `/app` | Workspace root | Read-only | Source code and simulation files |
| `/target` | Cargo target dir | Read-only | Build artifacts |
| `/work` | Work directory | Read-write | Simulation output and logs |

### QEMU backend

`patchbay-vm` downloads a Debian cloud image (cached in
`~/.local/share/patchbay/qemu-images/`), creates a COW disk backed by
it, and boots QEMU with cloud-init for initial provisioning. The guest
gets SSH access via a host-forwarded port (default 2222).

File sharing uses virtiofs when available (faster, requires virtiofsd on
the host) and falls back to 9p. Hardware acceleration is auto-detected:
KVM on Linux, HVF on macOS, TCG emulation as a last resort.

### Apple container backend

The container backend uses Apple's
[Containerization](https://github.com/apple/containerization) framework,
which runs each container inside its own lightweight Linux VM powered by
the Virtualization.framework hypervisor. Apple's default kernel ships
with everything patchbay needs built-in: network namespaces, nftables,
netem/HTB/TBF qdiscs, veth pairs, and bridges.

Instead of SSH, commands execute through `container exec`. Directories
are shared via native VirtioFS mounts (no separate virtiofsd process).
On first boot the guest installs required userspace tools (iproute2,
nftables, etc.) from the Debian repositories; subsequent runs skip this
step.

## Setting up the QEMU backend on macOS

1. Install QEMU:

```bash
brew install qemu
```

2. For faster file sharing, install virtiofsd (optional but recommended):

```bash
brew install virtiofsd
```

3. Build the musl runner binary. On Apple Silicon:

```bash
rustup target add aarch64-unknown-linux-musl
brew install filosottile/musl-cross/musl-cross
```

Add to `.cargo/config.toml`:

```toml
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
```

Then build:

```bash
cargo build --release --target aarch64-unknown-linux-musl -p patchbay-runner --bin patchbay
```

On Intel Macs, replace `aarch64` with `x86_64` throughout.

4. Run:

```bash
patchbay-vm --backend qemu run \
--patchbay-version "path:target/aarch64-unknown-linux-musl/release/patchbay" \
./path/to/sim.toml
```

The first boot downloads a Debian cloud image and provisions the VM,
which takes 1-2 minutes. Subsequent runs reuse the running VM.

## Setting up the Apple container backend

### Requirements

- Mac with Apple Silicon (M1 or later)
- macOS 26 (Tahoe) or later
- [container CLI](https://github.com/apple/container) installed

### Installation

1. Download the latest signed installer package from the
[container releases page](https://github.com/apple/container/releases).

2. Double-click the package and follow the prompts. The installer places
binaries under `/usr/local`.

3. Start the system service:

```bash
container system start
```

4. Verify it works:

```bash
container run --rm debian:trixie-slim echo "hello from container"
```

### Building the musl target

Simulations run inside an ARM64 Linux VM, so the patchbay runner binary
must be cross-compiled for `aarch64-unknown-linux-musl`.

1. Install the Rust target and a musl cross-compiler:

```bash
rustup target add aarch64-unknown-linux-musl
brew install filosottile/musl-cross/musl-cross
```

2. Tell Cargo which linker to use. Add to `.cargo/config.toml` (create
it if it does not exist):

```toml
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
```

3. Build the runner binary:

```bash
cargo build --release --target aarch64-unknown-linux-musl -p patchbay-runner --bin patchbay
```

### Running a simulation

```bash
patchbay-vm --backend container run \
--patchbay-version "path:target/aarch64-unknown-linux-musl/release/patchbay" ./path/to/sim.toml
```

On the first run the container backend pulls the Debian base image and
installs packages (takes about 15 seconds). Subsequent runs reuse the
existing container and skip provisioning entirely.

## Configuration

All settings have sensible defaults. Override them through environment
variables when needed:
variables when needed.

### QEMU backend

| Variable | Default | Description |
|----------|---------|-------------|
| `QEMU_VM_MEM_MB` | 4096 | Guest RAM in megabytes |
| `QEMU_VM_CPUS` | 4 | Guest CPU count |
| `QEMU_VM_MEM_MB` | 8192 | Guest RAM in megabytes |
| `QEMU_VM_CPUS` | all | Guest CPU count (defaults to all host CPUs) |
| `QEMU_VM_SSH_PORT` | 2222 | Host port forwarded to guest SSH |
| `QEMU_VM_NAME` | patchbay-vm | VM instance name |
| `QEMU_VM_DISK_GB` | 40 | Disk size in gigabytes |

VM state lives in `.qemu-vm/<name>/` in your project directory. The disk
image uses COW backing, so it only consumes space for blocks that differ
from the base image.

### Apple container backend

| Variable | Default | Description |
|----------|---------|-------------|
| `CONTAINER_VM_MEM_MB` | 8192 | Guest RAM in megabytes |
| `CONTAINER_VM_CPUS` | all | Guest CPU count (defaults to all host CPUs) |
| `CONTAINER_VM_IMAGE` | debian:trixie-slim | OCI image to use |
| `CONTAINER_VM_NAME` | patchbay | Container instance name |

Container state lives in `.container-vm/<name>/` in your project
directory.
4 changes: 2 additions & 2 deletions patchbay-vm/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[package]
name = "patchbay-vm"
version = "0.1.0"
description = "QEMU VM wrapper for running patchbay simulations on macOS"
keywords = ["network", "simulation", "qemu", "vm"]
description = "VM wrapper for running patchbay simulations (QEMU or Apple container backend)"
keywords = ["network", "simulation", "qemu", "vm", "container"]
edition.workspace = true
license.workspace = true
authors.workspace = true
Expand Down
Loading
Loading