Skip to content

microvm: serve durable-dir volumes through the one kataShared virtiofsd - #1034

Open
Lucky Abolorunke (Oneimu) wants to merge 1 commit into
agent-substrate:mainfrom
Oneimu:durdir-single-share
Open

microvm: serve durable-dir volumes through the one kataShared virtiofsd#1034
Lucky Abolorunke (Oneimu) wants to merge 1 commit into
agent-substrate:mainfrom
Oneimu:durdir-single-share

Conversation

@Oneimu

Copy link
Copy Markdown
Contributor

What Changed and Why

Durable-dir volumes were previously served to the guest by a second per-actor virtiofsd. That arrangement predates the writable kataShared share: when the rootfs share was a read-only lower, a writable durable share had to be its own device. Since #846, the kataShared tree is writable and served with --announce-submounts, making the second daemon redundant—it cost an extra process and vhost socket per actor, an extra fs device in every snapshot config, and a restore-time revival of all three.

This PR folds the durable-dir volumes into the single existing share:

  • Subtree Bind Mounting (_durable):
    kata.BindIntoShare bind-mounts the atelet-owned volumes directory into the served tree as its _durable subtree. The leading underscore keeps it out of the container-ID namespace (container names are RFC 1123 labels and cannot begin with an underscore). The guest sees it as a submount of the kataShared mount, and containers bind their volumes from <shared>/_durable/<volume> exactly as they previously did from the second share.
  • Streamlined VM Configuration:
    Cold boot no longer spawns the durable virtiofsd, and the VM configuration carries exactly one virtio-fs device.
  • Unchanged Ownership Semantics:
    atelet still owns the directory (creates it before boot, wipes it on actor reset). The bind is ateom-owned mount state, detached by CleanupSandboxState before any removal, ensuring atelet's data is never touched through it.
  • Unchanged Snapshot Content:
    Checkpoints tar the host directory directly under every scope, exactly as before.

Note

The bind uses the same mechanism the per-container merged rootfs mounts already use through this virtiofsd (submounts inside the served tree, re-opened by find-paths on restore), introducing no new guest-side behavior. BindIntoShare's doc comment establishes this pattern for future per-actor shares: mount a reserved subtree rather than adding a device (relevant to in-flight work such as #803; #923 already follows this subtree approach).


Compatibility with Existing Snapshots

Restore is self-describing in both directions:

  • Two-Share Era Snapshots:
    A snapshot whose config.json carries an ateDurable fs device originates from the two-share era. The resumed guest still expects that device, so restore revives the second virtiofsd exactly as before (stageLegacyDurableShare). Such a lineage remains two-share across its own re-checkpoints since cloud-hypervisor re-emits the device.
  • Single-Share Snapshots:
    A snapshot without the device gets the volumes re-bound into the shared tree before virtiofsd starts, allowing find-paths to re-open the guest's open durable files at their _durable/... paths, which the restored tar reproduces exactly.

Detection logic lives in rewriteSnapshotSocketPaths, which already inspects the configuration's fs devices. The configuration acts as the authority because the device is what cloud-hypervisor re-opens, regardless of the actor's spec.


How This Was Tested

  • Unit Tests:
    • Added assertions verifying that rewriteSnapshotSocketPaths classifies single-device and two-device snapshot configs correctly (routing restore to the bind vs. the legacy share).
    • Added a new kata-package test pinning _durable subtree invariants: the underscore namespace reservation, the guest path residing inside the single kataShared mount, and host/guest agreement on the relative path re-opened by find-paths.
    • Verified that existing durable-volume unit tests (tar/untar round-trip, container mount construction, spec isolation) continue to pass unchanged.
  • End-to-End Verification:
    • Ran the counter demo on a GKE cluster with nested virtualization: verified cold boot, suspend, and resume of a durable-volume actor on this branch (volume contents survived across worker pods).
    • Verified restore of a legacy snapshot taken on main before this change to exercise the legacy two-share fallback path.

@Oneimu

Copy link
Copy Markdown
Contributor Author

/assign Benjamin Elder (@BenTheElder)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant