vm: per-VM cgroup v2 CPU scope with Guaranteed-at-N defaults (#182) - #183
Merged
Conversation
vCPU count only bounds guest parallelism: virtio queue threads and io-wq workers burn host CPU outside it (measured 111-113% of a core for a 1-vCPU VM under I/O). Spawn the VMM via CLONE_INTO_CGROUP into cocoon.slice/vm-<id>.scope with raw knobs (cpu.weight / cpu.max / cpu.max.burst) defaulting to Guaranteed-at-N: quota = N x period, weight = N, burst = 0. Scope lifecycle is converge-friendly: create-or-reconfigure before launch (relaunch reuses an occupied scope), owned removal after the VMM is confirmed dead on stop/hibernate/teardown/converge-dead paths with cgroup.kill sweeping stragglers, and a GC module that removes empty scopes owned by no backend. subtree_control writes are read-gated so steady-state launches stay off the kernel's hierarchy-wide cgroup_mutex.
…cmp.Or/DeleteFunc reductions MergeSets folded into FilterUnreferenced's existing exclude variadic; MapValues inlined at its only caller; applyFilters kept non-destructive (its test encodes that contract) while filterRecords, with no observer of the input, takes DeleteFunc.
Contributor
Author
Hardware acceptance — bare-metal 16-core testbed, binary pr183-1b07c48, CH v54.0.0Environment: isolated root, non-root operator with a systemd user-delegated parent (
Two findings worth recording:
|
… the VM's Codex P1: restore launched the VMM under the old record's cgroup config while persisting the snapshot's afterwards, and Prepare could not lower quota past a leftover burst. Knobs now follow the Network precedent: restore keeps the VM's knobs (the snapshot's describe its source VM), clone inherits from the snapshot with per-knob flag overrides like --nics. Restore still launches under the target config so a --force cross-config restore derives the scope from the new vCPU count. Prepare zeroes a leftover burst before writing cpu.max (kernel requires burst <= quota).
Contributor
Author
cb4e9d3 — cgroup knobs are host policy (addresses the restore P1)Direction settled in discussion: knobs follow the Network precedent — they are host-side policy that stays with the VM, not guest state that travels in snapshots.
Coverage requested: unit tests for restore-keeps-knobs and clone-flag-overrides (incl. invalid-override rejection); hardware evidence on the 16-core testbed:
|
… alone, restore preflights fit A snapshot's cgroup knobs describe its source VM; applying them on clone contradicted restore's keep-the-VM's-policy rule and made an explicit zero flag inexpressible. Restore now rejects kept knobs that no longer fit the snapshot's vCPU count before the destructive phase — the same combination previously failed only at Prepare, after the VMM was gone, and retried into the same failure.
…urst zeroing to reused scopes LaunchSpec.SockPath/PIDPath became pure derivations of Rec.RunDir once Rec landed; computing them inside LaunchVMProcess removes the only way a caller could set them inconsistently. Fresh scopes get the kernel's burst=0 default, so the zeroing write now runs only when Mkdir reports the scope pre-existed.
This was referenced Aug 5, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #182 (frozen after three review rounds).
What
Every VMM now spawns directly into
<cgroup2 root>/cocoon.slice/vm-<id>.scopevia clone3CLONE_INTO_CGROUPat the sharedLaunchVMProcessseam — one integration point covers CH and FC across start, restore, and clone. Three raw knobs ride intypes.Configand asvm runflags (--cpu-weight,--cpu-quota-us+--cpu-period-us,--cpu-burst-us); absent knobs resolve to the Guaranteed-at-N defaults: quota = N x period, weight = N, burst = 0.--cpu Nfinally means what it says.Motivation is measured, not theoretical: on the 16-core testbed a 1-vCPU VM pegs 100.0% with pure CPU load, but 111–113.5% once it does I/O — virtio queue threads and io_uring io-wq workers live outside the vCPU count, and that overhead scales with vCPUs and NICs.
Lifecycle
cgroup.killsweeping the CH pty child and stray forks before rmdir (EBUSY-aware poll).+cpuinsubtree_control, end-state verified) fails with an error naming the exact missing file.subtree_controlwrites are read-gated — they take the kernel-widecgroup_mutex, so steady-state launches must not pay a no-op write (clone-burst path).vm statusgains a THROTTLED column fromcpu.stat.Not in this PR
Hardware acceptance on the testbed (default-cap enforcement, capped-vs-raised-quota tax A/B, 1:3 weight discrimination, start-latency delta) runs as a follow-up round per the issue's acceptance list.
Second commit is the whole-repo loc-justify cut application (net −41 prod): single-user map helpers inlined (
utils/map.godeleted),OrDefault/cmp.Or/DeleteFuncreductions.applyFiltersdeliberately kept non-destructive — its test encodes that contract.