cpu: map pod CPU QoS onto cocoon cgroup knobs, read scope-based stats - #67
Merged
Conversation
Cocoon now caps every VM at Guaranteed-at-N via a per-VM cgroup scope, so a rounded-up request would silently hard-cap burstable pods. Bound vCPU and quota at the pod's CPU limit, derive weight from requests with kubelet's cgroup v2 conversion, and pass the policy on every clone path too — cocoon never inherits cgroup knobs from snapshots. Per-VM CPU and throttling stats come from the scope's cpu.stat, which sees the virtio/io_uring workers /proc utime+stime misses. vk-cocoon.service raises CPUWeight within system.slice and packaging documents pairing VK_RESERVE_PERCENT with cocoon's cgroup_cpus fence.
…rity A 1m-9m CPU limit produced quota below cocoon's MinQuotaUs and every run/clone was rejected; clamp to 1000us like kubelet. A pod with no CPU resources left weight unset, falling to cocoon's vCPU-count default instead of kubelet's BestEffort minimum share; always derive weight.
…, cgroup layout into vm/ Comment budget: provider.go's handleVMGone bullets restated the switch, two recheck docs restated their callers, evictPod trimmed. Simplify: podCPUPolicy's hand-rolled fallback uses selectQuantity; the redundant min() guard in runDeferredRecheck matches the package's other backoff loops; the twin patch-retry loops share patchWithRetry; the stats path reads pod fields instead of re-splitting the map key; reconcileStaleCreates uses the package fanOut idiom; the SAC guard has one definition. Altitude: cgroup scope path + cpu.stat parsing move into vm/ beside COWSize and ConsoleSocketPath — vm/ stays the single owner of cocoon's host layout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
cocoon now spawns every VMM into a per-VM cgroup v2 CPU scope with Guaranteed-at-N defaults (cocoonstack/cocoon#183) and a host-level cpuset fence (cocoonstack/cocoon#185). Two consequences for vk-cocoon:
requests: 500m, limits: 2) would get hard-capped at 1 core after the node upgrade — K8s semantics inverted.What
--cpu-quota-usbound at the pod's CPU limit (requests only as fallback when no limit);--cpu-weightderived from requests via kubelet's cgroup v2 shares→weight conversion, so contention shares match what kubelet grants the same pod.--cpu-period-usis passed alongside the quota so the conversion never drifts from cocoon's default period.vm.CPUPolicyembedded inRunOptions/CloneOptions.cpu.statusage_usec(includes virtio/io_uring kernel workers that/proc/<pid>/statutime+stime misses; RSS stays on /proc). Throttling exposed on all three metric surfaces: cAdvisor-stylecontainer_cpu_cfs_throttled_{seconds,periods}_totalon/metrics/resourceandcocoon_vk_vm_cpu_throttled_{seconds,periods}_totalon:9091.vk-cocoon.servicesetsCPUWeight=1000(wins contention against co-located system.slice daemons); env example + docs pairVK_RESERVE_PERCENTwith cocoon'scgroup_cpusfence and addCOCOON_CGROUP_PARENTfor non-default slices.Notes
cpu.statparsing intentionally duplicate a small part of cocoon'scgrouppackage (vk-cocoon has no dependency edge to cocoon); hoisting the scope-naming contract into cocoon-common is a follow-up.limits: 500m→ half a core) instead of a rounded-up full core.Verification
go test -race -count=1 ./...— all packages passmake lint— 0 issues on GOOS=linux and GOOS=darwinasl ./...— clean on both GOOS