design-proposals: VM import from VMware vSphere - #62
Draft
Andrei Kvapil (kvaps) wants to merge 7 commits into
Draft
design-proposals: VM import from VMware vSphere#62Andrei Kvapil (kvaps) wants to merge 7 commits into
Andrei Kvapil (kvaps) wants to merge 7 commits into
Conversation
Proposes a tenant self-service path for migrating VMware VMs into Cozystack on the Konveyor Forklift engine, splitting the tenant API by lifecycle: a VMImport app registers a provider connection, and a new vmware source on VMDisk expresses the per-disk import intent. A fulfillment controller drives Forklift Plans and hands each transferred volume into its VMDisk without a second copy. The proprietary VDDK image stays out of the tree as an operator-set platform value on the existing _cluster channel. Verification against Forklift v2.11.5 and CDI v1.64.0 establishes that the raw-copy transfer path is restricted-PSS clean, and that guest conversion needs a narrow Localhost seccomp profile rather than a privileged namespace. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…p profile The Localhost/unshare.json branch is not gone on Forklift main -- a refactor moved it into pkg/controller/conversion/builder.go and duplicated it across the conversion and deep-inspection pods. That makes the upstream ask smaller than described: the mechanism exists and only needs a second way to reach it, instead of being restored. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
…le's blast radius The conversion pod needs four syscalls, not one: passt calls unshare(CLONE_NEWUSER) and then mount, umount2 and pivot_root, none of which a runtime default profile permits without CAP_SYS_ADMIN -- though the capability itself is not required, since the calls happen inside the namespace passt just created. More importantly, a node-level seccomp profile is addressable by any pod that names it, and PSA admits such a pod. Installing it therefore grants every tenant an opt-in to unshare+mount+pivot_root unless a validating policy fences off who may reference it. Enabling conversion ships two policies, not one. Also: the webhook route needs nothing from upstream, because mutating admission precedes the PodSecurity validating plugin and Forklift never reconciles a conversion pod's spec after creating it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Replace the catalog-app shape (VMImport app + VMDisk.source.vmware) with two CRDs in a new migration.cozystack.io group, reconciled by a controller in core, modeled on the Backup API: VMImportSource holds the provider connection and credentials, VMImportTask runs the one-shot import and produces VMDisk/VMInstance objects that survive its deletion. The previous revision held that no tenant-facing one-shot precedent existed; BackupJob/RestoreJob are exactly that, with the RBAC aggregation, dashboard section and credential-projection patterns included. Fold in the live-cluster verification results (restricted PSS admission, copy-free handoff with two new constraints) and the built-and-tested Forklift seccomp patch. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The map shapes the interfaces of the Forklift-created KubeVirt VM, which this design discards unstarted; the final network config belongs to the VMInstance the controller creates, and VMInstance attaches to the pod network only. The controller auto-generates the NetworkMap purely to satisfy Plan validation; the field returns additively with the network-placement design. Also spell out the empty-storageMap behavior (inventory-completed default class, Immediate-binding validation, stock default 'local' fails with a named remedy) and the static-guest-network caveat. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Most tenants want one class for every disk of an import; make that a single spec.storageClass field (the standard dashboard picker), with storageMap reduced to per-datastore overrides. Resolution order per disk: storageMap entry -> spec.storageClass -> cluster default, all subject to the Immediate-binding validation. An in-list default entry (no source, or a wildcard) was rejected as schema magic. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
One class per Task covers the real common case; the controller still builds the Forklift StorageMap internally (all datastores -> that class) to satisfy Plan validation. Splitting one VM's disks across classes by source datastore projects the old infrastructure onto the new one — if the need proves real, the field joins the spec additively. The v1 Task spec is now sourceRef, vms, storageClass. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design proposal for migrating VMware vSphere virtual machines into Cozystack, on the Konveyor Forklift engine that cozystack/cozystack#1982 already vendors.
Opened as a draft: the shape is settled, the load-bearing mechanics are verified (see below), but the upstream Forklift seccomp knob described in Design §4 is not yet submitted, and the full flow has not been run against a real vCenter yet.
Why this is here rather than in the implementation PR
cozystack/cozystack#1982 works and has been validated by its author on a live cluster, but its review surfaced questions that are design questions rather than defects: whether a VMware import is a tenant application at all, how a tenant supplies credentials when tenants cannot create Secrets, and whether an import produces managed Cozystack objects or leaves raw Forklift PVCs behind. Settling those in an 81-file PR is more expensive than settling them here.
What it proposes
The tenant API is a pair of CRDs in a new
migration.cozystack.iogroup, reconciled by a controller in core — the shape of Cozystack's own Backup API (BackupJob/RestoreJob), not a Helm chart in the catalog. AVMImportSourceregisters a long-lived connection to a provider: type, endpoint, credentials entered on the spec, materialized into a Secret by the controller, validated against the live endpoint before anything is created. AVMImportTaskruns the one-shot operation: reference a source, name the VMs, getVMDisks andVMInstances. The outputs carry no ownerReference to the Task, so deleting a Task removes the migration machinery and never the imported objects; everything Forklift-internal is owned and garbage-collected.This follows the shape proposed in the implementation PR's review. An earlier revision of this proposal kept a catalog app plus a
VMDisksource extension; it lost to the Backup precedent, which already answers one-shot lifecycle, tenant RBAC (aggregation labels shipped by the owning package), dashboard rendering (schema-generated forms and option pickers on plain CRDs), and controller-side credential materialization. The Alternatives section records the comparison.A fulfillment controller drives one Forklift Plan per source VM and hands each transferred volume into its VMDisk without a second copy, using
cdi.kubevirt.io/storage.populatedFor— the same primitive Cozystack's Velero VM-restore path already depends on in production. That closes both the raw-PVC gap raised in review and the duplicate-copy waste tracked in cozystack/cozystack#3924.The proprietary VDDK image never enters the tree: it is an optional platform value the operator sets, delivered to the migration controller, and no image field appears on any tenant API.
The privilege story, verified live
On the raw-copy path no Forklift pod moves the bytes — CDI's importer does, through
SetRestrictedSecurityContext. Verified in source against Forklift v2.11.5 / CDI v1.64.0 and empirically on a running Cozystack v1.6.2 cluster: the import was admitted into arestricted-enforcing namespace with no PodSecurity denial and ran to completion. Tenant self-service is structurally possible today.Guest conversion is the constrained path, and its requirement is four syscalls (
unshare,mount,umount2,pivot_root), not privilege: libguestfs startspasst, which sandboxes itself into fresh namespaces.baselineandrestrictedboth acceptseccompProfile.type: Localhost, and Forklift already selects aLocalhostprofile — but only when it detects OpenShift (pkg/controller/conversion/builder.goon currentmain). kubev2v/forklift#4491 is open on exactly this. The proposal pairs a node-side profile shipped through Talosmachine.seccompProfileswith a small upstream setting in theVIRT_V2V_*family; a patch in that shape is already built and unit-tested. The privileged-namespace-plus-Unconfined-webhook shape is explicitly rejected.Status of the claims
Everything load-bearing is cited to
file:line. Two claims were additionally verified on a live cluster: restricted-PSS admission of the transfer pod, and the copy-freepopulatedForhandoff (same PV throughout, no importer pod, data byte-identical) — which also surfaced two constraints now folded into §5 (volumeMode/accessModesmust be copied from the PV; the owning DataVolume must be deleted before re-pointing). Nothing has been measured against a real vCenter — the timings quoted are the implementation author's. The## Open questionssection carries a recommended default for each unsettled point.Requesting review from the reviewers of cozystack/cozystack#1982, whose findings shaped this, and from the contributor whose implementation it builds on.