Follow-up to #17 (ComputePlane): deliver it as composable presets on `kind: Kubernetes`, not a new kind

[myasnikovdaniil]

Follow-up to #17 (ComputePlane for untrusted-code workloads, merged). The mechanism #17 settled on is right — remote Flux apply onto a Kamaji+KubeVirt cluster, untrusted code behind a per-VM kernel boundary. This is about the user-facing surface: I'd argue ComputePlane should not be a new top-level kind. It should be the existing kind: Kubernetes carrying composable presets along two orthogonal axes, plus a tenant-side access-control extension that lets some apps be hidden from regular tenant users.

This is a small generalization of something #17 already does: it defines a ComputePlane "<profile>" (node groups, GPU types, k8s version, autoscaling bounds) "defined once at the tenant/platform level and referenced by name." I'm proposing to lift that profile idea onto kind: Kubernetes itself, split it into two axes, and drop the separate kind.

Why not a new kind

• design-proposal: compute plane for untrusted-code workloads #17 concedes they're the same object. From the approval summary: "from the management plane, a ComputePlane and a regular managed kind: Kubernetes are the same object." A distinct kind is UX sugar over identical machinery (Kamaji control plane, KubeVirt nodes, autoscaler, addon Flux apply). Make the variation a field, not a type.
• Kinds don't compose; presets do. "Isolation posture" and "what's preinstalled" are independent. As kinds you'd get a combinatorial explosion (a kind per cell: sandbox-gpu, sandbox-minimal, meshed-gpu…). As two preset fields you get the whole matrix for free.
• One substrate to maintain. Every future isolation/connectivity need becomes a new preset value, not a new CRD + reconcile path + RBAC surface.

Axis 1 — isolation / trust posture (isolationProfile)

Where the cluster sits on the trust ↔ containment spectrum; drives credentials, hardening, and network policy.

Preset	Tenant holds admin?	Network posture	For
standard (default)	yes — tenant administers	normal egress; tenant-trusted	today's behavior — clusters a tenant runs themselves
sandbox	no — kubeconfig withheld (managedDataplane)	hardened: restricted PSA + admission; deny egress → management kube-apiserver; scoped per-service egress only	#17's ComputePlane — untrusted-code catalog apps; hardening is tamper-proof because the tenant can't get admin
cluster-meshed	yes — tenant-trusted	wired into the cross-cluster data plane (PR #7 mesh / TenantMeshLink) to reach shared/tenant services (Ceph, Postgres), still denied host kube-API	trusted tenant clusters that need tight coupling to tenant data

sandbox vs cluster-meshed is exactly the distinction review #4 drew: a broad node-to-node mesh is fine for a tenant-trusted cluster, but untrusted sandbox code needs narrow per-service egress — never a wide mesh. Encoding that as two named presets makes the connectivity contract explicit instead of a hand-tuned policy. (#17 §5's scoped egress thus becomes the definition of the sandbox preset, not a separate section.)

Axis 2 — component bundle (componentProfile)

What's preinstalled inside, independent of trust posture. #17 already notes the kubernetes app deploys its own addons (cert-manager, ingress-nginx, CNI, monitoring) via remote Flux apply — a bundle preset just parameterizes that set.

• minimal — CNI only, bare cluster
• standard-addons — current default set (ingress-nginx, cert-manager, monitoring agents, autoscaler)
• gpu — adds GPU node groups + operator/drivers
• named app bundles — e.g. a notebook/code-exec runtime stack

Orthogonal to axis 1: isolationProfile: sandbox + componentProfile: gpu is a hardened GPU cluster for notebooks; isolationProfile: standard + componentProfile: gpu is a tenant-run GPU cluster — same substrate, two field values.

Hiding / blocking app access inside a tenant (RBAC / cozystack-api extension) — initial sketch

A capability #17 only touches obliquely (its review-#1 resolution: "withhold admin/write, allow scoped read") but that deserves first-class treatment, because it does double duty — a general tenant feature and the enforcement substrate for the sandbox preset above.

Idea. Let managed apps be marked so a tenant's regular users cannot see or manage them, while privileged subjects (tenant-admin / parent-tenant operator / superadmin) still can. Two separable things to control:

• Visibility — does the app object appear in kubectl get / the dashboard for a given subject.
• Mutation — can a subject create / update / delete it.

Where it's enforced. In the aggregated cozystack-api apiserver (pkg/registry/apps/application/rest.go), which already serves apps.cozystack.io/* and converts them to HelmReleases — not UI-only. A dashboard-only filter is bypassable by anyone holding a kubeconfig, so the gate must live at the API layer.

Granularity vs. RBAC limits.

• Per-kind ("regular users can't touch Jupyter at all") → plain Kubernetes RBAC (a tenant Role that omits the verbs on that kind).
• Per-instance ("this Jupyter, not that one") or label-driven ("hide apps labelled X") → cannot ride vanilla RBAC, because list/watch can't be filtered by name or label. Needs cozystack-api to filter responses by caller identity. Feasible since Cozystack owns the apiserver, but it's apiserver work, not a Role manifest.

Shape sketch (early, names TBD). An annotation/field on ApplicationDefinition — likely extending spec.dashboard, which already drives UI presentation / module: true — marking an app (or instance) as restricted, plus a tenant-role tier (regular vs privileged) that cozystack-api checks before listing/mutating.

How it composes.

• It is the enforcement layer for isolationProfile: sandbox: "withhold the admin kubeconfig" and "regular tenant users can't manage the sandbox cluster object or its hardening" are the same RBAC-scoping mechanism, applied to the <cluster>-admin-kubeconfig secret and the cluster CR. Note "hide the app CR from the dashboard" ≠ "withhold the kubeconfig secret" — different objects; sandbox needs both.
• Standalone, it's intra-tenant privilege separation: a parent/operator deploys infra or sensitive apps into a child namespace that the child's regular users shouldn't see or break.

What carries over from #17 unchanged

• placement: { ManagementPlane | <named cluster> } on ApplicationDefinition — routes a catalog app into a named kind: Kubernetes cluster regardless of its presets. (design-proposal: compute plane for untrusted-code workloads #17 Open Q [docs] Fixup links in the desgin proposal guide #2 already wanted placement generalized to a named target.)
• Single-tenant (design-proposal: compute plane for untrusted-code workloads #17 §3): each kind: Kubernetes is already a single tenant's cluster; presets don't change that, and the model now naturally allows N sandbox clusters per tenant (the merged single-string computePlane: module caps it at one).
• Sharing node pools / capacity across tenants stays fine; sharing a cluster does not.

Open questions

• Preset naming + defaults (standard as default keeps every existing manifest valid).
• Are axis-1 presets mutually exclusive, or can sandbox + a scoped mesh-link coexist (hardened cluster that still reaches one tenant service)? The matrix suggests posture might be better as composable flags than a single enum.
• How bundles are defined — as platform-level profile objects referenced by name (echoing design-proposal: compute plane for untrusted-code workloads #17's single-source-of-truth profile argument), so a tenant references a preset, never inlines an override blob.
• Access-control model: is the visibility/mutation split worth exposing, or is a single restricted flag enough for v1? And do we start per-kind only (plain RBAC) and defer per-instance/label filtering to a later iteration?

cc the #17 discussion.

[kvaps]

Strongly agree — and Timofei's review on #17 converged on the same surface (generalize placement to a named cluster; deliver the posture as a managedDataplane mode rather than a distinct kind). I've folded your proposal into the design: #27.

It adopts:

• ComputePlane = a configuration of kind: Kubernetes — two orthogonal presets isolationProfile × componentProfile, not a new kind.
• placement → named cluster (allows N sandbox clusters per tenant; the single-string computePlane: module is dropped).
• sandbox = managedDataplane + hardening + scoped per-service egress; cluster-meshed = the PR design-proposal: cross-cluster mesh for tenant access to host services #7 mesh for trusted clusters.
• Your visibility/mutation control as a first-class cozystack-api section (v1 per-kind via RBAC; per-instance/label filtering deferred) — the enforcement substrate for sandbox, and a standalone intra-tenant privilege-separation feature.

The distinct-kind framing + single-string module are moved to Alternatives (rejected); the mechanism (remote Flux apply, per-VM boundary, single-tenant) is unchanged. Thanks for the push — this is a better surface.