From eb2c262de9b3b6ecb8c43e7f4100a41009555ff5 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Wed, 10 Jun 2026 12:28:03 +0300 Subject: [PATCH 01/18] Add v2 proposals for review --- docs/proposals/v2-extended.md | 262 ++++++++++++++ docs/proposals/v2-slim-core.md | 601 +++++++++++++++++++++++++++++++++ 2 files changed, 863 insertions(+) create mode 100644 docs/proposals/v2-extended.md create mode 100644 docs/proposals/v2-slim-core.md diff --git a/docs/proposals/v2-extended.md b/docs/proposals/v2-extended.md new file mode 100644 index 000000000..6796ae048 --- /dev/null +++ b/docs/proposals/v2-extended.md @@ -0,0 +1,262 @@ +# kube-bind v2 Extended: Backend API, CLI, UI + +* Status: **DRAFT — for iteration** +* Authors: @mjudeikis +* Date: 2026-06-10 +* Builds on: [v2-slim-core.md](v2-slim-core.md) (Proposed)) + +## Summary + +The v2 core contract is up for discussion: a binding is one `kubectl apply` of a Secret + +`Connection` + bindings, consumed by the konnector, with zero kube-bind components on +the provider. This proposal designs everything *around* that contract — the optional +service layer that answers the questions the core deliberately doesn't: + +* **Who are you?** (auth: OIDC, sessions) +* **What may you have?** (catalog: curated offerings on top of raw exported APIs) +* **Here are your credentials.** (issuer: per-consumer SA/RBAC/kubeconfig, tenancy) +* **Here is your bundle.** (gateway: HTTP API whose terminal output is the one-apply file) +* **You stopped coming.** (reaper: GC keyed off the core's Lease) +* **Make it pleasant.** (CLI, UI) + +The defining rule, inherited from the core: **every path through this layer terminates +in the same one-apply bundle.** The service layer negotiates; it never syncs. If the +backend is deleted the day after binding, sync is unaffected. + +``` + provider cluster consumer cluster + ┌────────────────────────────────────┐ + │ extended layer (this proposal) │ + │ ┌──────────┐ ┌────────┐ ┌──────┐ │ bundle + │ │ gateway │ │ issuer │ │reaper│ │ (one-apply file) ┌───────────┐ + │ │ auth, UI │ │ creds, │ │ Lease│ │ ──────────────────────▶│ konnector │ + │ │ catalog │ │ tenancy│ │ GC │ │ via CLI / UI / GitOps│ (core) │ + │ └──────────┘ └────────┘ └──────┘ │ └─────┬─────┘ + └────────────────────────────────────┘ │ + ▲ sync (core contract: CRDs, spec ⇧, status ⇩) │ + └────────────────────────────────────────────────────────┘ +``` + +## Goals + +* Every component independently deployable and optional; any subset works. The core + (GitOps-only, no extended layer at all) remains a first-class path forever. +* The backend's terminal output is **exactly** the core's one-apply bundle — no + intermediate request/response CRDs on the consumer, no phase-gated handshake. +* Tenancy lives here: per-consumer provider namespaces / kcp workspaces are an issuer + concern, invisible to the core (which just sees a kubeconfig whose RBAC fences it in). +* Pluggable auth from day one — OIDC is the reference implementation, not the contract. +* HA-capable by construction (the v1 in-memory-session/single-replica limitation, + roadmap #424/#488, must not survive into v2). + +## Non-Goals + +* Anything that changes core sync semantics — the core contract is immutable, this layer must adapt around it. +* Marketplace/billing/quotas (a future layer above this one; the catalog leaves room). +* Re-implementing v1's wire protocol (`BindingProvider`, `BindingResourceResponse`, + `APIServiceExportRequest` flow). v2 extended is a clean protocol. + +## Components + +### 1. Catalog (provider-side CRDs) + +Raw discovery already exists in core (`Connection.status.exportedAPIs` from labeled +CRDs / the workspace boundary). The catalog adds **curation**: human-facing metadata +and sensible defaults that turn "a list of CRD names" into "a service you'd choose". + +Group: `catalog.kube-bind.io`. Two kinds, successors of v1's +`APIServiceExportTemplate` and `Collection`: + +```yaml +apiVersion: catalog.kube-bind.io/v1alpha1 +kind: Export # one offering +metadata: + name: mangodb +spec: + title: MangoDB + description: Managed MangoDB instances with automated backups. + icon: { url: … } # optional + docs: https://… # optional + apis: # what a binding to this offering syncs + - name: mangodbs.mangodb.io + - name: mangodbbackups.mangodb.io + defaults: # copied into the generated ClusterBinding + conflictPolicy: Fail + relatedResources: + - group: "" + resource: secrets + direction: FromProvider + selector: + labelSelector: + matchLabels: + mangodb.io/managed: "true" +``` + +```yaml +apiVersion: catalog.kube-bind.io/v1alpha1 +kind: Collection # grouping for UI/CLI browsing +metadata: + name: databases +spec: + title: Databases + exports: + - name: mangodb +``` + +Notes: + +* The catalog is **derived-from-core-truth**: an `Export` listing an API that isn't + actually exported (label/boundary) gets a condition; the gateway hides it. The label + remains the source of truth, the catalog is presentation + defaults. +* These CRDs live on the provider and are read only by the gateway/UI/CLI. The + konnector never sees them. + +### 2. Issuer (provider-side controller) + +Everything v1's `backend/kubernetes` did, made a named component. The issuer is a Go +interface (provision boundary, mint credentials, revoke); **the in-tree implementation +is plain Kubernetes only** — the kcp issuer lives in the separate `contrib/kcp` +distribution, which wires its own implementation against the same interface. + +* Per consumer identity: provision the tenancy boundary — a namespace set on plain + Kubernetes (workspaces, in the contrib/kcp issuer). +* Mint credentials: ServiceAccount + RBAC scoped to exactly the exported APIs (+ + declared related resources) within that boundary + kubeconfig. This fixes v1's + cluster-admin-ish `kube-binder` ClusterRole (roadmap #303: reduced footprint). +* **Credential mechanism: long-lived SA token** (v1 behavior, secret-based + ServiceAccount token). Trade-off accepted deliberately: zero rotation friction and no + konnector-side refresh machinery, at the cost of security posture — and noting + upstream Kubernetes is steering away from secret-based SA tokens, so this is + revisitable without API change (the bundle's Secret is replaceable; a bounded-token + + reissue mode can be added later behind the same interface). Revocation = delete the + `Grant` → issuer deletes the SA/token. +* Records issuance in **`Grant`** (`catalog.kube-bind.io`): "identity X was issued + credentials Y for export Z". The anchor for revocation, audit, and the reaper. + +### 3. Gateway (HTTP API) + +Stateless HTTP server (sessions externalized), serving: + +One gateway fronts exactly **one provider** — catalog aggregation across providers is a +future layer above this one, already possible externally because the protocol's output +is just a bundle. + +| Endpoint | Purpose | +|---|---| +| `GET /api/provider` | provider metadata + supported auth methods (successor of v1 `/api/exports`) | +| `GET /api/catalog` | `Export`s + `Collection`s visible to the caller | +| `POST /api/bind` | input: export name + consumer identity → drives issuer → returns a **one-time pickup URL** for the bundle | +| `GET /api/bundle/` | single-use, short-TTL (5 min) bundle pickup — **returns the bundle**, then the token is dead | +| `POST /api/apply` | optional, flag-gated: browser-apply path — gateway applies the bundle (+ konnector install) into a consumer cluster using a caller-supplied consumer kubeconfig | +| `GET /api/authorize`, `/api/callback` | auth flow (delegated to authenticator plugin) | +| `GET /api/healthz` | health | + +**The bundle is the one-apply file**, delivered through a one-time pickup URL so live +credentials are fetched exactly once and never stored at rest in the gateway. Content +negotiation on pickup: `application/yaml` returns the literal multi-doc bundle (Secret + +`Connection` + `ClusterBinding`); `application/json` wraps the same objects in a thin +envelope (`{ bundle: [...] }`). There is no other handshake state: no request objects to +poll, no phases to wait on. `curl` bind + pickup piped to `kubectl apply -f -` is a +complete client. + +### 4. Auth (pluggable) + +* `Authenticator` interface: `Routes()` (mounted under `/api/auth/…`) + + `Authenticate(r) (Identity, error)`. Reference implementations: OIDC (code grant + + PKCE, as v1) and `kubernetes` (SubjectAccessReview against the provider — for + in-platform UIs that already hold a cluster identity). +* The embedded mock-OIDC server survives **only** as a dev-mode flag. +* Session store stays an interface (memory, Redis as today) but the gateway must be + fully functional with ≥2 replicas out of the box — Redis (or any external store) is + the documented production default, memory is dev-only. +* Identity → tenancy key: `issuer + "/" + subject` hash, as v1, so the same human gets + the same boundary on re-bind. + +### 5. Reaper (provider-side, optional) + +The core leaves dead-consumer GC explicitly to this layer, keyed off the per-Connection +`Lease` the konnector maintains: + +* Lease expired beyond TTL → mark the issuance stale → (configurably) revoke + credentials, then delete kube-bind-created namespaces and synced objects. +* TTLs and the destructive step are opt-in and conservative by default (revoke ≠ + delete; deletion requires explicit enablement). + +### 6. CLI (`kubectl bind`) + +Thin client over the gateway; everything it does is reproducible by hand: + +```sh +kubectl bind login https://mangodb.example.com # auth, cache token +kubectl bind catalog # list Exports/Collections +kubectl bind mangodb # bind an Export: + # POST /api/bind → bundle + # → kubectl apply (or -o yaml) +kubectl bind mangodb -o yaml > binding.yaml # GitOps mode: print, don't apply +``` + +* `--install-konnector` (default on for interactive use) installs/upgrades the v2 + konnector, as v1 did. +* The CLI never creates bespoke objects — it applies the gateway's bundle verbatim. + `-o yaml` output committed to git is byte-for-byte the GitOps path. +* v1 subcommands that existed to ferry the old handshake (`apiservice`, `deploy`, + per-resource polling) disappear. + +### 7. UI (SPA) + +Browse catalog → authenticate → bind → then either: + +* **download the bundle** (via the one-time pickup URL) / copy a `kubectl bind` + one-liner, or +* **browser-apply** (v1's UI-only flow, roadmap #406, kept): the user supplies a + consumer-cluster kubeconfig (or the UI runs in-platform where one is already held), + and the gateway's `/api/apply` applies the bundle and installs the konnector into the + consumer cluster. + +The UI is a pure gateway client; it holds no flow state the gateway doesn't have. +Browser-apply is flag-gated on the gateway and off by default — it means consumer +credentials transit the gateway, which deployments must consciously accept. + +## Packaging & repo + +Per the frozen core layout: `v2/backend` (gateway + issuer + reaper + auth), `v2/cli`, +`v2/web`. The backend ships as **one binary with module flags** (`--enable-gateway`, +`--enable-issuer`, `--enable-reaper`, `--enable-apply`) — operational simplicity over +purity; the boundaries stay as Go packages so a future split costs a `main.go`, not a +refactor. All of it depends on `v2/sdk` only. The kcp distribution (`contrib/kcp`) +remains separate, providing its own issuer implementation behind the same interface. + +## Migration notes + +* The v1 wire protocol is not bridged: v1 CLI cannot talk to a v2 gateway. Both stacks + can run side by side on one provider during transition (different endpoints, disjoint + CRD groups). +* v1 catalog objects (`APIServiceExportTemplate`/`Collection`) translate mechanically + to `Export`/`Collection`; a converter script ships with the backend. + +## Decided + +* **Packaging**: one `kube-bind-backend` binary; gateway/issuer/reaper/apply are module + flags, boundaries kept as Go packages. +* **Issuance anchor**: `Grant` in `catalog.kube-bind.io` — the typed record of + "identity X was issued credentials Y for export Z"; anchor for revocation, audit, + reaper. +* **Credentials**: long-lived secret-based SA token (v1 behavior) — zero rotation + friction accepted over security posture; revocation via `Grant` deletion; bounded + tokens addable later behind the same issuer interface without API change. +* **Catalog vocabulary**: `Export` + `Collection`. +* **Bundle delivery**: one-time pickup URL, 5-minute TTL, single use; bundle never + stored at rest in the gateway. +* **kcp**: stays a separate distribution (`contrib/kcp`) providing its own issuer + implementation; the in-tree backend issuer is plain Kubernetes only. +* **UI reach**: browser-apply path **kept** (roadmap #406) — gateway `/api/apply` + applies bundle + installs konnector into the consumer cluster with a caller-supplied + kubeconfig; flag-gated, off by default, consumer credentials transiting the gateway + is an explicitly accepted trade-off when enabled. +* **Federation**: one gateway = one provider; cross-provider aggregation is a future + layer above the bundle protocol. + +## Open questions + +None — initial design questions resolved (see **Decided**). New questions raised during +review go here. diff --git a/docs/proposals/v2-slim-core.md b/docs/proposals/v2-slim-core.md new file mode 100644 index 000000000..ae51432a4 --- /dev/null +++ b/docs/proposals/v2-slim-core.md @@ -0,0 +1,601 @@ +# kube-bind v2: Slim Core, Pluggable Everything Else + +* Status: **Proposed** +* Authors: @mjudeikis +* Date: 2026-06-10 +* Supersedes parts of: [backend-only-bindings.md](backend-only-bindings.md) +* Follow-up: [v2-extended.md](v2-extended.md) — the optional service layer (backend API, + CLI, UI) built on this contract + +Changes to this document now require re-opening the decision log in **Decided**; the +remaining **Open questions** (OpenAPI fidelity spike, built-ins filter) are tracked for +implementation, not re-design. + +## Summary + +kube-bind v2 splits the project into a **slim sync core** and an **optional service +layer**. The core does exactly one thing: given a Secret containing a kubeconfig, a +connection object, and a binding per API, it syncs CRDs and resource instances between a +consumer and a provider cluster — same scope, same names, no transformation — and reports +conflicts instead of papering over them. + +Everything else — OIDC, the auth handshake, the web UI, the CLI, templates, collections, +service-account provisioning — moves out of the core into optional, pluggable components +whose only job is to *produce the core inputs* (the kubeconfig Secret, the `Connection`, +and the `Binding`/`ClusterBinding` objects). + +``` + v2 contract + + ┌────────────────────────────────────────────────────────┐ + │ optional layer (any of: backend+OIDC+UI, CLI, Helm, │ + │ GitOps, kcp integration, your own controller…) │ + └───────────────┬────────────────────────────────────────┘ + │ produces + ▼ + Secret (kubeconfig) + Connection (provider link + schema pull) + + Binding / ClusterBinding (per bound API) + │ consumed by + ▼ + ┌────────────────────────────────────────────────────────┐ + │ core: konnector — CRD sync, spec ⇩up / status ⇩down, │ + │ related resources, conflict detection │ + └────────────────────────────────────────────────────────┘ +``` + +## Motivation + +### What v1 got right + +* The split of *spec flows consumer → provider, status flows provider → consumer* is sound + and battle-tested ([spec_reconcile.go](../../pkg/konnector/controllers/cluster/serviceexport/spec/spec_reconcile.go), + [status_reconcile.go](../../pkg/konnector/controllers/cluster/serviceexport/status/status_reconcile.go)). +* The finalizer pattern (`kubebind.io/syncer` blocks consumer deletion until the provider + copy is gone) prevents data loss. +* `APIServiceBinding` already approximates the minimal object: it is essentially + `kubeconfigSecretRef` + conditions. +* The backend-only-bindings proposal proved there is real demand for a flow with no HTTP, + no OIDC, no browser. + +### What makes v1 heavy + +1. **The isolation/scope machinery.** Three isolation modes (`Prefixed`, `Namespaced`, + `None`) × two informer scopes (`Cluster`, `Namespaced`) × `APIServiceNamespace` + namespace-mapping objects. Cluster-scoped resources can be *converted* to namespaced on + the provider, names get rewritten, namespaces get re-mapped. This is the single largest + source of complexity in the syncer (the whole + [isolation/](../../pkg/konnector/controllers/cluster/serviceexport/isolation/) strategy + layer plus `APIServiceNamespace` round-trips in every reconcile), and the source of the + most confusing API fields (`informerScope` vs `isolation` vs deprecated + `clusterScopedIsolation` vs CRD `scope`). + +2. **The object zoo.** A bind today touches up to ten CRDs: `APIServiceExportTemplate`, + `Collection`, `APIServiceExportRequest`, `APIServiceExport`, `BoundSchema`, + `APIServiceNamespace`, `ClusterBinding`, `APIServiceBinding`, + `APIServiceBindingBundle`, `BindableResourcesRequest`. Several exist only to ferry the + *handshake* (template discovery, request/response), not to drive sync. + +3. **Coupled service layer.** The backend bundles OIDC (even an embedded mock OIDC + server), session storage, cookie handling, an SPA, kubeconfig minting, and the + provider-side controllers in one binary. `--frontend-disabled` exists but is a flag on + a monolith, not an architecture. + +4. **Sync engine inefficiencies.** Per-export dynamic controller spawning duplicates + informers per GVR; the konnector watches *all* secrets cluster-wide; heartbeat interval + and many behaviors are hardcoded. + +## Goals + +* **One apply.** A full end-to-end binding to a provider is a single + `kubectl apply -f` of one file containing all objects (Secret + `Connection` + + bindings). Nothing else, nothing more: no handshake, no waiting on phases, no ordering + requirements — the konnector converges from whatever has been applied. +* Scope-preserving, name-preserving sync ("plain sync"): namespaced ↔ namespaced in the + same namespace name, cluster-scoped ↔ cluster-scoped under the same name. +* First-class conflict handling: never silently fight over or adopt objects another actor + owns. +* CRD/schema delivery in core: applying the core objects yields working APIs on the + consumer. +* Related-resource sync (Secrets/ConfigMaps referenced by synced objects) in core. +* Zero kube-bind components required on the provider cluster for the core path. +* Portability and extension points so v1-style behaviors (tenancy mapping, handshakes, + UIs) can be rebuilt *on top*. + +## Non-Goals + +* In-core multi-consumer isolation on a shared provider namespace-space. Tenancy becomes a + deployment concern (dedicated namespaces/cluster/kcp workspace per consumer) or an + extension (see [Mapper](#extension-point-1-mapper)). +* Scope conversion (cluster-scoped → namespaced) of any kind. +* In-core auth: OIDC, sessions, browser flows, token minting. +* Automatic conversion from v1alpha2 objects (see [Migration](#migration)). + +## Design + +### The core API: three kinds + a Secret + +**0. Provider opt-in is a label or a boundary, not a CRD.** On a plain Kubernetes +provider, the operator marks which CRDs are exported by labeling them: + +```sh +kubectl label crd mangodbs.mangodb.io "core.kube-bind.io/exported=true" +``` + +Whatever carries the label *and* is readable by the consumer's credentials is exported. + +On **CRD-less providers** (kcp and kcp-like systems — APIs served from APIBindings / +logical clusters, no CRD objects to label), the export boundary is the **logical cluster +itself**: the kubeconfig points at a workspace that was deliberately populated with +exactly the APIs to offer, so everything discoverable there (minus built-ins) is +exported. No label needed — curation happened when the workspace was assembled. + +Either way this is the entire provider-side footprint of the core: a label (or a +workspace boundary) plus RBAC. The credentials' RBAC *is* the authorization model; there +is no separate permission/claim grant object in core. + +**1. A Secret with a kubeconfig** pointing at the provider cluster, living in the +konnector's designated namespace (default `kube-bind`). + +**2. `Connection`** (cluster-scoped) — the link to one provider. Owns credentials and +schema delivery, and surfaces what the provider offers: + +```yaml +apiVersion: core.kube-bind.io/v1alpha1 +kind: Connection +metadata: + name: mangodb-provider +spec: + # Immutable. The only credential reference in the core. + kubeconfigSecretRef: + namespace: kube-bind + name: mangodb-provider + key: kubeconfig + + # How exported APIs reach the consumer. + schema: + source: Auto # Auto (default): CRD if readable on the provider, else OpenAPI + # CRD: read apiextensions CRDs verbatim + # OpenAPI: synthesize CRDs from discovery + /openapi/v3 + # (CRD-less providers: kcp, kcp-like systems) + pullPolicy: Bound # Bound (default): pull only APIs that have a Binding/ClusterBinding + # All: eagerly pull every exported API readable by the credentials + # None: never install CRDs (user/extension manages them) + updatePolicy: Always # Always: follow provider schema changes | Once: pin at first pull + + # Bind everything: konnector maintains a managed ClusterBinding (named after this + # Connection) covering all exported APIs, kept in sync with status.exportedAPIs. + autoBind: false +status: + exportedAPIs: # discovery result: exported CRDs visible to these credentials + - name: mangodbs.mangodb.io + scope: Namespaced + versions: ["v1"] + conditions: [] # SecretValid, Connected, SchemaInSync +``` + +**3. `ClusterBinding`** (cluster-scoped) and **`Binding`** (namespaced) — activate +instance sync for one or more APIs, following the `ClusterRole`/`Role` convention. APIs +are referenced by their **CRD name** on the provider (`.`) — no +group/version/resource triples to get out of sync with the schema: + +```yaml +apiVersion: core.kube-bind.io/v1alpha1 +kind: ClusterBinding # syncs objects of these APIs cluster-wide +metadata: + name: mangodb +spec: + connectionRef: + name: mangodb-provider + + # One or more exported CRDs to sync, by CRD name on the provider. + apis: + - name: mangodbs.mangodb.io + - name: mangodbbackups.mangodb.io + + # Conflict behavior for pre-existing target objects (both directions). + conflictPolicy: Fail # Fail | Adopt + + # Related resources synced alongside instances of the bound APIs. + # Trimmed v1 PermissionClaims: explicit, reviewable, no JSONPath references in core v1. + relatedResources: + - group: "" + resource: secrets + direction: FromProvider # FromProvider | FromConsumer + selector: + labelSelector: + matchLabels: + mangodb.io/managed: "true" +status: + conditions: [] # Connected, Synced, Conflicts + boundAPIs: # per-API observed state + - name: mangodbs.mangodb.io + crdHash: sha256:… # applied schema version + conflicts: [] # objects skipped due to foreign ownership +``` + +```yaml +apiVersion: core.kube-bind.io/v1alpha1 +kind: Binding # same spec, but syncs only objects in its own namespace +metadata: + name: mangodb + namespace: team-a +spec: + connectionRef: + name: mangodb-provider + apis: + - name: mangodbs.mangodb.io +``` + +The namespaced `Binding` gives teams self-service sync scoped to their namespace with +plain namespace RBAC, and is the v2 answer to v1's `informerScope: Namespaced`. A +`Binding` listing a cluster-scoped CRD is invalid for that entry (per-API condition, no +sync). Where a `ClusterBinding` and a `Binding` cover the same API on the same +connection, the `ClusterBinding` wins and the `Binding` gets a per-API condition. +"Bind everything" is `Connection.spec.autoBind: true` — the konnector then maintains a +managed `ClusterBinding` mirroring `status.exportedAPIs`. + +That is the **entire core API**: three kinds plus the Secret. No kube-bind CRDs are +required on the provider; the konnector reads provider CRDs through the ordinary +apiextensions API and reads/writes instances through the dynamic client. + +### The whole UX: one apply + +The acceptance test for the core is that a complete binding is **one file, one command** +(konnector already installed, once, like any controller): + +```yaml +# mangodb-binding.yaml +apiVersion: v1 +kind: Secret +metadata: + name: mangodb-provider + namespace: kube-bind +stringData: + kubeconfig: | + +--- +apiVersion: core.kube-bind.io/v1alpha1 +kind: Connection +metadata: + name: mangodb-provider +spec: + kubeconfigSecretRef: + namespace: kube-bind + name: mangodb-provider + key: kubeconfig +--- +apiVersion: core.kube-bind.io/v1alpha1 +kind: ClusterBinding +metadata: + name: mangodb +spec: + connectionRef: + name: mangodb-provider + apis: + - name: mangodbs.mangodb.io +``` + +```sh +kubectl apply -f mangodb-binding.yaml +``` + +…and the APIs are live on the consumer. Design rule this imposes: **all core objects are +order-independent and level-triggered**. A binding referencing a not-yet-existing +`Connection`, a `Connection` referencing a not-yet-existing Secret, a binding listing a +not-yet-exported CRD — none of these are errors, all are `Pending` conditions that +resolve when the missing piece arrives. No phase gating, no request/response objects, no +controller that must answer before the user may apply the next thing. (Equivalently: +`kubectl delete -f mangodb-binding.yaml` is the complete, ordered-don't-care unbind.) + +### Schema sources: CRD and OpenAPI + +The konnector must install a working CRD on the consumer for every bound API. Two ways +to obtain it, selected by `Connection.spec.schema.source` (`Auto` probes CRD first, +falls back to OpenAPI): + +* **`CRD`** — read the CRD from the provider's apiextensions API and apply it (minus + webhook conversion, status). Highest fidelity; requires the provider to *have* CRDs + and the credentials to read them. +* **`OpenAPI`** — for CRD-less providers (kcp and kcp-like systems, aggregated APIs): + synthesize a CRD from what every Kubernetes-shaped API server already serves: + * **discovery** (`/apis//`) → plural/singular/kind/shortNames, + categories, `namespaced`, verbs, and the served versions of the group; + * **`/openapi/v3/apis//`** → the structural schema per version, + including `x-kubernetes-*` extensions and defaults; + * subresource detection (status/scale) from the OpenAPI paths. + + Known fidelity limits, stated honestly: CEL validation rules and other + server-side-only constraints may not round-trip through OpenAPI — the consumer-side + CRD can be *looser* than the provider's real validation, and the provider remains the + enforcing side (a consumer object that passes locally can still be rejected upstream; + that surfaces as a per-object sync condition, not silent loss). + +Binding by CRD name (`.`) works identically under both sources — it's +just resource + group; with the OpenAPI source there simply is no CRD object behind the +name on the provider. `status.exportedAPIs` is computed per source: labeled CRDs for +`CRD`, discovery (minus built-in groups) for `OpenAPI`. + +### Sync semantics + +| Aspect | v2 behavior | +|---|---| +| Identity | `ns/name` on consumer == `ns/name` on provider. Cluster-scoped stays cluster-scoped. No prefixes, no remapping, no `APIServiceNamespace`. | +| Selection | `ClusterBinding` syncs all instances of its listed APIs cluster-wide; namespaced `Binding` syncs only instances in its own namespace. `Connection.spec.autoBind` binds everything exported. | +| Spec | consumer → provider, server-side apply with a dedicated field manager. | +| Status | provider → consumer, status subresource update. | +| Namespaces | If the consumer namespace doesn't exist on the provider, the konnector creates it (annotated as kube-bind-created; only kube-bind-created namespaces are cleaned up on unbind). | +| Deletion | Finalizer on consumer object (as today); consumer deletion propagates to provider, finalizer released when provider copy is gone. Provider-side deletion of a synced object is treated as drift and re-created (consumer is source of truth for spec). | +| Schema | Per `Connection.spec.schema`: konnector obtains schemas via `source: CRD` (read apiextensions) or `OpenAPI` (synthesize from discovery + `/openapi/v3` — CRD-less providers like kcp), pulls per `pullPolicy: All` or `Bound`, applies on the consumer (owner-ref to the `Connection`). `updatePolicy: Always` keeps following provider schema changes; `Once` pins. CRDs with `strategy: Webhook` are refused (per-API condition + skip). | +| Discovery | `Connection.status.exportedAPIs`: labeled CRDs (`source: CRD`) or discovery minus built-ins (`source: OpenAPI`, where the logical-cluster boundary is the export boundary) — core-level discovery with zero provider CRDs. | +| Related resources | Selected Secrets/ConfigMaps sync in the declared direction, same identity rules, scoped like their binding. | +| Informers | One shared dynamic informer per GVR per connection. Cluster-wide informers for `ClusterBinding`; namespace-scoped informers where only namespaced `Binding`s exist. | + +### Conflict handling (the new hard part) + +Identity mapping means two writers can legitimately collide. Core rules: + +1. Every object the konnector writes carries ownership markers (annotation pair: + binding UID + source cluster UID — same idea as today's + `kube-bind.io/consumer-uid` / `provider-uid` annotations, kept). +2. Before first write to a target object that already exists **without** our markers: + * `conflictPolicy: Fail` (default) — do not touch it. Record it in + `status.boundResources[].conflicts`, set `Conflicts` condition, emit an Event, + keep syncing everything else. + * `conflictPolicy: Adopt` — take ownership (stamp markers, SSA force-apply). +3. Two `Binding`s (or two consumers against one provider target) claiming the same + object: second writer sees foreign markers → always a conflict regardless of policy. + First-writer-wins, deterministically. +4. Field-level conflicts within an owned object are resolved by SSA with + `force=true` for our field manager on our sync direction only (spec fields upstream, + status fields downstream) — same as today, but now stated as the contract. + +### Topology: the konnector is the only running component + +Consumer-side pull, as today — and **the core has no backend**. The konnector is the +single process in the entire core path; everything v1's backend did for sync either +moved into the konnector or out of core: + +| v1 backend job | v2 home | +|---|---| +| serve template/catalog discovery | konnector reads exported-CRD labels → `Connection.status.exportedAPIs` | +| materialize exports/schemas (`BoundSchema`) | konnector reads provider CRDs via apiextensions API directly | +| create provider namespaces (`APIServiceNamespace` controller) | konnector creates them (iff RBAC allows) | +| track consumer liveness (`ClusterBinding` heartbeat) | konnector maintains a `Lease` on the provider | +| mint kubeconfigs / service accounts, rotate credentials | **out of core** — service layer or manual | +| GC artifacts of dead/vanished consumers | **out of core** — service layer can reap based on expired Leases | + +* The konnector runs in the consumer cluster (or out-of-cluster with a consumer + kubeconfig — it must not assume in-cluster). One konnector may serve many consumer + logical clusters (kcp workspaces, a cluster fleet) — see + [Engine](#engine-built-on-multicluster-runtime). +* It watches `Connection`, `ClusterBinding`, and `Binding` objects, and only the Secrets + that `Connection`s reference (fixes the v1 watch-all-secrets issue). +* One `Connection` = one provider client/informer context; all bindings referencing it + share that context. +* The provider cluster needs: reachable API server + RBAC for the supplied credentials. + No backend, no controllers, no CRDs. + +The honest cost of zero provider-side runtime: credential issuance/rotation and cleanup +after consumers that disappear forever are nobody's job in core. The Lease per +`Connection` is the hook — an optional provider-side reaper (service layer) can GC +kube-bind-created namespaces and synced objects whose Lease has expired. + +Heartbeat keeps the zero-CRD property: the konnector maintains a plain +`coordination.k8s.io/Lease` per `Connection` in a designated provider namespace. +Anything richer (v1 `ClusterBinding`-style status) is an optional provider-side +component in the service layer. + +### Engine: built on multicluster-runtime + +The v2 konnector is built on +[multicluster-runtime](https://github.com/kubernetes-sigs/multicluster-runtime) +(roadmap #299), not hand-rolled informer plumbing — and structured as a **bridge between +two cluster sets**, each fronted by an mcr provider: + +* **Provider side**: a custom mcr provider discovers "clusters" from `Connection` + objects: one `Connection` = one logical cluster (engaged when the Connection's secret + resolves, disengaged on deletion). +* **Consumer side**: also behind an mcr provider abstraction. The default is the trivial + single-cluster provider (today's shape: one konnector in one consumer cluster). But + nothing in the engine assumes one consumer cluster — swapping the consumer-side + provider scales the same binary out: + * **kcp provider** → one konnector per kcp instance, serving *every* workspace; each + workspace carries its own `Connection`/`Binding` objects and is its own sync domain. + * **fleet provider** (kubeconfig/cluster-inventory) → one konnector serving many + physical consumer clusters, connecting a *set* of consumer clusters to a *set* of + providers. +* The sync domain is always the pair *(consumer logical cluster, `Connection` within + it)* — the core API is unchanged regardless of how many consumer clusters one + konnector serves. Multi-consumer is purely an engine/deployment dimension, never an + API dimension. +* Sync controllers are written once as multi-cluster-aware reconcilers; per-connection + client/cache lifecycle, informer dedup per GVR, and teardown come from the framework + instead of v1's contextstore + dynamic controller spawning. +* The backend already uses multicluster-runtime (`mcmanager` + the kcp apiexport + provider) — v2 aligns consumer and provider sides on one runtime model, and the kcp + flavor falls out of swapping the mcr provider rather than special-casing the engine. + +Alpha ships the single-cluster consumer provider only; the kcp and fleet consumer +providers are the explicit scale-out path, unlocked by the architecture rather than +designed later. + +### Extension point 1: Mapper + +Core ships **identity mapping only**, but the place where v1's isolation logic lived +becomes a narrow Go interface so out-of-tree builds can restore tenancy mapping without +forking the engine: + +```go +// Mapper translates object identity between consumer and provider. +// Core registers exactly one implementation: Identity. +type Mapper interface { + // ToProvider maps a consumer object key to its provider key. + ToProvider(gvr schema.GroupVersionResource, key ObjectKey) (ObjectKey, error) + // ToConsumer is the inverse; must round-trip. + ToConsumer(gvr schema.GroupVersionResource, key ObjectKey) (ObjectKey, error) +} +``` + +Notes: + +* This is a *compile-time* extension (custom konnector build), not a CRD-configurable + one. Keeping it out of the API keeps the API honest: the core CRD never promises + renaming. +* The v1 `Prefixed` behavior is implementable as a Mapper; `Namespaced` (scope + conversion) is intentionally **not** implementable — the interface maps keys, it cannot + change scope. That's the hard line v2 draws. +* Possible second interface later: `Transformer` (mutate object payload before write — + label injection, field stripping). Deliberately deferred. + +### Extension point 2: the handshake + +Everything that today negotiates *which* resources to bind and *how to get credentials* +becomes "anything that can create a Secret, a `Connection`, and bindings": + +| v1 component | v2 home | +|---|---| +| backend HTTP API, OIDC, sessions, SPA/UI | optional `kube-bind-backend` component (own module/repo dir, own release cycle). Its output is exactly the core objects. | +| `kubectl bind` CLI + browser dance | optional `cli/` plugin, talks to the backend, ends by applying the core objects + (optionally) installing the konnector. | +| `APIServiceExportTemplate`, `Collection` | backend-layer CRDs (rich catalog: descriptions, grouping, permission review). Raw discovery is core (`Connection.status.exportedAPIs`); curation is service layer. | +| `APIServiceExportRequest`, `BindableResourcesRequest`, `BindingResourceResponse` | backend-layer wire/handshake types. | +| `APIServiceBindingBundle` ("bind everything") | absorbed into core as `Connection.spec.autoBind: true` (managed `ClusterBinding` mirroring `exportedAPIs`). | +| service-account + kubeconfig minting ([backend/kubernetes/resources/](../../backend/kubernetes/resources/)) | backend layer (it's credential issuance, i.e. auth). | +| kcp integration ([contrib/kcp/](../../contrib/kcp/)) | a backend *flavor*: kcp workspaces give per-consumer isolation for free, which is exactly what identity-mapping core wants. The core already speaks to kcp-like providers natively via `schema.source: OpenAPI` (no CRDs needed); the backend flavor only adds workspace provisioning/handshake. Likely the best-fit v2 deployment. | +| heartbeat / `ClusterBinding` | core keeps a plain `Lease` on the provider; anything richer is an optional provider-side visibility component. | + +GitOps is the degenerate case: a human or pipeline commits the one-apply file (Secret + +`Connection` + bindings), no optional layer at all. + +### What gets deleted outright + +* `Isolation` / `ClusterScopedIsolation` / `informerScope` fields and the + isolation strategy implementations (`prefixed.go`, `namespaced.go`, `none.go`). +* `APIServiceNamespace` and the namespace-lifecycle controller. +* `BoundSchema` (core reads provider CRDs directly; the provider-side copy-of-a-CRD + object disappears). +* `ClusterBinding` from core (possibly resurrected in the backend layer). +* Embedded OIDC, sessions, cookies, SPA from anything called "core". +* The hardcoded claimable-APIs list (`claimable_apis.go`) — replaced by + `relatedResources` limited to `secrets`/`configmaps`, label + named selectors only. + +### Repo & module layout (new group, same repo, `v2/` prefix) + +One rule: **the path tells you the version**. Everything v2 lives under `v2/`; every +path outside it is v1 by definition — frozen on main, maintained on a release branch, +deleted from main at v2 GA. + +``` +kube-bind/ +├── v2/ # ── ALL v2 code lives here ── +│ ├── sdk/ # Go module: type-only API (core.kube-bind.io) +│ │ └── apis/core/v1alpha1/ # Connection, ClusterBinding, Binding +│ ├── konnector/ # Go module: the slim core engine + binary +│ │ ├── cmd/konnector/ +│ │ ├── engine/ # sync, schema sources (CRD/OpenAPI), conflicts +│ │ └── providers/ # mcr consumer-side providers (single, kcp, fleet) +│ ├── backend/ # future optional layer (follow-up proposal) +│ └── cli/ # future optional layer (follow-up proposal) +│ +├── sdk/apis/kubebind/v1alpha2/ # v1 — frozen; types kept on main for consumers +├── pkg/ cmd/ backend/ cli/ web/ # v1 — frozen on main, fixes on release-1.x, +└── contrib/kcp/ # deleted from main at v2 GA +``` + +Rules: + +* **No imports across the boundary, in either direction**: nothing under `v2/` imports + v1 packages, nothing outside `v2/` imports v2 packages. Enforced in CI. Shared code is + copied, not linked — the freedom to diverge is the point of the split. +* `v2/sdk` is a type-only module so consumers (backends, integrations) can depend on the + API without pulling the engine; `v2/konnector` depends on `v2/sdk`, never vice versa. +* Within v2, optional layers (`v2/backend`, `v2/cli`) depend on `v2/sdk` (and at most + the konnector's library surface), never the other way. +* The v2 konnector binary serves **only** `core.kube-bind.io`. v1alpha2 konnector is + maintained on the release branch until deprecation; no dual-stack binary. +* Engine implementation: built on multicluster-runtime (see + [Engine](#engine-built-on-multicluster-runtime)); one shared informer set per provider + connection, no hand-rolled informer plumbing. +* Images follow the same rule: `ghcr.io/kube-bind/konnector:v2.*` is built from + `v2/konnector`; `v1.*` / `v0.*` tags only ever come from the release branch. + +## Migration + +* v1alpha2 and core/v1alpha1 are **parallel universes**: no conversion webhooks, no + in-place upgrade. The semantics (isolation, namespace mapping) don't map. +* Migration tooling = a documented procedure + (maybe) a `kubectl bind migrate` helper + that, for bindings using `None` isolation + identity-compatible layouts, generates the + equivalent v2 objects. Anything using `Prefixed`/`Namespaced` isolation cannot migrate + to core semantics and stays on v1 / moves to a per-consumer provider deployment first. +* v1alpha2 enters maintenance on the day v2 core reaches alpha; removal horizon TBD. + +## Decided + +* **One apply**: a complete e2e binding is a single `kubectl apply -f` of one file + (Secret + `Connection` + bindings); all core objects are order-independent and + level-triggered — missing references are `Pending` conditions, never errors. +* **Naming**: group `core.kube-bind.io`, kinds `Connection`, `ClusterBinding`, `Binding`. + The agent stays **konnector**. +* **Binding shape & scope**: a binding lists one or more CRD names (`spec.apis`) plus + its related resources; cluster-wide vs per-namespace is expressed by kind + (`ClusterBinding`/`Binding`), not by fields. +* **Bind everything**: `Connection.spec.autoBind: true` — konnector maintains a managed + `ClusterBinding` mirroring `status.exportedAPIs`. Replaces `APIServiceBindingBundle`. +* **Discovery**: provider opt-in via `core.kube-bind.io/exported=true` CRD label on + plain Kubernetes; on CRD-less providers (kcp, kcp-like) the logical-cluster boundary + is the export boundary (discovery minus built-ins). Exported APIs surfaced on + `Connection.status.exportedAPIs`. No catalog CRDs in core. +* **Schema delivery**: owned by `Connection` (`source`, `pullPolicy`, `updatePolicy`). + `source: Auto` (default) reads provider CRDs when present, else synthesizes CRDs from + discovery + `/openapi/v3` — so kcp-like, CRD-less providers work in core. + `pullPolicy` defaults to `Bound` — CRDs are installed only when a binding references + them; `All` is the eager opt-in. +* **Conversion webhooks**: CRDs with `strategy: Webhook` are refused in core alpha + (per-API condition + skip); revisit on demand. +* **Sync direction**: fixed — spec consumer→provider, status provider→consumer. A + `syncMode` field name is reserved per-API but not implemented in alpha. +* **Related resources**: `secrets` + `configmaps` only; `labelSelector` + named + selectors only. JSONPath reference-following selectors do not enter core. +* **Conflicts**: `conflictPolicy: Fail | Adopt`; `Adopt` never touches objects carrying + another binding's/consumer's markers — cross-binding collisions are always conflicts, + first-writer-wins. +* **Provider namespaces**: konnector creates missing namespaces iff RBAC allows + (annotated kube-bind-created, cleaned up on unbind); otherwise condition + wait. +* **Heartbeat**: a plain `coordination.k8s.io/Lease` per Connection, maintained by the + konnector in a designated provider namespace. Still zero kube-bind CRDs on the + provider. +* **Mapper**: identity only in-tree; compile-time interface for out-of-tree key mapping; + scope conversion impossible by construction. +* **Topology**: consumer-side pull; the konnector is the **only running component** in + the core — no backend required. Provider needs zero kube-bind CRDs, controllers, or + processes. Provider-side credential lifecycle and dead-consumer GC are service-layer + jobs (keyed off the Lease). +* **Engine**: built on multicluster-runtime as a bridge between two cluster sets — a + custom mcr provider turns each `Connection` into a logical provider cluster, and the + consumer side sits behind an mcr provider too (single-cluster in alpha; kcp provider = + one konnector per kcp instance serving all workspaces; fleet provider = sets of + consumer clusters to sets of providers). The sync domain is always *(consumer logical + cluster, Connection)*; multi-consumer is an engine/deployment dimension, never an API + dimension. Sync controllers are multi-cluster-aware reconcilers, no hand-rolled + informer/contextstore machinery. +* **Repo**: new API group, same repo, `v2/` prefix directory — the path tells you the + version. `v2/sdk` (types) + `v2/konnector` (engine) as separate Go modules; no imports + across the v1/v2 boundary in either direction (CI-enforced); v1 paths frozen on main + and deleted at v2 GA. +* **Dual-stack**: none. v2 konnector serves only `core.kube-bind.io`; the v1alpha2 + konnector is maintained on a release branch until deprecation. +* **Backend/UI/CLI redesign**: separate follow-up proposal; this doc only fixes the + contract boundary. + +## Open questions + +1. **OpenAPI synthesis fidelity.** How lossy is discovery + `/openapi/v3` → CRD in + practice (CEL rules, defaulting edge cases, multi-version with conversion)? Needs a + spike against kcp and a kcp-like system (e.g. kplane) before the `Auto` default is + locked. Mitigation already in the contract: the provider is the enforcing side; + upstream rejections surface as per-object sync conditions. + + Do we need different way to deliver schema as we do now in v1? + +2. **Built-ins filter for `source: OpenAPI`.** Exact exclusion list/heuristic for "minus + built-ins" in `exportedAPIs` (core groups, `*.k8s.io`, kcp's own groups?) — and + whether it should be overridable on the `Connection`. From 28bbc3ed50b584d6e63b453eddb883fea9765657 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Wed, 10 Jun 2026 13:01:02 +0300 Subject: [PATCH 02/18] address offline review --- docs/proposals/v2-extended.md | 34 ++++++++++--- docs/proposals/v2-slim-core.md | 90 +++++++++++++++++++++++++--------- 2 files changed, 92 insertions(+), 32 deletions(-) diff --git a/docs/proposals/v2-extended.md b/docs/proposals/v2-extended.md index 6796ae048..913df826d 100644 --- a/docs/proposals/v2-extended.md +++ b/docs/proposals/v2-extended.md @@ -122,7 +122,12 @@ distribution, which wires its own implementation against the same interface. Kubernetes (workspaces, in the contrib/kcp issuer). * Mint credentials: ServiceAccount + RBAC scoped to exactly the exported APIs (+ declared related resources) within that boundary + kubeconfig. This fixes v1's - cluster-admin-ish `kube-binder` ClusterRole (roadmap #303: reduced footprint). + cluster-admin-ish `kube-binder` ClusterRole (roadmap #303: reduced footprint). On a + plain-Kubernetes provider "scoped to the exported APIs" is an explicit Role enumerating + those resources; on the kcp/CRD-less flavor the tenancy boundary *is* the workspace, so + scoping is the workspace grant itself (everything in it is exported by construction) + rather than a per-resource enumeration — same `Issuer` interface, two scoping mechanisms + matching the core's two schema sources. * **Credential mechanism: long-lived SA token** (v1 behavior, secret-based ServiceAccount token). Trade-off accepted deliberately: zero rotation friction and no konnector-side refresh machinery, at the cost of security posture — and noting @@ -130,8 +135,9 @@ distribution, which wires its own implementation against the same interface. revisitable without API change (the bundle's Secret is replaceable; a bounded-token + reissue mode can be added later behind the same interface). Revocation = delete the `Grant` → issuer deletes the SA/token. -* Records issuance in **`Grant`** (`catalog.kube-bind.io`): "identity X was issued - credentials Y for export Z". The anchor for revocation, audit, and the reaper. +* Records issuance in **`Grant`** (`iam.kube-bind.io` — an issuance/identity record, not + catalog presentation): "identity X was issued credentials Y for export Z". The anchor + for revocation, audit, and the reaper. ### 3. Gateway (HTTP API) @@ -159,6 +165,14 @@ envelope (`{ bundle: [...] }`). There is no other handshake state: no request ob poll, no phases to wait on. `curl` bind + pickup piped to `kubectl apply -f -` is a complete client. +The single-use, short-TTL property applies to the **pickup URL**, not to the credential +inside it: the bundle's Secret carries the long-lived SA token minted by the issuer +(§2), so a bundle that has been picked up once stays valid and re-appliable. That is what +makes `-o yaml > binding.yaml` committed to git a real GitOps artifact — the pickup is +consumed once, the token it delivered keeps working until its `Grant` is revoked. +Re-running `bind` mints a fresh `Grant`/token and a new pickup; it does not invalidate a +previously committed bundle unless that `Grant` is explicitly revoked. + ### 4. Auth (pluggable) * `Authenticator` interface: `Routes()` (mounted under `/api/auth/…`) + @@ -175,7 +189,10 @@ complete client. ### 5. Reaper (provider-side, optional) The core leaves dead-consumer GC explicitly to this layer, keyed off the per-Connection -`Lease` the konnector maintains: +`Lease` the konnector maintains. **This component is blocked on that core primitive:** the +konnector's provider-side `Lease` is specified in the core proposal but is new (no v1 +equivalent), so the reaper ships only once the konnector actually maintains the Lease; +until then dead-consumer GC is manual. * Lease expired beyond TTL → mark the issuance stale → (configurably) revoke credentials, then delete kube-bind-created namespaces and synced objects. @@ -238,15 +255,16 @@ remains separate, providing its own issuer implementation behind the same interf * **Packaging**: one `kube-bind-backend` binary; gateway/issuer/reaper/apply are module flags, boundaries kept as Go packages. -* **Issuance anchor**: `Grant` in `catalog.kube-bind.io` — the typed record of +* **Issuance anchor**: `Grant` in `iam.kube-bind.io` — the typed record of "identity X was issued credentials Y for export Z"; anchor for revocation, audit, - reaper. + reaper. Kept out of `catalog.kube-bind.io` so that group stays purely presentation+defaults. * **Credentials**: long-lived secret-based SA token (v1 behavior) — zero rotation friction accepted over security posture; revocation via `Grant` deletion; bounded tokens addable later behind the same issuer interface without API change. * **Catalog vocabulary**: `Export` + `Collection`. -* **Bundle delivery**: one-time pickup URL, 5-minute TTL, single use; bundle never - stored at rest in the gateway. +* **Bundle delivery**: one-time pickup URL, 5-minute TTL, single use — the TTL/single-use + applies to the *pickup URL*, not the long-lived SA token inside, so a picked-up bundle + stays re-appliable (GitOps-safe). The bundle is never stored at rest in the gateway. * **kcp**: stays a separate distribution (`contrib/kcp`) providing its own issuer implementation; the in-tree backend issuer is plain Kubernetes only. * **UI reach**: browser-apply path **kept** (roadmap #406) — gateway `/api/apply` diff --git a/docs/proposals/v2-slim-core.md b/docs/proposals/v2-slim-core.md index ae51432a4..630ea97d5 100644 --- a/docs/proposals/v2-slim-core.md +++ b/docs/proposals/v2-slim-core.md @@ -206,11 +206,12 @@ spec: matchLabels: mangodb.io/managed: "true" status: - conditions: [] # Connected, Synced, Conflicts + conditions: [] # Connected, Synced, Conflicts, PermissionDenied boundAPIs: # per-API observed state - name: mangodbs.mangodb.io crdHash: sha256:… # applied schema version - conflicts: [] # objects skipped due to foreign ownership + conflictCount: 0 # objects skipped due to foreign ownership; + # per-object detail lives on each object's own condition ``` ```yaml @@ -284,8 +285,17 @@ order-independent and level-triggered**. A binding referencing a not-yet-existin `Connection`, a `Connection` referencing a not-yet-existing Secret, a binding listing a not-yet-exported CRD — none of these are errors, all are `Pending` conditions that resolve when the missing piece arrives. No phase gating, no request/response objects, no -controller that must answer before the user may apply the next thing. (Equivalently: -`kubectl delete -f mangodb-binding.yaml` is the complete, ordered-don't-care unbind.) +controller that must answer before the user may apply the next thing. + +Unbind has one ordering constraint that apply does not. `kubectl delete -f +mangodb-binding.yaml` deletes the `Connection`, bindings, and synced objects in whatever +order the server processes them, but every synced consumer object carries a finalizer +whose release requires the konnector to reach the provider through the *very* `Connection` +being deleted. The konnector therefore drains object finalizers before a referenced +`Connection`/binding is allowed to finalize: one that still has synced objects bound to it +stays in `Terminating` with a `DrainingObjects` condition until those objects' provider +copies are gone, then releases. "Delete the whole file" converges — it does not deadlock — +but the `Connection` is the *last* object to disappear, not the first. ### Schema sources: CRD and OpenAPI @@ -293,9 +303,14 @@ The konnector must install a working CRD on the consumer for every bound API. Tw to obtain it, selected by `Connection.spec.schema.source` (`Auto` probes CRD first, falls back to OpenAPI): -* **`CRD`** — read the CRD from the provider's apiextensions API and apply it (minus - webhook conversion, status). Highest fidelity; requires the provider to *have* CRDs - and the credentials to read them. +* **`CRD`** — read the CRD from the provider's apiextensions API and apply it on the + consumer with mechanical adjustments: never copy provider-side `conversion.strategy: + Webhook` or its caBundle (such CRDs are refused — per-API condition + skip, see the + sync table), add an owner-ref to the `Connection`, and inject UX-only printer + columns/categories. To keep a multi-version CRD installable on the consumer *without* a + conversion webhook, install only the provider's **storage/served version** rather than + every historical version. Highest fidelity; requires the provider to *have* CRDs and + the credentials to read them. * **`OpenAPI`** — for CRD-less providers (kcp and kcp-like systems, aggregated APIs): synthesize a CRD from what every Kubernetes-shaped API server already serves: * **discovery** (`/apis//`) → plural/singular/kind/shortNames, @@ -324,27 +339,39 @@ name on the provider. `status.exportedAPIs` is computed per source: labeled CRDs | Spec | consumer → provider, server-side apply with a dedicated field manager. | | Status | provider → consumer, status subresource update. | | Namespaces | If the consumer namespace doesn't exist on the provider, the konnector creates it (annotated as kube-bind-created; only kube-bind-created namespaces are cleaned up on unbind). | -| Deletion | Finalizer on consumer object (as today); consumer deletion propagates to provider, finalizer released when provider copy is gone. Provider-side deletion of a synced object is treated as drift and re-created (consumer is source of truth for spec). | +| Deletion | Finalizer on consumer object (as today); consumer deletion propagates to provider, finalizer released only when the provider copy is fully gone (a provider-side finalizer holds the consumer object in `Terminating` until it clears). `kube-bind.io/deletion-policy: Orphan` on a consumer object releases the finalizer without deleting the provider copy. Provider-side deletion of a synced object is treated as drift and re-created **unless** the provider copy has a non-zero deletionTimestamp — then the konnector waits for it to finalize rather than racing a re-create. Consumer is source of truth for spec. | | Schema | Per `Connection.spec.schema`: konnector obtains schemas via `source: CRD` (read apiextensions) or `OpenAPI` (synthesize from discovery + `/openapi/v3` — CRD-less providers like kcp), pulls per `pullPolicy: All` or `Bound`, applies on the consumer (owner-ref to the `Connection`). `updatePolicy: Always` keeps following provider schema changes; `Once` pins. CRDs with `strategy: Webhook` are refused (per-API condition + skip). | | Discovery | `Connection.status.exportedAPIs`: labeled CRDs (`source: CRD`) or discovery minus built-ins (`source: OpenAPI`, where the logical-cluster boundary is the export boundary) — core-level discovery with zero provider CRDs. | -| Related resources | Selected Secrets/ConfigMaps sync in the declared direction, same identity rules, scoped like their binding. | +| Related resources | Selected Secrets/ConfigMaps sync in the declared direction, same identity rules, scoped like their binding. They are owned by the **binding** (not by individual instances): an object is synced while it matches the selector and is garbage-collected when it stops matching or the binding is removed. The same ownership markers and `conflictPolicy` apply — a related object already owned by another binding/consumer is a conflict, never silently overwritten. | +| RBAC | The supplied credentials' RBAC *is* the authorization model; partial RBAC is an expected steady state, not a failure. Each forbidden operation surfaces as a typed `PermissionDenied` condition naming the verb+resource refused (per-API on the binding, per-object on the instance); the konnector keeps syncing everything it *is* allowed to and never fails a whole binding because one resource or namespace is out of reach. | | Informers | One shared dynamic informer per GVR per connection. Cluster-wide informers for `ClusterBinding`; namespace-scoped informers where only namespaced `Binding`s exist. | ### Conflict handling (the new hard part) Identity mapping means two writers can legitimately collide. Core rules: -1. Every object the konnector writes carries ownership markers (annotation pair: - binding UID + source cluster UID — same idea as today's - `kube-bind.io/consumer-uid` / `provider-uid` annotations, kept). -2. Before first write to a target object that already exists **without** our markers: - * `conflictPolicy: Fail` (default) — do not touch it. Record it in - `status.boundResources[].conflicts`, set `Conflicts` condition, emit an Event, - keep syncing everything else. - * `conflictPolicy: Adopt` — take ownership (stamp markers, SSA force-apply). -3. Two `Binding`s (or two consumers against one provider target) claiming the same - object: second writer sees foreign markers → always a conflict regardless of policy. - First-writer-wins, deterministically. +1. Every object the konnector writes carries ownership markers: the **binding UID** plus + the **source cluster UID**. The source cluster UID is the identity the `Connection` + pins in its status the first time it resolves its credentials — *not* something read + from a fixed object like the `kube-system` namespace, which does not exist on + CRD-less/logical-cluster providers (kcp). This keeps the marker well-defined under both + schema sources. (Same idea as today's `kube-bind.io/consumer-uid` / `provider-uid` + annotations, kept.) +2. Before first write, the konnector classifies the existing target object three ways: + * **No markers** (a foreign, un-owned object): `conflictPolicy: Fail` (default) does + not touch it; `conflictPolicy: Adopt` stamps markers and SSA force-applies. + * **Our markers, our current binding + object UID**: ours — normal SSA update. + * **Our cluster's markers but a stale/foreign binding-or-object UID**, *or* **another + cluster's markers**: always a conflict, regardless of `conflictPolicy`. `Adopt` + never steals an object another binding/consumer owns; first-writer-wins, + deterministically. +3. Conflicts are recorded **on the conflicting object itself** as a typed condition that + distinguishes the two cases an operator remediates differently — `ForeignObjectExists` + (no markers; rename or switch to `Adopt`) vs. `OwnedByAnother` (claimed by a different + binding/consumer; pick another name). The binding does **not** inline an unbounded list + of object names: its status carries only a `Conflicts` condition and a count + (`boundAPIs[].conflictCount`), plus an Event. The konnector keeps syncing everything + else. 4. Field-level conflicts within an owned object are resolved by SSA with `force=true` for our field manager on our sync direction only (spec fields upstream, status fields downstream) — same as today, but now stated as the contract. @@ -369,7 +396,11 @@ moved into the konnector or out of core: logical clusters (kcp workspaces, a cluster fleet) — see [Engine](#engine-built-on-multicluster-runtime). * It watches `Connection`, `ClusterBinding`, and `Binding` objects, and only the Secrets - that `Connection`s reference (fixes the v1 watch-all-secrets issue). + that `Connection`s reference (fixes the v1 watch-all-secrets issue). A `Connection`'s + `kubeconfigSecretRef` must resolve to the konnector's own designated namespace (default + `kube-bind`): `Connection` is cluster-scoped, so letting it name a Secret in any + namespace would let anyone who can create a `Connection` read any Secret the konnector's + ServiceAccount can. Cross-namespace refs are rejected with `SecretValid=False`. * One `Connection` = one provider client/informer context; all bindings referencing it share that context. * The provider cluster needs: reachable API server + RBAC for the supplied credentials. @@ -555,9 +586,20 @@ Rules: `syncMode` field name is reserved per-API but not implemented in alpha. * **Related resources**: `secrets` + `configmaps` only; `labelSelector` + named selectors only. JSONPath reference-following selectors do not enter core. -* **Conflicts**: `conflictPolicy: Fail | Adopt`; `Adopt` never touches objects carrying - another binding's/consumer's markers — cross-binding collisions are always conflicts, - first-writer-wins. +* **Conflicts**: `conflictPolicy: Fail | Adopt` with three-way classification (no markers + / ours / foreign-or-stale). `Adopt` only takes *un-owned* objects and never steals one + carrying another binding's/consumer's markers — cross-binding collisions are always + conflicts, first-writer-wins. Per-object detail lives on the conflicting object's own + condition (`ForeignObjectExists` vs `OwnedByAnother`); the binding holds only a + `Conflicts` condition + count. The marker's source cluster UID is the + `Connection`-pinned identity, so conflicts stay well-defined on CRD-less providers too. +* **Cluster identity**: each `Connection` pins the resolved provider (and local) cluster + UID in its status on first connect and is immutable thereafter; a Secret later pointing + at a *different* cluster is rejected (`Connected=False`) rather than silently re-homing + synced objects. +* **Deletion**: consumer deletion propagates to the provider and the binding/`Connection` + drains object finalizers before finalizing (`DrainingObjects`); `deletion-policy: + Orphan` opts an object out of provider-side deletion. * **Provider namespaces**: konnector creates missing namespaces iff RBAC allows (annotated kube-bind-created, cleaned up on unbind); otherwise condition + wait. * **Heartbeat**: a plain `coordination.k8s.io/Lease` per Connection, maintained by the From 05dd17b631f0fc1353ccaf5284947310ad2db60c Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Thu, 11 Jun 2026 17:38:50 +0300 Subject: [PATCH 03/18] V2 init --- docs/proposals/v2-slim-core.md | 26 +- v2/Makefile | 50 ++ v2/README.md | 75 +++ v2/hack/demo.sh | 72 +++ v2/konnector/cmd/konnector/main.go | 132 ++++++ v2/konnector/config/samples/binding.yaml | 23 + .../config/samples/provider-widget-crd.yaml | 42 ++ v2/konnector/config/samples/widget.yaml | 10 + v2/konnector/engine/binding/reconciler.go | 322 +++++++++++++ v2/konnector/engine/connection/reconciler.go | 210 +++++++++ .../engine/provider/connection_provider.go | 217 +++++++++ v2/konnector/engine/remote/remote.go | 89 ++++ v2/konnector/engine/sync/crd_controller.go | 317 +++++++++++++ v2/konnector/engine/sync/resolve.go | 89 ++++ v2/konnector/engine/sync/syncer.go | 335 ++++++++++++++ v2/konnector/go.mod | 72 +++ v2/konnector/go.sum | 199 ++++++++ v2/konnector/test/e2e/framework/framework.go | 287 ++++++++++++ v2/konnector/test/e2e/sync_test.go | 256 +++++++++++ v2/sdk/apis/core/v1alpha1/binding_types.go | 53 +++ .../core/v1alpha1/clusterbinding_types.go | 67 +++ v2/sdk/apis/core/v1alpha1/conditions.go | 70 +++ v2/sdk/apis/core/v1alpha1/connection_types.go | 168 +++++++ v2/sdk/apis/core/v1alpha1/doc.go | 23 + .../apis/core/v1alpha1/groupversion_info.go | 54 +++ v2/sdk/apis/core/v1alpha1/helpers.go | 62 +++ v2/sdk/apis/core/v1alpha1/labels.go | 41 ++ v2/sdk/apis/core/v1alpha1/shared_types.go | 209 +++++++++ .../core/v1alpha1/zz_generated.deepcopy.go | 428 ++++++++++++++++++ .../crd/core.kube-bind.io_bindings.yaml | 297 ++++++++++++ .../core.kube-bind.io_clusterbindings.yaml | 295 ++++++++++++ .../crd/core.kube-bind.io_connections.yaml | 253 +++++++++++ v2/sdk/go.mod | 30 ++ v2/sdk/go.sum | 105 +++++ 34 files changed, 4968 insertions(+), 10 deletions(-) create mode 100644 v2/Makefile create mode 100644 v2/README.md create mode 100755 v2/hack/demo.sh create mode 100644 v2/konnector/cmd/konnector/main.go create mode 100644 v2/konnector/config/samples/binding.yaml create mode 100644 v2/konnector/config/samples/provider-widget-crd.yaml create mode 100644 v2/konnector/config/samples/widget.yaml create mode 100644 v2/konnector/engine/binding/reconciler.go create mode 100644 v2/konnector/engine/connection/reconciler.go create mode 100644 v2/konnector/engine/provider/connection_provider.go create mode 100644 v2/konnector/engine/remote/remote.go create mode 100644 v2/konnector/engine/sync/crd_controller.go create mode 100644 v2/konnector/engine/sync/resolve.go create mode 100644 v2/konnector/engine/sync/syncer.go create mode 100644 v2/konnector/go.mod create mode 100644 v2/konnector/go.sum create mode 100644 v2/konnector/test/e2e/framework/framework.go create mode 100644 v2/konnector/test/e2e/sync_test.go create mode 100644 v2/sdk/apis/core/v1alpha1/binding_types.go create mode 100644 v2/sdk/apis/core/v1alpha1/clusterbinding_types.go create mode 100644 v2/sdk/apis/core/v1alpha1/conditions.go create mode 100644 v2/sdk/apis/core/v1alpha1/connection_types.go create mode 100644 v2/sdk/apis/core/v1alpha1/doc.go create mode 100644 v2/sdk/apis/core/v1alpha1/groupversion_info.go create mode 100644 v2/sdk/apis/core/v1alpha1/helpers.go create mode 100644 v2/sdk/apis/core/v1alpha1/labels.go create mode 100644 v2/sdk/apis/core/v1alpha1/shared_types.go create mode 100644 v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go create mode 100644 v2/sdk/config/crd/core.kube-bind.io_bindings.yaml create mode 100644 v2/sdk/config/crd/core.kube-bind.io_clusterbindings.yaml create mode 100644 v2/sdk/config/crd/core.kube-bind.io_connections.yaml create mode 100644 v2/sdk/go.mod create mode 100644 v2/sdk/go.sum diff --git a/docs/proposals/v2-slim-core.md b/docs/proposals/v2-slim-core.md index 630ea97d5..60c88bb23 100644 --- a/docs/proposals/v2-slim-core.md +++ b/docs/proposals/v2-slim-core.md @@ -365,13 +365,17 @@ Identity mapping means two writers can legitimately collide. Core rules: cluster's markers**: always a conflict, regardless of `conflictPolicy`. `Adopt` never steals an object another binding/consumer owns; first-writer-wins, deterministically. -3. Conflicts are recorded **on the conflicting object itself** as a typed condition that - distinguishes the two cases an operator remediates differently — `ForeignObjectExists` - (no markers; rename or switch to `Adopt`) vs. `OwnedByAnother` (claimed by a different - binding/consumer; pick another name). The binding does **not** inline an unbounded list - of object names: its status carries only a `Conflicts` condition and a count - (`boundAPIs[].conflictCount`), plus an Event. The konnector keeps syncing everything - else. +3. Conflicts are surfaced by **two authoritative signals**: a Kubernetes **Event** on the + consumer object, and a **count + `Conflicts` condition on the binding** + (`boundAPIs[].conflictCount`) — never an unbounded list of object names. Both carry the + reason that tells an operator how to remediate: `ForeignObjectExists` (no markers; + rename or switch to `Adopt`) vs. `OwnedByAnother` (claimed by a different + binding/consumer; pick another name). The konnector *also* writes that reason as a + condition on the conflicting object itself, but only **best-effort**: a structural CRD + whose `status` schema has no `conditions` field will have it pruned by the API server + (confirmed in the POC against a status-only CRD). So the Event and the binding count are + the contract; the per-object condition is a convenience present only when the synced + type defines `status.conditions`. The konnector keeps syncing everything else. 4. Field-level conflicts within an owned object are resolved by SSA with `force=true` for our field manager on our sync direction only (spec fields upstream, status fields downstream) — same as today, but now stated as the contract. @@ -589,9 +593,11 @@ Rules: * **Conflicts**: `conflictPolicy: Fail | Adopt` with three-way classification (no markers / ours / foreign-or-stale). `Adopt` only takes *un-owned* objects and never steals one carrying another binding's/consumer's markers — cross-binding collisions are always - conflicts, first-writer-wins. Per-object detail lives on the conflicting object's own - condition (`ForeignObjectExists` vs `OwnedByAnother`); the binding holds only a - `Conflicts` condition + count. The marker's source cluster UID is the + conflicts, first-writer-wins. Conflicts are surfaced authoritatively by a Kubernetes + **Event** on the consumer object plus a **`Conflicts` condition + `conflictCount`** on + the binding (reasons `ForeignObjectExists` vs `OwnedByAnother`); a per-object condition + is written best-effort only (the API server prunes it on CRDs whose `status` schema has + no `conditions` field — confirmed in the POC). The marker's source cluster UID is the `Connection`-pinned identity, so conflicts stay well-defined on CRD-less providers too. * **Cluster identity**: each `Connection` pins the resolved provider (and local) cluster UID in its status on first connect and is immutable thereafter; a Secret later pointing diff --git a/v2/Makefile b/v2/Makefile new file mode 100644 index 000000000..50587acb7 --- /dev/null +++ b/v2/Makefile @@ -0,0 +1,50 @@ +# Copyright 2026 The Kube Bind Authors. Licensed under the Apache License 2.0. +# +# v2 slim-core. Run from the v2/ directory. + +CONTROLLER_GEN ?= go run sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.2 + +.PHONY: all +all: codegen build + +.PHONY: build +build: + cd konnector && go build ./... + cd sdk && go build ./... + +.PHONY: konnector +konnector: + cd konnector && go build -o bin/konnector ./cmd/konnector + +.PHONY: codegen +codegen: + cd sdk && $(CONTROLLER_GEN) object paths=./apis/... + cd sdk && $(CONTROLLER_GEN) crd paths=./apis/... output:crd:dir=./config/crd + +ENVTEST_K8S_VERSION ?= 1.34.1 +SETUP_ENVTEST ?= go run sigs.k8s.io/controller-runtime/tools/setup-envtest@release-0.21 + +.PHONY: test +test: + cd konnector && go test ./... + cd sdk && go test ./... + +# Run the envtest-based e2e (two in-process API servers + the engine). +.PHONY: test-e2e +test-e2e: + cd konnector && KUBEBUILDER_ASSETS="$$($(SETUP_ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" \ + go test ./test/e2e/... -count=1 -timeout 600s + +.PHONY: vet +vet: + cd konnector && go vet ./... + cd sdk && go vet ./... + +.PHONY: tidy +tidy: + cd konnector && GOWORK=off go mod tidy + cd sdk && GOWORK=off go mod tidy + +.PHONY: demo +demo: + ./hack/demo.sh diff --git a/v2/README.md b/v2/README.md new file mode 100644 index 000000000..563f2b85c --- /dev/null +++ b/v2/README.md @@ -0,0 +1,75 @@ +# kube-bind v2 — slim core (POC) + +This directory is the v2 "slim core" implementation. See +[../docs/proposals/v2-slim-core.md](../docs/proposals/v2-slim-core.md) for the design. + +Everything v2 lives under `v2/`; nothing here imports v1 packages and nothing +outside imports v2 (the path tells you the version). + +## Layout + +``` +v2/ +├── go.work # ties the two modules for local dev +├── sdk/ # module: type-only API (core.kube-bind.io) +│ └── apis/core/v1alpha1/ # Connection, ClusterBinding, Binding +│ └── config/crd/ # generated CRD manifests +├── konnector/ # module: the slim sync engine + binary +│ ├── cmd/konnector/ # main: local mgr + mcmanager + reconcilers +│ └── engine/ +│ ├── provider/ # mcr provider: Connection -> engaged cluster +│ ├── connection/ # resolve secret, pin identity, discover exports +│ ├── binding/ # validate + pull CRDs (schema.source: CRD) +│ ├── sync/ # per-GVR spec-up / status-down + conflicts +│ └── remote/ # kubeconfig + cluster identity helpers +└── hack/demo.sh # two-kind-cluster end-to-end demo +``` + +## What works today (POC milestone: E2E single-API sync) + +- `Connection` resolves a kubeconfig Secret, pins provider/consumer cluster + identity, and discovers label-gated exported CRDs into `status.exportedAPIs`. +- `ClusterBinding` / `Binding` validate the connection, pull the listed CRDs + (`schema.source: CRD`, single served version, no conversion webhook) onto the + consumer, and report `Ready` + `boundAPIs`. +- A dynamic per-GVR syncer copies instance **spec up** (server-side apply with + ownership markers + a finalizer) and **status down**, with `conflictPolicy: + Fail` surfaced on the object's own condition. + +Known POC simplifications (tracked against the proposal): status sync polls +(no provider watch yet); the sync path uses direct provider clients rather than +the mcr-engaged cluster cache; `schema.source: OpenAPI`, `Adopt`, related +resources, autoBind and the provider `Lease` are not implemented yet. + +## Build + +```sh +cd v2/konnector && go build ./... # workspace mode (go.work) +``` + +## Codegen (after editing types) + +```sh +cd v2/sdk +go run sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.2 object paths=./apis/... +go run sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.2 crd paths=./apis/... output:crd:dir=./config/crd +``` + +## Tests + +End-to-end test ([konnector/test/e2e](konnector/test/e2e)) runs two in-process +**envtest** API servers (provider + consumer) with the real engine reconcilers, and +mirrors the v1 happy-case step pattern: Connection Ready + discovery → ClusterBinding +Ready + CRD pull → spec up → status down → spec update → conflict (foreign object not +overwritten) → deletion (incl. the foreign-object guard). + +```sh +make test-e2e # downloads envtest assets and runs the suite +``` + +## Run the demo (two kind clusters) + +```sh +./hack/demo.sh # creates two kind clusters and wires the bundle +# then follow the printed instructions to run the konnector and sync a Widget +``` diff --git a/v2/hack/demo.sh b/v2/hack/demo.sh new file mode 100755 index 000000000..0353ca735 --- /dev/null +++ b/v2/hack/demo.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# Copyright 2026 The Kube Bind Authors. Licensed under the Apache License 2.0. +# +# Stands up two kind clusters (provider + consumer) and wires the v2 slim-core +# demo: a provider exports the Widget CRD, the consumer binds it, and the +# konnector syncs instances. Run the konnector separately (printed at the end). +set -euo pipefail + +PROVIDER=kube-bind-provider +CONSUMER=kube-bind-consumer +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +V2="$(cd "${HERE}/.." && pwd)" +SAMPLES="${V2}/konnector/config/samples" +PROVIDER_KC=/tmp/${PROVIDER}.kubeconfig +CONSUMER_KC=/tmp/${CONSUMER}.kubeconfig + +echo "==> Creating kind clusters" +kind create cluster --name "${PROVIDER}" || true +kind create cluster --name "${CONSUMER}" || true + +# Internal kubeconfig so the konnector (running on the host) can reach the +# provider API server via its host-mapped port. +kind get kubeconfig --name "${PROVIDER}" > "${PROVIDER_KC}" +kind get kubeconfig --name "${CONSUMER}" > "${CONSUMER_KC}" + +kp() { kubectl --kubeconfig "${PROVIDER_KC}" "$@"; } +kc() { kubectl --kubeconfig "${CONSUMER_KC}" "$@"; } + +echo "==> Provider: install + export the Widget CRD" +kp apply -f "${SAMPLES}/provider-widget-crd.yaml" + +echo "==> Consumer: install core CRDs and the kube-bind namespace" +kc apply -f "${V2}/sdk/config/crd" +kc create namespace kube-bind --dry-run=client -o yaml | kc apply -f - + +echo "==> Consumer: store the provider kubeconfig as a Secret" +kc -n kube-bind delete secret demo-provider-kubeconfig --ignore-not-found +kc -n kube-bind create secret generic demo-provider-kubeconfig \ + --from-file=kubeconfig="${PROVIDER_KC}" + +echo "==> Consumer: the one-apply bundle (Connection + ClusterBinding)" +kc apply -f "${SAMPLES}/binding.yaml" + +cat < provider clusters. + mcMgr, err := mcmanager.New(cfg, connProvider, mcmanager.Options{ + Scheme: scheme, + Metrics: metricsserver.Options{BindAddress: "0"}, + }) + if err != nil { + return err + } + + // Consumer-side reconcilers (run on the local manager). + if err := (&connection.Reconciler{}).SetupWithManager(localMgr); err != nil { + return err + } + if err := (&binding.ClusterReconciler{}).SetupWithManager(localMgr); err != nil { + return err + } + if err := (&binding.NamespacedReconciler{}).SetupWithManager(localMgr); err != nil { + return err + } + if err := syncengine.SetupWithManager(localMgr); err != nil { + return err + } + + log.Info("starting konnector") + g, ctx := errgroup.WithContext(ctx) + g.Go(func() error { return ignoreCanceled(localMgr.Start(ctx)) }) + g.Go(func() error { return ignoreCanceled(connProvider.Run(ctx, mcMgr)) }) + g.Go(func() error { return ignoreCanceled(mcMgr.Start(ctx)) }) + return g.Wait() +} + +func ignoreCanceled(err error) error { + if errors.Is(err, context.Canceled) { + return nil + } + return err +} + +func utilRuntimeMust(err error) { + if err != nil { + panic(err) + } +} diff --git a/v2/konnector/config/samples/binding.yaml b/v2/konnector/config/samples/binding.yaml new file mode 100644 index 000000000..008e85afc --- /dev/null +++ b/v2/konnector/config/samples/binding.yaml @@ -0,0 +1,23 @@ +# The "one apply" bundle. Apply this on the CONSUMER cluster. +# (The kubeconfig Secret is created separately by the demo script.) +apiVersion: core.kube-bind.io/v1alpha1 +kind: Connection +metadata: + name: demo-provider +spec: + kubeconfigSecretRef: + namespace: kube-bind + name: demo-provider-kubeconfig + key: kubeconfig + schema: + source: CRD +--- +apiVersion: core.kube-bind.io/v1alpha1 +kind: ClusterBinding +metadata: + name: widgets +spec: + connectionRef: + name: demo-provider + apis: + - name: widgets.example.org diff --git a/v2/konnector/config/samples/provider-widget-crd.yaml b/v2/konnector/config/samples/provider-widget-crd.yaml new file mode 100644 index 000000000..78071ec07 --- /dev/null +++ b/v2/konnector/config/samples/provider-widget-crd.yaml @@ -0,0 +1,42 @@ +# A sample provider CRD, labeled as exported. Apply this on the PROVIDER cluster. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.org + labels: + core.kube-bind.io/exported: "true" +spec: + group: example.org + names: + plural: widgets + singular: widget + kind: Widget + listKind: WidgetList + scope: Namespaced + versions: + - name: v1 + served: true + storage: true + subresources: + status: {} + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + size: + type: string + status: + type: object + properties: + phase: + type: string + additionalPrinterColumns: + - name: Size + type: string + jsonPath: .spec.size + - name: Phase + type: string + jsonPath: .status.phase diff --git a/v2/konnector/config/samples/widget.yaml b/v2/konnector/config/samples/widget.yaml new file mode 100644 index 000000000..21fd4bc93 --- /dev/null +++ b/v2/konnector/config/samples/widget.yaml @@ -0,0 +1,10 @@ +# A consumer-side instance. Apply on the CONSUMER cluster after the binding is +# Ready (the Widget CRD will have been pulled from the provider). The konnector +# syncs this object's spec up to the provider; provider status flows back here. +apiVersion: example.org/v1 +kind: Widget +metadata: + name: my-widget + namespace: default +spec: + size: large diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go new file mode 100644 index 000000000..591dde44b --- /dev/null +++ b/v2/konnector/engine/binding/reconciler.go @@ -0,0 +1,322 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package binding reconciles ClusterBinding and Binding objects: it validates +// the referenced Connection, pulls the CRDs for the listed APIs onto the +// consumer (schema.source: CRD), and reports per-API state. +package binding + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "fmt" + "sort" + "strings" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + + "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// base holds the shared dependencies and logic for both binding kinds. +type base struct { + client client.Client + scheme *runtime.Scheme + newClient func(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) +} + +func (b *base) providerClient(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) { + if b.newClient != nil { + return b.newClient(ctx, conn) + } + return remote.ProviderClient(ctx, b.client, conn, b.scheme) +} + +// reconcileAccessor drives a binding (cluster or namespaced) toward Ready. +func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAccessor) error { + spec := obj.BindingSpecP() + status := obj.BindingStatusP() + + // Resolve the Connection. + conn := &corev1alpha1.Connection{} + if err := b.client.Get(ctx, client.ObjectKey{Name: spec.ConnectionRef.Name}, conn); err != nil { + if apierrors.IsNotFound(err) { + setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonPending, + fmt.Sprintf("Connection %q not found yet", spec.ConnectionRef.Name)) + return nil + } + return fmt.Errorf("getting Connection %q: %w", spec.ConnectionRef.Name, err) + } + if !apimeta.IsStatusConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionReady) { + setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonPending, + fmt.Sprintf("Connection %q not ready yet", spec.ConnectionRef.Name)) + return nil + } + + providerClient, err := b.providerClient(ctx, conn) + if err != nil { + return fmt.Errorf("building provider client: %w", err) + } + + var ( + boundAPIs []corev1alpha1.BoundAPI + notExported []string + ) + for _, api := range spec.APIs { + exported, ok := conn.Status.ExportsAPI(api.Name) + if !ok { + notExported = append(notExported, api.Name) + continue + } + hash, err := b.pullCRD(ctx, providerClient, api.Name, conn.Name) + if err != nil { + return fmt.Errorf("pulling CRD %q: %w", api.Name, err) + } + boundAPIs = append(boundAPIs, corev1alpha1.BoundAPI{Name: exported.Name, CRDHash: hash}) + } + sort.Slice(boundAPIs, func(i, j int) bool { return boundAPIs[i].Name < boundAPIs[j].Name }) + status.BoundAPIs = boundAPIs + + if len(notExported) > 0 { + setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonAPINotExported, + fmt.Sprintf("APIs not exported by the provider yet: %s", strings.Join(notExported, ", "))) + return nil + } + + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionSynced, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "all APIs installed")) + setReady(status, obj, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "binding ready") + return nil +} + +// pullCRD reads the provider CRD by name and installs it on the consumer +// (create-if-absent; POC pulls once). It returns a content hash for status. +func (b *base) pullCRD(ctx context.Context, providerClient client.Client, crdName, connName string) (string, error) { + var remoteCRD apiextensionsv1.CustomResourceDefinition + if err := providerClient.Get(ctx, client.ObjectKey{Name: crdName}, &remoteCRD); err != nil { + return "", fmt.Errorf("reading provider CRD: %w", err) + } + + consumerCRD := crdForConsumer(&remoteCRD, connName) + hash := crdHash(consumerCRD) + + var existing apiextensionsv1.CustomResourceDefinition + err := b.client.Get(ctx, client.ObjectKey{Name: crdName}, &existing) + switch { + case apierrors.IsNotFound(err): + if err := b.client.Create(ctx, consumerCRD); err != nil { + return "", fmt.Errorf("creating consumer CRD: %w", err) + } + case err != nil: + return "", fmt.Errorf("getting consumer CRD: %w", err) + default: + // Already present. updatePolicy: Once for the POC — do not update. + } + return hash, nil +} + +// crdForConsumer strips the provider CRD down to something installable on the +// consumer without a conversion webhook: single served/storage version, +// conversion forced to None, no webhook/caBundle, no ownerRefs/status. +func crdForConsumer(in *apiextensionsv1.CustomResourceDefinition, connName string) *apiextensionsv1.CustomResourceDefinition { + out := in.DeepCopy() + out.ResourceVersion = "" + out.UID = "" + out.ManagedFields = nil + out.OwnerReferences = nil + out.Status = apiextensionsv1.CustomResourceDefinitionStatus{} + + // Keep only the storage version (fallback: first served), drop the rest, so + // no conversion webhook is needed on the consumer. + var keep *apiextensionsv1.CustomResourceDefinitionVersion + for i := range out.Spec.Versions { + v := &out.Spec.Versions[i] + if v.Storage { + keep = v + break + } + if keep == nil && v.Served { + keep = v + } + } + if keep != nil { + k := *keep + k.Storage = true + k.Served = true + out.Spec.Versions = []apiextensionsv1.CustomResourceDefinitionVersion{k} + } + out.Spec.Conversion = &apiextensionsv1.CustomResourceConversion{Strategy: apiextensionsv1.NoneConverter} + + if out.Labels == nil { + out.Labels = map[string]string{} + } + out.Labels[corev1alpha1.LabelManaged] = "true" + if out.Annotations == nil { + out.Annotations = map[string]string{} + } + out.Annotations[annotationConnection] = connName + return out +} + +const annotationConnection = "core.kube-bind.io/connection" + +func crdHash(crd *apiextensionsv1.CustomResourceDefinition) string { + h := sha256.New() + for _, v := range crd.Spec.Versions { + _, _ = h.Write([]byte(v.Name)) + } + return "sha256:" + hex.EncodeToString(h.Sum(nil))[:16] +} + +func condition(obj client.Object, t string, s metav1.ConditionStatus, reason, msg string) metav1.Condition { + return metav1.Condition{ + Type: t, + Status: s, + Reason: reason, + Message: msg, + ObservedGeneration: obj.GetGeneration(), + } +} + +func setReady(status *corev1alpha1.BindingStatus, obj client.Object, s metav1.ConditionStatus, reason, msg string) { + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionReady, s, reason, msg)) +} + +// ----- ClusterBinding ----- + +// ClusterReconciler reconciles ClusterBinding objects. +type ClusterReconciler struct{ base } + +// SetupWithManager registers the ClusterBinding reconciler. It also watches +// Connection so bindings re-reconcile when their connection becomes Ready or +// its exported APIs change (level-triggered). +func (r *ClusterReconciler) SetupWithManager(mgr ctrl.Manager) error { + r.base.client = mgr.GetClient() + r.base.scheme = mgr.GetScheme() + return ctrl.NewControllerManagedBy(mgr). + For(&corev1alpha1.ClusterBinding{}). + Watches(&corev1alpha1.Connection{}, handler.EnqueueRequestsFromMapFunc(r.bindingsForConnection)). + Named("clusterbinding"). + Complete(r) +} + +func (r *ClusterReconciler) bindingsForConnection(ctx context.Context, conn client.Object) []reconcile.Request { + var list corev1alpha1.ClusterBindingList + if err := r.base.client.List(ctx, &list); err != nil { + return nil + } + var reqs []reconcile.Request + for i := range list.Items { + if list.Items[i].Spec.ConnectionRef.Name == conn.GetName() { + reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{Name: list.Items[i].Name}}) + } + } + return reqs +} + +// Reconcile reconciles a ClusterBinding. +func (r *ClusterReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + cb := &corev1alpha1.ClusterBinding{} + if err := r.base.client.Get(ctx, req.NamespacedName, cb); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + orig := cb.DeepCopy() + rerr := r.base.reconcileAccessor(ctx, cb) + if !equalBindingStatus(&orig.Status, &cb.Status) { + if err := r.base.client.Status().Update(ctx, cb); err != nil { + return ctrl.Result{}, err + } + } + return ctrl.Result{}, rerr +} + +// ----- Binding (namespaced) ----- + +// NamespacedReconciler reconciles Binding objects. +type NamespacedReconciler struct{ base } + +// SetupWithManager registers the Binding reconciler. It also watches Connection +// so bindings re-reconcile when their connection becomes Ready (level-triggered). +func (r *NamespacedReconciler) SetupWithManager(mgr ctrl.Manager) error { + r.base.client = mgr.GetClient() + r.base.scheme = mgr.GetScheme() + return ctrl.NewControllerManagedBy(mgr). + For(&corev1alpha1.Binding{}). + Watches(&corev1alpha1.Connection{}, handler.EnqueueRequestsFromMapFunc(r.bindingsForConnection)). + Named("binding"). + Complete(r) +} + +func (r *NamespacedReconciler) bindingsForConnection(ctx context.Context, conn client.Object) []reconcile.Request { + var list corev1alpha1.BindingList + if err := r.base.client.List(ctx, &list); err != nil { + return nil + } + var reqs []reconcile.Request + for i := range list.Items { + if list.Items[i].Spec.ConnectionRef.Name == conn.GetName() { + reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{Namespace: list.Items[i].Namespace, Name: list.Items[i].Name}}) + } + } + return reqs +} + +// Reconcile reconciles a Binding. +func (r *NamespacedReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + b := &corev1alpha1.Binding{} + if err := r.base.client.Get(ctx, req.NamespacedName, b); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + orig := b.DeepCopy() + rerr := r.base.reconcileAccessor(ctx, b) + if !equalBindingStatus(&orig.Status, &b.Status) { + if err := r.base.client.Status().Update(ctx, b); err != nil { + return ctrl.Result{}, err + } + } + return ctrl.Result{}, rerr +} + +func equalBindingStatus(a, b *corev1alpha1.BindingStatus) bool { + if len(a.BoundAPIs) != len(b.BoundAPIs) { + return false + } + for i := range a.BoundAPIs { + if a.BoundAPIs[i] != b.BoundAPIs[i] { + return false + } + } + if len(a.Conditions) != len(b.Conditions) { + return false + } + for i := range a.Conditions { + ac, bc := a.Conditions[i], b.Conditions[i] + if ac.Type != bc.Type || ac.Status != bc.Status || ac.Reason != bc.Reason || ac.Message != bc.Message { + return false + } + } + return true +} diff --git a/v2/konnector/engine/connection/reconciler.go b/v2/konnector/engine/connection/reconciler.go new file mode 100644 index 000000000..0392d9e50 --- /dev/null +++ b/v2/konnector/engine/connection/reconciler.go @@ -0,0 +1,210 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package connection reconciles Connection objects: it resolves the provider +// credentials, pins the provider/consumer cluster identity, and discovers the +// APIs the provider exports to those credentials. +package connection + +import ( + "context" + "fmt" + "sort" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// Reconciler reconciles Connection objects on the consumer cluster. +type Reconciler struct { + // Client is the consumer-cluster client. + Client client.Client + // Scheme is used to build the provider-cluster client. + Scheme *runtime.Scheme + // NewProviderClient builds a direct client for the provider cluster from a + // Connection. Overridable in tests; defaults to a fresh client from the + // resolved kubeconfig. + NewProviderClient func(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) +} + +// SetupWithManager registers the reconciler with the consumer manager. +func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error { + r.Client = mgr.GetClient() + r.Scheme = mgr.GetScheme() + if r.NewProviderClient == nil { + r.NewProviderClient = r.defaultProviderClient + } + return ctrl.NewControllerManagedBy(mgr). + For(&corev1alpha1.Connection{}). + Named("connection"). + Complete(r) +} + +func (r *Reconciler) defaultProviderClient(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) { + cfg, err := remote.RestConfigFromConnection(ctx, r.Client, conn) + if err != nil { + return nil, err + } + return client.New(cfg, client.Options{Scheme: r.Scheme}) +} + +// Reconcile drives a Connection toward Ready: SecretValid, Connected (identity +// pinned), and exportedAPIs discovered. +func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := ctrl.LoggerFrom(ctx) + + conn := &corev1alpha1.Connection{} + if err := r.Client.Get(ctx, req.NamespacedName, conn); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + orig := conn.DeepCopy() + reconcileErr := r.reconcile(ctx, conn) + + if !apiequalStatus(orig, conn) { + if err := r.Client.Status().Update(ctx, conn); err != nil { + log.Error(err, "updating Connection status") + return ctrl.Result{}, err + } + } + return ctrl.Result{}, reconcileErr +} + +func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connection) error { + // 1. Build the provider client (validates the secret + kubeconfig). + providerClient, err := r.NewProviderClient(ctx, conn) + if err != nil { + setCondition(conn, corev1alpha1.ConditionSecretValid, metav1.ConditionFalse, corev1alpha1.ReasonSecretNotFound, err.Error()) + setNotReady(conn, corev1alpha1.ReasonSecretNotFound, "kubeconfig secret not usable yet") + return nil // level-triggered: resolves when the secret arrives + } + setCondition(conn, corev1alpha1.ConditionSecretValid, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "kubeconfig secret resolved") + + // 2. Pin and verify cluster identity. + remoteUID, err := remote.ClusterUID(ctx, providerClient) + if err != nil { + setCondition(conn, corev1alpha1.ConditionConnected, metav1.ConditionFalse, corev1alpha1.ReasonPending, err.Error()) + setNotReady(conn, corev1alpha1.ReasonPending, "cannot reach provider cluster") + return nil + } + localUID, err := remote.ClusterUID(ctx, r.Client) + if err != nil { + return fmt.Errorf("determining local cluster identity: %w", err) + } + if conn.Status.RemoteClusterUID != "" && conn.Status.RemoteClusterUID != remoteUID { + setCondition(conn, corev1alpha1.ConditionConnected, metav1.ConditionFalse, corev1alpha1.ReasonClusterIdentityChanged, + fmt.Sprintf("kubeconfig now points at cluster %s, but %s was pinned", remoteUID, conn.Status.RemoteClusterUID)) + setNotReady(conn, corev1alpha1.ReasonClusterIdentityChanged, "provider cluster identity changed") + return nil + } + conn.Status.RemoteClusterUID = remoteUID + if conn.Status.LocalClusterUID == "" { + conn.Status.LocalClusterUID = localUID + } + setCondition(conn, corev1alpha1.ConditionConnected, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "connected to provider") + + // 3. Discover exported APIs (schema.source: CRD — label-gated). + exported, err := discoverExportedCRDs(ctx, providerClient) + if err != nil { + return fmt.Errorf("discovering exported APIs: %w", err) + } + conn.Status.ExportedAPIs = exported + + setCondition(conn, corev1alpha1.ConditionReady, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "connection ready") + return nil +} + +// discoverExportedCRDs lists provider CRDs carrying the exported label and maps +// them to ExportedAPI entries. +func discoverExportedCRDs(ctx context.Context, providerClient client.Client) ([]corev1alpha1.ExportedAPI, error) { + var crds apiextensionsv1.CustomResourceDefinitionList + if err := providerClient.List(ctx, &crds, client.MatchingLabels{corev1alpha1.LabelExported: "true"}); err != nil { + return nil, err + } + + out := make([]corev1alpha1.ExportedAPI, 0, len(crds.Items)) + for i := range crds.Items { + crd := &crds.Items[i] + versions := make([]string, 0, len(crd.Spec.Versions)) + for _, v := range crd.Spec.Versions { + if v.Served { + versions = append(versions, v.Name) + } + } + if len(versions) == 0 { + continue + } + out = append(out, corev1alpha1.ExportedAPI{ + Name: crd.Name, + Group: crd.Spec.Group, + Resource: crd.Spec.Names.Plural, + Scope: crd.Spec.Scope, + Versions: versions, + }) + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return out, nil +} + +func setCondition(conn *corev1alpha1.Connection, condType string, status metav1.ConditionStatus, reason, msg string) { + apimeta.SetStatusCondition(&conn.Status.Conditions, metav1.Condition{ + Type: condType, + Status: status, + Reason: reason, + Message: msg, + ObservedGeneration: conn.Generation, + }) +} + +func setNotReady(conn *corev1alpha1.Connection, reason, msg string) { + setCondition(conn, corev1alpha1.ConditionReady, metav1.ConditionFalse, reason, msg) +} + +func apiequalStatus(a, b *corev1alpha1.Connection) bool { + return apiequal(a.Status, b.Status) +} + +// apiequal compares two ConnectionStatus values ignoring condition timestamps. +func apiequal(a, b corev1alpha1.ConnectionStatus) bool { + if a.RemoteClusterUID != b.RemoteClusterUID || a.LocalClusterUID != b.LocalClusterUID { + return false + } + if len(a.ExportedAPIs) != len(b.ExportedAPIs) { + return false + } + for i := range a.ExportedAPIs { + if a.ExportedAPIs[i].Name != b.ExportedAPIs[i].Name { + return false + } + } + if len(a.Conditions) != len(b.Conditions) { + return false + } + for i := range a.Conditions { + ac, bc := a.Conditions[i], b.Conditions[i] + if ac.Type != bc.Type || ac.Status != bc.Status || ac.Reason != bc.Reason || ac.Message != bc.Message { + return false + } + } + return true +} diff --git a/v2/konnector/engine/provider/connection_provider.go b/v2/konnector/engine/provider/connection_provider.go new file mode 100644 index 000000000..6f22cbea5 --- /dev/null +++ b/v2/konnector/engine/provider/connection_provider.go @@ -0,0 +1,217 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package provider implements the multicluster-runtime provider that turns each +// ready Connection into an engaged provider cluster. This is the consumer<->provider +// bridge from the v2 proposal: one Connection == one logical provider cluster. +package provider + +import ( + "context" + "fmt" + "sync" + "time" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" + "k8s.io/client-go/rest" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/builder" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/cluster" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/manager" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + mcmanager "sigs.k8s.io/multicluster-runtime/pkg/manager" + "sigs.k8s.io/multicluster-runtime/pkg/multicluster" + + "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +var _ multicluster.Provider = &ConnectionProvider{} + +// Options configures the ConnectionProvider. +type Options struct { + // ClusterOptions are passed to the cluster constructor. + ClusterOptions []cluster.Option + // NewCluster builds (but does not start) a cluster from a rest.Config. + NewCluster func(ctx context.Context, cfg *rest.Config, opts ...cluster.Option) (cluster.Cluster, error) +} + +func setDefaults(o *Options) { + if o.NewCluster == nil { + o.NewCluster = func(_ context.Context, cfg *rest.Config, opts ...cluster.Option) (cluster.Cluster, error) { + return cluster.New(cfg, opts...) + } + } +} + +type index struct { + object client.Object + field string + extractValue client.IndexerFunc +} + +// ConnectionProvider discovers provider clusters from Connection objects and +// engages them with the multicluster manager. The cluster key is the +// Connection name. +type ConnectionProvider struct { + opts Options + hcClient client.Client + + mcMgr mcmanager.Manager + + lock sync.Mutex + clusters map[string]cluster.Cluster + cancelFns map[string]context.CancelFunc + indexers []index +} + +// New registers the provider as a controller on the local (consumer) manager, +// watching Connection objects. +func New(localMgr manager.Manager, opts Options) (*ConnectionProvider, error) { + p := &ConnectionProvider{ + opts: opts, + hcClient: localMgr.GetClient(), + clusters: map[string]cluster.Cluster{}, + cancelFns: map[string]context.CancelFunc{}, + } + setDefaults(&p.opts) + + if err := builder.ControllerManagedBy(localMgr). + For(&corev1alpha1.Connection{}). + WithOptions(controller.Options{MaxConcurrentReconciles: 1}). + Named("connection-provider"). + Complete(p); err != nil { + return nil, fmt.Errorf("creating connection-provider controller: %w", err) + } + return p, nil +} + +// Reconcile engages a provider cluster once its Connection is Ready, and +// disengages it on deletion. +func (p *ConnectionProvider) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) { + log := ctrl.LoggerFrom(ctx).WithValues("connection", req.Name) + key := req.Name + + conn := &corev1alpha1.Connection{} + if err := p.hcClient.Get(ctx, req.NamespacedName, conn); err != nil { + if apierrors.IsNotFound(err) { + p.disengage(key) + return reconcile.Result{}, nil + } + return reconcile.Result{}, fmt.Errorf("getting Connection: %w", err) + } + + p.lock.Lock() + defer p.lock.Unlock() + + if p.mcMgr == nil { + return reconcile.Result{RequeueAfter: 2 * time.Second}, nil + } + if _, ok := p.clusters[key]; ok { + return reconcile.Result{}, nil + } + if !isReady(conn) { + log.V(4).Info("Connection not ready yet, not engaging") + return reconcile.Result{}, nil + } + + cfg, err := remote.RestConfigFromConnection(ctx, p.hcClient, conn) + if err != nil { + return reconcile.Result{}, fmt.Errorf("building provider rest config: %w", err) + } + + cl, err := p.opts.NewCluster(ctx, cfg, p.opts.ClusterOptions...) + if err != nil { + return reconcile.Result{}, fmt.Errorf("creating provider cluster: %w", err) + } + for _, idx := range p.indexers { + if err := cl.GetCache().IndexField(ctx, idx.object, idx.field, idx.extractValue); err != nil { + return reconcile.Result{}, fmt.Errorf("indexing field %q: %w", idx.field, err) + } + } + + clusterCtx, cancel := context.WithCancel(ctx) + go func() { + if err := cl.Start(clusterCtx); err != nil { + log.Error(err, "provider cluster stopped") + } + }() + if !cl.GetCache().WaitForCacheSync(clusterCtx) { + cancel() + return reconcile.Result{}, fmt.Errorf("provider cluster cache failed to sync") + } + + p.clusters[key] = cl + p.cancelFns[key] = cancel + + if err := p.mcMgr.Engage(clusterCtx, key, cl); err != nil { + cancel() + delete(p.clusters, key) + delete(p.cancelFns, key) + return reconcile.Result{}, fmt.Errorf("engaging provider cluster: %w", err) + } + log.Info("Engaged provider cluster") + return reconcile.Result{}, nil +} + +func (p *ConnectionProvider) disengage(key string) { + p.lock.Lock() + defer p.lock.Unlock() + if cancel, ok := p.cancelFns[key]; ok { + cancel() + delete(p.cancelFns, key) + } + delete(p.clusters, key) +} + +// Get returns the engaged cluster for the given Connection name. +func (p *ConnectionProvider) Get(_ context.Context, clusterName string) (cluster.Cluster, error) { + p.lock.Lock() + defer p.lock.Unlock() + if cl, ok := p.clusters[clusterName]; ok { + return cl, nil + } + return nil, multicluster.ErrClusterNotFound +} + +// Run records the multicluster manager and blocks until the context is done. +func (p *ConnectionProvider) Run(ctx context.Context, mgr mcmanager.Manager) error { + p.lock.Lock() + p.mcMgr = mgr + p.lock.Unlock() + <-ctx.Done() + return ctx.Err() +} + +// IndexField indexes a field on all engaged and future provider clusters. +func (p *ConnectionProvider) IndexField(ctx context.Context, obj client.Object, field string, extractValue client.IndexerFunc) error { + p.lock.Lock() + defer p.lock.Unlock() + p.indexers = append(p.indexers, index{object: obj, field: field, extractValue: extractValue}) + for name, cl := range p.clusters { + if err := cl.GetCache().IndexField(ctx, obj, field, extractValue); err != nil { + return fmt.Errorf("indexing field %q on cluster %q: %w", field, name, err) + } + } + return nil +} + +func isReady(conn *corev1alpha1.Connection) bool { + return apimeta.IsStatusConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionReady) +} diff --git a/v2/konnector/engine/remote/remote.go b/v2/konnector/engine/remote/remote.go new file mode 100644 index 000000000..16496062f --- /dev/null +++ b/v2/konnector/engine/remote/remote.go @@ -0,0 +1,89 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package remote resolves provider credentials and cluster identity for a +// Connection. +package remote + +import ( + "context" + "fmt" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/clientcmd" + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// ProviderClient builds a direct (uncached) client for the provider cluster a +// Connection points at. Used for one-shot reads/writes (discovery, CRD pull) +// that do not warrant a cache. +func ProviderClient(ctx context.Context, local client.Client, conn *corev1alpha1.Connection, scheme *runtime.Scheme) (client.Client, error) { + cfg, err := RestConfigFromConnection(ctx, local, conn) + if err != nil { + return nil, err + } + return client.New(cfg, client.Options{Scheme: scheme}) +} + +// RestConfigFromConnection reads the Connection's kubeconfig Secret and returns +// a rest.Config for the provider cluster. +func RestConfigFromConnection(ctx context.Context, c client.Client, conn *corev1alpha1.Connection) (*rest.Config, error) { + ref := conn.Spec.KubeconfigSecretRef + key := ref.Key + if key == "" { + key = "kubeconfig" + } + + var secret corev1.Secret + if err := c.Get(ctx, types.NamespacedName{Namespace: ref.Namespace, Name: ref.Name}, &secret); err != nil { + return nil, fmt.Errorf("getting kubeconfig secret %s/%s: %w", ref.Namespace, ref.Name, err) + } + + data, ok := secret.Data[key] + if !ok { + return nil, fmt.Errorf("kubeconfig secret %s/%s has no key %q", ref.Namespace, ref.Name, key) + } + if len(data) == 0 { + return nil, fmt.Errorf("kubeconfig secret %s/%s key %q is empty", ref.Namespace, ref.Name, key) + } + + cfg, err := clientcmd.RESTConfigFromKubeConfig(data) + if err != nil { + return nil, fmt.Errorf("parsing kubeconfig: %w", err) + } + return cfg, nil +} + +// ClusterUID returns a stable identity for the cluster behind c, derived from +// the kube-system namespace UID. This works on plain Kubernetes providers. +// +// TODO(v2): kcp / logical-cluster providers have no kube-system namespace; the +// OpenAPI path will need a different identity source (proposal gap #7). +func ClusterUID(ctx context.Context, c client.Client) (string, error) { + var ns corev1.Namespace + if err := c.Get(ctx, types.NamespacedName{Name: "kube-system"}, &ns); err != nil { + return "", fmt.Errorf("getting kube-system namespace for cluster identity: %w", err) + } + if ns.UID == "" { + return "", fmt.Errorf("kube-system namespace UID is empty") + } + return string(ns.UID), nil +} diff --git a/v2/konnector/engine/sync/crd_controller.go b/v2/konnector/engine/sync/crd_controller.go new file mode 100644 index 000000000..0e216ac8c --- /dev/null +++ b/v2/konnector/engine/sync/crd_controller.go @@ -0,0 +1,317 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package sync implements the dynamic per-GVR instance sync. A CRD controller +// watches the CRDs installed by bindings and starts/stops a spec/status syncer +// for each. +package sync + +import ( + "context" + "fmt" + "sync" + "time" + + "github.com/go-logr/logr" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/util/runtime" + dynamicclient "k8s.io/client-go/dynamic" + "k8s.io/client-go/dynamic/dynamicinformer" + "k8s.io/client-go/dynamic/dynamiclister" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/cache" + "k8s.io/client-go/tools/record" + "k8s.io/client-go/util/workqueue" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/predicate" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +const cleanupTimeout = 10 * time.Second + +// CRDController watches managed CRDs and runs a per-GVR syncer for each. +type CRDController struct { + mgr ctrl.Manager + consumerConfig *rest.Config + consumerClient client.Client + providers *providerCache + recorder record.EventRecorder + resync time.Duration + + lock sync.Mutex + contexts map[string]*syncContext // by CRD name +} + +type syncContext struct { + generation int64 + cancel context.CancelFunc + wg *sync.WaitGroup +} + +// SetupWithManager registers the CRD controller. It uses a direct (uncached) +// client for consumer object reads/writes inside the syncers, built from the +// manager's rest config. +func SetupWithManager(mgr ctrl.Manager) error { + consumerClient, err := client.New(mgr.GetConfig(), client.Options{Scheme: mgr.GetScheme()}) + if err != nil { + return fmt.Errorf("building direct consumer client: %w", err) + } + r := &CRDController{ + mgr: mgr, + consumerConfig: mgr.GetConfig(), + consumerClient: consumerClient, + providers: newProviderCache(consumerClient, mgr.GetScheme()), + recorder: mgr.GetEventRecorderFor("kube-bind-konnector"), + resync: time.Minute, + contexts: map[string]*syncContext{}, + } + return ctrl.NewControllerManagedBy(mgr). + For(&apiextensionsv1.CustomResourceDefinition{}). + WithEventFilter(predicate.NewPredicateFuncs(func(o client.Object) bool { + return o.GetLabels()[corev1alpha1.LabelManaged] == "true" + })). + Named("managed-crds"). + Complete(r) +} + +// Reconcile starts or stops the syncer for a managed CRD. +func (r *CRDController) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) { + crd := &apiextensionsv1.CustomResourceDefinition{} + if err := r.consumerClient.Get(ctx, req.NamespacedName, crd); err != nil { + if apierrors.IsNotFound(err) { + crd = nil + } else { + return reconcile.Result{}, err + } + } + if err := r.reconcile(ctx, req.Name, crd); err != nil { + return reconcile.Result{}, err + } + return reconcile.Result{}, nil +} + +func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiextensionsv1.CustomResourceDefinition) error { + log := ctrl.LoggerFrom(ctx).WithValues("crd", name) + deleted := crd == nil || crd.DeletionTimestamp != nil + var generation int64 + if crd != nil { + generation = crd.Generation + } + + r.lock.Lock() + c, found := r.contexts[name] + if found || deleted { + if !deleted && c.generation >= generation { + r.lock.Unlock() + return nil + } + if found { + log.Info("stopping syncer") + c.cancel() + waitWithTimeout(c.wg, cleanupTimeout) + delete(r.contexts, name) + } + } + r.lock.Unlock() + if deleted { + return nil + } + + version := storageOrServedVersion(crd) + if version == "" { + return fmt.Errorf("CRD %s has no served version", name) + } + gvr := schema.GroupVersionResource{Group: crd.Spec.Group, Version: version, Resource: crd.Spec.Names.Plural} + gvk := schema.GroupVersionKind{Group: crd.Spec.Group, Version: version, Kind: crd.Spec.Names.Kind} + + dyn, err := dynamicclient.NewForConfig(r.consumerConfig) + if err != nil { + return err + } + inf := dynamicinformer.NewFilteredDynamicInformer(dyn, gvr, metav1.NamespaceAll, r.resync, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, nil) + lister := dynamiclister.New(inf.Informer().GetIndexer(), gvr) + + rec := &specReconciler{ + gvr: gvr, + gvk: gvk, + scope: crd.Spec.Scope, + consumerLister: lister, + consumerClient: r.consumerClient, + providers: r.providers, + recorder: r.recorder, + } + + syncLog := ctrl.Log.WithName("sync").WithValues("gvr", gvr.String()) + ctrlr, err := controller.NewTypedUnmanaged(fmt.Sprintf("sync-%s", gvr.GroupResource().String()), controller.TypedOptions[reconcile.Request]{ + Reconciler: rec, + LogConstructor: func(*reconcile.Request) logr.Logger { return syncLog }, + }) + if err != nil { + return fmt.Errorf("creating sync controller: %w", err) + } + + src := newInformerSource(inf) + if err := ctrlr.Watch(src); err != nil { + return fmt.Errorf("watching consumer informer: %w", err) + } + if err := ctrlr.Watch(r.bindingSource(gvr.GroupResource(), lister)); err != nil { + return fmt.Errorf("watching cluster bindings: %w", err) + } + if err := ctrlr.Watch(r.namespacedBindingSource(gvr.GroupResource(), lister)); err != nil { + return fmt.Errorf("watching bindings: %w", err) + } + + syncCtx, cancel := context.WithCancel(ctx) + wg := &sync.WaitGroup{} + wg.Add(2) + go func() { defer wg.Done(); inf.Informer().Run(syncCtx.Done()) }() + go func() { + defer wg.Done() + if err := ctrlr.Start(syncCtx); err != nil { + runtime.HandleErrorWithContext(syncCtx, err, "sync controller stopped") + } + }() + if !cache.WaitForCacheSync(syncCtx.Done(), inf.Informer().HasSynced) { + cancel() + return fmt.Errorf("consumer informer for %s failed to sync", gvr) + } + + r.lock.Lock() + r.contexts[name] = &syncContext{generation: generation, cancel: cancel, wg: wg} + r.lock.Unlock() + log.Info("started syncer", "gvr", gvr.String()) + return nil +} + +func (r *CRDController) bindingSource(gr schema.GroupResource, lister dynamiclister.Lister) source.TypedSource[reconcile.Request] { + return source.Kind(r.mgr.GetCache(), &corev1alpha1.ClusterBinding{}, handler.TypedEnqueueRequestsFromMapFunc( + func(_ context.Context, cb *corev1alpha1.ClusterBinding) []reconcile.Request { + if !listsAPI(cb.Spec.APIs, gr.String()) { + return nil + } + return requestsForAll(lister, "") + })) +} + +func (r *CRDController) namespacedBindingSource(gr schema.GroupResource, lister dynamiclister.Lister) source.TypedSource[reconcile.Request] { + return source.Kind(r.mgr.GetCache(), &corev1alpha1.Binding{}, handler.TypedEnqueueRequestsFromMapFunc( + func(_ context.Context, b *corev1alpha1.Binding) []reconcile.Request { + if !listsAPI(b.Spec.APIs, gr.String()) { + return nil + } + return requestsForAll(lister, b.GetNamespace()) + })) +} + +func requestsForAll(lister dynamiclister.Lister, namespace string) []reconcile.Request { + var list []any + if namespace != "" { + l, err := lister.Namespace(namespace).List(labels.Everything()) + if err != nil { + return nil + } + for _, o := range l { + list = append(list, o) + } + } else { + l, err := lister.List(labels.Everything()) + if err != nil { + return nil + } + for _, o := range l { + list = append(list, o) + } + } + out := make([]reconcile.Request, 0, len(list)) + for _, o := range list { + acc, ok := o.(interface { + GetName() string + GetNamespace() string + }) + if !ok { + continue + } + out = append(out, reconcile.Request{NamespacedName: types.NamespacedName{Namespace: acc.GetNamespace(), Name: acc.GetName()}}) + } + return out +} + +func storageOrServedVersion(crd *apiextensionsv1.CustomResourceDefinition) string { + var served string + for _, v := range crd.Spec.Versions { + if v.Storage { + return v.Name + } + if served == "" && v.Served { + served = v.Name + } + } + return served +} + +func waitWithTimeout(wg *sync.WaitGroup, timeout time.Duration) { + done := make(chan struct{}) + go func() { wg.Wait(); close(done) }() + select { + case <-done: + case <-time.After(timeout): + } +} + +// newInformerSource adapts a dynamic informer to a controller-runtime source. +func newInformerSource(gi interface { + Informer() cache.SharedIndexInformer +}) source.TypedSource[reconcile.Request] { + return &informerSourceImpl{informer: gi.Informer()} +} + +type informerSourceImpl struct { + informer cache.SharedIndexInformer +} + +func (s *informerSourceImpl) Start(ctx context.Context, q workqueue.TypedRateLimitingInterface[reconcile.Request]) error { + _, err := s.informer.AddEventHandler(cache.ResourceEventHandlerFuncs{ + AddFunc: func(o any) { enqueue(o, q) }, + UpdateFunc: func(_, o any) { enqueue(o, q) }, + DeleteFunc: func(o any) { enqueue(o, q) }, + }) + return err +} + +func enqueue(o any, q workqueue.TypedRateLimitingInterface[reconcile.Request]) { + key, err := cache.DeletionHandlingMetaNamespaceKeyFunc(o) + if err != nil { + return + } + ns, name, err := cache.SplitMetaNamespaceKey(key) + if err != nil { + return + } + q.Add(reconcile.Request{NamespacedName: types.NamespacedName{Namespace: ns, Name: name}}) +} + +var _ source.TypedSource[reconcile.Request] = &informerSourceImpl{} diff --git a/v2/konnector/engine/sync/resolve.go b/v2/konnector/engine/sync/resolve.go new file mode 100644 index 000000000..80e0e32b5 --- /dev/null +++ b/v2/konnector/engine/sync/resolve.go @@ -0,0 +1,89 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package sync + +import ( + "context" + + apimeta "k8s.io/apimachinery/pkg/api/meta" + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// CRDNameForGVR returns the "." CRD name for a GVR. +func CRDNameForGVR(gvr schema.GroupVersionResource) string { + return gvr.Resource + "." + gvr.Group +} + +// Resolution is the result of resolving which binding covers an API. +type Resolution struct { + // Found is true if a binding lists the API. + Found bool + // Ready is true if that binding is Ready. + Ready bool + // ConnectionName is the Connection the binding references. + ConnectionName string +} + +// ResolveConnection finds the binding that covers crdName for the given +// namespace. A ClusterBinding wins over a namespaced Binding (proposal rule). +func ResolveConnection(ctx context.Context, c client.Client, crdName, namespace string) (Resolution, error) { + var cbs corev1alpha1.ClusterBindingList + if err := c.List(ctx, &cbs); err != nil { + return Resolution{}, err + } + for i := range cbs.Items { + cb := &cbs.Items[i] + if listsAPI(cb.Spec.APIs, crdName) { + return Resolution{ + Found: true, + Ready: apimeta.IsStatusConditionTrue(cb.Status.Conditions, corev1alpha1.ConditionReady), + ConnectionName: cb.Spec.ConnectionRef.Name, + }, nil + } + } + + if namespace != "" { + var bs corev1alpha1.BindingList + if err := c.List(ctx, &bs, client.InNamespace(namespace)); err != nil { + return Resolution{}, err + } + for i := range bs.Items { + b := &bs.Items[i] + if listsAPI(b.Spec.APIs, crdName) { + return Resolution{ + Found: true, + Ready: apimeta.IsStatusConditionTrue(b.Status.Conditions, corev1alpha1.ConditionReady), + ConnectionName: b.Spec.ConnectionRef.Name, + }, nil + } + } + } + + return Resolution{Found: false}, nil +} + +func listsAPI(apis []corev1alpha1.APIRef, crdName string) bool { + for _, a := range apis { + if a.Name == crdName { + return true + } + } + return false +} diff --git a/v2/konnector/engine/sync/syncer.go b/v2/konnector/engine/sync/syncer.go new file mode 100644 index 000000000..2642b9982 --- /dev/null +++ b/v2/konnector/engine/sync/syncer.go @@ -0,0 +1,335 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package sync + +import ( + "context" + "fmt" + "sync" + "time" + + corev1 "k8s.io/api/core/v1" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/dynamic/dynamiclister" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + + "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +const ( + // fieldOwner is the server-side-apply field manager for spec writes. + fieldOwner = "kube-bind-konnector" + // statusResyncInterval re-checks provider status for drift. POC uses + // polling; an event-driven provider watch is the follow-up (proposal #2). + statusResyncInterval = 30 * time.Second +) + +// providerCache builds and caches a direct provider client per Connection. +// +// TODO(v2): use the multicluster-runtime engaged cluster's client/cache instead +// of a direct client, so per-connection lifecycle comes from the framework. +type providerCache struct { + consumer client.Client + scheme *runtime.Scheme + + mu sync.Mutex + clients map[string]client.Client +} + +func newProviderCache(consumer client.Client, scheme *runtime.Scheme) *providerCache { + return &providerCache{consumer: consumer, scheme: scheme, clients: map[string]client.Client{}} +} + +func (p *providerCache) For(ctx context.Context, connName string) (client.Client, *corev1alpha1.Connection, error) { + conn := &corev1alpha1.Connection{} + if err := p.consumer.Get(ctx, client.ObjectKey{Name: connName}, conn); err != nil { + return nil, nil, err + } + + p.mu.Lock() + defer p.mu.Unlock() + if cl, ok := p.clients[connName]; ok { + return cl, conn, nil + } + cl, err := remote.ProviderClient(ctx, p.consumer, conn, p.scheme) + if err != nil { + return nil, nil, err + } + p.clients[connName] = cl + return cl, conn, nil +} + +// specReconciler syncs instances of one GVR: spec consumer->provider (SSA), +// status provider->consumer, with a finalizer and ownership markers. +type specReconciler struct { + gvr schema.GroupVersionResource + gvk schema.GroupVersionKind + scope apiextensionsv1.ResourceScope + + consumerLister dynamiclister.Lister + consumerClient client.Client // direct, uncached + providers *providerCache + recorder record.EventRecorder +} + +func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) { + log := ctrl.LoggerFrom(ctx).WithValues("gvr", r.gvr.String(), "key", req.String()) + + obj, err := r.getConsumerObject(req) + if apierrors.IsNotFound(err) { + return reconcile.Result{}, nil + } + if err != nil { + return reconcile.Result{}, err + } + obj = obj.DeepCopy() + + crdName := CRDNameForGVR(r.gvr) + res, err := ResolveConnection(ctx, r.consumerClient, crdName, req.Namespace) + if err != nil { + return reconcile.Result{}, fmt.Errorf("resolving connection: %w", err) + } + if !res.Found || !res.Ready { + // Not bound (or binding not ready): nothing to do; binding changes will + // re-trigger. If the object carries our finalizer with no binding, we + // still try to release it on deletion below. + if obj.GetDeletionTimestamp() == nil { + return reconcile.Result{}, nil + } + } + + var ( + providerClient client.Client + conn *corev1alpha1.Connection + ) + if res.Found && res.Ready { + providerClient, conn, err = r.providers.For(ctx, res.ConnectionName) + if err != nil { + return reconcile.Result{}, fmt.Errorf("provider client: %w", err) + } + } + + localUID := "" + if conn != nil { + localUID = conn.Status.LocalClusterUID + } + + // Deletion. + if obj.GetDeletionTimestamp() != nil { + return r.reconcileDelete(ctx, obj, providerClient, localUID) + } + + // Ensure finalizer before first write. + if controllerutil.AddFinalizer(obj, corev1alpha1.FinalizerSyncer) { + if err := r.consumerClient.Update(ctx, obj); err != nil { + return reconcile.Result{}, fmt.Errorf("adding finalizer: %w", err) + } + return reconcile.Result{Requeue: true}, nil + } + + if providerClient == nil { + return reconcile.Result{}, nil + } + + // Conflict check before first write. + existing := newUnstructured(r.gvk) + getErr := providerClient.Get(ctx, client.ObjectKeyFromObject(obj), existing) + switch { + case getErr == nil: + if owner, ours := ownershipOf(existing, localUID, string(obj.GetUID())); !ours { + reason := conflictReason(owner) + msg := fmt.Sprintf("provider object %s already exists and is %s; not overwriting (conflictPolicy: Fail)", client.ObjectKeyFromObject(obj), owner) + log.Info("conflict: refusing to overwrite provider object", "owner", owner) + // Surface on the object's own condition (best-effort: pruned if the + // CRD's status schema has no conditions) AND as an Event (always). + setObjCondition(obj, metav1.ConditionFalse, reason, msg) + _ = r.consumerClient.Status().Update(ctx, obj) + if r.recorder != nil { + r.recorder.Event(obj, corev1.EventTypeWarning, reason, msg) + } + return reconcile.Result{}, nil + } + case apierrors.IsNotFound(getErr): + if r.scope == apiextensionsv1.NamespaceScoped { + if err := ensureNamespace(ctx, providerClient, obj.GetNamespace(), localUID); err != nil { + return reconcile.Result{}, err + } + } + default: + return reconcile.Result{}, fmt.Errorf("provider get: %w", getErr) + } + + // Spec consumer -> provider (SSA). + patch := r.providerPatch(obj, localUID) + if err := providerClient.Patch(ctx, patch, client.Apply, client.FieldOwner(fieldOwner), client.ForceOwnership); err != nil { + return reconcile.Result{}, fmt.Errorf("applying spec to provider: %w", err) + } + + // Status provider -> consumer. + got := newUnstructured(r.gvk) + if err := providerClient.Get(ctx, client.ObjectKeyFromObject(obj), got); err != nil { + return reconcile.Result{}, fmt.Errorf("reading provider status: %w", err) + } + if err := r.copyStatus(ctx, got, obj); err != nil { + return reconcile.Result{}, err + } + + return reconcile.Result{RequeueAfter: statusResyncInterval}, nil +} + +func (r *specReconciler) reconcileDelete(ctx context.Context, obj *unstructured.Unstructured, providerClient client.Client, localUID string) (reconcile.Result, error) { + if !controllerutil.ContainsFinalizer(obj, corev1alpha1.FinalizerSyncer) { + return reconcile.Result{}, nil + } + if providerClient != nil { + existing := newUnstructured(r.gvk) + err := providerClient.Get(ctx, client.ObjectKeyFromObject(obj), existing) + switch { + case err == nil: + // Only delete the provider copy if it is ours. A foreign object that + // merely shares the name (a conflict) must never be deleted by us. + if _, ours := ownershipOf(existing, localUID, string(obj.GetUID())); ours { + if err := providerClient.Delete(ctx, existing); client.IgnoreNotFound(err) != nil { + return reconcile.Result{}, fmt.Errorf("deleting provider object: %w", err) + } + // Wait for the provider copy to be fully gone before releasing. + return reconcile.Result{RequeueAfter: 2 * time.Second}, nil + } + case !apierrors.IsNotFound(err): + return reconcile.Result{}, fmt.Errorf("provider get during delete: %w", err) + } + } + controllerutil.RemoveFinalizer(obj, corev1alpha1.FinalizerSyncer) + if err := r.consumerClient.Update(ctx, obj); err != nil { + return reconcile.Result{}, fmt.Errorf("removing finalizer: %w", err) + } + return reconcile.Result{}, nil +} + +func (r *specReconciler) getConsumerObject(req reconcile.Request) (*unstructured.Unstructured, error) { + if r.scope == apiextensionsv1.NamespaceScoped { + return r.consumerLister.Namespace(req.Namespace).Get(req.Name) + } + return r.consumerLister.Get(req.Name) +} + +// providerPatch builds the SSA patch: spec + identity, ownership markers, no +// status, no consumer-only metadata. +func (r *specReconciler) providerPatch(obj *unstructured.Unstructured, localUID string) *unstructured.Unstructured { + patch := newUnstructured(r.gvk) + patch.SetName(obj.GetName()) + patch.SetNamespace(obj.GetNamespace()) + patch.SetLabels(map[string]string{corev1alpha1.LabelManaged: "true"}) + patch.SetAnnotations(map[string]string{ + corev1alpha1.AnnotationConsumerClusterUID: localUID, + corev1alpha1.AnnotationConsumerObjectUID: string(obj.GetUID()), + }) + if spec, ok, _ := unstructured.NestedFieldCopy(obj.Object, "spec"); ok { + _ = unstructured.SetNestedField(patch.Object, spec, "spec") + } + return patch +} + +func (r *specReconciler) copyStatus(ctx context.Context, from, to *unstructured.Unstructured) error { + status, ok, err := unstructured.NestedFieldCopy(from.Object, "status") + if err != nil { + return err + } + if !ok { + return nil + } + if err := unstructured.SetNestedField(to.Object, status, "status"); err != nil { + return err + } + return r.consumerClient.Status().Update(ctx, to) +} + +func newUnstructured(gvk schema.GroupVersionKind) *unstructured.Unstructured { + u := &unstructured.Unstructured{} + u.SetGroupVersionKind(gvk) + return u +} + +// ownershipOf reports the marker owner of a provider object and whether it is +// ours (our consumer cluster + our consumer object UID). +func ownershipOf(obj *unstructured.Unstructured, localUID, consumerObjUID string) (string, bool) { + ann := obj.GetAnnotations() + cluster := ann[corev1alpha1.AnnotationConsumerClusterUID] + objUID := ann[corev1alpha1.AnnotationConsumerObjectUID] + if cluster == "" && objUID == "" { + return "foreign-unmanaged", false + } + if cluster == localUID && objUID == consumerObjUID { + return "ours", true + } + return "owned-by-another", false +} + +func conflictReason(owner string) string { + if owner == "foreign-unmanaged" { + return corev1alpha1.ReasonForeignObjectExists + } + return corev1alpha1.ReasonOwnedByAnother +} + +func setObjCondition(obj *unstructured.Unstructured, status metav1.ConditionStatus, reason, msg string) { + cond := map[string]any{ + "type": corev1alpha1.ConditionSynced, + "status": string(status), + "reason": reason, + "message": msg, + } + conds, _, _ := unstructured.NestedSlice(obj.Object, "status", "conditions") + // Replace any existing Synced condition. + out := make([]any, 0, len(conds)+1) + for _, c := range conds { + if m, ok := c.(map[string]any); ok && m["type"] == corev1alpha1.ConditionSynced { + continue + } + out = append(out, c) + } + out = append(out, cond) + _ = unstructured.SetNestedSlice(obj.Object, out, "status", "conditions") +} + +func ensureNamespace(ctx context.Context, c client.Client, name, localUID string) error { + if name == "" { + return nil + } + ns := &unstructured.Unstructured{} + ns.SetGroupVersionKind(schema.GroupVersionKind{Version: "v1", Kind: "Namespace"}) + ns.SetName(name) + ns.SetLabels(map[string]string{ + corev1alpha1.LabelManaged: "true", + corev1alpha1.AnnotationConsumerClusterUID: localUID, + }) + err := c.Create(ctx, ns) + if apierrors.IsAlreadyExists(err) { + return nil + } + return err +} diff --git a/v2/konnector/go.mod b/v2/konnector/go.mod new file mode 100644 index 000000000..6697fd5b2 --- /dev/null +++ b/v2/konnector/go.mod @@ -0,0 +1,72 @@ +module github.com/kube-bind/kube-bind/v2/konnector + +go 1.24.0 + +require ( + github.com/go-logr/logr v1.4.2 + github.com/kube-bind/kube-bind/v2/sdk v0.0.0-00010101000000-000000000000 + github.com/stretchr/testify v1.10.0 + golang.org/x/sync v0.12.0 + k8s.io/api v0.33.4 + k8s.io/apiextensions-apiserver v0.33.4 + k8s.io/apimachinery v0.33.4 + k8s.io/client-go v0.33.4 + sigs.k8s.io/controller-runtime v0.21.0 + sigs.k8s.io/multicluster-runtime v0.21.0-alpha.9 +) + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/blang/semver/v4 v4.0.0 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/emicklei/go-restful/v3 v3.11.0 // indirect + github.com/evanphx/json-patch/v5 v5.9.11 // indirect + github.com/fsnotify/fsnotify v1.7.0 // indirect + github.com/fxamacker/cbor/v2 v2.7.0 // indirect + github.com/go-logr/zapr v1.3.0 // indirect + github.com/go-openapi/jsonpointer v0.21.0 // indirect + github.com/go-openapi/jsonreference v0.20.2 // indirect + github.com/go-openapi/swag v0.23.0 // indirect + github.com/gogo/protobuf v1.3.2 // indirect + github.com/google/btree v1.1.3 // indirect + github.com/google/gnostic-models v0.6.9 // indirect + github.com/google/go-cmp v0.7.0 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/josharian/intern v1.0.0 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/mailru/easyjson v0.7.7 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/pkg/errors v0.9.1 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/prometheus/client_golang v1.22.0 // indirect + github.com/prometheus/client_model v0.6.1 // indirect + github.com/prometheus/common v0.62.0 // indirect + github.com/prometheus/procfs v0.15.1 // indirect + github.com/spf13/pflag v1.0.5 // indirect + github.com/x448/float16 v0.8.4 // indirect + go.uber.org/multierr v1.11.0 // indirect + go.uber.org/zap v1.27.0 // indirect + golang.org/x/net v0.38.0 // indirect + golang.org/x/oauth2 v0.27.0 // indirect + golang.org/x/sys v0.31.0 // indirect + golang.org/x/term v0.30.0 // indirect + golang.org/x/text v0.23.0 // indirect + golang.org/x/time v0.9.0 // indirect + gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect + google.golang.org/protobuf v1.36.5 // indirect + gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect + gopkg.in/inf.v0 v0.9.1 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect + k8s.io/klog/v2 v2.130.1 // indirect + k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff // indirect + k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect + sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect + sigs.k8s.io/randfill v1.0.0 // indirect + sigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect + sigs.k8s.io/yaml v1.4.0 // indirect +) + +replace github.com/kube-bind/kube-bind/v2/sdk => ../sdk diff --git a/v2/konnector/go.sum b/v2/konnector/go.sum new file mode 100644 index 000000000..c00ea24f9 --- /dev/null +++ b/v2/konnector/go.sum @@ -0,0 +1,199 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= +github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g= +github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= +github.com/evanphx/json-patch v0.5.2 h1:xVCHIVMUu1wtM/VkR9jVZ45N3FhZfYMMYGorLCR8P3k= +github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= +github.com/evanphx/json-patch/v5 v5.9.11 h1:/8HVnzMq13/3x9TPvjG08wUGqBTmZBsCWzjTM0wiaDU= +github.com/evanphx/json-patch/v5 v5.9.11/go.mod h1:3j+LviiESTElxA4p3EMKAB9HXj3/XEtnUf6OZxqIQTM= +github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA= +github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM= +github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E= +github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ= +github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= +github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ= +github.com/go-logr/zapr v1.3.0/go.mod h1:YKepepNBd1u/oyhd/yQmtjVXmm9uML4IXUgMOwR8/Gg= +github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs= +github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ= +github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY= +github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE= +github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k= +github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= +github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE= +github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ= +github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI= +github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg= +github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4= +github.com/google/gnostic-models v0.6.9 h1:MU/8wDLif2qCXZmzncUQ/BOfxWfthHi63KqpoNbWqVw= +github.com/google/gnostic-models v0.6.9/go.mod h1:CiWsm0s6BSQd1hRn8/QmxqB6BesYcbSZxsz9b0KuDBw= +github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= +github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db h1:097atOisP2aRj7vFgYQBbFN4U4JNXUNYpxael3UzMyo= +github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db/go.mod h1:vavhavw2zAxS5dIdcRluK6cSGGPlZynqzFM8NdvU144= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= +github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= +github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= +github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/onsi/ginkgo/v2 v2.22.0 h1:Yed107/8DjTr0lKCNt7Dn8yQ6ybuDRQoMGrNFKzMfHg= +github.com/onsi/ginkgo/v2 v2.22.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo= +github.com/onsi/gomega v1.36.1 h1:bJDPBO7ibjxcbHMgSCoo4Yj18UWbKDlLwX1x9sybDcw= +github.com/onsi/gomega v1.36.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q= +github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0= +github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= +github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= +github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io= +github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I= +github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= +github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= +github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= +github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o= +github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= +github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= +github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= +go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= +go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= +golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= +golang.org/x/oauth2 v0.27.0 h1:da9Vo7/tDv5RH/7nZDz1eMGS/q1Vv1N/7FCrBhI9I3M= +golang.org/x/oauth2 v0.27.0/go.mod h1:onh5ek6nERTohokkhCD/y2cV4Do3fxFHFuAejCkRWT8= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw= +golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik= +golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/term v0.30.0 h1:PQ39fJZ+mfadBm0y5WlL4vlM7Sx1Hgf13sMIY2+QS9Y= +golang.org/x/term v0.30.0/go.mod h1:NYYFdzHoI5wRh/h5tDMdMqCqPJZEuNqVR5xJLd/n67g= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY= +golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4= +golang.org/x/time v0.9.0 h1:EsRrnYcQiGH+5FfbgvV4AP7qEZstoyrHB0DzarOQ4ZY= +golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.26.0 h1:v/60pFQmzmT9ExmjDv2gGIfi3OqfKoEP6I5+umXlbnQ= +golang.org/x/tools v0.26.0/go.mod h1:TPVVj70c7JJ3WCazhD8OdXcZg/og+b9+tH/KxylGwH0= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw= +gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY= +google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM= +google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4= +gopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M= +gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= +gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +k8s.io/api v0.33.4 h1:oTzrFVNPXBjMu0IlpA2eDDIU49jsuEorGHB4cvKupkk= +k8s.io/api v0.33.4/go.mod h1:VHQZ4cuxQ9sCUMESJV5+Fe8bGnqAARZ08tSTdHWfeAc= +k8s.io/apiextensions-apiserver v0.33.4 h1:rtq5SeXiDbXmSwxsF0MLe2Mtv3SwprA6wp+5qh/CrOU= +k8s.io/apiextensions-apiserver v0.33.4/go.mod h1:mWXcZQkQV1GQyxeIjYApuqsn/081hhXPZwZ2URuJeSs= +k8s.io/apimachinery v0.33.4 h1:SOf/JW33TP0eppJMkIgQ+L6atlDiP/090oaX0y9pd9s= +k8s.io/apimachinery v0.33.4/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM= +k8s.io/client-go v0.33.4 h1:TNH+CSu8EmXfitntjUPwaKVPN0AYMbc9F1bBS8/ABpw= +k8s.io/client-go v0.33.4/go.mod h1:LsA0+hBG2DPwovjd931L/AoaezMPX9CmBgyVyBZmbCY= +k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk= +k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE= +k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff h1:/usPimJzUKKu+m+TE36gUyGcf03XZEP0ZIKgKj35LS4= +k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff/go.mod h1:5jIi+8yX4RIb8wk3XwBo5Pq2ccx4FP10ohkbSKCZoK8= +k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 h1:M3sRQVHv7vB20Xc2ybTt7ODCeFj6JSWYFzOFnYeS6Ro= +k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +sigs.k8s.io/controller-runtime v0.21.0 h1:CYfjpEuicjUecRk+KAeyYh+ouUBn4llGyDYytIGcJS8= +sigs.k8s.io/controller-runtime v0.21.0/go.mod h1:OSg14+F65eWqIu4DceX7k/+QRAbTTvxeQSNSOQpukWM= +sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8= +sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo= +sigs.k8s.io/multicluster-runtime v0.21.0-alpha.9 h1:baonM4f081WWct3U7O4EfqrxcUGtmCrFDbsT1FQ8xlo= +sigs.k8s.io/multicluster-runtime v0.21.0-alpha.9/go.mod h1:CpBzLMLQKdm+UCchd2FiGPiDdCxM5dgCCPKuaQ6Fsv0= +sigs.k8s.io/randfill v0.0.0-20250304075658-069ef1bbf016/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY= +sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU= +sigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY= +sigs.k8s.io/structured-merge-diff/v4 v4.6.0 h1:IUA9nvMmnKWcj5jl84xn+T5MnlZKThmUW1TdblaLVAc= +sigs.k8s.io/structured-merge-diff/v4 v4.6.0/go.mod h1:dDy58f92j70zLsuZVuUX5Wp9vtxXpaZnkPGWeqDfCps= +sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E= +sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY= diff --git a/v2/konnector/test/e2e/framework/framework.go b/v2/konnector/test/e2e/framework/framework.go new file mode 100644 index 000000000..ef12dc1d3 --- /dev/null +++ b/v2/konnector/test/e2e/framework/framework.go @@ -0,0 +1,287 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package framework provides an envtest-based harness for v2 slim-core e2e +// tests: a provider API server and a consumer API server, with the konnector +// engine reconcilers running in-process against the consumer. +// +// Unlike the v1 framework (kcp + backend + browser auth), the v2 core has no +// backend, so the harness only needs two API servers, a kubeconfig Secret, and +// the one-apply bundle. +package framework + +import ( + "context" + "path/filepath" + "runtime" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/wait" + clientgoscheme "k8s.io/client-go/kubernetes/scheme" + "k8s.io/apimachinery/pkg/runtime/schema" + apimachineryruntime "k8s.io/apimachinery/pkg/runtime" + utilruntime "k8s.io/apimachinery/pkg/util/runtime" + "k8s.io/client-go/dynamic" + "k8s.io/client-go/rest" + clientcmdapi "k8s.io/client-go/tools/clientcmd/api" + "k8s.io/client-go/tools/clientcmd" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/envtest" + "sigs.k8s.io/controller-runtime/pkg/log/zap" + metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" + + "github.com/kube-bind/kube-bind/v2/konnector/engine/binding" + "github.com/kube-bind/kube-bind/v2/konnector/engine/connection" + syncengine "github.com/kube-bind/kube-bind/v2/konnector/engine/sync" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// KubeBindNamespace is the konnector's designated namespace on the consumer. +const KubeBindNamespace = "kube-bind" + +// Env is a running provider+consumer test environment with the engine wired up. +type Env struct { + Scheme *apimachineryruntime.Scheme + + ProviderCfg *rest.Config + ConsumerCfg *rest.Config + ProviderClient client.Client + ConsumerClient client.Client + ProviderDyn dynamic.Interface + ConsumerDyn dynamic.Interface +} + +// Start brings up two envtest API servers, installs the core CRDs on the +// consumer, creates the kube-system namespaces (for cluster identity) and the +// kube-bind namespace, stores the provider kubeconfig as a Secret on the +// consumer, and starts the engine reconcilers in-process against the consumer. +var setLoggerOnce sync.Once + +func Start(t *testing.T) *Env { + t.Helper() + setLoggerOnce.Do(func() { ctrl.SetLogger(zap.New(zap.UseDevMode(true))) }) + + scheme := apimachineryruntime.NewScheme() + utilruntime.Must(clientgoscheme.AddToScheme(scheme)) + utilruntime.Must(apiextensionsv1.AddToScheme(scheme)) + utilruntime.Must(corev1alpha1.AddToScheme(scheme)) + + // Consumer API server, pre-loaded with the core CRDs. + consumerEnv := &envtest.Environment{ + CRDDirectoryPaths: []string{coreCRDDir(t)}, + ErrorIfCRDPathMissing: true, + Scheme: scheme, + } + consumerCfg, err := consumerEnv.Start() + require.NoError(t, err, "starting consumer envtest") + t.Cleanup(func() { _ = consumerEnv.Stop() }) + + // Provider API server (plain). + providerEnv := &envtest.Environment{Scheme: scheme} + providerCfg, err := providerEnv.Start() + require.NoError(t, err, "starting provider envtest") + t.Cleanup(func() { _ = providerEnv.Stop() }) + + consumerClient, err := client.New(consumerCfg, client.Options{Scheme: scheme}) + require.NoError(t, err) + providerClient, err := client.New(providerCfg, client.Options{Scheme: scheme}) + require.NoError(t, err) + consumerDyn, err := dynamic.NewForConfig(consumerCfg) + require.NoError(t, err) + providerDyn, err := dynamic.NewForConfig(providerCfg) + require.NoError(t, err) + + ctx := context.Background() + + // kube-system in both clusters → stable cluster identity for ownership markers + // (envtest may already seed it, so tolerate AlreadyExists). + ensureNamespace(t, consumerClient, "kube-system") + ensureNamespace(t, providerClient, "kube-system") + ensureNamespace(t, consumerClient, KubeBindNamespace) + + // Store the provider kubeconfig as a Secret on the consumer (the credential + // the Connection references). + kubeconfig, err := kubeconfigFromRestConfig(providerCfg) + require.NoError(t, err) + require.NoError(t, consumerClient.Create(ctx, &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider-kubeconfig", Namespace: KubeBindNamespace}, + Data: map[string][]byte{"kubeconfig": kubeconfig}, + })) + + startEngine(t, consumerCfg, scheme) + + return &Env{ + Scheme: scheme, + ProviderCfg: providerCfg, + ConsumerCfg: consumerCfg, + ProviderClient: providerClient, + ConsumerClient: consumerClient, + ProviderDyn: providerDyn, + ConsumerDyn: consumerDyn, + } +} + +// startEngine runs the consumer-side reconcilers (connection, both bindings, and +// the dynamic sync controller) on a manager pointed at the consumer cluster. +// The mcr provider is not needed here: the sync path uses direct provider +// clients in the POC. +func startEngine(t *testing.T, consumerCfg *rest.Config, scheme *apimachineryruntime.Scheme) { + t.Helper() + mgr, err := ctrl.NewManager(consumerCfg, ctrl.Options{ + Scheme: scheme, + Metrics: metricsserver.Options{BindAddress: "0"}, + }) + require.NoError(t, err) + + require.NoError(t, (&connection.Reconciler{}).SetupWithManager(mgr)) + require.NoError(t, (&binding.ClusterReconciler{}).SetupWithManager(mgr)) + require.NoError(t, (&binding.NamespacedReconciler{}).SetupWithManager(mgr)) + require.NoError(t, syncengine.SetupWithManager(mgr)) + + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + go func() { + if err := mgr.Start(ctx); err != nil { + t.Logf("manager stopped: %v", err) + } + }() + require.True(t, mgr.GetCache().WaitForCacheSync(ctx), "engine cache sync") +} + +// InstallExportedWidgetCRD installs the demo Widget CRD on the provider, labeled +// as exported. +func (e *Env) InstallExportedWidgetCRD(t *testing.T) schema.GroupVersionResource { + t.Helper() + crd := widgetCRD() + require.NoError(t, e.ProviderClient.Create(context.Background(), crd)) + gvr := schema.GroupVersionResource{Group: "example.org", Version: "v1", Resource: "widgets"} + // Wait for the provider to serve the new API. + require.Eventually(t, func() bool { + _, err := e.ProviderDyn.Resource(gvr).Namespace("default").List(context.Background(), metav1.ListOptions{}) + return err == nil + }, 30*time.Second, 200*time.Millisecond, "provider should serve the Widget API") + return gvr +} + +// WidgetGVK returns the demo Widget GVK. +func WidgetGVK() schema.GroupVersionKind { + return schema.GroupVersionKind{Group: "example.org", Version: "v1", Kind: "Widget"} +} + +func widgetCRD() *apiextensionsv1.CustomResourceDefinition { + preserve := true + return &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{ + Name: "widgets.example.org", + Labels: map[string]string{corev1alpha1.LabelExported: "true"}, + }, + Spec: apiextensionsv1.CustomResourceDefinitionSpec{ + Group: "example.org", + Names: apiextensionsv1.CustomResourceDefinitionNames{ + Plural: "widgets", + Singular: "widget", + Kind: "Widget", + ListKind: "WidgetList", + }, + Scope: apiextensionsv1.NamespaceScoped, + Versions: []apiextensionsv1.CustomResourceDefinitionVersion{{ + Name: "v1", + Served: true, + Storage: true, + Subresources: &apiextensionsv1.CustomResourceSubresources{ + Status: &apiextensionsv1.CustomResourceSubresourceStatus{}, + }, + Schema: &apiextensionsv1.CustomResourceValidation{ + OpenAPIV3Schema: &apiextensionsv1.JSONSchemaProps{ + Type: "object", + Properties: map[string]apiextensionsv1.JSONSchemaProps{ + "spec": { + Type: "object", + XPreserveUnknownFields: &preserve, + }, + "status": { + Type: "object", + XPreserveUnknownFields: &preserve, + }, + }, + }, + }, + }}, + }, + } +} + +// kubeconfigFromRestConfig serializes an envtest rest.Config (client-cert auth) +// into a kubeconfig the konnector can load from a Secret. +func kubeconfigFromRestConfig(cfg *rest.Config) ([]byte, error) { + const name = "provider" + c := clientcmdapi.NewConfig() + c.Clusters[name] = &clientcmdapi.Cluster{ + Server: cfg.Host, + CertificateAuthorityData: cfg.CAData, + } + c.AuthInfos[name] = &clientcmdapi.AuthInfo{ + ClientCertificateData: cfg.CertData, + ClientKeyData: cfg.KeyData, + } + c.Contexts[name] = &clientcmdapi.Context{Cluster: name, AuthInfo: name} + c.CurrentContext = name + return clientcmd.Write(*c) +} + +func ensureNamespace(t *testing.T, c client.Client, name string) { + t.Helper() + err := c.Create(context.Background(), &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: name}}) + if err != nil && !apierrors.IsAlreadyExists(err) { + require.NoError(t, err, "creating namespace %s", name) + } +} + +func coreCRDDir(t *testing.T) string { + t.Helper() + _, thisFile, _, ok := runtime.Caller(0) + require.True(t, ok, "runtime.Caller") + // .../v2/konnector/test/e2e/framework/framework.go -> .../v2/sdk/config/crd + dir := filepath.Join(filepath.Dir(thisFile), "..", "..", "..", "..", "sdk", "config", "crd") + abs, err := filepath.Abs(dir) + require.NoError(t, err) + return abs +} + +// WaitForConditionTrue polls until the named condition on the object is True. +func WaitForConditionTrue(t *testing.T, get func() ([]metav1.Condition, error), condType string) { + t.Helper() + require.Eventually(t, func() bool { + conds, err := get() + if err != nil { + return false + } + for _, c := range conds { + if c.Type == condType { + return c.Status == metav1.ConditionTrue + } + } + return false + }, wait.ForeverTestTimeout, 200*time.Millisecond, "waiting for condition %s=True", condType) +} diff --git a/v2/konnector/test/e2e/sync_test.go b/v2/konnector/test/e2e/sync_test.go new file mode 100644 index 000000000..99827f719 --- /dev/null +++ b/v2/konnector/test/e2e/sync_test.go @@ -0,0 +1,256 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package e2e + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/util/wait" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" + "github.com/kube-bind/kube-bind/v2/konnector/test/e2e/framework" +) + +const ( + widgetCRDName = "widgets.example.org" + instanceNS = "default" + instanceName = "my-widget" +) + +// TestSlimCoreHappyCase exercises the full v2 slim-core flow against two +// envtest API servers, mirroring the v1 happy-case step pattern: the one-apply +// bundle (Secret + Connection + ClusterBinding) drives CRD delivery and +// bidirectional instance sync, with conflict and deletion behavior verified. +func TestSlimCoreHappyCase(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + + gvr := env.InstallExportedWidgetCRD(t) + consumerWidgets := env.ConsumerDyn.Resource(gvr).Namespace(instanceNS) + providerWidgets := env.ProviderDyn.Resource(gvr).Namespace(instanceNS) + + // The "one apply" bundle: Connection + ClusterBinding on the consumer. + // (The kubeconfig Secret was created by the framework.) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{ + Namespace: framework.KubeBindNamespace, + Name: "demo-provider-kubeconfig", + Key: "kubeconfig", + }, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + }, + })) + + for _, tc := range []struct { + name string + step func(t *testing.T) + }{ + { + name: "Connection becomes Ready and discovers the exported API", + step: func(t *testing.T) { + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + conn := &corev1alpha1.Connection{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn) + return conn.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + conn := &corev1alpha1.Connection{} + require.NoError(t, env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn)) + require.NotEmpty(t, conn.Status.RemoteClusterUID, "remote cluster UID must be pinned") + require.NotEmpty(t, conn.Status.LocalClusterUID, "local cluster UID must be pinned") + _, ok := conn.Status.ExportsAPI(widgetCRDName) + require.True(t, ok, "Connection must export %s", widgetCRDName) + }, + }, + { + name: "ClusterBinding becomes Ready and the CRD is pulled onto the consumer", + step: func(t *testing.T) { + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + crd := &apiextensionsv1.CustomResourceDefinition{} + require.Eventually(t, func() bool { + return env.ConsumerClient.Get(ctx, client.ObjectKey{Name: widgetCRDName}, crd) == nil + }, wait.ForeverTestTimeout, 200*time.Millisecond, "Widget CRD should be pulled onto the consumer") + require.Equal(t, "true", crd.GetLabels()[corev1alpha1.LabelManaged], "pulled CRD must be marked managed") + }, + }, + { + name: "instance spec syncs consumer -> provider with ownership markers and a finalizer", + step: func(t *testing.T) { + require.Eventually(t, func() bool { + _, err := consumerWidgets.Create(ctx, widget(instanceName, "large"), metav1.CreateOptions{}) + return err == nil || apierrors.IsAlreadyExists(err) + }, 30*time.Second, 200*time.Millisecond, "consumer Widget CRD should become creatable") + + var provObj *unstructured.Unstructured + require.Eventually(t, func() bool { + var err error + provObj, err = providerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + return err == nil + }, wait.ForeverTestTimeout, 200*time.Millisecond, "Widget should sync to the provider") + + size, _, _ := unstructured.NestedString(provObj.Object, "spec", "size") + require.Equal(t, "large", size) + require.Equal(t, "true", provObj.GetLabels()[corev1alpha1.LabelManaged]) + require.NotEmpty(t, provObj.GetAnnotations()[corev1alpha1.AnnotationConsumerClusterUID], "consumer-cluster-uid marker") + require.NotEmpty(t, provObj.GetAnnotations()[corev1alpha1.AnnotationConsumerObjectUID], "consumer-object-uid marker") + + require.Eventually(t, func() bool { + obj, err := consumerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + if err != nil { + return false + } + for _, f := range obj.GetFinalizers() { + if f == corev1alpha1.FinalizerSyncer { + return true + } + } + return false + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer object should carry the syncer finalizer") + }, + }, + { + name: "status syncs provider -> consumer", + step: func(t *testing.T) { + require.NoError(t, retry.RetryOnConflict(retry.DefaultRetry, func() error { + obj, err := providerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + if err != nil { + return err + } + _ = unstructured.SetNestedField(obj.Object, "Running", "status", "phase") + _, err = providerWidgets.UpdateStatus(ctx, obj, metav1.UpdateOptions{}) + return err + })) + require.Eventually(t, func() bool { + obj, err := consumerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + if err != nil { + return false + } + phase, _, _ := unstructured.NestedString(obj.Object, "status", "phase") + return phase == "Running" + }, wait.ForeverTestTimeout, 200*time.Millisecond, "provider status should flow to the consumer") + }, + }, + { + name: "spec update syncs consumer -> provider", + step: func(t *testing.T) { + require.NoError(t, retry.RetryOnConflict(retry.DefaultRetry, func() error { + obj, err := consumerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + if err != nil { + return err + } + _ = unstructured.SetNestedField(obj.Object, "xlarge", "spec", "size") + _, err = consumerWidgets.Update(ctx, obj, metav1.UpdateOptions{}) + return err + })) + require.Eventually(t, func() bool { + obj, err := providerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + if err != nil { + return false + } + size, _, _ := unstructured.NestedString(obj.Object, "spec", "size") + return size == "xlarge" + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer spec update should flow to the provider") + }, + }, + { + name: "conflict: a foreign provider object is not overwritten", + step: func(t *testing.T) { + // Pre-create a foreign object on the provider (no markers). + _, err := providerWidgets.Create(ctx, widget("conflict-widget", "PROVIDER-OWNED"), metav1.CreateOptions{}) + require.NoError(t, err) + // Create a same-named object on the consumer. + _, err = consumerWidgets.Create(ctx, widget("conflict-widget", "CONSUMER-WANTS"), metav1.CreateOptions{}) + require.NoError(t, err) + + // The provider object must stay foreign and untouched. + require.Never(t, func() bool { + obj, err := providerWidgets.Get(ctx, "conflict-widget", metav1.GetOptions{}) + if err != nil { + return false + } + size, _, _ := unstructured.NestedString(obj.Object, "spec", "size") + return size != "PROVIDER-OWNED" || obj.GetLabels()[corev1alpha1.LabelManaged] == "true" + }, 5*time.Second, 500*time.Millisecond, "foreign provider object must not be overwritten") + }, + }, + { + name: "deletion: consumer delete removes the provider copy and releases the finalizer", + step: func(t *testing.T) { + require.NoError(t, consumerWidgets.Delete(ctx, instanceName, metav1.DeleteOptions{})) + require.Eventually(t, func() bool { + _, err := providerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + return apierrors.IsNotFound(err) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "provider copy should be deleted") + require.Eventually(t, func() bool { + _, err := consumerWidgets.Get(ctx, instanceName, metav1.GetOptions{}) + return apierrors.IsNotFound(err) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer object should finalize and disappear") + }, + }, + { + name: "deletion of a conflicting consumer object leaves the foreign provider object intact", + step: func(t *testing.T) { + require.NoError(t, consumerWidgets.Delete(ctx, "conflict-widget", metav1.DeleteOptions{})) + require.Eventually(t, func() bool { + _, err := consumerWidgets.Get(ctx, "conflict-widget", metav1.GetOptions{}) + return apierrors.IsNotFound(err) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer conflict object should finalize") + // Foreign provider object must survive. + obj, err := providerWidgets.Get(ctx, "conflict-widget", metav1.GetOptions{}) + require.NoError(t, err, "foreign provider object must survive consumer deletion") + size, _, _ := unstructured.NestedString(obj.Object, "spec", "size") + require.Equal(t, "PROVIDER-OWNED", size) + }, + }, + } { + t.Run(tc.name, tc.step) + } +} + +func widget(name, size string) *unstructured.Unstructured { + u := &unstructured.Unstructured{} + u.SetGroupVersionKind(framework.WidgetGVK()) + u.SetNamespace(instanceNS) + u.SetName(name) + _ = unstructured.SetNestedField(u.Object, size, "spec", "size") + return u +} + +func apiextensionsGVK() (gvk struct{}) { panic("unused") } diff --git a/v2/sdk/apis/core/v1alpha1/binding_types.go b/v2/sdk/apis/core/v1alpha1/binding_types.go new file mode 100644 index 000000000..1ee547d02 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/binding_types.go @@ -0,0 +1,53 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// Binding activates instance sync for one or more exported APIs within its own +// namespace. Namespaced, following the Role convention. It is the v2 answer to +// v1's informerScope: Namespaced. Where a ClusterBinding and a Binding cover the +// same API on the same connection, the ClusterBinding wins. +// +// +kubebuilder:object:root=true +// +kubebuilder:resource:scope=Namespaced,categories=kube-bind +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="Connection",type=string,JSONPath=`.spec.connectionRef.name` +// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` +// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` +type Binding struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +required + // +kubebuilder:validation:Required + Spec BindingSpec `json:"spec"` + + // +optional + Status BindingStatus `json:"status,omitempty"` +} + +// BindingList contains a list of Binding. +// +// +kubebuilder:object:root=true +type BindingList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []Binding `json:"items"` +} diff --git a/v2/sdk/apis/core/v1alpha1/clusterbinding_types.go b/v2/sdk/apis/core/v1alpha1/clusterbinding_types.go new file mode 100644 index 000000000..d1b9c8ebc --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/clusterbinding_types.go @@ -0,0 +1,67 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// ClusterBinding activates instance sync for one or more exported APIs +// cluster-wide. Cluster-scoped, following the ClusterRole convention. +// +// +kubebuilder:object:root=true +// +kubebuilder:resource:scope=Cluster,categories=kube-bind +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="Connection",type=string,JSONPath=`.spec.connectionRef.name` +// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` +// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` +type ClusterBinding struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +required + // +kubebuilder:validation:Required + Spec BindingSpec `json:"spec"` + + // +optional + Status BindingStatus `json:"status,omitempty"` +} + +// BindingStatus is the observed state shared by ClusterBinding and Binding. +type BindingStatus struct { + // boundAPIs is per-API observed state. + // + // +optional + // +listType=atomic + BoundAPIs []BoundAPI `json:"boundAPIs,omitempty"` + + // conditions: Connected, Synced, Conflicts, PermissionDenied, Ready. + // + // +optional + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +// ClusterBindingList contains a list of ClusterBinding. +// +// +kubebuilder:object:root=true +type ClusterBindingList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []ClusterBinding `json:"items"` +} diff --git a/v2/sdk/apis/core/v1alpha1/conditions.go b/v2/sdk/apis/core/v1alpha1/conditions.go new file mode 100644 index 000000000..3e8240431 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/conditions.go @@ -0,0 +1,70 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +// Condition types used across the core kinds. v2 uses metav1.Condition; the +// konnector sets these with apimachinery's meta.SetStatusCondition helpers. +const ( + // ConditionReady is the top-level summary condition on every core kind. + ConditionReady = "Ready" + + // Connection conditions. + + // ConditionSecretValid is true when the kubeconfigSecretRef resolves to a + // usable kubeconfig in the konnector's namespace. + ConditionSecretValid = "SecretValid" + // ConditionConnected is true when the konnector can reach the provider and + // the pinned cluster identity still matches. + ConditionConnected = "Connected" + // ConditionSchemaInSync is true when exported schemas are installed on the + // consumer per the Connection's schema policy. + ConditionSchemaInSync = "SchemaInSync" + + // Binding conditions. + + // ConditionSynced is true when all listed APIs are installed and their + // instances are syncing. + ConditionSynced = "Synced" + // ConditionConflicts is true when at least one object was skipped due to + // foreign ownership; see boundAPIs[].conflictCount and per-object conditions. + ConditionConflicts = "Conflicts" + // ConditionPermissionDenied is true when an operation was refused by + // provider RBAC; the binding keeps syncing what it can. + ConditionPermissionDenied = "PermissionDenied" +) + +// Condition reasons. +const ( + // ReasonAsExpected is the success reason for a True Ready/Synced condition. + ReasonAsExpected = "AsExpected" + // ReasonPending marks a missing-but-expected dependency (referenced + // Connection/Secret/CRD not present yet). Not an error; resolves on arrival. + ReasonPending = "Pending" + // ReasonSecretNotFound marks an unresolvable kubeconfigSecretRef. + ReasonSecretNotFound = "SecretNotFound" + // ReasonClusterIdentityChanged marks a Secret now pointing at a different + // provider cluster than the one pinned in status. + ReasonClusterIdentityChanged = "ClusterIdentityChanged" + // ReasonAPINotExported marks a binding listing an API the Connection does + // not export to these credentials. + ReasonAPINotExported = "APINotExported" + // ReasonForeignObjectExists marks a conflict with an un-owned target object. + ReasonForeignObjectExists = "ForeignObjectExists" + // ReasonOwnedByAnother marks a conflict with an object owned by a different + // binding/consumer. + ReasonOwnedByAnother = "OwnedByAnother" +) diff --git a/v2/sdk/apis/core/v1alpha1/connection_types.go b/v2/sdk/apis/core/v1alpha1/connection_types.go new file mode 100644 index 000000000..8004c001a --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/connection_types.go @@ -0,0 +1,168 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// Connection is the link to one provider cluster. It owns the credentials and +// schema delivery, and surfaces what the provider exports. Cluster-scoped. +// +// +kubebuilder:object:root=true +// +kubebuilder:resource:scope=Cluster,categories=kube-bind +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` +// +kubebuilder:printcolumn:name="Secret",type=string,JSONPath=`.spec.kubeconfigSecretRef.name` +// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` +type Connection struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +required + // +kubebuilder:validation:Required + Spec ConnectionSpec `json:"spec"` + + // +optional + Status ConnectionStatus `json:"status,omitempty"` +} + +// ConnectionSpec defines the desired provider link. +type ConnectionSpec struct { + // kubeconfigSecretRef points at the Secret holding the provider kubeconfig. + // Immutable. The only credential reference in the core. + // + // +required + // +kubebuilder:validation:Required + // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="kubeconfigSecretRef is immutable" + KubeconfigSecretRef SecretKeyRef `json:"kubeconfigSecretRef"` + + // schema controls how exported APIs reach the consumer. + // + // +optional + // +kubebuilder:default={} + Schema SchemaPolicy `json:"schema,omitempty"` + + // autoBind, when true, makes the konnector maintain a managed ClusterBinding + // (named after this Connection) covering all exported APIs. + // + // +kubebuilder:default=false + AutoBind bool `json:"autoBind,omitempty"` +} + +// SchemaPolicy controls schema source, pull and update behavior. +type SchemaPolicy struct { + // source selects how schemas are obtained: + // Auto - CRD if readable on the provider, else OpenAPI (default). + // CRD - read apiextensions CRDs verbatim. + // OpenAPI - synthesize CRDs from discovery + /openapi/v3. + // + // +kubebuilder:validation:Enum=Auto;CRD;OpenAPI + // +kubebuilder:default=Auto + Source SchemaSource `json:"source,omitempty"` + + // pullPolicy selects which exported APIs are installed: + // Bound - only APIs referenced by a Binding/ClusterBinding (default). + // All - every exported API readable by the credentials. + // None - never install CRDs (user/extension manages them). + // + // +kubebuilder:validation:Enum=Bound;All;None + // +kubebuilder:default=Bound + PullPolicy PullPolicy `json:"pullPolicy,omitempty"` + + // updatePolicy selects whether installed schemas follow provider changes: + // Always - follow provider schema changes (default). + // Once - pin at first pull. + // + // +kubebuilder:validation:Enum=Always;Once + // +kubebuilder:default=Always + UpdatePolicy UpdatePolicy `json:"updatePolicy,omitempty"` +} + +// SchemaSource selects how schemas are obtained from the provider. +type SchemaSource string + +const ( + // SchemaSourceAuto probes CRD first, falls back to OpenAPI. + SchemaSourceAuto SchemaSource = "Auto" + // SchemaSourceCRD reads apiextensions CRDs from the provider. + SchemaSourceCRD SchemaSource = "CRD" + // SchemaSourceOpenAPI synthesizes CRDs from discovery + /openapi/v3. + SchemaSourceOpenAPI SchemaSource = "OpenAPI" +) + +// PullPolicy selects which exported APIs are installed on the consumer. +type PullPolicy string + +const ( + // PullPolicyBound installs only APIs referenced by a binding. + PullPolicyBound PullPolicy = "Bound" + // PullPolicyAll installs every exported API. + PullPolicyAll PullPolicy = "All" + // PullPolicyNone never installs CRDs. + PullPolicyNone PullPolicy = "None" +) + +// UpdatePolicy selects whether installed schemas track provider changes. +type UpdatePolicy string + +const ( + // UpdatePolicyAlways follows provider schema changes. + UpdatePolicyAlways UpdatePolicy = "Always" + // UpdatePolicyOnce pins the schema at first pull. + UpdatePolicyOnce UpdatePolicy = "Once" +) + +// ConnectionStatus is the observed state of a Connection. +type ConnectionStatus struct { + // remoteClusterUID is the identity of the provider cluster, pinned on first + // connect and immutable thereafter. A Secret later pointing at a different + // cluster is rejected rather than silently re-homing synced objects. + // + // +optional + // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="remoteClusterUID is immutable" + RemoteClusterUID string `json:"remoteClusterUID,omitempty"` + + // localClusterUID is the identity of the consumer cluster, pinned on first + // connect and immutable thereafter. + // + // +optional + // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="localClusterUID is immutable" + LocalClusterUID string `json:"localClusterUID,omitempty"` + + // exportedAPIs is the discovery result: APIs exported to these credentials. + // + // +optional + // +listType=atomic + ExportedAPIs []ExportedAPI `json:"exportedAPIs,omitempty"` + + // conditions: SecretValid, Connected, SchemaInSync, Ready. + // + // +optional + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +// ConnectionList contains a list of Connection. +// +// +kubebuilder:object:root=true +type ConnectionList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []Connection `json:"items"` +} diff --git a/v2/sdk/apis/core/v1alpha1/doc.go b/v2/sdk/apis/core/v1alpha1/doc.go new file mode 100644 index 000000000..89c3ab2c8 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/doc.go @@ -0,0 +1,23 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package v1alpha1 contains the v2 "slim core" API for kube-bind: +// the Connection, ClusterBinding and Binding kinds in the core.kube-bind.io +// group. See docs/proposals/v2-slim-core.md for the design. +// +// +kubebuilder:object:generate=true +// +groupName=core.kube-bind.io +package v1alpha1 diff --git a/v2/sdk/apis/core/v1alpha1/groupversion_info.go b/v2/sdk/apis/core/v1alpha1/groupversion_info.go new file mode 100644 index 000000000..e80ea8de7 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/groupversion_info.go @@ -0,0 +1,54 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +const ( + // GroupName is the API group for the v2 slim core. + GroupName = "core.kube-bind.io" + + // Version is the API version of this package. + Version = "v1alpha1" +) + +var ( + // GroupVersion is group version used to register these objects. + GroupVersion = schema.GroupVersion{Group: GroupName, Version: Version} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme. + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) + +// Resource takes an unqualified resource and returns a Group qualified GroupResource. +func Resource(resource string) schema.GroupResource { + return GroupVersion.WithResource(resource).GroupResource() +} + +func init() { + SchemeBuilder.Register( + &Connection{}, &ConnectionList{}, + &ClusterBinding{}, &ClusterBindingList{}, + &Binding{}, &BindingList{}, + ) +} diff --git a/v2/sdk/apis/core/v1alpha1/helpers.go b/v2/sdk/apis/core/v1alpha1/helpers.go new file mode 100644 index 000000000..2a60b8495 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/helpers.go @@ -0,0 +1,62 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +// BindingAccessor is implemented by both ClusterBinding and Binding so a single +// reconciler can drive either kind. The only difference between them is scope. +// It is also a client.Object (metav1.Object + runtime.Object). +type BindingAccessor interface { + metav1.Object + runtime.Object + // BindingSpecP returns a pointer to the shared spec. + BindingSpecP() *BindingSpec + // BindingStatusP returns a pointer to the shared status. + BindingStatusP() *BindingStatus +} + +// BindingSpecP returns a pointer to the ClusterBinding spec. +func (b *ClusterBinding) BindingSpecP() *BindingSpec { return &b.Spec } + +// BindingStatusP returns a pointer to the ClusterBinding status. +func (b *ClusterBinding) BindingStatusP() *BindingStatus { return &b.Status } + +// BindingSpecP returns a pointer to the Binding spec. +func (b *Binding) BindingSpecP() *BindingSpec { return &b.Spec } + +// BindingStatusP returns a pointer to the Binding status. +func (b *Binding) BindingStatusP() *BindingStatus { return &b.Status } + +var ( + _ BindingAccessor = &ClusterBinding{} + _ BindingAccessor = &Binding{} +) + +// ExportsAPI reports whether the Connection exports an API with the given CRD +// name ("."). +func (s *ConnectionStatus) ExportsAPI(crdName string) (ExportedAPI, bool) { + for i := range s.ExportedAPIs { + if s.ExportedAPIs[i].Name == crdName { + return s.ExportedAPIs[i], true + } + } + return ExportedAPI{}, false +} diff --git a/v2/sdk/apis/core/v1alpha1/labels.go b/v2/sdk/apis/core/v1alpha1/labels.go new file mode 100644 index 000000000..54e6d4e69 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/labels.go @@ -0,0 +1,41 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +const ( + // LabelExported marks a provider CRD as exported to consumers. The + // konnector's discovery (schema.source: CRD) lists CRDs carrying this label. + // On CRD-less providers the logical-cluster boundary is the export boundary + // instead, and no label is needed. + LabelExported = "core.kube-bind.io/exported" + + // LabelManaged marks an object (or namespace) written by the konnector on + // the provider. + LabelManaged = "core.kube-bind.io/managed" + + // AnnotationConsumerClusterUID records the consumer cluster identity on a + // synced provider object (ownership marker). + AnnotationConsumerClusterUID = "core.kube-bind.io/consumer-cluster-uid" + + // AnnotationConsumerObjectUID records the source consumer object UID on a + // synced provider object (ownership marker). + AnnotationConsumerObjectUID = "core.kube-bind.io/consumer-object-uid" + + // FinalizerSyncer blocks consumer-object deletion until the provider copy + // has been removed. + FinalizerSyncer = "core.kube-bind.io/syncer" +) diff --git a/v2/sdk/apis/core/v1alpha1/shared_types.go b/v2/sdk/apis/core/v1alpha1/shared_types.go new file mode 100644 index 000000000..7cfcaf696 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/shared_types.go @@ -0,0 +1,209 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// SecretKeyRef references a key within a Secret. The Secret must live in the +// konnector's designated namespace; a cluster-scoped Connection may not reach +// into arbitrary namespaces (privilege-escalation guard, see proposal F1). +type SecretKeyRef struct { + // namespace of the Secret. Must be the konnector's designated namespace. + // + // +required + // +kubebuilder:validation:Required + // +kubebuilder:validation:MinLength=1 + Namespace string `json:"namespace"` + + // name of the Secret. + // + // +required + // +kubebuilder:validation:Required + // +kubebuilder:validation:MinLength=1 + Name string `json:"name"` + + // key within the Secret holding the kubeconfig. + // + // +kubebuilder:default=kubeconfig + Key string `json:"key,omitempty"` +} + +// ConnectionRef references a Connection by name. Connection is cluster-scoped, +// so no namespace is needed. +type ConnectionRef struct { + // name of the Connection. + // + // +required + // +kubebuilder:validation:Required + // +kubebuilder:validation:MinLength=1 + Name string `json:"name"` +} + +// APIRef identifies an exported API by its CRD name on the provider, +// i.e. "." (for example "mangodbs.mangodb.io"). Under the +// OpenAPI schema source there is no CRD object behind the name; it is still +// just resource + group. +type APIRef struct { + // name is the CRD name of the exported API, ".". + // + // +required + // +kubebuilder:validation:Required + // +kubebuilder:validation:MinLength=1 + Name string `json:"name"` +} + +// BindingSpec is the spec shared by ClusterBinding and Binding. The only +// difference between the two kinds is scope (cluster-wide vs. one namespace), +// which is expressed by the kind, not by a field. +type BindingSpec struct { + // connectionRef points at the Connection that provides the provider link + // and credentials for the listed APIs. + // + // +required + // +kubebuilder:validation:Required + ConnectionRef ConnectionRef `json:"connectionRef"` + + // apis lists one or more exported CRDs to sync, by CRD name on the provider. + // + // +required + // +kubebuilder:validation:Required + // +kubebuilder:validation:MinItems=1 + APIs []APIRef `json:"apis"` + + // conflictPolicy controls what happens when a target object already exists. + // Fail (default) leaves a foreign object untouched and records a conflict; + // Adopt takes ownership of an un-owned object. Adopt never steals an object + // already carrying another binding's/consumer's markers. + // + // +kubebuilder:validation:Enum=Fail;Adopt + // +kubebuilder:default=Fail + ConflictPolicy ConflictPolicy `json:"conflictPolicy,omitempty"` + + // relatedResources are Secrets/ConfigMaps synced alongside instances of the + // bound APIs. Not yet synced in the alpha POC. + // + // +optional + // +listType=atomic + RelatedResources []RelatedResource `json:"relatedResources,omitempty"` +} + +// ConflictPolicy is the behavior for pre-existing target objects. +type ConflictPolicy string + +const ( + // ConflictPolicyFail leaves a foreign target object untouched and records + // the collision as a conflict. This is the default. + ConflictPolicyFail ConflictPolicy = "Fail" + + // ConflictPolicyAdopt takes ownership of an un-owned target object. + ConflictPolicyAdopt ConflictPolicy = "Adopt" +) + +// RelatedResource selects auxiliary objects (Secrets/ConfigMaps) to sync +// alongside the bound instances. +type RelatedResource struct { + // group of the related resource. Empty string for the core group. + // + // +kubebuilder:default="" + Group string `json:"group"` + + // resource is the plural resource name. Only "secrets" and "configmaps" + // are permitted in core. + // + // +kubebuilder:validation:Enum=secrets;configmaps + // +required + // +kubebuilder:validation:Required + Resource string `json:"resource"` + + // direction the related resource flows. + // + // +kubebuilder:validation:Enum=FromProvider;FromConsumer + // +required + // +kubebuilder:validation:Required + Direction SyncDirection `json:"direction"` + + // selector restricts which objects are synced. Only labelSelector and + // named selectors are supported in core (no JSONPath reference-following). + // + // +optional + Selector *RelatedResourceSelector `json:"selector,omitempty"` +} + +// SyncDirection is the direction a related resource is synced. +type SyncDirection string + +const ( + // FromProvider syncs the related resource provider -> consumer. + FromProvider SyncDirection = "FromProvider" + // FromConsumer syncs the related resource consumer -> provider. + FromConsumer SyncDirection = "FromConsumer" +) + +// RelatedResourceSelector restricts which related objects are synced. +type RelatedResourceSelector struct { + // labelSelector selects related objects by label. + // + // +optional + LabelSelector *metav1.LabelSelector `json:"labelSelector,omitempty"` + + // names selects related objects by exact name. + // + // +optional + // +listType=set + Names []string `json:"names,omitempty"` +} + +// BoundAPI is the per-API observed state recorded on a binding's status. +type BoundAPI struct { + // name is the CRD name of the bound API, ".". + Name string `json:"name"` + + // crdHash is a hash of the schema currently applied on the consumer for + // this API. Empty until the schema has been installed. + // + // +optional + CRDHash string `json:"crdHash,omitempty"` + + // conflictCount is the number of objects skipped due to foreign ownership. + // Per-object detail lives on each object's own condition, not here. + // + // +optional + ConflictCount int32 `json:"conflictCount,omitempty"` +} + +// ExportedAPI describes one API the provider exports to these credentials. +type ExportedAPI struct { + // name is the CRD name of the exported API, ".". + Name string `json:"name"` + + // group of the exported API. + Group string `json:"group"` + + // resource is the plural resource name of the exported API. + Resource string `json:"resource"` + + // scope is whether the API is Namespaced or Cluster scoped. + Scope apiextensionsv1.ResourceScope `json:"scope"` + + // versions are the served versions of the exported API. + // + // +listType=set + Versions []string `json:"versions"` +} diff --git a/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go b/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 000000000..e5d637040 --- /dev/null +++ b/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,428 @@ +//go:build !ignore_autogenerated + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *APIRef) DeepCopyInto(out *APIRef) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new APIRef. +func (in *APIRef) DeepCopy() *APIRef { + if in == nil { + return nil + } + out := new(APIRef) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Binding) DeepCopyInto(out *Binding) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Binding. +func (in *Binding) DeepCopy() *Binding { + if in == nil { + return nil + } + out := new(Binding) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *Binding) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *BindingList) DeepCopyInto(out *BindingList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]Binding, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BindingList. +func (in *BindingList) DeepCopy() *BindingList { + if in == nil { + return nil + } + out := new(BindingList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *BindingList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *BindingSpec) DeepCopyInto(out *BindingSpec) { + *out = *in + out.ConnectionRef = in.ConnectionRef + if in.APIs != nil { + in, out := &in.APIs, &out.APIs + *out = make([]APIRef, len(*in)) + copy(*out, *in) + } + if in.RelatedResources != nil { + in, out := &in.RelatedResources, &out.RelatedResources + *out = make([]RelatedResource, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BindingSpec. +func (in *BindingSpec) DeepCopy() *BindingSpec { + if in == nil { + return nil + } + out := new(BindingSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *BindingStatus) DeepCopyInto(out *BindingStatus) { + *out = *in + if in.BoundAPIs != nil { + in, out := &in.BoundAPIs, &out.BoundAPIs + *out = make([]BoundAPI, len(*in)) + copy(*out, *in) + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BindingStatus. +func (in *BindingStatus) DeepCopy() *BindingStatus { + if in == nil { + return nil + } + out := new(BindingStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *BoundAPI) DeepCopyInto(out *BoundAPI) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BoundAPI. +func (in *BoundAPI) DeepCopy() *BoundAPI { + if in == nil { + return nil + } + out := new(BoundAPI) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ClusterBinding) DeepCopyInto(out *ClusterBinding) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ClusterBinding. +func (in *ClusterBinding) DeepCopy() *ClusterBinding { + if in == nil { + return nil + } + out := new(ClusterBinding) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ClusterBinding) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ClusterBindingList) DeepCopyInto(out *ClusterBindingList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]ClusterBinding, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ClusterBindingList. +func (in *ClusterBindingList) DeepCopy() *ClusterBindingList { + if in == nil { + return nil + } + out := new(ClusterBindingList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ClusterBindingList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Connection) DeepCopyInto(out *Connection) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + out.Spec = in.Spec + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Connection. +func (in *Connection) DeepCopy() *Connection { + if in == nil { + return nil + } + out := new(Connection) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *Connection) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionList) DeepCopyInto(out *ConnectionList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]Connection, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionList. +func (in *ConnectionList) DeepCopy() *ConnectionList { + if in == nil { + return nil + } + out := new(ConnectionList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ConnectionList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionRef) DeepCopyInto(out *ConnectionRef) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionRef. +func (in *ConnectionRef) DeepCopy() *ConnectionRef { + if in == nil { + return nil + } + out := new(ConnectionRef) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionSpec) DeepCopyInto(out *ConnectionSpec) { + *out = *in + out.KubeconfigSecretRef = in.KubeconfigSecretRef + out.Schema = in.Schema +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionSpec. +func (in *ConnectionSpec) DeepCopy() *ConnectionSpec { + if in == nil { + return nil + } + out := new(ConnectionSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionStatus) DeepCopyInto(out *ConnectionStatus) { + *out = *in + if in.ExportedAPIs != nil { + in, out := &in.ExportedAPIs, &out.ExportedAPIs + *out = make([]ExportedAPI, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStatus. +func (in *ConnectionStatus) DeepCopy() *ConnectionStatus { + if in == nil { + return nil + } + out := new(ConnectionStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ExportedAPI) DeepCopyInto(out *ExportedAPI) { + *out = *in + if in.Versions != nil { + in, out := &in.Versions, &out.Versions + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExportedAPI. +func (in *ExportedAPI) DeepCopy() *ExportedAPI { + if in == nil { + return nil + } + out := new(ExportedAPI) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RelatedResource) DeepCopyInto(out *RelatedResource) { + *out = *in + if in.Selector != nil { + in, out := &in.Selector, &out.Selector + *out = new(RelatedResourceSelector) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RelatedResource. +func (in *RelatedResource) DeepCopy() *RelatedResource { + if in == nil { + return nil + } + out := new(RelatedResource) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RelatedResourceSelector) DeepCopyInto(out *RelatedResourceSelector) { + *out = *in + if in.LabelSelector != nil { + in, out := &in.LabelSelector, &out.LabelSelector + *out = new(v1.LabelSelector) + (*in).DeepCopyInto(*out) + } + if in.Names != nil { + in, out := &in.Names, &out.Names + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RelatedResourceSelector. +func (in *RelatedResourceSelector) DeepCopy() *RelatedResourceSelector { + if in == nil { + return nil + } + out := new(RelatedResourceSelector) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SchemaPolicy) DeepCopyInto(out *SchemaPolicy) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SchemaPolicy. +func (in *SchemaPolicy) DeepCopy() *SchemaPolicy { + if in == nil { + return nil + } + out := new(SchemaPolicy) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SecretKeyRef) DeepCopyInto(out *SecretKeyRef) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SecretKeyRef. +func (in *SecretKeyRef) DeepCopy() *SecretKeyRef { + if in == nil { + return nil + } + out := new(SecretKeyRef) + in.DeepCopyInto(out) + return out +} diff --git a/v2/sdk/config/crd/core.kube-bind.io_bindings.yaml b/v2/sdk/config/crd/core.kube-bind.io_bindings.yaml new file mode 100644 index 000000000..946edc752 --- /dev/null +++ b/v2/sdk/config/crd/core.kube-bind.io_bindings.yaml @@ -0,0 +1,297 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: bindings.core.kube-bind.io +spec: + group: core.kube-bind.io + names: + categories: + - kube-bind + kind: Binding + listKind: BindingList + plural: bindings + singular: binding + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.connectionRef.name + name: Connection + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + Binding activates instance sync for one or more exported APIs within its own + namespace. Namespaced, following the Role convention. It is the v2 answer to + v1's informerScope: Namespaced. Where a ClusterBinding and a Binding cover the + same API on the same connection, the ClusterBinding wins. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + BindingSpec is the spec shared by ClusterBinding and Binding. The only + difference between the two kinds is scope (cluster-wide vs. one namespace), + which is expressed by the kind, not by a field. + properties: + apis: + description: apis lists one or more exported CRDs to sync, by CRD + name on the provider. + items: + description: |- + APIRef identifies an exported API by its CRD name on the provider, + i.e. "." (for example "mangodbs.mangodb.io"). Under the + OpenAPI schema source there is no CRD object behind the name; it is still + just resource + group. + properties: + name: + description: name is the CRD name of the exported API, ".". + minLength: 1 + type: string + required: + - name + type: object + minItems: 1 + type: array + conflictPolicy: + default: Fail + description: |- + conflictPolicy controls what happens when a target object already exists. + Fail (default) leaves a foreign object untouched and records a conflict; + Adopt takes ownership of an un-owned object. Adopt never steals an object + already carrying another binding's/consumer's markers. + enum: + - Fail + - Adopt + type: string + connectionRef: + description: |- + connectionRef points at the Connection that provides the provider link + and credentials for the listed APIs. + properties: + name: + description: name of the Connection. + minLength: 1 + type: string + required: + - name + type: object + relatedResources: + description: |- + relatedResources are Secrets/ConfigMaps synced alongside instances of the + bound APIs. Not yet synced in the alpha POC. + items: + description: |- + RelatedResource selects auxiliary objects (Secrets/ConfigMaps) to sync + alongside the bound instances. + properties: + direction: + description: direction the related resource flows. + enum: + - FromProvider + - FromConsumer + type: string + group: + default: "" + description: group of the related resource. Empty string for + the core group. + type: string + resource: + description: |- + resource is the plural resource name. Only "secrets" and "configmaps" + are permitted in core. + enum: + - secrets + - configmaps + type: string + selector: + description: |- + selector restricts which objects are synced. Only labelSelector and + named selectors are supported in core (no JSONPath reference-following). + properties: + labelSelector: + description: labelSelector selects related objects by label. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + names: + description: names selects related objects by exact name. + items: + type: string + type: array + x-kubernetes-list-type: set + type: object + required: + - direction + - group + - resource + type: object + type: array + x-kubernetes-list-type: atomic + required: + - apis + - connectionRef + type: object + status: + description: BindingStatus is the observed state shared by ClusterBinding + and Binding. + properties: + boundAPIs: + description: boundAPIs is per-API observed state. + items: + description: BoundAPI is the per-API observed state recorded on + a binding's status. + properties: + conflictCount: + description: |- + conflictCount is the number of objects skipped due to foreign ownership. + Per-object detail lives on each object's own condition, not here. + format: int32 + type: integer + crdHash: + description: |- + crdHash is a hash of the schema currently applied on the consumer for + this API. Empty until the schema has been installed. + type: string + name: + description: name is the CRD name of the bound API, ".". + type: string + required: + - name + type: object + type: array + x-kubernetes-list-type: atomic + conditions: + description: 'conditions: Connected, Synced, Conflicts, PermissionDenied, + Ready.' + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/v2/sdk/config/crd/core.kube-bind.io_clusterbindings.yaml b/v2/sdk/config/crd/core.kube-bind.io_clusterbindings.yaml new file mode 100644 index 000000000..5506921f1 --- /dev/null +++ b/v2/sdk/config/crd/core.kube-bind.io_clusterbindings.yaml @@ -0,0 +1,295 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: clusterbindings.core.kube-bind.io +spec: + group: core.kube-bind.io + names: + categories: + - kube-bind + kind: ClusterBinding + listKind: ClusterBindingList + plural: clusterbindings + singular: clusterbinding + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .spec.connectionRef.name + name: Connection + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + ClusterBinding activates instance sync for one or more exported APIs + cluster-wide. Cluster-scoped, following the ClusterRole convention. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + BindingSpec is the spec shared by ClusterBinding and Binding. The only + difference between the two kinds is scope (cluster-wide vs. one namespace), + which is expressed by the kind, not by a field. + properties: + apis: + description: apis lists one or more exported CRDs to sync, by CRD + name on the provider. + items: + description: |- + APIRef identifies an exported API by its CRD name on the provider, + i.e. "." (for example "mangodbs.mangodb.io"). Under the + OpenAPI schema source there is no CRD object behind the name; it is still + just resource + group. + properties: + name: + description: name is the CRD name of the exported API, ".". + minLength: 1 + type: string + required: + - name + type: object + minItems: 1 + type: array + conflictPolicy: + default: Fail + description: |- + conflictPolicy controls what happens when a target object already exists. + Fail (default) leaves a foreign object untouched and records a conflict; + Adopt takes ownership of an un-owned object. Adopt never steals an object + already carrying another binding's/consumer's markers. + enum: + - Fail + - Adopt + type: string + connectionRef: + description: |- + connectionRef points at the Connection that provides the provider link + and credentials for the listed APIs. + properties: + name: + description: name of the Connection. + minLength: 1 + type: string + required: + - name + type: object + relatedResources: + description: |- + relatedResources are Secrets/ConfigMaps synced alongside instances of the + bound APIs. Not yet synced in the alpha POC. + items: + description: |- + RelatedResource selects auxiliary objects (Secrets/ConfigMaps) to sync + alongside the bound instances. + properties: + direction: + description: direction the related resource flows. + enum: + - FromProvider + - FromConsumer + type: string + group: + default: "" + description: group of the related resource. Empty string for + the core group. + type: string + resource: + description: |- + resource is the plural resource name. Only "secrets" and "configmaps" + are permitted in core. + enum: + - secrets + - configmaps + type: string + selector: + description: |- + selector restricts which objects are synced. Only labelSelector and + named selectors are supported in core (no JSONPath reference-following). + properties: + labelSelector: + description: labelSelector selects related objects by label. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + names: + description: names selects related objects by exact name. + items: + type: string + type: array + x-kubernetes-list-type: set + type: object + required: + - direction + - group + - resource + type: object + type: array + x-kubernetes-list-type: atomic + required: + - apis + - connectionRef + type: object + status: + description: BindingStatus is the observed state shared by ClusterBinding + and Binding. + properties: + boundAPIs: + description: boundAPIs is per-API observed state. + items: + description: BoundAPI is the per-API observed state recorded on + a binding's status. + properties: + conflictCount: + description: |- + conflictCount is the number of objects skipped due to foreign ownership. + Per-object detail lives on each object's own condition, not here. + format: int32 + type: integer + crdHash: + description: |- + crdHash is a hash of the schema currently applied on the consumer for + this API. Empty until the schema has been installed. + type: string + name: + description: name is the CRD name of the bound API, ".". + type: string + required: + - name + type: object + type: array + x-kubernetes-list-type: atomic + conditions: + description: 'conditions: Connected, Synced, Conflicts, PermissionDenied, + Ready.' + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/v2/sdk/config/crd/core.kube-bind.io_connections.yaml b/v2/sdk/config/crd/core.kube-bind.io_connections.yaml new file mode 100644 index 000000000..29daa8109 --- /dev/null +++ b/v2/sdk/config/crd/core.kube-bind.io_connections.yaml @@ -0,0 +1,253 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: connections.core.kube-bind.io +spec: + group: core.kube-bind.io + names: + categories: + - kube-bind + kind: Connection + listKind: ConnectionList + plural: connections + singular: connection + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .spec.kubeconfigSecretRef.name + name: Secret + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + Connection is the link to one provider cluster. It owns the credentials and + schema delivery, and surfaces what the provider exports. Cluster-scoped. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: ConnectionSpec defines the desired provider link. + properties: + autoBind: + default: false + description: |- + autoBind, when true, makes the konnector maintain a managed ClusterBinding + (named after this Connection) covering all exported APIs. + type: boolean + kubeconfigSecretRef: + description: |- + kubeconfigSecretRef points at the Secret holding the provider kubeconfig. + Immutable. The only credential reference in the core. + properties: + key: + default: kubeconfig + description: key within the Secret holding the kubeconfig. + type: string + name: + description: name of the Secret. + minLength: 1 + type: string + namespace: + description: namespace of the Secret. Must be the konnector's + designated namespace. + minLength: 1 + type: string + required: + - name + - namespace + type: object + x-kubernetes-validations: + - message: kubeconfigSecretRef is immutable + rule: self == oldSelf + schema: + default: {} + description: schema controls how exported APIs reach the consumer. + properties: + pullPolicy: + default: Bound + description: |- + pullPolicy selects which exported APIs are installed: + Bound - only APIs referenced by a Binding/ClusterBinding (default). + All - every exported API readable by the credentials. + None - never install CRDs (user/extension manages them). + enum: + - Bound + - All + - None + type: string + source: + default: Auto + description: |- + source selects how schemas are obtained: + Auto - CRD if readable on the provider, else OpenAPI (default). + CRD - read apiextensions CRDs verbatim. + OpenAPI - synthesize CRDs from discovery + /openapi/v3. + enum: + - Auto + - CRD + - OpenAPI + type: string + updatePolicy: + default: Always + description: |- + updatePolicy selects whether installed schemas follow provider changes: + Always - follow provider schema changes (default). + Once - pin at first pull. + enum: + - Always + - Once + type: string + type: object + required: + - kubeconfigSecretRef + type: object + status: + description: ConnectionStatus is the observed state of a Connection. + properties: + conditions: + description: 'conditions: SecretValid, Connected, SchemaInSync, Ready.' + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + exportedAPIs: + description: 'exportedAPIs is the discovery result: APIs exported + to these credentials.' + items: + description: ExportedAPI describes one API the provider exports + to these credentials. + properties: + group: + description: group of the exported API. + type: string + name: + description: name is the CRD name of the exported API, ".". + type: string + resource: + description: resource is the plural resource name of the exported + API. + type: string + scope: + description: scope is whether the API is Namespaced or Cluster + scoped. + type: string + versions: + description: versions are the served versions of the exported + API. + items: + type: string + type: array + x-kubernetes-list-type: set + required: + - group + - name + - resource + - scope + - versions + type: object + type: array + x-kubernetes-list-type: atomic + localClusterUID: + description: |- + localClusterUID is the identity of the consumer cluster, pinned on first + connect and immutable thereafter. + type: string + x-kubernetes-validations: + - message: localClusterUID is immutable + rule: self == oldSelf + remoteClusterUID: + description: |- + remoteClusterUID is the identity of the provider cluster, pinned on first + connect and immutable thereafter. A Secret later pointing at a different + cluster is rejected rather than silently re-homing synced objects. + type: string + x-kubernetes-validations: + - message: remoteClusterUID is immutable + rule: self == oldSelf + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/v2/sdk/go.mod b/v2/sdk/go.mod new file mode 100644 index 000000000..f1d3f342d --- /dev/null +++ b/v2/sdk/go.mod @@ -0,0 +1,30 @@ +module github.com/kube-bind/kube-bind/v2/sdk + +go 1.24.0 + +require ( + k8s.io/apiextensions-apiserver v0.33.4 + k8s.io/apimachinery v0.33.4 + sigs.k8s.io/controller-runtime v0.21.0 +) + +require ( + github.com/fxamacker/cbor/v2 v2.7.0 // indirect + github.com/go-logr/logr v1.4.2 // indirect + github.com/gogo/protobuf v1.3.2 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/kr/pretty v0.3.1 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/x448/float16 v0.8.4 // indirect + golang.org/x/net v0.38.0 // indirect + golang.org/x/text v0.23.0 // indirect + gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect + gopkg.in/inf.v0 v0.9.1 // indirect + k8s.io/klog/v2 v2.130.1 // indirect + k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect + sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect + sigs.k8s.io/randfill v1.0.0 // indirect + sigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect + sigs.k8s.io/yaml v1.4.0 // indirect +) diff --git a/v2/sdk/go.sum b/v2/sdk/go.sum new file mode 100644 index 000000000..e60cd8e92 --- /dev/null +++ b/v2/sdk/go.sum @@ -0,0 +1,105 @@ +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E= +github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ= +github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= +github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI= +github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db h1:097atOisP2aRj7vFgYQBbFN4U4JNXUNYpxael3UzMyo= +github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db/go.mod h1:vavhavw2zAxS5dIdcRluK6cSGGPlZynqzFM8NdvU144= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= +github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/onsi/ginkgo/v2 v2.22.0 h1:Yed107/8DjTr0lKCNt7Dn8yQ6ybuDRQoMGrNFKzMfHg= +github.com/onsi/ginkgo/v2 v2.22.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo= +github.com/onsi/gomega v1.36.1 h1:bJDPBO7ibjxcbHMgSCoo4Yj18UWbKDlLwX1x9sybDcw= +github.com/onsi/gomega v1.36.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= +github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= +github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= +github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= +golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik= +golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY= +golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.26.0 h1:v/60pFQmzmT9ExmjDv2gGIfi3OqfKoEP6I5+umXlbnQ= +golang.org/x/tools v0.26.0/go.mod h1:TPVVj70c7JJ3WCazhD8OdXcZg/og+b9+tH/KxylGwH0= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= +gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +k8s.io/api v0.33.4 h1:oTzrFVNPXBjMu0IlpA2eDDIU49jsuEorGHB4cvKupkk= +k8s.io/api v0.33.4/go.mod h1:VHQZ4cuxQ9sCUMESJV5+Fe8bGnqAARZ08tSTdHWfeAc= +k8s.io/apiextensions-apiserver v0.33.4 h1:rtq5SeXiDbXmSwxsF0MLe2Mtv3SwprA6wp+5qh/CrOU= +k8s.io/apiextensions-apiserver v0.33.4/go.mod h1:mWXcZQkQV1GQyxeIjYApuqsn/081hhXPZwZ2URuJeSs= +k8s.io/apimachinery v0.33.4 h1:SOf/JW33TP0eppJMkIgQ+L6atlDiP/090oaX0y9pd9s= +k8s.io/apimachinery v0.33.4/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM= +k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk= +k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE= +k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 h1:M3sRQVHv7vB20Xc2ybTt7ODCeFj6JSWYFzOFnYeS6Ro= +k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +sigs.k8s.io/controller-runtime v0.21.0 h1:CYfjpEuicjUecRk+KAeyYh+ouUBn4llGyDYytIGcJS8= +sigs.k8s.io/controller-runtime v0.21.0/go.mod h1:OSg14+F65eWqIu4DceX7k/+QRAbTTvxeQSNSOQpukWM= +sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8= +sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo= +sigs.k8s.io/randfill v0.0.0-20250304075658-069ef1bbf016/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY= +sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU= +sigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY= +sigs.k8s.io/structured-merge-diff/v4 v4.6.0 h1:IUA9nvMmnKWcj5jl84xn+T5MnlZKThmUW1TdblaLVAc= +sigs.k8s.io/structured-merge-diff/v4 v4.6.0/go.mod h1:dDy58f92j70zLsuZVuUX5Wp9vtxXpaZnkPGWeqDfCps= +sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E= +sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY= From f0af66ab2a1cf085c23f91484fa8c931d3d9a087 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Thu, 11 Jun 2026 21:24:08 +0300 Subject: [PATCH 04/18] move to mcr --- v2/README.md | 15 +- v2/konnector/cmd/konnector/main.go | 2 +- v2/konnector/engine/binding/reconciler.go | 4 +- v2/konnector/engine/sync/crd_controller.go | 78 +++++++--- v2/konnector/engine/sync/syncer.go | 145 +++++++------------ v2/konnector/test/e2e/framework/framework.go | 53 +++++-- v2/konnector/test/e2e/sync_test.go | 4 +- v2/sdk/apis/core/v1alpha1/labels.go | 5 + 8 files changed, 165 insertions(+), 141 deletions(-) diff --git a/v2/README.md b/v2/README.md index 563f2b85c..28d61bc2b 100644 --- a/v2/README.md +++ b/v2/README.md @@ -34,12 +34,15 @@ v2/ consumer, and report `Ready` + `boundAPIs`. - A dynamic per-GVR syncer copies instance **spec up** (server-side apply with ownership markers + a finalizer) and **status down**, with `conflictPolicy: - Fail` surfaced on the object's own condition. - -Known POC simplifications (tracked against the proposal): status sync polls -(no provider watch yet); the sync path uses direct provider clients rather than -the mcr-engaged cluster cache; `schema.source: OpenAPI`, `Adopt`, related -resources, autoBind and the provider `Lease` are not implemented yet. + Fail` surfaced as an Event (+ best-effort object condition). +- The provider side is the **multicluster-runtime engaged cluster** for each + Connection: writes go through its client, fresh reads through its API reader, + and status/drift events arrive via a **watch on its cache** (event-driven, not + polled — a low-frequency resync is only a backstop). + +Known POC simplifications (tracked against the proposal): `schema.source: +OpenAPI`, `conflictPolicy: Adopt`, related resources, `autoBind` and the +provider `Lease` are not implemented yet. ## Build diff --git a/v2/konnector/cmd/konnector/main.go b/v2/konnector/cmd/konnector/main.go index 85c941806..abf0bfc08 100644 --- a/v2/konnector/cmd/konnector/main.go +++ b/v2/konnector/cmd/konnector/main.go @@ -106,7 +106,7 @@ func run(metricsAddr string) error { if err := (&binding.NamespacedReconciler{}).SetupWithManager(localMgr); err != nil { return err } - if err := syncengine.SetupWithManager(localMgr); err != nil { + if err := syncengine.SetupWithManager(localMgr, connProvider); err != nil { return err } diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go index 591dde44b..2c8e0b183 100644 --- a/v2/konnector/engine/binding/reconciler.go +++ b/v2/konnector/engine/binding/reconciler.go @@ -177,12 +177,10 @@ func crdForConsumer(in *apiextensionsv1.CustomResourceDefinition, connName strin if out.Annotations == nil { out.Annotations = map[string]string{} } - out.Annotations[annotationConnection] = connName + out.Annotations[corev1alpha1.AnnotationConnection] = connName return out } -const annotationConnection = "core.kube-bind.io/connection" - func crdHash(crd *apiextensionsv1.CustomResourceDefinition) string { h := sha256.New() for _, v := range crd.Spec.Versions { diff --git a/v2/konnector/engine/sync/crd_controller.go b/v2/konnector/engine/sync/crd_controller.go index 0e216ac8c..be1236cf0 100644 --- a/v2/konnector/engine/sync/crd_controller.go +++ b/v2/konnector/engine/sync/crd_controller.go @@ -42,6 +42,7 @@ import ( "k8s.io/client-go/util/workqueue" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/cluster" "sigs.k8s.io/controller-runtime/pkg/controller" "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/predicate" @@ -53,12 +54,19 @@ import ( const cleanupTimeout = 10 * time.Second -// CRDController watches managed CRDs and runs a per-GVR syncer for each. +// ClusterGetter resolves a Connection name to its engaged provider cluster. +// The multicluster-runtime ConnectionProvider satisfies it. +type ClusterGetter interface { + Get(ctx context.Context, name string) (cluster.Cluster, error) +} + +// CRDController watches managed CRDs and runs a per-GVR syncer for each, wired +// to the provider's multicluster-runtime engaged cluster. type CRDController struct { mgr ctrl.Manager consumerConfig *rest.Config consumerClient client.Client - providers *providerCache + clusters ClusterGetter recorder record.EventRecorder resync time.Duration @@ -72,10 +80,10 @@ type syncContext struct { wg *sync.WaitGroup } -// SetupWithManager registers the CRD controller. It uses a direct (uncached) -// client for consumer object reads/writes inside the syncers, built from the -// manager's rest config. -func SetupWithManager(mgr ctrl.Manager) error { +// SetupWithManager registers the CRD controller. The consumer side uses a +// direct (uncached) client built from the manager's rest config; the provider +// side comes from the multicluster-runtime engaged cluster resolved via clusters. +func SetupWithManager(mgr ctrl.Manager, clusters ClusterGetter) error { consumerClient, err := client.New(mgr.GetConfig(), client.Options{Scheme: mgr.GetScheme()}) if err != nil { return fmt.Errorf("building direct consumer client: %w", err) @@ -84,7 +92,7 @@ func SetupWithManager(mgr ctrl.Manager) error { mgr: mgr, consumerConfig: mgr.GetConfig(), consumerClient: consumerClient, - providers: newProviderCache(consumerClient, mgr.GetScheme()), + clusters: clusters, recorder: mgr.GetEventRecorderFor("kube-bind-konnector"), resync: time.Minute, contexts: map[string]*syncContext{}, @@ -108,13 +116,10 @@ func (r *CRDController) Reconcile(ctx context.Context, req reconcile.Request) (r return reconcile.Result{}, err } } - if err := r.reconcile(ctx, req.Name, crd); err != nil { - return reconcile.Result{}, err - } - return reconcile.Result{}, nil + return r.reconcile(ctx, req.Name, crd) } -func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiextensionsv1.CustomResourceDefinition) error { +func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiextensionsv1.CustomResourceDefinition) (reconcile.Result, error) { log := ctrl.LoggerFrom(ctx).WithValues("crd", name) deleted := crd == nil || crd.DeletionTimestamp != nil var generation int64 @@ -127,7 +132,7 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte if found || deleted { if !deleted && c.generation >= generation { r.lock.Unlock() - return nil + return reconcile.Result{}, nil } if found { log.Info("stopping syncer") @@ -138,19 +143,39 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte } r.lock.Unlock() if deleted { - return nil + return reconcile.Result{}, nil + } + + // The provider is pinned by the binding that pulled this CRD. + connName := crd.Annotations[corev1alpha1.AnnotationConnection] + if connName == "" { + return reconcile.Result{}, fmt.Errorf("managed CRD %s has no %s annotation", name, corev1alpha1.AnnotationConnection) + } + // Resolve the multicluster-runtime engaged provider cluster. Until the + // Connection is engaged (it engages once Ready), requeue. + providerCluster, err := r.clusters.Get(ctx, connName) + if err != nil { + log.V(4).Info("provider cluster not engaged yet, requeueing", "connection", connName) + return reconcile.Result{RequeueAfter: 2 * time.Second}, nil + } + conn := &corev1alpha1.Connection{} + if err := r.consumerClient.Get(ctx, client.ObjectKey{Name: connName}, conn); err != nil { + return reconcile.Result{}, fmt.Errorf("getting Connection %s: %w", connName, err) + } + if conn.Status.LocalClusterUID == "" { + return reconcile.Result{RequeueAfter: 2 * time.Second}, nil } version := storageOrServedVersion(crd) if version == "" { - return fmt.Errorf("CRD %s has no served version", name) + return reconcile.Result{}, fmt.Errorf("CRD %s has no served version", name) } gvr := schema.GroupVersionResource{Group: crd.Spec.Group, Version: version, Resource: crd.Spec.Names.Plural} gvk := schema.GroupVersionKind{Group: crd.Spec.Group, Version: version, Kind: crd.Spec.Names.Kind} dyn, err := dynamicclient.NewForConfig(r.consumerConfig) if err != nil { - return err + return reconcile.Result{}, err } inf := dynamicinformer.NewFilteredDynamicInformer(dyn, gvr, metav1.NamespaceAll, r.resync, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, nil) lister := dynamiclister.New(inf.Informer().GetIndexer(), gvr) @@ -161,7 +186,9 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte scope: crd.Spec.Scope, consumerLister: lister, consumerClient: r.consumerClient, - providers: r.providers, + providerClient: providerCluster.GetClient(), + providerReader: providerCluster.GetAPIReader(), + localUID: conn.Status.LocalClusterUID, recorder: r.recorder, } @@ -171,18 +198,23 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte LogConstructor: func(*reconcile.Request) logr.Logger { return syncLog }, }) if err != nil { - return fmt.Errorf("creating sync controller: %w", err) + return reconcile.Result{}, fmt.Errorf("creating sync controller: %w", err) } src := newInformerSource(inf) if err := ctrlr.Watch(src); err != nil { - return fmt.Errorf("watching consumer informer: %w", err) + return reconcile.Result{}, fmt.Errorf("watching consumer informer: %w", err) + } + // Provider-side watch via the engaged cluster's cache: status/drift events + // arrive here instead of being polled. + if err := ctrlr.Watch(providerSource(providerCluster.GetCache(), gvk, conn.Status.LocalClusterUID)); err != nil { + return reconcile.Result{}, fmt.Errorf("watching provider cluster: %w", err) } if err := ctrlr.Watch(r.bindingSource(gvr.GroupResource(), lister)); err != nil { - return fmt.Errorf("watching cluster bindings: %w", err) + return reconcile.Result{}, fmt.Errorf("watching cluster bindings: %w", err) } if err := ctrlr.Watch(r.namespacedBindingSource(gvr.GroupResource(), lister)); err != nil { - return fmt.Errorf("watching bindings: %w", err) + return reconcile.Result{}, fmt.Errorf("watching bindings: %w", err) } syncCtx, cancel := context.WithCancel(ctx) @@ -197,14 +229,14 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte }() if !cache.WaitForCacheSync(syncCtx.Done(), inf.Informer().HasSynced) { cancel() - return fmt.Errorf("consumer informer for %s failed to sync", gvr) + return reconcile.Result{}, fmt.Errorf("consumer informer for %s failed to sync", gvr) } r.lock.Lock() r.contexts[name] = &syncContext{generation: generation, cancel: cancel, wg: wg} r.lock.Unlock() log.Info("started syncer", "gvr", gvr.String()) - return nil + return reconcile.Result{}, nil } func (r *CRDController) bindingSource(gr schema.GroupResource, lister dynamiclister.Lister) source.TypedSource[reconcile.Request] { diff --git a/v2/konnector/engine/sync/syncer.go b/v2/konnector/engine/sync/syncer.go index 2642b9982..8340c541c 100644 --- a/v2/konnector/engine/sync/syncer.go +++ b/v2/konnector/engine/sync/syncer.go @@ -19,7 +19,6 @@ package sync import ( "context" "fmt" - "sync" "time" corev1 "k8s.io/api/core/v1" @@ -27,64 +26,34 @@ import ( apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" - "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/client-go/dynamic/dynamiclister" "k8s.io/client-go/tools/record" ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/cache" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" - "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) const ( // fieldOwner is the server-side-apply field manager for spec writes. fieldOwner = "kube-bind-konnector" - // statusResyncInterval re-checks provider status for drift. POC uses - // polling; an event-driven provider watch is the follow-up (proposal #2). - statusResyncInterval = 30 * time.Second + // statusResyncBackstop is a low-frequency safety net. Provider-originated + // changes (status, drift) are primarily delivered by the engaged-cluster + // watch; this just heals any missed watch event. + statusResyncBackstop = 10 * time.Minute ) -// providerCache builds and caches a direct provider client per Connection. -// -// TODO(v2): use the multicluster-runtime engaged cluster's client/cache instead -// of a direct client, so per-connection lifecycle comes from the framework. -type providerCache struct { - consumer client.Client - scheme *runtime.Scheme - - mu sync.Mutex - clients map[string]client.Client -} - -func newProviderCache(consumer client.Client, scheme *runtime.Scheme) *providerCache { - return &providerCache{consumer: consumer, scheme: scheme, clients: map[string]client.Client{}} -} - -func (p *providerCache) For(ctx context.Context, connName string) (client.Client, *corev1alpha1.Connection, error) { - conn := &corev1alpha1.Connection{} - if err := p.consumer.Get(ctx, client.ObjectKey{Name: connName}, conn); err != nil { - return nil, nil, err - } - - p.mu.Lock() - defer p.mu.Unlock() - if cl, ok := p.clients[connName]; ok { - return cl, conn, nil - } - cl, err := remote.ProviderClient(ctx, p.consumer, conn, p.scheme) - if err != nil { - return nil, nil, err - } - p.clients[connName] = cl - return cl, conn, nil -} - // specReconciler syncs instances of one GVR: spec consumer->provider (SSA), -// status provider->consumer, with a finalizer and ownership markers. +// status provider->consumer, with a finalizer and ownership markers. The +// provider side is the multicluster-runtime engaged cluster for one Connection: +// writes go through its client, reads through its API reader (no cache +// staleness), and provider events arrive via a watch on its cache. type specReconciler struct { gvr schema.GroupVersionResource gvk schema.GroupVersionKind @@ -92,8 +61,28 @@ type specReconciler struct { consumerLister dynamiclister.Lister consumerClient client.Client // direct, uncached - providers *providerCache - recorder record.EventRecorder + + providerClient client.Client // engaged-cluster client: SSA + delete + providerReader client.Reader // engaged-cluster API reader: fresh reads + localUID string // consumer cluster identity for ownership markers + + recorder record.EventRecorder +} + +// providerSource turns the engaged provider cluster's cache into an event +// source: a provider object owned by us (matching localUID) enqueues its +// identity-mapped consumer object. This is what makes status/drift sync +// event-driven instead of polled. +func providerSource(c cache.Cache, gvk schema.GroupVersionKind, localUID string) source.TypedSource[reconcile.Request] { + obj := &unstructured.Unstructured{} + obj.SetGroupVersionKind(gvk) + return source.Kind(c, client.Object(obj), handler.TypedEnqueueRequestsFromMapFunc( + func(_ context.Context, o client.Object) []reconcile.Request { + if o.GetAnnotations()[corev1alpha1.AnnotationConsumerClusterUID] != localUID { + return nil // not ours (another consumer on a shared provider) + } + return []reconcile.Request{{NamespacedName: client.ObjectKey{Namespace: o.GetNamespace(), Name: o.GetName()}}} + })) } func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) { @@ -108,39 +97,21 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( } obj = obj.DeepCopy() - crdName := CRDNameForGVR(r.gvr) - res, err := ResolveConnection(ctx, r.consumerClient, crdName, req.Namespace) + // Gate: is there a ready binding covering this object? (Namespaced Bindings + // scope which namespaces sync; ClusterBindings cover everything.) + res, err := ResolveConnection(ctx, r.consumerClient, CRDNameForGVR(r.gvr), req.Namespace) if err != nil { return reconcile.Result{}, fmt.Errorf("resolving connection: %w", err) } - if !res.Found || !res.Ready { - // Not bound (or binding not ready): nothing to do; binding changes will - // re-trigger. If the object carries our finalizer with no binding, we - // still try to release it on deletion below. - if obj.GetDeletionTimestamp() == nil { - return reconcile.Result{}, nil - } - } - - var ( - providerClient client.Client - conn *corev1alpha1.Connection - ) - if res.Found && res.Ready { - providerClient, conn, err = r.providers.For(ctx, res.ConnectionName) - if err != nil { - return reconcile.Result{}, fmt.Errorf("provider client: %w", err) - } - } + bound := res.Found && res.Ready - localUID := "" - if conn != nil { - localUID = conn.Status.LocalClusterUID - } - - // Deletion. + // Deletion: always release our copy, even if the binding is already gone. if obj.GetDeletionTimestamp() != nil { - return r.reconcileDelete(ctx, obj, providerClient, localUID) + return r.reconcileDelete(ctx, obj) + } + if !bound { + // Not bound yet; binding changes re-trigger this reconcile. + return reconcile.Result{}, nil } // Ensure finalizer before first write. @@ -151,16 +122,12 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( return reconcile.Result{Requeue: true}, nil } - if providerClient == nil { - return reconcile.Result{}, nil - } - - // Conflict check before first write. + // Conflict check before first write (fresh read via the API reader). existing := newUnstructured(r.gvk) - getErr := providerClient.Get(ctx, client.ObjectKeyFromObject(obj), existing) + getErr := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), existing) switch { case getErr == nil: - if owner, ours := ownershipOf(existing, localUID, string(obj.GetUID())); !ours { + if owner, ours := ownershipOf(existing, r.localUID, string(obj.GetUID())); !ours { reason := conflictReason(owner) msg := fmt.Sprintf("provider object %s already exists and is %s; not overwriting (conflictPolicy: Fail)", client.ObjectKeyFromObject(obj), owner) log.Info("conflict: refusing to overwrite provider object", "owner", owner) @@ -175,7 +142,7 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( } case apierrors.IsNotFound(getErr): if r.scope == apiextensionsv1.NamespaceScoped { - if err := ensureNamespace(ctx, providerClient, obj.GetNamespace(), localUID); err != nil { + if err := ensureNamespace(ctx, r.providerClient, obj.GetNamespace(), r.localUID); err != nil { return reconcile.Result{}, err } } @@ -184,36 +151,36 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( } // Spec consumer -> provider (SSA). - patch := r.providerPatch(obj, localUID) - if err := providerClient.Patch(ctx, patch, client.Apply, client.FieldOwner(fieldOwner), client.ForceOwnership); err != nil { + patch := r.providerPatch(obj, r.localUID) + if err := r.providerClient.Patch(ctx, patch, client.Apply, client.FieldOwner(fieldOwner), client.ForceOwnership); err != nil { return reconcile.Result{}, fmt.Errorf("applying spec to provider: %w", err) } // Status provider -> consumer. got := newUnstructured(r.gvk) - if err := providerClient.Get(ctx, client.ObjectKeyFromObject(obj), got); err != nil { + if err := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), got); err != nil { return reconcile.Result{}, fmt.Errorf("reading provider status: %w", err) } if err := r.copyStatus(ctx, got, obj); err != nil { return reconcile.Result{}, err } - return reconcile.Result{RequeueAfter: statusResyncInterval}, nil + return reconcile.Result{RequeueAfter: statusResyncBackstop}, nil } -func (r *specReconciler) reconcileDelete(ctx context.Context, obj *unstructured.Unstructured, providerClient client.Client, localUID string) (reconcile.Result, error) { +func (r *specReconciler) reconcileDelete(ctx context.Context, obj *unstructured.Unstructured) (reconcile.Result, error) { if !controllerutil.ContainsFinalizer(obj, corev1alpha1.FinalizerSyncer) { return reconcile.Result{}, nil } - if providerClient != nil { + { existing := newUnstructured(r.gvk) - err := providerClient.Get(ctx, client.ObjectKeyFromObject(obj), existing) + err := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), existing) switch { case err == nil: // Only delete the provider copy if it is ours. A foreign object that // merely shares the name (a conflict) must never be deleted by us. - if _, ours := ownershipOf(existing, localUID, string(obj.GetUID())); ours { - if err := providerClient.Delete(ctx, existing); client.IgnoreNotFound(err) != nil { + if _, ours := ownershipOf(existing, r.localUID, string(obj.GetUID())); ours { + if err := r.providerClient.Delete(ctx, existing); client.IgnoreNotFound(err) != nil { return reconcile.Result{}, fmt.Errorf("deleting provider object: %w", err) } // Wait for the provider copy to be fully gone before releasing. diff --git a/v2/konnector/test/e2e/framework/framework.go b/v2/konnector/test/e2e/framework/framework.go index ef12dc1d3..7708b1ca0 100644 --- a/v2/konnector/test/e2e/framework/framework.go +++ b/v2/konnector/test/e2e/framework/framework.go @@ -36,23 +36,25 @@ import ( apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/util/wait" - clientgoscheme "k8s.io/client-go/kubernetes/scheme" - "k8s.io/apimachinery/pkg/runtime/schema" apimachineryruntime "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" utilruntime "k8s.io/apimachinery/pkg/util/runtime" + "k8s.io/apimachinery/pkg/util/wait" "k8s.io/client-go/dynamic" + clientgoscheme "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" - clientcmdapi "k8s.io/client-go/tools/clientcmd/api" "k8s.io/client-go/tools/clientcmd" + clientcmdapi "k8s.io/client-go/tools/clientcmd/api" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/envtest" "sigs.k8s.io/controller-runtime/pkg/log/zap" metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" + mcmanager "sigs.k8s.io/multicluster-runtime/pkg/manager" "github.com/kube-bind/kube-bind/v2/konnector/engine/binding" "github.com/kube-bind/kube-bind/v2/konnector/engine/connection" + "github.com/kube-bind/kube-bind/v2/konnector/engine/provider" syncengine "github.com/kube-bind/kube-bind/v2/konnector/engine/sync" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) @@ -142,31 +144,50 @@ func Start(t *testing.T) *Env { } } -// startEngine runs the consumer-side reconcilers (connection, both bindings, and -// the dynamic sync controller) on a manager pointed at the consumer cluster. -// The mcr provider is not needed here: the sync path uses direct provider -// clients in the POC. +// startEngine wires the full engine the way main.go does: a local (consumer) +// manager, the mcr ConnectionProvider (each Connection -> engaged provider +// cluster), the multicluster manager, and the reconcilers — including the +// sync engine using the engaged cluster (Option B). func startEngine(t *testing.T, consumerCfg *rest.Config, scheme *apimachineryruntime.Scheme) { t.Helper() - mgr, err := ctrl.NewManager(consumerCfg, ctrl.Options{ + localMgr, err := ctrl.NewManager(consumerCfg, ctrl.Options{ Scheme: scheme, Metrics: metricsserver.Options{BindAddress: "0"}, }) require.NoError(t, err) - require.NoError(t, (&connection.Reconciler{}).SetupWithManager(mgr)) - require.NoError(t, (&binding.ClusterReconciler{}).SetupWithManager(mgr)) - require.NoError(t, (&binding.NamespacedReconciler{}).SetupWithManager(mgr)) - require.NoError(t, syncengine.SetupWithManager(mgr)) + connProvider, err := provider.New(localMgr, provider.Options{}) + require.NoError(t, err) + + mcMgr, err := mcmanager.New(consumerCfg, connProvider, mcmanager.Options{ + Scheme: scheme, + Metrics: metricsserver.Options{BindAddress: "0"}, + }) + require.NoError(t, err) + + require.NoError(t, (&connection.Reconciler{}).SetupWithManager(localMgr)) + require.NoError(t, (&binding.ClusterReconciler{}).SetupWithManager(localMgr)) + require.NoError(t, (&binding.NamespacedReconciler{}).SetupWithManager(localMgr)) + require.NoError(t, syncengine.SetupWithManager(localMgr, connProvider)) ctx, cancel := context.WithCancel(context.Background()) t.Cleanup(cancel) go func() { - if err := mgr.Start(ctx); err != nil { - t.Logf("manager stopped: %v", err) + if err := localMgr.Start(ctx); err != nil { + t.Logf("local manager stopped: %v", err) + } + }() + go func() { + if err := connProvider.Run(ctx, mcMgr); err != nil { + t.Logf("connection provider stopped: %v", err) + } + }() + go func() { + if err := mcMgr.Start(ctx); err != nil { + t.Logf("mc manager stopped: %v", err) } }() - require.True(t, mgr.GetCache().WaitForCacheSync(ctx), "engine cache sync") + require.True(t, localMgr.GetCache().WaitForCacheSync(ctx), "engine cache sync") } // InstallExportedWidgetCRD installs the demo Widget CRD on the provider, labeled diff --git a/v2/konnector/test/e2e/sync_test.go b/v2/konnector/test/e2e/sync_test.go index 99827f719..aa99ed95a 100644 --- a/v2/konnector/test/e2e/sync_test.go +++ b/v2/konnector/test/e2e/sync_test.go @@ -30,8 +30,8 @@ import ( "k8s.io/client-go/util/retry" "sigs.k8s.io/controller-runtime/pkg/client" - corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" "github.com/kube-bind/kube-bind/v2/konnector/test/e2e/framework" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) const ( @@ -252,5 +252,3 @@ func widget(name, size string) *unstructured.Unstructured { _ = unstructured.SetNestedField(u.Object, size, "spec", "size") return u } - -func apiextensionsGVK() (gvk struct{}) { panic("unused") } diff --git a/v2/sdk/apis/core/v1alpha1/labels.go b/v2/sdk/apis/core/v1alpha1/labels.go index 54e6d4e69..d25709dce 100644 --- a/v2/sdk/apis/core/v1alpha1/labels.go +++ b/v2/sdk/apis/core/v1alpha1/labels.go @@ -27,6 +27,11 @@ const ( // the provider. LabelManaged = "core.kube-bind.io/managed" + // AnnotationConnection records, on a pulled consumer CRD, the name of the + // Connection whose provider it was pulled from. The sync engine uses it to + // pin the provider cluster for that API. + AnnotationConnection = "core.kube-bind.io/connection" + // AnnotationConsumerClusterUID records the consumer cluster identity on a // synced provider object (ownership marker). AnnotationConsumerClusterUID = "core.kube-bind.io/consumer-cluster-uid" From f4ac110f25a4d0ff8adf5c6936710f88e4935b07 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 09:50:26 +0300 Subject: [PATCH 05/18] Wire in finalizer flow --- v2/README.md | 9 + v2/konnector/engine/binding/cleanup.go | 222 +++++++++++++++++++ v2/konnector/engine/binding/reconciler.go | 42 ++++ v2/konnector/engine/connection/reconciler.go | 159 ++++++++++++- v2/konnector/engine/sync/crd_controller.go | 5 +- v2/konnector/engine/sync/resolve.go | 9 + v2/konnector/test/e2e/framework/framework.go | 14 ++ v2/konnector/test/e2e/sync_test.go | 181 +++++++++++++++ v2/sdk/apis/core/v1alpha1/labels.go | 7 + 9 files changed, 645 insertions(+), 3 deletions(-) create mode 100644 v2/konnector/engine/binding/cleanup.go diff --git a/v2/README.md b/v2/README.md index 28d61bc2b..59854ee51 100644 --- a/v2/README.md +++ b/v2/README.md @@ -39,6 +39,15 @@ v2/ Connection: writes go through its client, fresh reads through its API reader, and status/drift events arrive via a **watch on its cache** (event-driven, not polled — a low-frequency resync is only a backstop). +- **Order-independent apply**: a `Connection` created before its Secret resolves + when the Secret arrives (the konnector watches referenced Secrets); a binding + created before its Connection resolves when the Connection goes Ready. +- **Complete unbind**: `Connection` and bindings carry a cleanup finalizer. + Deleting a `ClusterBinding` deletes the provider copies of synced instances, + releases instance finalizers, and removes the pulled CRD (cascading the + instances). A `Connection` blocks (`DrainingBindings`) until its bindings are + gone, and keeps its Secret alive (via a finalizer) so teardown can still reach + the provider — so `kubectl delete -f bundle.yaml` is order-don't-care. Known POC simplifications (tracked against the proposal): `schema.source: OpenAPI`, `conflictPolicy: Adopt`, related resources, `autoBind` and the diff --git a/v2/konnector/engine/binding/cleanup.go b/v2/konnector/engine/binding/cleanup.go new file mode 100644 index 000000000..f156f7322 --- /dev/null +++ b/v2/konnector/engine/binding/cleanup.go @@ -0,0 +1,222 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package binding + +import ( + "context" + "fmt" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// cleanup unwinds what a binding created when it is deleted. For each API the +// binding bound that no *other* binding still covers, it deletes the provider +// copies of synced instances (best effort), releases the syncer finalizer on +// the consumer instances, and — for a cluster-wide unbind — removes the pulled +// CRD (which cascade-deletes the now finalizer-free instances). +// +// - namespace: "" for a cluster-wide unbind; otherwise the Binding's namespace. +// - removeCRD: true for ClusterBinding (owns the CRD), false for Binding. +func (b *base) cleanup(ctx context.Context, obj corev1alpha1.BindingAccessor, namespace string, removeCRD bool) error { + log := ctrl.LoggerFrom(ctx) + spec := obj.BindingSpecP() + + // Best-effort provider client: the Secret/Connection may already be gone + // during a `delete -f`. If unavailable, we still release finalizers so the + // consumer never wedges; orphaned provider copies are the reaper's job. + var providerClient client.Client + conn := &corev1alpha1.Connection{} + if err := b.client.Get(ctx, client.ObjectKey{Name: spec.ConnectionRef.Name}, conn); err == nil { + if pc, perr := b.providerClient(ctx, conn); perr == nil { + providerClient = pc + } else { + log.V(2).Info("provider client unavailable during cleanup; releasing finalizers only", "err", perr.Error()) + } + } + localUID := conn.Status.LocalClusterUID + + for _, api := range spec.APIs { + if b.otherBindingCovers(ctx, api.Name, obj) { + continue + } + gvr, ok, err := b.gvrFor(ctx, api.Name) + if err != nil { + return err + } + if !ok { + continue // CRD already gone + } + + if err := b.drainInstances(ctx, gvr, namespace, providerClient, localUID); err != nil { + return err + } + + if removeCRD { + crd := &apiextensionsv1.CustomResourceDefinition{ObjectMeta: metav1.ObjectMeta{Name: api.Name}} + if err := b.client.Delete(ctx, crd); client.IgnoreNotFound(err) != nil { + return fmt.Errorf("deleting pulled CRD %s: %w", api.Name, err) + } + } + } + return nil +} + +// drainInstances deletes provider copies and releases the syncer finalizer on +// every consumer instance of gvr in scope. +func (b *base) drainInstances(ctx context.Context, gvr schema.GroupVersionResource, namespace string, providerClient client.Client, localUID string) error { + ri := b.dyn.Resource(gvr) + var lister dynamicLister = ri + if namespace != "" { + lister = ri.Namespace(namespace) + } + list, err := lister.List(ctx, metav1.ListOptions{}) + if err != nil { + if apierrors.IsNotFound(err) { + return nil + } + return fmt.Errorf("listing instances of %s: %w", gvr.Resource, err) + } + + for i := range list.Items { + inst := &list.Items[i] + if !controllerutil.ContainsFinalizer(inst, corev1alpha1.FinalizerSyncer) { + continue + } + // Delete the provider copy if it is ours (best effort). + if providerClient != nil { + if err := deleteProviderCopy(ctx, providerClient, gvr, inst, localUID); err != nil { + return err + } + } + // Release the syncer finalizer so the consumer instance can be deleted. + controllerutil.RemoveFinalizer(inst, corev1alpha1.FinalizerSyncer) + var updErr error + if inst.GetNamespace() == "" { + _, updErr = ri.Update(ctx, inst, metav1.UpdateOptions{}) + } else { + _, updErr = ri.Namespace(inst.GetNamespace()).Update(ctx, inst, metav1.UpdateOptions{}) + } + if client.IgnoreNotFound(updErr) != nil { + return fmt.Errorf("releasing finalizer on %s/%s: %w", inst.GetNamespace(), inst.GetName(), updErr) + } + } + return nil +} + +func deleteProviderCopy(ctx context.Context, providerClient client.Client, gvr schema.GroupVersionResource, inst *unstructured.Unstructured, localUID string) error { + existing := &unstructured.Unstructured{} + existing.SetGroupVersionKind(gvkFromGVR(gvr, inst)) + err := providerClient.Get(ctx, client.ObjectKey{Namespace: inst.GetNamespace(), Name: inst.GetName()}, existing) + if apierrors.IsNotFound(err) { + return nil + } + if err != nil { + // Best effort: if we can't read the provider object during teardown we + // can't confirm ownership, so skip rather than fail the whole unbind. + return nil //nolint:nilerr // intentional best-effort skip + } + // Only delete if it carries our markers. + ann := existing.GetAnnotations() + if ann[corev1alpha1.AnnotationConsumerClusterUID] != localUID || ann[corev1alpha1.AnnotationConsumerObjectUID] != string(inst.GetUID()) { + return nil + } + return client.IgnoreNotFound(providerClient.Delete(ctx, existing)) +} + +func gvkFromGVR(gvr schema.GroupVersionResource, inst *unstructured.Unstructured) schema.GroupVersionKind { + if gvk := inst.GroupVersionKind(); gvk.Kind != "" { + return gvk + } + return schema.GroupVersionKind{Group: gvr.Group, Version: gvr.Version} +} + +// otherBindingCovers reports whether a binding other than self lists crdName. +func (b *base) otherBindingCovers(ctx context.Context, crdName string, self corev1alpha1.BindingAccessor) bool { + var cbs corev1alpha1.ClusterBindingList + if err := b.client.List(ctx, &cbs); err == nil { + for i := range cbs.Items { + if cbs.Items[i].DeletionTimestamp != nil || cbs.Items[i].UID == self.GetUID() { + continue + } + if listsAPI(cbs.Items[i].Spec.APIs, crdName) { + return true + } + } + } + var bs corev1alpha1.BindingList + if err := b.client.List(ctx, &bs); err == nil { + for i := range bs.Items { + if bs.Items[i].DeletionTimestamp != nil || bs.Items[i].UID == self.GetUID() { + continue + } + if listsAPI(bs.Items[i].Spec.APIs, crdName) { + return true + } + } + } + return false +} + +// gvrFor resolves the GVR for an exported API CRD name from the pulled consumer +// CRD (storage/served version). Returns ok=false if the CRD is already gone. +func (b *base) gvrFor(ctx context.Context, crdName string) (schema.GroupVersionResource, bool, error) { + crd := &apiextensionsv1.CustomResourceDefinition{} + if err := b.client.Get(ctx, client.ObjectKey{Name: crdName}, crd); err != nil { + if apierrors.IsNotFound(err) { + return schema.GroupVersionResource{}, false, nil + } + return schema.GroupVersionResource{}, false, err + } + version := "" + for _, v := range crd.Spec.Versions { + if v.Storage { + version = v.Name + break + } + if version == "" && v.Served { + version = v.Name + } + } + if version == "" { + return schema.GroupVersionResource{}, false, nil + } + return schema.GroupVersionResource{Group: crd.Spec.Group, Version: version, Resource: crd.Spec.Names.Plural}, true, nil +} + +func listsAPI(apis []corev1alpha1.APIRef, crdName string) bool { + for _, a := range apis { + if a.Name == crdName { + return true + } + } + return false +} + +// dynamicLister is the common List surface of a (namespaced or not) dynamic +// resource interface. +type dynamicLister interface { + List(ctx context.Context, opts metav1.ListOptions) (*unstructured.UnstructuredList, error) +} diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go index 2c8e0b183..1b23f35c0 100644 --- a/v2/konnector/engine/binding/reconciler.go +++ b/v2/konnector/engine/binding/reconciler.go @@ -33,8 +33,10 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/dynamic" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/reconcile" @@ -45,6 +47,7 @@ import ( // base holds the shared dependencies and logic for both binding kinds. type base struct { client client.Client + dyn dynamic.Interface scheme *runtime.Scheme newClient func(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) } @@ -214,6 +217,11 @@ type ClusterReconciler struct{ base } func (r *ClusterReconciler) SetupWithManager(mgr ctrl.Manager) error { r.base.client = mgr.GetClient() r.base.scheme = mgr.GetScheme() + dyn, err := dynamic.NewForConfig(mgr.GetConfig()) + if err != nil { + return err + } + r.base.dyn = dyn return ctrl.NewControllerManagedBy(mgr). For(&corev1alpha1.ClusterBinding{}). Watches(&corev1alpha1.Connection{}, handler.EnqueueRequestsFromMapFunc(r.bindingsForConnection)). @@ -241,6 +249,20 @@ func (r *ClusterReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ct if err := r.base.client.Get(ctx, req.NamespacedName, cb); err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } + if cb.DeletionTimestamp != nil { + if !controllerutil.ContainsFinalizer(cb, corev1alpha1.FinalizerCleanup) { + return ctrl.Result{}, nil + } + // Cluster-wide unbind: clean up cluster-wide and remove the pulled CRDs. + if err := r.base.cleanup(ctx, cb, "", true); err != nil { + return ctrl.Result{}, err + } + controllerutil.RemoveFinalizer(cb, corev1alpha1.FinalizerCleanup) + return ctrl.Result{}, client.IgnoreNotFound(r.base.client.Update(ctx, cb)) + } + if controllerutil.AddFinalizer(cb, corev1alpha1.FinalizerCleanup) { + return ctrl.Result{}, r.base.client.Update(ctx, cb) + } orig := cb.DeepCopy() rerr := r.base.reconcileAccessor(ctx, cb) if !equalBindingStatus(&orig.Status, &cb.Status) { @@ -261,6 +283,11 @@ type NamespacedReconciler struct{ base } func (r *NamespacedReconciler) SetupWithManager(mgr ctrl.Manager) error { r.base.client = mgr.GetClient() r.base.scheme = mgr.GetScheme() + dyn, err := dynamic.NewForConfig(mgr.GetConfig()) + if err != nil { + return err + } + r.base.dyn = dyn return ctrl.NewControllerManagedBy(mgr). For(&corev1alpha1.Binding{}). Watches(&corev1alpha1.Connection{}, handler.EnqueueRequestsFromMapFunc(r.bindingsForConnection)). @@ -288,6 +315,21 @@ func (r *NamespacedReconciler) Reconcile(ctx context.Context, req ctrl.Request) if err := r.base.client.Get(ctx, req.NamespacedName, b); err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } + if b.DeletionTimestamp != nil { + if !controllerutil.ContainsFinalizer(b, corev1alpha1.FinalizerCleanup) { + return ctrl.Result{}, nil + } + // Namespaced unbind: clean up only instances in this namespace; the CRD + // is shared and stays. + if err := r.base.cleanup(ctx, b, b.Namespace, false); err != nil { + return ctrl.Result{}, err + } + controllerutil.RemoveFinalizer(b, corev1alpha1.FinalizerCleanup) + return ctrl.Result{}, client.IgnoreNotFound(r.base.client.Update(ctx, b)) + } + if controllerutil.AddFinalizer(b, corev1alpha1.FinalizerCleanup) { + return ctrl.Result{}, r.base.client.Update(ctx, b) + } orig := b.DeepCopy() rerr := r.base.reconcileAccessor(ctx, b) if !equalBindingStatus(&orig.Status, &b.Status) { diff --git a/v2/konnector/engine/connection/reconciler.go b/v2/konnector/engine/connection/reconciler.go index 0392d9e50..1dbc6fe57 100644 --- a/v2/konnector/engine/connection/reconciler.go +++ b/v2/konnector/engine/connection/reconciler.go @@ -23,13 +23,20 @@ import ( "context" "fmt" "sort" + "time" + corev1 "k8s.io/api/core/v1" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/reconcile" "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" @@ -47,7 +54,9 @@ type Reconciler struct { NewProviderClient func(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) } -// SetupWithManager registers the reconciler with the consumer manager. +// SetupWithManager registers the reconciler with the consumer manager. It also +// watches Secrets so a Connection created before its kubeconfig Secret resolves +// as soon as the Secret arrives (order-independent "one apply"). func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error { r.Client = mgr.GetClient() r.Scheme = mgr.GetScheme() @@ -56,10 +65,28 @@ func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error { } return ctrl.NewControllerManagedBy(mgr). For(&corev1alpha1.Connection{}). + Watches(&corev1.Secret{}, handler.EnqueueRequestsFromMapFunc(r.connectionsForSecret)). Named("connection"). Complete(r) } +// connectionsForSecret enqueues every Connection whose kubeconfigSecretRef +// points at the given Secret. +func (r *Reconciler) connectionsForSecret(ctx context.Context, secret client.Object) []reconcile.Request { + var list corev1alpha1.ConnectionList + if err := r.Client.List(ctx, &list); err != nil { + return nil + } + var reqs []reconcile.Request + for i := range list.Items { + ref := list.Items[i].Spec.KubeconfigSecretRef + if ref.Namespace == secret.GetNamespace() && ref.Name == secret.GetName() { + reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{Name: list.Items[i].Name}}) + } + } + return reqs +} + func (r *Reconciler) defaultProviderClient(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) { cfg, err := remote.RestConfigFromConnection(ctx, r.Client, conn) if err != nil { @@ -78,6 +105,18 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu return ctrl.Result{}, client.IgnoreNotFound(err) } + if conn.DeletionTimestamp != nil { + return r.reconcileDelete(ctx, conn) + } + + // Ensure cleanup finalizers on the Connection and (best effort) its Secret + // before doing any work, so teardown can always unwind. + if added, err := r.ensureFinalizers(ctx, conn); err != nil { + return ctrl.Result{}, err + } else if added { + return ctrl.Result{}, nil + } + orig := conn.DeepCopy() reconcileErr := r.reconcile(ctx, conn) @@ -134,6 +173,124 @@ func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connectio return nil } +// ensureFinalizers adds the cleanup finalizer to the Connection and to its +// referenced Secret (so the credential outlives the Connection during a +// `delete -f` teardown). Returns true if it changed the Connection. +func (r *Reconciler) ensureFinalizers(ctx context.Context, conn *corev1alpha1.Connection) (bool, error) { + // Secret finalizer (best effort: the Secret may not exist yet). + ref := conn.Spec.KubeconfigSecretRef + var secret corev1.Secret + if err := r.Client.Get(ctx, types.NamespacedName{Namespace: ref.Namespace, Name: ref.Name}, &secret); err == nil { + if controllerutil.AddFinalizer(&secret, corev1alpha1.FinalizerCleanup) { + if err := r.Client.Update(ctx, &secret); err != nil { + return false, fmt.Errorf("adding finalizer to secret: %w", err) + } + } + } else if !apierrors.IsNotFound(err) { + return false, err + } + + if controllerutil.AddFinalizer(conn, corev1alpha1.FinalizerCleanup) { + if err := r.Client.Update(ctx, conn); err != nil { + return false, fmt.Errorf("adding finalizer to connection: %w", err) + } + return true, nil + } + return false, nil +} + +// reconcileDelete blocks the Connection from finalizing until no Binding +// references it (so binding cleanup can still reach the provider through it), +// then releases the Secret's finalizer and its own. +func (r *Reconciler) reconcileDelete(ctx context.Context, conn *corev1alpha1.Connection) (ctrl.Result, error) { + if !controllerutil.ContainsFinalizer(conn, corev1alpha1.FinalizerCleanup) { + return ctrl.Result{}, nil + } + + refs, err := r.bindingsReferencing(ctx, conn.Name) + if err != nil { + return ctrl.Result{}, err + } + if refs > 0 { + setCondition(conn, corev1alpha1.ConditionReady, metav1.ConditionFalse, "DrainingBindings", + fmt.Sprintf("waiting for %d binding(s) to finish unbinding", refs)) + if err := r.Client.Status().Update(ctx, conn); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + return ctrl.Result{RequeueAfter: 2 * time.Second}, nil + } + + // Release the Secret finalizer — but only if no other Connection still + // references the same Secret. + ref := conn.Spec.KubeconfigSecretRef + if others, err := r.otherConnectionsUsingSecret(ctx, ref, conn.Name); err != nil { + return ctrl.Result{}, err + } else if others == 0 { + var secret corev1.Secret + if err := r.Client.Get(ctx, types.NamespacedName{Namespace: ref.Namespace, Name: ref.Name}, &secret); err == nil { + if controllerutil.RemoveFinalizer(&secret, corev1alpha1.FinalizerCleanup) { + if err := r.Client.Update(ctx, &secret); err != nil { + return ctrl.Result{}, fmt.Errorf("releasing secret finalizer: %w", err) + } + } + } else if !apierrors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + controllerutil.RemoveFinalizer(conn, corev1alpha1.FinalizerCleanup) + if err := r.Client.Update(ctx, conn); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + return ctrl.Result{}, nil +} + +// otherConnectionsUsingSecret counts Connections other than self whose +// kubeconfigSecretRef points at the same Secret, so we don't release the +// Secret finalizer while another Connection still depends on it. +func (r *Reconciler) otherConnectionsUsingSecret(ctx context.Context, ref corev1alpha1.SecretKeyRef, selfName string) (int, error) { + var list corev1alpha1.ConnectionList + if err := r.Client.List(ctx, &list); err != nil { + return 0, err + } + var n int + for i := range list.Items { + c := &list.Items[i] + if c.Name == selfName || c.DeletionTimestamp != nil { + continue + } + if c.Spec.KubeconfigSecretRef.Namespace == ref.Namespace && c.Spec.KubeconfigSecretRef.Name == ref.Name { + n++ + } + } + return n, nil +} + +// bindingsReferencing counts ClusterBindings and Bindings whose connectionRef +// names this Connection. +func (r *Reconciler) bindingsReferencing(ctx context.Context, name string) (int, error) { + var n int + var cbs corev1alpha1.ClusterBindingList + if err := r.Client.List(ctx, &cbs); err != nil { + return 0, err + } + for i := range cbs.Items { + if cbs.Items[i].Spec.ConnectionRef.Name == name { + n++ + } + } + var bs corev1alpha1.BindingList + if err := r.Client.List(ctx, &bs); err != nil { + return 0, err + } + for i := range bs.Items { + if bs.Items[i].Spec.ConnectionRef.Name == name { + n++ + } + } + return n, nil +} + // discoverExportedCRDs lists provider CRDs carrying the exported label and maps // them to ExportedAPI entries. func discoverExportedCRDs(ctx context.Context, providerClient client.Client) ([]corev1alpha1.ExportedAPI, error) { diff --git a/v2/konnector/engine/sync/crd_controller.go b/v2/konnector/engine/sync/crd_controller.go index be1236cf0..3fc4023ad 100644 --- a/v2/konnector/engine/sync/crd_controller.go +++ b/v2/konnector/engine/sync/crd_controller.go @@ -155,8 +155,9 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte // Connection is engaged (it engages once Ready), requeue. providerCluster, err := r.clusters.Get(ctx, connName) if err != nil { - log.V(4).Info("provider cluster not engaged yet, requeueing", "connection", connName) - return reconcile.Result{RequeueAfter: 2 * time.Second}, nil + // Not an error: the Connection just isn't engaged yet — requeue and wait. + log.V(4).Info("provider cluster not engaged yet, requeueing", "connection", connName, "reason", err.Error()) + return reconcile.Result{RequeueAfter: 2 * time.Second}, nil //nolint:nilerr // not-engaged-yet is a requeue, not an error } conn := &corev1alpha1.Connection{} if err := r.consumerClient.Get(ctx, client.ObjectKey{Name: connName}, conn); err != nil { diff --git a/v2/konnector/engine/sync/resolve.go b/v2/konnector/engine/sync/resolve.go index 80e0e32b5..33ca29258 100644 --- a/v2/konnector/engine/sync/resolve.go +++ b/v2/konnector/engine/sync/resolve.go @@ -50,6 +50,12 @@ func ResolveConnection(ctx context.Context, c client.Client, crdName, namespace } for i := range cbs.Items { cb := &cbs.Items[i] + // A binding being deleted is no longer a valid sync source — this gates + // the syncer off during unbind so it doesn't re-add finalizers or + // re-create provider copies while cleanup runs. + if cb.DeletionTimestamp != nil { + continue + } if listsAPI(cb.Spec.APIs, crdName) { return Resolution{ Found: true, @@ -66,6 +72,9 @@ func ResolveConnection(ctx context.Context, c client.Client, crdName, namespace } for i := range bs.Items { b := &bs.Items[i] + if b.DeletionTimestamp != nil { + continue + } if listsAPI(b.Spec.APIs, crdName) { return Resolution{ Found: true, diff --git a/v2/konnector/test/e2e/framework/framework.go b/v2/konnector/test/e2e/framework/framework.go index 7708b1ca0..732b60b0f 100644 --- a/v2/konnector/test/e2e/framework/framework.go +++ b/v2/konnector/test/e2e/framework/framework.go @@ -190,6 +190,20 @@ func startEngine(t *testing.T, consumerCfg *rest.Config, scheme *apimachineryrun require.True(t, localMgr.GetCache().WaitForCacheSync(ctx), "engine cache sync") } +// CopyProviderSecret returns a new Secret named `name` in the kube-bind +// namespace carrying the same provider kubeconfig as the one created by Start. +// Used to test Connection-before-Secret ordering. +func (e *Env) CopyProviderSecret(t *testing.T, name string) *corev1.Secret { + t.Helper() + var src corev1.Secret + require.NoError(t, e.ConsumerClient.Get(context.Background(), + client.ObjectKey{Namespace: KubeBindNamespace, Name: "demo-provider-kubeconfig"}, &src)) + return &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: KubeBindNamespace}, + Data: src.Data, + } +} + // InstallExportedWidgetCRD installs the demo Widget CRD on the provider, labeled // as exported. func (e *Env) InstallExportedWidgetCRD(t *testing.T) schema.GroupVersionResource { diff --git a/v2/konnector/test/e2e/sync_test.go b/v2/konnector/test/e2e/sync_test.go index aa99ed95a..9510404a3 100644 --- a/v2/konnector/test/e2e/sync_test.go +++ b/v2/konnector/test/e2e/sync_test.go @@ -22,6 +22,7 @@ import ( "time" "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -239,11 +240,191 @@ func TestSlimCoreHappyCase(t *testing.T) { require.Equal(t, "PROVIDER-OWNED", size) }, }, + { + // Gap 1: a Connection created before its Secret must resolve once the + // Secret arrives (order-independent "one apply"). + name: "Connection created before its Secret resolves when the Secret arrives", + step: func(t *testing.T) { + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "late-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{ + Namespace: framework.KubeBindNamespace, + Name: "late-secret", + Key: "kubeconfig", + }, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + // It must be not-Ready while the Secret is missing. + require.Never(t, func() bool { + conn := &corev1alpha1.Connection{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "late-provider"}, conn); err != nil { + return false + } + return isConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionReady) + }, 3*time.Second, 500*time.Millisecond, "must not be Ready without its Secret") + + // Now create the Secret (copy of the working one). + require.NoError(t, env.ConsumerClient.Create(ctx, env.CopyProviderSecret(t, "late-secret"))) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + conn := &corev1alpha1.Connection{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "late-provider"}, conn) + return conn.Status.Conditions, err + }, corev1alpha1.ConditionReady) + }, + }, + { + // The Secret cannot be hard-deleted while its Connection exists (it + // carries a cleanup finalizer); updates stay allowed. It is released + // only when the Connection is deleted. + name: "Secret survives deletion while its Connection exists, released on Connection delete", + step: func(t *testing.T) { + secretKey := client.ObjectKey{Namespace: framework.KubeBindNamespace, Name: "late-secret"} + // The Connection reconcile should have stamped the finalizer. + require.Eventually(t, func() bool { + s := &corev1.Secret{} + if err := env.ConsumerClient.Get(ctx, secretKey, s); err != nil { + return false + } + for _, f := range s.Finalizers { + if f == corev1alpha1.FinalizerCleanup { + return true + } + } + return false + }, wait.ForeverTestTimeout, 200*time.Millisecond, "Secret should carry the cleanup finalizer") + + // Deleting the Secret must NOT remove it while the Connection lives. + require.NoError(t, env.ConsumerClient.Delete(ctx, &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Namespace: secretKey.Namespace, Name: secretKey.Name}, + })) + require.Never(t, func() bool { + s := &corev1.Secret{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, secretKey, s)) + }, 3*time.Second, 500*time.Millisecond, "Secret must persist (Terminating) while its Connection exists") + + // Deleting the Connection releases the Secret. + require.NoError(t, env.ConsumerClient.Delete(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "late-provider"}, + })) + require.Eventually(t, func() bool { + s := &corev1.Secret{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, secretKey, s)) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "Secret should be released once the Connection is deleted") + require.Eventually(t, func() bool { + c := &corev1alpha1.Connection{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "late-provider"}, c)) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "Connection should finalize") + }, + }, + { + // A Secret shared by multiple Connections keeps its finalizer until + // the LAST Connection referencing it is gone. + name: "Secret shared by multiple Connections is released only when the last one is deleted", + step: func(t *testing.T) { + sharedKey := client.ObjectKey{Namespace: framework.KubeBindNamespace, Name: "shared-secret"} + require.NoError(t, env.ConsumerClient.Create(ctx, env.CopyProviderSecret(t, "shared-secret"))) + for _, name := range []string{"shared-a", "shared-b"} { + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: name}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{ + Namespace: framework.KubeBindNamespace, Name: "shared-secret", Key: "kubeconfig", + }, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + } + for _, name := range []string{"shared-a", "shared-b"} { + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + c := &corev1alpha1.Connection{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: name}, c) + return c.Status.Conditions, err + }, corev1alpha1.ConditionReady) + } + + // Request deletion of the shared Secret — it must stay (finalizer). + require.NoError(t, env.ConsumerClient.Delete(ctx, &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Namespace: sharedKey.Namespace, Name: sharedKey.Name}, + })) + + // Delete the first Connection — the Secret must survive (the second + // Connection still holds it). + require.NoError(t, env.ConsumerClient.Delete(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "shared-a"}, + })) + require.Eventually(t, func() bool { + c := &corev1alpha1.Connection{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "shared-a"}, c)) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "shared-a should finalize") + require.Never(t, func() bool { + s := &corev1.Secret{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, sharedKey, s)) + }, 3*time.Second, 500*time.Millisecond, "shared Secret must survive while shared-b still references it") + + // Delete the second (last) Connection — now the Secret is released + // and its pending deletion proceeds. + require.NoError(t, env.ConsumerClient.Delete(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "shared-b"}, + })) + require.Eventually(t, func() bool { + s := &corev1.Secret{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, sharedKey, s)) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "shared Secret should be released after the last Connection is deleted") + }, + }, + { + // Gap 2: deleting the ClusterBinding unbinds — provider copies gone, + // pulled CRD removed, consumer instances cascade away, finalizer freed. + name: "deleting the ClusterBinding unbinds and cleans up", + step: func(t *testing.T) { + // Seed a fresh synced instance. + _, err := consumerWidgets.Create(ctx, widget("unbind-widget", "small"), metav1.CreateOptions{}) + require.NoError(t, err) + require.Eventually(t, func() bool { + _, err := providerWidgets.Get(ctx, "unbind-widget", metav1.GetOptions{}) + return err == nil + }, wait.ForeverTestTimeout, 200*time.Millisecond, "unbind-widget should sync to the provider") + + // Unbind. + require.NoError(t, env.ConsumerClient.Delete(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + })) + + // ClusterBinding finalizes and disappears. + require.Eventually(t, func() bool { + cb := &corev1alpha1.ClusterBinding{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb)) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "ClusterBinding should finalize") + + // Provider copy deleted. + require.Eventually(t, func() bool { + _, err := providerWidgets.Get(ctx, "unbind-widget", metav1.GetOptions{}) + return apierrors.IsNotFound(err) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "provider copy should be cleaned up on unbind") + + // Pulled CRD removed → consumer instances cascade away. + require.Eventually(t, func() bool { + crd := &apiextensionsv1.CustomResourceDefinition{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, client.ObjectKey{Name: widgetCRDName}, crd)) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "pulled CRD should be removed on unbind") + }, + }, } { t.Run(tc.name, tc.step) } } +func isConditionTrue(conds []metav1.Condition, t string) bool { + for _, c := range conds { + if c.Type == t { + return c.Status == metav1.ConditionTrue + } + } + return false +} + func widget(name, size string) *unstructured.Unstructured { u := &unstructured.Unstructured{} u.SetGroupVersionKind(framework.WidgetGVK()) diff --git a/v2/sdk/apis/core/v1alpha1/labels.go b/v2/sdk/apis/core/v1alpha1/labels.go index d25709dce..c0b453c83 100644 --- a/v2/sdk/apis/core/v1alpha1/labels.go +++ b/v2/sdk/apis/core/v1alpha1/labels.go @@ -43,4 +43,11 @@ const ( // FinalizerSyncer blocks consumer-object deletion until the provider copy // has been removed. FinalizerSyncer = "core.kube-bind.io/syncer" + + // FinalizerCleanup blocks deletion of a Connection or Binding until the + // konnector has unwound what it created: provider copies of synced + // instances, pulled CRDs, and instance finalizers. It is also placed on a + // Connection's referenced Secret so the credential survives long enough to + // reach the provider during teardown. + FinalizerCleanup = "core.kube-bind.io/cleanup" ) From 2d39682bd3380e912d3852723217858d637ebb2f Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 10:28:08 +0300 Subject: [PATCH 06/18] lint --- v2/konnector/engine/sync/crd_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2/konnector/engine/sync/crd_controller.go b/v2/konnector/engine/sync/crd_controller.go index 3fc4023ad..e01e56efb 100644 --- a/v2/konnector/engine/sync/crd_controller.go +++ b/v2/konnector/engine/sync/crd_controller.go @@ -157,7 +157,7 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte if err != nil { // Not an error: the Connection just isn't engaged yet — requeue and wait. log.V(4).Info("provider cluster not engaged yet, requeueing", "connection", connName, "reason", err.Error()) - return reconcile.Result{RequeueAfter: 2 * time.Second}, nil //nolint:nilerr // not-engaged-yet is a requeue, not an error + return reconcile.Result{RequeueAfter: 2 * time.Second}, nil } conn := &corev1alpha1.Connection{} if err := r.consumerClient.Get(ctx, client.ObjectKey{Name: connName}, conn); err != nil { From 496d5424f4ccc6841630376e7a4156bf8ead677a Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 11:12:35 +0300 Subject: [PATCH 07/18] wire in pooling --- v2/README.md | 13 ++- v2/konnector/engine/binding/cleanup.go | 21 +++++ v2/konnector/engine/binding/reconciler.go | 57 ++++++++++-- v2/konnector/engine/connection/reconciler.go | 18 +++- v2/konnector/engine/sync/resolve.go | 4 + v2/konnector/engine/sync/syncer.go | 79 ++++++++++++++--- v2/konnector/test/e2e/framework/framework.go | 45 +++++----- v2/konnector/test/e2e/sync_test.go | 91 ++++++++++++++++++++ v2/sdk/apis/core/v1alpha1/labels.go | 6 ++ v2/sdk/go.sum | 10 +++ 10 files changed, 294 insertions(+), 50 deletions(-) diff --git a/v2/README.md b/v2/README.md index 59854ee51..2e1ba9cf1 100644 --- a/v2/README.md +++ b/v2/README.md @@ -33,8 +33,12 @@ v2/ (`schema.source: CRD`, single served version, no conversion webhook) onto the consumer, and report `Ready` + `boundAPIs`. - A dynamic per-GVR syncer copies instance **spec up** (server-side apply with - ownership markers + a finalizer) and **status down**, with `conflictPolicy: - Fail` surfaced as an Event (+ best-effort object condition). + ownership markers + a finalizer) and **status down**. `conflictPolicy: Fail` + refuses a foreign provider target (Event + conflict annotation, counted on the + binding's `conflictCount` + `Conflicts` condition); `conflictPolicy: Adopt` + takes over an *un-owned* provider object (never one owned by another binding). +- The Connection **re-discovers** exported APIs periodically, so a CRD labeled + `exported` after connect is picked up and its binding goes Ready. - The provider side is the **multicluster-runtime engaged cluster** for each Connection: writes go through its client, fresh reads through its API reader, and status/drift events arrive via a **watch on its cache** (event-driven, not @@ -50,8 +54,9 @@ v2/ the provider — so `kubectl delete -f bundle.yaml` is order-don't-care. Known POC simplifications (tracked against the proposal): `schema.source: -OpenAPI`, `conflictPolicy: Adopt`, related resources, `autoBind` and the -provider `Lease` are not implemented yet. +OpenAPI`, related resources, `pullPolicy: All`/`None`, `updatePolicy: Always`, +`autoBind`, `deletion-policy: Orphan` and the provider `Lease` are not +implemented yet. ## Build diff --git a/v2/konnector/engine/binding/cleanup.go b/v2/konnector/engine/binding/cleanup.go index f156f7322..583e9071f 100644 --- a/v2/konnector/engine/binding/cleanup.go +++ b/v2/konnector/engine/binding/cleanup.go @@ -84,6 +84,27 @@ func (b *base) cleanup(ctx context.Context, obj corev1alpha1.BindingAccessor, na return nil } +// countConflicts counts consumer instances of gvr in scope that the syncer +// marked as conflicting (the conflict annotation). +func (b *base) countConflicts(ctx context.Context, gvr schema.GroupVersionResource, namespace string) int32 { + ri := b.dyn.Resource(gvr) + var lister dynamicLister = ri + if namespace != "" { + lister = ri.Namespace(namespace) + } + list, err := lister.List(ctx, metav1.ListOptions{}) + if err != nil { + return 0 + } + var n int32 + for i := range list.Items { + if list.Items[i].GetAnnotations()[corev1alpha1.AnnotationConflict] != "" { + n++ + } + } + return n +} + // drainInstances deletes provider copies and releases the syncer finalizer on // every consumer instance of gvr in scope. func (b *base) drainInstances(ctx context.Context, gvr schema.GroupVersionResource, namespace string, providerClient client.Client, localUID string) error { diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go index 1b23f35c0..f32f99bfd 100644 --- a/v2/konnector/engine/binding/reconciler.go +++ b/v2/konnector/engine/binding/reconciler.go @@ -26,6 +26,7 @@ import ( "fmt" "sort" "strings" + "time" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -49,9 +50,19 @@ type base struct { client client.Client dyn dynamic.Interface scheme *runtime.Scheme + resync time.Duration newClient func(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) } +// resyncInterval is how often a Ready binding re-reconciles to refresh +// conflictCount (conflicts are detected by the syncer, not via binding events). +func (b *base) resyncInterval() time.Duration { + if b.resync > 0 { + return b.resync + } + return 30 * time.Second +} + func (b *base) providerClient(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) { if b.newClient != nil { return b.newClient(ctx, conn) @@ -60,7 +71,9 @@ func (b *base) providerClient(ctx context.Context, conn *corev1alpha1.Connection } // reconcileAccessor drives a binding (cluster or namespaced) toward Ready. -func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAccessor) error { +// namespace scopes conflict counting: "" for a ClusterBinding (cluster-wide), +// the Binding's namespace otherwise. +func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAccessor, namespace string) error { spec := obj.BindingSpecP() status := obj.BindingStatusP() @@ -99,7 +112,12 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc if err != nil { return fmt.Errorf("pulling CRD %q: %w", api.Name, err) } - boundAPIs = append(boundAPIs, corev1alpha1.BoundAPI{Name: exported.Name, CRDHash: hash}) + // Count instances we refused to sync due to a foreign provider target. + var conflicts int32 + if gvr, ok, gerr := b.gvrFor(ctx, api.Name); gerr == nil && ok { + conflicts = b.countConflicts(ctx, gvr, namespace) + } + boundAPIs = append(boundAPIs, corev1alpha1.BoundAPI{Name: exported.Name, CRDHash: hash, ConflictCount: conflicts}) } sort.Slice(boundAPIs, func(i, j int) bool { return boundAPIs[i].Name < boundAPIs[j].Name }) status.BoundAPIs = boundAPIs @@ -110,6 +128,17 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc return nil } + var totalConflicts int32 + for _, ba := range boundAPIs { + totalConflicts += ba.ConflictCount + } + if totalConflicts > 0 { + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionConflicts, metav1.ConditionTrue, corev1alpha1.ReasonForeignObjectExists, + fmt.Sprintf("%d object(s) skipped due to foreign ownership; see object Events", totalConflicts))) + } else { + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionConflicts, metav1.ConditionFalse, corev1alpha1.ReasonAsExpected, "no conflicts")) + } + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionSynced, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "all APIs installed")) setReady(status, obj, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "binding ready") return nil @@ -209,7 +238,12 @@ func setReady(status *corev1alpha1.BindingStatus, obj client.Object, s metav1.Co // ----- ClusterBinding ----- // ClusterReconciler reconciles ClusterBinding objects. -type ClusterReconciler struct{ base } +type ClusterReconciler struct { + base + // Resync is how often a Ready binding re-reconciles to refresh conflictCount + // (0 = default 30s). + Resync time.Duration +} // SetupWithManager registers the ClusterBinding reconciler. It also watches // Connection so bindings re-reconcile when their connection becomes Ready or @@ -217,6 +251,7 @@ type ClusterReconciler struct{ base } func (r *ClusterReconciler) SetupWithManager(mgr ctrl.Manager) error { r.base.client = mgr.GetClient() r.base.scheme = mgr.GetScheme() + r.base.resync = r.Resync dyn, err := dynamic.NewForConfig(mgr.GetConfig()) if err != nil { return err @@ -264,25 +299,31 @@ func (r *ClusterReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ct return ctrl.Result{}, r.base.client.Update(ctx, cb) } orig := cb.DeepCopy() - rerr := r.base.reconcileAccessor(ctx, cb) + rerr := r.base.reconcileAccessor(ctx, cb, "") if !equalBindingStatus(&orig.Status, &cb.Status) { if err := r.base.client.Status().Update(ctx, cb); err != nil { return ctrl.Result{}, err } } - return ctrl.Result{}, rerr + return ctrl.Result{RequeueAfter: r.base.resyncInterval()}, rerr } // ----- Binding (namespaced) ----- // NamespacedReconciler reconciles Binding objects. -type NamespacedReconciler struct{ base } +type NamespacedReconciler struct { + base + // Resync is how often a Ready binding re-reconciles to refresh conflictCount + // (0 = default 30s). + Resync time.Duration +} // SetupWithManager registers the Binding reconciler. It also watches Connection // so bindings re-reconcile when their connection becomes Ready (level-triggered). func (r *NamespacedReconciler) SetupWithManager(mgr ctrl.Manager) error { r.base.client = mgr.GetClient() r.base.scheme = mgr.GetScheme() + r.base.resync = r.Resync dyn, err := dynamic.NewForConfig(mgr.GetConfig()) if err != nil { return err @@ -331,13 +372,13 @@ func (r *NamespacedReconciler) Reconcile(ctx context.Context, req ctrl.Request) return ctrl.Result{}, r.base.client.Update(ctx, b) } orig := b.DeepCopy() - rerr := r.base.reconcileAccessor(ctx, b) + rerr := r.base.reconcileAccessor(ctx, b, b.Namespace) if !equalBindingStatus(&orig.Status, &b.Status) { if err := r.base.client.Status().Update(ctx, b); err != nil { return ctrl.Result{}, err } } - return ctrl.Result{}, rerr + return ctrl.Result{RequeueAfter: r.base.resyncInterval()}, rerr } func equalBindingStatus(a, b *corev1alpha1.BindingStatus) bool { diff --git a/v2/konnector/engine/connection/reconciler.go b/v2/konnector/engine/connection/reconciler.go index 1dbc6fe57..1d9e90ae7 100644 --- a/v2/konnector/engine/connection/reconciler.go +++ b/v2/konnector/engine/connection/reconciler.go @@ -52,6 +52,16 @@ type Reconciler struct { // Connection. Overridable in tests; defaults to a fresh client from the // resolved kubeconfig. NewProviderClient func(ctx context.Context, conn *corev1alpha1.Connection) (client.Client, error) + // DiscoveryResync is how often a Ready Connection re-discovers exported APIs + // (so a CRD labeled exported after connect is picked up). 0 = default 30s. + DiscoveryResync time.Duration +} + +func (r *Reconciler) discoveryResync() time.Duration { + if r.DiscoveryResync > 0 { + return r.DiscoveryResync + } + return 30 * time.Second } // SetupWithManager registers the reconciler with the consumer manager. It also @@ -126,7 +136,13 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu return ctrl.Result{}, err } } - return ctrl.Result{}, reconcileErr + // Periodically re-discover so a CRD labeled exported after connect is picked + // up (the binding watches the Connection and reacts to exportedAPIs changes). + result := ctrl.Result{} + if reconcileErr == nil { + result.RequeueAfter = r.discoveryResync() + } + return result, reconcileErr } func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connection) error { diff --git a/v2/konnector/engine/sync/resolve.go b/v2/konnector/engine/sync/resolve.go index 33ca29258..924a8b999 100644 --- a/v2/konnector/engine/sync/resolve.go +++ b/v2/konnector/engine/sync/resolve.go @@ -39,6 +39,8 @@ type Resolution struct { Ready bool // ConnectionName is the Connection the binding references. ConnectionName string + // ConflictPolicy is the binding's conflict policy (Fail if unset). + ConflictPolicy corev1alpha1.ConflictPolicy } // ResolveConnection finds the binding that covers crdName for the given @@ -61,6 +63,7 @@ func ResolveConnection(ctx context.Context, c client.Client, crdName, namespace Found: true, Ready: apimeta.IsStatusConditionTrue(cb.Status.Conditions, corev1alpha1.ConditionReady), ConnectionName: cb.Spec.ConnectionRef.Name, + ConflictPolicy: cb.Spec.ConflictPolicy, }, nil } } @@ -80,6 +83,7 @@ func ResolveConnection(ctx context.Context, c client.Client, crdName, namespace Found: true, Ready: apimeta.IsStatusConditionTrue(b.Status.Conditions, corev1alpha1.ConditionReady), ConnectionName: b.Spec.ConnectionRef.Name, + ConflictPolicy: b.Spec.ConflictPolicy, }, nil } } diff --git a/v2/konnector/engine/sync/syncer.go b/v2/konnector/engine/sync/syncer.go index 8340c541c..a53e909b5 100644 --- a/v2/konnector/engine/sync/syncer.go +++ b/v2/konnector/engine/sync/syncer.go @@ -27,6 +27,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/dynamic/dynamiclister" "k8s.io/client-go/tools/record" ctrl "sigs.k8s.io/controller-runtime" @@ -128,17 +129,18 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( switch { case getErr == nil: if owner, ours := ownershipOf(existing, r.localUID, string(obj.GetUID())); !ours { - reason := conflictReason(owner) - msg := fmt.Sprintf("provider object %s already exists and is %s; not overwriting (conflictPolicy: Fail)", client.ObjectKeyFromObject(obj), owner) - log.Info("conflict: refusing to overwrite provider object", "owner", owner) - // Surface on the object's own condition (best-effort: pruned if the - // CRD's status schema has no conditions) AND as an Event (always). - setObjCondition(obj, metav1.ConditionFalse, reason, msg) - _ = r.consumerClient.Status().Update(ctx, obj) - if r.recorder != nil { - r.recorder.Event(obj, corev1.EventTypeWarning, reason, msg) + // Adopt only takes an un-owned (markerless) object; it never steals + // one already owned by another binding/consumer. + if owner != ownerForeignUnmanaged || res.ConflictPolicy != corev1alpha1.ConflictPolicyAdopt { + reason := conflictReason(owner) + msg := fmt.Sprintf("provider object %s already exists and is %s; not overwriting (conflictPolicy: %s)", client.ObjectKeyFromObject(obj), owner, policyOrFail(res.ConflictPolicy)) + log.Info("conflict: refusing to overwrite provider object", "owner", owner) + if err := r.markConflict(ctx, obj, reason, msg); err != nil { + return reconcile.Result{}, err + } + return reconcile.Result{}, nil } - return reconcile.Result{}, nil + log.Info("adopting un-owned provider object", "key", client.ObjectKeyFromObject(obj)) } case apierrors.IsNotFound(getErr): if r.scope == apiextensionsv1.NamespaceScoped { @@ -150,6 +152,11 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( return reconcile.Result{}, fmt.Errorf("provider get: %w", getErr) } + // We are syncing — clear any stale conflict marker from a previous round. + if err := r.clearConflict(ctx, obj); err != nil { + return reconcile.Result{}, err + } + // Spec consumer -> provider (SSA). patch := r.providerPatch(obj, r.localUID) if err := r.providerClient.Patch(ctx, patch, client.Apply, client.FieldOwner(fieldOwner), client.ForceOwnership); err != nil { @@ -168,6 +175,36 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( return reconcile.Result{RequeueAfter: statusResyncBackstop}, nil } +// markConflict records a conflict durably (an annotation that survives status +// pruning, used by bindings to count conflicts), best-effort on the object's +// own condition, and as an Event. +func (r *specReconciler) markConflict(ctx context.Context, obj *unstructured.Unstructured, reason, msg string) error { + if obj.GetAnnotations()[corev1alpha1.AnnotationConflict] != reason { + p := fmt.Appendf(nil, `{"metadata":{"annotations":{%q:%q}}}`, corev1alpha1.AnnotationConflict, reason) + if err := r.consumerClient.Patch(ctx, obj, client.RawPatch(types.MergePatchType, p)); client.IgnoreNotFound(err) != nil { + return fmt.Errorf("marking conflict: %w", err) + } + } + setObjCondition(obj, metav1.ConditionFalse, reason, msg) + _ = r.consumerClient.Status().Update(ctx, obj) + if r.recorder != nil { + r.recorder.Event(obj, corev1.EventTypeWarning, reason, msg) + } + return nil +} + +// clearConflict removes the conflict marker once the object syncs cleanly. +func (r *specReconciler) clearConflict(ctx context.Context, obj *unstructured.Unstructured) error { + if obj.GetAnnotations()[corev1alpha1.AnnotationConflict] == "" { + return nil + } + p := fmt.Appendf(nil, `{"metadata":{"annotations":{%q:null}}}`, corev1alpha1.AnnotationConflict) + if err := r.consumerClient.Patch(ctx, obj, client.RawPatch(types.MergePatchType, p)); client.IgnoreNotFound(err) != nil { + return fmt.Errorf("clearing conflict: %w", err) + } + return nil +} + func (r *specReconciler) reconcileDelete(ctx context.Context, obj *unstructured.Unstructured) (reconcile.Result, error) { if !controllerutil.ContainsFinalizer(obj, corev1alpha1.FinalizerSyncer) { return reconcile.Result{}, nil @@ -241,6 +278,13 @@ func newUnstructured(gvk schema.GroupVersionKind) *unstructured.Unstructured { return u } +// Ownership classifications returned by ownershipOf. +const ( + ownerForeignUnmanaged = "foreign-unmanaged" + ownerOurs = "ours" + ownerByAnother = "owned-by-another" +) + // ownershipOf reports the marker owner of a provider object and whether it is // ours (our consumer cluster + our consumer object UID). func ownershipOf(obj *unstructured.Unstructured, localUID, consumerObjUID string) (string, bool) { @@ -248,21 +292,28 @@ func ownershipOf(obj *unstructured.Unstructured, localUID, consumerObjUID string cluster := ann[corev1alpha1.AnnotationConsumerClusterUID] objUID := ann[corev1alpha1.AnnotationConsumerObjectUID] if cluster == "" && objUID == "" { - return "foreign-unmanaged", false + return ownerForeignUnmanaged, false } if cluster == localUID && objUID == consumerObjUID { - return "ours", true + return ownerOurs, true } - return "owned-by-another", false + return ownerByAnother, false } func conflictReason(owner string) string { - if owner == "foreign-unmanaged" { + if owner == ownerForeignUnmanaged { return corev1alpha1.ReasonForeignObjectExists } return corev1alpha1.ReasonOwnedByAnother } +func policyOrFail(p corev1alpha1.ConflictPolicy) corev1alpha1.ConflictPolicy { + if p == "" { + return corev1alpha1.ConflictPolicyFail + } + return p +} + func setObjCondition(obj *unstructured.Unstructured, status metav1.ConditionStatus, reason, msg string) { cond := map[string]any{ "type": corev1alpha1.ConditionSynced, diff --git a/v2/konnector/test/e2e/framework/framework.go b/v2/konnector/test/e2e/framework/framework.go index 732b60b0f..778f33d59 100644 --- a/v2/konnector/test/e2e/framework/framework.go +++ b/v2/konnector/test/e2e/framework/framework.go @@ -165,9 +165,10 @@ func startEngine(t *testing.T, consumerCfg *rest.Config, scheme *apimachineryrun }) require.NoError(t, err) - require.NoError(t, (&connection.Reconciler{}).SetupWithManager(localMgr)) - require.NoError(t, (&binding.ClusterReconciler{}).SetupWithManager(localMgr)) - require.NoError(t, (&binding.NamespacedReconciler{}).SetupWithManager(localMgr)) + // Short resync intervals so re-discovery and conflictCount refresh fast in tests. + require.NoError(t, (&connection.Reconciler{DiscoveryResync: time.Second}).SetupWithManager(localMgr)) + require.NoError(t, (&binding.ClusterReconciler{Resync: time.Second}).SetupWithManager(localMgr)) + require.NoError(t, (&binding.NamespacedReconciler{Resync: time.Second}).SetupWithManager(localMgr)) require.NoError(t, syncengine.SetupWithManager(localMgr, connProvider)) ctx, cancel := context.WithCancel(context.Background()) @@ -207,15 +208,19 @@ func (e *Env) CopyProviderSecret(t *testing.T, name string) *corev1.Secret { // InstallExportedWidgetCRD installs the demo Widget CRD on the provider, labeled // as exported. func (e *Env) InstallExportedWidgetCRD(t *testing.T) schema.GroupVersionResource { + return e.InstallExportedCRD(t, "example.org", "widgets", "widget", "Widget") +} + +// InstallExportedCRD installs a namespaced CRD on the provider labeled as +// exported, and waits for the provider to serve it. +func (e *Env) InstallExportedCRD(t *testing.T, group, plural, singular, kind string) schema.GroupVersionResource { t.Helper() - crd := widgetCRD() - require.NoError(t, e.ProviderClient.Create(context.Background(), crd)) - gvr := schema.GroupVersionResource{Group: "example.org", Version: "v1", Resource: "widgets"} - // Wait for the provider to serve the new API. + require.NoError(t, e.ProviderClient.Create(context.Background(), exportedCRD(group, plural, singular, kind))) + gvr := schema.GroupVersionResource{Group: group, Version: "v1", Resource: plural} require.Eventually(t, func() bool { _, err := e.ProviderDyn.Resource(gvr).Namespace("default").List(context.Background(), metav1.ListOptions{}) return err == nil - }, 30*time.Second, 200*time.Millisecond, "provider should serve the Widget API") + }, 30*time.Second, 200*time.Millisecond, "provider should serve the %s API", kind) return gvr } @@ -224,20 +229,20 @@ func WidgetGVK() schema.GroupVersionKind { return schema.GroupVersionKind{Group: "example.org", Version: "v1", Kind: "Widget"} } -func widgetCRD() *apiextensionsv1.CustomResourceDefinition { +func exportedCRD(group, plural, singular, kind string) *apiextensionsv1.CustomResourceDefinition { preserve := true return &apiextensionsv1.CustomResourceDefinition{ ObjectMeta: metav1.ObjectMeta{ - Name: "widgets.example.org", + Name: plural + "." + group, Labels: map[string]string{corev1alpha1.LabelExported: "true"}, }, Spec: apiextensionsv1.CustomResourceDefinitionSpec{ - Group: "example.org", + Group: group, Names: apiextensionsv1.CustomResourceDefinitionNames{ - Plural: "widgets", - Singular: "widget", - Kind: "Widget", - ListKind: "WidgetList", + Plural: plural, + Singular: singular, + Kind: kind, + ListKind: kind + "List", }, Scope: apiextensionsv1.NamespaceScoped, Versions: []apiextensionsv1.CustomResourceDefinitionVersion{{ @@ -251,14 +256,8 @@ func widgetCRD() *apiextensionsv1.CustomResourceDefinition { OpenAPIV3Schema: &apiextensionsv1.JSONSchemaProps{ Type: "object", Properties: map[string]apiextensionsv1.JSONSchemaProps{ - "spec": { - Type: "object", - XPreserveUnknownFields: &preserve, - }, - "status": { - Type: "object", - XPreserveUnknownFields: &preserve, - }, + "spec": {Type: "object", XPreserveUnknownFields: &preserve}, + "status": {Type: "object", XPreserveUnknownFields: &preserve}, }, }, }, diff --git a/v2/konnector/test/e2e/sync_test.go b/v2/konnector/test/e2e/sync_test.go index 9510404a3..b8e114ee5 100644 --- a/v2/konnector/test/e2e/sync_test.go +++ b/v2/konnector/test/e2e/sync_test.go @@ -27,6 +27,7 @@ import ( apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/apimachinery/pkg/util/wait" "k8s.io/client-go/util/retry" "sigs.k8s.io/controller-runtime/pkg/client" @@ -209,6 +210,27 @@ func TestSlimCoreHappyCase(t *testing.T) { size, _, _ := unstructured.NestedString(obj.Object, "spec", "size") return size != "PROVIDER-OWNED" || obj.GetLabels()[corev1alpha1.LabelManaged] == "true" }, 5*time.Second, 500*time.Millisecond, "foreign provider object must not be overwritten") + + // The consumer object is marked with the conflict annotation. + require.Eventually(t, func() bool { + o, err := consumerWidgets.Get(ctx, "conflict-widget", metav1.GetOptions{}) + return err == nil && o.GetAnnotations()[corev1alpha1.AnnotationConflict] != "" + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer object should carry the conflict annotation") + + // The ClusterBinding surfaces conflictCount + the Conflicts condition. + require.Eventually(t, func() bool { + cb := &corev1alpha1.ClusterBinding{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb); err != nil { + return false + } + var cnt int32 + for _, ba := range cb.Status.BoundAPIs { + if ba.Name == widgetCRDName { + cnt = ba.ConflictCount + } + } + return cnt == 1 && isConditionTrue(cb.Status.Conditions, corev1alpha1.ConditionConflicts) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "ClusterBinding should report conflictCount=1 and Conflicts=True") }, }, { @@ -240,6 +262,75 @@ func TestSlimCoreHappyCase(t *testing.T) { require.Equal(t, "PROVIDER-OWNED", size) }, }, + { + // Tier 1: a Connection already Ready picks up a CRD exported later. + name: "re-discovery: a CRD exported after connect is picked up", + step: func(t *testing.T) { + env.InstallExportedCRD(t, "gadget.io", "gadgets", "gadget", "Gadget") + require.Eventually(t, func() bool { + conn := &corev1alpha1.Connection{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn); err != nil { + return false + } + _, ok := conn.Status.ExportsAPI("gadgets.gadget.io") + return ok + }, 30*time.Second, 200*time.Millisecond, "Connection should re-discover the newly-exported gadgets API") + }, + }, + { + // Tier 1: conflictPolicy Adopt takes over an un-owned provider object. + name: "conflictPolicy Adopt takes over an un-owned provider object", + step: func(t *testing.T) { + gadgetGVR := schema.GroupVersionResource{Group: "gadget.io", Version: "v1", Resource: "gadgets"} + gadgetGVK := schema.GroupVersionKind{Group: "gadget.io", Version: "v1", Kind: "Gadget"} + consumerGadgets := env.ConsumerDyn.Resource(gadgetGVR).Namespace(instanceNS) + providerGadgets := env.ProviderDyn.Resource(gadgetGVR).Namespace(instanceNS) + gadget := func(name, size string) *unstructured.Unstructured { + u := &unstructured.Unstructured{} + u.SetGroupVersionKind(gadgetGVK) + u.SetNamespace(instanceNS) + u.SetName(name) + _ = unstructured.SetNestedField(u.Object, size, "spec", "size") + return u + } + + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "gadgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: "gadgets.gadget.io"}}, + ConflictPolicy: corev1alpha1.ConflictPolicyAdopt, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "gadgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + // Pre-create a FOREIGN gadget on the provider (no markers). + _, err := providerGadgets.Create(ctx, gadget("shared-gadget", "PROVIDER-OWNED"), metav1.CreateOptions{}) + require.NoError(t, err) + // Create the same-named gadget on the consumer (CRD now pulled). + require.Eventually(t, func() bool { + _, err := consumerGadgets.Create(ctx, gadget("shared-gadget", "ADOPTED"), metav1.CreateOptions{}) + return err == nil || apierrors.IsAlreadyExists(err) + }, 30*time.Second, 200*time.Millisecond, "consumer Gadget CRD should become creatable") + + // Adopt: the provider object gets our markers and its spec is + // overwritten to the consumer's value. + require.Eventually(t, func() bool { + o, err := providerGadgets.Get(ctx, "shared-gadget", metav1.GetOptions{}) + if err != nil { + return false + } + size, _, _ := unstructured.NestedString(o.Object, "spec", "size") + return size == "ADOPTED" && + o.GetLabels()[corev1alpha1.LabelManaged] == "true" && + o.GetAnnotations()[corev1alpha1.AnnotationConsumerObjectUID] != "" + }, wait.ForeverTestTimeout, 200*time.Millisecond, "Adopt should take over the un-owned provider object") + }, + }, { // Gap 1: a Connection created before its Secret must resolve once the // Secret arrives (order-independent "one apply"). diff --git a/v2/sdk/apis/core/v1alpha1/labels.go b/v2/sdk/apis/core/v1alpha1/labels.go index c0b453c83..ced9ab379 100644 --- a/v2/sdk/apis/core/v1alpha1/labels.go +++ b/v2/sdk/apis/core/v1alpha1/labels.go @@ -40,6 +40,12 @@ const ( // synced provider object (ownership marker). AnnotationConsumerObjectUID = "core.kube-bind.io/consumer-object-uid" + // AnnotationConflict marks a consumer instance the konnector refused to sync + // because the provider target is owned by another binding/consumer. The + // value is the conflict reason. Bindings count annotated instances to report + // conflictCount. Survives status-schema pruning (it is metadata, not status). + AnnotationConflict = "core.kube-bind.io/conflict" + // FinalizerSyncer blocks consumer-object deletion until the provider copy // has been removed. FinalizerSyncer = "core.kube-bind.io/syncer" diff --git a/v2/sdk/go.sum b/v2/sdk/go.sum index e60cd8e92..89ad74c6c 100644 --- a/v2/sdk/go.sum +++ b/v2/sdk/go.sum @@ -1,3 +1,4 @@ +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -19,8 +20,13 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -30,9 +36,12 @@ github.com/onsi/ginkgo/v2 v2.22.0 h1:Yed107/8DjTr0lKCNt7Dn8yQ6ybuDRQoMGrNFKzMfHg github.com/onsi/ginkgo/v2 v2.22.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo= github.com/onsi/gomega v1.36.1 h1:bJDPBO7ibjxcbHMgSCoo4Yj18UWbKDlLwX1x9sybDcw= github.com/onsi/gomega v1.36.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog= +github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= +github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= @@ -78,6 +87,7 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= From 21f2e58f5febf795bed43353a7fc8ecc44fc4ecf Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 11:16:41 +0300 Subject: [PATCH 08/18] go mod bump --- v2/konnector/go.mod | 2 +- v2/sdk/go.mod | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/v2/konnector/go.mod b/v2/konnector/go.mod index 6697fd5b2..44278124e 100644 --- a/v2/konnector/go.mod +++ b/v2/konnector/go.mod @@ -1,6 +1,6 @@ module github.com/kube-bind/kube-bind/v2/konnector -go 1.24.0 +go 1.26.2 require ( github.com/go-logr/logr v1.4.2 diff --git a/v2/sdk/go.mod b/v2/sdk/go.mod index f1d3f342d..a4a314081 100644 --- a/v2/sdk/go.mod +++ b/v2/sdk/go.mod @@ -1,6 +1,6 @@ module github.com/kube-bind/kube-bind/v2/sdk -go 1.24.0 +go 1.26.2 require ( k8s.io/apiextensions-apiserver v0.33.4 From f189b384ff0cbd5abbb3b82ed3e9dbb1f0c1f6f6 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 11:50:13 +0300 Subject: [PATCH 09/18] wire in crdPuller, fix name clashes, deletion policies --- v2/README.md | 9 +- v2/konnector/engine/binding/cleanup.go | 5 +- v2/konnector/engine/binding/reconciler.go | 93 ++-------- v2/konnector/engine/connection/reconciler.go | 73 ++++++++ v2/konnector/engine/crdpull/crdpull.go | 177 +++++++++++++++++++ v2/konnector/engine/crdpull/crdpull_test.go | 148 ++++++++++++++++ v2/konnector/engine/sync/crd_controller.go | 4 + v2/konnector/engine/sync/syncer.go | 20 ++- v2/konnector/test/e2e/framework/framework.go | 23 +++ v2/konnector/test/e2e/tier2_test.go | 176 ++++++++++++++++++ v2/sdk/apis/core/v1alpha1/conditions.go | 2 + v2/sdk/apis/core/v1alpha1/labels.go | 10 ++ 12 files changed, 653 insertions(+), 87 deletions(-) create mode 100644 v2/konnector/engine/crdpull/crdpull.go create mode 100644 v2/konnector/engine/crdpull/crdpull_test.go create mode 100644 v2/konnector/test/e2e/tier2_test.go diff --git a/v2/README.md b/v2/README.md index 2e1ba9cf1..5a67c20e4 100644 --- a/v2/README.md +++ b/v2/README.md @@ -39,6 +39,10 @@ v2/ takes over an *un-owned* provider object (never one owned by another binding). - The Connection **re-discovers** exported APIs periodically, so a CRD labeled `exported` after connect is picked up and its binding goes Ready. +- Schema knobs are honored: `pullPolicy: Bound`/`All`/`None`, `updatePolicy: + Always`/`Once`, and `autoBind` (a managed ClusterBinding mirroring exported + APIs). `deletion-policy: Orphan` keeps a provider copy on delete/unbind. + Provider RBAC denials surface as a `PermissionDenied` condition / Event. - The provider side is the **multicluster-runtime engaged cluster** for each Connection: writes go through its client, fresh reads through its API reader, and status/drift events arrive via a **watch on its cache** (event-driven, not @@ -54,9 +58,8 @@ v2/ the provider — so `kubectl delete -f bundle.yaml` is order-don't-care. Known POC simplifications (tracked against the proposal): `schema.source: -OpenAPI`, related resources, `pullPolicy: All`/`None`, `updatePolicy: Always`, -`autoBind`, `deletion-policy: Orphan` and the provider `Lease` are not -implemented yet. +OpenAPI` (the kcp/CRD-less path), related resources, the `Mapper` extension, +and the provider `Lease` are not implemented yet. ## Build diff --git a/v2/konnector/engine/binding/cleanup.go b/v2/konnector/engine/binding/cleanup.go index 583e9071f..dd66486db 100644 --- a/v2/konnector/engine/binding/cleanup.go +++ b/v2/konnector/engine/binding/cleanup.go @@ -126,8 +126,9 @@ func (b *base) drainInstances(ctx context.Context, gvr schema.GroupVersionResour if !controllerutil.ContainsFinalizer(inst, corev1alpha1.FinalizerSyncer) { continue } - // Delete the provider copy if it is ours (best effort). - if providerClient != nil { + // Delete the provider copy if it is ours (best effort), unless the + // instance opted out with deletion-policy: Orphan. + if providerClient != nil && inst.GetAnnotations()[corev1alpha1.AnnotationDeletionPolicy] != corev1alpha1.DeletionPolicyOrphan { if err := deleteProviderCopy(ctx, providerClient, gvr, inst, localUID); err != nil { return err } diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go index f32f99bfd..05137f7de 100644 --- a/v2/konnector/engine/binding/reconciler.go +++ b/v2/konnector/engine/binding/reconciler.go @@ -21,14 +21,11 @@ package binding import ( "context" - "crypto/sha256" - "encoding/hex" "fmt" "sort" "strings" "time" - apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -41,6 +38,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/reconcile" + "github.com/kube-bind/kube-bind/v2/konnector/engine/crdpull" "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) @@ -108,8 +106,17 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc notExported = append(notExported, api.Name) continue } - hash, err := b.pullCRD(ctx, providerClient, api.Name, conn.Name) + hash, _, err := crdpull.Pull(ctx, b.client, providerClient, api.Name, conn.Name, crdpull.Options{ + Create: conn.Spec.Schema.PullPolicy != corev1alpha1.PullPolicyNone, + Update: conn.Spec.Schema.UpdatePolicy != corev1alpha1.UpdatePolicyOnce, + }) if err != nil { + if apierrors.IsForbidden(err) { + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionPermissionDenied, metav1.ConditionTrue, corev1alpha1.ReasonForbidden, + fmt.Sprintf("pulling CRD %q is forbidden by provider RBAC", api.Name))) + setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonForbidden, "provider RBAC denies reading the CRD") + return nil + } return fmt.Errorf("pulling CRD %q: %w", api.Name, err) } // Count instances we refused to sync due to a foreign provider target. @@ -139,88 +146,12 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionConflicts, metav1.ConditionFalse, corev1alpha1.ReasonAsExpected, "no conflicts")) } + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionPermissionDenied, metav1.ConditionFalse, corev1alpha1.ReasonAsExpected, "no RBAC denials")) apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionSynced, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "all APIs installed")) setReady(status, obj, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "binding ready") return nil } -// pullCRD reads the provider CRD by name and installs it on the consumer -// (create-if-absent; POC pulls once). It returns a content hash for status. -func (b *base) pullCRD(ctx context.Context, providerClient client.Client, crdName, connName string) (string, error) { - var remoteCRD apiextensionsv1.CustomResourceDefinition - if err := providerClient.Get(ctx, client.ObjectKey{Name: crdName}, &remoteCRD); err != nil { - return "", fmt.Errorf("reading provider CRD: %w", err) - } - - consumerCRD := crdForConsumer(&remoteCRD, connName) - hash := crdHash(consumerCRD) - - var existing apiextensionsv1.CustomResourceDefinition - err := b.client.Get(ctx, client.ObjectKey{Name: crdName}, &existing) - switch { - case apierrors.IsNotFound(err): - if err := b.client.Create(ctx, consumerCRD); err != nil { - return "", fmt.Errorf("creating consumer CRD: %w", err) - } - case err != nil: - return "", fmt.Errorf("getting consumer CRD: %w", err) - default: - // Already present. updatePolicy: Once for the POC — do not update. - } - return hash, nil -} - -// crdForConsumer strips the provider CRD down to something installable on the -// consumer without a conversion webhook: single served/storage version, -// conversion forced to None, no webhook/caBundle, no ownerRefs/status. -func crdForConsumer(in *apiextensionsv1.CustomResourceDefinition, connName string) *apiextensionsv1.CustomResourceDefinition { - out := in.DeepCopy() - out.ResourceVersion = "" - out.UID = "" - out.ManagedFields = nil - out.OwnerReferences = nil - out.Status = apiextensionsv1.CustomResourceDefinitionStatus{} - - // Keep only the storage version (fallback: first served), drop the rest, so - // no conversion webhook is needed on the consumer. - var keep *apiextensionsv1.CustomResourceDefinitionVersion - for i := range out.Spec.Versions { - v := &out.Spec.Versions[i] - if v.Storage { - keep = v - break - } - if keep == nil && v.Served { - keep = v - } - } - if keep != nil { - k := *keep - k.Storage = true - k.Served = true - out.Spec.Versions = []apiextensionsv1.CustomResourceDefinitionVersion{k} - } - out.Spec.Conversion = &apiextensionsv1.CustomResourceConversion{Strategy: apiextensionsv1.NoneConverter} - - if out.Labels == nil { - out.Labels = map[string]string{} - } - out.Labels[corev1alpha1.LabelManaged] = "true" - if out.Annotations == nil { - out.Annotations = map[string]string{} - } - out.Annotations[corev1alpha1.AnnotationConnection] = connName - return out -} - -func crdHash(crd *apiextensionsv1.CustomResourceDefinition) string { - h := sha256.New() - for _, v := range crd.Spec.Versions { - _, _ = h.Write([]byte(v.Name)) - } - return "sha256:" + hex.EncodeToString(h.Sum(nil))[:16] -} - func condition(obj client.Object, t string, s metav1.ConditionStatus, reason, msg string) metav1.Condition { return metav1.Condition{ Type: t, diff --git a/v2/konnector/engine/connection/reconciler.go b/v2/konnector/engine/connection/reconciler.go index 1d9e90ae7..a9aa47155 100644 --- a/v2/konnector/engine/connection/reconciler.go +++ b/v2/konnector/engine/connection/reconciler.go @@ -38,6 +38,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/reconcile" + "github.com/kube-bind/kube-bind/v2/konnector/engine/crdpull" "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) @@ -158,6 +159,9 @@ func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connectio // 2. Pin and verify cluster identity. remoteUID, err := remote.ClusterUID(ctx, providerClient) if err != nil { + if apierrors.IsForbidden(err) { + return r.denyPermission(conn, "reading provider cluster identity is forbidden by RBAC: "+err.Error()) + } setCondition(conn, corev1alpha1.ConditionConnected, metav1.ConditionFalse, corev1alpha1.ReasonPending, err.Error()) setNotReady(conn, corev1alpha1.ReasonPending, "cannot reach provider cluster") return nil @@ -181,14 +185,65 @@ func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connectio // 3. Discover exported APIs (schema.source: CRD — label-gated). exported, err := discoverExportedCRDs(ctx, providerClient) if err != nil { + if apierrors.IsForbidden(err) { + return r.denyPermission(conn, "listing exported CRDs is forbidden by RBAC: "+err.Error()) + } return fmt.Errorf("discovering exported APIs: %w", err) } conn.Status.ExportedAPIs = exported + setCondition(conn, corev1alpha1.ConditionPermissionDenied, metav1.ConditionFalse, corev1alpha1.ReasonAsExpected, "no RBAC denials") + + // pullPolicy: All — eagerly install every exported CRD on the consumer, + // without waiting for a binding. (Bound/None defer to the binding.) + if conn.Spec.Schema.PullPolicy == corev1alpha1.PullPolicyAll { + for i := range exported { + if _, _, err := crdpull.Pull(ctx, r.Client, providerClient, exported[i].Name, conn.Name, crdpull.Options{ + Create: true, + Update: conn.Spec.Schema.UpdatePolicy != corev1alpha1.UpdatePolicyOnce, + }); err != nil { + return fmt.Errorf("eager-pulling CRD %q: %w", exported[i].Name, err) + } + } + } + + // autoBind: maintain a managed ClusterBinding (named after the Connection) + // mirroring the exported APIs. + if conn.Spec.AutoBind { + if err := r.reconcileAutoBind(ctx, conn, exported); err != nil { + return fmt.Errorf("reconciling autoBind: %w", err) + } + } setCondition(conn, corev1alpha1.ConditionReady, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "connection ready") return nil } +// reconcileAutoBind keeps a managed ClusterBinding named after the Connection in +// sync with the exported APIs. With no exports it removes the managed binding. +func (r *Reconciler) reconcileAutoBind(ctx context.Context, conn *corev1alpha1.Connection, exported []corev1alpha1.ExportedAPI) error { + cb := &corev1alpha1.ClusterBinding{ObjectMeta: metav1.ObjectMeta{Name: conn.Name}} + if len(exported) == 0 { + return client.IgnoreNotFound(r.Client.Delete(ctx, cb)) + } + apis := make([]corev1alpha1.APIRef, 0, len(exported)) + for i := range exported { + apis = append(apis, corev1alpha1.APIRef{Name: exported[i].Name}) + } + _, err := controllerutil.CreateOrUpdate(ctx, r.Client, cb, func() error { + if err := controllerutil.SetControllerReference(conn, cb, r.Scheme); err != nil { + return err + } + if cb.Labels == nil { + cb.Labels = map[string]string{} + } + cb.Labels[corev1alpha1.LabelManaged] = "true" + cb.Spec.ConnectionRef = corev1alpha1.ConnectionRef{Name: conn.Name} + cb.Spec.APIs = apis + return nil + }) + return err +} + // ensureFinalizers adds the cleanup finalizer to the Connection and to its // referenced Secret (so the credential outlives the Connection during a // `delete -f` teardown). Returns true if it changed the Connection. @@ -223,6 +278,17 @@ func (r *Reconciler) reconcileDelete(ctx context.Context, conn *corev1alpha1.Con return ctrl.Result{}, nil } + // Explicitly delete the managed autoBind ClusterBinding so it drains; relying + // on owner-ref GC would deadlock (this Connection won't finalize until its + // bindings are gone, but GC won't remove the owned binding until the + // Connection is gone). + if conn.Spec.AutoBind { + managed := &corev1alpha1.ClusterBinding{ObjectMeta: metav1.ObjectMeta{Name: conn.Name}} + if err := r.Client.Delete(ctx, managed); client.IgnoreNotFound(err) != nil { + return ctrl.Result{}, err + } + } + refs, err := r.bindingsReferencing(ctx, conn.Name) if err != nil { return ctrl.Result{}, err @@ -353,6 +419,13 @@ func setNotReady(conn *corev1alpha1.Connection, reason, msg string) { setCondition(conn, corev1alpha1.ConditionReady, metav1.ConditionFalse, reason, msg) } +// denyPermission marks the Connection as blocked by provider RBAC. +func (r *Reconciler) denyPermission(conn *corev1alpha1.Connection, msg string) error { + setCondition(conn, corev1alpha1.ConditionPermissionDenied, metav1.ConditionTrue, corev1alpha1.ReasonForbidden, msg) + setNotReady(conn, corev1alpha1.ReasonForbidden, msg) + return nil // level-triggered: resolves when RBAC is granted (re-discovery requeue) +} + func apiequalStatus(a, b *corev1alpha1.Connection) bool { return apiequal(a.Status, b.Status) } diff --git a/v2/konnector/engine/crdpull/crdpull.go b/v2/konnector/engine/crdpull/crdpull.go new file mode 100644 index 000000000..d7ec17a12 --- /dev/null +++ b/v2/konnector/engine/crdpull/crdpull.go @@ -0,0 +1,177 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package crdpull centralizes how a provider CRD is installed on the consumer: +// stripping it to a consumer-installable form, creating it, and (per +// updatePolicy) updating it when the provider schema changes. Shared by the +// Connection (pullPolicy: All) and the binding (pullPolicy: Bound). +package crdpull + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// annotationSchemaHash records the hash of the installed schema so updatePolicy +// can detect provider changes without re-deriving. +const annotationSchemaHash = "core.kube-bind.io/schema-hash" + +// Options control how the CRD is reconciled on the consumer. +type Options struct { + // Create installs the CRD if absent (pullPolicy Bound/All). When false + // (pullPolicy None) an absent CRD is left to the user; an existing one is + // only stamped with the managed/connection markers so sync can find it. + Create bool + // Update re-applies the CRD spec when the provider schema changes + // (updatePolicy Always). When false (updatePolicy Once) the schema is pinned. + Update bool +} + +// Pull reconciles the consumer CRD for crdName from the provider. It returns the +// schema hash and whether the CRD is installed on the consumer. +func Pull(ctx context.Context, consumer client.Client, provider client.Reader, crdName, connName string, opts Options) (hash string, installed bool, err error) { + var remoteCRD apiextensionsv1.CustomResourceDefinition + if err := provider.Get(ctx, client.ObjectKey{Name: crdName}, &remoteCRD); err != nil { + return "", false, fmt.Errorf("reading provider CRD: %w", err) + } + desired := ForConsumer(&remoteCRD, connName) + hash = Hash(desired) + desired.Annotations[annotationSchemaHash] = hash + + var existing apiextensionsv1.CustomResourceDefinition + getErr := consumer.Get(ctx, client.ObjectKey{Name: crdName}, &existing) + switch { + case apierrors.IsNotFound(getErr): + if !opts.Create { + return hash, false, nil // None + absent: user manages it + } + if err := consumer.Create(ctx, desired); err != nil { + return "", false, fmt.Errorf("creating consumer CRD: %w", err) + } + return hash, true, nil + + case getErr != nil: + return "", false, getErr + + default: + if !opts.Create { + // None + present: only stamp the markers so the syncer can find it. + if stampMarkers(&existing, connName) { + if err := consumer.Update(ctx, &existing); err != nil { + return "", false, fmt.Errorf("stamping consumer CRD: %w", err) + } + } + return hash, true, nil + } + if opts.Update && existing.Annotations[annotationSchemaHash] != hash { + existing.Spec = desired.Spec + if existing.Annotations == nil { + existing.Annotations = map[string]string{} + } + existing.Annotations[annotationSchemaHash] = hash + existing.Annotations[corev1alpha1.AnnotationConnection] = connName + if existing.Labels == nil { + existing.Labels = map[string]string{} + } + existing.Labels[corev1alpha1.LabelManaged] = "true" + if err := consumer.Update(ctx, &existing); err != nil { + return "", false, fmt.Errorf("updating consumer CRD: %w", err) + } + } + return hash, true, nil + } +} + +// ForConsumer strips a provider CRD to something installable on the consumer +// without a conversion webhook: single storage/served version, conversion +// forced to None, no webhook/caBundle, no ownerRefs/status. It is marked managed +// and annotated with the source Connection. +func ForConsumer(in *apiextensionsv1.CustomResourceDefinition, connName string) *apiextensionsv1.CustomResourceDefinition { + out := in.DeepCopy() + out.ResourceVersion = "" + out.UID = "" + out.ManagedFields = nil + out.OwnerReferences = nil + out.Status = apiextensionsv1.CustomResourceDefinitionStatus{} + + var keep *apiextensionsv1.CustomResourceDefinitionVersion + for i := range out.Spec.Versions { + v := &out.Spec.Versions[i] + if v.Storage { + keep = v + break + } + if keep == nil && v.Served { + keep = v + } + } + if keep != nil { + k := *keep + k.Storage = true + k.Served = true + out.Spec.Versions = []apiextensionsv1.CustomResourceDefinitionVersion{k} + } + out.Spec.Conversion = &apiextensionsv1.CustomResourceConversion{Strategy: apiextensionsv1.NoneConverter} + + if out.Labels == nil { + out.Labels = map[string]string{} + } + out.Labels[corev1alpha1.LabelManaged] = "true" + if out.Annotations == nil { + out.Annotations = map[string]string{} + } + out.Annotations[corev1alpha1.AnnotationConnection] = connName + return out +} + +// Hash is a content hash over the consumer CRD spec, so updatePolicy: Always can +// detect provider schema changes. +func Hash(crd *apiextensionsv1.CustomResourceDefinition) string { + b, err := json.Marshal(crd.Spec) + if err != nil { + return "" + } + sum := sha256.Sum256(b) + return "sha256:" + hex.EncodeToString(sum[:])[:16] +} + +func stampMarkers(crd *apiextensionsv1.CustomResourceDefinition, connName string) bool { + changed := false + if crd.Labels[corev1alpha1.LabelManaged] != "true" { + if crd.Labels == nil { + crd.Labels = map[string]string{} + } + crd.Labels[corev1alpha1.LabelManaged] = "true" + changed = true + } + if crd.Annotations[corev1alpha1.AnnotationConnection] != connName { + if crd.Annotations == nil { + crd.Annotations = map[string]string{} + } + crd.Annotations[corev1alpha1.AnnotationConnection] = connName + changed = true + } + return changed +} diff --git a/v2/konnector/engine/crdpull/crdpull_test.go b/v2/konnector/engine/crdpull/crdpull_test.go new file mode 100644 index 000000000..c11796c3b --- /dev/null +++ b/v2/konnector/engine/crdpull/crdpull_test.go @@ -0,0 +1,148 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package crdpull + +import ( + "context" + "testing" + + "github.com/stretchr/testify/require" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +func scheme(t *testing.T) *runtime.Scheme { + t.Helper() + s := runtime.NewScheme() + require.NoError(t, apiextensionsv1.AddToScheme(s)) + return s +} + +func providerCRD(printerCol string) *apiextensionsv1.CustomResourceDefinition { + return &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets.example.org"}, + Spec: apiextensionsv1.CustomResourceDefinitionSpec{ + Group: "example.org", + Names: apiextensionsv1.CustomResourceDefinitionNames{Plural: "widgets", Kind: "Widget"}, + Scope: apiextensionsv1.NamespaceScoped, + Versions: []apiextensionsv1.CustomResourceDefinitionVersion{{ + Name: "v1", + Served: true, + Storage: true, + AdditionalPrinterColumns: []apiextensionsv1.CustomResourceColumnDefinition{ + {Name: printerCol, Type: "string", JSONPath: ".spec." + printerCol}, + }, + }}, + }, + } +} + +func TestPull_BoundCreatesCRD(t *testing.T) { + s := scheme(t) + provider := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("size")).Build() + consumer := fake.NewClientBuilder().WithScheme(s).Build() + + hash, installed, err := Pull(context.Background(), consumer, provider, "widgets.example.org", "conn", Options{Create: true}) + require.NoError(t, err) + require.True(t, installed) + require.NotEmpty(t, hash) + + got := &apiextensionsv1.CustomResourceDefinition{} + require.NoError(t, consumer.Get(context.Background(), client.ObjectKey{Name: "widgets.example.org"}, got)) + require.Equal(t, "true", got.Labels[corev1alpha1.LabelManaged]) + require.Equal(t, "conn", got.Annotations[corev1alpha1.AnnotationConnection]) +} + +func TestPull_UpdatePolicyOnce_DoesNotUpdate(t *testing.T) { + s := scheme(t) + provider := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("size")).Build() + consumer := fake.NewClientBuilder().WithScheme(s).Build() + + // First pull installs it. + _, _, err := Pull(context.Background(), consumer, provider, "widgets.example.org", "conn", Options{Create: true, Update: true}) + require.NoError(t, err) + + // Provider schema changes, but updatePolicy: Once (Update=false) pins it. + provider2 := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("colour")).Build() + _, _, err = Pull(context.Background(), consumer, provider2, "widgets.example.org", "conn", Options{Create: true, Update: false}) + require.NoError(t, err) + + got := &apiextensionsv1.CustomResourceDefinition{} + require.NoError(t, consumer.Get(context.Background(), client.ObjectKey{Name: "widgets.example.org"}, got)) + require.Equal(t, "size", got.Spec.Versions[0].AdditionalPrinterColumns[0].Name, "Once must not follow provider changes") +} + +func TestPull_UpdatePolicyAlways_FollowsProviderChanges(t *testing.T) { + s := scheme(t) + provider := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("size")).Build() + consumer := fake.NewClientBuilder().WithScheme(s).Build() + + _, hash1, err := pull3(t, consumer, provider, Options{Create: true, Update: true}) + require.NoError(t, err) + + provider2 := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("colour")).Build() + _, hash2, err := pull3(t, consumer, provider2, Options{Create: true, Update: true}) + require.NoError(t, err) + require.NotEqual(t, hash1, hash2, "schema hash must change when provider schema changes") + + got := &apiextensionsv1.CustomResourceDefinition{} + require.NoError(t, consumer.Get(context.Background(), client.ObjectKey{Name: "widgets.example.org"}, got)) + require.Equal(t, "colour", got.Spec.Versions[0].AdditionalPrinterColumns[0].Name, "Always must follow provider changes") +} + +func TestPull_NoneAbsent_NotInstalled(t *testing.T) { + s := scheme(t) + provider := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("size")).Build() + consumer := fake.NewClientBuilder().WithScheme(s).Build() + + _, installed, err := Pull(context.Background(), consumer, provider, "widgets.example.org", "conn", Options{Create: false}) + require.NoError(t, err) + require.False(t, installed, "pullPolicy None must not create the CRD") + + got := &apiextensionsv1.CustomResourceDefinition{} + require.True(t, client.IgnoreNotFound(consumer.Get(context.Background(), client.ObjectKey{Name: "widgets.example.org"}, got)) == nil) + require.Empty(t, got.Name) +} + +func TestPull_NonePresent_StampsMarkers(t *testing.T) { + s := scheme(t) + // A user-installed CRD already on the consumer, without our markers. + existing := providerCRD("size") + provider := fake.NewClientBuilder().WithScheme(s).WithObjects(providerCRD("size")).Build() + consumer := fake.NewClientBuilder().WithScheme(s).WithObjects(existing.DeepCopy()).Build() + + _, installed, err := Pull(context.Background(), consumer, provider, "widgets.example.org", "conn", Options{Create: false}) + require.NoError(t, err) + require.True(t, installed) + + got := &apiextensionsv1.CustomResourceDefinition{} + require.NoError(t, consumer.Get(context.Background(), client.ObjectKey{Name: "widgets.example.org"}, got)) + require.Equal(t, "true", got.Labels[corev1alpha1.LabelManaged], "None must stamp an existing CRD so the syncer finds it") + require.Equal(t, "conn", got.Annotations[corev1alpha1.AnnotationConnection]) +} + +// pull3 returns (installed, hash, err) reordered for convenience. +func pull3(t *testing.T, consumer client.Client, provider client.Reader, opts Options) (bool, string, error) { + t.Helper() + hash, installed, err := Pull(context.Background(), consumer, provider, "widgets.example.org", "conn", opts) + return installed, hash, err +} diff --git a/v2/konnector/engine/sync/crd_controller.go b/v2/konnector/engine/sync/crd_controller.go index e01e56efb..d81b1c7d4 100644 --- a/v2/konnector/engine/sync/crd_controller.go +++ b/v2/konnector/engine/sync/crd_controller.go @@ -40,6 +40,7 @@ import ( "k8s.io/client-go/tools/cache" "k8s.io/client-go/tools/record" "k8s.io/client-go/util/workqueue" + "k8s.io/utils/ptr" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/cluster" @@ -197,6 +198,9 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte ctrlr, err := controller.NewTypedUnmanaged(fmt.Sprintf("sync-%s", gvr.GroupResource().String()), controller.TypedOptions[reconcile.Request]{ Reconciler: rec, LogConstructor: func(*reconcile.Request) logr.Logger { return syncLog }, + // A CRD can be removed and re-added (same GVR), recreating this + // controller with the same name — skip the global uniqueness check. + SkipNameValidation: ptr.To(true), }) if err != nil { return reconcile.Result{}, fmt.Errorf("creating sync controller: %w", err) diff --git a/v2/konnector/engine/sync/syncer.go b/v2/konnector/engine/sync/syncer.go index a53e909b5..d16830e0d 100644 --- a/v2/konnector/engine/sync/syncer.go +++ b/v2/konnector/engine/sync/syncer.go @@ -145,6 +145,9 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( case apierrors.IsNotFound(getErr): if r.scope == apiextensionsv1.NamespaceScoped { if err := ensureNamespace(ctx, r.providerClient, obj.GetNamespace(), r.localUID); err != nil { + if apierrors.IsForbidden(err) { + return r.permissionDenied(obj, "creating provider namespace", err), nil + } return reconcile.Result{}, err } } @@ -160,6 +163,9 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( // Spec consumer -> provider (SSA). patch := r.providerPatch(obj, r.localUID) if err := r.providerClient.Patch(ctx, patch, client.Apply, client.FieldOwner(fieldOwner), client.ForceOwnership); err != nil { + if apierrors.IsForbidden(err) { + return r.permissionDenied(obj, "applying spec to provider", err), nil + } return reconcile.Result{}, fmt.Errorf("applying spec to provider: %w", err) } @@ -193,6 +199,17 @@ func (r *specReconciler) markConflict(ctx context.Context, obj *unstructured.Uns return nil } +// permissionDenied surfaces a provider RBAC denial as an Event and requeues — +// it never fails the whole controller, so other objects keep syncing. The denial +// resolves on its own once RBAC is granted. +func (r *specReconciler) permissionDenied(obj *unstructured.Unstructured, op string, err error) reconcile.Result { + if r.recorder != nil { + r.recorder.Event(obj, corev1.EventTypeWarning, corev1alpha1.ReasonForbidden, + fmt.Sprintf("%s is forbidden by provider RBAC: %v", op, err)) + } + return reconcile.Result{RequeueAfter: 30 * time.Second} +} + // clearConflict removes the conflict marker once the object syncs cleanly. func (r *specReconciler) clearConflict(ctx context.Context, obj *unstructured.Unstructured) error { if obj.GetAnnotations()[corev1alpha1.AnnotationConflict] == "" { @@ -209,7 +226,8 @@ func (r *specReconciler) reconcileDelete(ctx context.Context, obj *unstructured. if !controllerutil.ContainsFinalizer(obj, corev1alpha1.FinalizerSyncer) { return reconcile.Result{}, nil } - { + // deletion-policy: Orphan keeps the provider copy — just release the finalizer. + if obj.GetAnnotations()[corev1alpha1.AnnotationDeletionPolicy] != corev1alpha1.DeletionPolicyOrphan { existing := newUnstructured(r.gvk) err := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), existing) switch { diff --git a/v2/konnector/test/e2e/framework/framework.go b/v2/konnector/test/e2e/framework/framework.go index 778f33d59..456df2202 100644 --- a/v2/konnector/test/e2e/framework/framework.go +++ b/v2/konnector/test/e2e/framework/framework.go @@ -45,8 +45,10 @@ import ( "k8s.io/client-go/rest" "k8s.io/client-go/tools/clientcmd" clientcmdapi "k8s.io/client-go/tools/clientcmd/api" + "k8s.io/utils/ptr" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/config" "sigs.k8s.io/controller-runtime/pkg/envtest" "sigs.k8s.io/controller-runtime/pkg/log/zap" metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" @@ -72,6 +74,23 @@ type Env struct { ConsumerClient client.Client ProviderDyn dynamic.Interface ConsumerDyn dynamic.Interface + + providerEnv *envtest.Environment +} + +// RestrictedProviderSecret creates a Secret on the consumer holding a kubeconfig +// for a provider user with no RBAC, so the konnector's provider operations are +// forbidden. Used to exercise the PermissionDenied path. +func (e *Env) RestrictedProviderSecret(t *testing.T, name string) *corev1.Secret { + t.Helper() + au, err := e.providerEnv.AddUser(envtest.User{Name: "restricted-konnector"}, nil) + require.NoError(t, err, "adding restricted provider user") + kubeconfig, err := kubeconfigFromRestConfig(au.Config()) + require.NoError(t, err) + return &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: KubeBindNamespace}, + Data: map[string][]byte{"kubeconfig": kubeconfig}, + } } // Start brings up two envtest API servers, installs the core CRDs on the @@ -141,6 +160,7 @@ func Start(t *testing.T) *Env { ConsumerClient: consumerClient, ProviderDyn: providerDyn, ConsumerDyn: consumerDyn, + providerEnv: providerEnv, } } @@ -153,6 +173,9 @@ func startEngine(t *testing.T, consumerCfg *rest.Config, scheme *apimachineryrun localMgr, err := ctrl.NewManager(consumerCfg, ctrl.Options{ Scheme: scheme, Metrics: metricsserver.Options{BindAddress: "0"}, + // Multiple test envs run in one process; skip the global controller-name + // uniqueness check. + Controller: config.Controller{SkipNameValidation: ptr.To(true)}, }) require.NoError(t, err) diff --git a/v2/konnector/test/e2e/tier2_test.go b/v2/konnector/test/e2e/tier2_test.go new file mode 100644 index 000000000..dcc690b3f --- /dev/null +++ b/v2/konnector/test/e2e/tier2_test.go @@ -0,0 +1,176 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package e2e + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/wait" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/kube-bind/kube-bind/v2/konnector/test/e2e/framework" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// TestSlimCoreTier2 exercises the Tier-2 "Decided" features. It shares one +// envtest pair (spinning a fresh one per case exhausts envtest resources): +// deletion-policy Orphan, updatePolicy Always, autoBind, pullPolicy All, and +// PermissionDenied. The widgets-dependent cases run first, before the extra +// Connections (autoBind/All) re-stamp the widgets CRD. +func TestSlimCoreTier2(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + gvr := env.InstallExportedWidgetCRD(t) + consumerWidgets := env.ConsumerDyn.Resource(gvr).Namespace(instanceNS) + providerWidgets := env.ProviderDyn.Resource(gvr).Namespace(instanceNS) + + // Base bundle: demo-provider Connection + widgets ClusterBinding. + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + t.Run("deletion-policy Orphan keeps the provider copy", func(t *testing.T) { + w := widget("orphan-widget", "keep") + w.SetAnnotations(map[string]string{corev1alpha1.AnnotationDeletionPolicy: corev1alpha1.DeletionPolicyOrphan}) + _, err := consumerWidgets.Create(ctx, w, metav1.CreateOptions{}) + require.NoError(t, err) + require.Eventually(t, func() bool { + _, err := providerWidgets.Get(ctx, "orphan-widget", metav1.GetOptions{}) + return err == nil + }, wait.ForeverTestTimeout, 200*time.Millisecond, "orphan-widget should sync to the provider") + + require.NoError(t, consumerWidgets.Delete(ctx, "orphan-widget", metav1.DeleteOptions{})) + require.Eventually(t, func() bool { + _, err := consumerWidgets.Get(ctx, "orphan-widget", metav1.GetOptions{}) + return apierrors.IsNotFound(err) + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer object should finalize despite Orphan") + require.Never(t, func() bool { + _, err := providerWidgets.Get(ctx, "orphan-widget", metav1.GetOptions{}) + return apierrors.IsNotFound(err) + }, 3*time.Second, 500*time.Millisecond, "Orphan must keep the provider copy") + }) + + t.Run("updatePolicy Always follows provider CRD changes", func(t *testing.T) { + require.NoError(t, retry.RetryOnConflict(retry.DefaultRetry, func() error { + crd := &apiextensionsv1.CustomResourceDefinition{} + if err := env.ProviderClient.Get(ctx, client.ObjectKey{Name: widgetCRDName}, crd); err != nil { + return err + } + crd.Spec.Versions[0].AdditionalPrinterColumns = append(crd.Spec.Versions[0].AdditionalPrinterColumns, + apiextensionsv1.CustomResourceColumnDefinition{Name: "colour", Type: "string", JSONPath: ".spec.colour"}) + return env.ProviderClient.Update(ctx, crd) + })) + require.Eventually(t, func() bool { + crd := &apiextensionsv1.CustomResourceDefinition{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: widgetCRDName}, crd); err != nil { + return false + } + for _, v := range crd.Spec.Versions { + for _, col := range v.AdditionalPrinterColumns { + if col.Name == "colour" { + return true + } + } + } + return false + }, wait.ForeverTestTimeout, 200*time.Millisecond, "consumer CRD should follow the provider schema change (updatePolicy: Always)") + }) + + t.Run("autoBind maintains a managed ClusterBinding", func(t *testing.T) { + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "auto-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + AutoBind: true, + }, + })) + require.Eventually(t, func() bool { + cb := &corev1alpha1.ClusterBinding{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "auto-provider"}, cb); err != nil { + return false + } + if cb.Labels[corev1alpha1.LabelManaged] != "true" || len(cb.OwnerReferences) == 0 { + return false + } + for _, a := range cb.Spec.APIs { + if a.Name == widgetCRDName { + return true + } + } + return false + }, 30*time.Second, 200*time.Millisecond, "autoBind should maintain a managed ClusterBinding mirroring exportedAPIs") + }) + + t.Run("pullPolicy All installs CRDs without a binding", func(t *testing.T) { + env.InstallExportedCRD(t, "doodad.io", "doodads", "doodad", "Doodad") + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "all-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD, PullPolicy: corev1alpha1.PullPolicyAll}, + }, + })) + require.Eventually(t, func() bool { + crd := &apiextensionsv1.CustomResourceDefinition{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "doodads.doodad.io"}, crd) + return err == nil && crd.Labels[corev1alpha1.LabelManaged] == "true" + }, 30*time.Second, 200*time.Millisecond, "pullPolicy: All should install the CRD without a binding") + }) + + t.Run("PermissionDenied surfaces on a restricted Connection", func(t *testing.T) { + require.NoError(t, env.ConsumerClient.Create(ctx, env.RestrictedProviderSecret(t, "restricted-secret"))) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "restricted-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "restricted-secret", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + require.Eventually(t, func() bool { + conn := &corev1alpha1.Connection{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "restricted-provider"}, conn); err != nil { + return false + } + return isConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionPermissionDenied) && + !isConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionReady) + }, 30*time.Second, 200*time.Millisecond, "a restricted Connection should report PermissionDenied and not be Ready") + }) +} diff --git a/v2/sdk/apis/core/v1alpha1/conditions.go b/v2/sdk/apis/core/v1alpha1/conditions.go index 3e8240431..05f143e76 100644 --- a/v2/sdk/apis/core/v1alpha1/conditions.go +++ b/v2/sdk/apis/core/v1alpha1/conditions.go @@ -67,4 +67,6 @@ const ( // ReasonOwnedByAnother marks a conflict with an object owned by a different // binding/consumer. ReasonOwnedByAnother = "OwnedByAnother" + // ReasonForbidden marks an operation refused by provider RBAC. + ReasonForbidden = "Forbidden" ) diff --git a/v2/sdk/apis/core/v1alpha1/labels.go b/v2/sdk/apis/core/v1alpha1/labels.go index ced9ab379..8cbe581c9 100644 --- a/v2/sdk/apis/core/v1alpha1/labels.go +++ b/v2/sdk/apis/core/v1alpha1/labels.go @@ -46,6 +46,16 @@ const ( // conflictCount. Survives status-schema pruning (it is metadata, not status). AnnotationConflict = "core.kube-bind.io/conflict" + // AnnotationDeletionPolicy on a consumer instance controls what happens to + // its provider copy when the consumer object is deleted or unbound. The only + // non-default value is "Orphan": release the finalizer without deleting the + // provider copy (keep the managed object on the provider). + AnnotationDeletionPolicy = "core.kube-bind.io/deletion-policy" + + // DeletionPolicyOrphan keeps the provider copy when the consumer object is + // deleted or unbound. + DeletionPolicyOrphan = "Orphan" + // FinalizerSyncer blocks consumer-object deletion until the provider copy // has been removed. FinalizerSyncer = "core.kube-bind.io/syncer" From 190bd72e72e9860df8040905cd153b1a70705675 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 11:50:17 +0300 Subject: [PATCH 10/18] tidy --- v2/konnector/go.mod | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2/konnector/go.mod b/v2/konnector/go.mod index 44278124e..ee3b816fe 100644 --- a/v2/konnector/go.mod +++ b/v2/konnector/go.mod @@ -11,6 +11,7 @@ require ( k8s.io/apiextensions-apiserver v0.33.4 k8s.io/apimachinery v0.33.4 k8s.io/client-go v0.33.4 + k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 sigs.k8s.io/controller-runtime v0.21.0 sigs.k8s.io/multicluster-runtime v0.21.0-alpha.9 ) @@ -62,7 +63,6 @@ require ( gopkg.in/yaml.v3 v3.0.1 // indirect k8s.io/klog/v2 v2.130.1 // indirect k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff // indirect - k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect sigs.k8s.io/randfill v1.0.0 // indirect sigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect From 983bd4f04252f2087e199beaece3c91e6eb73722 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 12:28:22 +0300 Subject: [PATCH 11/18] Implement provider lease, related resources, schema --- v2/README.md | 16 +- v2/konnector/engine/binding/cleanup.go | 4 +- v2/konnector/engine/binding/reconciler.go | 45 +++- v2/konnector/engine/binding/related.go | 219 +++++++++++++++++ v2/konnector/engine/connection/reconciler.go | 121 +++++++++- v2/konnector/engine/crdpull/crdpull.go | 37 ++- v2/konnector/engine/openapi/openapi.go | 223 ++++++++++++++++++ v2/konnector/test/e2e/sync_test.go | 33 +++ v2/konnector/test/e2e/tier3_test.go | 166 +++++++++++++ v2/sdk/apis/core/v1alpha1/connection_types.go | 7 + v2/sdk/apis/core/v1alpha1/helpers.go | 2 + v2/sdk/apis/core/v1alpha1/labels.go | 5 + .../core/v1alpha1/zz_generated.deepcopy.go | 2 +- .../crd/core.kube-bind.io_connections.yaml | 6 + 14 files changed, 860 insertions(+), 26 deletions(-) create mode 100644 v2/konnector/engine/binding/related.go create mode 100644 v2/konnector/engine/openapi/openapi.go create mode 100644 v2/konnector/test/e2e/tier3_test.go diff --git a/v2/README.md b/v2/README.md index 5a67c20e4..9579d06cb 100644 --- a/v2/README.md +++ b/v2/README.md @@ -43,6 +43,14 @@ v2/ Always`/`Once`, and `autoBind` (a managed ClusterBinding mirroring exported APIs). `deletion-policy: Orphan` keeps a provider copy on delete/unbind. Provider RBAC denials surface as a `PermissionDenied` condition / Event. +- `schema.source: OpenAPI` (and `Auto`) synthesizes the consumer CRD from the + provider's discovery + `/openapi/v3` — the CRD-less (kcp-like) path — and the + Connection installs it. Known fidelity limits (CEL, defaulting, `$ref`, + multi-version) are accepted; the provider stays the enforcing side. +- The konnector maintains a `coordination.k8s.io/Lease` per Connection on the + provider (heartbeat) — the hook a service-layer reaper keys off. +- `relatedResources` sync selected Secrets/ConfigMaps in the declared direction, + scoped like the binding, GC'd when they stop matching or on unbind. - The provider side is the **multicluster-runtime engaged cluster** for each Connection: writes go through its client, fresh reads through its API reader, and status/drift events arrive via a **watch on its cache** (event-driven, not @@ -57,9 +65,11 @@ v2/ gone, and keeps its Secret alive (via a finalizer) so teardown can still reach the provider — so `kubectl delete -f bundle.yaml` is order-don't-care. -Known POC simplifications (tracked against the proposal): `schema.source: -OpenAPI` (the kcp/CRD-less path), related resources, the `Mapper` extension, -and the provider `Lease` are not implemented yet. +Known POC simplifications (tracked against the proposal): the `Mapper` +extension is not implemented; OpenAPI synthesis is best-effort (fidelity limits +above); cluster identity still derives from the `kube-system` namespace UID, so +true kcp logical clusters (no kube-system) need an alternative identity source; +and syncer stop-on-disengage + productionization (RBAC/HA/Helm) remain. ## Build diff --git a/v2/konnector/engine/binding/cleanup.go b/v2/konnector/engine/binding/cleanup.go index dd66486db..516fc506f 100644 --- a/v2/konnector/engine/binding/cleanup.go +++ b/v2/konnector/engine/binding/cleanup.go @@ -81,7 +81,9 @@ func (b *base) cleanup(ctx context.Context, obj corev1alpha1.BindingAccessor, na } } } - return nil + + // Delete related Secret/ConfigMap copies this binding created. + return b.cleanupRelated(ctx, obj, namespace, providerClient) } // countConflicts counts consumer instances of gvr in scope that the syncer diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go index 05137f7de..8b8635eeb 100644 --- a/v2/konnector/engine/binding/reconciler.go +++ b/v2/konnector/engine/binding/reconciler.go @@ -26,6 +26,7 @@ import ( "strings" "time" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -106,18 +107,34 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc notExported = append(notExported, api.Name) continue } - hash, _, err := crdpull.Pull(ctx, b.client, providerClient, api.Name, conn.Name, crdpull.Options{ - Create: conn.Spec.Schema.PullPolicy != corev1alpha1.PullPolicyNone, - Update: conn.Spec.Schema.UpdatePolicy != corev1alpha1.UpdatePolicyOnce, - }) - if err != nil { - if apierrors.IsForbidden(err) { - apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionPermissionDenied, metav1.ConditionTrue, corev1alpha1.ReasonForbidden, - fmt.Sprintf("pulling CRD %q is forbidden by provider RBAC", api.Name))) - setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonForbidden, "provider RBAC denies reading the CRD") - return nil + var hash string + if conn.Status.ActiveSchemaSource == corev1alpha1.SchemaSourceOpenAPI { + // CRD-less provider: the Connection synthesized + installed the CRD. + // The binding just consumes it (no provider apiextensions to read). + crd := &apiextensionsv1.CustomResourceDefinition{} + if err := b.client.Get(ctx, client.ObjectKey{Name: api.Name}, crd); err != nil { + if apierrors.IsNotFound(err) { + notExported = append(notExported, api.Name) + continue + } + return fmt.Errorf("getting synthesized CRD %q: %w", api.Name, err) } - return fmt.Errorf("pulling CRD %q: %w", api.Name, err) + hash = crd.Annotations[crdpull.AnnotationSchemaHash] + } else { + h, _, err := crdpull.Pull(ctx, b.client, providerClient, api.Name, conn.Name, crdpull.Options{ + Create: conn.Spec.Schema.PullPolicy != corev1alpha1.PullPolicyNone, + Update: conn.Spec.Schema.UpdatePolicy != corev1alpha1.UpdatePolicyOnce, + }) + if err != nil { + if apierrors.IsForbidden(err) { + apimeta.SetStatusCondition(&status.Conditions, condition(obj, corev1alpha1.ConditionPermissionDenied, metav1.ConditionTrue, corev1alpha1.ReasonForbidden, + fmt.Sprintf("pulling CRD %q is forbidden by provider RBAC", api.Name))) + setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonForbidden, "provider RBAC denies reading the CRD") + return nil + } + return fmt.Errorf("pulling CRD %q: %w", api.Name, err) + } + hash = h } // Count instances we refused to sync due to a foreign provider target. var conflicts int32 @@ -135,6 +152,12 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc return nil } + // Sync related Secrets/ConfigMaps in the declared direction, scoped like the + // binding. + if err := b.reconcileRelated(ctx, obj, namespace, providerClient, conn.Status.LocalClusterUID); err != nil { + return fmt.Errorf("syncing related resources: %w", err) + } + var totalConflicts int32 for _, ba := range boundAPIs { totalConflicts += ba.ConflictCount diff --git a/v2/konnector/engine/binding/related.go b/v2/konnector/engine/binding/related.go new file mode 100644 index 000000000..8ee1cde72 --- /dev/null +++ b/v2/konnector/engine/binding/related.go @@ -0,0 +1,219 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package binding + +import ( + "context" + "fmt" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// relatedFieldOwner is the SSA field manager for related-resource writes. +const relatedFieldOwner = "kube-bind-konnector-related" + +// relatedGVK maps a related-resource kind to its GVK. Only secrets/configmaps. +func relatedGVK(resource string) (schema.GroupVersionKind, bool) { + switch resource { + case "secrets": + return schema.GroupVersionKind{Version: "v1", Kind: "Secret"}, true + case "configmaps": + return schema.GroupVersionKind{Version: "v1", Kind: "ConfigMap"}, true + default: + return schema.GroupVersionKind{}, false + } +} + +// reconcileRelated syncs the binding's related Secrets/ConfigMaps in the +// declared direction, scoped like the binding, and garbage-collects copies that +// no longer match. Identity-preserving, marked with the binding UID. +func (b *base) reconcileRelated(ctx context.Context, obj corev1alpha1.BindingAccessor, namespace string, providerClient client.Client, localUID string) error { + spec := obj.BindingSpecP() + bindingUID := string(obj.GetUID()) + for i := range spec.RelatedResources { + rr := &spec.RelatedResources[i] + gvk, ok := relatedGVK(rr.Resource) + if !ok { + continue + } + source, target := b.client, providerClient // FromConsumer + if rr.Direction == corev1alpha1.FromProvider { + source, target = providerClient, b.client + } + + sel := labels.Everything() + if rr.Selector != nil && rr.Selector.LabelSelector != nil { + s, err := metav1.LabelSelectorAsSelector(rr.Selector.LabelSelector) + if err != nil { + return fmt.Errorf("invalid related selector: %w", err) + } + sel = s + } + names := nameSet(rr.Selector) + + list := &unstructured.UnstructuredList{} + list.SetGroupVersionKind(gvk.GroupVersion().WithKind(gvk.Kind + "List")) + opts := []client.ListOption{client.MatchingLabelsSelector{Selector: sel}} + if namespace != "" { + opts = append(opts, client.InNamespace(namespace)) + } + if err := source.List(ctx, list, opts...); err != nil { + return fmt.Errorf("listing related %s: %w", rr.Resource, err) + } + + matched := map[client.ObjectKey]bool{} + for j := range list.Items { + src := &list.Items[j] + if names != nil && !names[src.GetName()] { + continue + } + key := client.ObjectKey{Namespace: src.GetNamespace(), Name: src.GetName()} + matched[key] = true + if err := applyRelated(ctx, target, gvk, src, localUID, bindingUID); err != nil { + return err + } + } + if err := b.gcRelated(ctx, target, gvk, namespace, bindingUID, matched); err != nil { + return err + } + } + return nil +} + +// applyRelated copies a related object to the target side (identity-preserving) +// with our markers, refusing to overwrite a foreign object of the same name. +func applyRelated(ctx context.Context, target client.Client, gvk schema.GroupVersionKind, src *unstructured.Unstructured, localUID, bindingUID string) error { + key := client.ObjectKey{Namespace: src.GetNamespace(), Name: src.GetName()} + + existing := &unstructured.Unstructured{} + existing.SetGroupVersionKind(gvk) + switch err := target.Get(ctx, key, existing); { + case err == nil: + if existing.GetAnnotations()[corev1alpha1.AnnotationRelatedBinding] != bindingUID { + return nil // foreign object — never overwrite + } + case apierrors.IsNotFound(err): + if src.GetNamespace() != "" { + if err := ensureNamespace(ctx, target, src.GetNamespace(), localUID); err != nil { + return err + } + } + default: + return fmt.Errorf("getting related target: %w", err) + } + + out := &unstructured.Unstructured{} + out.SetGroupVersionKind(gvk) + out.SetNamespace(src.GetNamespace()) + out.SetName(src.GetName()) + out.SetLabels(map[string]string{corev1alpha1.LabelManaged: "true"}) + out.SetAnnotations(map[string]string{ + corev1alpha1.AnnotationConsumerClusterUID: localUID, + corev1alpha1.AnnotationRelatedBinding: bindingUID, + }) + for _, f := range []string{"data", "stringData", "binaryData", "type", "immutable"} { + if v, ok, _ := unstructured.NestedFieldCopy(src.Object, f); ok { + _ = unstructured.SetNestedField(out.Object, v, f) + } + } + if err := target.Patch(ctx, out, client.Apply, client.FieldOwner(relatedFieldOwner), client.ForceOwnership); err != nil { + return fmt.Errorf("applying related %s/%s: %w", src.GetNamespace(), src.GetName(), err) + } + return nil +} + +// gcRelated deletes target copies we own (this binding) that are no longer in +// the matched set. +func (b *base) gcRelated(ctx context.Context, target client.Client, gvk schema.GroupVersionKind, namespace, bindingUID string, matched map[client.ObjectKey]bool) error { + list := &unstructured.UnstructuredList{} + list.SetGroupVersionKind(gvk.GroupVersion().WithKind(gvk.Kind + "List")) + opts := []client.ListOption{client.MatchingLabels{corev1alpha1.LabelManaged: "true"}} + if namespace != "" { + opts = append(opts, client.InNamespace(namespace)) + } + if err := target.List(ctx, list, opts...); err != nil { + return fmt.Errorf("listing managed related %s: %w", gvk.Kind, err) + } + for i := range list.Items { + o := &list.Items[i] + if o.GetAnnotations()[corev1alpha1.AnnotationRelatedBinding] != bindingUID { + continue + } + if matched[client.ObjectKey{Namespace: o.GetNamespace(), Name: o.GetName()}] { + continue + } + if err := target.Delete(ctx, o); client.IgnoreNotFound(err) != nil { + return fmt.Errorf("gc related %s/%s: %w", o.GetNamespace(), o.GetName(), err) + } + } + return nil +} + +// cleanupRelated deletes all related copies this binding created, on unbind. +func (b *base) cleanupRelated(ctx context.Context, obj corev1alpha1.BindingAccessor, namespace string, providerClient client.Client) error { + spec := obj.BindingSpecP() + bindingUID := string(obj.GetUID()) + for i := range spec.RelatedResources { + rr := &spec.RelatedResources[i] + gvk, ok := relatedGVK(rr.Resource) + if !ok { + continue + } + target := providerClient // FromConsumer copies live on the provider + if rr.Direction == corev1alpha1.FromProvider { + target = b.client + } + if target == nil { + continue + } + if err := b.gcRelated(ctx, target, gvk, namespace, bindingUID, nil); err != nil { + return err + } + } + return nil +} + +func nameSet(sel *corev1alpha1.RelatedResourceSelector) map[string]bool { + if sel == nil || len(sel.Names) == 0 { + return nil + } + m := make(map[string]bool, len(sel.Names)) + for _, n := range sel.Names { + m[n] = true + } + return m +} + +// ensureNamespace creates the target namespace if absent (best-effort markers). +func ensureNamespace(ctx context.Context, c client.Client, name, localUID string) error { + ns := &unstructured.Unstructured{} + ns.SetGroupVersionKind(schema.GroupVersionKind{Version: "v1", Kind: "Namespace"}) + ns.SetName(name) + ns.SetLabels(map[string]string{corev1alpha1.LabelManaged: "true"}) + ns.SetAnnotations(map[string]string{corev1alpha1.AnnotationConsumerClusterUID: localUID}) + if err := c.Create(ctx, ns); err != nil && !apierrors.IsAlreadyExists(err) { + return err + } + return nil +} diff --git a/v2/konnector/engine/connection/reconciler.go b/v2/konnector/engine/connection/reconciler.go index a9aa47155..837393699 100644 --- a/v2/konnector/engine/connection/reconciler.go +++ b/v2/konnector/engine/connection/reconciler.go @@ -25,6 +25,7 @@ import ( "sort" "time" + coordinationv1 "k8s.io/api/coordination/v1" corev1 "k8s.io/api/core/v1" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -32,6 +33,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" + "k8s.io/utils/ptr" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" @@ -39,6 +41,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/reconcile" "github.com/kube-bind/kube-bind/v2/konnector/engine/crdpull" + "github.com/kube-bind/kube-bind/v2/konnector/engine/openapi" "github.com/kube-bind/kube-bind/v2/konnector/engine/remote" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) @@ -56,6 +59,21 @@ type Reconciler struct { // DiscoveryResync is how often a Ready Connection re-discovers exported APIs // (so a CRD labeled exported after connect is picked up). 0 = default 30s. DiscoveryResync time.Duration + // LeaseNamespace is the provider namespace where the konnector maintains its + // heartbeat Lease. 0 = default "kube-bind". + LeaseNamespace string +} + +const ( + leaseNamespaceDefault = "kube-bind" + leaseDurationSeconds = 60 +) + +func (r *Reconciler) leaseNamespace() string { + if r.LeaseNamespace != "" { + return r.LeaseNamespace + } + return leaseNamespaceDefault } func (r *Reconciler) discoveryResync() time.Duration { @@ -182,14 +200,15 @@ func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connectio } setCondition(conn, corev1alpha1.ConditionConnected, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "connected to provider") - // 3. Discover exported APIs (schema.source: CRD — label-gated). - exported, err := discoverExportedCRDs(ctx, providerClient) + // 3. Discover exported APIs and install schemas, per schema.source. + source, exported, err := r.discoverAndInstall(ctx, conn, providerClient) if err != nil { if apierrors.IsForbidden(err) { - return r.denyPermission(conn, "listing exported CRDs is forbidden by RBAC: "+err.Error()) + return r.denyPermission(conn, "discovery is forbidden by provider RBAC: "+err.Error()) } return fmt.Errorf("discovering exported APIs: %w", err) } + conn.Status.ActiveSchemaSource = source conn.Status.ExportedAPIs = exported setCondition(conn, corev1alpha1.ConditionPermissionDenied, metav1.ConditionFalse, corev1alpha1.ReasonAsExpected, "no RBAC denials") @@ -214,10 +233,58 @@ func (r *Reconciler) reconcile(ctx context.Context, conn *corev1alpha1.Connectio } } + // Heartbeat: maintain a Lease on the provider so a service-layer reaper can + // detect a consumer that stopped checking in. Best-effort — a missing Lease + // never blocks sync. + if err := r.heartbeat(ctx, providerClient, conn); err != nil { + ctrl.LoggerFrom(ctx).V(2).Info("heartbeat lease update failed (continuing)", "err", err.Error()) + } + setCondition(conn, corev1alpha1.ConditionReady, metav1.ConditionTrue, corev1alpha1.ReasonAsExpected, "connection ready") return nil } +// heartbeat creates or renews a coordination.k8s.io/Lease on the provider in the +// designated namespace, keyed by the consumer cluster identity. Zero kube-bind +// CRDs on the provider — just a plain Lease. +func (r *Reconciler) heartbeat(ctx context.Context, providerClient client.Client, conn *corev1alpha1.Connection) error { + if conn.Status.LocalClusterUID == "" { + return nil + } + ns := r.leaseNamespace() + if err := providerClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: ns}}); err != nil && + !apierrors.IsAlreadyExists(err) && !apierrors.IsForbidden(err) { + return fmt.Errorf("ensuring lease namespace: %w", err) + } + + now := metav1.NowMicro() + lease := &coordinationv1.Lease{ObjectMeta: metav1.ObjectMeta{Name: leaseName(conn), Namespace: ns}} + _, err := controllerutil.CreateOrUpdate(ctx, providerClient, lease, func() error { + if lease.Labels == nil { + lease.Labels = map[string]string{} + } + lease.Labels[corev1alpha1.LabelManaged] = "true" + if lease.Annotations == nil { + lease.Annotations = map[string]string{} + } + lease.Annotations[corev1alpha1.AnnotationConsumerClusterUID] = conn.Status.LocalClusterUID + lease.Spec.HolderIdentity = ptr.To(conn.Status.LocalClusterUID) + lease.Spec.LeaseDurationSeconds = ptr.To(int32(leaseDurationSeconds)) + if lease.Spec.AcquireTime == nil { + lease.Spec.AcquireTime = &now + } + lease.Spec.RenewTime = &now + return nil + }) + return err +} + +// leaseName keys the heartbeat Lease by the consumer cluster identity, so a +// provider can track which consumers are alive. +func leaseName(conn *corev1alpha1.Connection) string { + return "consumer-" + conn.Status.LocalClusterUID +} + // reconcileAutoBind keeps a managed ClusterBinding named after the Connection in // sync with the exported APIs. With no exports it removes the managed binding. func (r *Reconciler) reconcileAutoBind(ctx context.Context, conn *corev1alpha1.Connection, exported []corev1alpha1.ExportedAPI) error { @@ -375,6 +442,54 @@ func (r *Reconciler) bindingsReferencing(ctx context.Context, name string) (int, // discoverExportedCRDs lists provider CRDs carrying the exported label and maps // them to ExportedAPI entries. +// discoverAndInstall resolves the schema source and returns the active source + +// exported APIs. For CRD it lists label-gated provider CRDs (the binding pulls +// them later); for OpenAPI it synthesizes CRDs from discovery + /openapi/v3 and +// installs them on the consumer (CRD-less providers like kcp). Auto probes CRD +// first and falls back to OpenAPI. +func (r *Reconciler) discoverAndInstall(ctx context.Context, conn *corev1alpha1.Connection, providerClient client.Client) (corev1alpha1.SchemaSource, []corev1alpha1.ExportedAPI, error) { + src := conn.Spec.Schema.Source + if src == "" { + src = corev1alpha1.SchemaSourceAuto + } + + if src == corev1alpha1.SchemaSourceCRD || src == corev1alpha1.SchemaSourceAuto { + exported, err := discoverExportedCRDs(ctx, providerClient) + if err != nil { + return "", nil, err + } + if len(exported) > 0 || src == corev1alpha1.SchemaSourceCRD { + return corev1alpha1.SchemaSourceCRD, exported, nil + } + // Auto with no labeled CRDs → fall through to OpenAPI. + } + + // OpenAPI: synthesize CRDs from discovery + /openapi/v3 and install them. + cfg, err := remote.RestConfigFromConnection(ctx, r.Client, conn) + if err != nil { + return "", nil, err + } + crds, err := openapi.SynthesizeCRDs(ctx, cfg) + if err != nil { + return "", nil, err + } + exported := make([]corev1alpha1.ExportedAPI, 0, len(crds)) + for _, crd := range crds { + if _, err := crdpull.Install(ctx, r.Client, crd, conn.Name, conn.Spec.Schema.UpdatePolicy != corev1alpha1.UpdatePolicyOnce); err != nil { + return "", nil, fmt.Errorf("installing synthesized CRD %q: %w", crd.Name, err) + } + exported = append(exported, corev1alpha1.ExportedAPI{ + Name: crd.Name, + Group: crd.Spec.Group, + Resource: crd.Spec.Names.Plural, + Scope: crd.Spec.Scope, + Versions: []string{crd.Spec.Versions[0].Name}, + }) + } + sort.Slice(exported, func(i, j int) bool { return exported[i].Name < exported[j].Name }) + return corev1alpha1.SchemaSourceOpenAPI, exported, nil +} + func discoverExportedCRDs(ctx context.Context, providerClient client.Client) ([]corev1alpha1.ExportedAPI, error) { var crds apiextensionsv1.CustomResourceDefinitionList if err := providerClient.List(ctx, &crds, client.MatchingLabels{corev1alpha1.LabelExported: "true"}); err != nil { diff --git a/v2/konnector/engine/crdpull/crdpull.go b/v2/konnector/engine/crdpull/crdpull.go index d7ec17a12..95ab0b824 100644 --- a/v2/konnector/engine/crdpull/crdpull.go +++ b/v2/konnector/engine/crdpull/crdpull.go @@ -34,9 +34,9 @@ import ( corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) -// annotationSchemaHash records the hash of the installed schema so updatePolicy +// AnnotationSchemaHash records the hash of the installed schema so updatePolicy // can detect provider changes without re-deriving. -const annotationSchemaHash = "core.kube-bind.io/schema-hash" +const AnnotationSchemaHash = "core.kube-bind.io/schema-hash" // Options control how the CRD is reconciled on the consumer. type Options struct { @@ -57,11 +57,34 @@ func Pull(ctx context.Context, consumer client.Client, provider client.Reader, c return "", false, fmt.Errorf("reading provider CRD: %w", err) } desired := ForConsumer(&remoteCRD, connName) - hash = Hash(desired) - desired.Annotations[annotationSchemaHash] = hash + return install(ctx, consumer, desired, connName, opts) +} + +// Install installs an already-built (e.g. OpenAPI-synthesized) consumer CRD, +// stamping the managed/connection/schema-hash markers. updatePolicy: Always +// follows changes; Once pins. +func Install(ctx context.Context, consumer client.Client, crd *apiextensionsv1.CustomResourceDefinition, connName string, update bool) (hash string, err error) { + desired := crd.DeepCopy() + if desired.Labels == nil { + desired.Labels = map[string]string{} + } + desired.Labels[corev1alpha1.LabelManaged] = "true" + if desired.Annotations == nil { + desired.Annotations = map[string]string{} + } + desired.Annotations[corev1alpha1.AnnotationConnection] = connName + h, _, err := install(ctx, consumer, desired, connName, Options{Create: true, Update: update}) + return h, err +} + +// install reconciles a fully-built consumer CRD (create-if-absent, update on +// schema change per opts.Update, stamp-only when opts.Create is false). +func install(ctx context.Context, consumer client.Client, desired *apiextensionsv1.CustomResourceDefinition, connName string, opts Options) (string, bool, error) { + hash := Hash(desired) + desired.Annotations[AnnotationSchemaHash] = hash var existing apiextensionsv1.CustomResourceDefinition - getErr := consumer.Get(ctx, client.ObjectKey{Name: crdName}, &existing) + getErr := consumer.Get(ctx, client.ObjectKey{Name: desired.Name}, &existing) switch { case apierrors.IsNotFound(getErr): if !opts.Create { @@ -85,12 +108,12 @@ func Pull(ctx context.Context, consumer client.Client, provider client.Reader, c } return hash, true, nil } - if opts.Update && existing.Annotations[annotationSchemaHash] != hash { + if opts.Update && existing.Annotations[AnnotationSchemaHash] != hash { existing.Spec = desired.Spec if existing.Annotations == nil { existing.Annotations = map[string]string{} } - existing.Annotations[annotationSchemaHash] = hash + existing.Annotations[AnnotationSchemaHash] = hash existing.Annotations[corev1alpha1.AnnotationConnection] = connName if existing.Labels == nil { existing.Labels = map[string]string{} diff --git a/v2/konnector/engine/openapi/openapi.go b/v2/konnector/engine/openapi/openapi.go new file mode 100644 index 000000000..f90669cc3 --- /dev/null +++ b/v2/konnector/engine/openapi/openapi.go @@ -0,0 +1,223 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package openapi synthesizes CRDs for a CRD-less provider (kcp, aggregated +// APIs) from server discovery + the published /openapi/v3 schemas. This is the +// schema.source: OpenAPI path. Known fidelity limits (CEL, defaulting, +// multi-version conversion) are accepted — the provider remains the enforcing +// side. +package openapi + +import ( + "context" + "encoding/json" + "fmt" + "strings" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/discovery" + "k8s.io/client-go/rest" +) + +// builtinGroups are never exported via the OpenAPI boundary. +var builtinGroups = map[string]bool{ + "": true, // core + "apps": true, + "batch": true, + "extensions": true, + "policy": true, + "autoscaling": true, + "networking.k8s.io": true, +} + +// isBuiltinGroup reports whether a group is a Kubernetes built-in (excluded from +// the OpenAPI export boundary). +func isBuiltinGroup(group string) bool { + return builtinGroups[group] || strings.HasSuffix(group, ".k8s.io") || group == "apiregistration.k8s.io" +} + +// SynthesizeCRDs discovers every served, non-built-in API on the provider and +// synthesizes a consumer-installable CRD for each from its /openapi/v3 schema. +func SynthesizeCRDs(ctx context.Context, cfg *rest.Config) ([]*apiextensionsv1.CustomResourceDefinition, error) { + dc, err := discovery.NewDiscoveryClientForConfig(cfg) + if err != nil { + return nil, err + } + _, resourceLists, err := dc.ServerGroupsAndResources() + if err != nil { + // Partial discovery (an aggregated API may be down) is tolerable. + if len(resourceLists) == 0 { + return nil, fmt.Errorf("discovery: %w", err) + } + } + + root := dc.OpenAPIV3() + paths, err := root.Paths() + if err != nil { + return nil, fmt.Errorf("openapi v3 paths: %w", err) + } + + var out []*apiextensionsv1.CustomResourceDefinition + for _, rl := range resourceLists { + gv, err := schema.ParseGroupVersion(rl.GroupVersion) + if err != nil || isBuiltinGroup(gv.Group) { + continue + } + + // Which resources have a status subresource (discovery lists "/status"). + hasStatus := map[string]bool{} + for _, r := range rl.APIResources { + if base, sub, ok := strings.Cut(r.Name, "/"); ok && sub == "status" { + hasStatus[base] = true + } + } + + gvObj, ok := paths["apis/"+gv.Group+"/"+gv.Version] + if !ok { + continue + } + doc, err := gvObj.Schema("application/json") + if err != nil { + continue + } + + for _, r := range rl.APIResources { + if strings.Contains(r.Name, "/") { // skip subresources + continue + } + gvk := schema.GroupVersionKind{Group: gv.Group, Version: gv.Version, Kind: r.Kind} + props, err := schemaForGVK(doc, gvk) + if err != nil { + continue // no usable schema; skip this resource + } + out = append(out, buildCRD(gv, r, hasStatus[r.Name], props)) + } + } + return out, nil +} + +// schemaForGVK finds the component schema for gvk in an OpenAPI v3 document and +// converts it to a structural CRD schema (JSON round-trip + cleanup). +func schemaForGVK(doc []byte, gvk schema.GroupVersionKind) (*apiextensionsv1.JSONSchemaProps, error) { + var parsed struct { + Components struct { + Schemas map[string]json.RawMessage `json:"schemas"` + } `json:"components"` + } + if err := json.Unmarshal(doc, &parsed); err != nil { + return nil, err + } + for _, raw := range parsed.Components.Schemas { + var meta struct { + GVK []struct { + Group string `json:"group"` + Version string `json:"version"` + Kind string `json:"kind"` + } `json:"x-kubernetes-group-version-kind"` + } + if err := json.Unmarshal(raw, &meta); err != nil { + continue + } + for _, g := range meta.GVK { + if g.Group == gvk.Group && g.Version == gvk.Version && g.Kind == gvk.Kind { + var props apiextensionsv1.JSONSchemaProps + if err := json.Unmarshal(raw, &props); err != nil { + return nil, err + } + return cleanSchema(&props), nil + } + } + } + return nil, fmt.Errorf("no openapi schema for %s", gvk) +} + +// cleanSchema turns a top-level object schema into a structural CRD schema: drop +// the apiserver-managed apiVersion/kind/metadata (metadata is a $ref that won't +// resolve in a CRD), and guarantee an object type with unknown fields preserved +// where a sub-schema is otherwise empty. +func cleanSchema(p *apiextensionsv1.JSONSchemaProps) *apiextensionsv1.JSONSchemaProps { + p.Type = "object" + p.Ref = nil + if p.Properties != nil { + delete(p.Properties, "apiVersion") + delete(p.Properties, "kind") + delete(p.Properties, "metadata") + for k, v := range p.Properties { + p.Properties[k] = *normalize(&v) + } + } else { + preserve := true + p.XPreserveUnknownFields = &preserve + } + return p +} + +// normalize makes a sub-schema structural: drop $ref (unresolvable in a CRD) and +// preserve unknown fields where the type is unknown. +func normalize(p *apiextensionsv1.JSONSchemaProps) *apiextensionsv1.JSONSchemaProps { + if p.Ref != nil { + preserve := true + return &apiextensionsv1.JSONSchemaProps{Type: "object", XPreserveUnknownFields: &preserve} + } + if p.Type == "" && p.XPreserveUnknownFields == nil { + preserve := true + p.Type = "object" + p.XPreserveUnknownFields = &preserve + } + for k, v := range p.Properties { + p.Properties[k] = *normalize(&v) + } + if p.Items != nil && p.Items.Schema != nil { + p.Items.Schema = normalize(p.Items.Schema) + } + return p +} + +func buildCRD(gv schema.GroupVersion, r metav1.APIResource, hasStatus bool, props *apiextensionsv1.JSONSchemaProps) *apiextensionsv1.CustomResourceDefinition { + scope := apiextensionsv1.ClusterScoped + if r.Namespaced { + scope = apiextensionsv1.NamespaceScoped + } + singular := r.SingularName + if singular == "" { + singular = strings.ToLower(r.Kind) + } + version := apiextensionsv1.CustomResourceDefinitionVersion{ + Name: gv.Version, + Served: true, + Storage: true, + Schema: &apiextensionsv1.CustomResourceValidation{OpenAPIV3Schema: props}, + } + if hasStatus { + version.Subresources = &apiextensionsv1.CustomResourceSubresources{Status: &apiextensionsv1.CustomResourceSubresourceStatus{}} + } + return &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: r.Name + "." + gv.Group}, + Spec: apiextensionsv1.CustomResourceDefinitionSpec{ + Group: gv.Group, + Names: apiextensionsv1.CustomResourceDefinitionNames{ + Plural: r.Name, + Singular: singular, + Kind: r.Kind, + ListKind: r.Kind + "List", + }, + Scope: scope, + Versions: []apiextensionsv1.CustomResourceDefinitionVersion{version}, + }, + } +} diff --git a/v2/konnector/test/e2e/sync_test.go b/v2/konnector/test/e2e/sync_test.go index b8e114ee5..9bec21bd5 100644 --- a/v2/konnector/test/e2e/sync_test.go +++ b/v2/konnector/test/e2e/sync_test.go @@ -22,6 +22,7 @@ import ( "time" "github.com/stretchr/testify/require" + coordinationv1 "k8s.io/api/coordination/v1" corev1 "k8s.io/api/core/v1" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -96,6 +97,38 @@ func TestSlimCoreHappyCase(t *testing.T) { require.True(t, ok, "Connection must export %s", widgetCRDName) }, }, + { + name: "provider heartbeat Lease is maintained and renewed", + step: func(t *testing.T) { + leaseFor := func() (*coordinationv1.Lease, bool) { + conn := &corev1alpha1.Connection{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn); err != nil || conn.Status.LocalClusterUID == "" { + return nil, false + } + l := &coordinationv1.Lease{} + key := client.ObjectKey{Namespace: framework.KubeBindNamespace, Name: "consumer-" + conn.Status.LocalClusterUID} + if err := env.ProviderClient.Get(ctx, key, l); err != nil { + return nil, false + } + return l, l.Spec.HolderIdentity != nil && *l.Spec.HolderIdentity == conn.Status.LocalClusterUID + } + + var renew1 metav1.MicroTime + require.Eventually(t, func() bool { + l, ok := leaseFor() + if !ok || l.Labels[corev1alpha1.LabelManaged] != "true" || l.Spec.RenewTime == nil { + return false + } + renew1 = *l.Spec.RenewTime + return true + }, 30*time.Second, 200*time.Millisecond, "a heartbeat Lease should appear on the provider") + + require.Eventually(t, func() bool { + l, ok := leaseFor() + return ok && l.Spec.RenewTime != nil && l.Spec.RenewTime.After(renew1.Time) + }, 30*time.Second, 200*time.Millisecond, "the heartbeat Lease should be renewed") + }, + }, { name: "ClusterBinding becomes Ready and the CRD is pulled onto the consumer", step: func(t *testing.T) { diff --git a/v2/konnector/test/e2e/tier3_test.go b/v2/konnector/test/e2e/tier3_test.go new file mode 100644 index 000000000..bc065cf07 --- /dev/null +++ b/v2/konnector/test/e2e/tier3_test.go @@ -0,0 +1,166 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package e2e + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/kube-bind/kube-bind/v2/konnector/test/e2e/framework" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// TestSlimCoreRelatedResources binds the Widget API with a relatedResources rule +// that syncs label-selected Secrets FromProvider, and verifies sync + GC. +func TestSlimCoreRelatedResources(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + env.InstallExportedWidgetCRD(t) + + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + RelatedResources: []corev1alpha1.RelatedResource{{ + Group: "", + Resource: "secrets", + Direction: corev1alpha1.FromProvider, + Selector: &corev1alpha1.RelatedResourceSelector{ + LabelSelector: &metav1.LabelSelector{MatchLabels: map[string]string{"app": "widget"}}, + }, + }}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + secretKey := client.ObjectKey{Namespace: "default", Name: "widget-creds"} + + t.Run("a label-selected provider Secret syncs to the consumer", func(t *testing.T) { + require.NoError(t, env.ProviderClient.Create(ctx, &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "widget-creds", Labels: map[string]string{"app": "widget"}}, + StringData: map[string]string{"token": "s3cr3t"}, + })) + require.Eventually(t, func() bool { + s := &corev1.Secret{} + if err := env.ConsumerClient.Get(ctx, secretKey, s); err != nil { + return false + } + return string(s.Data["token"]) == "s3cr3t" && + s.Labels[corev1alpha1.LabelManaged] == "true" && + s.Annotations[corev1alpha1.AnnotationRelatedBinding] != "" + }, 30*time.Second, 200*time.Millisecond, "the label-selected provider Secret should sync to the consumer") + }) + + t.Run("the synced copy is GC'd when it stops matching", func(t *testing.T) { + s := &corev1.Secret{} + require.NoError(t, env.ProviderClient.Get(ctx, secretKey, s)) + delete(s.Labels, "app") + require.NoError(t, env.ProviderClient.Update(ctx, s)) + + require.Eventually(t, func() bool { + c := &corev1.Secret{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, secretKey, c)) + }, 30*time.Second, 200*time.Millisecond, "the consumer copy should be GC'd once the Secret stops matching the selector") + }) +} + +// TestSlimCoreOpenAPISource exercises schema.source: OpenAPI — the Connection +// synthesizes the consumer CRD from the provider's discovery + /openapi/v3 +// (no provider CRD read), then a binding syncs instances. +func TestSlimCoreOpenAPISource(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + gvr := env.InstallExportedWidgetCRD(t) // provider serves Widget; the label is ignored by OpenAPI + + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceOpenAPI}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + conn := &corev1alpha1.Connection{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn) + return conn.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + t.Run("Connection synthesizes and installs the CRD via OpenAPI", func(t *testing.T) { + conn := &corev1alpha1.Connection{} + require.NoError(t, env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn)) + require.Equal(t, corev1alpha1.SchemaSourceOpenAPI, conn.Status.ActiveSchemaSource) + _, ok := conn.Status.ExportsAPI(widgetCRDName) + require.True(t, ok, "OpenAPI discovery should export %s", widgetCRDName) + + crd := &apiextensionsv1.CustomResourceDefinition{} + require.Eventually(t, func() bool { + return env.ConsumerClient.Get(ctx, client.ObjectKey{Name: widgetCRDName}, crd) == nil + }, 30*time.Second, 200*time.Millisecond, "the synthesized CRD should be installed on the consumer") + require.Equal(t, "true", crd.Labels[corev1alpha1.LabelManaged]) + }) + + t.Run("a binding syncs an instance over the synthesized CRD", func(t *testing.T) { + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + consumerWidgets := env.ConsumerDyn.Resource(gvr).Namespace(instanceNS) + providerWidgets := env.ProviderDyn.Resource(gvr).Namespace(instanceNS) + require.Eventually(t, func() bool { + _, err := consumerWidgets.Create(ctx, widget("openapi-widget", "large"), metav1.CreateOptions{}) + return err == nil || apierrors.IsAlreadyExists(err) + }, 30*time.Second, 200*time.Millisecond, "the synthesized Widget CRD should become creatable") + require.Eventually(t, func() bool { + o, err := providerWidgets.Get(ctx, "openapi-widget", metav1.GetOptions{}) + if err != nil { + return false + } + size, _, _ := unstructured.NestedString(o.Object, "spec", "size") + return size == "large" + }, 30*time.Second, 200*time.Millisecond, "the instance should sync to the provider") + }) +} diff --git a/v2/sdk/apis/core/v1alpha1/connection_types.go b/v2/sdk/apis/core/v1alpha1/connection_types.go index 8004c001a..3e6964171 100644 --- a/v2/sdk/apis/core/v1alpha1/connection_types.go +++ b/v2/sdk/apis/core/v1alpha1/connection_types.go @@ -144,6 +144,13 @@ type ConnectionStatus struct { // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="localClusterUID is immutable" LocalClusterUID string `json:"localClusterUID,omitempty"` + // activeSchemaSource is the schema source actually in effect after resolving + // "Auto" (CRD or OpenAPI). The binding uses it to decide whether the + // Connection already installed the CRDs (OpenAPI) or it should pull them (CRD). + // + // +optional + ActiveSchemaSource SchemaSource `json:"activeSchemaSource,omitempty"` + // exportedAPIs is the discovery result: APIs exported to these credentials. // // +optional diff --git a/v2/sdk/apis/core/v1alpha1/helpers.go b/v2/sdk/apis/core/v1alpha1/helpers.go index 2a60b8495..37929ebbf 100644 --- a/v2/sdk/apis/core/v1alpha1/helpers.go +++ b/v2/sdk/apis/core/v1alpha1/helpers.go @@ -24,6 +24,8 @@ import ( // BindingAccessor is implemented by both ClusterBinding and Binding so a single // reconciler can drive either kind. The only difference between them is scope. // It is also a client.Object (metav1.Object + runtime.Object). +// +// +kubebuilder:object:generate=false type BindingAccessor interface { metav1.Object runtime.Object diff --git a/v2/sdk/apis/core/v1alpha1/labels.go b/v2/sdk/apis/core/v1alpha1/labels.go index 8cbe581c9..b67299af7 100644 --- a/v2/sdk/apis/core/v1alpha1/labels.go +++ b/v2/sdk/apis/core/v1alpha1/labels.go @@ -40,6 +40,11 @@ const ( // synced provider object (ownership marker). AnnotationConsumerObjectUID = "core.kube-bind.io/consumer-object-uid" + // AnnotationRelatedBinding records, on a synced related Secret/ConfigMap, the + // UID of the binding that pulled it in, so it can be GC'd when it stops + // matching the selector or the binding is removed. + AnnotationRelatedBinding = "core.kube-bind.io/related-binding" + // AnnotationConflict marks a consumer instance the konnector refused to sync // because the provider target is owned by another binding/consumer. The // value is the conflict reason. Bindings count annotated instances to report diff --git a/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go b/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go index e5d637040..86057e718 100644 --- a/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go +++ b/v2/sdk/apis/core/v1alpha1/zz_generated.deepcopy.go @@ -6,7 +6,7 @@ package v1alpha1 import ( "k8s.io/apimachinery/pkg/apis/meta/v1" - runtime "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime" ) // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. diff --git a/v2/sdk/config/crd/core.kube-bind.io_connections.yaml b/v2/sdk/config/crd/core.kube-bind.io_connections.yaml index 29daa8109..92437e2c4 100644 --- a/v2/sdk/config/crd/core.kube-bind.io_connections.yaml +++ b/v2/sdk/config/crd/core.kube-bind.io_connections.yaml @@ -129,6 +129,12 @@ spec: status: description: ConnectionStatus is the observed state of a Connection. properties: + activeSchemaSource: + description: |- + activeSchemaSource is the schema source actually in effect after resolving + "Auto" (CRD or OpenAPI). The binding uses it to decide whether the + Connection already installed the CRDs (OpenAPI) or it should pull them (CRD). + type: string conditions: description: 'conditions: SecretValid, Connected, SchemaInSync, Ready.' items: From b2567c7b83692785d7e5234c4cff58b863534eaf Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 13:49:45 +0300 Subject: [PATCH 12/18] add identity resolution --- v2/README.md | 9 +- v2/konnector/engine/remote/remote.go | 40 ++++-- v2/konnector/engine/remote/remote_test.go | 91 ++++++++++++ v2/konnector/test/e2e/framework/framework.go | 53 +++++++ .../e2e/{tier2_test.go => policies_test.go} | 6 +- .../test/e2e/related_resources_test.go | 98 +++++++++++++ .../{tier3_test.go => schema_source_test.go} | 129 +++++++++--------- v2/konnector/test/e2e/sync_test.go | 4 +- 8 files changed, 346 insertions(+), 84 deletions(-) create mode 100644 v2/konnector/engine/remote/remote_test.go rename v2/konnector/test/e2e/{tier2_test.go => policies_test.go} (97%) create mode 100644 v2/konnector/test/e2e/related_resources_test.go rename v2/konnector/test/e2e/{tier3_test.go => schema_source_test.go} (67%) diff --git a/v2/README.md b/v2/README.md index 9579d06cb..ff4618944 100644 --- a/v2/README.md +++ b/v2/README.md @@ -49,6 +49,11 @@ v2/ multi-version) are accepted; the provider stays the enforcing side. - The konnector maintains a `coordination.k8s.io/Lease` per Connection on the provider (heartbeat) — the hook a service-layer reaper keys off. +- **kcp-aware cluster identity**: the provider's stable identity is the kcp + `LogicalCluster` ("cluster") UID when present (a kcp workspace serves + `core.kcp.io`, and has no `kube-system`), falling back to the `kube-system` + namespace UID on plain Kubernetes. A provider RBAC denial on the identity read + surfaces as `PermissionDenied`. - `relatedResources` sync selected Secrets/ConfigMaps in the declared direction, scoped like the binding, GC'd when they stop matching or on unbind. - The provider side is the **multicluster-runtime engaged cluster** for each @@ -67,9 +72,7 @@ v2/ Known POC simplifications (tracked against the proposal): the `Mapper` extension is not implemented; OpenAPI synthesis is best-effort (fidelity limits -above); cluster identity still derives from the `kube-system` namespace UID, so -true kcp logical clusters (no kube-system) need an alternative identity source; -and syncer stop-on-disengage + productionization (RBAC/HA/Helm) remain. +above); and syncer stop-on-disengage + productionization (RBAC/HA/Helm) remain. ## Build diff --git a/v2/konnector/engine/remote/remote.go b/v2/konnector/engine/remote/remote.go index 16496062f..df79bc8ca 100644 --- a/v2/konnector/engine/remote/remote.go +++ b/v2/konnector/engine/remote/remote.go @@ -23,7 +23,10 @@ import ( "fmt" corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/rest" "k8s.io/client-go/tools/clientcmd" @@ -72,18 +75,37 @@ func RestConfigFromConnection(ctx context.Context, c client.Client, conn *corev1 return cfg, nil } -// ClusterUID returns a stable identity for the cluster behind c, derived from -// the kube-system namespace UID. This works on plain Kubernetes providers. +// logicalClusterGVK is kcp's per-workspace singleton identity object. +var logicalClusterGVK = schema.GroupVersionKind{Group: "core.kcp.io", Version: "v1alpha1", Kind: "LogicalCluster"} + +// ClusterUID returns a stable identity for the cluster behind c. It tries, in +// order: +// - the kcp LogicalCluster "cluster" object's UID (kcp / logical clusters, +// which have no kube-system namespace) — preferred when present, since only +// a kcp-shaped cluster serves core.kcp.io; then +// - the kube-system namespace UID (plain Kubernetes). // -// TODO(v2): kcp / logical-cluster providers have no kube-system namespace; the -// OpenAPI path will need a different identity source (proposal gap #7). +// A Forbidden error from a source is surfaced (so the caller can report +// PermissionDenied) rather than silently falling through. func ClusterUID(ctx context.Context, c client.Client) (string, error) { + lc := &unstructured.Unstructured{} + lc.SetGroupVersionKind(logicalClusterGVK) + lcErr := c.Get(ctx, types.NamespacedName{Name: "cluster"}, lc) + if lcErr == nil && lc.GetUID() != "" { + return string(lc.GetUID()), nil + } + if apierrors.IsForbidden(lcErr) { + return "", lcErr + } + var ns corev1.Namespace - if err := c.Get(ctx, types.NamespacedName{Name: "kube-system"}, &ns); err != nil { - return "", fmt.Errorf("getting kube-system namespace for cluster identity: %w", err) + nsErr := c.Get(ctx, types.NamespacedName{Name: "kube-system"}, &ns) + if nsErr == nil && ns.UID != "" { + return string(ns.UID), nil } - if ns.UID == "" { - return "", fmt.Errorf("kube-system namespace UID is empty") + if apierrors.IsForbidden(nsErr) { + return "", nsErr } - return string(ns.UID), nil + + return "", fmt.Errorf("cannot determine cluster identity (no LogicalCluster: %v; no kube-system namespace: %v)", lcErr, nsErr) } diff --git a/v2/konnector/engine/remote/remote_test.go b/v2/konnector/engine/remote/remote_test.go new file mode 100644 index 000000000..70083b0a4 --- /dev/null +++ b/v2/konnector/engine/remote/remote_test.go @@ -0,0 +1,91 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package remote + +import ( + "context" + "testing" + + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + clientgoscheme "k8s.io/client-go/kubernetes/scheme" + "sigs.k8s.io/controller-runtime/pkg/client/fake" +) + +func identityScheme(t *testing.T) *runtime.Scheme { + t.Helper() + s := runtime.NewScheme() + require.NoError(t, clientgoscheme.AddToScheme(s)) + // Register the kcp LogicalCluster GVK as unstructured so the fake client can + // serve it. + s.AddKnownTypeWithName(logicalClusterGVK, &unstructured.Unstructured{}) + s.AddKnownTypeWithName(logicalClusterGVK.GroupVersion().WithKind("LogicalClusterList"), &unstructured.UnstructuredList{}) + return s +} + +func logicalCluster(uid string) *unstructured.Unstructured { + lc := &unstructured.Unstructured{} + lc.SetGroupVersionKind(logicalClusterGVK) + lc.SetName("cluster") + lc.SetUID(types.UID(uid)) + return lc +} + +func TestClusterUID_KubeSystem(t *testing.T) { + s := identityScheme(t) + c := fake.NewClientBuilder().WithScheme(s).WithObjects( + &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: "kube-system", UID: "ks-uid"}}, + ).Build() + + uid, err := ClusterUID(context.Background(), c) + require.NoError(t, err) + require.Equal(t, "ks-uid", uid, "plain Kubernetes should use the kube-system namespace UID") +} + +func TestClusterUID_KCPLogicalClusterFallback(t *testing.T) { + s := identityScheme(t) + // No kube-system namespace (a kcp logical cluster) — only a LogicalCluster. + c := fake.NewClientBuilder().WithScheme(s).WithObjects(logicalCluster("lc-uid")).Build() + + uid, err := ClusterUID(context.Background(), c) + require.NoError(t, err) + require.Equal(t, "lc-uid", uid, "a kcp logical cluster should fall back to the LogicalCluster UID") +} + +func TestClusterUID_LogicalClusterWinsWhenBothPresent(t *testing.T) { + s := identityScheme(t) + c := fake.NewClientBuilder().WithScheme(s).WithObjects( + &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: "kube-system", UID: "ks-uid"}}, + logicalCluster("lc-uid"), + ).Build() + + uid, err := ClusterUID(context.Background(), c) + require.NoError(t, err) + require.Equal(t, "lc-uid", uid, "a kcp-shaped cluster (LogicalCluster present) should prefer the LogicalCluster UID") +} + +func TestClusterUID_NoSource(t *testing.T) { + s := identityScheme(t) + c := fake.NewClientBuilder().WithScheme(s).Build() + + _, err := ClusterUID(context.Background(), c) + require.Error(t, err, "with neither source the identity is undeterminable") +} diff --git a/v2/konnector/test/e2e/framework/framework.go b/v2/konnector/test/e2e/framework/framework.go index 456df2202..8e927afc3 100644 --- a/v2/konnector/test/e2e/framework/framework.go +++ b/v2/konnector/test/e2e/framework/framework.go @@ -36,6 +36,7 @@ import ( apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" apimachineryruntime "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" utilruntime "k8s.io/apimachinery/pkg/util/runtime" @@ -228,6 +229,58 @@ func (e *Env) CopyProviderSecret(t *testing.T, name string) *corev1.Secret { } } +// MakeProviderKCPLike turns the provider into a kcp-shaped cluster: it installs +// a LogicalCluster CRD + the singleton "cluster" object (the kcp per-workspace +// identity object). On a real kcp workspace there is no kube-system namespace, +// so identity comes from the LogicalCluster; the konnector prefers it whenever +// it is present (which only a kcp-shaped cluster serves), so we don't need to +// remove kube-system here (envtest has no namespace controller to finalize it +// away anyway). Returns the LogicalCluster's UID. +func (e *Env) MakeProviderKCPLike(t *testing.T, ctx context.Context) string { + t.Helper() + + require.NoError(t, e.ProviderClient.Create(ctx, logicalClusterCRD())) + lcGVR := schema.GroupVersionResource{Group: "core.kcp.io", Version: "v1alpha1", Resource: "logicalclusters"} + require.Eventually(t, func() bool { + _, err := e.ProviderDyn.Resource(lcGVR).List(ctx, metav1.ListOptions{}) + return err == nil + }, 30*time.Second, 200*time.Millisecond, "provider should serve LogicalCluster") + + lc := &unstructured.Unstructured{} + lc.SetGroupVersionKind(schema.GroupVersionKind{Group: "core.kcp.io", Version: "v1alpha1", Kind: "LogicalCluster"}) + lc.SetName("cluster") + require.NoError(t, e.ProviderClient.Create(ctx, lc)) + require.NoError(t, e.ProviderClient.Get(ctx, client.ObjectKey{Name: "cluster"}, lc)) + uid := string(lc.GetUID()) + require.NotEmpty(t, uid) + return uid +} + +func logicalClusterCRD() *apiextensionsv1.CustomResourceDefinition { + preserve := true + return &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: "logicalclusters.core.kcp.io"}, + Spec: apiextensionsv1.CustomResourceDefinitionSpec{ + Group: "core.kcp.io", + Names: apiextensionsv1.CustomResourceDefinitionNames{ + Plural: "logicalclusters", Singular: "logicalcluster", Kind: "LogicalCluster", ListKind: "LogicalClusterList", + }, + Scope: apiextensionsv1.ClusterScoped, + Versions: []apiextensionsv1.CustomResourceDefinitionVersion{{ + Name: "v1alpha1", Served: true, Storage: true, + Schema: &apiextensionsv1.CustomResourceValidation{ + OpenAPIV3Schema: &apiextensionsv1.JSONSchemaProps{ + Type: "object", + Properties: map[string]apiextensionsv1.JSONSchemaProps{ + "spec": {Type: "object", XPreserveUnknownFields: &preserve}, + }, + }, + }, + }}, + }, + } +} + // InstallExportedWidgetCRD installs the demo Widget CRD on the provider, labeled // as exported. func (e *Env) InstallExportedWidgetCRD(t *testing.T) schema.GroupVersionResource { diff --git a/v2/konnector/test/e2e/tier2_test.go b/v2/konnector/test/e2e/policies_test.go similarity index 97% rename from v2/konnector/test/e2e/tier2_test.go rename to v2/konnector/test/e2e/policies_test.go index dcc690b3f..837b16d54 100644 --- a/v2/konnector/test/e2e/tier2_test.go +++ b/v2/konnector/test/e2e/policies_test.go @@ -33,12 +33,12 @@ import ( corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) -// TestSlimCoreTier2 exercises the Tier-2 "Decided" features. It shares one -// envtest pair (spinning a fresh one per case exhausts envtest resources): +// TestSlimCorePolicies exercises the binding/connection policy knobs. It shares +// one envtest pair (spinning a fresh one per case exhausts envtest resources): // deletion-policy Orphan, updatePolicy Always, autoBind, pullPolicy All, and // PermissionDenied. The widgets-dependent cases run first, before the extra // Connections (autoBind/All) re-stamp the widgets CRD. -func TestSlimCoreTier2(t *testing.T) { +func TestSlimCorePolicies(t *testing.T) { env := framework.Start(t) ctx := context.Background() gvr := env.InstallExportedWidgetCRD(t) diff --git a/v2/konnector/test/e2e/related_resources_test.go b/v2/konnector/test/e2e/related_resources_test.go new file mode 100644 index 000000000..163d638d0 --- /dev/null +++ b/v2/konnector/test/e2e/related_resources_test.go @@ -0,0 +1,98 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package e2e + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/kube-bind/kube-bind/v2/konnector/test/e2e/framework" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// TestSlimCoreRelatedResources binds the Widget API with a relatedResources rule +// that syncs label-selected Secrets FromProvider, and verifies sync + GC. +func TestSlimCoreRelatedResources(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + env.InstallExportedWidgetCRD(t) + + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + RelatedResources: []corev1alpha1.RelatedResource{{ + Group: "", + Resource: "secrets", + Direction: corev1alpha1.FromProvider, + Selector: &corev1alpha1.RelatedResourceSelector{ + LabelSelector: &metav1.LabelSelector{MatchLabels: map[string]string{"app": "widget"}}, + }, + }}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + secretKey := client.ObjectKey{Namespace: "default", Name: "widget-creds"} + + t.Run("a label-selected provider Secret syncs to the consumer", func(t *testing.T) { + require.NoError(t, env.ProviderClient.Create(ctx, &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "widget-creds", Labels: map[string]string{"app": "widget"}}, + StringData: map[string]string{"token": "s3cr3t"}, + })) + require.Eventually(t, func() bool { + s := &corev1.Secret{} + if err := env.ConsumerClient.Get(ctx, secretKey, s); err != nil { + return false + } + return string(s.Data["token"]) == "s3cr3t" && + s.Labels[corev1alpha1.LabelManaged] == "true" && + s.Annotations[corev1alpha1.AnnotationRelatedBinding] != "" + }, 30*time.Second, 200*time.Millisecond, "the label-selected provider Secret should sync to the consumer") + }) + + t.Run("the synced copy is GC'd when it stops matching", func(t *testing.T) { + s := &corev1.Secret{} + require.NoError(t, env.ProviderClient.Get(ctx, secretKey, s)) + delete(s.Labels, "app") + require.NoError(t, env.ProviderClient.Update(ctx, s)) + + require.Eventually(t, func() bool { + c := &corev1.Secret{} + return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, secretKey, c)) + }, 30*time.Second, 200*time.Millisecond, "the consumer copy should be GC'd once the Secret stops matching the selector") + }) +} diff --git a/v2/konnector/test/e2e/tier3_test.go b/v2/konnector/test/e2e/schema_source_test.go similarity index 67% rename from v2/konnector/test/e2e/tier3_test.go rename to v2/konnector/test/e2e/schema_source_test.go index bc065cf07..1884af172 100644 --- a/v2/konnector/test/e2e/tier3_test.go +++ b/v2/konnector/test/e2e/schema_source_test.go @@ -22,7 +22,6 @@ import ( "time" "github.com/stretchr/testify/require" - corev1 "k8s.io/api/core/v1" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -33,72 +32,6 @@ import ( corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) -// TestSlimCoreRelatedResources binds the Widget API with a relatedResources rule -// that syncs label-selected Secrets FromProvider, and verifies sync + GC. -func TestSlimCoreRelatedResources(t *testing.T) { - env := framework.Start(t) - ctx := context.Background() - env.InstallExportedWidgetCRD(t) - - require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ - ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, - Spec: corev1alpha1.ConnectionSpec{ - KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, - Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, - }, - })) - require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, - Spec: corev1alpha1.BindingSpec{ - ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, - APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, - RelatedResources: []corev1alpha1.RelatedResource{{ - Group: "", - Resource: "secrets", - Direction: corev1alpha1.FromProvider, - Selector: &corev1alpha1.RelatedResourceSelector{ - LabelSelector: &metav1.LabelSelector{MatchLabels: map[string]string{"app": "widget"}}, - }, - }}, - }, - })) - framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { - cb := &corev1alpha1.ClusterBinding{} - err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) - return cb.Status.Conditions, err - }, corev1alpha1.ConditionReady) - - secretKey := client.ObjectKey{Namespace: "default", Name: "widget-creds"} - - t.Run("a label-selected provider Secret syncs to the consumer", func(t *testing.T) { - require.NoError(t, env.ProviderClient.Create(ctx, &corev1.Secret{ - ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "widget-creds", Labels: map[string]string{"app": "widget"}}, - StringData: map[string]string{"token": "s3cr3t"}, - })) - require.Eventually(t, func() bool { - s := &corev1.Secret{} - if err := env.ConsumerClient.Get(ctx, secretKey, s); err != nil { - return false - } - return string(s.Data["token"]) == "s3cr3t" && - s.Labels[corev1alpha1.LabelManaged] == "true" && - s.Annotations[corev1alpha1.AnnotationRelatedBinding] != "" - }, 30*time.Second, 200*time.Millisecond, "the label-selected provider Secret should sync to the consumer") - }) - - t.Run("the synced copy is GC'd when it stops matching", func(t *testing.T) { - s := &corev1.Secret{} - require.NoError(t, env.ProviderClient.Get(ctx, secretKey, s)) - delete(s.Labels, "app") - require.NoError(t, env.ProviderClient.Update(ctx, s)) - - require.Eventually(t, func() bool { - c := &corev1.Secret{} - return apierrors.IsNotFound(env.ConsumerClient.Get(ctx, secretKey, c)) - }, 30*time.Second, 200*time.Millisecond, "the consumer copy should be GC'd once the Secret stops matching the selector") - }) -} - // TestSlimCoreOpenAPISource exercises schema.source: OpenAPI — the Connection // synthesizes the consumer CRD from the provider's discovery + /openapi/v3 // (no provider CRD read), then a binding syncs instances. @@ -164,3 +97,65 @@ func TestSlimCoreOpenAPISource(t *testing.T) { }, 30*time.Second, 200*time.Millisecond, "the instance should sync to the provider") }) } + +// TestSlimCoreKCPLikeProvider runs the full OpenAPI flow against a kcp-shaped +// provider (a LogicalCluster object is present, as on a kcp workspace; identity +// is taken from it rather than kube-system) and verifies the pinned remote +// identity + sync. +func TestSlimCoreKCPLikeProvider(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + lcUID := env.MakeProviderKCPLike(t, ctx) + gvr := env.InstallExportedWidgetCRD(t) + + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "kcp-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceOpenAPI}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + conn := &corev1alpha1.Connection{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "kcp-provider"}, conn) + return conn.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + t.Run("identity is pinned from the LogicalCluster", func(t *testing.T) { + conn := &corev1alpha1.Connection{} + require.NoError(t, env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "kcp-provider"}, conn)) + require.Equal(t, lcUID, conn.Status.RemoteClusterUID, "remote identity must come from the LogicalCluster on a kcp-like provider") + _, ok := conn.Status.ExportsAPI(widgetCRDName) + require.True(t, ok) + }) + + t.Run("instances sync against the kcp-like provider", func(t *testing.T) { + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "kcp-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + }, + })) + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) + + consumerWidgets := env.ConsumerDyn.Resource(gvr).Namespace(instanceNS) + providerWidgets := env.ProviderDyn.Resource(gvr).Namespace(instanceNS) + require.Eventually(t, func() bool { + _, err := consumerWidgets.Create(ctx, widget("kcp-widget", "large"), metav1.CreateOptions{}) + return err == nil || apierrors.IsAlreadyExists(err) + }, 30*time.Second, 200*time.Millisecond, "the synthesized Widget CRD should become creatable") + require.Eventually(t, func() bool { + o, err := providerWidgets.Get(ctx, "kcp-widget", metav1.GetOptions{}) + if err != nil { + return false + } + // The provider copy carries our consumer markers. + return o.GetAnnotations()[corev1alpha1.AnnotationConsumerObjectUID] != "" + }, 30*time.Second, 200*time.Millisecond, "the instance should sync to the kcp-like provider") + }) +} diff --git a/v2/konnector/test/e2e/sync_test.go b/v2/konnector/test/e2e/sync_test.go index 9bec21bd5..4f521ce27 100644 --- a/v2/konnector/test/e2e/sync_test.go +++ b/v2/konnector/test/e2e/sync_test.go @@ -296,7 +296,7 @@ func TestSlimCoreHappyCase(t *testing.T) { }, }, { - // Tier 1: a Connection already Ready picks up a CRD exported later. + // Re-discovery: a Connection already Ready picks up a CRD exported later. name: "re-discovery: a CRD exported after connect is picked up", step: func(t *testing.T) { env.InstallExportedCRD(t, "gadget.io", "gadgets", "gadget", "Gadget") @@ -311,7 +311,7 @@ func TestSlimCoreHappyCase(t *testing.T) { }, }, { - // Tier 1: conflictPolicy Adopt takes over an un-owned provider object. + // conflictPolicy Adopt takes over an un-owned provider object. name: "conflictPolicy Adopt takes over an un-owned provider object", step: func(t *testing.T) { gadgetGVR := schema.GroupVersionResource{Group: "gadget.io", Version: "v1", Resource: "gadgets"} From 62e40749c384830b427d56c82429302e2c498cf2 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 14:19:14 +0300 Subject: [PATCH 13/18] add mapper & identity --- v2/README.md | 19 ++- v2/konnector/engine/mapper/mapper.go | 67 ++++++++ v2/konnector/engine/mapper/mapper_test.go | 70 ++++++++ .../engine/provider/connection_provider.go | 28 +++- v2/konnector/engine/sync/crd_controller.go | 152 +++++++++++++----- v2/konnector/engine/sync/syncer.go | 56 +++++-- v2/konnector/test/e2e/disengage_test.go | 119 ++++++++++++++ 7 files changed, 448 insertions(+), 63 deletions(-) create mode 100644 v2/konnector/engine/mapper/mapper.go create mode 100644 v2/konnector/engine/mapper/mapper_test.go create mode 100644 v2/konnector/test/e2e/disengage_test.go diff --git a/v2/README.md b/v2/README.md index ff4618944..a91976a02 100644 --- a/v2/README.md +++ b/v2/README.md @@ -60,6 +60,18 @@ v2/ Connection: writes go through its client, fresh reads through its API reader, and status/drift events arrive via a **watch on its cache** (event-driven, not polled — a low-frequency resync is only a backstop). +- **Stop-on-disengage**: a Connection that loses readiness (revoked credential, + unreachable provider, withdrawn RBAC) is disengaged, and its per-GVR syncers + are torn down rather than left running against a dead cluster. When it becomes + Ready again the provider re-engages as a fresh cluster and the syncers are + rebuilt against it (a stale syncer would otherwise hold a dead client forever). +- **Mapper extension point** (`engine/mapper`): the syncer routes every + provider-side operation through a `Mapper` that translates the consumer object + key to its provider key. Core ships only `Identity` (ns/name unchanged); an + out-of-tree build supplies its own via `sync.WithMapper(...)` to restore v1's + "Prefixed" key isolation without forking the engine. The interface maps keys + only — it cannot change scope (cluster-scoped stays cluster-scoped), and it is + deliberately kept out of the CRD API so the core API never promises renaming. - **Order-independent apply**: a `Connection` created before its Secret resolves when the Secret arrives (the konnector watches referenced Secrets); a binding created before its Connection resolves when the Connection goes Ready. @@ -70,9 +82,10 @@ v2/ gone, and keeps its Secret alive (via a finalizer) so teardown can still reach the provider — so `kubectl delete -f bundle.yaml` is order-don't-care. -Known POC simplifications (tracked against the proposal): the `Mapper` -extension is not implemented; OpenAPI synthesis is best-effort (fidelity limits -above); and syncer stop-on-disengage + productionization (RBAC/HA/Helm) remain. +Known POC simplifications (tracked against the proposal): OpenAPI synthesis is +best-effort (fidelity limits above); the `Mapper` seam exists but only `Identity` +ships and `relatedResources` are not yet routed through it; and productionization +(RBAC/HA/Helm) remains. ## Build diff --git a/v2/konnector/engine/mapper/mapper.go b/v2/konnector/engine/mapper/mapper.go new file mode 100644 index 000000000..4f31954b6 --- /dev/null +++ b/v2/konnector/engine/mapper/mapper.go @@ -0,0 +1,67 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package mapper defines the Mapper extension point: how the konnector +// translates object identity (namespace/name) between the consumer and the +// provider when syncing instances. +// +// Core ships exactly one implementation — Identity — and the interface is a +// compile-time seam so out-of-tree konnector builds can restore tenancy +// key-mapping (e.g. v1's "Prefixed" isolation) without forking the engine. It +// is deliberately kept out of the CRD API: the core API never promises +// renaming. +// +// The interface maps KEYS only; it cannot change an object's scope +// (cluster-scoped stays cluster-scoped). That is the hard line v2 draws — v1's +// "Namespaced" scope-conversion isolation is intentionally not expressible here. +// A later, separate Transformer interface (mutating the object payload — label +// injection, field stripping) is possible but deliberately deferred. +package mapper + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// ObjectKey is a namespace/name identity. It aliases controller-runtime's +// client.ObjectKey for drop-in interop with the syncer. +type ObjectKey = client.ObjectKey + +// Mapper translates object identity between consumer and provider. Core +// registers exactly one implementation: Identity. +type Mapper interface { + // ToProvider maps a consumer object key to its provider key. + ToProvider(gvr schema.GroupVersionResource, key ObjectKey) (ObjectKey, error) + // ToConsumer is the inverse of ToProvider; it must round-trip, i.e. + // ToConsumer(gvr, ToProvider(gvr, k)) == k. + ToConsumer(gvr schema.GroupVersionResource, key ObjectKey) (ObjectKey, error) +} + +// Identity is the core Mapper: the consumer ns/name equals the provider ns/name +// with no transformation. It is the only mapping core ships. +type Identity struct{} + +// ToProvider returns key unchanged. +func (Identity) ToProvider(_ schema.GroupVersionResource, key ObjectKey) (ObjectKey, error) { + return key, nil +} + +// ToConsumer returns key unchanged. +func (Identity) ToConsumer(_ schema.GroupVersionResource, key ObjectKey) (ObjectKey, error) { + return key, nil +} + +var _ Mapper = Identity{} diff --git a/v2/konnector/engine/mapper/mapper_test.go b/v2/konnector/engine/mapper/mapper_test.go new file mode 100644 index 000000000..389f55719 --- /dev/null +++ b/v2/konnector/engine/mapper/mapper_test.go @@ -0,0 +1,70 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package mapper_test + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/runtime/schema" + + "github.com/kube-bind/kube-bind/v2/konnector/engine/mapper" +) + +var widgetGVR = schema.GroupVersionResource{Group: "example.org", Version: "v1", Resource: "widgets"} + +func TestIdentity_RoundTrips(t *testing.T) { + m := mapper.Identity{} + in := mapper.ObjectKey{Namespace: "team-a", Name: "w1"} + + prov, err := m.ToProvider(widgetGVR, in) + require.NoError(t, err) + require.Equal(t, in, prov, "Identity must not change the key on the way to the provider") + + back, err := m.ToConsumer(widgetGVR, prov) + require.NoError(t, err) + require.Equal(t, in, back, "Identity must round-trip") +} + +// prefixMapper is an out-of-tree-style Mapper that prefixes the provider +// namespace, exercising the interface the way a custom build would (v1's +// "Prefixed" isolation). It lives in the test to prove the seam is usable and +// that the round-trip contract is satisfiable by a non-identity mapping. +type prefixMapper struct{ prefix string } + +func (p prefixMapper) ToProvider(_ schema.GroupVersionResource, key mapper.ObjectKey) (mapper.ObjectKey, error) { + return mapper.ObjectKey{Namespace: p.prefix + key.Namespace, Name: key.Name}, nil +} + +func (p prefixMapper) ToConsumer(_ schema.GroupVersionResource, key mapper.ObjectKey) (mapper.ObjectKey, error) { + return mapper.ObjectKey{Namespace: strings.TrimPrefix(key.Namespace, p.prefix), Name: key.Name}, nil +} + +func TestMapper_NonIdentityRoundTrips(t *testing.T) { + var m mapper.Mapper = prefixMapper{prefix: "consumer-7-"} + in := mapper.ObjectKey{Namespace: "team-a", Name: "w1"} + + prov, err := m.ToProvider(widgetGVR, in) + require.NoError(t, err) + require.Equal(t, "consumer-7-team-a", prov.Namespace) + require.Equal(t, "w1", prov.Name) + + back, err := m.ToConsumer(widgetGVR, prov) + require.NoError(t, err) + require.Equal(t, in, back, "a Mapper must round-trip: ToConsumer(ToProvider(k)) == k") +} diff --git a/v2/konnector/engine/provider/connection_provider.go b/v2/konnector/engine/provider/connection_provider.go index 6f22cbea5..6ded9f883 100644 --- a/v2/konnector/engine/provider/connection_provider.go +++ b/v2/konnector/engine/provider/connection_provider.go @@ -118,16 +118,34 @@ func (p *ConnectionProvider) Reconcile(ctx context.Context, req reconcile.Reques } p.lock.Lock() - defer p.lock.Unlock() + mcMgr := p.mcMgr + _, engaged := p.clusters[key] + p.lock.Unlock() - if p.mcMgr == nil { + if mcMgr == nil { return reconcile.Result{RequeueAfter: 2 * time.Second}, nil } - if _, ok := p.clusters[key]; ok { + // A Connection that is no longer Ready (e.g. its credential was revoked, the + // provider became unreachable, or RBAC was withdrawn) must be disengaged, not + // just left running against a dead cluster. This also covers a Connection + // mid-deletion that has already flipped not-Ready. + if !isReady(conn) { + if engaged { + p.disengage(key) + log.Info("Disengaged provider cluster (Connection no longer ready)") + } else { + log.V(4).Info("Connection not ready yet, not engaging") + } return reconcile.Result{}, nil } - if !isReady(conn) { - log.V(4).Info("Connection not ready yet, not engaging") + if engaged { + return reconcile.Result{}, nil + } + + p.lock.Lock() + defer p.lock.Unlock() + // Re-check under the lock in case a concurrent reconcile engaged it. + if _, ok := p.clusters[key]; ok { return reconcile.Result{}, nil } diff --git a/v2/konnector/engine/sync/crd_controller.go b/v2/konnector/engine/sync/crd_controller.go index d81b1c7d4..c3bfaaf33 100644 --- a/v2/konnector/engine/sync/crd_controller.go +++ b/v2/konnector/engine/sync/crd_controller.go @@ -28,6 +28,7 @@ import ( "github.com/go-logr/logr" apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" "k8s.io/apimachinery/pkg/runtime/schema" @@ -42,6 +43,7 @@ import ( "k8s.io/client-go/util/workqueue" "k8s.io/utils/ptr" ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/builder" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/cluster" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -50,6 +52,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/reconcile" "sigs.k8s.io/controller-runtime/pkg/source" + "github.com/kube-bind/kube-bind/v2/konnector/engine/mapper" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) @@ -70,21 +73,37 @@ type CRDController struct { clusters ClusterGetter recorder record.EventRecorder resync time.Duration + // mapper translates consumer<->provider object keys. Defaults to + // mapper.Identity; an out-of-tree build swaps it via WithMapper. + mapper mapper.Mapper lock sync.Mutex contexts map[string]*syncContext // by CRD name } +// Option configures the CRDController at setup. It is the compile-time seam for +// out-of-tree konnector builds (e.g. a custom Mapper). +type Option func(*CRDController) + +// WithMapper overrides the consumer<->provider key Mapper (default: Identity). +func WithMapper(m mapper.Mapper) Option { + return func(c *CRDController) { c.mapper = m } +} + type syncContext struct { generation int64 - cancel context.CancelFunc - wg *sync.WaitGroup + // cluster is the engaged provider cluster this syncer was built against. A + // different instance means the Connection re-engaged and the syncer must be + // rebuilt (the old one holds a dead client/cache). + cluster cluster.Cluster + cancel context.CancelFunc + wg *sync.WaitGroup } // SetupWithManager registers the CRD controller. The consumer side uses a // direct (uncached) client built from the manager's rest config; the provider // side comes from the multicluster-runtime engaged cluster resolved via clusters. -func SetupWithManager(mgr ctrl.Manager, clusters ClusterGetter) error { +func SetupWithManager(mgr ctrl.Manager, clusters ClusterGetter, opts ...Option) error { consumerClient, err := client.New(mgr.GetConfig(), client.Options{Scheme: mgr.GetScheme()}) if err != nil { return fmt.Errorf("building direct consumer client: %w", err) @@ -96,17 +115,42 @@ func SetupWithManager(mgr ctrl.Manager, clusters ClusterGetter) error { clusters: clusters, recorder: mgr.GetEventRecorderFor("kube-bind-konnector"), resync: time.Minute, + mapper: mapper.Identity{}, contexts: map[string]*syncContext{}, } + for _, o := range opts { + o(r) + } return ctrl.NewControllerManagedBy(mgr). - For(&apiextensionsv1.CustomResourceDefinition{}). - WithEventFilter(predicate.NewPredicateFuncs(func(o client.Object) bool { - return o.GetLabels()[corev1alpha1.LabelManaged] == "true" - })). + For(&apiextensionsv1.CustomResourceDefinition{}, builder.WithPredicates(predicate.NewPredicateFuncs(managedCRD))). + // Watch Connections so an engage/disengage (Ready flip) re-evaluates the + // syncers of every CRD that Connection pulled — stopping them on disengage + // and rebuilding them against the freshly-engaged cluster on re-engage. + Watches(&corev1alpha1.Connection{}, handler.EnqueueRequestsFromMapFunc(r.crdsForConnection)). Named("managed-crds"). Complete(r) } +func managedCRD(o client.Object) bool { + return o.GetLabels()[corev1alpha1.LabelManaged] == "true" +} + +// crdsForConnection enqueues every managed CRD pulled through the given +// Connection, so its readiness transitions re-evaluate the per-GVR syncers. +func (r *CRDController) crdsForConnection(ctx context.Context, conn client.Object) []reconcile.Request { + var crds apiextensionsv1.CustomResourceDefinitionList + if err := r.consumerClient.List(ctx, &crds, client.MatchingLabels{corev1alpha1.LabelManaged: "true"}); err != nil { + return nil + } + var reqs []reconcile.Request + for i := range crds.Items { + if crds.Items[i].Annotations[corev1alpha1.AnnotationConnection] == conn.GetName() { + reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{Name: crds.Items[i].Name}}) + } + } + return reqs +} + // Reconcile starts or stops the syncer for a managed CRD. func (r *CRDController) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) { crd := &apiextensionsv1.CustomResourceDefinition{} @@ -122,52 +166,60 @@ func (r *CRDController) Reconcile(ctx context.Context, req reconcile.Request) (r func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiextensionsv1.CustomResourceDefinition) (reconcile.Result, error) { log := ctrl.LoggerFrom(ctx).WithValues("crd", name) - deleted := crd == nil || crd.DeletionTimestamp != nil - var generation int64 - if crd != nil { - generation = crd.Generation - } - r.lock.Lock() - c, found := r.contexts[name] - if found || deleted { - if !deleted && c.generation >= generation { - r.lock.Unlock() - return reconcile.Result{}, nil - } - if found { - log.Info("stopping syncer") - c.cancel() - waitWithTimeout(c.wg, cleanupTimeout) - delete(r.contexts, name) - } - } - r.lock.Unlock() - if deleted { + // The CRD is gone (unbind removed it) — stop the syncer. + if crd == nil || crd.DeletionTimestamp != nil { + r.stopSyncer(name, log) return reconcile.Result{}, nil } + generation := crd.Generation // The provider is pinned by the binding that pulled this CRD. connName := crd.Annotations[corev1alpha1.AnnotationConnection] if connName == "" { return reconcile.Result{}, fmt.Errorf("managed CRD %s has no %s annotation", name, corev1alpha1.AnnotationConnection) } - // Resolve the multicluster-runtime engaged provider cluster. Until the - // Connection is engaged (it engages once Ready), requeue. - providerCluster, err := r.clusters.Get(ctx, connName) - if err != nil { - // Not an error: the Connection just isn't engaged yet — requeue and wait. - log.V(4).Info("provider cluster not engaged yet, requeueing", "connection", connName, "reason", err.Error()) - return reconcile.Result{RequeueAfter: 2 * time.Second}, nil - } + + // The syncer must only run while the Connection is Ready (engaged). If the + // Connection is gone or no longer Ready, stop the syncer; the Connection watch + // re-triggers this reconcile when it becomes Ready again. conn := &corev1alpha1.Connection{} if err := r.consumerClient.Get(ctx, client.ObjectKey{Name: connName}, conn); err != nil { + if apierrors.IsNotFound(err) { + r.stopSyncer(name, log) + return reconcile.Result{}, nil + } return reconcile.Result{}, fmt.Errorf("getting Connection %s: %w", connName, err) } - if conn.Status.LocalClusterUID == "" { + if !connectionReady(conn) || conn.Status.LocalClusterUID == "" { + r.stopSyncer(name, log) + log.V(4).Info("provider not ready; syncer stopped", "connection", connName) + return reconcile.Result{}, nil + } + + // Resolve the multicluster-runtime engaged provider cluster. A Ready + // Connection engages asynchronously, so it may be briefly absent — stop any + // stale syncer and requeue. + providerCluster, err := r.clusters.Get(ctx, connName) + if err != nil { + r.stopSyncer(name, log) + log.V(4).Info("provider cluster not engaged yet, requeueing", "connection", connName, "reason", err.Error()) return reconcile.Result{RequeueAfter: 2 * time.Second}, nil } + // Already syncing this generation against this exact engaged cluster? Done. + // A different cluster instance means the Connection re-engaged (after a + // disengage) — fall through to rebuild so the syncer tracks the live cluster. + r.lock.Lock() + if c, found := r.contexts[name]; found && c.generation >= generation && c.cluster == providerCluster { + r.lock.Unlock() + return reconcile.Result{}, nil + } + r.lock.Unlock() + + // (Re)build the syncer. Stop any prior one (older generation or dead cluster). + r.stopSyncer(name, log) + version := storageOrServedVersion(crd) if version == "" { return reconcile.Result{}, fmt.Errorf("CRD %s has no served version", name) @@ -191,6 +243,7 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte providerClient: providerCluster.GetClient(), providerReader: providerCluster.GetAPIReader(), localUID: conn.Status.LocalClusterUID, + mapper: r.mapper, recorder: r.recorder, } @@ -212,7 +265,7 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte } // Provider-side watch via the engaged cluster's cache: status/drift events // arrive here instead of being polled. - if err := ctrlr.Watch(providerSource(providerCluster.GetCache(), gvk, conn.Status.LocalClusterUID)); err != nil { + if err := ctrlr.Watch(providerSource(providerCluster.GetCache(), gvr, gvk, conn.Status.LocalClusterUID, r.mapper)); err != nil { return reconcile.Result{}, fmt.Errorf("watching provider cluster: %w", err) } if err := ctrlr.Watch(r.bindingSource(gvr.GroupResource(), lister)); err != nil { @@ -238,7 +291,7 @@ func (r *CRDController) reconcile(ctx context.Context, name string, crd *apiexte } r.lock.Lock() - r.contexts[name] = &syncContext{generation: generation, cancel: cancel, wg: wg} + r.contexts[name] = &syncContext{generation: generation, cluster: providerCluster, cancel: cancel, wg: wg} r.lock.Unlock() log.Info("started syncer", "gvr", gvr.String()) return reconcile.Result{}, nil @@ -310,6 +363,27 @@ func storageOrServedVersion(crd *apiextensionsv1.CustomResourceDefinition) strin return served } +// stopSyncer tears down the running syncer for a CRD, if any, and blocks (up to +// cleanupTimeout) for its goroutines to exit. A no-op when nothing is running. +func (r *CRDController) stopSyncer(name string, log logr.Logger) { + r.lock.Lock() + c, found := r.contexts[name] + if found { + delete(r.contexts, name) + } + r.lock.Unlock() + if !found { + return + } + log.Info("stopping syncer") + c.cancel() + waitWithTimeout(c.wg, cleanupTimeout) +} + +func connectionReady(conn *corev1alpha1.Connection) bool { + return apimeta.IsStatusConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionReady) +} + func waitWithTimeout(wg *sync.WaitGroup, timeout time.Duration) { done := make(chan struct{}) go func() { wg.Wait(); close(done) }() diff --git a/v2/konnector/engine/sync/syncer.go b/v2/konnector/engine/sync/syncer.go index d16830e0d..c0b6b1c27 100644 --- a/v2/konnector/engine/sync/syncer.go +++ b/v2/konnector/engine/sync/syncer.go @@ -38,6 +38,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/reconcile" "sigs.k8s.io/controller-runtime/pkg/source" + "github.com/kube-bind/kube-bind/v2/konnector/engine/mapper" corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" ) @@ -67,14 +68,19 @@ type specReconciler struct { providerReader client.Reader // engaged-cluster API reader: fresh reads localUID string // consumer cluster identity for ownership markers + // mapper translates the consumer object key to its provider key. Core uses + // mapper.Identity (ns/name unchanged); an out-of-tree build can swap it. + mapper mapper.Mapper + recorder record.EventRecorder } // providerSource turns the engaged provider cluster's cache into an event -// source: a provider object owned by us (matching localUID) enqueues its -// identity-mapped consumer object. This is what makes status/drift sync -// event-driven instead of polled. -func providerSource(c cache.Cache, gvk schema.GroupVersionKind, localUID string) source.TypedSource[reconcile.Request] { +// source: a provider object owned by us (matching localUID) enqueues its mapped +// consumer object. This is what makes status/drift sync event-driven instead of +// polled. The provider key is mapped back to the consumer key via the Mapper so +// the request targets the right consumer object under non-identity mappings. +func providerSource(c cache.Cache, gvr schema.GroupVersionResource, gvk schema.GroupVersionKind, localUID string, m mapper.Mapper) source.TypedSource[reconcile.Request] { obj := &unstructured.Unstructured{} obj.SetGroupVersionKind(gvk) return source.Kind(c, client.Object(obj), handler.TypedEnqueueRequestsFromMapFunc( @@ -82,7 +88,11 @@ func providerSource(c cache.Cache, gvk schema.GroupVersionKind, localUID string) if o.GetAnnotations()[corev1alpha1.AnnotationConsumerClusterUID] != localUID { return nil // not ours (another consumer on a shared provider) } - return []reconcile.Request{{NamespacedName: client.ObjectKey{Namespace: o.GetNamespace(), Name: o.GetName()}}} + consumerKey, err := m.ToConsumer(gvr, client.ObjectKey{Namespace: o.GetNamespace(), Name: o.GetName()}) + if err != nil { + return nil + } + return []reconcile.Request{{NamespacedName: consumerKey}} })) } @@ -115,6 +125,14 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( return reconcile.Result{}, nil } + // Map the consumer key to the provider key. Core maps identity; an out-of-tree + // Mapper can rename/re-namespace. Every provider-side operation below uses + // providerKey, never the consumer object's own key. + providerKey, err := r.mapper.ToProvider(r.gvr, client.ObjectKeyFromObject(obj)) + if err != nil { + return reconcile.Result{}, fmt.Errorf("mapping %s to provider: %w", client.ObjectKeyFromObject(obj), err) + } + // Ensure finalizer before first write. if controllerutil.AddFinalizer(obj, corev1alpha1.FinalizerSyncer) { if err := r.consumerClient.Update(ctx, obj); err != nil { @@ -125,7 +143,7 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( // Conflict check before first write (fresh read via the API reader). existing := newUnstructured(r.gvk) - getErr := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), existing) + getErr := r.providerReader.Get(ctx, providerKey, existing) switch { case getErr == nil: if owner, ours := ownershipOf(existing, r.localUID, string(obj.GetUID())); !ours { @@ -133,18 +151,18 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( // one already owned by another binding/consumer. if owner != ownerForeignUnmanaged || res.ConflictPolicy != corev1alpha1.ConflictPolicyAdopt { reason := conflictReason(owner) - msg := fmt.Sprintf("provider object %s already exists and is %s; not overwriting (conflictPolicy: %s)", client.ObjectKeyFromObject(obj), owner, policyOrFail(res.ConflictPolicy)) + msg := fmt.Sprintf("provider object %s already exists and is %s; not overwriting (conflictPolicy: %s)", providerKey, owner, policyOrFail(res.ConflictPolicy)) log.Info("conflict: refusing to overwrite provider object", "owner", owner) if err := r.markConflict(ctx, obj, reason, msg); err != nil { return reconcile.Result{}, err } return reconcile.Result{}, nil } - log.Info("adopting un-owned provider object", "key", client.ObjectKeyFromObject(obj)) + log.Info("adopting un-owned provider object", "key", providerKey) } case apierrors.IsNotFound(getErr): if r.scope == apiextensionsv1.NamespaceScoped { - if err := ensureNamespace(ctx, r.providerClient, obj.GetNamespace(), r.localUID); err != nil { + if err := ensureNamespace(ctx, r.providerClient, providerKey.Namespace, r.localUID); err != nil { if apierrors.IsForbidden(err) { return r.permissionDenied(obj, "creating provider namespace", err), nil } @@ -161,7 +179,7 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( } // Spec consumer -> provider (SSA). - patch := r.providerPatch(obj, r.localUID) + patch := r.providerPatch(obj, providerKey, r.localUID) if err := r.providerClient.Patch(ctx, patch, client.Apply, client.FieldOwner(fieldOwner), client.ForceOwnership); err != nil { if apierrors.IsForbidden(err) { return r.permissionDenied(obj, "applying spec to provider", err), nil @@ -171,7 +189,7 @@ func (r *specReconciler) Reconcile(ctx context.Context, req reconcile.Request) ( // Status provider -> consumer. got := newUnstructured(r.gvk) - if err := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), got); err != nil { + if err := r.providerReader.Get(ctx, providerKey, got); err != nil { return reconcile.Result{}, fmt.Errorf("reading provider status: %w", err) } if err := r.copyStatus(ctx, got, obj); err != nil { @@ -228,8 +246,12 @@ func (r *specReconciler) reconcileDelete(ctx context.Context, obj *unstructured. } // deletion-policy: Orphan keeps the provider copy — just release the finalizer. if obj.GetAnnotations()[corev1alpha1.AnnotationDeletionPolicy] != corev1alpha1.DeletionPolicyOrphan { + providerKey, err := r.mapper.ToProvider(r.gvr, client.ObjectKeyFromObject(obj)) + if err != nil { + return reconcile.Result{}, fmt.Errorf("mapping %s to provider: %w", client.ObjectKeyFromObject(obj), err) + } existing := newUnstructured(r.gvk) - err := r.providerReader.Get(ctx, client.ObjectKeyFromObject(obj), existing) + err = r.providerReader.Get(ctx, providerKey, existing) switch { case err == nil: // Only delete the provider copy if it is ours. A foreign object that @@ -260,11 +282,13 @@ func (r *specReconciler) getConsumerObject(req reconcile.Request) (*unstructured } // providerPatch builds the SSA patch: spec + identity, ownership markers, no -// status, no consumer-only metadata. -func (r *specReconciler) providerPatch(obj *unstructured.Unstructured, localUID string) *unstructured.Unstructured { +// status, no consumer-only metadata. The provider key (from the Mapper) sets the +// name/namespace, so a non-identity mapping writes to the mapped location while +// the ownership markers still record the consumer object's identity. +func (r *specReconciler) providerPatch(obj *unstructured.Unstructured, providerKey mapper.ObjectKey, localUID string) *unstructured.Unstructured { patch := newUnstructured(r.gvk) - patch.SetName(obj.GetName()) - patch.SetNamespace(obj.GetNamespace()) + patch.SetName(providerKey.Name) + patch.SetNamespace(providerKey.Namespace) patch.SetLabels(map[string]string{corev1alpha1.LabelManaged: "true"}) patch.SetAnnotations(map[string]string{ corev1alpha1.AnnotationConsumerClusterUID: localUID, diff --git a/v2/konnector/test/e2e/disengage_test.go b/v2/konnector/test/e2e/disengage_test.go new file mode 100644 index 000000000..a069ca75b --- /dev/null +++ b/v2/konnector/test/e2e/disengage_test.go @@ -0,0 +1,119 @@ +/* +Copyright 2026 The Kube Bind Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package e2e + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/kube-bind/kube-bind/v2/konnector/test/e2e/framework" + corev1alpha1 "github.com/kube-bind/kube-bind/v2/sdk/apis/core/v1alpha1" +) + +// TestSlimCoreStopOnDisengage verifies the syncer lifecycle around a Connection +// losing and regaining readiness: when the Connection stops being Ready the +// provider disengages and the per-GVR syncer is torn down; when it becomes Ready +// again the syncer is rebuilt against the freshly engaged provider cluster +// (rather than left pointing at a dead one), so sync resumes. +func TestSlimCoreStopOnDisengage(t *testing.T) { + env := framework.Start(t) + ctx := context.Background() + gvr := env.InstallExportedWidgetCRD(t) + + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.Connection{ + ObjectMeta: metav1.ObjectMeta{Name: "demo-provider"}, + Spec: corev1alpha1.ConnectionSpec{ + KubeconfigSecretRef: corev1alpha1.SecretKeyRef{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig", Key: "kubeconfig"}, + Schema: corev1alpha1.SchemaPolicy{Source: corev1alpha1.SchemaSourceCRD}, + }, + })) + require.NoError(t, env.ConsumerClient.Create(ctx, &corev1alpha1.ClusterBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets"}, + Spec: corev1alpha1.BindingSpec{ + ConnectionRef: corev1alpha1.ConnectionRef{Name: "demo-provider"}, + APIs: []corev1alpha1.APIRef{{Name: widgetCRDName}}, + }, + })) + waitBindingReady(t, env, ctx) + + consumerWidgets := env.ConsumerDyn.Resource(gvr).Namespace(instanceNS) + providerWidgets := env.ProviderDyn.Resource(gvr).Namespace(instanceNS) + + // Baseline: the syncer is running — an instance syncs to the provider. + _, err := consumerWidgets.Create(ctx, widget("w1", "small"), metav1.CreateOptions{}) + require.NoError(t, err) + require.Eventually(t, func() bool { + _, err := providerWidgets.Get(ctx, "w1", metav1.GetOptions{}) + return err == nil + }, 30*time.Second, 200*time.Millisecond, "baseline instance should sync while the Connection is engaged") + + secretKey := client.ObjectKey{Namespace: framework.KubeBindNamespace, Name: "demo-provider-kubeconfig"} + good := &corev1.Secret{} + require.NoError(t, env.ConsumerClient.Get(ctx, secretKey, good)) + goodKubeconfig := good.Data["kubeconfig"] + + t.Run("a Connection that loses readiness disengages", func(t *testing.T) { + // Corrupt the kubeconfig so the konnector can no longer build a provider + // client → the Connection goes not-Ready → the provider disengages. + bad := good.DeepCopy() + bad.Data["kubeconfig"] = []byte("not-a-kubeconfig") + require.NoError(t, env.ConsumerClient.Update(ctx, bad)) + + require.Eventually(t, func() bool { + conn := &corev1alpha1.Connection{} + if err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "demo-provider"}, conn); err != nil { + return false + } + return !apimeta.IsStatusConditionTrue(conn.Status.Conditions, corev1alpha1.ConditionReady) + }, 30*time.Second, 200*time.Millisecond, "Connection should go not-Ready once its kubeconfig is broken") + }) + + t.Run("re-engage rebuilds the syncer and sync resumes", func(t *testing.T) { + // Restore the good kubeconfig → the Connection becomes Ready again and the + // provider re-engages as a fresh cluster. + cur := &corev1.Secret{} + require.NoError(t, env.ConsumerClient.Get(ctx, secretKey, cur)) + cur.Data["kubeconfig"] = goodKubeconfig + require.NoError(t, env.ConsumerClient.Update(ctx, cur)) + waitBindingReady(t, env, ctx) + + // A NEW instance created after re-engage must sync — proving the syncer was + // rebuilt against the live cluster rather than left pointing at the dead one. + _, err := consumerWidgets.Create(ctx, widget("w2", "large"), metav1.CreateOptions{}) + require.NoError(t, err) + require.Eventually(t, func() bool { + _, err := providerWidgets.Get(ctx, "w2", metav1.GetOptions{}) + return err == nil + }, 60*time.Second, 200*time.Millisecond, "a post-re-engage instance should sync through the rebuilt syncer") + }) +} + +func waitBindingReady(t *testing.T, env *framework.Env, ctx context.Context) { + t.Helper() + framework.WaitForConditionTrue(t, func() ([]metav1.Condition, error) { + cb := &corev1alpha1.ClusterBinding{} + err := env.ConsumerClient.Get(ctx, client.ObjectKey{Name: "widgets"}, cb) + return cb.Status.Conditions, err + }, corev1alpha1.ConditionReady) +} From bffe2da7c6d0feacc44f3ef1ff05c5a303095cd4 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Fri, 12 Jun 2026 14:28:30 +0300 Subject: [PATCH 14/18] wire in production setup/deployments --- v2/Makefile | 21 ++ v2/README.md | 32 +- v2/konnector/Dockerfile | 44 +++ v2/konnector/cmd/konnector/main.go | 38 ++- .../deploy/charts/konnector/.helmignore | 8 + .../deploy/charts/konnector/Chart.yaml | 16 + .../crds/core.kube-bind.io_bindings.yaml | 297 ++++++++++++++++++ .../core.kube-bind.io_clusterbindings.yaml | 295 +++++++++++++++++ .../crds/core.kube-bind.io_connections.yaml | 259 +++++++++++++++ .../charts/konnector/templates/NOTES.txt | 21 ++ .../charts/konnector/templates/_helpers.tpl | 53 ++++ .../konnector/templates/clusterrole.yaml | 54 ++++ .../templates/clusterrolebinding.yaml | 16 + .../charts/konnector/templates/crds.yaml | 6 + .../konnector/templates/deployment.yaml | 76 +++++ .../charts/konnector/templates/role.yaml | 17 + .../konnector/templates/rolebinding.yaml | 17 + .../konnector/templates/serviceaccount.yaml | 13 + .../deploy/charts/konnector/values.yaml | 77 +++++ 19 files changed, 1352 insertions(+), 8 deletions(-) create mode 100644 v2/konnector/Dockerfile create mode 100644 v2/konnector/deploy/charts/konnector/.helmignore create mode 100644 v2/konnector/deploy/charts/konnector/Chart.yaml create mode 100644 v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_bindings.yaml create mode 100644 v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_clusterbindings.yaml create mode 100644 v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/NOTES.txt create mode 100644 v2/konnector/deploy/charts/konnector/templates/_helpers.tpl create mode 100644 v2/konnector/deploy/charts/konnector/templates/clusterrole.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/clusterrolebinding.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/crds.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/deployment.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/role.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/rolebinding.yaml create mode 100644 v2/konnector/deploy/charts/konnector/templates/serviceaccount.yaml create mode 100644 v2/konnector/deploy/charts/konnector/values.yaml diff --git a/v2/Makefile b/v2/Makefile index 50587acb7..5d6a2b973 100644 --- a/v2/Makefile +++ b/v2/Makefile @@ -16,6 +16,27 @@ build: konnector: cd konnector && go build -o bin/konnector ./cmd/konnector +# Container image. Build context is v2/ so the konnector's `replace ../sdk` resolves. +IMAGE ?= ghcr.io/kube-bind/konnector:dev +.PHONY: image +image: + docker build -f konnector/Dockerfile -t $(IMAGE) . + +CHART ?= konnector/deploy/charts/konnector +.PHONY: helm-lint +helm-lint: + helm lint $(CHART) + +# Render the chart to stdout for review (extra flags via HELM_ARGS=...). +.PHONY: helm-template +helm-template: + helm template konnector $(CHART) -n kube-bind $(HELM_ARGS) + +# Refresh the chart's bundled CRDs from the generated sdk CRDs. +.PHONY: helm-sync-crds +helm-sync-crds: codegen + cp sdk/config/crd/core.kube-bind.io_*.yaml $(CHART)/files/crds/ + .PHONY: codegen codegen: cd sdk && $(CONTROLLER_GEN) object paths=./apis/... diff --git a/v2/README.md b/v2/README.md index a91976a02..115523f12 100644 --- a/v2/README.md +++ b/v2/README.md @@ -84,8 +84,7 @@ v2/ Known POC simplifications (tracked against the proposal): OpenAPI synthesis is best-effort (fidelity limits above); the `Mapper` seam exists but only `Identity` -ships and `relatedResources` are not yet routed through it; and productionization -(RBAC/HA/Helm) remains. +ships and `relatedResources` are not yet routed through it. ## Build @@ -93,6 +92,35 @@ ships and `relatedResources` are not yet routed through it; and productionizatio cd v2/konnector && go build ./... # workspace mode (go.work) ``` +## Deploy + +The konnector runs in (or against) the **consumer** cluster — it is the only +running component of the core (no backend, no provider-side controllers). + +```sh +cd v2 +make image IMAGE=ghcr.io/kube-bind/konnector:dev # build the image +helm install konnector konnector/deploy/charts/konnector \ + -n kube-bind --create-namespace \ + --set image.repository=ghcr.io/kube-bind/konnector --set image.tag=dev +``` + +The chart ([konnector/deploy/charts/konnector](konnector/deploy/charts/konnector)) +ships the core CRDs (`installCRDs`, default on), a `ServiceAccount`, the consumer +RBAC (`ClusterRole`/`ClusterRoleBinding` + a namespaced leader-election `Role`), +and the `Deployment` with liveness/readiness probes. Notable values: + +- `replicaCount` / `leaderElect` — HA. Leader election gates all consumer-side + controllers, so standby replicas engage no providers until they win the lease. + It is forced on automatically when `replicaCount > 1`. +- `rbac.boundResourceGroups` (default `["*"]`) — the API groups the konnector may + sync. The bound APIs are open-ended, so this defaults to all groups; narrow it + to the specific groups your providers export to shrink the blast radius. + +Provider credentials are governed by the kubeconfig in each `Connection`'s +Secret, **not** the konnector's ServiceAccount — the RBAC above is consumer-side +only. + ## Codegen (after editing types) ```sh diff --git a/v2/konnector/Dockerfile b/v2/konnector/Dockerfile new file mode 100644 index 000000000..85ca258f3 --- /dev/null +++ b/v2/konnector/Dockerfile @@ -0,0 +1,44 @@ +# Copyright 2026 The Kube Bind Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# v2 slim-core konnector image. +# +# Build context is the v2/ directory (the konnector module's `replace +# github.com/kube-bind/kube-bind/v2/sdk => ../sdk` needs the sibling sdk module): +# +# docker build -f konnector/Dockerfile -t kube-bind/konnector:dev . +# +# from inside v2/. +FROM golang:1.26.2 AS builder +WORKDIR /workspace + +ARG TARGETARCH +ARG TARGETOS +ARG LDFLAGS + +# Both modules are needed: konnector requires sdk via a local replace. Build +# standalone (GOWORK=off) so the image does not depend on the go.work file. +COPY sdk/ sdk/ +COPY konnector/ konnector/ + +WORKDIR /workspace/konnector +ENV GOWORK=off +RUN go mod download +RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \ + go build -ldflags="${LDFLAGS}" -o /bin/konnector ./cmd/konnector + +FROM gcr.io/distroless/static:nonroot +COPY --from=builder /bin/konnector /bin/konnector +USER 65532:65532 +ENTRYPOINT ["/bin/konnector"] diff --git a/v2/konnector/cmd/konnector/main.go b/v2/konnector/cmd/konnector/main.go index abf0bfc08..dc49cfae4 100644 --- a/v2/konnector/cmd/konnector/main.go +++ b/v2/konnector/cmd/konnector/main.go @@ -30,6 +30,7 @@ import ( "k8s.io/apimachinery/pkg/runtime" clientgoscheme "k8s.io/client-go/kubernetes/scheme" ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/healthz" "sigs.k8s.io/controller-runtime/pkg/log/zap" metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" mcmanager "sigs.k8s.io/multicluster-runtime/pkg/manager" @@ -49,21 +50,34 @@ func init() { utilRuntimeMust(corev1alpha1.AddToScheme(scheme)) } +// options holds the konnector's runtime flags. +type options struct { + metricsAddr string + probeAddr string + leaderElect bool + leaderElectionID string +} + func main() { - var metricsAddr string - flag.StringVar(&metricsAddr, "metrics-bind-address", ":8085", "address the metric endpoint binds to") + var o options + flag.StringVar(&o.metricsAddr, "metrics-bind-address", ":8085", "address the metric endpoint binds to") + flag.StringVar(&o.probeAddr, "health-probe-bind-address", ":8081", "address the health/readiness probe endpoint binds to") + flag.BoolVar(&o.leaderElect, "leader-elect", false, + "enable leader election, ensuring only one active konnector replica (required for HA / multiple replicas)") + flag.StringVar(&o.leaderElectionID, "leader-election-id", "konnector.kube-bind.io", + "name of the Lease used for leader election") opts := zap.Options{Development: true} opts.BindFlags(flag.CommandLine) flag.Parse() ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts))) - if err := run(metricsAddr); err != nil { + if err := run(o); err != nil { ctrl.Log.Error(err, "konnector exited with error") os.Exit(1) } } -func run(metricsAddr string) error { +func run(o options) error { ctx := ctrl.SetupSignalHandler() log := ctrl.Log.WithName("konnector") @@ -73,13 +87,25 @@ func run(metricsAddr string) error { } // Local (consumer) manager: owns the core CRDs and the consumer-side caches. + // Leader election gates all consumer-side controllers, so standby replicas + // engage no provider clusters and run no syncers until they win the lease. localMgr, err := ctrl.NewManager(cfg, ctrl.Options{ - Scheme: scheme, - Metrics: metricsserver.Options{BindAddress: metricsAddr}, + Scheme: scheme, + Metrics: metricsserver.Options{BindAddress: o.metricsAddr}, + HealthProbeBindAddress: o.probeAddr, + LeaderElection: o.leaderElect, + LeaderElectionID: o.leaderElectionID, + LeaderElectionReleaseOnCancel: true, }) if err != nil { return err } + if err := localMgr.AddHealthzCheck("healthz", healthz.Ping); err != nil { + return err + } + if err := localMgr.AddReadyzCheck("readyz", healthz.Ping); err != nil { + return err + } // Connection provider: each ready Connection becomes an engaged provider cluster. connProvider, err := provider.New(localMgr, provider.Options{}) diff --git a/v2/konnector/deploy/charts/konnector/.helmignore b/v2/konnector/deploy/charts/konnector/.helmignore new file mode 100644 index 000000000..cb365c23b --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/.helmignore @@ -0,0 +1,8 @@ +# Patterns to ignore when building Helm packages. +.DS_Store +.git/ +.gitignore +*.tmpl +*.gotmpl +.idea/ +*.tgz diff --git a/v2/konnector/deploy/charts/konnector/Chart.yaml b/v2/konnector/deploy/charts/konnector/Chart.yaml new file mode 100644 index 000000000..e606f3946 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/Chart.yaml @@ -0,0 +1,16 @@ +apiVersion: v2 +name: konnector +description: kube-bind v2 slim-core konnector — the consumer-side sync engine. +type: application + +# version / appVersion are placeholders bumped by the release tooling. +version: 0.0.0 +appVersion: "0.0.0" + +home: https://kube-bind.io +sources: + - https://github.com/kube-bind/kube-bind +keywords: + - kube-bind + - multicluster + - api-binding diff --git a/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_bindings.yaml b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_bindings.yaml new file mode 100644 index 000000000..946edc752 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_bindings.yaml @@ -0,0 +1,297 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: bindings.core.kube-bind.io +spec: + group: core.kube-bind.io + names: + categories: + - kube-bind + kind: Binding + listKind: BindingList + plural: bindings + singular: binding + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.connectionRef.name + name: Connection + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + Binding activates instance sync for one or more exported APIs within its own + namespace. Namespaced, following the Role convention. It is the v2 answer to + v1's informerScope: Namespaced. Where a ClusterBinding and a Binding cover the + same API on the same connection, the ClusterBinding wins. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + BindingSpec is the spec shared by ClusterBinding and Binding. The only + difference between the two kinds is scope (cluster-wide vs. one namespace), + which is expressed by the kind, not by a field. + properties: + apis: + description: apis lists one or more exported CRDs to sync, by CRD + name on the provider. + items: + description: |- + APIRef identifies an exported API by its CRD name on the provider, + i.e. "." (for example "mangodbs.mangodb.io"). Under the + OpenAPI schema source there is no CRD object behind the name; it is still + just resource + group. + properties: + name: + description: name is the CRD name of the exported API, ".". + minLength: 1 + type: string + required: + - name + type: object + minItems: 1 + type: array + conflictPolicy: + default: Fail + description: |- + conflictPolicy controls what happens when a target object already exists. + Fail (default) leaves a foreign object untouched and records a conflict; + Adopt takes ownership of an un-owned object. Adopt never steals an object + already carrying another binding's/consumer's markers. + enum: + - Fail + - Adopt + type: string + connectionRef: + description: |- + connectionRef points at the Connection that provides the provider link + and credentials for the listed APIs. + properties: + name: + description: name of the Connection. + minLength: 1 + type: string + required: + - name + type: object + relatedResources: + description: |- + relatedResources are Secrets/ConfigMaps synced alongside instances of the + bound APIs. Not yet synced in the alpha POC. + items: + description: |- + RelatedResource selects auxiliary objects (Secrets/ConfigMaps) to sync + alongside the bound instances. + properties: + direction: + description: direction the related resource flows. + enum: + - FromProvider + - FromConsumer + type: string + group: + default: "" + description: group of the related resource. Empty string for + the core group. + type: string + resource: + description: |- + resource is the plural resource name. Only "secrets" and "configmaps" + are permitted in core. + enum: + - secrets + - configmaps + type: string + selector: + description: |- + selector restricts which objects are synced. Only labelSelector and + named selectors are supported in core (no JSONPath reference-following). + properties: + labelSelector: + description: labelSelector selects related objects by label. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + names: + description: names selects related objects by exact name. + items: + type: string + type: array + x-kubernetes-list-type: set + type: object + required: + - direction + - group + - resource + type: object + type: array + x-kubernetes-list-type: atomic + required: + - apis + - connectionRef + type: object + status: + description: BindingStatus is the observed state shared by ClusterBinding + and Binding. + properties: + boundAPIs: + description: boundAPIs is per-API observed state. + items: + description: BoundAPI is the per-API observed state recorded on + a binding's status. + properties: + conflictCount: + description: |- + conflictCount is the number of objects skipped due to foreign ownership. + Per-object detail lives on each object's own condition, not here. + format: int32 + type: integer + crdHash: + description: |- + crdHash is a hash of the schema currently applied on the consumer for + this API. Empty until the schema has been installed. + type: string + name: + description: name is the CRD name of the bound API, ".". + type: string + required: + - name + type: object + type: array + x-kubernetes-list-type: atomic + conditions: + description: 'conditions: Connected, Synced, Conflicts, PermissionDenied, + Ready.' + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_clusterbindings.yaml b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_clusterbindings.yaml new file mode 100644 index 000000000..5506921f1 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_clusterbindings.yaml @@ -0,0 +1,295 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: clusterbindings.core.kube-bind.io +spec: + group: core.kube-bind.io + names: + categories: + - kube-bind + kind: ClusterBinding + listKind: ClusterBindingList + plural: clusterbindings + singular: clusterbinding + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .spec.connectionRef.name + name: Connection + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + ClusterBinding activates instance sync for one or more exported APIs + cluster-wide. Cluster-scoped, following the ClusterRole convention. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + BindingSpec is the spec shared by ClusterBinding and Binding. The only + difference between the two kinds is scope (cluster-wide vs. one namespace), + which is expressed by the kind, not by a field. + properties: + apis: + description: apis lists one or more exported CRDs to sync, by CRD + name on the provider. + items: + description: |- + APIRef identifies an exported API by its CRD name on the provider, + i.e. "." (for example "mangodbs.mangodb.io"). Under the + OpenAPI schema source there is no CRD object behind the name; it is still + just resource + group. + properties: + name: + description: name is the CRD name of the exported API, ".". + minLength: 1 + type: string + required: + - name + type: object + minItems: 1 + type: array + conflictPolicy: + default: Fail + description: |- + conflictPolicy controls what happens when a target object already exists. + Fail (default) leaves a foreign object untouched and records a conflict; + Adopt takes ownership of an un-owned object. Adopt never steals an object + already carrying another binding's/consumer's markers. + enum: + - Fail + - Adopt + type: string + connectionRef: + description: |- + connectionRef points at the Connection that provides the provider link + and credentials for the listed APIs. + properties: + name: + description: name of the Connection. + minLength: 1 + type: string + required: + - name + type: object + relatedResources: + description: |- + relatedResources are Secrets/ConfigMaps synced alongside instances of the + bound APIs. Not yet synced in the alpha POC. + items: + description: |- + RelatedResource selects auxiliary objects (Secrets/ConfigMaps) to sync + alongside the bound instances. + properties: + direction: + description: direction the related resource flows. + enum: + - FromProvider + - FromConsumer + type: string + group: + default: "" + description: group of the related resource. Empty string for + the core group. + type: string + resource: + description: |- + resource is the plural resource name. Only "secrets" and "configmaps" + are permitted in core. + enum: + - secrets + - configmaps + type: string + selector: + description: |- + selector restricts which objects are synced. Only labelSelector and + named selectors are supported in core (no JSONPath reference-following). + properties: + labelSelector: + description: labelSelector selects related objects by label. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + names: + description: names selects related objects by exact name. + items: + type: string + type: array + x-kubernetes-list-type: set + type: object + required: + - direction + - group + - resource + type: object + type: array + x-kubernetes-list-type: atomic + required: + - apis + - connectionRef + type: object + status: + description: BindingStatus is the observed state shared by ClusterBinding + and Binding. + properties: + boundAPIs: + description: boundAPIs is per-API observed state. + items: + description: BoundAPI is the per-API observed state recorded on + a binding's status. + properties: + conflictCount: + description: |- + conflictCount is the number of objects skipped due to foreign ownership. + Per-object detail lives on each object's own condition, not here. + format: int32 + type: integer + crdHash: + description: |- + crdHash is a hash of the schema currently applied on the consumer for + this API. Empty until the schema has been installed. + type: string + name: + description: name is the CRD name of the bound API, ".". + type: string + required: + - name + type: object + type: array + x-kubernetes-list-type: atomic + conditions: + description: 'conditions: Connected, Synced, Conflicts, PermissionDenied, + Ready.' + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml new file mode 100644 index 000000000..92437e2c4 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml @@ -0,0 +1,259 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: connections.core.kube-bind.io +spec: + group: core.kube-bind.io + names: + categories: + - kube-bind + kind: Connection + listKind: ConnectionList + plural: connections + singular: connection + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .spec.kubeconfigSecretRef.name + name: Secret + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + Connection is the link to one provider cluster. It owns the credentials and + schema delivery, and surfaces what the provider exports. Cluster-scoped. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: ConnectionSpec defines the desired provider link. + properties: + autoBind: + default: false + description: |- + autoBind, when true, makes the konnector maintain a managed ClusterBinding + (named after this Connection) covering all exported APIs. + type: boolean + kubeconfigSecretRef: + description: |- + kubeconfigSecretRef points at the Secret holding the provider kubeconfig. + Immutable. The only credential reference in the core. + properties: + key: + default: kubeconfig + description: key within the Secret holding the kubeconfig. + type: string + name: + description: name of the Secret. + minLength: 1 + type: string + namespace: + description: namespace of the Secret. Must be the konnector's + designated namespace. + minLength: 1 + type: string + required: + - name + - namespace + type: object + x-kubernetes-validations: + - message: kubeconfigSecretRef is immutable + rule: self == oldSelf + schema: + default: {} + description: schema controls how exported APIs reach the consumer. + properties: + pullPolicy: + default: Bound + description: |- + pullPolicy selects which exported APIs are installed: + Bound - only APIs referenced by a Binding/ClusterBinding (default). + All - every exported API readable by the credentials. + None - never install CRDs (user/extension manages them). + enum: + - Bound + - All + - None + type: string + source: + default: Auto + description: |- + source selects how schemas are obtained: + Auto - CRD if readable on the provider, else OpenAPI (default). + CRD - read apiextensions CRDs verbatim. + OpenAPI - synthesize CRDs from discovery + /openapi/v3. + enum: + - Auto + - CRD + - OpenAPI + type: string + updatePolicy: + default: Always + description: |- + updatePolicy selects whether installed schemas follow provider changes: + Always - follow provider schema changes (default). + Once - pin at first pull. + enum: + - Always + - Once + type: string + type: object + required: + - kubeconfigSecretRef + type: object + status: + description: ConnectionStatus is the observed state of a Connection. + properties: + activeSchemaSource: + description: |- + activeSchemaSource is the schema source actually in effect after resolving + "Auto" (CRD or OpenAPI). The binding uses it to decide whether the + Connection already installed the CRDs (OpenAPI) or it should pull them (CRD). + type: string + conditions: + description: 'conditions: SecretValid, Connected, SchemaInSync, Ready.' + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + exportedAPIs: + description: 'exportedAPIs is the discovery result: APIs exported + to these credentials.' + items: + description: ExportedAPI describes one API the provider exports + to these credentials. + properties: + group: + description: group of the exported API. + type: string + name: + description: name is the CRD name of the exported API, ".". + type: string + resource: + description: resource is the plural resource name of the exported + API. + type: string + scope: + description: scope is whether the API is Namespaced or Cluster + scoped. + type: string + versions: + description: versions are the served versions of the exported + API. + items: + type: string + type: array + x-kubernetes-list-type: set + required: + - group + - name + - resource + - scope + - versions + type: object + type: array + x-kubernetes-list-type: atomic + localClusterUID: + description: |- + localClusterUID is the identity of the consumer cluster, pinned on first + connect and immutable thereafter. + type: string + x-kubernetes-validations: + - message: localClusterUID is immutable + rule: self == oldSelf + remoteClusterUID: + description: |- + remoteClusterUID is the identity of the provider cluster, pinned on first + connect and immutable thereafter. A Secret later pointing at a different + cluster is rejected rather than silently re-homing synced objects. + type: string + x-kubernetes-validations: + - message: remoteClusterUID is immutable + rule: self == oldSelf + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/v2/konnector/deploy/charts/konnector/templates/NOTES.txt b/v2/konnector/deploy/charts/konnector/templates/NOTES.txt new file mode 100644 index 000000000..688e9afa2 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/NOTES.txt @@ -0,0 +1,21 @@ +The kube-bind konnector is installed as "{{ include "konnector.fullname" . }}" in namespace "{{ .Release.Namespace }}". + +Replicas: {{ .Values.replicaCount }} +Leader election: {{ include "konnector.leaderElect" . }} + +Check it is running: + + kubectl -n {{ .Release.Namespace }} rollout status deploy/{{ include "konnector.fullname" . }} + +Bind a provider API (one apply): create the kubeconfig Secret, a Connection +referencing it, and a ClusterBinding/Binding selecting the exported APIs. See +the konnector/config/samples/ directory for examples. + +{{- if eq (include "konnector.leaderElect" .) "false" }} +{{- if gt (int .Values.replicaCount) 1 }} + +WARNING: replicaCount > 1 with leader election disabled runs multiple active +konnectors against the same cluster, which will fight over the same objects. +Set leaderElect: true (or it is forced on automatically for replicaCount > 1). +{{- end }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/_helpers.tpl b/v2/konnector/deploy/charts/konnector/templates/_helpers.tpl new file mode 100644 index 000000000..f0392e9c6 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/_helpers.tpl @@ -0,0 +1,53 @@ +{{/* Chart name (overridable). */}} +{{- define "konnector.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* Fully qualified app name. */}} +{{- define "konnector.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* Chart label. */}} +{{- define "konnector.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* Common labels. */}} +{{- define "konnector.labels" -}} +helm.sh/chart: {{ include "konnector.chart" . }} +{{ include "konnector.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* Selector labels. */}} +{{- define "konnector.selectorLabels" -}} +app.kubernetes.io/name: {{ include "konnector.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* ServiceAccount name. */}} +{{- define "konnector.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "konnector.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + +{{/* Whether leader election is effectively on (forced when replicaCount > 1). */}} +{{- define "konnector.leaderElect" -}} +{{- if or .Values.leaderElect (gt (int .Values.replicaCount) 1) }}true{{ else }}false{{ end }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/clusterrole.yaml b/v2/konnector/deploy/charts/konnector/templates/clusterrole.yaml new file mode 100644 index 000000000..b890c84e3 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/clusterrole.yaml @@ -0,0 +1,54 @@ +{{- if .Values.rbac.create }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ include "konnector.fullname" . }} + labels: + {{- include "konnector.labels" . | nindent 4 }} +rules: + # Core kube-bind objects the konnector reconciles (+ status, + finalizers). + - apiGroups: ["core.kube-bind.io"] + resources: ["connections", "clusterbindings", "bindings"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] + - apiGroups: ["core.kube-bind.io"] + resources: ["connections/status", "clusterbindings/status", "bindings/status"] + verbs: ["get", "update", "patch"] + - apiGroups: ["core.kube-bind.io"] + resources: ["connections/finalizers", "clusterbindings/finalizers", "bindings/finalizers"] + verbs: ["update"] + # CRDs: the konnector pulls/synthesizes and installs the bound APIs, stamps + # markers, and adds/removes finalizers. + - apiGroups: ["apiextensions.k8s.io"] + resources: ["customresourcedefinitions"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] + # Provider kubeconfig Secrets: read + finalizer (never deleted by the konnector). + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "list", "watch", "update", "patch"] + # Namespaces: read for cluster identity; create when materializing + # related-resource copies into consumer namespaces. + - apiGroups: [""] + resources: ["namespaces"] + verbs: ["get", "list", "watch", "create"] + # Cluster identity on kcp-shaped consumers (best-effort; harmless if absent). + - apiGroups: ["core.kcp.io"] + resources: ["logicalclusters"] + verbs: ["get", "list", "watch"] + # Events emitted by the reconcilers (conflicts, permission denials). + - apiGroups: [""] + resources: ["events"] + verbs: ["create", "patch"] + - apiGroups: ["events.k8s.io"] + resources: ["events"] + verbs: ["create", "patch"] + # Instances of the bound APIs synced between consumer and provider. These are + # open-ended (whatever providers export), so access is granted per + # .Values.rbac.boundResourceGroups — default "*". Narrow it to the specific + # provider API groups to reduce the blast radius. + {{- with .Values.rbac.boundResourceGroups }} + - apiGroups: + {{- toYaml . | nindent 6 }} + resources: ["*"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] + {{- end }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/clusterrolebinding.yaml b/v2/konnector/deploy/charts/konnector/templates/clusterrolebinding.yaml new file mode 100644 index 000000000..b1011da50 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/clusterrolebinding.yaml @@ -0,0 +1,16 @@ +{{- if .Values.rbac.create }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "konnector.fullname" . }} + labels: + {{- include "konnector.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ include "konnector.fullname" . }} +subjects: + - kind: ServiceAccount + name: {{ include "konnector.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/crds.yaml b/v2/konnector/deploy/charts/konnector/templates/crds.yaml new file mode 100644 index 000000000..693db13a2 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/crds.yaml @@ -0,0 +1,6 @@ +{{- if .Values.installCRDs }} +{{- range $path, $_ := .Files.Glob "files/crds/*.yaml" }} +--- +{{ $.Files.Get $path }} +{{- end }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/deployment.yaml b/v2/konnector/deploy/charts/konnector/templates/deployment.yaml new file mode 100644 index 000000000..7efc03147 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/deployment.yaml @@ -0,0 +1,76 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "konnector.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "konnector.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.replicaCount }} + selector: + matchLabels: + {{- include "konnector.selectorLabels" . | nindent 6 }} + template: + metadata: + labels: + {{- include "konnector.selectorLabels" . | nindent 8 }} + {{- with .Values.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.podAnnotations }} + annotations: + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + serviceAccountName: {{ include "konnector.serviceAccountName" . }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + securityContext: + {{- toYaml .Values.podSecurityContext | nindent 8 }} + containers: + - name: konnector + image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --metrics-bind-address=:{{ .Values.metrics.port }} + - --health-probe-bind-address=:{{ .Values.healthProbe.port }} + - --leader-elect={{ include "konnector.leaderElect" . }} + - --leader-election-id={{ .Values.leaderElectionID }} + {{- with .Values.extraArgs }} + {{- toYaml . | nindent 12 }} + {{- end }} + ports: + - name: metrics + containerPort: {{ .Values.metrics.port }} + - name: probe + containerPort: {{ .Values.healthProbe.port }} + livenessProbe: + httpGet: + path: /healthz + port: probe + initialDelaySeconds: 15 + periodSeconds: 20 + readinessProbe: + httpGet: + path: /readyz + port: probe + initialDelaySeconds: 5 + periodSeconds: 10 + securityContext: + {{- toYaml .Values.securityContext | nindent 12 }} + resources: + {{- toYaml .Values.resources | nindent 12 }} + {{- with .Values.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/role.yaml b/v2/konnector/deploy/charts/konnector/templates/role.yaml new file mode 100644 index 000000000..209a59d68 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/role.yaml @@ -0,0 +1,17 @@ +{{- if .Values.rbac.create }} +# Leader-election lease + bootstrap, scoped to the release namespace. +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "konnector.fullname" . }}-leaderelection + namespace: {{ .Release.Namespace }} + labels: + {{- include "konnector.labels" . | nindent 4 }} +rules: + - apiGroups: ["coordination.k8s.io"] + resources: ["leases"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] + - apiGroups: [""] + resources: ["events"] + verbs: ["create", "patch"] +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/rolebinding.yaml b/v2/konnector/deploy/charts/konnector/templates/rolebinding.yaml new file mode 100644 index 000000000..251a12a27 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/rolebinding.yaml @@ -0,0 +1,17 @@ +{{- if .Values.rbac.create }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "konnector.fullname" . }}-leaderelection + namespace: {{ .Release.Namespace }} + labels: + {{- include "konnector.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "konnector.fullname" . }}-leaderelection +subjects: + - kind: ServiceAccount + name: {{ include "konnector.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/templates/serviceaccount.yaml b/v2/konnector/deploy/charts/konnector/templates/serviceaccount.yaml new file mode 100644 index 000000000..73a4c86a0 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/templates/serviceaccount.yaml @@ -0,0 +1,13 @@ +{{- if .Values.serviceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "konnector.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "konnector.labels" . | nindent 4 }} + {{- with .Values.serviceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/v2/konnector/deploy/charts/konnector/values.yaml b/v2/konnector/deploy/charts/konnector/values.yaml new file mode 100644 index 000000000..075398588 --- /dev/null +++ b/v2/konnector/deploy/charts/konnector/values.yaml @@ -0,0 +1,77 @@ +# Default values for the konnector chart. + +image: + repository: ghcr.io/kube-bind/konnector + # Defaults to the chart appVersion when empty. + tag: "" + pullPolicy: IfNotPresent + +imagePullSecrets: [] +nameOverride: "" +fullnameOverride: "" + +# Number of konnector replicas. >1 requires leaderElect (only one is active at a +# time; the rest are warm standbys). +replicaCount: 1 + +# Leader election ensures a single active replica. Enable for HA / replicaCount>1. +# It is forced on automatically whenever replicaCount > 1. +leaderElect: false +leaderElectionID: konnector.kube-bind.io + +serviceAccount: + create: true + name: "" + annotations: {} + +rbac: + create: true + # The konnector syncs instances of the bound (provider-exported) APIs on this + # consumer cluster. Those APIs are open-ended, so by default it is granted the + # standard verbs on all API groups. Narrow this to the specific groups your + # providers export (e.g. ["mounts.example.org", "example.org"]) to tighten the + # blast radius — but bindings for groups not listed here will fail to sync. + boundResourceGroups: + - "*" + +# Extra args appended to the konnector command. +extraArgs: [] + +metrics: + # Port the controller-runtime metrics endpoint binds to (matches --metrics-bind-address). + port: 8085 + +healthProbe: + # Port the health/readiness probe endpoint binds to (matches --health-probe-bind-address). + port: 8081 + +resources: + requests: + cpu: 100m + memory: 128Mi + limits: + cpu: 500m + memory: 256Mi + +podAnnotations: {} +podLabels: {} + +podSecurityContext: + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault + +securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + drop: + - ALL + +nodeSelector: {} +tolerations: [] +affinity: {} + +# Install the core CRDs (Connection, ClusterBinding, Binding) with the release. +# Set false if they are managed out-of-band. +installCRDs: true From a2f2986ad423af4b7fb7b5efd79b90cc2146d9d6 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Mon, 15 Jun 2026 09:51:13 +0300 Subject: [PATCH 15/18] fix boilerplate --- v2/Makefile | 14 +++++++++++++- v2/hack/demo.sh | 15 ++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/v2/Makefile b/v2/Makefile index 5d6a2b973..86dd70e2d 100644 --- a/v2/Makefile +++ b/v2/Makefile @@ -1,5 +1,17 @@ -# Copyright 2026 The Kube Bind Authors. Licensed under the Apache License 2.0. +# Copyright 2026 The Kube Bind Authors. # +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + # v2 slim-core. Run from the v2/ directory. CONTROLLER_GEN ?= go run sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.2 diff --git a/v2/hack/demo.sh b/v2/hack/demo.sh index 0353ca735..fb166889f 100755 --- a/v2/hack/demo.sh +++ b/v2/hack/demo.sh @@ -1,6 +1,19 @@ #!/usr/bin/env bash -# Copyright 2026 The Kube Bind Authors. Licensed under the Apache License 2.0. + +# Copyright 2026 The Kube Bind Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at # +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + # Stands up two kind clusters (provider + consumer) and wires the v2 slim-core # demo: a provider exports the Widget CRD, the consumer binds it, and the # konnector syncs instances. Run the konnector separately (printed at the end). From dea02851a9a34cb3cd19d36bd82bf53fe1ba6c26 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Mon, 15 Jun 2026 11:06:14 +0300 Subject: [PATCH 16/18] wire in v2 CI --- .github/workflows/ci-docs-only.yaml | 6 ++++++ .github/workflows/ci.yaml | 15 +++++++++++++++ v2/Makefile | 4 +++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci-docs-only.yaml b/.github/workflows/ci-docs-only.yaml index 26526240b..b48c44f0c 100644 --- a/.github/workflows/ci-docs-only.yaml +++ b/.github/workflows/ci-docs-only.yaml @@ -37,6 +37,12 @@ jobs: steps: - run: 'echo "No build required"' + v2-test: + name: v2-test + runs-on: ubuntu-latest + steps: + - run: 'echo "No build required"' + lint: name: lint runs-on: ubuntu-latest diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 08de66a12..28dc683d9 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -63,6 +63,21 @@ jobs: go-version: v1.26.2 - run: make test + # The v2 slim-core lives in its own go.work workspace (v2/sdk + v2/konnector) + # which the root `make test` does not descend into. lint/verify already cover + # v2 via GOMODS; this job adds its build, vet, unit, and envtest e2e suites. + v2-test: + name: v2-test + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + - uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 + with: + go-version: v1.26.2 + - run: cd v2 && make build vet test + # test-e2e downloads envtest binaries on demand via setup-envtest. + - run: cd v2 && make test-e2e + lint: name: lint runs-on: ubuntu-latest diff --git a/v2/Makefile b/v2/Makefile index 86dd70e2d..103e0447e 100644 --- a/v2/Makefile +++ b/v2/Makefile @@ -57,9 +57,11 @@ codegen: ENVTEST_K8S_VERSION ?= 1.34.1 SETUP_ENVTEST ?= go run sigs.k8s.io/controller-runtime/tools/setup-envtest@release-0.21 +# Unit tests only. The envtest-based e2e (which needs KUBEBUILDER_ASSETS) is a +# separate target so `make test` runs with no external setup. .PHONY: test test: - cd konnector && go test ./... + cd konnector && go test $$(go list ./... | grep -v /test/e2e) cd sdk && go test ./... # Run the envtest-based e2e (two in-process API servers + the engine). From 6a4b199b81062f1fd2fcbb2d5a93b0d44f3e66a0 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Mon, 15 Jun 2026 11:23:32 +0300 Subject: [PATCH 17/18] Add Ci for V2 images --- .github/workflows/ci.yaml | 16 ++++++ .github/workflows/image-konnector-v2.yaml | 64 +++++++++++++++++++++++ v2/konnector/Dockerfile | 4 +- 3 files changed, 83 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/image-konnector-v2.yaml diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 28dc683d9..cec96c951 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -108,3 +108,19 @@ jobs: go-version: v1.26.2 - uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 - run: make image-local + + # Verify the v2 konnector image builds for all release architectures, without + # pushing. Mirrors the v2 release workflow's multi-arch buildx invocation. + v2-image-build: + name: v2-image-build + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + - uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + - name: Build v2 konnector image (multi-arch, no push) + run: | + docker buildx build \ + --platform linux/amd64,linux/arm64 \ + --file v2/konnector/Dockerfile \ + --provenance=false \ + v2 diff --git a/.github/workflows/image-konnector-v2.yaml b/.github/workflows/image-konnector-v2.yaml new file mode 100644 index 000000000..38e068149 --- /dev/null +++ b/.github/workflows/image-konnector-v2.yaml @@ -0,0 +1,64 @@ +name: Image konnector v2 + +# Publishes the v2 slim-core konnector image. Triggered ONLY by tags in the +# `konnector/v2*` namespace (e.g. `konnector/v2.0.0`), which deliberately do not +# match the `v*` tag filter the v1 image/goreleaser/CI workflows use — so this is +# fully independent of the existing v1 release infrastructure. +# +# The image is pushed to the shared `konnector` package under a v2 tag (the part +# after `konnector/`), e.g. ghcr.io//konnector:v2.0.0. It is NOT tagged +# `latest`: v1 and v2 are distinct builds and `latest` stays with v1. +on: + push: + tags: + - 'konnector/v2*' + +permissions: + contents: read + packages: write + +jobs: + image: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + + - name: Compute image ref + id: meta + run: | + # refs/tags/konnector/v2.0.0 -> GITHUB_REF_NAME=konnector/v2.0.0 -> version=v2.0.0 + version="${GITHUB_REF_NAME#konnector/}" + owner="$(echo "${{ github.repository_owner }}" | tr '[:upper:]' '[:lower:]')" + { + echo "version=${version}" + echo "image=ghcr.io/${owner}/konnector" + } >> "$GITHUB_OUTPUT" + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + + - name: Login to GitHub Container Registry + uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + # Multi-arch via in-Dockerfile cross-compile (builder is pinned to + # BUILDPLATFORM), so no QEMU is required. Build context is v2/ so the + # konnector module's `replace ../sdk` resolves. No :latest tag. + - name: Build and push v2 konnector image + run: | + docker buildx build \ + --platform linux/amd64,linux/arm64 \ + --file v2/konnector/Dockerfile \ + --tag "${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.version }}" \ + --label "org.opencontainers.image.source=https://github.com/${{ github.repository }}" \ + --label "org.opencontainers.image.revision=${{ github.sha }}" \ + --label "org.opencontainers.image.version=${{ steps.meta.outputs.version }}" \ + --label "org.opencontainers.image.title=konnector" \ + --cache-from type=gha \ + --cache-to type=gha,mode=max \ + --provenance=false \ + --push \ + v2 diff --git a/v2/konnector/Dockerfile b/v2/konnector/Dockerfile index 85ca258f3..ab80958b6 100644 --- a/v2/konnector/Dockerfile +++ b/v2/konnector/Dockerfile @@ -20,7 +20,9 @@ # docker build -f konnector/Dockerfile -t kube-bind/konnector:dev . # # from inside v2/. -FROM golang:1.26.2 AS builder +# Pin the builder to the build host's native platform and cross-compile via +# GOARCH/GOOS (CGO disabled), so multi-arch builds need no QEMU emulation. +FROM --platform=$BUILDPLATFORM golang:1.26.2 AS builder WORKDIR /workspace ARG TARGETARCH From d9ee090aa72249e9305cd3a1e7dc52cbb52ea758 Mon Sep 17 00:00:00 2001 From: Mangirdas Judeikis Date: Mon, 15 Jun 2026 12:29:40 +0300 Subject: [PATCH 18/18] adress reviews --- docs/proposals/v2-extended.md | 2 +- .../files/crds/core.kube-bind.io_connections.yaml | 8 ++++---- v2/konnector/engine/binding/reconciler.go | 14 +++++++++++--- v2/sdk/apis/core/v1alpha1/connection_types.go | 4 ++-- .../config/crd/core.kube-bind.io_connections.yaml | 8 ++++---- 5 files changed, 22 insertions(+), 14 deletions(-) diff --git a/docs/proposals/v2-extended.md b/docs/proposals/v2-extended.md index 913df826d..a78cc6b77 100644 --- a/docs/proposals/v2-extended.md +++ b/docs/proposals/v2-extended.md @@ -3,7 +3,7 @@ * Status: **DRAFT — for iteration** * Authors: @mjudeikis * Date: 2026-06-10 -* Builds on: [v2-slim-core.md](v2-slim-core.md) (Proposed)) +* Builds on: [v2-slim-core.md](v2-slim-core.md) (Proposed) ## Summary diff --git a/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml index 92437e2c4..59ea041a7 100644 --- a/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml +++ b/v2/konnector/deploy/charts/konnector/files/crds/core.kube-bind.io_connections.yaml @@ -238,8 +238,8 @@ spec: connect and immutable thereafter. type: string x-kubernetes-validations: - - message: localClusterUID is immutable - rule: self == oldSelf + - message: localClusterUID is immutable once set + rule: oldSelf == '' || self == oldSelf remoteClusterUID: description: |- remoteClusterUID is the identity of the provider cluster, pinned on first @@ -247,8 +247,8 @@ spec: cluster is rejected rather than silently re-homing synced objects. type: string x-kubernetes-validations: - - message: remoteClusterUID is immutable - rule: self == oldSelf + - message: remoteClusterUID is immutable once set + rule: oldSelf == '' || self == oldSelf type: object required: - spec diff --git a/v2/konnector/engine/binding/reconciler.go b/v2/konnector/engine/binding/reconciler.go index 8b8635eeb..25055f32d 100644 --- a/v2/konnector/engine/binding/reconciler.go +++ b/v2/konnector/engine/binding/reconciler.go @@ -98,8 +98,9 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc } var ( - boundAPIs []corev1alpha1.BoundAPI - notExported []string + boundAPIs []corev1alpha1.BoundAPI + notExported []string + pendingSchema []string ) for _, api := range spec.APIs { exported, ok := conn.Status.ExportsAPI(api.Name) @@ -114,7 +115,9 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc crd := &apiextensionsv1.CustomResourceDefinition{} if err := b.client.Get(ctx, client.ObjectKey{Name: api.Name}, crd); err != nil { if apierrors.IsNotFound(err) { - notExported = append(notExported, api.Name) + // The API is exported, but the Connection has not installed the + // synthesized CRD yet — a transient wait, not "not exported". + pendingSchema = append(pendingSchema, api.Name) continue } return fmt.Errorf("getting synthesized CRD %q: %w", api.Name, err) @@ -151,6 +154,11 @@ func (b *base) reconcileAccessor(ctx context.Context, obj corev1alpha1.BindingAc fmt.Sprintf("APIs not exported by the provider yet: %s", strings.Join(notExported, ", "))) return nil } + if len(pendingSchema) > 0 { + setReady(status, obj, metav1.ConditionFalse, corev1alpha1.ReasonPending, + fmt.Sprintf("waiting for the Connection to install the synthesized CRD(s): %s", strings.Join(pendingSchema, ", "))) + return nil + } // Sync related Secrets/ConfigMaps in the declared direction, scoped like the // binding. diff --git a/v2/sdk/apis/core/v1alpha1/connection_types.go b/v2/sdk/apis/core/v1alpha1/connection_types.go index 3e6964171..2a68ec73b 100644 --- a/v2/sdk/apis/core/v1alpha1/connection_types.go +++ b/v2/sdk/apis/core/v1alpha1/connection_types.go @@ -134,14 +134,14 @@ type ConnectionStatus struct { // cluster is rejected rather than silently re-homing synced objects. // // +optional - // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="remoteClusterUID is immutable" + // +kubebuilder:validation:XValidation:rule="oldSelf == '' || self == oldSelf",message="remoteClusterUID is immutable once set" RemoteClusterUID string `json:"remoteClusterUID,omitempty"` // localClusterUID is the identity of the consumer cluster, pinned on first // connect and immutable thereafter. // // +optional - // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="localClusterUID is immutable" + // +kubebuilder:validation:XValidation:rule="oldSelf == '' || self == oldSelf",message="localClusterUID is immutable once set" LocalClusterUID string `json:"localClusterUID,omitempty"` // activeSchemaSource is the schema source actually in effect after resolving diff --git a/v2/sdk/config/crd/core.kube-bind.io_connections.yaml b/v2/sdk/config/crd/core.kube-bind.io_connections.yaml index 92437e2c4..59ea041a7 100644 --- a/v2/sdk/config/crd/core.kube-bind.io_connections.yaml +++ b/v2/sdk/config/crd/core.kube-bind.io_connections.yaml @@ -238,8 +238,8 @@ spec: connect and immutable thereafter. type: string x-kubernetes-validations: - - message: localClusterUID is immutable - rule: self == oldSelf + - message: localClusterUID is immutable once set + rule: oldSelf == '' || self == oldSelf remoteClusterUID: description: |- remoteClusterUID is the identity of the provider cluster, pinned on first @@ -247,8 +247,8 @@ spec: cluster is rejected rather than silently re-homing synced objects. type: string x-kubernetes-validations: - - message: remoteClusterUID is immutable - rule: self == oldSelf + - message: remoteClusterUID is immutable once set + rule: oldSelf == '' || self == oldSelf type: object required: - spec