-
Notifications
You must be signed in to change notification settings - Fork 136
Document v3 CRD mode as the default for new installs #2805
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
ece3fb6
df6c3eb
8903b93
dffc935
6a5e00c
b431b19
ed16520
bdb349b
b263e52
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -45,6 +45,12 @@ The following steps assume the Calico deployment is installed on `tigera-operato | |
|
|
||
| 1. Install the $[prodname] custom resource definitions. | ||
|
|
||
| :::note[CRD mode] | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we're maintaining both modes, then a more user-friendly path might look like this: Prerequisite: You know what CRD mode you've got (Is there a simple check?)
TAB 1 / TAB 2 code code code |
||
|
|
||
| These steps apply to clusters using the aggregation API server, the default for existing and upgraded $[prodname] clusters. Upgrading does not change your cluster's CRD mode. If your cluster uses native v3 CRDs, apply `v3_projectcalico_org.yaml` in place of `operator-crds.yaml` below. See [Native v3 CRDs](../../../../operations/native-v3-crds.mdx). | ||
|
|
||
| ::: | ||
|
|
||
| ```bash | ||
| kubectl apply --server-side --force-conflicts -f $[filesUrl]/manifests/operator-crds.yaml | ||
| kubectl create -f $[filesUrl]/manifests/prometheus-operator-crds.yaml | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,18 +1,12 @@ | ||
| --- | ||
| description: Switch Calico Enterprise to native projectcalico.org/v3 CRDs so resources are stored directly as CRDs without the aggregated API server component. | ||
| description: New installs of Calico Enterprise use native projectcalico.org/v3 CRDs by default. This page explains how native v3 CRD mode works and how to install with the aggregation API server instead. | ||
| --- | ||
|
|
||
| # Enable native v3 CRDs | ||
|
|
||
| :::note | ||
|
|
||
| This feature is tech preview. Tech preview features may be subject to significant changes before they become GA. | ||
|
|
||
| ::: | ||
| # Native v3 CRDs | ||
|
|
||
| ## Big picture | ||
|
|
||
| Enable native `projectcalico.org/v3` CRDs so that Calico resources are backed directly by CRDs, eliminating the need for the Calico aggregation API server. | ||
| New installs of $[prodname] use native `projectcalico.org/v3` CRDs by default: Calico resources are backed directly by CRDs, eliminating the need for the Calico aggregation API server. This page explains how the mode works and how to install with the aggregation API server instead. | ||
|
|
||
| ## Value | ||
|
|
||
|
|
@@ -36,12 +30,12 @@ When using native `projectcalico.org/v3` CRDs: | |
|
|
||
| ### Validation and defaulting | ||
|
|
||
| When using native `projectcalico.org/v3` CRDs, resource validation and defaulting are handled by native CRD validation and defaulting, as well as ValidatingAdmissionPolicies and MutatingAdmissionPolicies. $[prodname] uses [MutatingAdmissionPolicies](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/) for defaulting, which require the beta `admissionregistration.k8s.io/v1beta1` API. That API is available in **Kubernetes 1.34 and later**. On Kubernetes 1.33 and earlier, MutatingAdmissionPolicy is only available as an alpha API (`v1alpha1`), which $[prodname] does not support. On Kubernetes 1.34 and 1.35, you must enable the `MutatingAdmissionPolicy` feature gate on your Kubernetes API server before using native `projectcalico.org/v3` CRDs, as it is not enabled by default. On Kubernetes 1.36 and later, the feature is GA and enabled by default. | ||
| When using native `projectcalico.org/v3` CRDs, resource validation and defaulting are handled by native CRD validation and defaulting, as well as ValidatingAdmissionPolicies and MutatingAdmissionPolicies. $[prodname] uses [MutatingAdmissionPolicies](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/) for defaulting, which require **Kubernetes 1.32 or later**. On clusters where the `MutatingAdmissionPolicy` API is not enabled by default, you must enable the `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) on the Kubernetes API server. | ||
|
|
||
| ## Before you begin | ||
|
|
||
| - A Kubernetes cluster **without** $[prodname] installed, or a cluster where you are performing a fresh install. To migrate an existing cluster from API server mode, see [Migrate from API server to native CRDs](crd-migration.mdx). | ||
| - **Kubernetes 1.34 or later.** $[prodname] uses the beta `admissionregistration.k8s.io/v1beta1` MutatingAdmissionPolicy API for defaulting, which is not available on Kubernetes 1.33 and earlier (where MutatingAdmissionPolicy is alpha only). On Kubernetes 1.34 and 1.35, the `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) must be enabled on the API server, as it is not enabled by default. On Kubernetes 1.36 and later, the feature is GA and enabled by default. | ||
| - **Kubernetes 1.32 or later.** $[prodname] uses the `MutatingAdmissionPolicy` API for defaulting. On clusters where it is not enabled by default, enable the `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) on the API server. | ||
|
|
||
| import Tabs from '@theme/Tabs'; | ||
| import TabItem from '@theme/TabItem'; | ||
|
|
@@ -50,76 +44,10 @@ import TabItem from '@theme/TabItem'; | |
|
|
||
| ### Install $[prodname] with native `projectcalico.org/v3` CRDs | ||
|
|
||
| Select the method below based on your preferred installation method. | ||
|
|
||
| <Tabs> | ||
| <TabItem label="Helm install" value="Helm install-0"> | ||
|
|
||
| 1. Add the $[prodname] Helm repo: | ||
|
|
||
| ```bash | ||
| helm repo add projectcalico https://docs.tigera.io/calico/charts | ||
| ``` | ||
|
|
||
| 1. Create the `tigera-operator` namespace: | ||
|
|
||
| ```bash | ||
| kubectl create namespace tigera-operator | ||
| ``` | ||
|
|
||
| 1. Install the v3 CRD chart instead of the default v1 CRD chart: | ||
|
|
||
| ```bash | ||
| helm template calico-crds projectcalico/projectcalico.org.v3 --version $[releaseTitle] | kubectl apply --server-side -f - | ||
| ``` | ||
| On $[prodname], the standard install guides default to native `projectcalico.org/v3` CRD mode, so follow your preferred method to install: | ||
|
|
||
| :::note | ||
|
|
||
| This replaces the `crd.projectcalico.org.v1` chart used in the default installation. Do not install both CRD charts. | ||
|
|
||
| ::: | ||
|
|
||
| 1. Install the Tigera Operator: | ||
|
|
||
| ```bash | ||
| helm install $[prodnamedash] projectcalico/tigera-operator --version $[releaseTitle] --namespace tigera-operator | ||
| ``` | ||
|
|
||
| If you have a `values.yaml` with custom configuration: | ||
|
|
||
| ```bash | ||
| helm install $[prodnamedash] projectcalico/tigera-operator --version $[releaseTitle] -f values.yaml --namespace tigera-operator | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| <TabItem label="Manifest install" value="Manifest install-1"> | ||
|
|
||
| 1. Install the v3 CRDs: | ||
|
|
||
| ```bash | ||
| kubectl create -f $[manifestsUrl]/manifests/v3_projectcalico_org.yaml | ||
| ``` | ||
|
|
||
| :::note | ||
|
|
||
| This replaces the `v1_crd_projectcalico_org.yaml` manifest used in the default installation. Do not install both CRD manifests. | ||
|
|
||
| ::: | ||
|
|
||
| 1. Install the Tigera Operator: | ||
|
|
||
| ```bash | ||
| kubectl create -f $[manifestsUrl]/manifests/tigera-operator.yaml | ||
| ``` | ||
|
|
||
| 1. Install $[prodname] by creating the necessary custom resources: | ||
|
|
||
| ```bash | ||
| kubectl create -f $[manifestsUrl]/manifests/custom-resources.yaml | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| </Tabs> | ||
| - [Helm install](../getting-started/install-on-clusters/kubernetes/helm.mdx) installs the `projectcalico.org.v3` CRD chart. | ||
| - [Manifest install](../getting-started/install-on-clusters/kubernetes/quickstart.mdx) applies the `v3_projectcalico_org.yaml` CRD bundle. | ||
|
|
||
| After installing, complete the following steps: | ||
|
|
||
|
|
@@ -181,6 +109,34 @@ In API server mode, tier RBAC is enforced for all operations (create, update, de | |
|
|
||
| When using native `projectcalico.org/v3` CRDs, tier RBAC is enforced via the admission webhook for **create, update, and delete** operations. However, **GET, LIST, and WATCH** operations on tiered policies are **not enforced** because admission webhooks cannot intercept read operations. This is a known limitation. | ||
|
|
||
| ## Install in API server mode (v1 CRDs) | ||
|
|
||
| New installs default to v3 CRD mode. To install in the aggregation API-server mode instead, install the v1 CRDs before the operator decides the mode - the operator runs in API-server mode whenever the `crd.projectcalico.org/v1` CRDs are present. | ||
|
|
||
| <Tabs> | ||
| <TabItem label="Helm install" value="apiserver-helm"> | ||
|
|
||
| Pull and install the v1 CRD chart before the Tigera Operator: | ||
|
|
||
| ```bash | ||
| helm pull tigera-ee/crd.projectcalico.org.v1 --version $[releaseTitle]$[helmPre] | ||
| helm template calico-crds crd.projectcalico.org.v1-$[chart_version_name].tgz | kubectl apply --server-side -f - | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| <TabItem label="Manifest install" value="apiserver-manifest"> | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't support manifest installations in Calico Enterprise. Is there something special about this case?
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps this is just Tigera Operator and Helm installation types? |
||
|
|
||
| Apply the v1 CRDs before the Tigera Operator: | ||
|
|
||
| ```bash | ||
| kubectl create -f $[filesUrl]/manifests/operator-crds.yaml | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| </Tabs> | ||
|
|
||
| Then continue with your normal install. The aggregation API server is deployed automatically. | ||
|
|
||
| ## Known limitations | ||
|
|
||
| - **GET/LIST/WATCH tier RBAC not enforced** — Admission webhooks cannot intercept read operations, so tier-based RBAC for GET, LIST, and WATCH is not enforced when using native `projectcalico.org/v3` CRDs. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,6 +18,8 @@ This version of Calico Enterprise is based on [Calico Open Source $[openSourceVe | |
|
|
||
| ## New features and enhancements | ||
|
|
||
| - New installs default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Existing and upgraded clusters are unaffected. To install with the aggregation API server instead, apply the v1 CRDs before installing the operator. v3 CRD mode requires Kubernetes 1.32 or later. See [Native v3 CRDs](../operations/native-v3-crds.mdx). | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll do more closer to release, but I suspect it would be good to add more here about how this works for upgrades. Would also be good to mention if we're deprecating the api server. Is that still the plan? |
||
|
|
||
| ### Deprecated and removed features | ||
|
|
||
| ## Technology Preview features | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should consider removing the note here and on the other installation guides. If someone performs an installation and notices the missing api server, they'll find plenty of information in your v3 CRDs pages and the release notes.
One reason to keep the note, or something similar, is to make things easier for people who really need the api server. Is this a significant group? From what I read, there is little reason that anyone would prefer that option.
Best path is to present one option in the main guide and provide the less common alternative in another location.