Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,14 @@ The geeky details of what you get:

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](../../operations/native-v3-crds.mdx).

:::

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl create -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ In this guide, you install the Tigera Calico operator and custom resource defini
helm repo add tigera-ee https://downloads.tigera.io/ee/charts
helm repo update
helm pull tigera-ee/tigera-operator --version $[releaseTitle]$[helmPre]
helm pull tigera-ee/crd.projectcalico.org.v1 --version $[releaseTitle]$[helmPre]
helm pull tigera-ee/projectcalico.org.v3 --version $[releaseTitle]$[helmPre]
```

### Prepare the Installation Configuration
Expand Down Expand Up @@ -95,8 +95,14 @@ To install a standard $[prodname] cluster with Helm:

1. Install the necessary custom resource definitions.

:::note[What changed]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider removing the note here and on the other installation guides. If someone performs an installation and notices the missing api server, they'll find plenty of information in your v3 CRDs pages and the release notes.

One reason to keep the note, or something similar, is to make things easier for people who really need the api server. Is this a significant group? From what I read, there is little reason that anyone would prefer that option.

Best path is to present one option in the main guide and provide the less common alternative in another location.


New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, pull and install the `tigera-ee/crd.projectcalico.org.v1` chart in place of `projectcalico.org.v3`. See [Native v3 CRDs](../../../operations/native-v3-crds.mdx).

:::

```bash
helm template calico-crds crd.projectcalico.org.v1-$[chart_version_name].tgz | kubectl apply --server-side -f -
helm template calico-crds projectcalico.org.v3-$[chart_version_name].tgz | kubectl apply --server-side -f -
```

1. Install the Tigera Operator using the Helm 3 chart:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,14 @@ A Linux host that meets the following requirements.

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](../../../operations/native-v3-crds.mdx).

:::

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl create -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,14 @@ The geeky details of what you get:

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](../../operations/native-v3-crds.mdx).

:::

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl create -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,14 @@ The geeky details of what you get:

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](../../operations/native-v3-crds.mdx).

:::

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl create -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,14 @@ The following steps will outline the installation of $[prodname] networking on t

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](../../../operations/native-v3-crds.mdx).

:::

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl create -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@ The following steps assume the Calico deployment is installed on `tigera-operato

1. Install the $[prodname] custom resource definitions.

:::note[CRD mode]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're maintaining both modes, then a more user-friendly path might look like this:

Prerequisite: You know what CRD mode you've got (Is there a simple check?)

  1. Install the custom resource definitions for your cluster's mode:

TAB 1 / TAB 2

code code code


These steps apply to clusters using the aggregation API server, the default for existing and upgraded $[prodname] clusters. Upgrading does not change your cluster's CRD mode. If your cluster uses native v3 CRDs, apply `v3_projectcalico_org.yaml` in place of `operator-crds.yaml` below. See [Native v3 CRDs](../../../../operations/native-v3-crds.mdx).

:::

```bash
kubectl apply --server-side --force-conflicts -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/prometheus-operator-crds.yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,12 @@ These steps differ based on your cluster type. If you are unsure of your cluster

1. Install the $[prodname] custom resource definitions.

:::note[CRD mode]

These steps apply to clusters using the aggregation API server, the default for existing and upgraded $[prodname] clusters. Upgrading does not change your cluster's CRD mode. If your cluster uses native v3 CRDs, apply `v3_projectcalico_org.yaml` in place of `operator-crds.yaml` below. See [Native v3 CRDs](../../../../operations/native-v3-crds.mdx).

:::

```bash
kubectl apply --server-side --force-conflicts -f $[filesUrl]/manifests/operator-crds.yaml
kubectl apply --server-side --force-conflicts -f $[filesUrl]/manifests/prometheus-operator-crds.yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,14 @@ In a new terminal, install the $[prodname] CNI.

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](../operations/native-v3-crds.mdx).

:::

```bash
kubectl apply --server-side -f $[filesUrl]/manifests/operator-crds.yaml
kubectl apply --server-side -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl apply -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
6 changes: 0 additions & 6 deletions calico-enterprise/operations/crd-migration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,6 @@ description: Migrate Calico Enterprise resources from the aggregated API server

# Migrate from API server to native CRDs

:::note

This feature is tech preview. Tech preview features may be subject to significant changes before they become GA.

:::

## Big picture

Automatically migrate $[prodname] resources from the aggregated API server's `crd.projectcalico.org/v1` backing storage to native `projectcalico.org/v3` CRDs, allowing you to remove the API server component.
Expand Down
116 changes: 36 additions & 80 deletions calico-enterprise/operations/native-v3-crds.mdx
Original file line number Diff line number Diff line change
@@ -1,18 +1,12 @@
---
description: Switch Calico Enterprise to native projectcalico.org/v3 CRDs so resources are stored directly as CRDs without the aggregated API server component.
description: New installs of Calico Enterprise use native projectcalico.org/v3 CRDs by default. This page explains how native v3 CRD mode works and how to install with the aggregation API server instead.
---

# Enable native v3 CRDs

:::note

This feature is tech preview. Tech preview features may be subject to significant changes before they become GA.

:::
# Native v3 CRDs

## Big picture

Enable native `projectcalico.org/v3` CRDs so that Calico resources are backed directly by CRDs, eliminating the need for the Calico aggregation API server.
New installs of $[prodname] use native `projectcalico.org/v3` CRDs by default: Calico resources are backed directly by CRDs, eliminating the need for the Calico aggregation API server. This page explains how the mode works and how to install with the aggregation API server instead.

## Value

Expand All @@ -36,12 +30,12 @@ When using native `projectcalico.org/v3` CRDs:

### Validation and defaulting

When using native `projectcalico.org/v3` CRDs, resource validation and defaulting are handled by native CRD validation and defaulting, as well as ValidatingAdmissionPolicies and MutatingAdmissionPolicies. $[prodname] uses [MutatingAdmissionPolicies](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/) for defaulting, which require the beta `admissionregistration.k8s.io/v1beta1` API. That API is available in **Kubernetes 1.34 and later**. On Kubernetes 1.33 and earlier, MutatingAdmissionPolicy is only available as an alpha API (`v1alpha1`), which $[prodname] does not support. On Kubernetes 1.34 and 1.35, you must enable the `MutatingAdmissionPolicy` feature gate on your Kubernetes API server before using native `projectcalico.org/v3` CRDs, as it is not enabled by default. On Kubernetes 1.36 and later, the feature is GA and enabled by default.
When using native `projectcalico.org/v3` CRDs, resource validation and defaulting are handled by native CRD validation and defaulting, as well as ValidatingAdmissionPolicies and MutatingAdmissionPolicies. $[prodname] uses [MutatingAdmissionPolicies](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/) for defaulting, which require **Kubernetes 1.32 or later**. On clusters where the `MutatingAdmissionPolicy` API is not enabled by default, you must enable the `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) on the Kubernetes API server.

## Before you begin

- A Kubernetes cluster **without** $[prodname] installed, or a cluster where you are performing a fresh install. To migrate an existing cluster from API server mode, see [Migrate from API server to native CRDs](crd-migration.mdx).
- **Kubernetes 1.34 or later.** $[prodname] uses the beta `admissionregistration.k8s.io/v1beta1` MutatingAdmissionPolicy API for defaulting, which is not available on Kubernetes 1.33 and earlier (where MutatingAdmissionPolicy is alpha only). On Kubernetes 1.34 and 1.35, the `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) must be enabled on the API server, as it is not enabled by default. On Kubernetes 1.36 and later, the feature is GA and enabled by default.
- **Kubernetes 1.32 or later.** $[prodname] uses the `MutatingAdmissionPolicy` API for defaulting. On clusters where it is not enabled by default, enable the `MutatingAdmissionPolicy` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) on the API server.

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Expand All @@ -50,76 +44,10 @@ import TabItem from '@theme/TabItem';

### Install $[prodname] with native `projectcalico.org/v3` CRDs

Select the method below based on your preferred installation method.

<Tabs>
<TabItem label="Helm install" value="Helm install-0">

1. Add the $[prodname] Helm repo:

```bash
helm repo add projectcalico https://docs.tigera.io/calico/charts
```

1. Create the `tigera-operator` namespace:

```bash
kubectl create namespace tigera-operator
```

1. Install the v3 CRD chart instead of the default v1 CRD chart:

```bash
helm template calico-crds projectcalico/projectcalico.org.v3 --version $[releaseTitle] | kubectl apply --server-side -f -
```
On $[prodname], the standard install guides default to native `projectcalico.org/v3` CRD mode, so follow your preferred method to install:

:::note

This replaces the `crd.projectcalico.org.v1` chart used in the default installation. Do not install both CRD charts.

:::

1. Install the Tigera Operator:

```bash
helm install $[prodnamedash] projectcalico/tigera-operator --version $[releaseTitle] --namespace tigera-operator
```

If you have a `values.yaml` with custom configuration:

```bash
helm install $[prodnamedash] projectcalico/tigera-operator --version $[releaseTitle] -f values.yaml --namespace tigera-operator
```

</TabItem>
<TabItem label="Manifest install" value="Manifest install-1">

1. Install the v3 CRDs:

```bash
kubectl create -f $[manifestsUrl]/manifests/v3_projectcalico_org.yaml
```

:::note

This replaces the `v1_crd_projectcalico_org.yaml` manifest used in the default installation. Do not install both CRD manifests.

:::

1. Install the Tigera Operator:

```bash
kubectl create -f $[manifestsUrl]/manifests/tigera-operator.yaml
```

1. Install $[prodname] by creating the necessary custom resources:

```bash
kubectl create -f $[manifestsUrl]/manifests/custom-resources.yaml
```

</TabItem>
</Tabs>
- [Helm install](../getting-started/install-on-clusters/kubernetes/helm.mdx) installs the `projectcalico.org.v3` CRD chart.
- [Manifest install](../getting-started/install-on-clusters/kubernetes/quickstart.mdx) applies the `v3_projectcalico_org.yaml` CRD bundle.

After installing, complete the following steps:

Expand Down Expand Up @@ -181,6 +109,34 @@ In API server mode, tier RBAC is enforced for all operations (create, update, de

When using native `projectcalico.org/v3` CRDs, tier RBAC is enforced via the admission webhook for **create, update, and delete** operations. However, **GET, LIST, and WATCH** operations on tiered policies are **not enforced** because admission webhooks cannot intercept read operations. This is a known limitation.

## Install in API server mode (v1 CRDs)

New installs default to v3 CRD mode. To install in the aggregation API-server mode instead, install the v1 CRDs before the operator decides the mode - the operator runs in API-server mode whenever the `crd.projectcalico.org/v1` CRDs are present.

<Tabs>
<TabItem label="Helm install" value="apiserver-helm">

Pull and install the v1 CRD chart before the Tigera Operator:

```bash
helm pull tigera-ee/crd.projectcalico.org.v1 --version $[releaseTitle]$[helmPre]
helm template calico-crds crd.projectcalico.org.v1-$[chart_version_name].tgz | kubectl apply --server-side -f -
```

</TabItem>
<TabItem label="Manifest install" value="apiserver-manifest">

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't support manifest installations in Calico Enterprise. Is there something special about this case?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this is just Tigera Operator and Helm installation types?


Apply the v1 CRDs before the Tigera Operator:

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
```

</TabItem>
</Tabs>

Then continue with your normal install. The aggregation API server is deployed automatically.

## Known limitations

- **GET/LIST/WATCH tier RBAC not enforced** — Admission webhooks cannot intercept read operations, so tier-based RBAC for GET, LIST, and WATCH is not enforced when using native `projectcalico.org/v3` CRDs.
8 changes: 7 additions & 1 deletion calico-enterprise/operations/nftables.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,14 @@ If you're using Kubernetes 1.33 or later, you can create a cluster with kubeadm

1. Install the Tigera Operator and custom resource definitions.

:::note[What changed]

New installs now default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Requires Kubernetes 1.32+. To install with the aggregation API server instead, apply `operator-crds.yaml` in place of `v3_projectcalico_org.yaml`. See [Native v3 CRDs](./native-v3-crds.mdx).

:::

```bash
kubectl create -f $[filesUrl]/manifests/operator-crds.yaml
kubectl create -f $[filesUrl]/manifests/v3_projectcalico_org.yaml
kubectl create -f $[filesUrl]/manifests/tigera-operator.yaml
```

Expand Down
2 changes: 2 additions & 0 deletions calico-enterprise/release-notes/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ This version of Calico Enterprise is based on [Calico Open Source $[openSourceVe

## New features and enhancements

- New installs default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. Existing and upgraded clusters are unaffected. To install with the aggregation API server instead, apply the v1 CRDs before installing the operator. v3 CRD mode requires Kubernetes 1.32 or later. See [Native v3 CRDs](../operations/native-v3-crds.mdx).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do more closer to release, but I suspect it would be good to add more here about how this works for upgrades.

Would also be good to mention if we're deprecating the api server. Is that still the plan?


### Deprecated and removed features

## Technology Preview features
Expand Down
12 changes: 3 additions & 9 deletions calico/getting-started/kubernetes/helm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,13 @@ For more information about configurable options via `values.yaml` please see [He

1. Install the necessary custom resource definitions.

```bash
helm template calico-crds projectcalico/crd.projectcalico.org.v1 --version $[releaseTitle] | kubectl apply --server-side -f -
```

:::tip

To install with [native v3 CRDs](../../operations/native-v3-crds.mdx) (tech preview) instead, use the v3 CRD chart:

```bash
helm template calico-crds projectcalico/projectcalico.org.v3 --version $[releaseTitle] | kubectl apply --server-side -f -
```

Native v3 CRDs eliminate the need for the aggregation API server and allows `kubectl` to manage `projectcalico.org/v3` resources directly.
:::note[What changed]

New installs default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. This requires Kubernetes 1.32 or later. To install with the aggregation API server instead, see [Install in API server mode](../../operations/install-apiserver.mdx#install-in-api-server-mode-v1-crds).

:::

Expand Down
15 changes: 10 additions & 5 deletions calico/getting-started/kubernetes/k3s/multi-node-install.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -94,19 +94,24 @@ curl -sfL https://get.k3s.io | K3S_URL=https://serverip:6443 K3S_TOKEN=mytoken s
<Tabs>
<TabItem label="Operator" value="Operator-0">

Install the $[prodname] operator and custom resource definitions.
Install the $[prodname] custom resource definitions.

```bash
kubectl create -f $[manifestsUrl]/manifests/v1_crd_projectcalico_org.yaml
kubectl create -f $[manifestsUrl]/manifests/tigera-operator.yaml
kubectl create -f $[manifestsUrl]/manifests/v3_projectcalico_org.yaml
```

:::note
:::note[What changed]

Due to the large size of the CRD bundle, `kubectl apply` might exceed request limits. Instead, use `kubectl create` or `kubectl replace`.
New installs default to v3 CRD mode: `projectcalico.org/v3` resources are served directly by CRDs, with no aggregation API server. This requires Kubernetes 1.32 or later. To install with the aggregation API server instead, see [Install in API server mode](../../../operations/install-apiserver.mdx#install-in-api-server-mode-v1-crds).

:::

Install the Tigera Operator.

```bash
kubectl create -f $[manifestsUrl]/manifests/tigera-operator.yaml
```

Install $[prodname] by creating the necessary custom resource. For more information on configuration options available in this manifest, see [the installation reference](../../../reference/installation/api.mdx).

```bash
Expand Down
Loading