Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
862 changes: 488 additions & 374 deletions content/openapi/oneks_v1.json

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,7 @@ A K8s Cluster includes:
| `name` | K8s Cluster name. |
| `description` | Optional human-readable description. |
| `kubernetes_version` | Kubernetes version used by the K8s Cluster. |
| `public_network` | OpenNebula public Virtual Network ID. |
| `private_network` | OpenNebula private Virtual Network ID. |
| `deployment` | OpenNebula placement used by the K8s Cluster, including the target OpenNebula Cluster and public and private Virtual Networks. |
| `spec` | Control-plane profile selection and input values. |
| `state` | Current K8s Cluster lifecycle state. |
| `control_plane` | Reference to the logical control-plane group. |
Expand Down
10 changes: 2 additions & 8 deletions content/platform_services/oneks/management/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,17 +61,14 @@ These options are used when OneKS runs Kubernetes commands from the Front-end Ho

## Readiness Check Configuration

The optional `:readiness` section enables the OneKS readiness check service. When it is enabled, users can validate the public and private Virtual Networks that will be used by a K8s Cluster before starting a deployment.
The optional `:readiness` section enables the OneKS readiness check service. When it is enabled, users can validate the full deployment placement that will be used by a K8s Cluster before starting a deployment.

On startup, OneKS verifies that the readiness appliance is available. If the appliance is not already imported, OneKS imports the configured marketplace appliance into the configured datastore, creating the OneKS readiness service VM template and its backing image as part of the import.
The readiness check uses the OneKS appliance dependency defined by the selected control-plane family. It deploys a temporary probe VM from the same appliance and image datastore resolution path used by the real K8s Cluster provisioning flow.

Comment out the full `:readiness` section to disable the readiness check service.

| Attribute | Default | Description |
|-----------------------------------|-----------------------------------------|-------------|
| `:readiness` / `:appliance_name` | `OneKS Readiness Service` | Marketplace appliance name used when importing the readiness probe template if it is missing. |
| `:readiness` / `:appliance_id` | `97383e01-6150-4a1f-8830-fc5d745056e0` | Marketplace appliance UUID used to find or import the readiness probe VM template. By default, this value points to an Alpine 3.20 appliance from the OpenNebula Marketplace. |
| `:readiness` / `:appliance_ds` | `1` | Datastore ID where the readiness appliance image is imported. |
| `:readiness` / `:external_url` | `https://get.rke2.io` | Public URL used by the probe VM to validate DNS resolution and outbound internet access. |
| `:readiness` / `:timeout` | `60` | Maximum time, in seconds, to wait for the probe VM and each readiness check step. |

Expand Down Expand Up @@ -203,9 +200,6 @@ OneKS relies on several OpenNebula services and network endpoints:
################################################################################

:readiness:
:appliance_name: 'OneKS Readiness Service'
:appliance_id: 97383e01-6150-4a1f-8830-fc5d745056e0
:appliance_ds: 1
:external_url: 'https://get.rke2.io'
:timeout: 60

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Before creating a K8s Cluster, verify that:
* **OneKS Service**: The OneKS service is configured and running.
* **OneGate Service**: OneGate is configured and reachable.
* **Transparent Proxy**: `tproxy` is configured for the required OneGate and OpenNebula XML-RPC ports.
* **OpenNebula Cluster**: The target OpenNebula Cluster ID is known.
* **Networks**: The OpenNebula public and private Virtual Network IDs are known.
* **Profiles**: The required family and flavour are available.
* **Kubernetes Version**: The target Kubernetes version is supported by the selected family.
Expand All @@ -41,6 +42,8 @@ Before creating a K8s Cluster, verify that:
For more detailed information about the basic configuration and requirements needed to create K8s Clusters, refer to the [Basic Configuration Guide]({{% relref "platform_services/oneks/getting_started/basic_configuration" %}}).
{{< /alert >}}

Deployment options let you choose where each K8s Cluster is placed and which OpenNebula resources it uses. This makes it possible to provision independent K8s Clusters in different OpenNebula Clusters from the same OneKS service. For each deployment, you can select the target OpenNebula Cluster and the public and private Virtual Networks that will be used by the K8s Cluster. The selected networks must be available in the target OpenNebula Cluster. The OneKS appliance required by the selected control-plane family must also be available in an image datastore that can be used from that placement.

{{< tabpane text=true right=false >}}
{{% tab header="**Interfaces**:" disabled=true /%}}

Expand All @@ -54,6 +57,7 @@ alt="OneKS create Cluster step 1" align="center" width="90%" mb="20px" >}}
The wizard guides you through the required configuration steps:

* **General**: K8s Cluster name and optional description.
* **Select a OpenNebula Cluster ID**: The OpenNebula Cluster used for K8s Cluster placement.
* **Select a Public Virtual Network**: Public network used for external connectivity and bootstrap paths.
* **Select a Private Virtual Network**: Private network used for internal K8s Cluster communication.
* **Kubernetes Version**: Kubernetes version to deploy.
Expand All @@ -79,6 +83,14 @@ ID USER GROUP NAME CLUSTERS BRIDGE STATE
0 oneadmin oneadmin public 0 br1 rdy
```

Also identify the OpenNebula Cluster where the K8s Cluster must be deployed:

```shell
$ onecluster list
ID NAME HOSTS VNETS DATASTORES
0 default 2 2 3
```

Launch the interactive K8s Cluster creation command. You can add `--wait` to subscribe to the deployment logs and keep the command attached until the operation completes or reaches a terminal state:

```shell
Expand All @@ -88,12 +100,43 @@ oneks create cluster --wait
The interactive CLI flow asks for:

* **K8s Cluster Name**: The name used to identify the OneKS K8s Cluster.
* **Kubernetes Version**: The Kubernetes version to deploy.
* **K8s Cluster Flavour**: The control-plane flavour, such as `standalone` or `ha`.
* **OpenNebula Cluster ID**: The OpenNebula Cluster used for K8s Cluster placement.
* **Public Network ID**: The OpenNebula public Virtual Network used by the K8s Cluster.
* **Private Network ID**: The OpenNebula private Virtual Network used by the K8s Cluster.
* **Kubernetes Version**: The Kubernetes version to deploy.
* **K8s Cluster Flavour**: The control-plane flavour, such as `standalone` or `ha`.

```shell
$ oneks create cluster
> Cluster name: example-oneks-cluster

DEPLOYMENT PLACEMENT
> OpenNebula cluster ID: 0
> Public network ID: 0
> Private network ID: 1

ONEKS SPEC
> Select a Kubernetes version for the Cluster:
0: v1.33.7
1: v1.34.2

{{< image path="/images/oneks/light/k8s_cluster_create_cli.png" alt="K8s Cluster create CLI menu" align="center" width="60%" mb="20px" >}}
Select an option by number: 1

> Select a flavour for the Cluster:
0: Single-Node Control Plane
Single Control Plane node deployment.
Suitable for development, evaluation, and non-critical workloads.
1 node | 2 CPU | 2 vCPU | 4 GB RAM | 16 GB Storage

1: Highly Available Control Plane
Three-node Control Plane deployment with built-in redundancy.
Suitable for production and other environments that require higher availability.
3 nodes | 2 CPU | 2 vCPU | 4 GB RAM | 16 GB Storage

Select an option by number: 1

ID: 10
```

You can also create a K8s Cluster from a JSON specification:

Expand All @@ -107,9 +150,20 @@ Example `spec.json`:
{
"name": "prod-cluster",
"description": "Production Kubernetes cluster",
"kubernetes_version": "v1.32.9",
"public_network": 12,
"private_network": 34,
"kubernetes_version": "v1.34.2",
"deployment": {
"cluster": {
"id": 0
},
"networks": {
"public": {
"id": 12
},
"private": {
"id": 34
}
}
},
"spec": {
"flavour": "ha"
}
Expand All @@ -129,9 +183,20 @@ curl -u "$(cat /var/lib/one/.one/one_auth)" \
-d '{
"name": "prod-cluster",
"description": "Production Kubernetes cluster",
"kubernetes_version": "v1.32.9",
"public_network": 0,
"private_network": 1,
"kubernetes_version": "v1.34.2",
"deployment": {
"cluster": {
"id": 0
},
"networks": {
"public": {
"id": 0
},
"private": {
"id": 1
}
}
},
"spec": {
"flavour": "ha"
}
Expand Down Expand Up @@ -236,7 +301,29 @@ The command starts an interactive creation flow. You will be asked to provide:
* **Flavour**: The worker node size profile to use.
* **Count**: The number of worker nodes to create.

{{< image path="/images/oneks/light/oneks_create_nodegroup_cli.png" alt="OneKS create nodegroup CLI" align="center" width="60%" mb="20px" >}}
```shell
$ oneks create group --cluster-id 10
> Nodegroup name: example-oneks-group
> Select a flavour for the Nodegroup:
0: Small Worker Nodes
Small worker node profile for lightweight workloads.
2 CPU | 2 vCPU | 4 GB RAM | 16 GB Storage

1: Medium Worker Nodes
Medium worker node profile for balanced workloads.
4 CPU | 4 vCPU | 8 GB RAM | 32 GB Storage

2: Large Worker Nodes
Large worker node profile for demanding workloads.
8 CPU | 8 vCPU | 16 GB RAM | 64 GB Storage

Select an option by number: 2
There are some parameters that require user input.
* (count) Number of Worker nodes [type: number]
Enter a number: 1

ID: 11
```

Scale a node group by specifying its ID and the desired number of worker nodes:

Expand All @@ -247,18 +334,18 @@ oneks scale nodegroup <nodegroup_id> --target <worker_count>
Example:

```shell
oneks scale nodegroup 7 --target 3
oneks scale nodegroup 11 --target 3
```

Validate the Kubernetes node list:

```shell
$ KUBECONFIG=./kubeconfig kubectl get nodes
NAME STATUS ROLES AGE VERSION
test-cluster-control-plane Ready control-plane 9m v1.31.4
test-cluster-worker-1 Ready <none> 2m v1.31.4
test-cluster-worker-2 Ready <none> 2m v1.31.4
test-cluster-worker-3 Ready <none> 2m v1.31.4
test-cluster-control-plane Ready control-plane 9m v1.34.2
test-cluster-worker-1 Ready <none> 2m v1.34.2
test-cluster-worker-2 Ready <none> 2m v1.34.2
test-cluster-worker-3 Ready <none> 2m v1.34.2
```

{{% /tab %}}
Expand Down Expand Up @@ -296,10 +383,10 @@ Validate the Kubernetes node list:
```shell
$ KUBECONFIG=./kubeconfig kubectl get nodes
NAME STATUS ROLES AGE VERSION
test-cluster-control-plane Ready control-plane 9m v1.31.4
test-cluster-worker-1 Ready <none> 2m v1.31.4
test-cluster-worker-2 Ready <none> 2m v1.31.4
test-cluster-worker-3 Ready <none> 2m v1.31.4
test-cluster-control-plane Ready control-plane 9m v1.34.2
test-cluster-worker-1 Ready <none> 2m v1.34.2
test-cluster-worker-2 Ready <none> 2m v1.34.2
test-cluster-worker-3 Ready <none> 2m v1.34.2
```

For further details about the API, see the [OneKS REST API Reference]({{% relref "platform_services/oneks/references/oneks_api/" %}}).
Expand Down Expand Up @@ -346,7 +433,7 @@ oneks upgrade cluster <cluster_id> --k8s-version <version>
Example:

```shell
oneks upgrade cluster 42 --k8s-version v1.32.9
oneks upgrade cluster 42 --k8s-version v1.34.2
```

After the upgrade starts, inspect the K8s Cluster state:
Expand All @@ -370,7 +457,7 @@ curl -u "$(cat /var/lib/one/.one/one_auth)" \
-X POST http://<oneks-server>:10780/api/v1/clusters/<cluster_id>/upgrade \
-H "Content-Type: application/json" \
-d '{
"kubernetes_version": "v1.32.9"
"kubernetes_version": "v1.34.2"
}'
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -150,9 +150,9 @@ K8s Cluster provisioning can fail for different infrastructure or network-relate

### OneKS Readiness Check

If the OneKS readiness check service is enabled, use it to validate the public and private Virtual Networks used by a K8s Cluster. The check deploys a temporary lightweight probe VM and verifies the main requirements needed by OneKS provisioning, including OneGate access, internet connectivity, and private network paths. For configuration details, see the [Readiness Check Configuration]({{% relref "platform_services/oneks/management/configuration#readiness-check-configuration" %}}). For command syntax, see the [`oneks check` CLI reference]({{% relref "platform_services/oneks/references/oneks_cli#readiness-checks" %}}).
If the OneKS readiness check service is enabled, use it to validate the deployment placement used by a K8s Cluster. The check deploys a temporary probe VM from the same OneKS appliance used by provisioning and verifies the main requirements needed by OneKS, including the target OpenNebula Cluster, selected networks, appliance availability, OneGate access, internet connectivity, and private network paths. For configuration details, see the [Readiness Check Configuration]({{% relref "platform_services/oneks/management/configuration#readiness-check-configuration" %}}). For command syntax, see the [`oneks check` CLI reference]({{% relref "platform_services/oneks/references/oneks_cli#readiness-checks" %}}).

When a deployment fails, run the readiness check against the failed K8s Cluster. OneKS resolves the networks from the Cluster document:
When a deployment fails, run the readiness check against the failed K8s Cluster. OneKS resolves the deployment placement from the Cluster document:

```shell
oneks check cluster <cluster_id>
Expand All @@ -172,10 +172,11 @@ When all checks pass, the command reports each validated step:
[OK] All OneKS readiness checks passed
```

You can also run the check before creating a K8s Cluster by passing the networks explicitly:
You can also run the check before creating a K8s Cluster by passing the deployment placement explicitly:

```shell
oneks check --public-network <network_id> --private-network <network_id>
oneks check --opennebula-cluster <one_cluster_id> \
--public-network <network_id> --private-network <network_id>
```

If the readiness check fails, the tool reports the failing step and the environment problem that must be fixed. For example, if the probe VM cannot access OneGate:
Expand Down
4 changes: 2 additions & 2 deletions content/platform_services/oneks/references/oneks_cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Some builds may expose node groups through the lower-level `group` resource in C
* `oneks create nodegroup --cluster-id <cluster_id>`: Create a node group.
* `oneks recover cluster <cluster_id>`: Recover a K8s Cluster from selected failure states.
* `oneks recover nodegroup <nodegroup_id>`: Recover a node group from selected failure states.
* `oneks check cluster <cluster_id>`: Run the OneKS readiness check using the networks from an existing K8s Cluster.
* `oneks check cluster <cluster_id>`: Run the OneKS readiness check using the deployment placement from an existing K8s Cluster.
* `oneks delete cluster <cluster_id>`: Delete a K8s Cluster.
* `oneks delete nodegroup <nodegroup_id>`: Delete a node group.
* `oneks logs cluster <cluster_id>`: Show K8s Cluster logs.
Expand Down Expand Up @@ -98,7 +98,7 @@ Run OneKS readiness checks:

```shell
oneks check cluster 42
oneks check --public-network 105 --private-network 106
oneks check --opennebula-cluster 0 --public-network 105 --private-network 106
```

Inspect logs:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ The OpenNebula team is excited to announce the availability of the **OpenNebula

## OpenNebula Form

## OpenNebula Elastic Kubernetes Service

* Added multi-cluster deployment support in OneKS, allowing users to select the target OpenNebula cluster and deployment networks when creating Kubernetes Clusters.

## Packaging

## Features Backported to 7.2.x
Expand Down
Loading