diff --git a/docs/self-hosted/deploy/machine-images/aws-ami.mdx b/docs/self-hosted/deploy/machine-images/aws-ami.mdx index db537c42d..ad94fdf70 100644 --- a/docs/self-hosted/deploy/machine-images/aws-ami.mdx +++ b/docs/self-hosted/deploy/machine-images/aws-ami.mdx @@ -62,7 +62,7 @@ To configure SSL, and lock down the instance from the public internet, see the [ ### Executors -Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes). +Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)*. Executors support [auto-indexing](/code-navigation/auto-indexing) and [server-side batch changes](/batch-changes/server-side). diff --git a/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx b/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx index 9520d2b9a..80075727d 100644 --- a/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx +++ b/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx @@ -74,7 +74,7 @@ Find the URL of your Sourcegraph instance in the **Outputs** section of the AWS ### Executors -Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes). +Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)*. Executors support [auto-indexing](/code-navigation/auto-indexing) and [server-side batch changes](/batch-changes/server-side). diff --git a/docs/self-hosted/deploy/machine-images/gce.mdx b/docs/self-hosted/deploy/machine-images/gce.mdx index ce90f8542..008664765 100644 --- a/docs/self-hosted/deploy/machine-images/gce.mdx +++ b/docs/self-hosted/deploy/machine-images/gce.mdx @@ -114,7 +114,7 @@ $ sudo su sourcegraph ### Executors -Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes). +Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)*. Executors support [auto-indexing](/code-navigation/auto-indexing) and [server-side batch changes](/batch-changes/server-side). diff --git a/docs/self-hosted/executors/deploy-executors-dind.mdx b/docs/self-hosted/executors/deploy-executors-dind.mdx index de42b8d1c..e2fabbcc5 100644 --- a/docs/self-hosted/executors/deploy-executors-dind.mdx +++ b/docs/self-hosted/executors/deploy-executors-dind.mdx @@ -1,6 +1,12 @@ # Deploying Sourcegraph executors on Kubernetes (docker-in-docker) -This feature is in Beta stage. + + Docker-in-Docker Kubernetes executors are in beta and are not recommended for production use. + This deployment mode requires privileged pod access and does not use Firecracker isolation. + For production workloads, deploy using + [Terraform](/self-hosted/executors/deploy-executors-terraform) or the + [Linux binary](/self-hosted/executors/deploy-executors-binary) instead. + [Kubernetes manifests](https://github.com/sourcegraph/deploy-sourcegraph-k8s) are provided to deploy Sourcegraph Executors on a running Kubernetes cluster. If you are deploying Sourcegraph with helm, charts are available [here](https://github.com/sourcegraph/deploy-sourcegraph-helm). diff --git a/docs/self-hosted/executors/deploy-executors-kubernetes.mdx b/docs/self-hosted/executors/deploy-executors-kubernetes.mdx index fbd2c221c..8f735c084 100644 --- a/docs/self-hosted/executors/deploy-executors-kubernetes.mdx +++ b/docs/self-hosted/executors/deploy-executors-kubernetes.mdx @@ -1,7 +1,9 @@ # Deploying Sourcegraph Executors natively on Kubernetes - - This feature is in beta and is available in Sourcegraph 5.1.0 and later. + + Native Kubernetes executors are in beta. For production workloads, consider deploying using + [Terraform](/self-hosted/executors/deploy-executors-terraform) or the + [Linux binary](/self-hosted/executors/deploy-executors-binary) for better long-term support. The native Kubernetes Executors have a master Executor pod that schedules worker pods via the Kubernetes API. The master Executor pod manages the lifecycle of jobs, while the worker pods process the actual [batch change](/batch-changes/server-side) or [precise auto indexing](/code-navigation/auto-indexing) job specs. For more details, see [how it works](/admin/executors#native-kubernetes). @@ -65,6 +67,9 @@ Native Kubernetes Executors can be deployed via either the `sourcegraph-executor ### Steps 1. If you are deploying Executors for processing Batch Changes, set `batchChanges.nativeServerSideExecution` to `true` in your [site configuration](/admin/config/site-config). Only enable this when using Executors on Kubernetes. + + > **Note:** Batch Changes on native Kubernetes executors inherits the beta limitations of + > the native Kubernetes runtime. See the warning at the top of this page. 2. Configure the following environment variables on the Executor Deployment: 1. `EXECUTOR_FRONTEND_URL` should match the URL of your Sourcegraph instance diff --git a/docs/self-hosted/executors/deploy-executors-terraform.mdx b/docs/self-hosted/executors/deploy-executors-terraform.mdx index dbb2b6bbc..3e9df238e 100644 --- a/docs/self-hosted/executors/deploy-executors-terraform.mdx +++ b/docs/self-hosted/executors/deploy-executors-terraform.mdx @@ -1,8 +1,8 @@ # Deploying Sourcegraph executors using Terraform on AWS or GCP [Terraform modules](https://learn.hashicorp.com/tutorials/terraform/module-use?in=terraform/modules) are provided to -provision machines running executors on [AWS](https://sourcegraph.com/github.com/sourcegraph/terraform-aws-executors) -and [Google Cloud](https://sourcegraph.com/github.com/sourcegraph/terraform-google-executors). +provision machines running executors on [AWS](https://github.com/sourcegraph/terraform-aws-executors) +and [Google Cloud](https://github.com/sourcegraph/terraform-google-executors). ## Basic Definition @@ -50,14 +50,14 @@ module "executors" { See the Terraform Modules for additional configurations. -- [AWS](https://sourcegraph.com/github.com/sourcegraph/terraform-aws-executors/-/blob/modules/executors/variables.tf) -- [Google](https://sourcegraph.com/github.com/sourcegraph/terraform-google-executors/-/blob/modules/executors/variables.tf) +- [AWS](https://github.com/sourcegraph/terraform-aws-executors/blob/main/modules/executors/variables.tf) +- [Google](https://github.com/sourcegraph/terraform-google-executors/blob/main/modules/executors/variables.tf) ## Terraform Version Terraform modules `4.2.x` and above allow Terraform from `1.1.x` to `< 2.x` to be used. -If using a Terraform module `4.1.x` or below, use [tfenv](https://sourcegraph.com/github.com/tfutils/tfenv) to install Terraform +If using a Terraform module `4.1.x` or below, use [tfenv](https://github.com/tfutils/tfenv) to install Terraform 1.1+. ```shell @@ -158,16 +158,16 @@ All regions are supported. The following examples provision a single executor to pull from the `codeintel` queue. -- [AWS example](https://sourcegraph.com/github.com/sourcegraph/terraform-aws-executors/-/tree/examples/single-executor) -- [Google example](https://sourcegraph.com/github.com/sourcegraph/terraform-google-executors/-/tree/examples/single-executor) +- [AWS example](https://github.com/sourcegraph/terraform-aws-executors/tree/main/examples/single-executor) +- [Google example](https://github.com/sourcegraph/terraform-google-executors/tree/main/examples/single-executor) ### Multiple Executors The following examples provision two executors, one to pull from the `codeintel` queue and the other for the `batches` queue. -- [AWS example](https://sourcegraph.com/github.com/sourcegraph/terraform-aws-executors/-/tree/examples/multiple-executors) -- [Google example](https://sourcegraph.com/github.com/sourcegraph/terraform-google-executors/-/tree/examples/multiple-executors) +- [AWS example](https://github.com/sourcegraph/terraform-aws-executors/tree/main/examples/multiple-executors) +- [Google example](https://github.com/sourcegraph/terraform-google-executors/tree/main/examples/multiple-executors) ### Step-by-step Guide @@ -192,7 +192,7 @@ The following is a step-by-step guide on provisioning a single `codeintel` execu - `"codeIntelAutoIndexing.enabled": true` - _This is only for `codeintel` executors._ 5. Download - the [example files](https://sourcegraph.com/github.com/sourcegraph/terraform-google-executors/-/blob/examples/single-executor) + the [example files](https://github.com/sourcegraph/terraform-google-executors/tree/main/examples/single-executor) 6. Change the following in `providers.tf` - `project` to the GCP project to provision the executor in - `region` to the GCP region to provision the executor in @@ -291,11 +291,11 @@ you@sourcegraph-executor-h0rv:~$ curl Auto-scaling of executor instances can help to increase concurrency of jobs, without paying for unused resources. With auto-scaling, you can scale down to 0 instances when no workload exist and scale up as far as you like and your cloud provider can support. Auto-scaling needs to be configured separately. -Auto-scaling makes use of the auto-scaling capabilities of the respective cloud provider (**AutoScalingGroups** on AWS and **Instance Groups** on GCP). Sourcegraph's `worker` service publishes a scaling metric (that is, the number of jobs in queue) to the cloud providers. Then, based on that reported value, the auto-scalers add and remove compute resources to match the required amount of compute. The autoscaler will attempt to hold 1 instance running per each [`executor_jobs_per_instance_scaling`](https://sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/terraform-.*-executors%24+variable+%22executor_jobs_per_instance_scaling%22&patternType=literal) items in queue. +Auto-scaling makes use of the auto-scaling capabilities of the respective cloud provider (**AutoScalingGroups** on AWS and **Instance Groups** on GCP). Sourcegraph's `worker` service publishes a scaling metric (that is, the number of jobs in queue) to the cloud providers. Then, based on that reported value, the auto-scalers add and remove compute resources to match the required amount of compute. The autoscaler will attempt to hold 1 instance running per each `executor_jobs_per_instance_scaling` items in queue. -For example, if `executor_jobs_per_instance_scaling` is set to `20` and the queue size is currently `400`, then `20`instances would be determined as required to handle the load. You might want to tweak this number based on the [machine type](https://sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/terraform-.*-executors%24+variable+%22machine_type%22+-f:docker-mirror&patternType=literal), [concurrency per machine](https://sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/terraform-.*-executors%24+variable+%22maximum_num_jobs%22&patternType=literal) and desired processing speed. +For example, if `executor_jobs_per_instance_scaling` is set to `20` and the queue size is currently `400`, then `20` instances would be determined as required to handle the load. You might want to tweak this number based on the `machine_type`, `maximum_num_jobs` (concurrency per machine), and desired processing speed. See the [AWS](https://github.com/sourcegraph/terraform-aws-executors/blob/main/modules/executors/variables.tf) and [Google](https://github.com/sourcegraph/terraform-google-executors/blob/main/modules/executors/variables.tf) variable definitions for details. -With the Terraform variables [`executor_min_replicas`](https://sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/terraform-.*-executors%24+variable+%22executor_min_replicas%22&patternType=literal) and [`executor_max_replicas`](https://sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/terraform-.*-executors%24+variable+%22executor_max_replicas%22&patternType=literal) in the Terraform modules linked to above, you can configure the minimum and maximum number of compute machines to be run at a given time. +With the Terraform variables `executor_min_replicas` and `executor_max_replicas` in the Terraform modules linked to above, you can configure the minimum and maximum number of compute machines to be run at a given time. For auto-scaling to work, two things must be true: @@ -305,7 +305,7 @@ For auto-scaling to work, two things must be true: For the latter to work, the Sourcegraph instance needs to be configured with the correct credentials that allow it to access the cloud provider. -The `credentials` submodule in both the [AWS](https://sourcegraph.com/github.com/sourcegraph/terraform-aws-executors/-/tree/modules/credentials) and [Google](https://sourcegraph.com/github.com/sourcegraph/terraform-google-executors/-/tree/modules/credentials) executor modules exists for that purpose. When used, the `credentials` module sets up the credentials on the cloud provider and returns them in the Terraform outputs. +The `credentials` submodule in both the [AWS](https://github.com/sourcegraph/terraform-aws-executors/tree/main/modules/credentials) and [Google](https://github.com/sourcegraph/terraform-google-executors/tree/main/modules/credentials) executor modules exists for that purpose. When used, the `credentials` module sets up the credentials on the cloud provider and returns them in the Terraform outputs. Here's an example of how one would configure auto-scaling. diff --git a/docs/self-hosted/executors/deploy-executors.mdx b/docs/self-hosted/executors/deploy-executors.mdx index e2e19a7c9..ccbb86b37 100644 --- a/docs/self-hosted/executors/deploy-executors.mdx +++ b/docs/self-hosted/executors/deploy-executors.mdx @@ -4,8 +4,8 @@ Executors can be deployed in a variety of manners. The supported deployment opti - [Linux Binary Service](/self-hosted/executors/deploy-executors-binary) ([Firecracker](./firecracker) compatible) - [Terraform on AWS or GCP](/self-hosted/executors/deploy-executors-terraform) ([Firecracker](./firecracker) compatible) -- [Native Kubernetes](/self-hosted/executors/deploy-executors-kubernetes) -- [Docker-in-Docker on Kubernetes](/self-hosted/executors/deploy-executors-dind) +- [Native Kubernetes](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)* +- [Docker-in-Docker on Kubernetes](/self-hosted/executors/deploy-executors-dind) *(Beta — not recommended for production)* - [Docker-Compose](/self-hosted/executors/deploy-executors-docker) See [deciding which executor deployment method to use ](../executors#deciding-which-executor-deployment-method-to-use) for more information on these different deployment options. @@ -110,13 +110,13 @@ Once the shared secret is set in Sourcegraph, you can start setting up executors title="Native Kubernetes" icon="installation" href="/self-hosted/executors/deploy-executors-kubernetes" - description="Run executors natively on kubernetes." + description="Run executors natively on kubernetes. (Beta — known reliability limitations, not recommended for production.)" /> **Note:** The flowchart above shows Kubernetes-based deployment options. Both Native Kubernetes +> and Docker-in-Docker Kubernetes executors are currently in beta with known production reliability +> limitations. For production workloads, Terraform or Linux binary deployments are recommended +> even when nested virtualization is not available. + ## How it works Executor instances are capable of being deployed in a variety of ways. Each runtime varies in how jobs are executed. @@ -122,7 +127,11 @@ Executor instances are capable of being deployed in a variety of ways. Each runt ### Native Kubernetes -> NOTE: This is an experimental feature. + + Native Kubernetes executors are in beta and are not recommended for production use. + See the [deployment guide](/self-hosted/executors/deploy-executors-kubernetes) for details + on known limitations. + ![Executors architecture - native kubernetes](https://storage.googleapis.com/sourcegraph-assets/executor_kubernetes_native_arch.png) @@ -142,7 +151,11 @@ Executor instances are capable of being deployed in a variety of ways. Each runt ### Docker-in-Docker Kubernetes -> NOTE: This is an experimental feature. + + Docker-in-Docker Kubernetes executors are in beta and are not recommended for production use. + See the [deployment guide](/self-hosted/executors/deploy-executors-dind) for details on known + limitations. + ![Executors architecture - docker in docker kubernetes](https://storage.googleapis.com/sourcegraph-assets/executor_kubernetes_dind_arch.png) diff --git a/docs/self-hosted/index.mdx b/docs/self-hosted/index.mdx index 1f66d8eda..7b56dff82 100644 --- a/docs/self-hosted/index.mdx +++ b/docs/self-hosted/index.mdx @@ -59,12 +59,12 @@ Get started running Sourcegraph on-prem. - [Executors overview](/self-hosted/executors/) - [Deploy executors](/self-hosted/executors/deploy-executors) -- [Kubernetes deployment](/self-hosted/executors/deploy-executors-kubernetes) +- [Kubernetes deployment](/self-hosted/executors/deploy-executors-kubernetes) *(Beta)* - [Terraform deployment](/self-hosted/executors/deploy-executors-terraform) - [Docker deployment](/self-hosted/executors/deploy-executors-docker) - [Binary deployment](/self-hosted/executors/deploy-executors-binary) - [Binary deployment (offline)](/self-hosted/executors/deploy-executors-binary-offline) -- [Docker-in-Docker](/self-hosted/executors/deploy-executors-dind) +- [Docker-in-Docker](/self-hosted/executors/deploy-executors-dind) *(Beta)* - [Firecracker](/self-hosted/executors/firecracker) - [Configuration](/self-hosted/executors/executors-config) - [Troubleshooting](/self-hosted/executors/executors-troubleshooting)