Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 47 additions & 14 deletions TEST.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Sourcegraph Helm Chart Test Guide

This is a temporary test guide before we have more thorough automated integration tests.
This is a temporary test guide before we have more thorough automated
integration tests.

## Lint

Expand All @@ -12,23 +13,42 @@ helm lint charts/sourcegraph/.

## Unit testing

We utilize [helm-unittest](https://github.com/helm-unittest/helm-unittest/), a BDD styled unit test framework, to validate our helm chart.
We utilize [helm-unittest](https://github.com/helm-unittest/helm-unittest/), a
BDD styled unit test framework, to validate our helm chart.

helm-unittest can be installed with:

```bash
helm plugin install https://github.com/helm-unittest/helm-unittest
```

Once the plugin is installed, you can run the unit tests using the following:
Once the plugin is installed, you can run the unit tests for a single chart:

```bash
helm unittest ./charts/sourcegraph
```

We currently do not have testing best practices or require unit tests for new changes, so add test cases at your best judgement if possible.
Or run every chart the same way CI does:

You may check out our [existing test cases](https://github.com/sourcegraph/deploy-sourcegraph-helm/tree/main/charts/sourcegraph/tests) and helm-unittest [docs](https://github.com/helm-unittest/helm-unittest/blob/master/DOCUMENT.md).
```bash
helm unittest \
./charts/sourcegraph \
./charts/sourcegraph-migrator \
./charts/sourcegraph-executor/k8s \
./charts/sourcegraph-executor/dind
```

Each chart keeps its tests in a `tests/` directory next to its `templates/`. CI
runs all of them via `scripts/ci/helm-unittest.sh`, so a new chart only needs a
`tests/` directory and an entry in that script.

We currently do not have testing best practices or require unit tests for new
changes, so add test cases at your best judgement if possible.

You may check out our [existing test
cases](https://github.com/sourcegraph/deploy-sourcegraph-helm/tree/main/charts/sourcegraph/tests)
and helm-unittest
[docs](https://github.com/helm-unittest/helm-unittest/blob/master/DOCUMENT.md).

## Manual testing

Expand All @@ -52,11 +72,16 @@ sourcegraph:
# More values to be added in order to test your change
```

Make sure you test both enabled and disabled toggles. For example, if you added a new values to conditional render some templates, turn it on and off in the `override.yaml` to make sure they both work. You can also include your `override.yaml` in the `Test plan` during PR review to help others understand your testing strategy.
Make sure you test both enabled and disabled toggles. For example, if you added
a new values to conditional render some templates, turn it on and off in the
`override.yaml` to make sure they both work. You can also include your
`override.yaml` in the `Test plan` during PR review to help others understand
your testing strategy.

### Update the image tag

You have two options to target specific Sourcegraph version. Add the below to your `override.yaml`:
You have two options to target specific Sourcegraph version. Add the below to
your `override.yaml`:

```yaml
sourcegraph:
Expand All @@ -65,11 +90,14 @@ sourcegraph:
useGlobalTagAsDefault: true
```

Alternatively, you can use `sg` to update the image tags directly in the default `values.yaml`, **DO NOT COMMIT** the changes:
Alternatively, you can use `sg` to update the image tags directly in the default
`values.yaml`, **DO NOT COMMIT** the changes:

```sh
cd charts/sourcegraph
sg ops update-images -kind helm -pin-tag 6.10.0 -cr-username=$DOCKER_HUB_USERNAME -cr-password=$DOCKER_HUB_PERSONAL_ACCESS_TOKEN .
sg ops update-images -kind helm -pin-tag 6.10.0 \
-cr-username=$DOCKER_HUB_USERNAME \
-cr-password=$DOCKER_HUB_PERSONAL_ACCESS_TOKEN .
```

### Inspect the entire rendered template
Expand All @@ -82,20 +110,25 @@ helm template -f ./override.yaml sourcegraph charts/sourcegraph/.

### Inspect the diff

Perform a diff of the rendered helm manifests before and after your change. There're many ways to produce the diff:
Perform a diff of the rendered helm manifests before and after your change.
There're many ways to produce the diff:

- Run `helm template` before and after the change, then run `diff bundle.old.yaml bundle.new.yaml`.
- Run `helm install` before the change, then run `helm diff` to inspect the diff.
- Run `helm template` before and after the change, then run `diff
bundle.old.yaml bundle.new.yaml`.
- Run `helm install` before the change, then run `helm diff` to inspect the
diff.

### Deploy the chart

You should make sure your change can be deployed

```sh
helm upgrade --install --create-namespace -n sourcegraph -f ./override.yaml sourcegraph charts/sourcegraph/.
helm upgrade --install --create-namespace -n sourcegraph \
-f ./override.yaml sourcegraph charts/sourcegraph/.
```

Follow the [confirm instance health] guide but use `kubectl port-forward` instead to validate instance health.
Follow the [confirm instance health] guide but use `kubectl port-forward`
instead to validate instance health.

```sh
kubectl -n sourcegraph port-forward svc/sourcegraph-frontend 30080
Expand Down
48 changes: 31 additions & 17 deletions charts/sourcegraph-executor/dind/tests/executor_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,46 @@ suite: executor
templates:
- executor/docker-daemon.ConfigMap.yaml
- executor/executor.Deployment.yaml
- executor/executor.Service.yaml
tests:
- it: should render the Deployment and Service if executor is enabled
- it: should render the ConfigMap and a single Deployment named after the queue
set:
executor:
enabled: true
queueName: batches
asserts:
- containsDocument:
kind: ConfigMap
apiVersion: v1
name: docker-config
template: executor/docker-daemon.ConfigMap.yaml
- containsDocument:
kind: Deployment
apiVersion: apps/v1
name: executor
name: executor-batches
template: executor/executor.Deployment.yaml
- contains:
path: spec.template.spec.containers[0].env
content:
name: EXECUTOR_QUEUE_NAME
value: batches
template: executor/executor.Deployment.yaml
- containsDocument:
kind: Service
apiVersion: v1
name: executor
template: executor/executor.Service.yaml

- it: should not render any resources if executor is disabled
- it: should render one Deployment per entry in queues
template: executor/executor.Deployment.yaml
set:
executor:
enabled: false
queues:
- name: codeintel
- name: batches
queueName: batches
asserts:
- hasDocuments:
count: 0
templates:
- executor/docker-daemon.ConfigMap.yaml
- executor/executor.Deployment.yaml
- executor/executor.Service.yaml
count: 2
- containsDocument:
kind: Deployment
apiVersion: apps/v1
name: executor-codeintel
documentIndex: 0
- containsDocument:
kind: Deployment
apiVersion: apps/v1
name: executor-batches
documentIndex: 1
Original file line number Diff line number Diff line change
@@ -1,29 +1,23 @@
suite: privateDockerRegistry
templates:
- private-docker-registry.Deployment.yaml
- private-docker-registry.PersistentVolumeClaim.yaml
- private-docker-registry.Service.yaml
- private-docker-registry/private-docker-registry.Statefulset.yaml
- private-docker-registry/private-docker-registry.Service.yaml
tests:
- it: should render the Deployment, Service and PVC if registry is enabled
- it: should render the StatefulSet and Service if registry is enabled
set:
privateDockerRegistry:
enabled: true
asserts:
- containsDocument:
kind: Deployment
kind: StatefulSet
apiVersion: apps/v1
name: private-docker-registry
template: private-docker-registry.Deployment.yaml
template: private-docker-registry/private-docker-registry.Statefulset.yaml
- containsDocument:
kind: Service
apiVersion: v1
name: private-docker-registry
template: private-docker-registry.Service.yaml
- containsDocument:
kind: PersistentVolumeClaim
apiVersion: v1
name: private-docker-registry
template: private-docker-registry.PersistentVolumeClaim.yaml
template: private-docker-registry/private-docker-registry.Service.yaml

- it: should not render any resources if registry is disabled
set:
Expand All @@ -32,7 +26,3 @@ tests:
asserts:
- hasDocuments:
count: 0
templates:
- private-docker-registry.Deployment.yaml
- private-docker-registry.PersistentVolumeClaim.yaml
- private-docker-registry.Service.yaml
129 changes: 129 additions & 0 deletions charts/sourcegraph-migrator/tests/migrator_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
suite: migrator
templates:
- migrator/sourcegraph-migrator.Job.yaml
tests:
- it: should render a Job with a random name suffix and default args
asserts:
- isKind:
of: Job
- matchRegex:
path: metadata.name
pattern: "^migrator-[a-z0-9]{5}$"
- matchRegex:
path: spec.template.spec.containers[0].image
pattern: "^index.docker.io/sourcegraph/migrator:"
- equal:
path: spec.template.spec.containers[0].args
value:
- up
- -db=all
- equal:
path: spec.template.spec.restartPolicy
value: Never

- it: should use migrator.name and migrator.args when set
set:
migrator:
name: db-migrate
args:
- upto
- -db=frontend
- -target=1648051770
asserts:
- matchRegex:
path: metadata.name
pattern: "^db-migrate-[a-z0-9]{5}$"
- equal:
path: spec.template.spec.containers[0].args
value:
- upto
- -db=frontend
- -target=1648051770

- it: should read all three database credentials from the default secrets
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: PGHOST
valueFrom:
secretKeyRef:
key: host
name: pgsql-auth
- contains:
path: spec.template.spec.containers[0].env
content:
name: CODEINTEL_PGHOST
valueFrom:
secretKeyRef:
key: host
name: codeintel-db-auth
- contains:
path: spec.template.spec.containers[0].env
content:
name: CODEINSIGHTS_PGHOST
valueFrom:
secretKeyRef:
key: host
name: codeinsights-db-auth

- it: should only swap the secret for the database that sets existingSecret
set:
pgsql:
auth:
existingSecret: "my-secret"
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: PGHOST
valueFrom:
secretKeyRef:
key: host
name: "my-secret"
- contains:
path: spec.template.spec.containers[0].env
content:
name: CODEINTEL_PGHOST
valueFrom:
secretKeyRef:
key: host
name: codeintel-db-auth

- it: should append migrator.env after the database credentials
set:
migrator:
env:
SRC_LOG_LEVEL:
value: dbug
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: SRC_LOG_LEVEL
value: dbug

- it: should prefer migrator.priorityClassName over the global one
set:
sourcegraph:
priorityClassName: global-class
migrator:
priorityClassName: migrator-class
asserts:
- equal:
path: spec.template.spec.priorityClassName
value: migrator-class

- it: should fall back to the global priorityClassName
set:
sourcegraph:
priorityClassName: global-class
asserts:
- equal:
path: spec.template.spec.priorityClassName
value: global-class

- it: should not set priorityClassName by default
asserts:
- isNull:
path: spec.template.spec.priorityClassName
8 changes: 7 additions & 1 deletion scripts/ci/helm-unittest.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,10 @@ HELM_UNITTEST_VERSION="v1.0.2"
helm plugin install https://github.com/helm-unittest/helm-unittest --version "$HELM_UNITTEST_VERSION" --verify=false

### Run the helm tests
helm unittest -q charts/sourcegraph
# helm unittest runs every chart even after a failure, prints a per-chart
# summary, and exits non-zero if any chart failed.
helm unittest \
charts/sourcegraph \
charts/sourcegraph-migrator \
charts/sourcegraph-executor/k8s \
charts/sourcegraph-executor/dind
Loading