Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### Improvements

- e2e: prefix API test run IDs with `cli-e2e-` so shared test org resources don't collide with other repos

## 1.95.1

### Bug fixes
Expand All @@ -27,7 +31,6 @@
- `--config` flag now takes precedence over `EXOSCALE_CONFIG` env var as expected #830
- Adding forgotten field `Inference Engine Version` in `exo ai deployment show` command output #833


## 1.94.2

### Bug fixes
Expand Down
39 changes: 39 additions & 0 deletions tests/e2e/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# E2E tests

Quick reference for running and writing CLI e2e tests.

## Run

```bash
make build
cd tests/e2e
go test -v # local only
go test -v -tags=api -run X # api only, set EXOSCALE_API_KEY and EXOSCALE_API_SECRET first
```

`-tags=api` needs `EXOSCALE_API_KEY`, `EXOSCALE_API_SECRET`. `EXOSCALE_ZONE` is optional, defaults to `ch-gva-2`.

## Layout

- `scenarios/without-api/` runs by default, no credentials needed.
- `scenarios/with-api/` runs with `-tags=api`, touches the real org.
- Custom commands are defined in `testscript_api_test.go`.

## Resource naming

API tests share an Exoscale org with other repos (terraform-provider, csi-driver, ...). The runner injects a unique `TEST_RUN_ID` prefixed with `cli-e2e-` into every scenario. Use it for every resource you create.

- good: `${TEST_RUN_ID}-nlb-a`, `pg-$TEST_RUN_ID`, `reboot-$TEST_RUN_ID`
- bad: hardcoded names, or `e2e-...` without the `cli-` part

Placeholders that intentionally don't exist (for not-found error tests) are fine as-is, e.g. `nonexistent-e2e-instance`.

## Writing a scenario

- File goes under `scenarios/with-api/<area>/`.
- `exec exo ...` runs the binary against the isolated workdir and env the runner sets up.
- `json-setenv VAR FIELD FILE` reads `FILE` (relative to workdir) and sets `VAR` to a top-level JSON field.
- `wait-instance-state ZONE ID TARGET [SECS]` polls instance show until state matches, default 300s.
- `wait-dbaas-state ZONE NAME TARGET [SECS]` same for dbaas, default 600s.
- `execpty --stdin=<file> <bin> [args...]` runs `<bin>` inside a PTY and feeds tokens from `<file>`. Tokens can be literals, arrow key names (`@down`, `@up`, ...), or `@wait:<pattern>` to delay the next input until `<pattern>` appears in PTY output.
- Each scenario creates and deletes its own resources. The runner does not clean up after you.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Full lifecycle: create a dedicated instance with reverse DNS, verify it, then delete it.
# TEST_ZONE and TEST_RUN_ID are injected by the API test runner.

exec exo --zone $TEST_ZONE --output-format json compute instance create cli-e2e-rdns-$TEST_RUN_ID --instance-type standard.tiny --template 'Linux Ubuntu 22.04 LTS 64-bit' --disk-size 10 --reverse-dns rdns-$TEST_RUN_ID.example.org
exec exo --zone $TEST_ZONE --output-format json compute instance create rdns-$TEST_RUN_ID --instance-type standard.tiny --template 'Linux Ubuntu 22.04 LTS 64-bit' --disk-size 10 --reverse-dns rdns-$TEST_RUN_ID.example.org

json-setenv INSTANCE_ID id stdout

Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/scenarios/with-api/compute/instance_reboot.txtar
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# TEST_ZONE and TEST_RUN_ID are injected by the API test runner.

# Create a fresh instance for this test; testscript captures stdout automatically
exec exo --zone $TEST_ZONE --output-format json compute instance create cli-e2e-reboot-$TEST_RUN_ID --instance-type standard.tiny --template 'Linux Ubuntu 22.04 LTS 64-bit' --disk-size 10
exec exo --zone $TEST_ZONE --output-format json compute instance create reboot-$TEST_RUN_ID --instance-type standard.tiny --template 'Linux Ubuntu 22.04 LTS 64-bit' --disk-size 10

# Extract the instance ID into INSTANCE_ID env var from the captured stdout
json-setenv INSTANCE_ID id stdout
Expand Down
14 changes: 7 additions & 7 deletions tests/e2e/scenarios/with-api/dbaas/pg_config_lifecycle.txtar
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,26 @@
# TEST_ZONE and TEST_RUN_ID are injected by the API test runner.

# Create a PG service with the new configuration options (disable termination protection for cleanup)
exec exo --zone $TEST_ZONE --output-format json dbaas create pg hobbyist-2 cli-e2e-pg-$TEST_RUN_ID --pg-shared-buffers-percentage 25 --pg-synchronous-replication off --pg-work-mem 128 --termination-protection=false
exec exo --zone $TEST_ZONE --output-format json dbaas create pg hobbyist-2 pg-$TEST_RUN_ID --pg-shared-buffers-percentage 25 --pg-synchronous-replication off --pg-work-mem 128 --termination-protection=false

# Wait for the service to be running (PG provisioning can take 5-10 minutes)
wait-dbaas-state $TEST_ZONE cli-e2e-pg-$TEST_RUN_ID running 600
wait-dbaas-state $TEST_ZONE pg-$TEST_RUN_ID running 600

# Show the service and verify the new fields are present in JSON output
exec exo --zone $TEST_ZONE --output-format json dbaas show cli-e2e-pg-$TEST_RUN_ID
exec exo --zone $TEST_ZONE --output-format json dbaas show pg-$TEST_RUN_ID
stdout '"shared_buffers_percentage"'
stdout '"synchronous_replication"'
stdout '"work_mem"'

# Update: change work_mem
exec exo --zone $TEST_ZONE dbaas update cli-e2e-pg-$TEST_RUN_ID --pg-work-mem 256 -f
exec exo --zone $TEST_ZONE dbaas update pg-$TEST_RUN_ID --pg-work-mem 256 -f

# Wait for the service to stabilise after update
wait-dbaas-state $TEST_ZONE cli-e2e-pg-$TEST_RUN_ID running 600
wait-dbaas-state $TEST_ZONE pg-$TEST_RUN_ID running 600

# Verify updated value
exec exo --zone $TEST_ZONE --output-format json dbaas show cli-e2e-pg-$TEST_RUN_ID
exec exo --zone $TEST_ZONE --output-format json dbaas show pg-$TEST_RUN_ID
stdout '"work_mem"'

# Teardown: delete the service
exec exo --zone $TEST_ZONE dbaas delete -f cli-e2e-pg-$TEST_RUN_ID
exec exo --zone $TEST_ZONE dbaas delete -f pg-$TEST_RUN_ID
5 changes: 4 additions & 1 deletion tests/e2e/testscript_api_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,10 @@ func runAPITestSuite(t *testing.T, dir string) {
zone = "ch-gva-2"
}

runID := fmt.Sprintf("e2e-%d-%s", time.Now().Unix(), randString(6))
// Prefix every test resource with `cli-e2e-` so runs against the shared
// Exoscale test organisation do not collide with resources created by
// other repositories (terraform-provider-exoscale, csi-driver, ...).
runID := fmt.Sprintf("cli-e2e-%d-%s", time.Now().Unix(), randString(6))
t.Logf("API test run ID: %s (zone: %s)", runID, zone)

suite := &APITestSuite{
Expand Down
Loading