From 0a58895652549fd47a59f49421f4aeec4a2cfa74 Mon Sep 17 00:00:00 2001 From: Sabine Maennel <5292683+sabinem@users.noreply.github.com> Date: Tue, 8 Sep 2026 07:38:02 +0200 Subject: [PATCH] feat(ci): publish the Helm chart, not just the container images MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI built and pushed the container images but never the chart, so deploying meant copying the chart out of a checkout by hand. It now publishes the chart to GHCR too, and a deployment can install a pinned version. The chart carries its own version, separate from the app's: a chart fix needs no app release, and an app release does not republish a chart that has not changed. Two things were broken underneath and are fixed here. The repo's version numbers had drifted apart (VERSION, both components and package.json each said something different), and `just version::bump` crashed instead of fixing them. CI also gained three checks: the versions agree, the chart lints, and the chart version was bumped whenever the chart changed. style: reformat ci-cd.md treefmt output only — realigns a markdown table column and rewraps two paragraphs. No wording changes. --- .github/ci-cd.md | 100 +++++++-- .github/workflows/ci.yml | 77 +++++++ CLAUDE.md | 29 ++- VERSION | 2 +- components/backend/.component.yaml | 2 +- components/frontend/.component.yaml | 2 +- components/frontend/package.json | 2 +- helm-chart/Chart.yaml | 8 +- helm-chart/templates/backend-deployment.yaml | 2 +- helm-chart/templates/frontend-deployment.yaml | 2 +- helm-chart/values.yaml | 27 ++- justfile | 1 + tools/helm/lint-values.yaml | 29 +++ tools/just/ci.just | 3 + tools/just/helm.just | 197 ++++++++++++++++++ tools/just/version.just | 99 ++++++++- tools/nix/hackagon/lib/toolchain.nix | 4 + 17 files changed, 537 insertions(+), 49 deletions(-) create mode 100644 tools/helm/lint-values.yaml create mode 100644 tools/just/helm.just diff --git a/.github/ci-cd.md b/.github/ci-cd.md index b2b26d95..279b192b 100644 --- a/.github/ci-cd.md +++ b/.github/ci-cd.md @@ -45,21 +45,76 @@ Woodpecker, Drone, GitLab CI, etc. ## Pipeline stages -Single `ci.yml` workflow triggered on PR and push to `main`: - -| # | Stage | Command | Purpose | -| --- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 1 | Setup Nix + cache | `cachix/install-nix-action` + `cachix/cachix-action` | Avoid rebuilding toolchains every run | -| 2 | Verify generated code is in sync | `just ci::codegen-check` | Catch PRs that forgot to regenerate stubs; uses `default` shell (buf + codegen tools not in CI shell); pnpm install required because `protoc-gen-ts_proto` is a node_modules binary | -| 3 | Format check | `just ci::run just check::format` | Backstop for developers who haven't opted into the githooks framework | -| 4 | Lint | `just ci::run bash -c "just check::lint -c backend && just check::lint -c frontend"` | Delegates to quitsh → golangci-lint, eslint, svelte-check, typos, yamllint | -| 5 | Build | `just ci::run bash -c "just build -c backend && just build -c frontend"` | All components via quitsh; catches compilation errors before running tests | -| 6 | Test | `just ci::run bash -c "just check::test -c backend && just check::test -c frontend"` | Go unit tests + Vitest; coverage upload per [codecov.yaml](../tools/configs/codecov/codecov.yaml) | -| 7 | Container images _(main only)_ | `nix build ./tools/nix#backend-service-dev` and `#frontend-service-dev` | Reproducible OCI images | -| 8 | Publish _(tag/main only)_ | Push images to registry; attach image digests or registry references to the GitHub release via `gh release edit` | Gated behind release trigger; links the release to its artifacts | - -Stages 2–6 are PR-blocking. Stage 7 runs on `main` only. Stage 8 only on tagged -releases. +Single `ci.yml` workflow triggered on PR, push to `main`, and `v*` tags: + +| # | Stage | Command | Purpose | +| --- | --------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------ | +| 1 | Setup Nix + cache | `install-nix-action` + `cachix-action` | Avoid rebuilding toolchains every run | +| 2 | Codegen in sync | `just ci::codegen-check` | Catch PRs that forgot to regenerate stubs; uses the `default` shell, where buf lives | +| 3 | Format check | `just ci::run just check::format` | Backstop for developers who haven't opted into the githooks framework | +| 4 | Lint | `just ci::run just check::lint -c ` | golangci-lint, eslint, svelte-check, typos, yamllint | +| 5 | Version consistency | `just ci::run just version::check` | `VERSION` of the app, both `.component.yaml` versions and `package.json` must agree | +| 6 | Helm chart lint | `just ci::run just helm::lint` | `helm lint` plus a full render — the chart cannot render with its own defaults | +| 7 | Chart version bumped _(PR)_ | `just ci::run just helm::check-bump` | A chart change without a version bump can never reach a cluster | +| 8 | Build | `just ci::run just check::build -c ` | All components via quitsh | +| 9 | Test | `just ci::run just check::test -c ` | Go unit tests + Vitest; coverage per [codecov.yaml](../tools/configs/codecov/codecov.yaml) | +| 10 | Container images | `quitsh image` (the `images` job) | PR: build only. `main`: push to `temporary/`. Tag: push to `release/` | +| 11 | Publish chart | `just helm::publish` | Pushes the chart when Chart.yaml holds an unpublished version; no-ops otherwise | + +Stages 2–9 are PR-blocking. Stage 10 builds on every PR and pushes on `main` and +on tags. Stage 11 runs on every push and does nothing unless the chart version +has moved. + +## Two release trains + +The app and the chart version are independent from each other: a chart fix with +no app change should not force an app release, and an app release should not +republish an unchanged chart. + +**The app** is versioned by `VERSION` at the repo root. `just version::bump` +moves it along with `.component.yaml` versions and `package.json`, commits, and +tags. Pushing that `v*` tag builds and pushes: + +``` +ghcr.io/swissdatasciencecenter/hackagon/release/backend-service:X.Y.Z +ghcr.io/swissdatasciencecenter/hackagon/release/frontend-service:X.Y.Z +``` + +Stage 5 refuses a tree where those files disagree, because the component version +_is_ the image tag. + +**The chart** is versioned by `helm-chart/Chart.yaml`, by hand: + +- `version` — the chart's own release. Bump it whenever anything under + `helm-chart/` changes; stage 7 fails the PR if you forget. +- `appVersion` — the app release this chart deploys. The deployment templates + fall back to it for the image tag, so it decides what a cluster actually runs. + +`just helm::publish` reads both from Chart.yaml and injects nothing. It pushes +to `oci://ghcr.io/swissdatasciencecenter/hackagon/charts/hackagon` and gives up +early in two cases, both reported and neither a failure: + +- **the chart version is already published** — the normal case on most pushes +- **`appVersion` has no published images** — the chart would not be installable + +The second is what makes "bump appVersion to an unreleased version" safe: the +chart waits, and the run that finally builds those images publishes it. + +Two consequences worth knowing: + +- **A new GHCR package is private by default.** The first `v*` tag creates + `release/*`, and the first chart publish creates `charts/hackagon`; until + someone sets those to public in the org's package settings, a cluster needs an + `imagePullSecret`. The existing `temporary/*` packages are already public. +- **A release image is never overwritten.** quitsh refuses to push a release tag + that already exists ([upload.go](../tools/quitsh/pkg/image/upload.go)), so + re-tagging a version fails rather than replacing an artifact someone has + deployed. Bump the version instead. + +To deploy the head of `main` rather than a release, override the image +repository and tag to the `temporary/` package — see the Images note in + +[helm-chart/values.yaml](../helm-chart/values.yaml). ## Parallelism @@ -88,10 +143,10 @@ Run the full CI pipeline locally from the repo root: just ci::all ``` -This mirrors stages 2–6 exactly as they run on GitHub Actions (generate check → -format → lint → build → test). Works from a bare shell or from a direnv-managed -shell — `ci::all` handles the Nix shell itself, so do **not** call it from -inside `nix develop`. +This mirrors stages 2–9 exactly as they run on GitHub Actions (generate check → +format → lint → version → helm → chart → build → test). Works from a bare shell +or from a direnv-managed shell — `ci::all` handles the Nix shell itself, so do +**not** call it from inside `nix develop`. Before pushing: `just ci::all` → if it passes locally, CI passes. @@ -120,8 +175,11 @@ Individual stages can also be isolated from the repo root: just ci::codegen-check # stage 2 — uses default shell just ci::run just check::format # stage 3 just ci::run just check::lint -c backend # stage 4 (one component) -just ci::run just build -c backend # stage 5 (one component) -just ci::run just check::test -c backend # stage 6 (one component) +just ci::run just version::check # stage 5 +just ci::run just helm::lint # stage 6 +just ci::run just helm::check-bump # stage 7 +just ci::run just build -c backend # stage 8 (one component) +just ci::run just check::test -c backend # stage 9 (one component) ``` On CI failure: copy the failing command from the workflow YAML, run it locally, diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index edc0bd7d..d348a395 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,6 +7,11 @@ on: tags: ["v*"] workflow_dispatch: +# Permissions default to read-only, so PRs from forks cannot push packages. +# The `images` jobs that publish packages overwrite this. +permissions: + contents: read + # Cancel in-progress runs on the same branch/PR to avoid wasted minutes. concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -18,7 +23,10 @@ jobs: timeout-minutes: 30 steps: + # `fetch-depth: 0` is required for `just version::check` to see the tags. - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + fetch-depth: 0 - uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27 with: @@ -63,6 +71,21 @@ jobs: just ci::run just check::lint -c backend just ci::run just check::lint -c frontend + # Make sure the version in AppVersion is consistent in all app components. + - name: Version consistency + run: just ci::run just version::check + + # Make sure the helm chart renders correctly and passes `helm lint`. + - name: Helm chart lint + run: just ci::run just helm::lint + + # Make sure the Chart version is raised when the helm chart changes. + - name: Helm chart version bumped + if: github.event_name == 'pull_request' + run: | + just ci::run just helm::check-bump \ + "origin/${{ github.event.pull_request.base.ref }}" + - name: Build run: | just ci::run just check::build -c backend @@ -81,6 +104,9 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 30 needs: [ci] + permissions: + contents: read + packages: write steps: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 @@ -135,3 +161,54 @@ jobs: --credential-user-env GH_USERNAME \ --credential-token-env GHCR_TOKEN fi + + # Publishes the chart as a versioned OCI artifact so a deployment can pin + # `oci://.../charts/hackagon --version X.Y.Z`. + # + # Deliberately not tag-driven. The chart carries its own version and its own + # appVersion, and the recipe reads both from Chart.yaml. + # + # An app release can be published without a chart bump, but a chart bump + # cannot be published without an app release. + # The `needs: [images]` ensures that the chart cannot be published unless + # the images for its appVersion are already published. + publish-chart: + if: github.event_name == 'push' + runs-on: ubuntu-24.04 + timeout-minutes: 30 + needs: [images] + permissions: + contents: read + packages: write + + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + + - uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27 + with: + extra_nix_config: | + experimental-features = nix-command flakes + + - uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15 + with: + name: hackagon + authToken: ${{ secrets.CACHIX_AUTH_TOKEN }} + + - name: Install bootstrap tools + run: nix profile install --accept-flake-config ./tools/nix#bootstrap + + # Two logins, two tools: helm pushes the chart, skopeo answers "is this + # already published?". They keep separate credential stores. + # Credentials are passed through env. + - name: Log in to GHCR + env: + GH_USERNAME: ${{ github.actor }} + GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + just ci::run bash -c ' + echo "$GHCR_TOKEN" | helm registry login ghcr.io -u "$GH_USERNAME" --password-stdin + echo "$GHCR_TOKEN" | skopeo login ghcr.io -u "$GH_USERNAME" --password-stdin + ' + + - name: Publish chart if its version is unpublished + run: just ci::run just helm::publish diff --git a/CLAUDE.md b/CLAUDE.md index 76ad8290..111993d8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -67,7 +67,9 @@ components/backend/ ├── internal/logx/logx.go # slog setup + Fatal helper └── Schema.md # human-readable DB reference (auto-generated) components/frontend/ # SvelteKit; generated gRPC clients under src/lib/server/grpc/generated/ +helm-chart/ # Helm chart; published as an OCI artifact on a v* tag mydocs/docs/backend-tickets/ # known gaps, one file per issue (README inside) +tools/helm/lint-values.yaml # throwaway values so the chart can be rendered in CI tools/nix/ # Nix flake + process-compose config (toolchain.nix) tools/just/*.just # just modules — see Dev commands justfile # root justfile, imports the modules above @@ -98,7 +100,13 @@ just check::lint -c backend # also: build, test, format; -c frontend just ci::all # everything CI runs, locally just version::show # declared version + what a build stamps in the footer -just version::bump patch|minor|major # edit VERSION, commit, annotated tag +just version::check # fail if a component version has drifted from VERSION +just version::bump patch|minor|major # edit VERSION everywhere, commit, annotated tag + +just helm::lint # helm lint + full render against tools/helm/lint-values.yaml +just helm::template [args] # render the chart to stdout +just helm::check-bump # fail if helm-chart/ changed without a version bump +just helm::publish # push the chart if Chart.yaml's version is unpublished ``` Backend listens on **:3000**, frontend on **:8081**. Dev users (Keycloak @@ -139,11 +147,20 @@ These hold across the whole codebase; the skills explain the mechanisms. running server — see **backend-api-explore**. Any inventory committed to a markdown file is a snapshot that starts rotting immediately, this one included. -- **`VERSION` at the repo root is the only declared version.** Bump it through - `just version::bump`, never by hand — the recipe is what also commits it, tags - it, and keeps `components/frontend/package.json` in step. The frontend reads - it at build time (`vite.config.ts` → `$lib/version` → the footer), so a - version edited without a rebuild will not show up. +- **`VERSION` at the repo root is the app's only declared version.** Bump it + with `just version::bump`, never by hand — the recipe moves `VERSION`, both + `components/*/.component.yaml` versions and `components/frontend/package.json` + together, then commits and tags. A component version _is_ its published image + tag, so `just version::check` (a CI stage) fails a tree where these disagree. + The frontend reads `VERSION` at build time (`vite.config.ts` → `$lib/version` + → the footer), so editing it without a rebuild changes nothing. +- **The chart versions itself; `VERSION` never touches it.** + `helm-chart/Chart.yaml` holds two hand-edited numbers: `version` is the + chart's own release, `appVersion` is the app release it deploys. + `just helm::publish` reads both and injects nothing, so a chart is published + only when its own `version` changes — never as a side effect of an app + release. It skips without failing when that version is already published, or + when `appVersion` has no images yet. ## Don't diff --git a/VERSION b/VERSION index 6e8bf73a..a3df0a69 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.0 +0.8.0 diff --git a/components/backend/.component.yaml b/components/backend/.component.yaml index 62ce0501..29fd63e3 100644 --- a/components/backend/.component.yaml +++ b/components/backend/.component.yaml @@ -1,5 +1,5 @@ name: backend -version: 0.1.0 +version: 0.8.0 language: go targets: diff --git a/components/frontend/.component.yaml b/components/frontend/.component.yaml index 29985ca0..74905662 100644 --- a/components/frontend/.component.yaml +++ b/components/frontend/.component.yaml @@ -1,5 +1,5 @@ name: frontend -version: 0.7.0 +version: 0.8.0 language: typescript targets: diff --git a/components/frontend/package.json b/components/frontend/package.json index 76fdd1d6..c9831e68 100644 --- a/components/frontend/package.json +++ b/components/frontend/package.json @@ -1,7 +1,7 @@ { "name": "frontend-v3", "private": true, - "version": "0.0.1", + "version": "0.8.0", "type": "module", "scripts": { "dev": "vite dev", diff --git a/helm-chart/Chart.yaml b/helm-chart/Chart.yaml index 619bcb6e..ee972225 100644 --- a/helm-chart/Chart.yaml +++ b/helm-chart/Chart.yaml @@ -4,8 +4,12 @@ description: A Helm chart for Hackagon on Kubernetes type: application -version: 0.1.0 -appVersion: "1.0.0" +# The chart's own version. Bumped by hand when the chart changes, and +# independent of the app: the CI publishes whatever it finds here. +version: 0.2.0 +# The app release this chart deploys. A new app release does +# not become deployable until someone points the chart at it. +appVersion: "0.8.0" dependencies: - name: postgresql diff --git a/helm-chart/templates/backend-deployment.yaml b/helm-chart/templates/backend-deployment.yaml index f543ab46..025ef7bf 100644 --- a/helm-chart/templates/backend-deployment.yaml +++ b/helm-chart/templates/backend-deployment.yaml @@ -24,7 +24,7 @@ spec: spec: containers: - name: backend - image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}" + image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.backend.image.pullPolicy }} args: - "--config-dir=/etc/hackagon/" diff --git a/helm-chart/templates/frontend-deployment.yaml b/helm-chart/templates/frontend-deployment.yaml index c31115ab..1a549966 100644 --- a/helm-chart/templates/frontend-deployment.yaml +++ b/helm-chart/templates/frontend-deployment.yaml @@ -24,7 +24,7 @@ spec: spec: containers: - name: frontend - image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }}" + image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.frontend.image.pullPolicy }} args: - "--config-dir=/etc/hackagon" diff --git a/helm-chart/values.yaml b/helm-chart/values.yaml index ecb154da..a1810889 100644 --- a/helm-chart/values.yaml +++ b/helm-chart/values.yaml @@ -4,13 +4,31 @@ baseDomain: "example.com" # -- Keycloak realm JSON (pass via --set-file: --set-file realmJson=@./tools/configs/keycloak/realm-hackagon.json) # realmJson: "" +# ============================================================ +# Images +# ============================================================ +# An image address is `repository:tag`. Two repositories exist: +# release/ one image per `v*` tag — what you deploy +# temporary/ one image per push to main — for testing, moves constantly +# +# If there is no `tag` specified: then the chart then uses its the +# `appVersion` from Chart.yaml, +# +# But for test deploys you might want to override this, with a +# tagged image from the `temporary/` repository. +# +# To test against the head of `main`, override both keys, e.g.: +# --set frontend.image.repository=/temporary/frontend-service +# --set frontend.image.tag=latest +# # ============================================================ # Frontend # ============================================================ frontend: image: - repository: ghcr.io/swissdatasciencecenter/hackagon/temporary/frontend-service - tag: "latest" + repository: ghcr.io/swissdatasciencecenter/hackagon/release/frontend-service + # Empty: falls back to .Chart.AppVersion — see the Images note above. + tag: "" pullPolicy: IfNotPresent replicaCount: 1 @@ -67,8 +85,9 @@ frontend: # ============================================================ backend: image: - repository: ghcr.io/swissdatasciencecenter/hackagon/temporary/backend-service - tag: "latest" + repository: ghcr.io/swissdatasciencecenter/hackagon/release/backend-service + # Empty: falls back to .Chart.AppVersion — see the Images note above. + tag: "" pullPolicy: IfNotPresent replicaCount: 1 diff --git a/justfile b/justfile index 3101a76e..6e4ab262 100644 --- a/justfile +++ b/justfile @@ -14,6 +14,7 @@ mod deploy "./tools/just/deploy.just" mod codegen "./tools/just/codegen.just" mod clean "./tools/just/clean.just" mod version "./tools/just/version.just" +mod helm "./tools/just/helm.just" [private] default: diff --git a/tools/helm/lint-values.yaml b/tools/helm/lint-values.yaml new file mode 100644 index 00000000..a856756d --- /dev/null +++ b/tools/helm/lint-values.yaml @@ -0,0 +1,29 @@ +# Throwaway values, used only to make the chart renderable so `helm lint` and +# `helm template` can check it. +# +# Used by `just helm::lint`. + +baseDomain: lint.invalid + +backend: + config: + server: + adminkeycloakid: "00000000-0000-0000-0000-000000000000" + database: + postgresPassword: "placeholder-not-a-real-password" + +keycloak: + hostname: + hostname: "https://auth.lint.invalid" + database: + external: + host: "hackagon-postgresql" + password: "placeholder-not-a-real-password" + +postgresql: + auth: + postgresPassword: "placeholder-not-a-real-password" + +frontendSecrets: + clientSecret: "placeholder-not-a-real-secret" + authSecret: "placeholder-not-a-real-secret" diff --git a/tools/just/ci.just b/tools/just/ci.just index 222dce8f..1d048f20 100644 --- a/tools/just/ci.just +++ b/tools/just/ci.just @@ -62,6 +62,9 @@ all: "codegen | just ci::codegen-check" "format | just ci::run just check::format" "lint | just ci::run bash -c 'just check::lint -c backend && just check::lint -c frontend'" + "version | just ci::run just version::check" + "helm | just ci::run just helm::lint" + "chart | just ci::run just helm::check-bump" "build | just ci::run bash -c 'just build -c backend && just build -c frontend'" "test | just ci::run bash -c 'just check::test -c backend && just check::test -c frontend'" ) diff --git a/tools/just/helm.just b/tools/just/helm.just new file mode 100644 index 00000000..c2704397 --- /dev/null +++ b/tools/just/helm.just @@ -0,0 +1,197 @@ +set positional-arguments +set shell := ["bash", "-cue"] + +root_dir := `git rev-parse --show-toplevel` +chart_dir := root_dir + "/helm-chart" +chart_file := chart_dir + "/Chart.yaml" +charts_dir := chart_dir + "/charts" +out_dir := root_dir + "/.output/helm" +lint_values := root_dir + "/tools/helm/lint-values.yaml" + +registry := "ghcr.io/swissdatasciencecenter/hackagon" +chart_ref := registry + "/charts/hackagon" +oci_repo := "oci://" + registry + "/charts" + +[private] +default: + just --list --unsorted -f "{{ source_file() }}" + +[group('helm')] +help: + #!/usr/bin/env bash + bold="\033[1m" + dim="\033[2m" + cyan="\033[36m" + reset="\033[0m" + + echo "" + echo -e " ${bold}Helm Commands${reset}" + echo "" + echo -e " ${cyan}just helm::lint${reset}" + echo -e " ${dim}Lint the chart and render it against tools/helm/lint-values.yaml${reset}" + echo "" + echo -e " ${cyan}just helm::template${reset} [helm args...]" + echo -e " ${dim}Render the chart manifests to stdout${reset}" + echo "" + echo -e " ${cyan}just helm::check-bump${reset} [base]" + echo -e " ${dim}Fail if helm-chart/ changed without a Chart.yaml version bump${reset}" + echo "" + echo -e " ${cyan}just helm::publish${reset}" + echo -e " ${dim}Publish the chart if its version is not already in the registry${reset}" + echo "" + echo -e " ${cyan}just helm::clean${reset}" + echo -e " ${dim}Remove fetched subchart archives and packaged charts${reset}" + echo "" + echo -e " ${bold}Two versions, two purposes${reset} ${dim}(helm-chart/Chart.yaml)${reset}" + echo -e " ${dim}version the chart's own release, bumped when the chart changes${reset}" + echo -e " ${dim}appVersion the app release the chart deploys${reset}" + echo -e " ${dim}Both are edited by hand. An app tag publishes images, never a chart.${reset}" + echo "" + echo -e " ${dim}The chart cannot render with its own defaults — several values${reset}" + echo -e " ${dim}(admin id, passwords, hostnames) are intentionally empty.${reset}" + echo "" + +# Remove fetched subchart archives and packaged charts. +[group('helm')] +clean: + rm -rf "{{ charts_dir }}" "{{ out_dir }}" + +# Fetch the subchart archives the chart declares. +[private] +[group('helm')] +fetch-deps: + #!/usr/bin/env bash + set -eu + + # `--force-update` keeps this idempotent: a plain `repo add` fails on a name + # that already exists. + helm repo add bitnami https://charts.bitnami.com/bitnami --force-update + helm repo add helmforge https://repo.helmforge.dev --force-update + + cd "{{ chart_dir }}" + # `build`, not `update`: build obeys the committed Chart.lock, so CI resolves + # the same subchart versions a developer does. `update` re-resolves the + # `18.x`/`3.x` ranges and moves them silently. + helm dependency build + +# Lint the chart and render it against throwaway values. +[group('helm')] +lint: fetch-deps + #!/usr/bin/env bash + set -eu + + helm lint "{{ chart_dir }}" --values "{{ lint_values }}" + + # `lint` does not catch a template that fails to render, so render it too. + helm template hackagon-lint "{{ chart_dir }}" --values "{{ lint_values }}" >/dev/null + +# Render the chart manifests to stdout. +# Usage: just helm::template --values my-values.yaml +[group('helm')] +[no-cd] +template *args: fetch-deps + #!/usr/bin/env bash + set -eu + helm template "$@" "{{ chart_dir }}" + +# Fail if anything under helm-chart/ changed without a Chart.yaml version bump. +# The chart is published from whatever version Chart.yaml holds, so an unbumped +# change is one that can never reach a cluster. +[group('helm')] +check-bump base="origin/main": + #!/usr/bin/env bash + set -eu + cd "{{ root_dir }}" + + if ! git rev-parse --verify -q "{{ base }}" >/dev/null; then + echo "Unknown base ref '{{ base }}'. In CI the checkout needs full history" >&2 + echo "(fetch-depth: 0); locally try 'just helm::check-bump main'." >&2 + exit 1 + fi + + base=$(git merge-base "{{ base }}" HEAD) + + if [ -z "$(git diff --name-only "$base" HEAD -- helm-chart/)" ]; then + echo "No chart changes against {{ base }}; nothing to check." + exit 0 + fi + + read_version() { sed -n 's/^version: *//p' | head -1; } + before=$(git show "$base:helm-chart/Chart.yaml" 2>/dev/null | read_version || true) + now=$(read_version <"{{ chart_file }}") + + if [ -n "$before" ] && [ "$before" = "$now" ]; then + echo "helm-chart/ changed but Chart.yaml still declares version $now." >&2 + echo "" >&2 + git diff --name-only "$base" HEAD -- helm-chart/ | sed 's/^/ /' >&2 + echo "" >&2 + echo "Bump 'version' in helm-chart/Chart.yaml. It is the chart's own" >&2 + echo "release number, not the app's — see just helm::help." >&2 + exit 1 + fi + + echo "Chart version $before -> $now." + +# Publish the chart to GHCR as an OCI artifact, if it is not already there. +# Both versions come from Chart.yaml and nothing is injected, so an app release +# never publishes a chart on its own — someone has to bump the chart first. +# Assumes `helm registry login` and `skopeo login` have already happened. +[group('helm')] +publish: + #!/usr/bin/env bash + set -eu + + chart_v=$(sed -n 's/^version: *//p' "{{ chart_file }}" | head -1) + app_v=$(sed -n 's/^appVersion: *"*\([^"]*\)"*/\1/p' "{{ chart_file }}" | head -1) + + if ! [[ "$chart_v" =~ ^[0-9]+\.[0-9]+\.[0-9]+ ]]; then + echo "Chart.yaml version '$chart_v' is not a semver." >&2 + exit 1 + fi + if [ -z "$app_v" ]; then + echo "Chart.yaml declares no appVersion." >&2 + exit 1 + fi + + exists() { skopeo inspect --raw "docker://$1" >/dev/null 2>&1; } + + if exists "{{ chart_ref }}:$chart_v"; then + echo "Chart $chart_v is already published; nothing to do." + echo "Bump 'version' in helm-chart/Chart.yaml to publish a new one." + exit 0 + fi + + # A published chart must be installable. Its image tags default to + # appVersion, so refuse to promise an app release that was never built. + for comp in backend frontend; do + ref="{{ registry }}/release/$comp-service:$app_v" + if ! exists "$ref"; then + echo "Not publishing chart $chart_v: appVersion $app_v has no images." >&2 + echo " missing: $ref" >&2 + echo "Push the v$app_v tag first, or point appVersion at a released version." >&2 + exit 0 + fi + done + + just helm::fetch-deps + + mkdir -p "{{ out_dir }}" + helm package "{{ chart_dir }}" --destination "{{ out_dir }}" + + pkg="{{ out_dir }}/hackagon-${chart_v}.tgz" + if [ ! -f "$pkg" ]; then + echo "Expected package '$pkg' was not produced." >&2 + exit 1 + fi + + helm push "$pkg" "{{ oci_repo }}" + + echo "" + echo " chart $chart_v" + echo " deploys app $app_v" + echo " pushed {{ chart_ref }}:$chart_v" + echo "" + echo " Install with:" + echo " helm install hackagon {{ oci_repo }}/hackagon --version $chart_v \\" + echo " --set baseDomain=example.com --set-file realmJson=@path/to/realm.json" + echo "" diff --git a/tools/just/version.just b/tools/just/version.just index 2ff4637e..309c8ebd 100644 --- a/tools/just/version.just +++ b/tools/just/version.just @@ -5,6 +5,11 @@ root_dir := `git rev-parse --show-toplevel` version_file := root_dir + "/VERSION" pkg_file := root_dir + "/components/frontend/package.json" +# Components whose `.component.yaml` version becomes an OCI image tag. Not +# `tools/quitsh`: vendored, carries its own upstream version, builds no image. +# The Helm chart is deliberately absent — see helm.just. +image_comps := "backend frontend" + [private] default: just --list --unsorted -f "{{ source_file() }}" @@ -23,14 +28,17 @@ help: echo -e " ${cyan}just version::show${reset}" echo -e " ${dim}Print the declared version and what a build stamps into the footer${reset}" echo "" + echo -e " ${cyan}just version::check${reset}" + echo -e " ${dim}Fail if any component version has drifted from VERSION${reset}" + echo "" echo -e " ${cyan}just version::bump${reset} patch|minor|major" - echo -e " ${dim}Edit VERSION, commit it, and create the matching annotated tag${reset}" + echo -e " ${dim}Edit VERSION, propagate it to every component, commit, and tag${reset}" echo "" echo -e " ${cyan}just version::tag${reset}" echo -e " ${dim}Tag HEAD with the version already declared in VERSION${reset}" echo "" - echo -e " ${dim}VERSION at the repo root is the single source of truth. The${reset}" - echo -e " ${dim}frontend reads it at build time — see components/frontend/vite.config.ts.${reset}" + echo -e " ${dim}VERSION covers the app only: the per-component copies become the${reset}" + echo -e " ${dim}published image tags. The Helm chart versions itself — see just helm::help.${reset}" echo "" # Print the declared version and the string a build stamps into the footer. @@ -64,7 +72,52 @@ show: echo "build $build" echo "tags $(git -C "{{ root_dir }}" tag --list 'v*' | tr '\n' ' ' | sed 's/ $//' || true)" -# Bump VERSION, commit it, and create the matching annotated tag. + echo "images" + for comp in {{ image_comps }}; do + v=$(sed -n 's/^version: *//p' "{{ root_dir }}/components/$comp/.component.yaml" | head -1) + marker="" + if [ "$v" != "$declared" ]; then + marker=" <- drifted from VERSION" + fi + echo " $comp-service:$v$marker" + done + +# Fail if VERSION, the component versions and package.json disagree. +# The component version is the published image tag, so a drifted tree releases +# an image named after the wrong version — and a second such release collides +# with the existing tag and is refused (tools/quitsh/pkg/image/upload.go). +[group('version')] +check: + #!/usr/bin/env bash + set -eu + declared=$(tr -d '[:space:]' <"{{ version_file }}") + failed=0 + + for comp in {{ image_comps }}; do + f="{{ root_dir }}/components/$comp/.component.yaml" + v=$(sed -n 's/^version: *//p' "$f" | head -1) + if [ "$v" != "$declared" ]; then + echo " $comp: .component.yaml declares '$v', VERSION declares '$declared'" >&2 + failed=1 + fi + done + + v=$(sed -n 's/.*"version": *"\([^"]*\)".*/\1/p' "{{ pkg_file }}" | head -1) + if [ "$v" != "$declared" ]; then + echo " frontend: package.json declares '$v', VERSION declares '$declared'" >&2 + failed=1 + fi + + if [ "$failed" -ne 0 ]; then + echo "" >&2 + echo "Version drift. Run 'just version::bump' to move every file at once," >&2 + echo "or align them by hand if this was an intentional out-of-band edit." >&2 + exit 1 + fi + + echo "All versions agree on $declared." + +# Bump VERSION, propagate it to the components, commit, and tag. # Usage: just version::bump patch|minor|major [group('version')] bump part="patch": @@ -102,12 +155,31 @@ bump part="patch": echo "$next" >"{{ version_file }}" - # package.json is private and never published, but a stale version field - # there is a trap for anyone who reads it as authoritative. Written without - # `sed -i`, whose syntax differs between GNU and BSD. - tmp=$(mktemp) - sed "s/\"version\": \"$current\"/\"version\": \"$next\"/" "{{ pkg_file }}" >"$tmp" - mv "$tmp" "{{ pkg_file }}" + # Replace whatever version a file holds, not `$current`: bump must work even + # when the files already disagree. `awk` because `sed -i` differs GNU/BSD + # and cannot portably stop at the first match. + replace_first() { + local file="$1" pattern="$2" replacement="$3" tmp + tmp=$(mktemp) + awk -v pat="$pattern" -v rep="$replacement" ' + !done && $0 ~ pat { sub(pat, rep); done = 1 } + { print } + ' "$file" >"$tmp" + mv "$tmp" "$file" + } + + for comp in {{ image_comps }}; do + f="{{ root_dir }}/components/$comp/.component.yaml" + replace_first "$f" "^version: .*" "version: $next" + if ! grep -qx "version: $next" "$f"; then + echo "Failed to update the version field in $f." >&2 + exit 1 + fi + git -C "{{ root_dir }}" add "$f" + done + + # Never published, but a stale version here misleads anyone reading it. + replace_first "{{ pkg_file }}" "\"version\": \"[^\"]*\"" "\"version\": \"$next\"" if ! grep -q "\"version\": \"$next\"" "{{ pkg_file }}"; then echo "Failed to update the version field in {{ pkg_file }}." >&2 exit 1 @@ -119,6 +191,9 @@ bump part="patch": echo "" echo " VERSION $current -> $next" + for comp in {{ image_comps }}; do + echo " image $comp-service:$next" + done echo " commit chore(release): v$next" echo " tag v$next (annotated)" echo "" @@ -138,6 +213,10 @@ tag: exit 1 fi + # Pushing this tag triggers the release image build, and CI reads the + # component versions rather than this one. + just version::check + git -C "{{ root_dir }}" tag -a "v$declared" -m "v$declared" echo " tag v$declared (annotated) at $(git -C "{{ root_dir }}" rev-parse --short=7 HEAD)" echo " Push with: git push origin v$declared" diff --git a/tools/nix/hackagon/lib/toolchain.nix b/tools/nix/hackagon/lib/toolchain.nix index ec9364be..e181c045 100644 --- a/tools/nix/hackagon/lib/toolchain.nix +++ b/tools/nix/hackagon/lib/toolchain.nix @@ -448,6 +448,10 @@ let pkgs.hackagon.quitsh pkgs.podman + # helm chart deployment and testing + pkgs.kubernetes-helm + pkgs.skopeo + pkgs.openssh # SSH agent ]; }