diff --git a/.github/ci-cd.md b/.github/ci-cd.md index b2b26d95..279b192b 100644 --- a/.github/ci-cd.md +++ b/.github/ci-cd.md @@ -45,21 +45,76 @@ Woodpecker, Drone, GitLab CI, etc. ## Pipeline stages -Single `ci.yml` workflow triggered on PR and push to `main`: - -| # | Stage | Command | Purpose | -| --- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 1 | Setup Nix + cache | `cachix/install-nix-action` + `cachix/cachix-action` | Avoid rebuilding toolchains every run | -| 2 | Verify generated code is in sync | `just ci::codegen-check` | Catch PRs that forgot to regenerate stubs; uses `default` shell (buf + codegen tools not in CI shell); pnpm install required because `protoc-gen-ts_proto` is a node_modules binary | -| 3 | Format check | `just ci::run just check::format` | Backstop for developers who haven't opted into the githooks framework | -| 4 | Lint | `just ci::run bash -c "just check::lint -c backend && just check::lint -c frontend"` | Delegates to quitsh → golangci-lint, eslint, svelte-check, typos, yamllint | -| 5 | Build | `just ci::run bash -c "just build -c backend && just build -c frontend"` | All components via quitsh; catches compilation errors before running tests | -| 6 | Test | `just ci::run bash -c "just check::test -c backend && just check::test -c frontend"` | Go unit tests + Vitest; coverage upload per [codecov.yaml](../tools/configs/codecov/codecov.yaml) | -| 7 | Container images _(main only)_ | `nix build ./tools/nix#backend-service-dev` and `#frontend-service-dev` | Reproducible OCI images | -| 8 | Publish _(tag/main only)_ | Push images to registry; attach image digests or registry references to the GitHub release via `gh release edit` | Gated behind release trigger; links the release to its artifacts | - -Stages 2–6 are PR-blocking. Stage 7 runs on `main` only. Stage 8 only on tagged -releases. +Single `ci.yml` workflow triggered on PR, push to `main`, and `v*` tags: + +| # | Stage | Command | Purpose | +| --- | --------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------ | +| 1 | Setup Nix + cache | `install-nix-action` + `cachix-action` | Avoid rebuilding toolchains every run | +| 2 | Codegen in sync | `just ci::codegen-check` | Catch PRs that forgot to regenerate stubs; uses the `default` shell, where buf lives | +| 3 | Format check | `just ci::run just check::format` | Backstop for developers who haven't opted into the githooks framework | +| 4 | Lint | `just ci::run just check::lint -c ` | golangci-lint, eslint, svelte-check, typos, yamllint | +| 5 | Version consistency | `just ci::run just version::check` | `VERSION` of the app, both `.component.yaml` versions and `package.json` must agree | +| 6 | Helm chart lint | `just ci::run just helm::lint` | `helm lint` plus a full render — the chart cannot render with its own defaults | +| 7 | Chart version bumped _(PR)_ | `just ci::run just helm::check-bump` | A chart change without a version bump can never reach a cluster | +| 8 | Build | `just ci::run just check::build -c ` | All components via quitsh | +| 9 | Test | `just ci::run just check::test -c ` | Go unit tests + Vitest; coverage per [codecov.yaml](../tools/configs/codecov/codecov.yaml) | +| 10 | Container images | `quitsh image` (the `images` job) | PR: build only. `main`: push to `temporary/`. Tag: push to `release/` | +| 11 | Publish chart | `just helm::publish` | Pushes the chart when Chart.yaml holds an unpublished version; no-ops otherwise | + +Stages 2–9 are PR-blocking. Stage 10 builds on every PR and pushes on `main` and +on tags. Stage 11 runs on every push and does nothing unless the chart version +has moved. + +## Two release trains + +The app and the chart version are independent from each other: a chart fix with +no app change should not force an app release, and an app release should not +republish an unchanged chart. + +**The app** is versioned by `VERSION` at the repo root. `just version::bump` +moves it along with `.component.yaml` versions and `package.json`, commits, and +tags. Pushing that `v*` tag builds and pushes: + +``` +ghcr.io/swissdatasciencecenter/hackagon/release/backend-service:X.Y.Z +ghcr.io/swissdatasciencecenter/hackagon/release/frontend-service:X.Y.Z +``` + +Stage 5 refuses a tree where those files disagree, because the component version +_is_ the image tag. + +**The chart** is versioned by `helm-chart/Chart.yaml`, by hand: + +- `version` — the chart's own release. Bump it whenever anything under + `helm-chart/` changes; stage 7 fails the PR if you forget. +- `appVersion` — the app release this chart deploys. The deployment templates + fall back to it for the image tag, so it decides what a cluster actually runs. + +`just helm::publish` reads both from Chart.yaml and injects nothing. It pushes +to `oci://ghcr.io/swissdatasciencecenter/hackagon/charts/hackagon` and gives up +early in two cases, both reported and neither a failure: + +- **the chart version is already published** — the normal case on most pushes +- **`appVersion` has no published images** — the chart would not be installable + +The second is what makes "bump appVersion to an unreleased version" safe: the +chart waits, and the run that finally builds those images publishes it. + +Two consequences worth knowing: + +- **A new GHCR package is private by default.** The first `v*` tag creates + `release/*`, and the first chart publish creates `charts/hackagon`; until + someone sets those to public in the org's package settings, a cluster needs an + `imagePullSecret`. The existing `temporary/*` packages are already public. +- **A release image is never overwritten.** quitsh refuses to push a release tag + that already exists ([upload.go](../tools/quitsh/pkg/image/upload.go)), so + re-tagging a version fails rather than replacing an artifact someone has + deployed. Bump the version instead. + +To deploy the head of `main` rather than a release, override the image +repository and tag to the `temporary/` package — see the Images note in + +[helm-chart/values.yaml](../helm-chart/values.yaml). ## Parallelism @@ -88,10 +143,10 @@ Run the full CI pipeline locally from the repo root: just ci::all ``` -This mirrors stages 2–6 exactly as they run on GitHub Actions (generate check → -format → lint → build → test). Works from a bare shell or from a direnv-managed -shell — `ci::all` handles the Nix shell itself, so do **not** call it from -inside `nix develop`. +This mirrors stages 2–9 exactly as they run on GitHub Actions (generate check → +format → lint → version → helm → chart → build → test). Works from a bare shell +or from a direnv-managed shell — `ci::all` handles the Nix shell itself, so do +**not** call it from inside `nix develop`. Before pushing: `just ci::all` → if it passes locally, CI passes. @@ -120,8 +175,11 @@ Individual stages can also be isolated from the repo root: just ci::codegen-check # stage 2 — uses default shell just ci::run just check::format # stage 3 just ci::run just check::lint -c backend # stage 4 (one component) -just ci::run just build -c backend # stage 5 (one component) -just ci::run just check::test -c backend # stage 6 (one component) +just ci::run just version::check # stage 5 +just ci::run just helm::lint # stage 6 +just ci::run just helm::check-bump # stage 7 +just ci::run just build -c backend # stage 8 (one component) +just ci::run just check::test -c backend # stage 9 (one component) ``` On CI failure: copy the failing command from the workflow YAML, run it locally, diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index edc0bd7d..d348a395 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,6 +7,11 @@ on: tags: ["v*"] workflow_dispatch: +# Permissions default to read-only, so PRs from forks cannot push packages. +# The `images` jobs that publish packages overwrite this. +permissions: + contents: read + # Cancel in-progress runs on the same branch/PR to avoid wasted minutes. concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -18,7 +23,10 @@ jobs: timeout-minutes: 30 steps: + # `fetch-depth: 0` is required for `just version::check` to see the tags. - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + fetch-depth: 0 - uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27 with: @@ -63,6 +71,21 @@ jobs: just ci::run just check::lint -c backend just ci::run just check::lint -c frontend + # Make sure the version in AppVersion is consistent in all app components. + - name: Version consistency + run: just ci::run just version::check + + # Make sure the helm chart renders correctly and passes `helm lint`. + - name: Helm chart lint + run: just ci::run just helm::lint + + # Make sure the Chart version is raised when the helm chart changes. + - name: Helm chart version bumped + if: github.event_name == 'pull_request' + run: | + just ci::run just helm::check-bump \ + "origin/${{ github.event.pull_request.base.ref }}" + - name: Build run: | just ci::run just check::build -c backend @@ -81,6 +104,9 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 30 needs: [ci] + permissions: + contents: read + packages: write steps: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 @@ -135,3 +161,54 @@ jobs: --credential-user-env GH_USERNAME \ --credential-token-env GHCR_TOKEN fi + + # Publishes the chart as a versioned OCI artifact so a deployment can pin + # `oci://.../charts/hackagon --version X.Y.Z`. + # + # Deliberately not tag-driven. The chart carries its own version and its own + # appVersion, and the recipe reads both from Chart.yaml. + # + # An app release can be published without a chart bump, but a chart bump + # cannot be published without an app release. + # The `needs: [images]` ensures that the chart cannot be published unless + # the images for its appVersion are already published. + publish-chart: + if: github.event_name == 'push' + runs-on: ubuntu-24.04 + timeout-minutes: 30 + needs: [images] + permissions: + contents: read + packages: write + + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + + - uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27 + with: + extra_nix_config: | + experimental-features = nix-command flakes + + - uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15 + with: + name: hackagon + authToken: ${{ secrets.CACHIX_AUTH_TOKEN }} + + - name: Install bootstrap tools + run: nix profile install --accept-flake-config ./tools/nix#bootstrap + + # Two logins, two tools: helm pushes the chart, skopeo answers "is this + # already published?". They keep separate credential stores. + # Credentials are passed through env. + - name: Log in to GHCR + env: + GH_USERNAME: ${{ github.actor }} + GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + just ci::run bash -c ' + echo "$GHCR_TOKEN" | helm registry login ghcr.io -u "$GH_USERNAME" --password-stdin + echo "$GHCR_TOKEN" | skopeo login ghcr.io -u "$GH_USERNAME" --password-stdin + ' + + - name: Publish chart if its version is unpublished + run: just ci::run just helm::publish diff --git a/CLAUDE.md b/CLAUDE.md index 76ad8290..111993d8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -67,7 +67,9 @@ components/backend/ ├── internal/logx/logx.go # slog setup + Fatal helper └── Schema.md # human-readable DB reference (auto-generated) components/frontend/ # SvelteKit; generated gRPC clients under src/lib/server/grpc/generated/ +helm-chart/ # Helm chart; published as an OCI artifact on a v* tag mydocs/docs/backend-tickets/ # known gaps, one file per issue (README inside) +tools/helm/lint-values.yaml # throwaway values so the chart can be rendered in CI tools/nix/ # Nix flake + process-compose config (toolchain.nix) tools/just/*.just # just modules — see Dev commands justfile # root justfile, imports the modules above @@ -98,7 +100,13 @@ just check::lint -c backend # also: build, test, format; -c frontend just ci::all # everything CI runs, locally just version::show # declared version + what a build stamps in the footer -just version::bump patch|minor|major # edit VERSION, commit, annotated tag +just version::check # fail if a component version has drifted from VERSION +just version::bump patch|minor|major # edit VERSION everywhere, commit, annotated tag + +just helm::lint # helm lint + full render against tools/helm/lint-values.yaml +just helm::template [args] # render the chart to stdout +just helm::check-bump # fail if helm-chart/ changed without a version bump +just helm::publish # push the chart if Chart.yaml's version is unpublished ``` Backend listens on **:3000**, frontend on **:8081**. Dev users (Keycloak @@ -139,11 +147,20 @@ These hold across the whole codebase; the skills explain the mechanisms. running server — see **backend-api-explore**. Any inventory committed to a markdown file is a snapshot that starts rotting immediately, this one included. -- **`VERSION` at the repo root is the only declared version.** Bump it through - `just version::bump`, never by hand — the recipe is what also commits it, tags - it, and keeps `components/frontend/package.json` in step. The frontend reads - it at build time (`vite.config.ts` → `$lib/version` → the footer), so a - version edited without a rebuild will not show up. +- **`VERSION` at the repo root is the app's only declared version.** Bump it + with `just version::bump`, never by hand — the recipe moves `VERSION`, both + `components/*/.component.yaml` versions and `components/frontend/package.json` + together, then commits and tags. A component version _is_ its published image + tag, so `just version::check` (a CI stage) fails a tree where these disagree. + The frontend reads `VERSION` at build time (`vite.config.ts` → `$lib/version` + → the footer), so editing it without a rebuild changes nothing. +- **The chart versions itself; `VERSION` never touches it.** + `helm-chart/Chart.yaml` holds two hand-edited numbers: `version` is the + chart's own release, `appVersion` is the app release it deploys. + `just helm::publish` reads both and injects nothing, so a chart is published + only when its own `version` changes — never as a side effect of an app + release. It skips without failing when that version is already published, or + when `appVersion` has no images yet. ## Don't diff --git a/VERSION b/VERSION index 6e8bf73a..a3df0a69 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.0 +0.8.0 diff --git a/components/backend/.component.yaml b/components/backend/.component.yaml index 62ce0501..29fd63e3 100644 --- a/components/backend/.component.yaml +++ b/components/backend/.component.yaml @@ -1,5 +1,5 @@ name: backend -version: 0.1.0 +version: 0.8.0 language: go targets: diff --git a/components/frontend/.component.yaml b/components/frontend/.component.yaml index 29985ca0..74905662 100644 --- a/components/frontend/.component.yaml +++ b/components/frontend/.component.yaml @@ -1,5 +1,5 @@ name: frontend -version: 0.7.0 +version: 0.8.0 language: typescript targets: diff --git a/components/frontend/package.json b/components/frontend/package.json index 76fdd1d6..c9831e68 100644 --- a/components/frontend/package.json +++ b/components/frontend/package.json @@ -1,7 +1,7 @@ { "name": "frontend-v3", "private": true, - "version": "0.0.1", + "version": "0.8.0", "type": "module", "scripts": { "dev": "vite dev", diff --git a/helm-chart/Chart.yaml b/helm-chart/Chart.yaml index 619bcb6e..ee972225 100644 --- a/helm-chart/Chart.yaml +++ b/helm-chart/Chart.yaml @@ -4,8 +4,12 @@ description: A Helm chart for Hackagon on Kubernetes type: application -version: 0.1.0 -appVersion: "1.0.0" +# The chart's own version. Bumped by hand when the chart changes, and +# independent of the app: the CI publishes whatever it finds here. +version: 0.2.0 +# The app release this chart deploys. A new app release does +# not become deployable until someone points the chart at it. +appVersion: "0.8.0" dependencies: - name: postgresql diff --git a/helm-chart/templates/backend-deployment.yaml b/helm-chart/templates/backend-deployment.yaml index f543ab46..025ef7bf 100644 --- a/helm-chart/templates/backend-deployment.yaml +++ b/helm-chart/templates/backend-deployment.yaml @@ -24,7 +24,7 @@ spec: spec: containers: - name: backend - image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}" + image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.backend.image.pullPolicy }} args: - "--config-dir=/etc/hackagon/" diff --git a/helm-chart/templates/frontend-deployment.yaml b/helm-chart/templates/frontend-deployment.yaml index c31115ab..1a549966 100644 --- a/helm-chart/templates/frontend-deployment.yaml +++ b/helm-chart/templates/frontend-deployment.yaml @@ -24,7 +24,7 @@ spec: spec: containers: - name: frontend - image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }}" + image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.frontend.image.pullPolicy }} args: - "--config-dir=/etc/hackagon" diff --git a/helm-chart/values.yaml b/helm-chart/values.yaml index ecb154da..a1810889 100644 --- a/helm-chart/values.yaml +++ b/helm-chart/values.yaml @@ -4,13 +4,31 @@ baseDomain: "example.com" # -- Keycloak realm JSON (pass via --set-file: --set-file realmJson=@./tools/configs/keycloak/realm-hackagon.json) # realmJson: "" +# ============================================================ +# Images +# ============================================================ +# An image address is `repository:tag`. Two repositories exist: +# release/ one image per `v*` tag — what you deploy +# temporary/ one image per push to main — for testing, moves constantly +# +# If there is no `tag` specified: then the chart then uses its the +# `appVersion` from Chart.yaml, +# +# But for test deploys you might want to override this, with a +# tagged image from the `temporary/` repository. +# +# To test against the head of `main`, override both keys, e.g.: +# --set frontend.image.repository=/temporary/frontend-service +# --set frontend.image.tag=latest +# # ============================================================ # Frontend # ============================================================ frontend: image: - repository: ghcr.io/swissdatasciencecenter/hackagon/temporary/frontend-service - tag: "latest" + repository: ghcr.io/swissdatasciencecenter/hackagon/release/frontend-service + # Empty: falls back to .Chart.AppVersion — see the Images note above. + tag: "" pullPolicy: IfNotPresent replicaCount: 1 @@ -67,8 +85,9 @@ frontend: # ============================================================ backend: image: - repository: ghcr.io/swissdatasciencecenter/hackagon/temporary/backend-service - tag: "latest" + repository: ghcr.io/swissdatasciencecenter/hackagon/release/backend-service + # Empty: falls back to .Chart.AppVersion — see the Images note above. + tag: "" pullPolicy: IfNotPresent replicaCount: 1 diff --git a/justfile b/justfile index 3101a76e..6e4ab262 100644 --- a/justfile +++ b/justfile @@ -14,6 +14,7 @@ mod deploy "./tools/just/deploy.just" mod codegen "./tools/just/codegen.just" mod clean "./tools/just/clean.just" mod version "./tools/just/version.just" +mod helm "./tools/just/helm.just" [private] default: diff --git a/tools/helm/lint-values.yaml b/tools/helm/lint-values.yaml new file mode 100644 index 00000000..a856756d --- /dev/null +++ b/tools/helm/lint-values.yaml @@ -0,0 +1,29 @@ +# Throwaway values, used only to make the chart renderable so `helm lint` and +# `helm template` can check it. +# +# Used by `just helm::lint`. + +baseDomain: lint.invalid + +backend: + config: + server: + adminkeycloakid: "00000000-0000-0000-0000-000000000000" + database: + postgresPassword: "placeholder-not-a-real-password" + +keycloak: + hostname: + hostname: "https://auth.lint.invalid" + database: + external: + host: "hackagon-postgresql" + password: "placeholder-not-a-real-password" + +postgresql: + auth: + postgresPassword: "placeholder-not-a-real-password" + +frontendSecrets: + clientSecret: "placeholder-not-a-real-secret" + authSecret: "placeholder-not-a-real-secret" diff --git a/tools/just/ci.just b/tools/just/ci.just index 222dce8f..1d048f20 100644 --- a/tools/just/ci.just +++ b/tools/just/ci.just @@ -62,6 +62,9 @@ all: "codegen | just ci::codegen-check" "format | just ci::run just check::format" "lint | just ci::run bash -c 'just check::lint -c backend && just check::lint -c frontend'" + "version | just ci::run just version::check" + "helm | just ci::run just helm::lint" + "chart | just ci::run just helm::check-bump" "build | just ci::run bash -c 'just build -c backend && just build -c frontend'" "test | just ci::run bash -c 'just check::test -c backend && just check::test -c frontend'" ) diff --git a/tools/just/helm.just b/tools/just/helm.just new file mode 100644 index 00000000..c2704397 --- /dev/null +++ b/tools/just/helm.just @@ -0,0 +1,197 @@ +set positional-arguments +set shell := ["bash", "-cue"] + +root_dir := `git rev-parse --show-toplevel` +chart_dir := root_dir + "/helm-chart" +chart_file := chart_dir + "/Chart.yaml" +charts_dir := chart_dir + "/charts" +out_dir := root_dir + "/.output/helm" +lint_values := root_dir + "/tools/helm/lint-values.yaml" + +registry := "ghcr.io/swissdatasciencecenter/hackagon" +chart_ref := registry + "/charts/hackagon" +oci_repo := "oci://" + registry + "/charts" + +[private] +default: + just --list --unsorted -f "{{ source_file() }}" + +[group('helm')] +help: + #!/usr/bin/env bash + bold="\033[1m" + dim="\033[2m" + cyan="\033[36m" + reset="\033[0m" + + echo "" + echo -e " ${bold}Helm Commands${reset}" + echo "" + echo -e " ${cyan}just helm::lint${reset}" + echo -e " ${dim}Lint the chart and render it against tools/helm/lint-values.yaml${reset}" + echo "" + echo -e " ${cyan}just helm::template${reset} [helm args...]" + echo -e " ${dim}Render the chart manifests to stdout${reset}" + echo "" + echo -e " ${cyan}just helm::check-bump${reset} [base]" + echo -e " ${dim}Fail if helm-chart/ changed without a Chart.yaml version bump${reset}" + echo "" + echo -e " ${cyan}just helm::publish${reset}" + echo -e " ${dim}Publish the chart if its version is not already in the registry${reset}" + echo "" + echo -e " ${cyan}just helm::clean${reset}" + echo -e " ${dim}Remove fetched subchart archives and packaged charts${reset}" + echo "" + echo -e " ${bold}Two versions, two purposes${reset} ${dim}(helm-chart/Chart.yaml)${reset}" + echo -e " ${dim}version the chart's own release, bumped when the chart changes${reset}" + echo -e " ${dim}appVersion the app release the chart deploys${reset}" + echo -e " ${dim}Both are edited by hand. An app tag publishes images, never a chart.${reset}" + echo "" + echo -e " ${dim}The chart cannot render with its own defaults — several values${reset}" + echo -e " ${dim}(admin id, passwords, hostnames) are intentionally empty.${reset}" + echo "" + +# Remove fetched subchart archives and packaged charts. +[group('helm')] +clean: + rm -rf "{{ charts_dir }}" "{{ out_dir }}" + +# Fetch the subchart archives the chart declares. +[private] +[group('helm')] +fetch-deps: + #!/usr/bin/env bash + set -eu + + # `--force-update` keeps this idempotent: a plain `repo add` fails on a name + # that already exists. + helm repo add bitnami https://charts.bitnami.com/bitnami --force-update + helm repo add helmforge https://repo.helmforge.dev --force-update + + cd "{{ chart_dir }}" + # `build`, not `update`: build obeys the committed Chart.lock, so CI resolves + # the same subchart versions a developer does. `update` re-resolves the + # `18.x`/`3.x` ranges and moves them silently. + helm dependency build + +# Lint the chart and render it against throwaway values. +[group('helm')] +lint: fetch-deps + #!/usr/bin/env bash + set -eu + + helm lint "{{ chart_dir }}" --values "{{ lint_values }}" + + # `lint` does not catch a template that fails to render, so render it too. + helm template hackagon-lint "{{ chart_dir }}" --values "{{ lint_values }}" >/dev/null + +# Render the chart manifests to stdout. +# Usage: just helm::template --values my-values.yaml +[group('helm')] +[no-cd] +template *args: fetch-deps + #!/usr/bin/env bash + set -eu + helm template "$@" "{{ chart_dir }}" + +# Fail if anything under helm-chart/ changed without a Chart.yaml version bump. +# The chart is published from whatever version Chart.yaml holds, so an unbumped +# change is one that can never reach a cluster. +[group('helm')] +check-bump base="origin/main": + #!/usr/bin/env bash + set -eu + cd "{{ root_dir }}" + + if ! git rev-parse --verify -q "{{ base }}" >/dev/null; then + echo "Unknown base ref '{{ base }}'. In CI the checkout needs full history" >&2 + echo "(fetch-depth: 0); locally try 'just helm::check-bump main'." >&2 + exit 1 + fi + + base=$(git merge-base "{{ base }}" HEAD) + + if [ -z "$(git diff --name-only "$base" HEAD -- helm-chart/)" ]; then + echo "No chart changes against {{ base }}; nothing to check." + exit 0 + fi + + read_version() { sed -n 's/^version: *//p' | head -1; } + before=$(git show "$base:helm-chart/Chart.yaml" 2>/dev/null | read_version || true) + now=$(read_version <"{{ chart_file }}") + + if [ -n "$before" ] && [ "$before" = "$now" ]; then + echo "helm-chart/ changed but Chart.yaml still declares version $now." >&2 + echo "" >&2 + git diff --name-only "$base" HEAD -- helm-chart/ | sed 's/^/ /' >&2 + echo "" >&2 + echo "Bump 'version' in helm-chart/Chart.yaml. It is the chart's own" >&2 + echo "release number, not the app's — see just helm::help." >&2 + exit 1 + fi + + echo "Chart version $before -> $now." + +# Publish the chart to GHCR as an OCI artifact, if it is not already there. +# Both versions come from Chart.yaml and nothing is injected, so an app release +# never publishes a chart on its own — someone has to bump the chart first. +# Assumes `helm registry login` and `skopeo login` have already happened. +[group('helm')] +publish: + #!/usr/bin/env bash + set -eu + + chart_v=$(sed -n 's/^version: *//p' "{{ chart_file }}" | head -1) + app_v=$(sed -n 's/^appVersion: *"*\([^"]*\)"*/\1/p' "{{ chart_file }}" | head -1) + + if ! [[ "$chart_v" =~ ^[0-9]+\.[0-9]+\.[0-9]+ ]]; then + echo "Chart.yaml version '$chart_v' is not a semver." >&2 + exit 1 + fi + if [ -z "$app_v" ]; then + echo "Chart.yaml declares no appVersion." >&2 + exit 1 + fi + + exists() { skopeo inspect --raw "docker://$1" >/dev/null 2>&1; } + + if exists "{{ chart_ref }}:$chart_v"; then + echo "Chart $chart_v is already published; nothing to do." + echo "Bump 'version' in helm-chart/Chart.yaml to publish a new one." + exit 0 + fi + + # A published chart must be installable. Its image tags default to + # appVersion, so refuse to promise an app release that was never built. + for comp in backend frontend; do + ref="{{ registry }}/release/$comp-service:$app_v" + if ! exists "$ref"; then + echo "Not publishing chart $chart_v: appVersion $app_v has no images." >&2 + echo " missing: $ref" >&2 + echo "Push the v$app_v tag first, or point appVersion at a released version." >&2 + exit 0 + fi + done + + just helm::fetch-deps + + mkdir -p "{{ out_dir }}" + helm package "{{ chart_dir }}" --destination "{{ out_dir }}" + + pkg="{{ out_dir }}/hackagon-${chart_v}.tgz" + if [ ! -f "$pkg" ]; then + echo "Expected package '$pkg' was not produced." >&2 + exit 1 + fi + + helm push "$pkg" "{{ oci_repo }}" + + echo "" + echo " chart $chart_v" + echo " deploys app $app_v" + echo " pushed {{ chart_ref }}:$chart_v" + echo "" + echo " Install with:" + echo " helm install hackagon {{ oci_repo }}/hackagon --version $chart_v \\" + echo " --set baseDomain=example.com --set-file realmJson=@path/to/realm.json" + echo "" diff --git a/tools/just/version.just b/tools/just/version.just index 2ff4637e..309c8ebd 100644 --- a/tools/just/version.just +++ b/tools/just/version.just @@ -5,6 +5,11 @@ root_dir := `git rev-parse --show-toplevel` version_file := root_dir + "/VERSION" pkg_file := root_dir + "/components/frontend/package.json" +# Components whose `.component.yaml` version becomes an OCI image tag. Not +# `tools/quitsh`: vendored, carries its own upstream version, builds no image. +# The Helm chart is deliberately absent — see helm.just. +image_comps := "backend frontend" + [private] default: just --list --unsorted -f "{{ source_file() }}" @@ -23,14 +28,17 @@ help: echo -e " ${cyan}just version::show${reset}" echo -e " ${dim}Print the declared version and what a build stamps into the footer${reset}" echo "" + echo -e " ${cyan}just version::check${reset}" + echo -e " ${dim}Fail if any component version has drifted from VERSION${reset}" + echo "" echo -e " ${cyan}just version::bump${reset} patch|minor|major" - echo -e " ${dim}Edit VERSION, commit it, and create the matching annotated tag${reset}" + echo -e " ${dim}Edit VERSION, propagate it to every component, commit, and tag${reset}" echo "" echo -e " ${cyan}just version::tag${reset}" echo -e " ${dim}Tag HEAD with the version already declared in VERSION${reset}" echo "" - echo -e " ${dim}VERSION at the repo root is the single source of truth. The${reset}" - echo -e " ${dim}frontend reads it at build time — see components/frontend/vite.config.ts.${reset}" + echo -e " ${dim}VERSION covers the app only: the per-component copies become the${reset}" + echo -e " ${dim}published image tags. The Helm chart versions itself — see just helm::help.${reset}" echo "" # Print the declared version and the string a build stamps into the footer. @@ -64,7 +72,52 @@ show: echo "build $build" echo "tags $(git -C "{{ root_dir }}" tag --list 'v*' | tr '\n' ' ' | sed 's/ $//' || true)" -# Bump VERSION, commit it, and create the matching annotated tag. + echo "images" + for comp in {{ image_comps }}; do + v=$(sed -n 's/^version: *//p' "{{ root_dir }}/components/$comp/.component.yaml" | head -1) + marker="" + if [ "$v" != "$declared" ]; then + marker=" <- drifted from VERSION" + fi + echo " $comp-service:$v$marker" + done + +# Fail if VERSION, the component versions and package.json disagree. +# The component version is the published image tag, so a drifted tree releases +# an image named after the wrong version — and a second such release collides +# with the existing tag and is refused (tools/quitsh/pkg/image/upload.go). +[group('version')] +check: + #!/usr/bin/env bash + set -eu + declared=$(tr -d '[:space:]' <"{{ version_file }}") + failed=0 + + for comp in {{ image_comps }}; do + f="{{ root_dir }}/components/$comp/.component.yaml" + v=$(sed -n 's/^version: *//p' "$f" | head -1) + if [ "$v" != "$declared" ]; then + echo " $comp: .component.yaml declares '$v', VERSION declares '$declared'" >&2 + failed=1 + fi + done + + v=$(sed -n 's/.*"version": *"\([^"]*\)".*/\1/p' "{{ pkg_file }}" | head -1) + if [ "$v" != "$declared" ]; then + echo " frontend: package.json declares '$v', VERSION declares '$declared'" >&2 + failed=1 + fi + + if [ "$failed" -ne 0 ]; then + echo "" >&2 + echo "Version drift. Run 'just version::bump' to move every file at once," >&2 + echo "or align them by hand if this was an intentional out-of-band edit." >&2 + exit 1 + fi + + echo "All versions agree on $declared." + +# Bump VERSION, propagate it to the components, commit, and tag. # Usage: just version::bump patch|minor|major [group('version')] bump part="patch": @@ -102,12 +155,31 @@ bump part="patch": echo "$next" >"{{ version_file }}" - # package.json is private and never published, but a stale version field - # there is a trap for anyone who reads it as authoritative. Written without - # `sed -i`, whose syntax differs between GNU and BSD. - tmp=$(mktemp) - sed "s/\"version\": \"$current\"/\"version\": \"$next\"/" "{{ pkg_file }}" >"$tmp" - mv "$tmp" "{{ pkg_file }}" + # Replace whatever version a file holds, not `$current`: bump must work even + # when the files already disagree. `awk` because `sed -i` differs GNU/BSD + # and cannot portably stop at the first match. + replace_first() { + local file="$1" pattern="$2" replacement="$3" tmp + tmp=$(mktemp) + awk -v pat="$pattern" -v rep="$replacement" ' + !done && $0 ~ pat { sub(pat, rep); done = 1 } + { print } + ' "$file" >"$tmp" + mv "$tmp" "$file" + } + + for comp in {{ image_comps }}; do + f="{{ root_dir }}/components/$comp/.component.yaml" + replace_first "$f" "^version: .*" "version: $next" + if ! grep -qx "version: $next" "$f"; then + echo "Failed to update the version field in $f." >&2 + exit 1 + fi + git -C "{{ root_dir }}" add "$f" + done + + # Never published, but a stale version here misleads anyone reading it. + replace_first "{{ pkg_file }}" "\"version\": \"[^\"]*\"" "\"version\": \"$next\"" if ! grep -q "\"version\": \"$next\"" "{{ pkg_file }}"; then echo "Failed to update the version field in {{ pkg_file }}." >&2 exit 1 @@ -119,6 +191,9 @@ bump part="patch": echo "" echo " VERSION $current -> $next" + for comp in {{ image_comps }}; do + echo " image $comp-service:$next" + done echo " commit chore(release): v$next" echo " tag v$next (annotated)" echo "" @@ -138,6 +213,10 @@ tag: exit 1 fi + # Pushing this tag triggers the release image build, and CI reads the + # component versions rather than this one. + just version::check + git -C "{{ root_dir }}" tag -a "v$declared" -m "v$declared" echo " tag v$declared (annotated) at $(git -C "{{ root_dir }}" rev-parse --short=7 HEAD)" echo " Push with: git push origin v$declared" diff --git a/tools/nix/hackagon/lib/toolchain.nix b/tools/nix/hackagon/lib/toolchain.nix index ec9364be..e181c045 100644 --- a/tools/nix/hackagon/lib/toolchain.nix +++ b/tools/nix/hackagon/lib/toolchain.nix @@ -448,6 +448,10 @@ let pkgs.hackagon.quitsh pkgs.podman + # helm chart deployment and testing + pkgs.kubernetes-helm + pkgs.skopeo + pkgs.openssh # SSH agent ]; }