ci(gha): adopt chart-testing for chart CI (lint + install)#95
Open
etgraylog wants to merge 16 commits into
Open
ci(gha): adopt chart-testing for chart CI (lint + install)#95etgraylog wants to merge 16 commits into
etgraylog wants to merge 16 commits into
Conversation
…tup-python, bump checkout/setup-helm, drop deprecated token)
…o helm-ct-install
24 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the bare
helm lintworkflow with a Helm chart testing tool ct-based CI flow that runs:ct lint(Chart.yaml schema, maintainer/icon checks, version-bump enforcement, yamllint over Chart.yaml + values.yaml + ci/overlays)ct install(kind cluster + MongoDB Kubernetes Operator +helm install+helm test), with a Kubernetes version matrix covering v1.32.11, v1.33.7, and v1.34.3 — the chart's stated minimum and recent stable versions perdocs/TESTING.md.Every PR against
mainnow automatically validates that the chart renders cleanly and deploys correctly on each supported Kubernetes version, with all 5 existinghelm testpods passing. Two pre-existing yamllint violations surfaced by the new linter are also fixed so the new workflow lands green.Details
Two-job workflow architecture
helm-ct-lintjob — runsct lint --allagainst the chart. ~10–15 seconds.helm-ct-installjob — runsct install --allagainst a kind cluster with the MongoDB Operator pre-installed. Matrix over three Kubernetes versions (v1.32.11, v1.33.7, v1.34.3), each in parallel. ~5–6 minutes wall time per matrix variant. Gated onhelm-ct-lintpassing (needs: helm-ct-lint) so the longer install only runs when lint is clean.Multi-Kubernetes version testing
The
helm-ct-installjob uses astrategy.matrixover Kubernetes versions, each variant spinning up its own kind cluster on a differentkindest/nodeimage.fail-fast: falseensures all three versions run to completion even if one fails, so reviewers see which versions are affected. The matrix covers the range stated indocs/TESTING.md("Kubernetes 1.32+"):v1.32.11— stated minimum supported versionv1.33.7— mid-range stablev1.34.3— recent stablev1.35.0is intentionally not included to keep the matrix at three parallel runners. Adding it later is a one-line change.What
ct installcoversOn every PR, each matrix variant's kind cluster gets a fresh chart install using the minimal-resource overlay at
charts/graylog/ci/ci-values.yaml. After install, ct runshelm testautomatically, exercising all 5 existing test pods atcharts/graylog/templates/tests/:test-credentials-secrettest-mongodb-connectivitytest-datanode-registrationtest-graylog-api-healthtest-graylog-cluster-statusThese test pods were previously exercised only manually per
docs/TESTING.mdPhase 3 — they are now enforced on every PR against three Kubernetes versions in parallel.ct.yamlvalidation policy.github/ct.yamlexplicitly enables ct's validation knobs (rather than relying on default values) so the chart's CI policy is self-documenting:check-version-increment: true— enforceChart.yamlversion bumps on chart changes (gated by ct's--allflag in our workflow, so currently informational; takes effect if--allis dropped)validate-chart-schema: true—Chart.yamlschema validationvalidate-yaml: true— yamllint overChart.yaml,values.yaml, andci/*-values.yamlupgrade: false— in-place upgrade testing is disabled. ct's--upgradedoesn't exercisehelm upgradeagainst an existing deployed release the way users would expect; it installs the previous revision into an ephemeral namespace, upgrades to current within that same ephemeral namespace, then tears down. Combined with the ~2x CI runtime cost it adds, not worth enabling right now. Configuration kept visible so it's easy to re-enable later if ct semantics change or maintainers want to opt in.Ephemeral rootPassword handling
The chart generates a random
rootPasswordwhen none is provided in values. To avoid leaking that random value into the public CI log (via NOTES.txt rendering), the workflow:openssl rand -hex 16.::add-mask::BEFOREct installruns, so all subsequent log output masks the value.ct install --helm-extra-set-args "--set graylog.config.rootPassword=<value>".Result: NOTES.txt's
EXTERNAL ACCESS → password: <value>renders as***in logs, and the "ADDITIONAL NOTES → randomly generated password" warning block is suppressed (becauserootPasswordis explicitly set via--set, the chart's{{- else if empty }}branch doesn't render).New files / modifications
.github/ct.yaml(new) — chart-testing config (chart-dirs, target-branch, validate-maintainers, explicit validation defaults,upgrade: falsewith documented rationale, helm timeout).charts/graylog/ci/ci-values.yaml(new) — minimal-resource overlay for thehelm-ct-installjob. Scales the chart down to fit a default GitHub-hosted runner (~7 GB RAM, 4 vCPU): single replica of each tier, reduced JVM heaps, MongoDBv8.0.23..github/workflows/lint-and-test.yaml— replaced. Two-job workflow with kind cluster setup, MongoDB Operator install, ephemeral rootPassword generation + masking, ct lint + ct install steps, and Kubernetes version matrix.MongoDB version override (workaround for #89)
The ci-values overlay pins
mongodb.version: "8.0.23". The chart's default is"7.0.25", which is the exact version affected by issue #89 ("MongoDB Version 7.0 silently fails"). Pinning to8.0.23in CI ensures the install completes cleanly; the chart's default value is unchanged and remains in scope for the #89 fix.Action version pinning (Node 24 readiness)
actions/checkout@v6(was v4)azure/setup-helm@v5(was v4; deprecatedtoken:input dropped)helm/chart-testing-action@v2.8.0(defaults to ct 3.14.0)helm/kind-action@v1.14.0All on Node 24-supporting versions, ahead of the June 2026 Node 20 runtime sunset.
yamllint violations fixed
charts/graylog/Chart.yaml— added missing trailing newline (rule:new-line-at-end-of-file).charts/graylog/values.yaml— RBAC role-rules block (lines 344-351): removed inner spaces in brackets ([ "" ]→[""]) per rulebrackets, added missing trailing newline.Linked issues
None directly. This converts
docs/TESTING.mdPhases 1-3 (Static Validation, Installation, Automated Test Suite) from documented-manual to enforced-by-CI, and adds Kubernetes version-matrix testing for the chart's stated supported range.PR Checklist
Testing Checklist
Static Validation
helm lint ./charts/grayloghelm template graylog ./charts/graylog --validateInstallation
helm install graylog ./charts/grayloghelm test graylogFunctional (if applicable)
Upgrade (if applicable)
Specific to this PR
ct lint --config .github/ct.yaml --allpasses locally against this branch.ct install --config .github/ct.yaml --allvalidated end-to-end via the PR-attached CI run, covering the chart install + all 5 helm test pods against Kubernetes v1.32.11, v1.33.7, and v1.34.3 in parallel.EXTERNAL ACCESSsection renderspassword: ***in workflow logs, and the "ADDITIONAL NOTES randomly generated password" warning block is correctly suppressed.26732173551, conclusionsuccess. All four jobs green:helm-ct-lintandhelm-ct-installmatrix variants for K8sv1.32.11,v1.33.7,v1.34.3.Notes for reviewers
charts/graylog/templates/tests/remain in place; they're now exercised byct installon every PR, against three Kubernetes versions. No changes to those files in this PR.ci-values.yaml(mongodb.version: "8.0.23") is a workaround for MongoDB Version 7.0 silently fails #89; can be removed when the chart's default (7.0.25) is updated post-MongoDB Version 7.0 silently fails #89 resolution. The chart's actual default value is unchanged.upgrade: truewas considered and explicitly disabled. ct's--upgradedoesn't exercisehelm upgradeagainst an existing deployed release in the way users/maintainers would expect — it installs the previous revision into an ephemeral namespace, upgrades to current within that same ephemeral namespace, and tears down. Combined with the ~2x CI runtime cost, not worth enabling now. The configuration line is kept visible in.github/ct.yamlso re-enablement is a single-character change if maintainers want it later.ct installcovershelm install+helm test, but not browser-driven UI verification. UI functional testing remains manual perdocs/TESTING.mdPhase 4 and could be addressed in a follow-up PR.v1.35.0is not in the K8s matrix. We chose three versions (1.32 minimum, 1.33 mid, 1.34 recent) to keep the matrix at three parallel runners. Easy to extend withv1.35.0(and beyond) askindest/nodereleases.astral-sh/setup-uv@v7.3.0(invoked internally byhelm/chart-testing-action@v2.8.0) emits "No file matched to ... The cache will never get invalidated" because this repo has no Python dependency files for the action's defaultcache-dependency-globto hash. Informational only, doesn't affect functionality. Not addressable from our workflow without forkingchart-testing-actionor replacing it with manual setup. Could file an upstream issue againsthelm/chart-testing-actionto exposeenable-cache(or similar) as a forwarded input.