Skip to content

test(unit): add helm-unittest layer for template-rendering regression tests#96

Open
etgraylog wants to merge 18 commits into
Graylog2:mainfrom
etgraylog:ci/helm-unittest
Open

test(unit): add helm-unittest layer for template-rendering regression tests#96
etgraylog wants to merge 18 commits into
Graylog2:mainfrom
etgraylog:ci/helm-unittest

Conversation

@etgraylog
Copy link
Copy Markdown

@etgraylog etgraylog commented Jun 1, 2026

⚠️ Stacked on #95. Once #95 merges, this branch will rebase onto main and the diff will reduce to the helm-unittest-specific changes (2 commits below). Until then, the PR's commit list shows #95's commits as well.

Summary

Add a helm-unittest layer to the chart's CI: a new helm-unittest job in the existing lint-and-test.yaml workflow that runs helm unittest against five starter test suites covering high-leverage conditional rendering across the chart. Complements #95's ct install coverage with fast, cluster-free template-rendering regression tests — runs in ~200 ms vs. ~6 min for ct install, exercising values combinations that ct install doesn't.

Why add helm-unittest on top of ct install?

#95's ct install provides strong end-to-end validation: chart installs, MongoDB Operator is in place, helm test pods pass against multiple Kubernetes versions. But it tests one specific values configuration (the minimal-resource overlay in charts/graylog/ci/ci-values.yaml). The chart has substantial conditional template logic — TLS on/off, plugins on/off, bundled vs. BYO MongoDB, ingress configurations, cert-manager annotation paths — that ct install doesn't exercise.

helm-unittest's distinct value:

  • Combinatorial values coverage. Each test sets specific values and asserts on the rendered output. Adding a values combination is a few lines, not another full helm install cycle.
  • Negative assertions. "When TLS is disabled, the StatefulSet should NOT mount the tls-creds volume" — easy in helm-unittest, hard in install testing.
  • Exact-value assertions. Specific replicas counts, port numbers, annotation values — directly asserted.
  • Speed. ~200 ms for 38 tests in the sandbox. Tight inner loop for chart authors.

Details

Workflow change

A new helm-unittest job in .github/workflows/lint-and-test.yaml:

  • Helm version matrix (v3.16.4 + v4.2.0) — exercises template-rendering logic across both supported Helm major versions
  • helm-unittest plugin pinned to v1.0.3 — latest release whose plugin.yaml does NOT use the platformHooks field (added in v1.1.0+ which Helm 3.16.4 rejects during plugin.yaml parsing)
  • Cloned from the v1.0.3 git tag explicitly, not via helm plugin install --version (which clones the default branch, not a specific tag)
  • Local-path install also sidesteps Helm 4's signature verification flow

The job runs in parallel with helm-ct-lint (no needs: between them — both are fast, independent signals).

Updated dependency graph

The existing helm-ct-install job now gates on both helm-ct-lint AND helm-unittest:

helm-ct-lint ─┐
├─→ helm-ct-install
helm-unittest ─┘

Install (~6 min × matrix) only starts if BOTH lint and unittest pass. Saves CI compute on PRs that fail either of the fast gates.

Starter test suites (5 files, 38 tests)

File Test count What it validates
charts/graylog/tests/service_test.yaml 7 Service type defaults / overrides, metrics port conditional, graylog.inputs port iteration, the always-present input-fwd-conf port
charts/graylog/tests/graylog_statefulset_test.yaml 6 graylog.replicas flows through, TLS volume + volumeMount conditional rendering, plugins emptyDir volume conditional
charts/graylog/tests/init_graylog_test.yaml 8 init-script.sh logic: MongoDB URI credential check always present, truststore keytool -importcert block gated on TLS + updateKeyStore, plugin-copy logic gated on graylog.config.plugins.enabled
charts/graylog/tests/mongodb_test.yaml 8 MongoDBCommunity CR rendering gated on mongodb.communityResource.enabled, version pin from mongodb.version, members/arbiters counts, arbiters >= replicas template failure, persistence sizing
charts/graylog/tests/ingress_test.yaml 9 Ingress gated on both ingress.enabled AND ingress.web.enabled, TLS section rendering, cert-manager cluster-issuer vs. issuer annotations

These are intentionally focused on conditional logic the chart actually has, not trivial templates. Templates like auth/sa.yaml and policy/pdb/graylog.yaml are deliberately not covered — simple enough that ct install is sufficient. Future PRs can extend the pattern.

Notes on template dependencies (for future test authors)

The Graylog StatefulSet template includes graylog.configChecksum and graylog.secretsChecksum helpers, which themselves include config/graylog.yaml and config/secret/secrets.yaml. helm-unittest's templates: filter must list those dependencies for the includes to resolve. The StatefulSet test suite uses template: field on each assertion to scope assertions to the StatefulSet specifically while still rendering the dependency templates.

Commit history

2 commits:

ci(gha): add helm-unittest job to lint-and-test workflow
test(unit): add starter helm-unittest suites (service, statefulset, init-graylog, mongodb, ingress)

Linked issues

None. Builds on the testing infrastructure introduced in #95.

PR Checklist

  • Tests added/updated
  • Documentation updated
  • This PR includes a new feature
  • This PR includes a bugfix
  • This PR includes a refactor

Testing Checklist

Static Validation

  • Linter check passes: helm lint ./charts/graylog
  • Helm renders local template successfully: helm template graylog ./charts/graylog --validate

Installation

Functional (if applicable)

  • Web UI accessible and login works
  • DataNodes visible in System > Cluster Configuration
  • Inputs can be created and receive data

Upgrade (if applicable)

  • Upgrade from previous release succeeds
  • Scaling up/down works correctly
  • Configuration changes apply correctly

Specific to this PR

  • helm unittest charts/graylog passes locally on Helm v4.2.0: 5 test suites, 38 tests, ~200 ms.
  • Workflow CI validated all 8 jobs green on the PR-attached run 26734982194: both helm-ct-lint Helm variants, both helm-unittest Helm variants, and all 4 helm-ct-install (Helm × K8s) variants.
  • helm-ct-install correctly gates on both helm-ct-lint AND helm-unittest — install starts only after the fast gates pass.

Notes for reviewers

  • Verify all applicable tests above pass
  • This PR stacks on ci(gha): adopt chart-testing for chart CI (lint + install) #95. Review ci(gha): adopt chart-testing for chart CI (lint + install) #95 first; this PR's diff will reduce to just the helm-unittest job + 5 test files once ci(gha): adopt chart-testing for chart CI (lint + install) #95 merges.
  • helm-unittest plugin pinned to v1.0.3, cloned from git tag. See the in-workflow comment explaining the pinning rationale: v1.1.0's plugin.yaml adds a platformHooks field that Helm 3.x rejects. Worth tracking upstream (helm-unittest/helm-unittest) for a Helm-3-compatible release.
  • Test files in charts/graylog/tests/ are helm-unittest test definitions (YAML configs with suite:/templates:/tests:/asserts:). They are NOT to be confused with the in-cluster helm test pods at charts/graylog/templates/tests/test-*.yaml, which are Pod manifests used by helm test post-install. Both layers coexist and serve different purposes.
  • Starter scope intentional: 5 test files cover the highest-value conditional logic in the chart, not the trivial templates. Future PRs (ours and maintainers') can extend the pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant