From e5a357505ec42a1193fcebf0695efaee14e36d3f Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 09:27:26 +1100 Subject: [PATCH 1/2] ci(musl): build the Alpine image from a pinned Dockerfile Dependabot can move The linux-x64-musl binaries of @cipherstash/auth and @cipherstash/protect-ffi were built in node:22-alpine named by digest inside a `run:` script, where Dependabot's github-actions ecosystem cannot see it, so nothing would ever move the digest; and the container ran an unpinned `apk add`, so two builds of one commit could use different compilers under the same provenance. The image is now .github/docker/musl-build/Dockerfile: the base image by digest, which a new Dependabot docker entry updates, and every apk package at an exact version. Both build workflows `docker build --pull` it and run in the result; both preflights read its FROM line for their smoke test, so the digest has one home. Alpine drops a package version from its index when it publishes the next one, so a pin can stop resolving between Dependabot runs; musl-build-image.yml builds the Dockerfile weekly and on every change so that is a failed check rather than a failed release. musl-build-image.test.mjs holds the digest pin, the version pins, the Dependabot entry, and that no workflow carries its own copy of the digest or its own `apk add`. Closes #1042. Found by cipherstash-bot in review of #1018. --- .changeset/musl-build-image-dependabot.md | 5 + .github/dependabot.yml | 25 +++ .github/docker/musl-build/Dockerfile | 22 ++ .github/workflows/_build-auth-artifacts.yml | 17 +- .github/workflows/_build-ffi-artifacts.yml | 6 +- .github/workflows/auth-preflight.yml | 8 +- .github/workflows/ffi-preflight.yml | 8 +- .github/workflows/musl-build-image.yml | 47 +++++ e2e/tests/supply-chain.e2e.test.ts | 1 + .../__tests__/auth-build-artifacts.test.mjs | 22 +- .../__tests__/ffi-build-artifacts.test.mjs | 37 +--- scripts/__tests__/musl-build-image.test.mjs | 189 ++++++++++++++++++ skills/stash-supply-chain-security/SKILL.md | 2 +- 13 files changed, 333 insertions(+), 56 deletions(-) create mode 100644 .changeset/musl-build-image-dependabot.md create mode 100644 .github/docker/musl-build/Dockerfile create mode 100644 .github/workflows/musl-build-image.yml create mode 100644 scripts/__tests__/musl-build-image.test.mjs diff --git a/.changeset/musl-build-image-dependabot.md b/.changeset/musl-build-image-dependabot.md new file mode 100644 index 000000000..5045b81fe --- /dev/null +++ b/.changeset/musl-build-image-dependabot.md @@ -0,0 +1,5 @@ +--- +'stash': patch +--- + +The supply-chain skill names every Dependabot ecosystem the repository monitors, including the Go module and the Docker image the musl binaries are built in. diff --git a/.github/dependabot.yml b/.github/dependabot.yml index b2dc14f31..c031cb820 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -296,3 +296,28 @@ updates: - dependency-name: "*" update-types: - version-update:semver-major + + # ── Docker ───────────────────────────────────────────────────── + # The Alpine image the musl binaries are built in. Only the `FROM` digest is + # read here: the `apk` pins in that Dockerfile are moved by hand, and + # musl-build-image.yml says when they stop resolving. + - package-ecosystem: docker + directory: /.github/docker/musl-build + schedule: + interval: weekly + day: monday + cooldown: + default-days: 7 + open-pull-requests-limit: 2 + labels: + - dependencies + - github-actions + commit-message: + prefix: "chore" + include: scope + ignore: + # A new Node.js major (node:24-alpine) is reviewed and applied manually, + # with the host legs' Node version, not by Dependabot. + - dependency-name: "*" + update-types: + - version-update:semver-major diff --git a/.github/docker/musl-build/Dockerfile b/.github/docker/musl-build/Dockerfile new file mode 100644 index 000000000..61f9217b2 --- /dev/null +++ b/.github/docker/musl-build/Dockerfile @@ -0,0 +1,22 @@ +# Builds the linux-x64-musl binaries of @cipherstash/auth and +# @cipherstash/protect-ffi. Alpine is a musl system, so its compiler links +# musl; the Ubuntu runner's compiler links glibc, and the result does not load +# on musl. +# +# Pinned because the output is published with provenance. Dependabot moves the +# digest; nothing moves the package versions, and Alpine drops a version from +# its index when it publishes the next one, so a pin can stop resolving at any +# time. musl-build-image.yml builds this file weekly so that is a failed check, +# not a failed release. Re-pin from the FROM image with +# `apk update >/dev/null && apk search --exact `. +FROM node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 + +# Alpine 3.24.2 +RUN apk add --no-cache \ + build-base=0.5-r4 \ + cmake=4.2.3-r0 \ + curl=8.22.0-r0 \ + git=2.54.0-r0 \ + linux-headers=7.0.0-r1 \ + perl=5.42.2-r1 \ + rustup=1.29.0-r0 diff --git a/.github/workflows/_build-auth-artifacts.yml b/.github/workflows/_build-auth-artifacts.yml index 913171280..efa6f6456 100644 --- a/.github/workflows/_build-auth-artifacts.yml +++ b/.github/workflows/_build-auth-artifacts.yml @@ -118,28 +118,29 @@ jobs: # suite's 0.44.0 has that fault. Ubuntu's musl-gcc cannot link a Rust # shared library (it has no musl libgcc_s), and the musl.cc toolchain # that _build-ffi-artifacts.yml downloads timed out from GitHub's runners - # on 2 October 2026. The image is pinned by digest because its output is - # published with provenance. `-crt-static` keeps the binary linked - # against musl at load time, which a Node.js native module needs, and - # which the C library check below reads. The same napi flags as the host - # build, for the same reasons. + # on 2 October 2026. The image is .github/docker/musl-build/Dockerfile, + # base image and packages pinned, because its output is published with + # provenance. `-crt-static` keeps the binary linked against musl at load + # time, which a Node.js native module needs, and which the C library + # check below reads. The same napi flags as the host build, for the same + # reasons. - name: Build the native binding in Alpine (linux-x64-musl) if: ${{ matrix.platform == 'linux-x64-musl' }} env: TARGET: ${{ matrix.target }} - ALPINE_NODE_IMAGE: node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 + MUSL_BUILD_IMAGE: cipherstash/musl-build:local run: | set -euo pipefail rust_version=$(mise current rust) pnpm_spec=$(node -p "require('./package.json').packageManager") + docker build --pull -t "$MUSL_BUILD_IMAGE" .github/docker/musl-build docker run --rm \ -v "$GITHUB_WORKSPACE:/build" -w /build \ -e TARGET -e RUST_VERSION="$rust_version" -e PNPM_SPEC="$pnpm_spec" \ -e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" \ -e RUSTFLAGS="-C target-feature=-crt-static" \ - "$ALPINE_NODE_IMAGE" sh -euc ' + "$MUSL_BUILD_IMAGE" sh -euc ' trap "chown -R \"\$HOST_UID:\$HOST_GID\" /build" EXIT - apk add --no-cache build-base cmake perl linux-headers git curl rustup rustup-init -y --profile minimal --default-toolchain "$RUST_VERSION" --target "$TARGET" . "$HOME/.cargo/env" corepack enable diff --git a/.github/workflows/_build-ffi-artifacts.yml b/.github/workflows/_build-ffi-artifacts.yml index d0d4d4410..14b454141 100644 --- a/.github/workflows/_build-ffi-artifacts.yml +++ b/.github/workflows/_build-ffi-artifacts.yml @@ -235,20 +235,20 @@ jobs: PLATFORM: ${{ matrix.cfg.platform }} BUILD_SCRIPT: ${{ matrix.cfg.script }} BUILD_LOG: ${{ matrix.cfg.log }} - ALPINE_NODE_IMAGE: node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 + MUSL_BUILD_IMAGE: cipherstash/musl-build:local run: | set -euo pipefail rust_version=$(rustc --version | cut -d' ' -f2) pnpm_spec=$(node -p "require('./package.json').packageManager") + docker build --pull -t "$MUSL_BUILD_IMAGE" .github/docker/musl-build docker run --rm \ -v "$GITHUB_WORKSPACE:/build" -w /build \ -e CARGO_BUILD_TARGET -e PLATFORM -e BUILD_SCRIPT -e BUILD_LOG \ -e RUST_VERSION="$rust_version" -e PNPM_SPEC="$pnpm_spec" \ -e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" \ -e RUSTFLAGS="-C target-feature=-crt-static" \ - "$ALPINE_NODE_IMAGE" sh -euc ' + "$MUSL_BUILD_IMAGE" sh -euc ' trap "chown -R \"\$HOST_UID:\$HOST_GID\" /build" EXIT - apk add --no-cache build-base cmake perl linux-headers git curl rustup rustup-init -y --profile minimal --default-toolchain "$RUST_VERSION" --target "$CARGO_BUILD_TARGET" . "$HOME/.cargo/env" corepack enable diff --git a/.github/workflows/auth-preflight.yml b/.github/workflows/auth-preflight.yml index 9f7f6a95b..9a36a530b 100644 --- a/.github/workflows/auth-preflight.yml +++ b/.github/workflows/auth-preflight.yml @@ -131,12 +131,14 @@ jobs: # The host step above installs only linux-x64-gnu, the runner's own # platform. The musl binary loads only where musl is the C library, so it - # is installed and loaded inside Alpine, from the image the build uses. + # is installed and loaded inside Alpine, on the base image the build + # image is made from: plain Node.js on musl, as a user's container is. + # Read from the Dockerfile so there is one digest for Dependabot to move. - name: Smoke-test the musl artifact inside Alpine - env: - ALPINE_NODE_IMAGE: node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 run: | set -euo pipefail + ALPINE_NODE_IMAGE=$(sed -n 's/^FROM //p' .github/docker/musl-build/Dockerfile) + test -n "$ALPINE_NODE_IMAGE" wrapper=$(basename "$(ls "$GITHUB_WORKSPACE"/auth-dist/cipherstash-auth-[0-9]*.tgz)") musl=$(basename "$(ls "$GITHUB_WORKSPACE"/auth-dist/cipherstash-auth-linux-x64-musl-*.tgz)") docker run --rm -v "$GITHUB_WORKSPACE/auth-dist:/dist:ro" \ diff --git a/.github/workflows/ffi-preflight.yml b/.github/workflows/ffi-preflight.yml index 7cf99328d..40ba51c6a 100644 --- a/.github/workflows/ffi-preflight.yml +++ b/.github/workflows/ffi-preflight.yml @@ -152,12 +152,14 @@ jobs: # The host steps above install only linux-x64-gnu, the runner's own # platform. The musl binary loads only where musl is the C library, so it - # is installed and loaded inside Alpine, from the image the build uses. + # is installed and loaded inside Alpine, on the base image the build + # image is made from: plain Node.js on musl, as a user's container is. + # Read from the Dockerfile so there is one digest for Dependabot to move. - name: Smoke-test the musl artifact inside Alpine - env: - ALPINE_NODE_IMAGE: node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 run: | set -euo pipefail + ALPINE_NODE_IMAGE=$(sed -n 's/^FROM //p' .github/docker/musl-build/Dockerfile) + test -n "$ALPINE_NODE_IMAGE" wrapper=$(basename "$(ls "$GITHUB_WORKSPACE"/ffi-dist/cipherstash-protect-ffi-[0-9]*.tgz)") musl=$(basename "$(ls "$GITHUB_WORKSPACE"/ffi-dist/cipherstash-protect-ffi-linux-x64-musl-*.tgz)") docker run --rm -v "$GITHUB_WORKSPACE/ffi-dist:/dist:ro" \ diff --git a/.github/workflows/musl-build-image.yml b/.github/workflows/musl-build-image.yml new file mode 100644 index 000000000..eef8d4d88 --- /dev/null +++ b/.github/workflows/musl-build-image.yml @@ -0,0 +1,47 @@ +name: musl build image + +# The Dockerfile pins every apk package to an exact version, and Alpine drops a +# version from its index when it publishes the next one. Nothing else builds +# the image between releases, so without this a pin that stopped resolving +# would first be seen by a release. +on: + push: + branches: + - main + paths: + - ".github/docker/musl-build/**" + - ".github/workflows/musl-build-image.yml" + pull_request: + paths: + - ".github/docker/musl-build/**" + - ".github/workflows/musl-build-image.yml" + schedule: + - cron: "0 7 * * 1" + workflow_dispatch: + +permissions: + contents: read + +jobs: + build: + name: Build the Alpine musl image + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + # `--pull` fetches the digest the Dockerfile names rather than a cached + # tag, so the check builds what a release would. + - name: Build the image + run: docker build --pull -t cipherstash/musl-build:check .github/docker/musl-build + # The build steps rely on these being present; a Dockerfile that builds + # but lost one would fail minutes into a release build instead. + - name: Check the tools the build steps use + run: | + set -euo pipefail + docker run --rm cipherstash/musl-build:check sh -euc ' + for tool in cc c++ cmake perl make curl git rustup-init node; do + command -v "$tool" >/dev/null || { echo "missing: $tool" >&2; exit 1; } + done + ' diff --git a/e2e/tests/supply-chain.e2e.test.ts b/e2e/tests/supply-chain.e2e.test.ts index 9fe55dc36..590179e01 100644 --- a/e2e/tests/supply-chain.e2e.test.ts +++ b/e2e/tests/supply-chain.e2e.test.ts @@ -625,6 +625,7 @@ const MANIFEST_BY_ECOSYSTEM: Record = { uv: 'pyproject.toml', mix: 'mix.exs', pub: 'pubspec.yaml', + docker: 'Dockerfile', // github-actions is special-cased: Dependabot requires `directory: /` and // discovers .github/workflows itself. 'github-actions': '.github/workflows', diff --git a/scripts/__tests__/auth-build-artifacts.test.mjs b/scripts/__tests__/auth-build-artifacts.test.mjs index da81fd261..c3623563f 100644 --- a/scripts/__tests__/auth-build-artifacts.test.mjs +++ b/scripts/__tests__/auth-build-artifacts.test.mjs @@ -80,19 +80,18 @@ describe('_build-auth-artifacts.yml', () => { for (const build of builds) expect(check).toBeGreaterThan(build) }) - it('builds the musl binding inside Alpine, from an image pinned by digest', () => { + it('builds the musl binding inside Alpine, from the pinned Dockerfile', () => { // Built on the Ubuntu runner, the musl binary linked glibc and failed to - // load on musl. Alpine is a musl system, so its compiler links musl. + // load on musl. Alpine is a musl system, so its compiler links musl. The + // image's pins, and that this is its one home, are musl-build-image.test.mjs's. const steps = binaries?.steps ?? [] const musl = steps.filter((step) => String(step?.if ?? '').includes("== 'linux-x64-musl'"), ) const run = musl.map((step) => String(step?.run ?? '')).join('\n') - const env = Object.assign({}, ...musl.map((step) => step?.env ?? {})) - expect(env.ALPINE_NODE_IMAGE).toMatch(/-alpine@sha256:[0-9a-f]{64}$/) - // The pinned image is the one that runs, not a mutable tag. - expect(run).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) - expect(run).not.toMatch(/\bnode:\d+-alpine(?!@)/) + expect(run).toMatch( + /docker build --pull -t "\$MUSL_BUILD_IMAGE" \.github\/docker\/musl-build\n[\s\S]*docker run[\s\S]*"\$MUSL_BUILD_IMAGE"/, + ) expect(run).toContain('RUSTFLAGS="-C target-feature=-crt-static"') // The container loads the binary it built, on musl. expect(run).toMatch( @@ -162,12 +161,11 @@ describe('_build-auth-artifacts.yml', () => { ) expect(alpine).toBeDefined() expect(String(alpine.run)).toContain('linux-x64-musl') - expect(String(alpine.run)).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) - // The same image as the build, so the load test matches the build. - const build = (binaries?.steps ?? []).find( - (step) => step?.env?.ALPINE_NODE_IMAGE, + // On the image the build image starts from, read from the Dockerfile. + expect(String(alpine.run)).toContain( + "ALPINE_NODE_IMAGE=$(sed -n 's/^FROM //p'", ) - expect(alpine.env?.ALPINE_NODE_IMAGE).toBe(build?.env?.ALPINE_NODE_IMAGE) + expect(String(alpine.run)).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) }) it('packs the wrapper with pnpm, which rewrites its workspace peers', () => { diff --git a/scripts/__tests__/ffi-build-artifacts.test.mjs b/scripts/__tests__/ffi-build-artifacts.test.mjs index b2f3d8b69..45343fd24 100644 --- a/scripts/__tests__/ffi-build-artifacts.test.mjs +++ b/scripts/__tests__/ffi-build-artifacts.test.mjs @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest' -import { readWorkflow, workflowFiles } from './lib/workflows.mjs' +import { readWorkflow } from './lib/workflows.mjs' /** * `_build-ffi-artifacts.yml` builds the seven `@cipherstash/protect-ffi` @@ -10,9 +10,10 @@ import { readWorkflow, workflowFiles } from './lib/workflows.mjs' * 1. Every Linux binary's C library is checked before it is packed. Without * the check, only a hand-run ffi-preflight sees a glibc-linked musl binary, * which fails to load on musl systems such as Alpine Linux. - * 2. The musl binary is built inside Alpine Linux, from an image pinned by - * digest, and loaded there. Its toolchain used to come from musl.cc, which - * timed out from GitHub's runners on six tries on 2 October 2026. + * 2. The musl binary is built inside Alpine Linux, from the pinned Dockerfile + * at .github/docker/musl-build, and loaded there. Its toolchain used to + * come from musl.cc, which timed out from GitHub's runners on six tries on + * 2 October 2026. The Dockerfile's pins are musl-build-image.test.mjs's. */ const workflow = readWorkflow('.github/workflows/_build-ffi-artifacts.yml') @@ -25,7 +26,6 @@ const gha = (expression) => `\${{ ${expression} }}` const MUSL = "matrix.cfg.platform == 'linux-x64-musl'" const NOT_MUSL = "matrix.cfg.platform != 'linux-x64-musl'" -const PINNED = /^node:\d+-alpine@sha256:[0-9a-f]{64}$/ describe('_build-ffi-artifacts.yml', () => { it('checks the C library of every Linux binary before it is packed', () => { @@ -51,15 +51,14 @@ describe('_build-ffi-artifacts.yml', () => { ) }) - it('builds the musl binding inside Alpine, from an image pinned by digest', () => { + it('builds the musl binding inside Alpine, from the pinned Dockerfile', () => { const musl = steps.filter((step) => String(step?.if ?? '').includes(MUSL)) expect(musl).toHaveLength(1) const [build] = musl const run = runOf(build) - expect(build.env?.ALPINE_NODE_IMAGE).toMatch(PINNED) - // The pinned image is the one that runs, not a mutable tag. - expect(run).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) - expect(run).not.toMatch(/\bnode:\d+-alpine(?!@)/) + expect(run).toMatch( + /docker build --pull -t "\$MUSL_BUILD_IMAGE" \.github\/docker\/musl-build\n[\s\S]*docker run[\s\S]*"\$MUSL_BUILD_IMAGE"/, + ) expect(run).toContain('RUSTFLAGS="-C target-feature=-crt-static"') // The container hands its files back even when the build fails. expect(run).toMatch(/^\s*trap "chown -R .*\/build" EXIT$/m) @@ -153,6 +152,8 @@ describe('ffi-preflight.yml', () => { const alpine = smoke.find((step) => /\bdocker run\b/.test(runOf(step))) expect(alpine).toBeDefined() const run = runOf(alpine) + // On the image the build image starts from, read from the Dockerfile. + expect(run).toContain("ALPINE_NODE_IMAGE=$(sed -n 's/^FROM //p'") expect(run).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) expect(run).toContain('cipherstash-protect-ffi-linux-x64-musl-') expect(run).toMatch( @@ -162,19 +163,3 @@ describe('ffi-preflight.yml', () => { expect(run).toContain('ffi.assertNativeBindingAvailable()') }) }) - -describe('the Alpine image', () => { - it('is one pinned image in every workflow, so a load test matches its build', () => { - const images = workflowFiles().flatMap((file) => - Object.values(readWorkflow(file)?.jobs ?? {}).flatMap((job) => - (job?.steps ?? []) - .map((step) => step?.env?.ALPINE_NODE_IMAGE) - .filter(Boolean), - ), - ) - // Both builds and both preflights. - expect(images.length).toBeGreaterThanOrEqual(4) - expect(new Set(images).size).toBe(1) - expect(images[0]).toMatch(PINNED) - }) -}) diff --git a/scripts/__tests__/musl-build-image.test.mjs b/scripts/__tests__/musl-build-image.test.mjs new file mode 100644 index 000000000..ca2ce7fd0 --- /dev/null +++ b/scripts/__tests__/musl-build-image.test.mjs @@ -0,0 +1,189 @@ +import { readFileSync } from 'node:fs' +import { join } from 'node:path' +import yaml from 'js-yaml' +import { describe, expect, it } from 'vitest' +import { REPO_ROOT } from './lib/repo-root.mjs' +import { readWorkflow, workflowFiles } from './lib/workflows.mjs' + +/** + * The linux-x64-musl binaries of `@cipherstash/auth` and + * `@cipherstash/protect-ffi` are built in Alpine Linux, from + * `.github/docker/musl-build/Dockerfile`, and published with provenance. Two + * things about that image drift silently if an edit undoes them (#1042): + * + * 1. The base image digest is updated by nothing unless Dependabot's `docker` + * ecosystem reads the Dockerfile. Its `github-actions` ecosystem reads + * `uses:` lines only, so an image named inside a `run:` script is + * invisible to it — which is how the digest sat at its 2 October 2026 + * value. + * 2. An unpinned `apk add` installs whatever Alpine serves that day, so two + * builds of one commit can use different compilers. + * + * Both are only as good as there being ONE image: a second copy of the digest + * in a workflow, or an `apk add` in a build script, is the drift this file + * exists to catch. + */ + +const DOCKERFILE_DIR = '.github/docker/musl-build' +const DOCKERFILE = `${DOCKERFILE_DIR}/Dockerfile` +const dockerfile = readFileSync(join(REPO_ROOT, DOCKERFILE), 'utf8') + +const PINNED = /^node:\d+-alpine@sha256:[0-9a-f]{64}$/ + +/** The build steps need these; a Dockerfile that lost one would still build. */ +const TOOLCHAIN = [ + 'build-base', + 'cmake', + 'curl', + 'git', + 'linux-headers', + 'perl', + 'rustup', +] + +/** Each `RUN apk add` in the Dockerfile, as its list of package arguments. */ +function apkAdds(text) { + // Join `\`-continued lines so one instruction is one string. + const instructions = text.replace(/\\\n/g, ' ').split('\n') + return instructions + .filter((line) => /^RUN\s+apk\s+add\b/.test(line)) + .map((line) => + line + .replace(/^RUN\s+apk\s+add\s+/, '') + .split(/\s+/) + .filter((word) => word && !word.startsWith('--')), + ) +} + +describe(DOCKERFILE, () => { + it('starts from a node Alpine image pinned by digest', () => { + const froms = dockerfile + .split('\n') + .filter((line) => /^FROM\b/.test(line)) + .map((line) => line.replace(/^FROM\s+/, '').trim()) + // One stage: the preflights read `FROM` with `sed -n 's/^FROM //p'` and + // run the smoke test in that image, so a second stage would hand them two. + expect(froms).toHaveLength(1) + expect(froms[0]).toMatch(PINNED) + }) + + it('pins every apk package to an exact version', () => { + const adds = apkAdds(dockerfile) + expect(adds.length).toBeGreaterThan(0) + const packages = adds.flat() + const unpinned = packages.filter( + (pkg) => !/^[a-z0-9][a-z0-9._+-]*=[^=\s]+$/.test(pkg), + ) + expect( + unpinned, + `These apk packages carry no \`=version\`, so \`apk add\` installs whatever Alpine serves on the day:\n${unpinned.map((p) => ` ${p}`).join('\n')}`, + ).toEqual([]) + }) + + it('installs the toolchain the build steps rely on', () => { + const names = apkAdds(dockerfile) + .flat() + .map((pkg) => pkg.split('=')[0]) + for (const tool of TOOLCHAIN) expect(names).toContain(tool) + }) +}) + +describe('the workflows that use the image', () => { + const workflows = workflowFiles().map((relPath) => ({ + relPath, + text: readFileSync(join(REPO_ROOT, relPath), 'utf8'), + wf: readWorkflow(relPath), + })) + const runsOf = (wf) => + Object.values(wf?.jobs ?? {}).flatMap((job) => + (Array.isArray(job?.steps) ? job.steps : []).map((step) => + String(step?.run ?? ''), + ), + ) + + it('build the musl binaries from the Dockerfile, not from an inline image', () => { + for (const relPath of [ + '.github/workflows/_build-auth-artifacts.yml', + '.github/workflows/_build-ffi-artifacts.yml', + ]) { + const { wf } = workflows.find((w) => w.relPath === relPath) + const run = runsOf(wf).find((text) => /\bdocker run\b/.test(text)) + expect(run, `${relPath} has no docker run step`).toBeDefined() + // `--pull` fetches the digest the Dockerfile names, not a tag cached on + // the runner, and the run step uses the tag the build step wrote. + expect(run).toMatch( + /docker build --pull -t "\$MUSL_BUILD_IMAGE" \.github\/docker\/musl-build\n[\s\S]*docker run[\s\S]*"\$MUSL_BUILD_IMAGE"/, + ) + } + }) + + it('name the base image digest nowhere, so the Dockerfile is its one home', () => { + const copies = workflows + .filter(({ text }) => /node:\d+-alpine@sha256:/.test(text)) + .map(({ relPath }) => relPath) + expect( + copies, + `These workflows carry their own copy of the Alpine image digest. Dependabot moves the one in ${DOCKERFILE}, and a copy is left behind — read it from the Dockerfile as the preflights do.`, + ).toEqual([]) + }) + + it('install no apk packages of their own', () => { + const offenders = workflows + .filter(({ wf }) => runsOf(wf).some((text) => /\bapk add\b/.test(text))) + .map(({ relPath }) => relPath) + expect( + offenders, + `These workflows run \`apk add\` in a step. Packages the musl build needs belong in ${DOCKERFILE}, pinned.`, + ).toEqual([]) + }) + + it('smoke-test the musl artifacts on the image the Dockerfile starts from', () => { + for (const relPath of [ + '.github/workflows/auth-preflight.yml', + '.github/workflows/ffi-preflight.yml', + ]) { + const { wf } = workflows.find((w) => w.relPath === relPath) + const run = runsOf(wf).find((text) => /\bdocker run\b/.test(text)) + expect(run, `${relPath} has no docker run step`).toBeDefined() + expect(run).toContain( + `ALPINE_NODE_IMAGE=$(sed -n 's/^FROM //p' ${DOCKERFILE})`, + ) + expect(run).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) + } + }) +}) + +describe('keeping the image current', () => { + it('has a Dependabot docker entry for the Dockerfile directory', () => { + const config = yaml.load( + readFileSync(join(REPO_ROOT, '.github/dependabot.yml'), 'utf8'), + ) + const docker = (config?.updates ?? []).filter( + (update) => update['package-ecosystem'] === 'docker', + ) + expect(docker.map((update) => update.directory)).toContain( + `/${DOCKERFILE_DIR}`, + ) + }) + + it('builds the Dockerfile on a schedule and on every change to it', () => { + const relPath = '.github/workflows/musl-build-image.yml' + const wf = readWorkflow(relPath) + const on = wf?.on ?? wf?.[true] + expect(on?.schedule?.length ?? 0).toBeGreaterThan(0) + for (const trigger of ['push', 'pull_request']) { + expect(on?.[trigger]?.paths).toEqual( + expect.arrayContaining([`${DOCKERFILE_DIR}/**`, relPath]), + ) + } + const runs = Object.values(wf?.jobs ?? {}).flatMap((job) => + (job?.steps ?? []).map((step) => String(step?.run ?? '')), + ) + expect( + runs.some( + (run) => + run.includes(`docker build --pull`) && run.includes(DOCKERFILE_DIR), + ), + ).toBe(true) + }) +}) diff --git a/skills/stash-supply-chain-security/SKILL.md b/skills/stash-supply-chain-security/SKILL.md index 6df77728f..125c3e74e 100644 --- a/skills/stash-supply-chain-security/SKILL.md +++ b/skills/stash-supply-chain-security/SKILL.md @@ -59,7 +59,7 @@ the registry. ### 5. Cooldown'd auto-updates — practice #6 -Dependabot opens grouped, cooldown'd PRs (7 days minor/patch) for `npm`, `cargo` and `github-actions`. Major bumps are not proposed at all — every entry ignores `version-update:semver-major`, so majors are reviewed and applied by hand. +Dependabot opens grouped, cooldown'd PRs (7 days minor/patch) for `npm`, `cargo`, `gomod`, `github-actions` and `docker` (the digest of the Alpine image the musl binaries are built in). Major bumps are not proposed at all — every entry ignores `version-update:semver-major`, so majors are reviewed and applied by hand. There is deliberately **no `semver-major-days` cooldown** on any entry. It would delay major *version update* PRs, which the `ignore` above means Dependabot never opens, and cooldown does not reach the security path either ("the cooldown option is only available for version updates, not security updates"). Don't add one back as a safety net for the day the `ignore` is dropped — dead config reads as policy, and the test below fails on the pair. From ee876696135bcc908389d199cd588c7bdada674f Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 16:02:41 +1100 Subject: [PATCH 2/2] fix(musl): check out the Dockerfile in the preflights, pin the compiler, and widen the guards MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The preflights sparse-checkout `scripts` alone, so the smoke test's `sed` over the Dockerfile read nothing and every dispatched run died before Docker started; the directory is in both checkouts now, and the test demands it. `build-base` is a metapackage whose gcc, g++, binutils, musl-dev and make dependencies carry no version, so pinning it alone left the compiler unpinned — the five are pinned beside it and demanded by name. Guards from the review: no workflow may name `node:-alpine` with or without a digest; every `apk add` in the Dockerfile must be one the pin check parses, and no `apk upgrade`; the FROM line is read as the preflights' `sed` reads it; the weekly cron, the tools check step and the Dependabot schedule are asserted; every Dependabot entry needs a cooldown, and the skill must name every monitored ecosystem. The weekly run opens an issue when it fails, since GitHub mails a scheduled failure to one person. The docker entry carries the `supply-chain` label the other non-Actions entries use, and the Alpine version is printed by the check rather than written in a comment Dependabot would leave stale. Review: cipherstash-bot on #1107. --- .github/dependabot.yml | 2 +- .github/docker/musl-build/Dockerfile | 8 +- .github/workflows/auth-preflight.yml | 5 +- .github/workflows/ffi-preflight.yml | 5 +- .github/workflows/musl-build-image.yml | 16 +++ e2e/tests/supply-chain.e2e.test.ts | 29 +++++ scripts/__tests__/musl-build-image.test.mjs | 115 +++++++++++++++++--- skills/stash-supply-chain-security/SKILL.md | 2 +- 8 files changed, 163 insertions(+), 19 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index c031cb820..6cb08638d 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -311,7 +311,7 @@ updates: open-pull-requests-limit: 2 labels: - dependencies - - github-actions + - supply-chain commit-message: prefix: "chore" include: scope diff --git a/.github/docker/musl-build/Dockerfile b/.github/docker/musl-build/Dockerfile index 61f9217b2..b4b3fc5ac 100644 --- a/.github/docker/musl-build/Dockerfile +++ b/.github/docker/musl-build/Dockerfile @@ -11,12 +11,18 @@ # `apk update >/dev/null && apk search --exact `. FROM node:22-alpine@sha256:0a7108bf6c7bf5de370ffb1a3ed6be93d405b43ff159f681a8d18c0e2bc2e402 -# Alpine 3.24.2 +# build-base is a metapackage whose compiler and linker dependencies carry no +# version, so gcc, g++, binutils, musl-dev and make are pinned beside it. RUN apk add --no-cache \ + binutils=2.45.1-r1 \ build-base=0.5-r4 \ cmake=4.2.3-r0 \ curl=8.22.0-r0 \ + g++=15.2.0-r5 \ + gcc=15.2.0-r5 \ git=2.54.0-r0 \ linux-headers=7.0.0-r1 \ + make=4.4.1-r4 \ + musl-dev=1.2.6-r2 \ perl=5.42.2-r1 \ rustup=1.29.0-r0 diff --git a/.github/workflows/auth-preflight.yml b/.github/workflows/auth-preflight.yml index 9a36a530b..124814b9e 100644 --- a/.github/workflows/auth-preflight.yml +++ b/.github/workflows/auth-preflight.yml @@ -47,7 +47,10 @@ jobs: with: ref: ${{ inputs.ref }} persist-credentials: false - sparse-checkout: scripts + # The musl smoke test reads the Dockerfile's FROM line. + sparse-checkout: | + scripts + .github/docker/musl-build - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 with: diff --git a/.github/workflows/ffi-preflight.yml b/.github/workflows/ffi-preflight.yml index 40ba51c6a..5372ea2fe 100644 --- a/.github/workflows/ffi-preflight.yml +++ b/.github/workflows/ffi-preflight.yml @@ -49,7 +49,10 @@ jobs: with: ref: ${{ inputs.ref }} persist-credentials: false - sparse-checkout: scripts + # The musl smoke test reads the Dockerfile's FROM line. + sparse-checkout: | + scripts + .github/docker/musl-build - uses: actions/download-artifact@v4 with: diff --git a/.github/workflows/musl-build-image.yml b/.github/workflows/musl-build-image.yml index eef8d4d88..89ef361e4 100644 --- a/.github/workflows/musl-build-image.yml +++ b/.github/workflows/musl-build-image.yml @@ -27,6 +27,10 @@ jobs: name: Build the Alpine musl image runs-on: ubuntu-latest timeout-minutes: 15 + # `issues: write` for the last step alone; see there. + permissions: + contents: read + issues: write steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 with: @@ -41,7 +45,19 @@ jobs: run: | set -euo pipefail docker run --rm cipherstash/musl-build:check sh -euc ' + echo "Alpine $(cat /etc/alpine-release)" for tool in cc c++ cmake perl make curl git rustup-init node; do command -v "$tool" >/dev/null || { echo "missing: $tool" >&2; exit 1; } done ' + # GitHub mails a scheduled run's failure to one person: whoever last + # edited the cron line. An issue is what the rest of the team sees. + - name: Open an issue when the weekly build fails + if: ${{ failure() && github.event_name == 'schedule' }} + env: + GH_TOKEN: ${{ github.token }} + run: | + gh issue create --repo "$GITHUB_REPOSITORY" \ + --title "The musl build image no longer builds" \ + --label "github-actions" \ + --body "The weekly build of .github/docker/musl-build/Dockerfile failed: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID. An apk pin has most likely stopped resolving; re-pin with the command in the Dockerfile's comment." diff --git a/e2e/tests/supply-chain.e2e.test.ts b/e2e/tests/supply-chain.e2e.test.ts index 590179e01..ad8e4292e 100644 --- a/e2e/tests/supply-chain.e2e.test.ts +++ b/e2e/tests/supply-chain.e2e.test.ts @@ -723,6 +723,35 @@ describe('supply chain — automated dependency updates (Dependabot)', () => { expect(gha?.cooldown?.['default-days']).toBeGreaterThanOrEqual(3) }) + it('every entry has a ≥ 3 day cooldown', () => { + // The two above are the named controls; this is the one that catches an + // entry added later without one, which would propose a dependency the + // day it is published. + expect(db.updates.length).toBeGreaterThan(0) + for (const entry of db.updates) { + expect( + entry.cooldown?.['default-days'] ?? 0, + `${entry['package-ecosystem']} at ${entry.directory} has no cooldown of at least 3 days`, + ).toBeGreaterThanOrEqual(3) + } + }) + + it('the skill names every ecosystem Dependabot monitors', () => { + // The sentence is what a customer's agent reads; it omitted gomod for + // months with nothing to say so. + const skill = read('skills/stash-supply-chain-security/SKILL.md') + const sentence = skill + .split('\n') + .find((line) => line.startsWith('Dependabot opens grouped')) + expect(sentence).toBeDefined() + const ecosystems = new Set(db.updates.map((u) => u['package-ecosystem'])) + for (const ecosystem of ecosystems) { + expect(sentence, `the skill does not name \`${ecosystem}\``).toContain( + `\`${ecosystem}\``, + ) + } + }) + it('every entry ignores majors, so none configures a major cooldown window', () => { // One relationship, asserted from both ends, because either end alone // passes on the drift that matters. diff --git a/scripts/__tests__/musl-build-image.test.mjs b/scripts/__tests__/musl-build-image.test.mjs index ca2ce7fd0..84188b357 100644 --- a/scripts/__tests__/musl-build-image.test.mjs +++ b/scripts/__tests__/musl-build-image.test.mjs @@ -30,22 +30,53 @@ const dockerfile = readFileSync(join(REPO_ROOT, DOCKERFILE), 'utf8') const PINNED = /^node:\d+-alpine@sha256:[0-9a-f]{64}$/ -/** The build steps need these; a Dockerfile that lost one would still build. */ +/** A GitHub Actions expression, as the parsed workflow holds it. */ +const gha = (expression) => `\${{ ${expression} }}` + +/** + * The build steps need these; a Dockerfile that lost one would still build. + * `build-base` is a metapackage whose compiler and linker dependencies carry + * no version, so the five it pulls in are pinned, and demanded, by name. + */ const TOOLCHAIN = [ + 'binutils', 'build-base', 'cmake', 'curl', + 'g++', + 'gcc', 'git', 'linux-headers', + 'make', + 'musl-dev', 'perl', 'rustup', ] +/** The tools `musl-build-image.yml` checks for inside the built image. */ +const TOOLS_CHECKED = [ + 'cc', + 'c++', + 'cmake', + 'perl', + 'make', + 'curl', + 'git', + 'rustup-init', + 'node', +] + +/** The Dockerfile's instructions, comments dropped and continuations joined. */ +const instructions = dockerfile + .split('\n') + .filter((line) => !/^\s*#/.test(line)) + .join('\n') + .replace(/\\\n/g, ' ') + /** Each `RUN apk add` in the Dockerfile, as its list of package arguments. */ function apkAdds(text) { - // Join `\`-continued lines so one instruction is one string. - const instructions = text.replace(/\\\n/g, ' ').split('\n') - return instructions + return text + .split('\n') .filter((line) => /^RUN\s+apk\s+add\b/.test(line)) .map((line) => line @@ -57,18 +88,33 @@ function apkAdds(text) { describe(DOCKERFILE, () => { it('starts from a node Alpine image pinned by digest', () => { + // Read exactly as the preflights do, `sed -n 's/^FROM //p'`: one space + // stripped, nothing trimmed, so a trailing space or CR that would make + // `docker run` fail with "invalid reference format" fails here. One + // stage, because a second would hand them two images. const froms = dockerfile .split('\n') - .filter((line) => /^FROM\b/.test(line)) - .map((line) => line.replace(/^FROM\s+/, '').trim()) - // One stage: the preflights read `FROM` with `sed -n 's/^FROM //p'` and - // run the smoke test in that image, so a second stage would hand them two. + .filter((line) => line.startsWith('FROM ')) + .map((line) => line.slice('FROM '.length)) expect(froms).toHaveLength(1) expect(froms[0]).toMatch(PINNED) }) + it('reads every apk add in the Dockerfile, so none skips the pin check', () => { + // `apkAdds` reads `RUN apk add …` lines only. An `apk add` after `&&` + // would install unpinned and pass the test below, and `apk upgrade` + // moves versions with no pin at all. + const written = instructions.match(/\bapk\s+add\b/g) ?? [] + expect(written.length).toBeGreaterThan(0) + expect( + apkAdds(instructions), + 'An `apk add` is not at the start of a `RUN` line, so its packages are not checked for a `=version`.', + ).toHaveLength(written.length) + expect(instructions).not.toMatch(/\bapk\s+upgrade\b/) + }) + it('pins every apk package to an exact version', () => { - const adds = apkAdds(dockerfile) + const adds = apkAdds(instructions) expect(adds.length).toBeGreaterThan(0) const packages = adds.flat() const unpinned = packages.filter( @@ -81,7 +127,7 @@ describe(DOCKERFILE, () => { }) it('installs the toolchain the build steps rely on', () => { - const names = apkAdds(dockerfile) + const names = apkAdds(instructions) .flat() .map((pkg) => pkg.split('=')[0]) for (const tool of TOOLCHAIN) expect(names).toContain(tool) @@ -117,13 +163,15 @@ describe('the workflows that use the image', () => { } }) - it('name the base image digest nowhere, so the Dockerfile is its one home', () => { + it('name the node Alpine image nowhere, so the Dockerfile is its one home', () => { + // Pinned or not: a copy with no digest is worse than a stale one, because + // the tag moves. const copies = workflows - .filter(({ text }) => /node:\d+-alpine@sha256:/.test(text)) + .filter(({ text }) => /\bnode:\d+-alpine\b/.test(text)) .map(({ relPath }) => relPath) expect( copies, - `These workflows carry their own copy of the Alpine image digest. Dependabot moves the one in ${DOCKERFILE}, and a copy is left behind — read it from the Dockerfile as the preflights do.`, + `These workflows name the node Alpine image themselves. Dependabot moves the digest in ${DOCKERFILE}, and a copy is left behind or was never pinned — read it from the Dockerfile as the preflights do.`, ).toEqual([]) }) @@ -149,6 +197,19 @@ describe('the workflows that use the image', () => { `ALPINE_NODE_IMAGE=$(sed -n 's/^FROM //p' ${DOCKERFILE})`, ) expect(run).toMatch(/docker run[\s\S]*"\$ALPINE_NODE_IMAGE"/) + // Both preflights sparse-checkout `scripts` alone; the Dockerfile has + // to be in that list or the `sed` reads nothing and the step dies. + const job = Object.values(wf.jobs).find((j) => + (j?.steps ?? []).some((s) => String(s?.run ?? '').includes(DOCKERFILE)), + ) + const checkout = job.steps.find((s) => + String(s?.uses ?? '').startsWith('actions/checkout'), + ) + const sparse = String(checkout?.with?.['sparse-checkout'] ?? '') + .split('\n') + .map((path) => path.trim()) + .filter(Boolean) + if (sparse.length > 0) expect(sparse).toContain(DOCKERFILE_DIR) } }) }) @@ -164,13 +225,19 @@ describe('keeping the image current', () => { expect(docker.map((update) => update.directory)).toContain( `/${DOCKERFILE_DIR}`, ) + // Weekly, as the skill says. The cooldown on every entry is + // supply-chain.e2e.test.ts's. + const entry = docker.find( + (update) => update.directory === `/${DOCKERFILE_DIR}`, + ) + expect(entry).toMatchObject({ schedule: { interval: 'weekly' } }) }) it('builds the Dockerfile on a schedule and on every change to it', () => { const relPath = '.github/workflows/musl-build-image.yml' const wf = readWorkflow(relPath) const on = wf?.on ?? wf?.[true] - expect(on?.schedule?.length ?? 0).toBeGreaterThan(0) + expect(on?.schedule).toEqual([{ cron: '0 7 * * 1' }]) for (const trigger of ['push', 'pull_request']) { expect(on?.[trigger]?.paths).toEqual( expect.arrayContaining([`${DOCKERFILE_DIR}/**`, relPath]), @@ -185,5 +252,25 @@ describe('keeping the image current', () => { run.includes(`docker build --pull`) && run.includes(DOCKERFILE_DIR), ), ).toBe(true) + // A Dockerfile that builds but lost a tool would fail minutes into a + // release build; the check step is what sees it first. + const check = runs.find((run) => run.includes('for tool in')) + expect( + check, + 'no step checks the tools inside the built image', + ).toBeDefined() + const listed = check + .match(/for tool in ([^;]+);/)[1] + .trim() + .split(/\s+/) + expect(listed).toEqual(TOOLS_CHECKED) + // A scheduled run's failure mails one person; the issue is for the rest. + const steps = Object.values(wf?.jobs ?? {}).flatMap( + (job) => job?.steps ?? [], + ) + const notify = steps.find((step) => + String(step?.run ?? '').includes('gh issue create'), + ) + expect(notify?.if).toBe(gha("failure() && github.event_name == 'schedule'")) }) }) diff --git a/skills/stash-supply-chain-security/SKILL.md b/skills/stash-supply-chain-security/SKILL.md index 125c3e74e..56ebcf476 100644 --- a/skills/stash-supply-chain-security/SKILL.md +++ b/skills/stash-supply-chain-security/SKILL.md @@ -66,7 +66,7 @@ There is deliberately **no `semver-major-days` cooldown** on any entry. It would `cargo` covers the in-tree Rust workspace at `languages/typescript/packages/protect-ffi` (**not** the repo root — that is where `Cargo.toml`/`Cargo.lock` live). It runs monthly rather than weekly because each bump costs a native rebuild to validate, and it ignores the exact-pinned CipherStash crates (`cipherstash-client`, `cts-common`, `stack-auth`, `stack-profile`, `eql-bindings`, `vitaminc`) — they share a release train with the `@cipherstash/auth` catalog and must be bumped together, manually. - **Where**: `.github/dependabot.yml` -- **Test asserts**: cooldown ≥ 3 days on npm/github-actions; every entry ignores `version-update:semver-major` for `*` **and** sets no `semver-major-days` (both ends, so neither half can drift alone); every lockfile present in the repo maps to a monitored `package-ecosystem`; every entry's `directory` actually contains the manifest its ecosystem reads +- **Test asserts**: cooldown ≥ 3 days on every entry; the sentence above names every monitored ecosystem; every entry ignores `version-update:semver-major` for `*` **and** sets no `semver-major-days` (both ends, so neither half can drift alone); every lockfile present in the repo maps to a monitored `package-ecosystem`; every entry's `directory` actually contains the manifest its ecosystem reads The ecosystem-coverage assertion is derived from the filesystem, so **adding a lockfile for a new language fails the suite until `dependabot.yml` covers it.** Two lockfiles are exempt because Dependabot has no ecosystem for them (`e2e/wasm/deno.lock`, `.flox/env/manifest.lock`); both are named with their reason in the test.