ci(supply-chain): sign runtime manifest, validate scan refs, asset SBOMs + train provenance gate (FIX-SUPPLY)

.github/README.md

@@ -9,7 +9,7 @@ the trailing comment) and every downloaded tool sha256-verified.
 | `images.yml` | push to `main`, `workflow_dispatch` | digest-only build of the seven images (`backend`, `frontend`, `operator`, `linux-base`, `linux-desktop`, `browser`, `kasm-adapter`) → isolated trivy gate + SBOM → promote `ghcr.io/tinyorbitvn/tinycdi-<name>:{sha-<short>,main}` + cosign keyless signature/SBOM attestation. Publishes only when `github.ref == refs/heads/main`; a dispatch elsewhere builds + scans without pushing. |
 | `release.yml` | tag `v*.*.*`, `workflow_dispatch` (dry-run only) | digest-only build of the `build/release-images.txt` set → isolated trivy gate → `environment: release` publish job: sign + attest digests, `helm push` to `oci://ghcr.io/tinyorbitvn/charts` + sign the chart, then promote `:<semver>`/`latest` tags, GitHub Release with binaries + CRDs + SBOMs + KasmVNC source bundle + `sha256sums.txt` + sigstore bundles |
 | `runtime-freshness.yml` | daily schedule, `workflow_dispatch` | runs `check-browser-freshness.sh` for **both** pinned engines (chromium and firefox-esr); when bookworm-security offers a newer build — or the pinned version no longer exists there ("pinned version gone": the browser image can no longer be built from scratch) — `bump-browser-pin.sh` repins `build/browser/Dockerfile` (and, for firefox-esr, `build/linux-desktop/Dockerfile` — the desktop image carries the same Firefox pin) + the doc pins (firefox-esr: with its deb sha256) and one pin-bump PR is opened (`gh pr create`). Also runs `check-runtime-image-age.sh`: fails when the newest `runtime-*` release is older than 14 days (D28) |
-| `runtime-images.yml` | push to `main` touching `build/{linux-base,linux-desktop,browser}/**`, weekly schedule, `workflow_dispatch` | the runtime image release train (D27): digest-only build of linux-base, then linux-desktop + browser (both `FROM` the base digest) → isolated trivy gate → cosign sign + SBOM attest → promote `rt-YYYYMMDD.N` tag (N = next free number over the day's published `rt-*` tags — successful publishes only) → `runtime-images.json` attached to GitHub Release `runtime-YYYY.MM.DD`. Never builds or tags control-plane images; publishes only on `refs/heads/main` |
+| `runtime-images.yml` | push to `main` touching `build/{linux-base,linux-desktop,browser}/**`, weekly schedule, `workflow_dispatch` | the runtime image release train (D27): digest-only build of linux-base, then linux-desktop + browser (both `FROM` the base digest) → isolated trivy gate → cosign sign + SBOM attest (+ `attest-build-provenance` under `TRAIN_ATTESTATIONS_ENABLED`, off by default) → promote `rt-YYYYMMDD.N` tag (N = next free number over the day's published `rt-*` tags — successful publishes only) → `runtime-images.json` sign-blob'd and attached to GitHub Release `runtime-YYYY.MM.DD`. Never builds or tags control-plane images; publishes only on `refs/heads/main` |
 
 ## Supply-chain pipeline shape
 
@@ -47,7 +47,9 @@ split publish rights from scanner execution (SEC-04/SEC-05):
    the chart tgz + static binaries + CRD bundle + KasmVNC source bundle in
    the release bundle, and the digests stamped into the packaged chart
    must equal the image refs. The images.yml promote job validates refs
-   against the same strict regex (`validate-image-refs.sh`).
+   against the same strict regex (`validate-image-refs.sh`). Scan jobs
+   apply the same strict form to artifact-supplied refs **before**
+   writing them to `$GITHUB_ENV` (SUPF-10).
 
 A non-publishing run (`release.yml` dry_run, `images.yml` on a non-main ref)
 exports the image as a `type=docker` tar artifact instead; the scan job scans
@@ -116,6 +118,7 @@ Created by `setup-repo-protection.sh --apply` (or manually under
 |---|---|---|
 | `CODE_SCANNING_ENABLED` | `true` | trivy SARIF also pushed to Security → Code scanning; dependency-review runs on PRs. Both need Advanced Security on private repos — off today |
 | `ATTESTATIONS_ENABLED` | `true` | additionally emits `actions/attest-build-provenance` for each release image. Attestation storage on private repos needs GitHub Enterprise (SEC-I12) — off until the repo goes public |
+| `TRAIN_ATTESTATIONS_ENABLED` | `true` | same for the runtime train's publish job (`runtime-images.yml`) — a separate variable so train provenance stays off until verified on a tag build. NOT set by `setup-repo-protection.sh`; create it manually when enabling |
 | `BASE_MIRROR_REGISTRY` | registry host | optional private mirror for Dockerfile `FROM` bases — when set, build jobs docker-login to it with the `BASE_MIRROR_*` secrets below |
 | unset | (default) | the gated steps are skipped; SARIF/SBOM artifacts and cosign signing are unaffected |
 
@@ -216,6 +219,18 @@ cosign verify ghcr.io/tinyorbitvn/tinycdi-browser:rt-<date>.<n> \
   'https://github.com/tinyorbitvn/tinycdi/.github/workflows/runtime-images.yml@refs/heads/main'
 ```
 
+`runtime-images.json` itself is signed — `cosign sign-blob` in the same
+publish job — and the release carries `runtime-images.json.sigstore.json`
+next to it. Verify the manifest before taking digests from it:
+
+```sh
+cosign verify-blob --bundle runtime-images.json.sigstore.json \
+  runtime-images.json \
+  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+  --certificate-identity \
+  'https://github.com/tinyorbitvn/tinycdi/.github/workflows/runtime-images.yml@refs/heads/main'
+```
+
 ## Released image set
 
 `release.yml` builds, scans, signs and publishes exactly the images listed

.github/scripts/collect-publish-inputs.sh

@@ -61,7 +61,7 @@ want=()
 for img in "${expected[@]}"; do
   want+=("image-ref-$img" "sbom-$img")
 done
-want+=(release-chart release-assets)
+want+=(release-chart release-assets sbom-chart sbom-binaries)
 mapfile -t got < <(find "$DL" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort)
 [ "$(printf '%s\n' "${got[@]:-}")" = "$(printf '%s\n' "${want[@]}" | sort)" ] \
   || die "artifact folder set is [${got[*]:-none}] — expected exactly [${want[*]}]"
@@ -77,8 +77,10 @@ done
 "$SCRIPT_DIR/validate-image-refs.sh" "$OUT/refs" "${expected[@]}"
 
 # ---- 2. SBOMs: one per image, allowlisted name, valid JSON -------------
+#     sbom-chart / sbom-binaries are the dedicated SBOMs for the packaged
+#     Helm chart and the release binaries (SEC-17) — same shape check.
 mkdir -p "$OUT/sboms"
-for img in "${expected[@]}"; do
+for img in "${expected[@]}" chart binaries; do
   exact_files "$DL/sbom-$img" "sbom-$img.spdx.json"
   f="$DL/sbom-$img/sbom-$img.spdx.json"
   [ -s "$f" ] || die "missing SBOM for '$img'"

.github/tests/publish-inputs.test.sh

@@ -35,6 +35,12 @@ build_store() {
     echo "{\"spdxVersion\":\"SPDX-2.3\",\"name\":\"$img\"}" \
       > "$dl/sbom-$img/sbom-$img.spdx.json"
   done
+  # dedicated SBOMs for the packaged chart and the release binaries (SEC-17)
+  for extra in chart binaries; do
+    mkdir -p "$dl/sbom-$extra"
+    echo "{\"spdxVersion\":\"SPDX-2.3\",\"name\":\"$extra\"}" \
+      > "$dl/sbom-$extra/sbom-$extra.spdx.json"
+  done
 
   # packaged chart: real stamp script + real tar layout
   local cd_="$WORK/chart-src"
@@ -92,8 +98,12 @@ expect_ok() {
     || { echo "FAIL: bundle lacks kasmvnc sha256"; fails=1; }
   [ "$(find "$WORK/out/refs" -name '*.ref' | wc -l)" -eq "${#IMGS[@]}" ] \
     || { echo "FAIL: refs dir wrong"; fails=1; }
-  [ "$(find "$WORK/out/sboms" -name '*.json' | wc -l)" -eq "${#IMGS[@]}" ] \
+  [ "$(find "$WORK/out/sboms" -name '*.json' | wc -l)" -eq "$(( ${#IMGS[@]} + 2 ))" ] \
     || { echo "FAIL: sboms dir wrong"; fails=1; }
+  for extra in chart binaries; do
+    [ -f "$WORK/out/bundle/sbom-$extra.spdx.json" ] \
+      || { echo "FAIL: bundle lacks sbom-$extra.spdx.json"; fails=1; }
+  done
   echo "ok: happy path"
 }
 
@@ -136,6 +146,8 @@ mut_bad_chart() {
 }
 mut_no_kasmvnc()     { rm -f "$1/release-assets/kasmvnc-$KV-corresponding-source.tar.gz.sha256"; }
 mut_bad_sbom()       { echo 'not json' > "$1/sbom-backend/sbom-backend.spdx.json"; }
+mut_no_chart_sbom()  { rm -rf "$1/sbom-chart"; }
+mut_bad_bins_sbom()  { echo 'not json' > "$1/sbom-binaries/sbom-binaries.spdx.json"; }
 mut_missing_bin()    { rm -f "$1/release-assets/tinycdi-backend-$VERSION-linux-amd64"; }
 # v0.2 removed the api/gateway commands — a stale binary must not ride along.
 mut_stale_bin()      { local c=api; echo old > "$1/release-assets/tinycdi-$c-$VERSION-linux-amd64"; }
@@ -151,6 +163,8 @@ expect_fail "ref digest != chart digest" "digest" mut_wrong_digest
 expect_fail "chart stamped with wrong digests" "digest" mut_bad_chart
 expect_fail "missing kasmvnc checksum" "KasmVNC" mut_no_kasmvnc
 expect_fail "invalid sbom json" "not valid JSON" mut_bad_sbom
+expect_fail "missing chart sbom artifact" "artifact folder set" mut_no_chart_sbom
+expect_fail "invalid binaries sbom json" "not valid JSON" mut_bad_bins_sbom
 expect_fail "missing release binary" "missing" mut_missing_bin
 expect_fail "stale removed-component binary" "unexpected release-assets file" mut_stale_bin
 expect_fail "removed image artifact" "artifact folder set" mut_removed_image

.github/tests/supply-chain-hardening.test.sh

@@ -0,0 +1,179 @@
+#!/usr/bin/env bash
+# supply-chain-hardening.test.sh — structural guards for the v1.0
+# supply-chain hardening pass on the publishing workflows:
+#
+#   * scan-env hygiene (SUPF-10): every scan job must validate the
+#     artifact-supplied image ref against the strict
+#     ghcr.io/tinyorbitvn/tinycdi-<img>@sha256:<64hex> regex BEFORE it
+#     lands in $GITHUB_ENV — a newline in a forged ref file would
+#     otherwise inject arbitrary env vars into the scan job.
+#   * runtime manifest signing (SEC-17): runtime-images.json is the
+#     deployment contract consumers pin digests from — it ships with a
+#     cosign sign-blob Sigstore bundle on the runtime-* release, signed
+#     in the same job that publishes it.
+#   * train provenance parity: the runtime train's publish job carries
+#     the same var-gated actions/attest-build-provenance legs as
+#     release.yml — gated on its own TRAIN_ATTESTATIONS_ENABLED variable
+#     (defaults OFF until verified), with attestations:write scoped to
+#     the publish job only.
+#   * release-asset SBOMs (SEC-17): the packaged Helm chart and the
+#     release binaries get dedicated syft SBOMs, validated by
+#     collect-publish-inputs.sh and signed/released like every other
+#     asset.
+#   * digest-addressability is documented: gate-failed digest-pushes
+#     stay pullable-by-digest but are never tagged/signed — the docs
+#     must say so.
+set -uo pipefail
+
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+IMG="$ROOT/.github/workflows/images.yml"
+REL="$ROOT/.github/workflows/release.yml"
+TRAIN="$ROOT/.github/workflows/runtime-images.yml"
+COLLECT="$ROOT/.github/scripts/collect-publish-inputs.sh"
+README="$ROOT/.github/README.md"
+PROV="$ROOT/docs/security/provenance.md"
+fails=0
+
+chk() { # chk <desc> <file> <regex>
+  local desc="$1" file="$2" pat="$3"
+  if ! grep -qE -e "$pat" "$file"; then
+    echo "FAIL: $desc"; fails=1
+  fi
+}
+chk_absent() { # chk_absent <desc> <file> <regex>
+  local desc="$1" file="$2" pat="$3"
+  if grep -qE -e "$pat" "$file"; then
+    echo "FAIL: $desc"; fails=1
+  fi
+}
+order() { # order <desc> <file> <earlier-regex> <later-regex>
+  local desc="$1" file="$2" a="$3" b="$4" la lb
+  la="$(grep -nE "$a" "$file" | head -1 | cut -d: -f1)"
+  lb="$(grep -nE "$b" "$file" | head -1 | cut -d: -f1)"
+  if [ -z "$la" ] || [ -z "$lb" ] || [ "$la" -ge "$lb" ]; then
+    echo "FAIL: $desc"; fails=1
+  fi
+}
+
+# ---------------------------------------------------------------
+# SUPF-10: scan-env hygiene — the ref is validated BEFORE the
+# $GITHUB_ENV write, inside the same 'resolve scan target' step.
+# ---------------------------------------------------------------
+for wf in "$IMG" "$REL" "$TRAIN"; do
+  name="$(basename "$wf")"
+  block="$(awk '
+    /^      - name: resolve scan target/ { inb=1 }
+    inb && /^      - / && !/resolve scan target/ { inb=0 }
+    inb { print }
+  ' "$wf")"
+  [ -n "$block" ] || { echo "FAIL: $name: no 'resolve scan target' step"; fails=1; continue; }
+  grep -qF 'SCAN_REF=' <<< "$block" \
+    || { echo "FAIL: $name: resolve step does not write SCAN_REF"; fails=1; }
+  grep -qF 'ghcr\.io/tinyorbitvn/tinycdi-${IMG}@sha256:[0-9a-f]{64}' <<< "$block" \
+    || { echo "FAIL: $name: resolve step lacks the strict ref regex"; fails=1; }
+  # the guard must precede the env write
+  la="$(grep -nF 'tinycdi-${IMG}@sha256' <<< "$block" | head -1 | cut -d: -f1)"
+  lb="$(grep -nF 'SCAN_REF=' <<< "$block" | head -1 | cut -d: -f1)"
+  { [ -n "$la" ] && [ -n "$lb" ] && [ "$la" -lt "$lb" ]; } \
+    || { echo "FAIL: $name: ref regex is not evaluated before the SCAN_REF env write"; fails=1; }
+  # a non-conforming ref must fail the step, not fall through
+  grep -qE 'exit 1' <<< "$block" \
+    || { echo "FAIL: $name: invalid ref does not fail the scan job"; fails=1; }
+done
+
+# Unit-test the guard regex itself — the same pattern the scan steps run.
+IMG=browser
+accept() { [[ "$1" =~ ^ghcr\.io/tinyorbitvn/tinycdi-${IMG}@sha256:[0-9a-f]{64}$ ]]; }
+D64="$(printf 'a%.0s' $(seq 64))"
+accept "ghcr.io/tinyorbitvn/tinycdi-browser@sha256:$D64" \
+  || { echo "FAIL: regex rejects a valid ref"; fails=1; }
+for bad in \
+  "ghcr.io/tinyorbitvn/tinycdi-browser@sha256:$D64
+POISONED=1" \
+  "ghcr.io/evil/tinycdi-browser@sha256:$D64" \
+  "ghcr.io/tinyorbitvn/tinycdi-backend@sha256:$D64" \
+  "ghcr.io/tinyorbitvn/tinycdi-browser@sha256:$(printf 'a%.0s' $(seq 63))" \
+  "ghcr.io/tinyorbitvn/tinycdi-browser@sha256:$(printf 'A%.0s' $(seq 64))" \
+  "ghcr.io/tinyorbitvn/tinycdi-browser:latest" \
+  " ghcr.io/tinyorbitvn/tinycdi-browser@sha256:$D64" \
+  "local"; do
+  if accept "$bad"; then
+    echo "FAIL: regex accepted $(printf '%q' "$bad")"; fails=1
+  fi
+done
+
+# ---------------------------------------------------------------
+# SEC-17: runtime-images.json is sign-blob'd by the publish job and
+# the bundle rides the runtime-* release next to the manifest.
+# ---------------------------------------------------------------
+chk "runtime-images: sign-blob the manifest" "$TRAIN" 'cosign sign-blob .*runtime-images\.json'
+chk "runtime-images: sigstore bundle for the manifest" "$TRAIN" 'runtime-images\.json\.sigstore\.json'
+chk "runtime-images: bundle uploaded to the release" "$TRAIN" 'gh release (upload|create).*runtime-images\.json\.sigstore\.json|runtime-images\.json runtime-images\.json\.sigstore\.json'
+order "runtime-images: manifest signed before the release write" "$TRAIN" \
+  'cosign sign-blob' 'create or update the runtime release'
+
+# the consumption docs carry the exact verify-blob line with the
+# pinned signer identity (workflow path + refs/heads/main)
+chk "docs: README documents manifest verify-blob" "$README" \
+  'cosign verify-blob .*runtime-images\.json'
+chk "docs: README pins the train signer identity" "$README" \
+  'runtime-images\.yml@refs/heads/main'
+chk "docs: images.md points at the signed manifest" "$ROOT/docs/images.md" \
+  'verify-blob|sigstore\.json'
+
+# ---------------------------------------------------------------
+# Train build-provenance parity with release.yml — var-gated OFF by
+# default (TRAIN_ATTESTATIONS_ENABLED), attestations:write only on
+# the publish job.
+# ---------------------------------------------------------------
+PUB="$(sed -n '/^  publish:/,$p' "$TRAIN")"
+chk "runtime-images: publish job holds attestations: write" <(echo "$PUB") \
+  'attestations: write'
+n="$(grep -c 'attestations: write' "$TRAIN")"
+[ "$n" -eq 1 ] \
+  || { echo "FAIL: runtime-images: attestations: write appears $n times (want exactly the publish job)"; fails=1; }
+chk "runtime-images: attestations gated on TRAIN_ATTESTATIONS_ENABLED" "$TRAIN" \
+  "TRAIN_ATTESTATIONS_ENABLED == 'true'"
+chk "runtime-images: sha-pinned attest-build-provenance" "$TRAIN" \
+  'actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8'
+for i in linux-base linux-desktop browser; do
+  chk "runtime-images: provenance attestation for $i" "$TRAIN" \
+    "attest build provenance — $i"
+done
+# each attest step is var-gated: three ifs + the subject-resolver
+n="$(grep -c "TRAIN_ATTESTATIONS_ENABLED == 'true'" "$TRAIN")"
+[ "$n" -ge 4 ] \
+  || { echo "FAIL: runtime-images: only $n TRAIN_ATTESTATIONS_ENABLED gates (want resolver + 3 legs)"; fails=1; }
+order "runtime-images: subjects resolved before attestation" "$TRAIN" \
+  'resolve attestation subjects' 'attest build provenance — linux-base'
+order "runtime-images: attestation before rt tag promotion" "$TRAIN" \
+  'attest build provenance — linux-base' 'name: promote rt tag'
+chk "docs: README documents TRAIN_ATTESTATIONS_ENABLED" "$README" \
+  'TRAIN_ATTESTATIONS_ENABLED'
+
+# ---------------------------------------------------------------
+# SEC-17: dedicated SBOMs for the packaged chart and the release
+# binaries — built by the producing job, validated by
+# collect-publish-inputs.sh, shipped + signed in the bundle.
+# ---------------------------------------------------------------
+chk "release: chart SBOM step" "$REL" 'sbom-chart\.spdx\.json'
+chk "release: chart SBOM artifact" "$REL" 'name: sbom-chart'
+chk "release: binaries SBOM step" "$REL" 'sbom-binaries\.spdx\.json'
+chk "release: binaries SBOM artifact" "$REL" 'name: sbom-binaries'
+chk "release: publish downloads sbom-chart" "$REL" 'gh run download.*-n sbom-chart'
+chk "release: publish downloads sbom-binaries" "$REL" 'gh run download.*-n sbom-binaries'
+chk "collect: validates sbom-chart" "$COLLECT" 'sbom-chart'
+chk "collect: validates sbom-binaries" "$COLLECT" 'sbom-binaries'
+chk "collect: extra SBOMs join the signed bundle" "$COLLECT" \
+  'cp .*sboms/.*\.spdx\.json.*bundle'
+
+# ---------------------------------------------------------------
+# F3 docs: digest-addressable gate-failed manifests are documented.
+# ---------------------------------------------------------------
+chk "docs: digest-addressable != released documented" "$PROV" \
+  'pullable|digest-addressable'
+chk "docs: unsigned digest fails cosign verify" "$PROV" \
+  'never (signed|tagged)|unsigned'
+
+[ "$fails" -eq 0 ] && echo "supply-chain-hardening: all guards pass"
+exit "$fails"

.github/workflows/images.yml

@@ -515,12 +515,18 @@ jobs:
 
       - name: resolve scan target
         run: |
+          # SUPF-10: the ref file is artifact content — validate the
+          # strict repo@sha256 form before it lands in $GITHUB_ENV; a
+          # newline in a forged ref would otherwise inject env vars.
           REF="$(cat "refs/$IMG.ref")"
           if [ "$REF" = "local" ]; then
             echo "SCAN_MODE=tar" >> "$GITHUB_ENV"
-          else
+          elif [[ "$REF" =~ ^ghcr\.io/tinyorbitvn/tinycdi-${IMG}@sha256:[0-9a-f]{64}$ ]]; then
             echo "SCAN_MODE=registry" >> "$GITHUB_ENV"
             echo "SCAN_REF=$REF" >> "$GITHUB_ENV"
+          else
+            echo "::error::ref for '$IMG' is not ghcr.io/tinyorbitvn/tinycdi-$IMG@sha256:<64hex>"
+            exit 1
           fi
 
       - name: download image tar (non-publishing runs)