From e38857baac88d695efa5edcdfa4eb89e7644efb3 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 21:49:23 +0000 Subject: [PATCH 1/2] fix(ci): re-attest hypatia credential-read-check at 29a9e4a8 after #686 #686 (Dependabot actions bump) moved hypatia-scan.yml's reusable call to hyperpolymath/hypatia/.github/workflows/credential-read-check.yml from b93f0615 to 29a9e4a8 without updating the pinned-source manifest or the closure receipts, so 'Pinned source closure' (and E2E, which requires it) has failed on main since 6a6b748 with SOURCE-ACTIVE-MISSING, and on every PR merged with main. Fetched both commits from hyperpolymath/hypatia: the workflow is byte- identical at both (blob 15c597ea, sha256 645e8ac8...), and its only uses are step-security/harden-runner@1d88384f. So only the commit moves: - pinned-reusable-sources.json: the record's commit -> 29a9e4a8 (blob, content_sha256 and uses unchanged, verified against the fetched file). - lock-closure-receipts.tsv: the receipt block re-emitted with audit-lock-closure.sh --emit-receipt from the fetched source. audit-pinned-reusable-sources.sh: findings 1 -> 0. audit-lock-closure.sh: fatal 1 -> 0 (review/accepted/not-run counts unchanged). pinned-reusable-sources-test: 55/55. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01J4QvDhFeexmokLqLRfZNce --- scripts/lib/lock-closure-receipts.tsv | 4 ++-- scripts/lib/pinned-reusable-sources.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/lib/lock-closure-receipts.tsv b/scripts/lib/lock-closure-receipts.tsv index d53a646d..1b854662 100644 --- a/scripts/lib/lock-closure-receipts.tsv +++ b/scripts/lib/lock-closure-receipts.tsv @@ -76,5 +76,5 @@ metadatastician/standards ea2f4d8a886585ed442687ae63d4ad7493379ba4 oven-sh/setup # Reviewed source: hyperpolymath/proven-cicd-and-workflows/.github/workflows/secret-scanner-reusable.yml@1fd5e6371a50f0fc3687206a5adca0cdf7b39aa6 hyperpolymath/proven-cicd-and-workflows 1fd5e6371a50f0fc3687206a5adca0cdf7b39aa6 actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 -# Reviewed source: hyperpolymath/hypatia/.github/workflows/credential-read-check.yml@b93f0615a61ee4c05af66eb526538eab47120aaa -hyperpolymath/hypatia b93f0615a61ee4c05af66eb526538eab47120aaa step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a +# Reviewed source: hyperpolymath/hypatia/.github/workflows/credential-read-check.yml@29a9e4a881b6a3cc4cd27fcbc0886ffbadcbc5fe +hyperpolymath/hypatia 29a9e4a881b6a3cc4cd27fcbc0886ffbadcbc5fe step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a diff --git a/scripts/lib/pinned-reusable-sources.json b/scripts/lib/pinned-reusable-sources.json index 02fc8533..8c666757 100644 --- a/scripts/lib/pinned-reusable-sources.json +++ b/scripts/lib/pinned-reusable-sources.json @@ -731,7 +731,7 @@ }, { "repository": "hyperpolymath/hypatia", - "commit": "b93f0615a61ee4c05af66eb526538eab47120aaa", + "commit": "29a9e4a881b6a3cc4cd27fcbc0886ffbadcbc5fe", "path": ".github/workflows/credential-read-check.yml", "blob": "15c597ea720a5affb861dab058084f7245ee4cbd", "content_sha256": "645e8ac8ff118c33f2dc941ec46f1abac895f7836339736841c269e4b9dc084d", From d74a1c4452e4d9e1b3330bc52066547a477a941b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 21:56:35 +0000 Subject: [PATCH 2/2] fix(ci): Dependabot leaves attested reusable workflows alone; add the re-attest tool #686 showed the gap: Dependabot bumped an attested reusable-workflow pin (hypatia credential-read-check) without re-attesting it, and its own 'Pinned source closure' and 'E2E tests' failures did not stop the merge, so main went red. Dependabot cannot re-attest, so these pins must not be its to move. - .github/dependabot.yml: a name-only ignore for each ACTIVE source in scripts/lib/pinned-reusable-sources.json, using Dependabot's own name for a reusable workflow (owner/repo/.github/workflows/file.yml, as in #686's body). Name-only, like the CodeQL hold: a SHA re-bump slips past a version-scoped ignore. Ordinary actions keep updating. - scripts/reattest-pinned-source.sh: the deliberate path. It moves one pin in the manifest and re-emits its receipt (audit-lock-closure.sh --emit-receipt) only when no judgement is needed: the file is byte- identical at the new commit, every caller already calls it, no other record shares the old commit, and the receipt block has the usual layout. Anything else is refused with the reason, exit 3, nothing changed. It ends by running the pinned-source audit. Replayed on main@dec1e7c it reproduces #694 byte for byte. - scripts/tests/dependabot-attested-ignore-test.sh: the ignore list equals the manifest's active sources (missing, stale and version-scoped entries each fail; negative controls). Ruby YAML, already on the runner. - scripts/tests/reattest-pinned-source-test.sh: fixture move, idempotent rerun, five refusals, bad arguments, and the #686 replay when github.com and the commit are available (NOT RUN otherwise, never passed). 29/29. - e2e.yml: both suites run in the 'Pinned source closure' job. No new action, so actions.lock is unchanged (check-lock-sync clean). Making 'Pinned source closure' and 'E2E tests' required on main is a branch-protection setting for the owner; this change cannot make it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01J4QvDhFeexmokLqLRfZNce --- .github/dependabot.yml | 27 ++++ .github/workflows/e2e.yml | 4 + scripts/reattest-pinned-source.sh | 144 ++++++++++++++++++ .../tests/dependabot-attested-ignore-test.sh | 76 +++++++++ scripts/tests/reattest-pinned-source-test.sh | 114 ++++++++++++++ 5 files changed, 365 insertions(+) create mode 100755 scripts/reattest-pinned-source.sh create mode 100644 scripts/tests/dependabot-attested-ignore-test.sh create mode 100644 scripts/tests/reattest-pinned-source-test.sh diff --git a/.github/dependabot.yml b/.github/dependabot.yml index b70be4a5..3d0969b4 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -24,6 +24,33 @@ { dependency-name: "github/codeql-action", }, + # ATTESTED reusable workflows: each is a record in + # scripts/lib/pinned-reusable-sources.json with closure receipts in + # scripts/lib/lock-closure-receipts.tsv. Dependabot cannot update those, + # so a bump from it breaks 'Pinned source closure' on main (#686 did). + # Move these pins only by a deliberate PR that runs + # scripts/reattest-pinned-source.sh. Name-only entries, as for the + # CodeQL hold: a SHA re-bump slips past version-scoped ignores. + # scripts/tests/dependabot-attested-ignore-test.sh keeps this list + # equal to the manifest's active sources. + { + dependency-name: "hyperpolymath/hypatia/.github/workflows/credential-read-check.yml", + }, + { + dependency-name: "hyperpolymath/proven-cicd-and-workflows/.github/workflows/secret-scanner-reusable.yml", + }, + { + dependency-name: "metadatastician/standards/.github/workflows/governance-reusable.yml", + }, + { + dependency-name: "metadatastician/standards/.github/workflows/hypatia-scan-reusable.yml", + }, + { + dependency-name: "metadatastician/standards/.github/workflows/mirror-reusable.yml", + }, + { + dependency-name: "metadatastician/standards/.github/workflows/scorecard-reusable.yml", + }, ], }, # Rust dependencies. Keep every Cargo project in one update entry so the diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml index 1c20f698..44173d77 100644 --- a/.github/workflows/e2e.yml +++ b/.github/workflows/e2e.yml @@ -78,6 +78,10 @@ jobs: run: bash scripts/tests/pinned-reusable-sources-test.sh - name: Authenticate pinned reusable sources run: bash scripts/audit-pinned-reusable-sources.sh --fetch-sources + - name: Dependabot never bumps an attested reusable workflow + run: bash scripts/tests/dependabot-attested-ignore-test.sh + - name: Re-attestation tool moves only an unchanged source + run: bash scripts/tests/reattest-pinned-source-test.sh - name: Verify live scanner caller and negative controls run: bash scripts/tests/scanner-caller-contract-test.sh - name: Authenticate historical scanner source for archived preflight diff --git a/scripts/reattest-pinned-source.sh b/scripts/reattest-pinned-source.sh new file mode 100755 index 00000000..0caf2375 --- /dev/null +++ b/scripts/reattest-pinned-source.sh @@ -0,0 +1,144 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell +# +# reattest-pinned-source.sh — move one attested reusable-workflow pin to a new +# commit, in scripts/lib/pinned-reusable-sources.json and +# scripts/lib/lock-closure-receipts.tsv, after the callers have moved. +# +# Dependabot may not bump these pins (.github/dependabot.yml ignores them; see +# scripts/tests/dependabot-attested-ignore-test.sh): it cannot re-attest, and +# its bump breaks 'Pinned source closure' on main (#686). This is the +# deliberate path. It automates ONLY the case that needs no judgement, and +# refuses (exit 3, nothing changed) every other case with the reason: +# +# - the workflow file is byte-identical at the new commit (same git blob and +# sha256 as attested), so its `uses:` edges are unchanged; a changed file +# needs a human review of its new edges and lock records; +# - every caller recorded in the manifest already calls the new commit; +# - no other active record shares the old commit (receipt rows are the union +# over all files called at one owner/repo@sha, so splitting them is a +# review decision); +# - the receipt block is the one-header-then-rows layout this file uses. +# +# The receipt rows are re-emitted with audit-lock-closure.sh --emit-receipt, +# never hand-written. Afterwards it runs audit-pinned-reusable-sources.sh when +# the repository has it, and fails if that reports a finding. +# +# Usage: +# reattest-pinned-source.sh OWNER/REPO PATH NEW_SHA [--source FILE] [--repo-root DIR] +# PATH the workflow path inside OWNER/REPO, e.g. .github/workflows/x.yml +# --source the file at NEW_SHA, read instead of fetching it (offline, tests) +# +# Exit: 0 re-attested, or already attested at NEW_SHA · 3 refused (needs a +# human) · 64 usage · 69 a tool or the fetch is unavailable. +set -euo pipefail + +SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +die() { printf 'reattest-pinned-source: %s\n' "$1" >&2; exit "${2:-3}"; } +usage() { sed -n '/^# Usage:/,/^# Exit:/p' "$0" | sed 's/^# \{0,1\}//' >&2; exit 64; } + +[[ $# -ge 3 ]] || usage +REPOSITORY="$1" WF_PATH="$2" NEW="$3"; shift 3 +SOURCE="" ROOT="" +while [[ $# -gt 0 ]]; do + case "$1" in + --source) [[ $# -ge 2 ]] || usage; SOURCE="$2"; shift 2 ;; + --repo-root) [[ $# -ge 2 ]] || usage; ROOT="$2"; shift 2 ;; + *) usage ;; + esac +done +[[ "$REPOSITORY" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]] || die "OWNER/REPO expected, got ${REPOSITORY@Q}" 64 +[[ "$WF_PATH" =~ ^\.github/workflows/[A-Za-z0-9_./-]+\.ya?ml$ && "$WF_PATH" != *..* ]] \ + || die "PATH must be a .github/workflows/*.yml path, got ${WF_PATH@Q}" 64 +[[ "$NEW" =~ ^[0-9a-f]{40}$ ]] || die "NEW_SHA must be 40 lowercase hex" 64 +for tool in jq git sha256sum; do command -v "$tool" >/dev/null || die "$tool is required" 69; done +if [[ -z "$ROOT" ]]; then ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" || die "not in a git work tree; pass --repo-root" 64; fi +MANIFEST="$ROOT/scripts/lib/pinned-reusable-sources.json" +RECEIPTS="$ROOT/scripts/lib/lock-closure-receipts.tsv" +[[ -f "$MANIFEST" && -f "$RECEIPTS" ]] || die "no manifest or receipts under $ROOT/scripts/lib" 64 + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/reattest.XXXXXX")"; trap 'rm -rf "$WORK"' EXIT + +# The one active record for this workflow. +jq -c --arg r "$REPOSITORY" --arg p "$WF_PATH" \ + '[.sources[] | select(.repository == $r and .path == $p and .role == "active")]' "$MANIFEST" > "$WORK/rec.json" +case "$(jq length "$WORK/rec.json")" in + 1) ;; + 0) die "no active record for $REPOSITORY/$WF_PATH in the manifest; attesting a new source is a review, not a move" ;; + *) die "more than one active record for $REPOSITORY/$WF_PATH; fix the manifest first" ;; +esac +OLD="$(jq -r '.[0].commit' "$WORK/rec.json")" +BLOB="$(jq -r '.[0].blob' "$WORK/rec.json")" SUM="$(jq -r '.[0].content_sha256' "$WORK/rec.json")" +if [[ "$OLD" == "$NEW" ]]; then echo "reattest-pinned-source: $REPOSITORY/$WF_PATH is already attested at $NEW"; exit 0; fi + +# Receipt rows are a union per owner/repo@sha: refuse to split a shared one. +shared="$(jq -r --arg r "$REPOSITORY" --arg p "$WF_PATH" --arg c "$OLD" \ + '.sources[] | select(.repository == $r and .commit == $c and .path != $p and .role == "active") | .path' "$MANIFEST")" +[[ -z "$shared" ]] || die "$REPOSITORY@${OLD:0:12} is also attested for: $shared. Its receipt rows are the union over those files, so moving one alone is a review decision; move them together by hand" + +# Every recorded caller must already call the new commit. +call="$REPOSITORY/$WF_PATH@" +while read -r wf; do + [[ -n "$wf" ]] || continue + [[ -f "$ROOT/$wf" ]] || die "caller $wf is in the manifest but not in the repository" + grep -qF -- "$call$NEW" "$ROOT/$wf" || die "caller $wf does not call $call$NEW yet; move the caller pin first" + ! grep -qF -- "$call$OLD" "$ROOT/$wf" || die "caller $wf still calls $call$OLD as well" +done < <(jq -r '.[0].callers[]?.workflow' "$WORK/rec.json" | sort -u) + +# The file at the new commit, from --source or fetched. +if [[ -n "$SOURCE" ]]; then + [[ -f "$SOURCE" ]] || die "--source $SOURCE is not a file" 64 + cp -- "$SOURCE" "$WORK/src.yml" +else + git init -q --bare "$WORK/fetch.git" + git -C "$WORK/fetch.git" fetch -q --depth=1 "https://github.com/$REPOSITORY" "$NEW" 2> "$WORK/fetch.err" \ + || die "could not fetch $REPOSITORY@$NEW: $(head -c 300 "$WORK/fetch.err")" 69 + git -C "$WORK/fetch.git" cat-file -p "$NEW:$WF_PATH" > "$WORK/src.yml" 2> "$WORK/fetch.err" \ + || die "$WF_PATH does not exist at $REPOSITORY@$NEW" 3 +fi +new_blob="$(git hash-object -- "$WORK/src.yml")" +new_sum="$(sha256sum -- "$WORK/src.yml" | cut -d' ' -f1)" +if [[ "$new_blob" != "$BLOB" || "$new_sum" != "$SUM" ]]; then + { + printf 'reattest-pinned-source: %s changed between %s and %s (blob %s -> %s).\n' "$WF_PATH" "${OLD:0:12}" "${NEW:0:12}" "${BLOB:0:12}" "${new_blob:0:12}" + printf 'reattest-pinned-source: a changed file needs a human review of its uses: edges and lock records; its uses: now are:\n' + grep -nE '^[[:space:]-]*uses:' "$WORK/src.yml" | sed 's/^/ /' || true + } >&2 + exit 3 +fi + +# The receipt block: one '# Reviewed source:' header for this file at OLD, +# followed by exactly the rows for owner/repoOLD. +header="# Reviewed source: $REPOSITORY/$WF_PATH@$OLD" +[[ "$(grep -cxF -- "$header" "$RECEIPTS")" -eq 1 ]] || die "expected one receipt header '$header' in $RECEIPTS" +rows_total="$(awk -F'\t' -v r="$REPOSITORY" -v c="$OLD" '$1 == r && $2 == c' "$RECEIPTS" | wc -l)" +rows_after="$(awk -F'\t' -v h="$header" -v r="$REPOSITORY" -v c="$OLD" ' + $0 == h { in_block = 1; next } + in_block && $1 == r && $2 == c { n++; next } + { in_block = 0 } + END { print n + 0 }' "$RECEIPTS")" +[[ "$rows_total" -eq "$rows_after" ]] \ + || die "receipt rows for $REPOSITORY@${OLD:0:12} are not all directly under '$header'; re-attest by hand" +bash "$SELF_DIR/audit-lock-closure.sh" --emit-receipt "$REPOSITORY@$NEW" --source "$WORK/src.yml" > "$WORK/rows.tsv" \ + || die "audit-lock-closure.sh --emit-receipt failed" 69 + +# Write both files from temporaries, so a failure leaves neither half-updated. +jq --arg r "$REPOSITORY" --arg p "$WF_PATH" --arg old "$OLD" --arg new "$NEW" ' + (.sources[] | select(.repository == $r and .path == $p and .role == "active")) |= + (.commit = $new | if .source_url then .source_url |= gsub($old; $new) else . end)' "$MANIFEST" > "$WORK/manifest.json" +awk -F'\t' -v h="$header" -v r="$REPOSITORY" -v c="$OLD" -v nh="# Reviewed source: $REPOSITORY/$WF_PATH@$NEW" -v rows="$WORK/rows.tsv" ' + $0 == h { print nh; while ((getline line < rows) > 0) print line; skip = 1; next } + skip && $1 == r && $2 == c { next } + { skip = 0; print }' "$RECEIPTS" > "$WORK/receipts.tsv" +cat -- "$WORK/manifest.json" > "$MANIFEST" +cat -- "$WORK/receipts.tsv" > "$RECEIPTS" +echo "reattest-pinned-source: $REPOSITORY/$WF_PATH ${OLD:0:12} -> ${NEW:0:12} (content unchanged: blob ${BLOB:0:12})" + +if [[ -f "$ROOT/scripts/audit-pinned-reusable-sources.sh" ]]; then + out="$(cd "$ROOT" && bash scripts/audit-pinned-reusable-sources.sh 2>&1)" || true + printf '%s\n' "$out" | grep -E '^(FATAL|pinned reusable sources:)' | sed 's/^/ /' + if grep -q '^FATAL' <<< "$out"; then die "the pinned-source audit still reports a finding (above)"; fi +else + echo "reattest-pinned-source: NOT RUN: no scripts/audit-pinned-reusable-sources.sh under $ROOT" +fi diff --git a/scripts/tests/dependabot-attested-ignore-test.sh b/scripts/tests/dependabot-attested-ignore-test.sh new file mode 100644 index 00000000..fae96ff4 --- /dev/null +++ b/scripts/tests/dependabot-attested-ignore-test.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# +# Every ACTIVE reusable-workflow source attested in +# scripts/lib/pinned-reusable-sources.json must be a name-only Dependabot +# ignore entry in .github/dependabot.yml (github-actions), and no ignore entry +# may name an attested-looking reusable workflow that is not active. Dependabot +# cannot re-attest a pin, so a bump from it breaks 'Pinned source closure' on +# main (#686 did). Such pins move only by a deliberate PR that runs +# scripts/reattest-pinned-source.sh. +# +# Usage: bash scripts/tests/dependabot-attested-ignore-test.sh +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +command -v ruby >/dev/null || { echo "NOT RUN: needs ruby (YAML parser)" >&2; exit 2; } +command -v jq >/dev/null || { echo "NOT RUN: needs jq" >&2; exit 2; } + +pass=0 fail=0 +ok() { echo "PASS: $1"; pass=$((pass+1)); } +no() { echo "FAIL: $1"; fail=$((fail+1)); } +TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT + +# drift -> prints differences, exit 1 if any. +drift() { + local want have + want="$(jq -r '.sources[] | select(.role == "active") | "\(.repository)/\(.path)"' "$2" | LC_ALL=C sort -u)" + have="$(ruby -ryaml -e ' + y = YAML.safe_load(File.read(ARGV[0])) + (y["updates"] || []).each do |u| + next unless u["package-ecosystem"] == "github-actions" + (u["ignore"] || []).each do |i| + n = i["dependency-name"].to_s + # A reusable-workflow ignore is any name with a workflow path in it. + next unless n.include?("/.github/workflows/") + # Name-only: a versions/update-types scope lets a SHA re-bump through. + puts "SCOPED\t#{n}" if i["versions"] || i["update-types"] + puts n + end + end' "$1" | LC_ALL=C sort -u)" + local scoped missing extra + scoped="$(grep '^SCOPED' <<< "$have" || true)" + have="$(grep -v '^SCOPED' <<< "$have" || true)" + missing="$(comm -23 <(printf '%s\n' "$want") <(printf '%s\n' "$have") | grep . || true)" + extra="$(comm -13 <(printf '%s\n' "$want") <(printf '%s\n' "$have") | grep . || true)" + [[ -z "$missing$extra$scoped" ]] && return 0 + local n + while read -r n; do [[ -n "$n" ]] && printf 'missing ignore (attested, Dependabot may bump it): %s\n' "$n"; done <<< "$missing" + while read -r n; do [[ -n "$n" ]] && printf 'stale ignore (not an active attested source): %s\n' "$n"; done <<< "$extra" + [[ -n "$scoped" ]] && printf '%s (must be name-only)\n' "$scoped" + return 1 +} + +DEP="$ROOT/.github/dependabot.yml" MAN="$ROOT/scripts/lib/pinned-reusable-sources.json" + +if out="$(drift "$DEP" "$MAN")"; then ok "the repo's ignore list equals the manifest's active sources" +else no "the repo's ignore list drifted from the manifest:"; printf ' %s\n' "$out"; fi + +# Negative controls: the check must catch each kind of drift. +jq '.sources += [{repository: "acme/new", path: ".github/workflows/x.yml", commit: "0000000000000000000000000000000000000000", role: "active"}]' "$MAN" > "$TMP/man-extra.json" +if out="$(drift "$DEP" "$TMP/man-extra.json")"; then no "a newly attested source without an ignore passed" +elif grep -q 'missing ignore.*acme/new/.github/workflows/x.yml' <<< "$out"; then ok "a newly attested source without an ignore is caught" +else no "wrong report for a missing ignore: $out"; fi + +jq '(.sources[] | select(.repository == "hyperpolymath/hypatia") | .role) = "retained-review"' "$MAN" > "$TMP/man-retired.json" +if out="$(drift "$DEP" "$TMP/man-retired.json")"; then no "an ignore for a no-longer-active source passed" +elif grep -q 'stale ignore.*hyperpolymath/hypatia' <<< "$out"; then ok "an ignore left behind for a retired source is caught" +else no "wrong report for a stale ignore: $out"; fi + +sed 's|dependency-name: "hyperpolymath/hypatia/.github/workflows/credential-read-check.yml",|&\n versions: [">= 0"],|' "$DEP" > "$TMP/dep-scoped.yml" +if out="$(drift "$TMP/dep-scoped.yml" "$MAN")"; then no "a version-scoped ignore passed" +elif grep -q 'SCOPED.*hypatia.*must be name-only' <<< "$out"; then ok "a version-scoped ignore is caught (a SHA re-bump would slip past it)" +else no "wrong report for a scoped ignore: $out"; fi + +echo "dependabot-attested-ignore-test: $pass passed, $fail failed" +[[ "$fail" -eq 0 ]] diff --git a/scripts/tests/reattest-pinned-source-test.sh b/scripts/tests/reattest-pinned-source-test.sh new file mode 100644 index 00000000..78063b82 --- /dev/null +++ b/scripts/tests/reattest-pinned-source-test.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# +# Tests for scripts/reattest-pinned-source.sh. A fixture repository covers the +# move and every refusal offline. A replay of the real #686 breakage (main at +# dec1e7c, hypatia credential-read-check b93f0615 -> 29a9e4a8) runs when +# github.com is reachable, and is reported NOT RUN, never passed, when not. +# +# Usage: bash scripts/tests/reattest-pinned-source-test.sh +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +RE="$ROOT/scripts/reattest-pinned-source.sh" +for t in jq git sha256sum awk; do command -v "$t" >/dev/null || { echo "NOT RUN: needs $t" >&2; exit 2; }; done + +pass=0 fail=0 notrun=0 +ok() { echo "PASS: $1"; pass=$((pass+1)); } +no() { echo "FAIL: $1"; fail=$((fail+1)); } +check() { if [[ "$2" == "$3" ]]; then ok "$1"; else no "$1 (want ${2@Q}, got ${3@Q})"; fi; } +TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT + +OLD=1111111111111111111111111111111111111111 +NEW=2222222222222222222222222222222222222222 +CO=3d3c42e5aac5ba805825da76410c181273ba90b1 + +# fixture [caller-ref] -> a minimal repository with one attested source. +fixture() { + local d="$1" ref="${2:-$NEW}" + rm -rf "$d"; mkdir -p "$d/.github/workflows" "$d/scripts/lib" + printf 'on: push\njobs:\n x:\n uses: acme/lib/.github/workflows/r.yml@%s\n' "$ref" > "$d/.github/workflows/caller.yml" + printf 'on: workflow_call\njobs:\n a:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@%s\n' "$CO" > "$d/src.yml" + local blob sum + blob="$(git hash-object "$d/src.yml")"; sum="$(sha256sum "$d/src.yml" | cut -d' ' -f1)" + jq -n --arg old "$OLD" --arg blob "$blob" --arg sum "$sum" --arg co "actions/checkout@$CO" '{ + version: 1, + sources: [{repository: "acme/lib", commit: $old, path: ".github/workflows/r.yml", + blob: $blob, content_sha256: $sum, role: "active", + callers: [{workflow: ".github/workflows/caller.yml", job: "x"}], + uses: [{kind: "step", ref: $co}], + source_url: ("https://github.com/acme/lib/blob/" + $old + "/.github/workflows/r.yml")}], + dependencies: {}}' > "$d/scripts/lib/pinned-reusable-sources.json" + { printf '# SPDX-License-Identifier: MPL-2.0\n# receipts\n\n' + printf '# Reviewed source: acme/lib/.github/workflows/r.yml@%s\n' "$OLD" + printf 'acme/lib\t%s\tactions/checkout@%s\n\n' "$OLD" "$CO" + printf '# Reviewed source: other/x/.github/workflows/y.yml@%s\n' "$OLD" + printf 'other/x\t%s\tactions/checkout@%s\n' "$OLD" "$CO"; } > "$d/scripts/lib/lock-closure-receipts.tsv" +} +run() { RC=0; bash "$RE" "$@" > "$TMP/out" 2> "$TMP/err" || RC=$?; } +digest() { cat "$1/scripts/lib/pinned-reusable-sources.json" "$1/scripts/lib/lock-closure-receipts.tsv" | sha256sum; } + +# --- the move ---------------------------------------------------------------- +F="$TMP/f"; fixture "$F" +run acme/lib .github/workflows/r.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +check "an unchanged file at a moved caller pin is re-attested -> exit 0" 0 "$RC" +check "the manifest record's commit moves" "$NEW" "$(jq -r '.sources[0].commit' "$F/scripts/lib/pinned-reusable-sources.json")" +check "its source_url moves with it" "https://github.com/acme/lib/blob/$NEW/.github/workflows/r.yml" "$(jq -r '.sources[0].source_url' "$F/scripts/lib/pinned-reusable-sources.json")" +check "blob, sha256 and uses are untouched" "$(fixture "$TMP/g"; jq -c '.sources[0] | [.blob, .content_sha256, .uses]' "$TMP/g/scripts/lib/pinned-reusable-sources.json")" \ + "$(jq -c '.sources[0] | [.blob, .content_sha256, .uses]' "$F/scripts/lib/pinned-reusable-sources.json")" +check "the receipt header names the new commit" 1 "$(grep -cxF "# Reviewed source: acme/lib/.github/workflows/r.yml@$NEW" "$F/scripts/lib/lock-closure-receipts.tsv")" +check "the receipt rows are re-emitted at the new commit" "acme/lib $NEW actions/checkout@$CO" "$(grep -P "^acme/lib\t" "$F/scripts/lib/lock-closure-receipts.tsv")" +check "another source's block (same old sha, other repo) is untouched" "other/x $OLD actions/checkout@$CO" "$(grep -P "^other/x\t" "$F/scripts/lib/lock-closure-receipts.tsv")" +check "the post-move audit is reported NOT RUN when the repo lacks it, never passed" 1 "$(grep -c 'NOT RUN' "$TMP/out")" +d1="$(digest "$F")"; run acme/lib .github/workflows/r.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +check "idempotent: a second run says already attested, changes nothing" "0 1 $d1" "$RC $(grep -c 'already attested' "$TMP/out") $(digest "$F")" + +# --- refusals: exit 3 and nothing changed --------------------------------------- +refused() { # name reason-regex + check "$1: refused, exit 3" 3 "$RC" + check "$1: nothing changed" "$d0" "$(digest "$F")" + if grep -qE "$2" "$TMP/err"; then ok "$1: the reason says so"; else no "$1: reason $(head -c 200 "$TMP/err")"; fi +} +F="$TMP/f"; fixture "$F"; printf ' - uses: evil/action@%s\n' "$CO" >> "$F/src.yml"; d0="$(digest "$F")" +run acme/lib .github/workflows/r.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +refused "a changed file" "needs a human review of its uses" +if grep -q 'evil/action' "$TMP/err"; then ok "a changed file: its new uses: are listed for review"; else no "a changed file: new uses: not listed"; fi + +fixture "$F" "$OLD"; d0="$(digest "$F")" +run acme/lib .github/workflows/r.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +refused "a caller that has not moved" "does not call .* yet; move the caller pin first" + +fixture "$F"; jq --arg old "$OLD" '.sources += [.sources[0] | .path = ".github/workflows/s.yml"]' "$F/scripts/lib/pinned-reusable-sources.json" > "$TMP/m" && mv "$TMP/m" "$F/scripts/lib/pinned-reusable-sources.json"; d0="$(digest "$F")" +run acme/lib .github/workflows/r.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +refused "an old commit shared with another attested file" "also attested for: .github/workflows/s.yml" + +fixture "$F"; d0="$(digest "$F")" +run acme/lib .github/workflows/none.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +refused "a source with no attested record" "no active record" + +fixture "$F"; printf 'acme/lib\t%s\tstray@%s\n' "$OLD" "$CO" >> "$F/scripts/lib/lock-closure-receipts.tsv"; d0="$(digest "$F")" +run acme/lib .github/workflows/r.yml "$NEW" --source "$F/src.yml" --repo-root "$F" +refused "receipt rows outside their block" "not all directly under" + +for args in "acme lib x" "acme/lib ../etc/passwd $NEW" "acme/lib .github/workflows/r.yml abc"; do + read -ra argv <<< "$args" + RC=0; bash "$RE" "${argv[@]}" --repo-root "$F" >/dev/null 2>&1 || RC=$? + check "bad arguments ($args): usage exit 64" 64 "$RC" +done + +# --- replay the real #686 breakage, when github.com is reachable -------------- +if git -C "$ROOT" cat-file -e 'dec1e7ce36dfa62ea121b530108de04cc8321e69^{commit}' 2>/dev/null \ + && timeout 20 git ls-remote https://github.com/hyperpolymath/hypatia HEAD >/dev/null 2>&1; then + W="$TMP/replay"; git -C "$ROOT" worktree add -q --detach "$W" dec1e7ce36dfa62ea121b530108de04cc8321e69 2>/dev/null + cp "$RE" "$W/scripts/reattest-pinned-source.sh" + RC=0; bash "$W/scripts/reattest-pinned-source.sh" hyperpolymath/hypatia .github/workflows/credential-read-check.yml \ + 29a9e4a881b6a3cc4cd27fcbc0886ffbadcbc5fe --repo-root "$W" > "$TMP/out" 2> "$TMP/err" || RC=$? + check "replay #686 on main@dec1e7c: re-attested, and the pinned-source audit is clean" "0 1" \ + "$RC $(grep -c 'findings=0' "$TMP/out")" + git -C "$ROOT" worktree remove --force "$W" +else + echo "NOT RUN: replay of #686 (needs github.com and commit dec1e7c in this clone)"; notrun=$((notrun+1)) +fi + +echo "reattest-pinned-source-test: $pass passed, $fail failed, $notrun not run" +[[ "$fail" -eq 0 ]]