From fadb168212980d568c81c610d61efd56630aa851 Mon Sep 17 00:00:00 2001 From: Stefan Kofler Date: Fri, 28 Aug 2026 14:16:17 +0200 Subject: [PATCH] Scan the image a Dockerfile produces, not the base image in its FROM line MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A container candidate takes its scan target from the Dockerfile FROM line, so what gets scanned is the base image. Everything the Dockerfile does after that is invisible in both directions: a package the final stage deletes is still reported against the product, and anything the final stage installs is missing from the inventory. A final stage that removes the base image's bundled npm still produces CVEs for that npm on an immediate track, while an `apk add` in the same stage appears nowhere. The scope template made this worse than a silent gap. It instructed every product to exclude its own deployed image because "its components are covered by dockerfiles-image-final, which builds it" — a coverage claim that does not hold while the candidate scans only its base. Corrected here, and the include entry now says what it scans. The same claim sat in discover.sh's note for release-versioned images and is corrected there too: that candidate covers the contents only where it declares built_image. An include entry may now carry `built_image`, with ${version} substituted from the run. Declared rather than derived: the reference lives in a compose file and a release workflow, and the naming between them is per-product convention. Deriving it would be a guess, and a wrong guess scans someone else's image and files it as ours. One string is not enough. A product may push -staging from its staging release workflow and from its production one, so a single reference can only ever be right for one of the two — declared as a string, the other tier's run pulls nothing and records gaps. built_image may therefore be a map of tier to reference, keyed on the tier the pipeline already resolves. A tier the map does not name is a gap, for the same reason an unknown version is. Three outcomes, and none of them is quietly keeping the base image. A declaration that cannot be applied stops the run, because ignoring it would leave the candidate on its base image while the scope file states otherwise. A version the run does not know, or a tier the declaration does not name, makes the candidate a reported gap — now carrying the candidate's own note rather than the fixed line about templated deploy references, since the two are unscannable for different reasons. Only a fully resolved reference redirects, and the FROM image stays on the record as base_image. Without the declaration nothing changes. Registry credentials already exist as registry-username / registry-password on the action. Comments in the files touched here no longer name products or cite individual reports; the same phrasing elsewhere in the tree is left for its own change. 234 passed offline, 240 with TEST_NETWORK=1, 0 failures either way. Co-Authored-By: Claude Opus 5 (1M context) --- soup-discovery/IMPLEMENTATION.md | 1 + .../examples/example.soup-scope.yml | 24 ++- soup-discovery/scripts/discover.sh | 6 +- soup-discovery/scripts/resolve-scope.sh | 82 +++++++++- soup-discovery/scripts/run-pipeline.sh | 13 +- soup-discovery/tests/run-tests.sh | 143 ++++++++++++++++-- 6 files changed, 243 insertions(+), 26 deletions(-) diff --git a/soup-discovery/IMPLEMENTATION.md b/soup-discovery/IMPLEMENTATION.md index 51da64b..d7cccca 100644 --- a/soup-discovery/IMPLEMENTATION.md +++ b/soup-discovery/IMPLEMENTATION.md @@ -111,6 +111,7 @@ File| Holds| Reviewer ---|---|--- .soup-policy.yml| The parameters of WI-006-09-02, Parameters defined per project. Only what differs from the defaults.| QM, for the service-level and scope parameters .soup-scope.yml| `include:` and `exclude:`, each entry an `id` and a `reason`. The run fails while a discovered candidate appears in neither.| Development lead +.soup-scope.yml, `built_image` on an include entry| The image a Dockerfile candidate actually produces, `${version}` substituted from the run. Without it the candidate is scanned from its FROM line, which is the *base* image: anything the Dockerfile adds afterwards is missing from the inventory and anything it deletes is still reported. Declared rather than derived — the reference lives in a compose file and a release workflow, and the naming between them is per-product. May be a single reference or a map of tier (`staging`, `candidate`, `branch`) to reference, for a product that pushes `-staging` from one release workflow and `` from another. A declaration that cannot be applied stops the run; a version the run does not know, or a tier the map does not name, makes the candidate a reported gap, never a silent fallback to the base image.| SOUP approver .soup-decisions.yml| Risk acceptances, revised remediation dates and `vendor_requests` with their follow-up dates.| SOUP approver ## Configuration keys diff --git a/soup-discovery/examples/example.soup-scope.yml b/soup-discovery/examples/example.soup-scope.yml index 1d749eb..64bea9c 100644 --- a/soup-discovery/examples/example.soup-scope.yml +++ b/soup-discovery/examples/example.soup-scope.yml @@ -29,9 +29,23 @@ include: - id: values-image-cache reason: Third-party image pinned in the deployment values and shipped as part of the stack. - # The final stage of a Dockerfile we build — this is what covers the components of our own image. + # The final stage of a Dockerfile we build. Discovery finds this candidate from the FROM line, so + # without `built_image` below the scan target is the *base image*, and everything the Dockerfile + # does afterwards is invisible: an `apk add` in the final stage is missing from the inventory, and + # a package the final stage deletes is still reported against it. + # + # `built_image` points the scan at the image this Dockerfile produces instead. `${version}` is the + # release version of the run. Declare it wherever the built image is pushed somewhere the run can + # pull it from; the action takes registry-username / registry-password for a private one. + # + # Where the reference differs per tier — a staging release pushing -staging and a + # production one pushing is the common case — key it by tier instead of writing one + # string. A tier the map does not name is reported as a gap, not scanned from its base image. - id: dockerfiles-image-final - reason: Final stage of the service Dockerfile — the base of the image we ship. + reason: Final stage of the service Dockerfile — the image we ship. + built_image: + staging: ourregistry/our-service:${version}-staging + candidate: ourregistry/our-service:${version} exclude: # Same image as an included candidate, in a different artefact. Legitimate, and the reason is what @@ -44,8 +58,10 @@ exclude: # Dockerfile candidate that builds it. - id: deployed-service--version- reason: >- - Our own image at the release version. The tag is substituted at deploy time, so it cannot be - resolved here; its components are covered by dockerfiles-image-final, which builds it. + Our own image at the release version, referenced from the deployment manifest where the tag is + substituted at deploy time and cannot be resolved here. The same image is scanned as + dockerfiles-image-final, which declares a built_image — without that declaration this exclusion + would leave our own image uncovered, because the Dockerfile candidate scans only its base. # Infrastructure, not shipped software. State which it is — "infrastructure" alone is not a decision. - id: terraform-root diff --git a/soup-discovery/scripts/discover.sh b/soup-discovery/scripts/discover.sh index 7fdb70f..a9e8321 100755 --- a/soup-discovery/scripts/discover.sh +++ b/soup-discovery/scripts/discover.sh @@ -119,7 +119,7 @@ while IFS= read -r f; do app_root="${f%%/android/*}" [[ "$app_root" == "$f" ]] && app_root=$(dirname "$dir") [[ -z "$app_root" || "$app_root" == "." ]] && app_root="app" - # Gradle dependency locking wins over the built artifact. Measured: syft reads ZERO + # Gradle dependency locking wins over the built artifact. syft reads ZERO # components out of an AAB — dex bytecode carries no package metadata — so scanning the # bundle would close the gap by claiming an empty inventory, which is worse than the gap. # The lockfile is the resolved android closure. Enabling it is a build-config decision, @@ -167,7 +167,7 @@ done <<<"$(grep -E '(^|/)pom\.xml$' <<<"$FILES" || true)" # one. Before this, the iOS half of a mobile product was in no inventory and was not reported # as missing either: the Android build is a candidate and can be recorded as a gap, while iOS # was absent from the candidate list, so a document could say complete with a whole platform -# never looked at. On one real product that was 39 pods. +# never looked at. # # Keyed on the app root rather than the file's directory, the same as the Android candidate: # Package.resolved sits several levels down inside Runner.xcworkspace, and an id built from @@ -436,7 +436,7 @@ while IFS= read -r ref; do # is the difference between a scope rule that can say what covers this and one that # can only say "some templated thing in this file". resolvable=false - note="OUR OWN IMAGE, RELEASE-VERSIONED — repository resolved from the chart values as ${img%%:*}, but the tag is the chart appVersion / --set version, so the concrete version comes from the deploy record. Its contents are covered by the Dockerfile candidate that builds it." + note="OUR OWN IMAGE, RELEASE-VERSIONED — repository resolved from the chart values as ${img%%:*}, but the tag is the chart appVersion / --set version, so the concrete version comes from the deploy record. The Dockerfile candidate that builds it covers its contents only where that candidate declares built_image; without one it scans the base image from its FROM line." elif [[ "$img" == *'@unresolved'* || "$img" == *'{{'* || "$img" == *'<'*'>'* || "$img" == *'${'* || "$img" =~ ^[A-Z][A-Z0-9_]*$ ]]; then resolvable=false note="TEMPLATED REFERENCE — the concrete version is substituted at deploy time and is not knowable from the repo. Its components cannot be enumerated here; resolving it needs the deploy record." diff --git a/soup-discovery/scripts/resolve-scope.sh b/soup-discovery/scripts/resolve-scope.sh index fe3048a..4694ae1 100755 --- a/soup-discovery/scripts/resolve-scope.sh +++ b/soup-discovery/scripts/resolve-scope.sh @@ -9,13 +9,28 @@ # silently omitted from it. Both are invisible failures, and the second is the one that # matters for CVE scope: WI-006-03 wants "a new SOUP appeared" to be a review event. # +# An include entry may also carry `built_image:`, and that is the one thing here that changes +# *what* is scanned rather than *whether* it is scanned. A Dockerfile candidate is discovered +# from its FROM line, so the scan target is the base image — everything the Dockerfile does +# afterwards (apk add, COPY, rm -rf) is invisible: a package the final stage deletes is still +# reported against the product, and one it installs is missing from the inventory. Declaring the +# image the Dockerfile produces points the scan at the artefact that actually ships. +# +# Declared rather than derived. The reference lives in a compose file and a release workflow, +# and the naming between them is per-product convention — deriving it would be a guess, and a +# wrong guess here scans somebody else's image and calls it ours. +# # Usage: resolve-scope.sh [.soup-scope.yml] +# SOUP_VERSION= substituted into ${version} in a built_image reference +# SOUP_TIER= selects the entry when built_image is keyed by tier set -uo pipefail CANDIDATES="${1:-candidates.json}" SCOPE_FILE="${2:-.soup-scope.yml}" OUTPUT="${SCOPE_OUTPUT:-scan-plan.json}" +VERSION="${SOUP_VERSION:-}" +TIER="${SOUP_TIER:-}" for tool in jq yq; do command -v "$tool" >/dev/null 2>&1 || { echo "::error::$tool required" >&2; exit 1; } @@ -45,6 +60,8 @@ SCOPE_JSON=$(yq -o=json '.' "$SCOPE_FILE") PLAN=$(jq -n \ --argjson cand "$(jq -c '.candidates' "$CANDIDATES")" \ --argjson scope "$SCOPE_JSON" \ + --arg version "$VERSION" \ + --arg tier "$TIER" \ ' def rules(k): ($scope[k] // []); def matches($c; $r): @@ -63,6 +80,51 @@ PLAN=$(jq -n \ | map(select((.path // null) as $p | $p != null and ($c.markers | any(startswith($p))))) | first); + # Redirect an included container candidate from its base image to the image the Dockerfile + # produces. Three outcomes, and none of them is "quietly keep scanning the base image": + # a misdeclaration is an error that stops the run, an unknowable version is a reported gap, + # and only a fully resolved reference actually redirects. + def apply_built($c; $inc): + ($inc.built_image // null) as $decl + | (if ($decl | type) == "object" then ($decl[$tier] // null) else $decl end) as $bi + | if $decl == null then $c + elif ($c.scan_source | startswith("registry:") | not) then + $c + {built_image_error: + "built_image is only meaningful for a container candidate; this one scans \($c.scan_source)"} + # Keyed by tier, because a product does not push the same reference from every workflow: + # one repository tags -staging from its staging release and from its + # production one. A tier the declaration does not name is a gap for the same reason an + # unknown version is: the alternative is scanning the base image and saying nothing. + elif ($decl | type) == "object" and $bi == null then + $c + {resolvable: false, + note: ("built_image names no image for the " + + (if $tier == "" then "unknown" else $tier end) + + " tier, so the shipped image cannot be addressed — reported as a gap " + + "rather than scanning the base image instead")} + elif ($bi | type) != "string" then + $c + {built_image_error: + "built_image must be an image reference or a map of tier to image reference"} + elif ($version == "" and ($bi | test("\\$\\{version\\}"))) then + # No version, no concrete image. Falling back to the base image would restore exactly + # the wrong answer this exists to remove, and it would do it silently. + $c + {resolvable: false, + note: ("built_image declared but this run knows no version, so the shipped image " + + "cannot be addressed — reported as a gap rather than scanning the base image instead")} + else + ($bi | gsub("\\$\\{version\\}"; $version)) as $resolved + | if ($resolved | test("\\$\\{")) then + $c + {built_image_error: + "built_image still carries an unsubstituted placeholder after resolution: \($resolved)"} + else + $c + {scan_source: ("registry:" + $resolved), + base_image: $c.scan_source, + built_image_declared: true, + note: ("scanning the image this Dockerfile produces (" + $resolved + + ") rather than the base image from its FROM line" + + (if ($c.note // "") == "" then "" else "; " + $c.note end))} + end + end; + def classify($c): by_id($c; "include") as $inc_id | by_id($c; "exclude") as $exc_id @@ -75,7 +137,7 @@ PLAN=$(jq -n \ | if $inc != null and $exc != null then $c + {decision:"conflict", reason:"matched both include and exclude at the same specificity"} elif $inc != null then - $c + {decision:"include", reason:($inc.reason // "")} + apply_built($c + {decision:"include", reason:($inc.reason // "")}; $inc) elif $exc != null then $c + {decision:"exclude", reason:($exc.reason // "")} else @@ -94,6 +156,10 @@ PLAN=$(jq -n \ }, unclassified: ($all | map(select(.decision=="unclassified")) | map({id, ecosystem, markers, ships})), conflicts: ($all | map(select(.decision=="conflict")) | map({id, markers})), + built_image_errors: ($all | map(select(.built_image_error != null)) + | map({id, error: .built_image_error})), + redirected: ($all | map(select(.built_image_declared == true)) + | map({id, base_image, scan_source})), # Includes as well as excludes: WI-006-09: Introduce says every entry carries a reason, and an # unexplained include is how test tooling ends up in the shipped inventory unnoticed. missing_reason: ($all | map(select((.decision=="exclude" or .decision=="include") @@ -120,6 +186,15 @@ if [[ "$(jq -r '.counts.conflict' "$OUTPUT")" != "0" ]]; then STATUS=1 fi +# A built_image that cannot be applied is refused rather than ignored. Ignoring it would leave +# the candidate scanning its base image while the scope file states otherwise, which is worse +# than never having declared it: the document would carry a claim nobody checked. +if [[ "$(jq -r '.built_image_errors | length' "$OUTPUT")" != "0" ]]; then + echo "::error::built_image declarations that cannot be applied" >&2 + jq -r '.built_image_errors[] | "::error:: \(.id): \(.error)"' "$OUTPUT" >&2 + STATUS=1 +fi + # An exclusion without a reason is the failure mode this whole file exists to prevent: # it is indistinguishable from an oversight six months later. if [[ "$(jq -r '.missing_reason | length' "$OUTPUT")" != "0" ]]; then @@ -128,6 +203,11 @@ if [[ "$(jq -r '.missing_reason | length' "$OUTPUT")" != "0" ]]; then STATUS=1 fi +# Named in the log, because a redirected scan target changes what the inventory means and +# should not be something only a diff of the BOM reveals. +jq -r '.redirected[]? | " scanning \(.id) as \(.scan_source | sub("^registry:";"")) (declared), not \(.base_image | sub("^registry:";""))"' \ + "$OUTPUT" >&2 + if [[ $STATUS -eq 0 ]]; then echo "wrote $OUTPUT — every candidate has a recorded decision" >&2 fi diff --git a/soup-discovery/scripts/run-pipeline.sh b/soup-discovery/scripts/run-pipeline.sh index 19d73ec..6708672 100755 --- a/soup-discovery/scripts/run-pipeline.sh +++ b/soup-discovery/scripts/run-pipeline.sh @@ -63,14 +63,16 @@ DISCOVER_OUTPUT="$CAND" bash "$HERE/discover.sh" "$REPO" || die "discovery faile # --- 2. scope --------------------------------------------------------------- SCOPE_FILE="${SOUP_SCOPE_FILE:-$REPO/.soup-scope.yml}" -SCOPE_OUTPUT="$PLAN" bash "$HERE/resolve-scope.sh" "$CAND" "$SCOPE_FILE" \ +# SOUP_VERSION feeds ${version} in a built_image declaration — the tag is the only part of a +# shipped image reference this pipeline can know, and it already has it. +SCOPE_OUTPUT="$PLAN" SOUP_VERSION="$VERSION" SOUP_TIER="${SBOM_TIER:-}" bash "$HERE/resolve-scope.sh" "$CAND" "$SCOPE_FILE" \ || die "scope resolution failed — every candidate needs a recorded decision before a BOM can be trusted" # --- 3. scan each in-scope target ------------------------------------------ BOMS=() GAPS=() -while IFS=$'\t' read -r id source resolvable ecosystem markers; do +while IFS=$'\t' read -r id source resolvable ecosystem markers note; do [[ -z "$id" ]] && continue raw="$OUT_DIR/bom/$id.raw.json" final="$OUT_DIR/bom/$id.cdx.json" @@ -78,7 +80,10 @@ while IFS=$'\t' read -r id source resolvable ecosystem markers; do arg="${source#*:}" if [[ "$resolvable" != "true" ]]; then - log " gap $id — reference is templated, concrete version not knowable from the repo" + # The candidate's own note: a templated deploy reference and a built_image on a run + # that knows no version are both unscannable for different reasons, and a gap naming + # the wrong one sends whoever reads it to the wrong file. + log " gap $id — ${note:-reference is templated, concrete version not knowable from the repo}" GAPS+=("$id"); continue fi @@ -205,7 +210,7 @@ while IFS=$'\t' read -r id source resolvable ecosystem markers; do || log "::warning::$id: dependency graph could not be derived" rm -f "$raw" "$native" BOMS+=("$final") -done < <(jq -r '.scan[] | "\(.id)\t\(.scan_source)\t\(.resolvable)\t\(.ecosystem)\t\(.markers | join(","))"' "$PLAN") +done < <(jq -r '.scan[] | "\(.id)\t\(.scan_source)\t\(.resolvable)\t\(.ecosystem)\t\(.markers | join(","))\t\(.note // "" | gsub("\t"; " "))"' "$PLAN") [[ ${#BOMS[@]} -gt 0 ]] || die "no target produced a BOM" diff --git a/soup-discovery/tests/run-tests.sh b/soup-discovery/tests/run-tests.sh index 9858ad0..c431e96 100755 --- a/soup-discovery/tests/run-tests.sh +++ b/soup-discovery/tests/run-tests.sh @@ -165,6 +165,123 @@ test_scope_same_specificity_is_a_conflict() { [[ "$(jq -r '.counts.conflict' "$TMP/p4.json")" == "1" ]] || { echo "conflict not detected"; return 1; } } +# A Dockerfile candidate is discovered from its FROM line, so without built_image the scan +# target is the base image and everything the Dockerfile does afterwards is invisible: a package +# the final stage deletes is still reported, and one it installs is missing. +mk_image_candidates() { + mk_candidates '[{"id":"img-final","ecosystem":"container","scan_source":"registry:node:22-alpine", + "markers":["Dockerfile:31"],"ships":true,"resolvable":true,"note":"NOT digest-pinned"}, + {"id":"web","ecosystem":"npm","scan_source":"file:web/yarn.lock", + "markers":["web/yarn.lock"],"ships":true,"resolvable":true,"note":""}]' > "$TMP/bi-c.json" +} + +test_scope_built_image_redirects_the_scan_target() { + mk_image_candidates + cat > "$TMP/bi1.yml" <<'EOF' +include: + - id: img-final + reason: Final stage of the service Dockerfile — the image we ship. + built_image: reg/app:${version} + - id: web + reason: The lockfile. +exclude: [] +EOF + SCOPE_OUTPUT="$TMP/bi1.json" SOUP_VERSION=v1.2.3 \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi1.yml" >/dev/null 2>&1 || return 1 + assert "$(jq -r '.scan[]|select(.id=="img-final")|.scan_source' "$TMP/bi1.json")" \ + "registry:reg/app:v1.2.3" || return 1 + # the FROM image is kept on the record rather than discarded — it is what was shipped from + assert "$(jq -r '.scan[]|select(.id=="img-final")|.base_image' "$TMP/bi1.json")" \ + "registry:node:22-alpine" || return 1 + assert "$(jq -r '.redirected|length' "$TMP/bi1.json")" "1" +} + +# The whole point is that a candidate without the declaration behaves exactly as before. +test_scope_without_built_image_keeps_the_from_line() { + mk_image_candidates + printf 'include:\n - id: img-final\n reason: r\n - id: web\n reason: r\nexclude: []\n' > "$TMP/bi2.yml" + SCOPE_OUTPUT="$TMP/bi2.json" SOUP_VERSION=v1 \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi2.yml" >/dev/null 2>&1 || return 1 + assert "$(jq -r '.scan[]|select(.id=="img-final")|.scan_source' "$TMP/bi2.json")" "registry:node:22-alpine" || return 1 + assert "$(jq -r '.redirected|length' "$TMP/bi2.json")" "0" +} + +# No version, no concrete image. Falling back to the base image would silently restore the +# exact wrong answer this exists to remove, so the candidate becomes a reported gap instead. +test_scope_built_image_without_a_version_is_a_gap_not_a_fallback() { + mk_image_candidates + printf 'include:\n - id: img-final\n reason: r\n built_image: reg/app:${version}\n - id: web\n reason: r\nexclude: []\n' > "$TMP/bi3.yml" + SCOPE_OUTPUT="$TMP/bi3.json" SOUP_VERSION="" \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi3.yml" >/dev/null 2>&1 || return 1 + assert "$(jq -r '.scan[]|select(.id=="img-final")|.resolvable' "$TMP/bi3.json")" "false" || return 1 + contains "$(jq -r '.scan[]|select(.id=="img-final")|.note' "$TMP/bi3.json")" "rather than scanning the base image" +} + +# One reference does not fit every tier: a product may push -staging from its staging +# release workflow and from its production one, so a single string can only ever be +# right for one of the two. +test_scope_built_image_selects_the_entry_for_the_tier() { + mk_image_candidates + cat > "$TMP/bi6.yml" <<'EOF' +include: + - id: img-final + reason: Final stage of the service Dockerfile — the image we ship. + built_image: + staging: reg/app:${version}-staging + candidate: reg/app:${version} + - id: web + reason: The lockfile. +exclude: [] +EOF + SCOPE_OUTPUT="$TMP/bi6a.json" SOUP_VERSION=v1.2.3 SOUP_TIER=staging bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi6.yml" >/dev/null 2>&1 || return 1 + assert "$(jq -r '.scan[]|select(.id=="img-final")|.scan_source' "$TMP/bi6a.json")" \ + "registry:reg/app:v1.2.3-staging" || return 1 + SCOPE_OUTPUT="$TMP/bi6b.json" SOUP_VERSION=v1.0.0 SOUP_TIER=candidate \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi6.yml" >/dev/null 2>&1 || return 1 + assert "$(jq -r '.scan[]|select(.id=="img-final")|.scan_source' "$TMP/bi6b.json")" \ + "registry:reg/app:v1.0.0" +} + +# A tier the map does not name is a gap for the same reason an unknown version is: the only +# alternative is scanning the base image and saying nothing about it. +test_scope_built_image_tier_not_named_is_a_gap() { + mk_image_candidates + cat > "$TMP/bi7.yml" <<'EOF' +include: + - id: img-final + reason: r + built_image: + candidate: reg/app:${version} + - id: web + reason: r +exclude: [] +EOF + SCOPE_OUTPUT="$TMP/bi7.json" SOUP_VERSION=v1 SOUP_TIER=branch \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi7.yml" >/dev/null 2>&1 || return 1 + assert "$(jq -r '.scan[]|select(.id=="img-final")|.resolvable' "$TMP/bi7.json")" "false" || return 1 + contains "$(jq -r '.scan[]|select(.id=="img-final")|.note' "$TMP/bi7.json")" "no image for the branch tier" +} + +# A declaration that cannot be applied stops the run. Ignoring it would leave the candidate on +# its base image while the scope file states otherwise — a claim in the document nobody checked. +test_scope_built_image_on_a_non_container_is_refused() { + mk_image_candidates + printf 'include:\n - id: img-final\n reason: r\n - id: web\n reason: r\n built_image: reg/app:1\nexclude: []\n' > "$TMP/bi4.yml" + out=$(SCOPE_OUTPUT="$TMP/bi4.json" SOUP_VERSION=v1 \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi4.yml" 2>&1) + assert "$?" "1" || return 1 + contains "$out" "only meaningful for a container candidate" +} + +test_scope_built_image_with_an_unknown_placeholder_is_refused() { + mk_image_candidates + printf 'include:\n - id: img-final\n reason: r\n built_image: reg/app-${service}:${version}\n - id: web\n reason: r\nexclude: []\n' > "$TMP/bi5.yml" + out=$(SCOPE_OUTPUT="$TMP/bi5.json" SOUP_VERSION=v1 \ + bash "$S/resolve-scope.sh" "$TMP/bi-c.json" "$TMP/bi5.yml" 2>&1) + assert "$?" "1" || return 1 + contains "$out" "unsubstituted placeholder" +} + # ---------------------------------------------------------------- discovery mkrepo() { rm -rf "$TMP/repo"; mkdir -p "$TMP/repo"; ( cd "$TMP/repo" && git init -q . ); } discover() { ( cd "$TMP/repo" && git add -A >/dev/null 2>&1; DISCOVER_OUTPUT="$TMP/cand.json" bash "$S/discover.sh" . >/dev/null 2>&1 ); } @@ -264,7 +381,7 @@ test_discover_dockerfile_id_survives_a_line_shift() { # Two Dockerfiles in one directory both have a builder and both have a final stage. Keyed on # the directory they collapse onto one candidate and one of the two shipped images is dropped -# without a word. Seen on a real project, where it hid a whole base image. +# without a word. test_discover_two_dockerfiles_in_one_directory_stay_apart() { mkrepo mkdir -p "$TMP/repo/web/dockerfiles" @@ -703,9 +820,9 @@ test_enrichment_epss_model_and_date_are_on_the_finding() { # ---------------------------------------------------------------- gradle lockfile filtering GLF() { bash "$S/filter-gradle-lockfile.sh" "$@"; } -# lockAllConfigurations() tags every configuration onto one line. Measured on a real -# lockfile: 56 of 143 components were tagged only with test/build-tooling configurations and -# never ship, yet got real classified findings and remediation burden. +# lockAllConfigurations() tags every configuration onto one line, so a component tagged only +# with test or build-tooling configurations reads as shipped and collects classified findings +# and remediation burden it should never have. test_gradle_lockfile_drops_test_only_entries() { cat > "$TMP/glf-in.lockfile" <<'EOF' # This is a Gradle generated file for dependency locking. @@ -807,8 +924,8 @@ test_classify_kev_gets_its_own_track_regardless_of_cvss() { } # The point of separating the two: "actively exploited" is an observation, "CVSS 10.0" is a -# score, and they no longer share a clock. Measured on kontina-backend, 0 of 23 Critical -# findings were in KEV — the 72h clock there was justified by a risk none of them carried. +# score, and they no longer share a clock. A Critical that is not in KEV would otherwise inherit +# the 72h clock, justified by a risk it does not carry. test_classify_kev_and_critical_have_different_clocks() { mkpolicy mkvuln CVE-1 "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H" true null # KEV, also 10.0 @@ -1610,8 +1727,7 @@ test_units_no_published_fix_is_not_an_upgrade() { # Regression: the action name dropped the npm scope, so @nestjs/core, @sigstore/core and # @strapi/core all produced actions titled "upgrade core" — and since the unit key is built -# from that name, two of them inside one artifact merged into a single action. Seen on -# curacoach v1.0.0-qa10, which listed three separate "upgrade core" rows. +# from that name, two of them inside one artifact merged into a single action. test_units_scoped_packages_do_not_collide() { jq -n '{bomFormat:"CycloneDX",specVersion:"1.6", metadata:{component:{name:"p","bom-ref":"p",type:"application"}}, @@ -2324,8 +2440,8 @@ test_scope_npm_an_undecidable_range_stays_direct() { # A pod written `- name (from `...`)` is not an iOS choice. On a Flutter app it is the iOS half # of a Dart package that pubspec.lock already records as direct, or the engine itself. Counting # it again would demand a second SOUP record for one choice and assess its currency twice, once -# under pkg:pub and once under pkg:cocoapods. On two real products that is 33 of 34 and 23 of 24 -# entries, so both come out with no direct pods at all. +# under pkg:pub and once under pkg:cocoapods. Nearly every entry in a Podfile.lock is of that +# shape, so the usual answer is no direct pods at all. test_scope_cocoapods_ignores_locally_sourced_pods() { mkdir -p "$TMP/sc-pods/app/ios" cat > "$TMP/sc-pods/app/ios/Podfile.lock" <<'EOF' @@ -3119,9 +3235,8 @@ EOF } # A fixed version that exists only as a prerelease is not a fix a released product can -# apply. Reported as "available" it produced the action "upgrade multer to 3.0.0-alpha.2" — -# on an expedited track, with a deadline nobody could meet by doing the thing it named. -# Seen on curacoach v1.0.0-qa10 for multer and @babel/core. +# apply. Pooled with the stable fixes it also sorted last, so the Fixed-in column named the +# prerelease while an adoptable version stood beside it. fake_osv() { # — serves querybatch and the per-advisory fetch from a stub curl. mkdir -p "$TMP/osvbin" @@ -3306,7 +3421,7 @@ test_render_vdr_report() { } # String order puts 8.0.5 above 8.0.16, and Fixed-in printed whichever sorted last — a -# version below the one carrying the fix. vite in the curacoach qa10 report is the real case. +# version below the one carrying the fix. test_render_fix_version_order_is_numeric() { need_reportlab || return 77 S="$S" python3 - <<'PYEOF'