Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1083,6 +1083,16 @@ jobs:
with:
go-version: ${{ env.GO_VERSION }}

- name: Assert the gosec version is pinned only in .gosec-version
# Issue #347: the version used to be hard-coded in four files, so a
# bump that missed one left local scans and this job's SARIF upload on
# different gosec rule sets. Fast (git grep only). always(): a finding
# here must not skip the scanners below; the job still fails.
if: always() && steps.checkout.outcome == 'success'
run: |
bash scripts/check-gosec-pin.sh
bash scripts/test-check-gosec-pin.sh

- name: Run govulncheck CVE scanner (all modules)
# always(): each scanner step below is independent evidence for the
# Security tab. Without this, one scanner failing (e.g. a live npm
Expand Down Expand Up @@ -1207,12 +1217,12 @@ jobs:
rm -f "$merged"
fi

# scripts/run-gosec.sh owns the pinned gosec version, toolchain,
# module list and rule set, shared with the pre-commit hook and
# `make security-scan-go` (issue #340). It installs gosec with the
# job's existing setup-go: the securego/gosec Docker action bundles
# its own Go toolchain, which cannot satisfy the module's go
# directive.
# scripts/run-gosec.sh owns the toolchain, module list and rule set,
# shared with the pre-commit hook and `make security-scan-go` (issue
# #340), and installs the gosec version in .gosec-version (issue
# #347). It installs gosec with the job's existing setup-go: the
# securego/gosec Docker action bundles its own Go toolchain, which
# cannot satisfy the module's go directive.
#
# gosec exits non-zero both for real findings and for a processing
# error. The script still scans every module after a failure, and
Expand Down
42 changes: 21 additions & 21 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -121,27 +121,26 @@ jobs:
echo "tflint --init failed after 6 attempts — GitHub release API likely unavailable" >&2
exit 1

# Cache the installed tool binaries (gosec, gocyclo). Keyed on the
# pinned version strings so a tool-version bump still triggers a
# fresh install. Both binaries land in ~/go/bin which setup-go@v6
# already adds to PATH. Restored BEFORE the install steps so the
# `if: cache-hit != 'true'` guards below can short-circuit them on
# cache-hit runs (the `go install` invocations cost ~3-5s each
# even when the module cache is warm; skipping them on cache-hit
# is worth the extra `if`).
- name: Cache Go-installed tools (gosec, gocyclo)
# Cache the installed tool binaries (gocyclo). Keyed on the pinned
# version string so a tool-version bump still triggers a fresh
# install. The binary lands in ~/go/bin which setup-go@v6 already
# adds to PATH. Restored BEFORE the install step so the
# `if: cache-hit != 'true'` guard below can short-circuit it on
# cache-hit runs (the `go install` invocation costs ~3-5s even
# when the module cache is warm; skipping it on cache-hit is worth
# the extra `if`).
#
# gosec is deliberately NOT installed here (issue #347): the `gosec`
# hook is SKIPped in this job (see the pre-commit step below), and
# the hook (via scripts/run-gosec.sh) installs its own copy under
# ~/.cache/gosec anyway, so a ~/go/bin/gosec was never run. ci.yml's
# Security Scanning job is the gosec gate.
- name: Cache Go-installed tools (gocyclo)
id: cache-go-tools
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ~/go/bin
key: go-tools-${{ runner.os }}-gosec-v2.28.0-gocyclo-v0.6.0

- name: Install gosec
if: steps.cache-go-tools.outputs.cache-hit != 'true'
# Pinned to the same version ci.yml's `securego/gosec` Action uses,
# so an upstream gosec release with rule changes can't silently
# downgrade the gate between the two workflows.
run: go install github.com/securego/gosec/v2/cmd/gosec@v2.28.0
key: go-tools-${{ runner.os }}-gocyclo-v0.6.0

- name: Install gocyclo
if: steps.cache-go-tools.outputs.cache-hit != 'true'
Expand Down Expand Up @@ -286,10 +285,11 @@ jobs:
# modules at once, and gosec's whole-repo analysis exhausts the
# runner's memory — the job dies with "The runner has received a
# shutdown signal" at this step on every run. gosec is NOT dropped:
# the dedicated `Security Scanning` job in ci.yml runs gosec v2.28.0
# per-module (SARIF) as the authoritative gate, so this only removes
# the duplicate that OOMs CI — the same dedup rationale as the
# already-skipped `terraform_validate` (covered by Validate Terraform).
# the dedicated `Security Scanning` job in ci.yml runs gosec (the
# version in .gosec-version) per-module (SARIF) as the
# authoritative gate, so this only removes the duplicate that OOMs
# CI — the same dedup rationale as the already-skipped
# `terraform_validate` (covered by Validate Terraform).
# Local developers still get the fast per-changed-package gosec hook.
SKIP: terraform_validate,gosec
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
1 change: 1 addition & 0 deletions .gosec-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v2.28.0
5 changes: 3 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,9 @@ repos:
# their owning module (root or tests/e2e).
# Fast: never whole-repo, always per-changed-package.
#
# Version, toolchain and rule set (no excludes) live in
# scripts/run-gosec.sh, shared with `make security-scan-go` and CI.
# Toolchain and rule set (no excludes) live in scripts/run-gosec.sh,
# shared with `make security-scan-go` and CI; it installs the gosec
# version in .gosec-version.
entry: bash scripts/gosec-hook.sh
language: system
pass_filenames: true
Expand Down
14 changes: 10 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,14 @@ BUILD_TIME?=$(shell date -u '+%Y-%m-%dT%H:%M:%SZ')
GIT_SHA?=$(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)

# Dev tool versions - keep in sync with the CI pins in
# .github/workflows/ci.yml, pre-commit.yml and database-migration.yml
# .github/workflows/ci.yml and pre-commit.yml
GOLANGCI_LINT_VERSION?=v2.10.1
GOSEC_VERSION?=v2.28.0
# gosec has one source of truth, .gosec-version, also read by
# scripts/run-gosec.sh (the pre-commit hook, security-scan-go and CI).
# scripts/check-gosec-pin.sh fails CI if the version is written anywhere else
# (issue #347).
GOSEC_VERSION_FILE:=$(dir $(abspath $(lastword $(MAKEFILE_LIST)))).gosec-version
GOSEC_VERSION?=$(shell cat $(GOSEC_VERSION_FILE))
GOCYCLO_VERSION?=v0.6.0
# golang-migrate deliberately has no version variable: it is installed as a
# package of this module (see install-tools), so its version and its whole
Expand Down Expand Up @@ -159,8 +164,9 @@ complexity-report:
# Security scanning
security-scan: security-scan-go security-scan-docker security-scan-terraform

# scripts/run-gosec.sh owns the gosec pin, module list and rule set, shared
# with the pre-commit hook and the CI gosec step (issue #340).
# scripts/run-gosec.sh owns the module list and rule set and installs the
# gosec version in .gosec-version, shared with the pre-commit hook and the CI
# gosec step (issues #340, #347).
security-scan-go:
@scripts/run-gosec.sh

Expand Down
149 changes: 149 additions & 0 deletions scripts/check-gosec-pin.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# check-gosec-pin.sh
#
# Fails when the gosec version is written anywhere other than .gosec-version.
#
# .gosec-version (repo root, one line such as `v2.28.0`) is the single source of
# truth for the gosec version. scripts/run-gosec.sh (which ci.yml's Security
# Scanning job, `make security-scan-go` and the pre-commit hook all call) and
# the Makefile's install-dev-tools target read it. Before issue #347 the same
# version was hard-coded in four places (ci.yml, pre-commit.yml, the Makefile
# and gosec-hook.sh) plus comments that restated it, so a bump only had to miss
# one of them for local scans and the CI scan that uploads to GitHub Security
# to run different rule sets.
#
# Every tracked text file is scanned (git grep), not a list of known pin sites,
# so a new workflow or script that hard-codes the version is caught without
# anyone remembering to add it here. Findings:
#
# 1. .gosec-version is missing, untracked, or not exactly `vMAJOR.MINOR.PATCH`.
# 2. A version literal written next to "gosec" (any case): an install ref
# (`gosec@v2.28.0`), an assignment (`GOSEC_VERSION?=v2.28.0`,
# `GOSEC_VERSION="2.28.0"`), a cache key (`gosec-v2.28.0`), a path
# (`pre-commit-gosec/v2.28.0/`), an image tag (`securego/gosec:2.28.0`),
# a go.mod requirement, or prose (`runs gosec v2.28.0`). Prose counts too:
# a comment that restates the version goes stale on the next bump.
# 3. A gosec install or action ref that names anything other than a variable,
# such as `gosec@latest`, `gosec@master` or `securego/gosec@<sha>`.
# 4. An install site that does not mention .gosec-version, i.e. one that
# takes its version from somewhere else. An install site is any file that
# names the gosec command at a version (`securego/gosec/v2/cmd/gosec@`, as
# `go install` and `go run` do; matched per line, so a `go install \`
# continued onto the next line still counts). At least one must exist, so
# a scan that silently matched nothing cannot pass.
#
# Not scanned: CHANGELOG.md and docs/audits/, which record history and may name
# the versions that were current at the time, and this script and its self-test
# (scripts/test-check-gosec-pin.sh), which spell out the patterns.
#
# LIMITATIONS. A version stored in a variable whose name does not contain
# "gosec" (e.g. `V=2.28.0` then `gosec@$V`) is not caught by rule 2; rule 4
# still requires that file to mention .gosec-version.
#
# Usage: check-gosec-pin.sh [ROOT]
# ROOT defaults to this repository. It must be the top level of a git work
# tree; only tracked files are scanned. The self-test points it at fixtures.
#
# Exit codes: 0 clean, 1 findings, 2 usage or setup error.

set -euo pipefail

if [[ $# -gt 1 ]]; then
echo "Usage: $0 [ROOT]" >&2
exit 2
fi
case "${1:-}" in
-*) echo "Unknown flag: $1" >&2; exit 2 ;;
esac

ROOT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
if [[ ! -d "$ROOT" ]]; then
echo "ERROR: not a directory: $ROOT" >&2
exit 2
fi
ROOT="$(cd "$ROOT" && pwd -P)"
toplevel="$(git -C "$ROOT" rev-parse --show-toplevel 2>/dev/null || true)"
if [[ -z "$toplevel" || "$(cd "$toplevel" && pwd -P)" != "$ROOT" ]]; then
echo "ERROR: $ROOT is not the top level of a git work tree" >&2
exit 2
fi

VERSION_FILE=".gosec-version"
# The guard and its self-test spell out the patterns they look for.
EXCLUDES=(
":(exclude)CHANGELOG.md"
":(exclude)docs/audits/**"
":(exclude)scripts/check-gosec-pin.sh"
":(exclude)scripts/test-check-gosec-pin.sh"
)

# Rule 2: "gosec", then any non-space run (e.g. `_VERSION`, `/v2/cmd/gosec`),
# an optional word "version", separators, then a version literal.
LITERAL_RE="gosec[^[:space:]]*([[:space:]]+version)?[\"'@:=?/_[:space:]-]+v?[0-9]+\.[0-9]+\.[0-9]+"
# Rule 3: a gosec ref whose value after `@` does not start with `$`.
REF_RE="gosec[^[:space:]@]*@[\"']?[^$\"'[:space:]]"
# Rule 4: an install site.
INSTALL_RE="securego/gosec(/v[0-9]+)?/cmd/gosec@"

findings=()

# git grep exits 1 for "no match"; anything above that is a real error.
grep_tree() {
local rc=0 out
out="$(git -C "$ROOT" grep "$@" -- . "${EXCLUDES[@]}")" || rc=$?
if [[ $rc -gt 1 ]]; then
echo "ERROR: git grep failed (exit $rc) in $ROOT" >&2
exit 2
fi
# $(...) strips the final newline; put it back so `read` sees the last line.
[[ -n "$out" ]] && printf '%s\n' "$out"
return 0
}

# Rule 1.
if [[ ! -f "$ROOT/$VERSION_FILE" ]]; then
findings+=("$VERSION_FILE: missing")
elif ! git -C "$ROOT" ls-files --error-unmatch -- "$VERSION_FILE" >/dev/null 2>&1; then
findings+=("$VERSION_FILE: not tracked by git, so CI's checkout would not have it")
else
version="$(cat "$ROOT/$VERSION_FILE")"
if [[ ! "$version" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
findings+=("$VERSION_FILE: must be one line of the form vMAJOR.MINOR.PATCH, got '${version}'")
fi
fi

# Rules 2 and 3.
while IFS= read -r hit; do
[[ -n "$hit" ]] && findings+=("$hit [hard-coded gosec version]")
done < <(grep_tree -n -I -i -E "$LITERAL_RE")
while IFS= read -r hit; do
[[ -n "$hit" ]] && findings+=("$hit [gosec ref not read from $VERSION_FILE]")
done < <(grep_tree -n -I -i -E "$REF_RE")

# Rule 4.
install_sites=0
while IFS= read -r f; do
[[ -z "$f" ]] && continue
install_sites=$((install_sites + 1))
if ! git -C "$ROOT" grep -q -F "$VERSION_FILE" -- "$f"; then
findings+=("$f: installs gosec but never mentions $VERSION_FILE")
fi
done < <(grep_tree -l -I -i -E "$INSTALL_RE")
if [[ $install_sites -eq 0 ]]; then
findings+=("no file names securego/gosec/v2/cmd/gosec@...; the scan found no install site, so it cannot vouch for any")
fi

if [[ ${#findings[@]} -gt 0 ]]; then
{
echo "FAILED: the gosec version must be written only in $VERSION_FILE."
echo ""
printf ' %s\n' "${findings[@]}"
echo ""
echo "To bump gosec, edit $VERSION_FILE and nothing else. Install sites read it:"
echo " go install \"github.com/securego/gosec/v2/cmd/gosec@\$(cat $VERSION_FILE)\""
echo "See the header of scripts/check-gosec-pin.sh for what is checked."
} >&2
exit 1
fi

echo "OK: gosec ${version} is pinned only in $VERSION_FILE; ${install_sites} install site(s) read it."
11 changes: 6 additions & 5 deletions scripts/gosec-hook.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,15 @@
# files, resolved to their owning module. Fast by design: never scans the whole
# repo; each commit triggers at most one gosec invocation per affected module.
#
# Rule set, gosec version and toolchain live in scripts/run-gosec.sh, shared
# with `make security-scan-go` and CI, so this hook enforces exactly what CI
# enforces (issue #340). This script only maps staged files to
# (module, package) pairs.
# Rule set, gosec version (read from .gosec-version, issue #347) and toolchain
# live in scripts/run-gosec.sh, shared with `make security-scan-go` and CI, so
# this hook enforces exactly what CI enforces (issue #340). This script only
# maps staged files to (module, package) pairs.
#
# Called by pre-commit with pass_filenames: true and files: \.go$.
# Exits 0 when no staged .go files survive filtering (deleted / testdata).
# Exits 1 on any gosec finding; exits 2 on setup failure.
# Exits 1 on any gosec finding; exits 2 on setup failure (including a missing
# or malformed .gosec-version).

set -euo pipefail

Expand Down
16 changes: 15 additions & 1 deletion scripts/run-gosec.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@
# where <tag> is "root" for the root module and the module path with slashes
# turned into dashes otherwise (tests/e2e -> tests-e2e).
#
# The gosec version is read from .gosec-version at the repo root, the single
# source of truth that the Makefile's install-dev-tools also reads (issue
# #347). A missing or malformed file (anything but one vMAJOR.MINOR.PATCH line)
# is a setup failure, caught before Go runs or the cache is touched.
#
# --print-versions prints gosec_version=, go_version= and bin_dir= lines (the
# pinned gosec, the effective Go version and the binary cache directory) and
# exits without installing or scanning. CI keys its gosec caches on them, so
Expand All @@ -34,7 +39,6 @@

set -euo pipefail

GOSEC_VERSION="v2.28.0"
# Every module in go.work.
MODULES=(. tests/e2e)

Expand Down Expand Up @@ -81,6 +85,16 @@ if [[ ${#scan_mods[@]} -eq 0 ]]; then
for _ in "${MODULES[@]}"; do scan_pkgs+=(""); done
fi

# Validate the pin before it reaches `go install` or the cache path: an empty
# or garbage value must not install some other gosec or escape the cache dir.
version_file="${REPO_ROOT}/.gosec-version"
[[ -f "$version_file" ]] || { echo "run-gosec: $version_file not found" >&2; exit 2; }
GOSEC_VERSION="$(cat "$version_file")"
if [[ ! "$GOSEC_VERSION" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "run-gosec: $version_file must be one line like vMAJOR.MINOR.PATCH, got '${GOSEC_VERSION}'" >&2
exit 2
fi

if [[ -z "${GOTOOLCHAIN:-}" ]]; then
go_directive=$(awk '$1=="go"{print $2; exit}' "${REPO_ROOT}/go.mod")
[[ -n "$go_directive" ]] || { echo "run-gosec: no go directive in go.mod" >&2; exit 2; }
Expand Down
7 changes: 4 additions & 3 deletions scripts/security-scan.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ print_section() {
}

# 1. Go Security Scanner (gosec)
# scripts/run-gosec.sh owns the gosec pin, module list and rule set, shared
# with the pre-commit hook, `make security-scan-go` and CI (issue #340). It
# installs the pinned gosec itself, so there is no not-installed branch.
# scripts/run-gosec.sh owns the module list and rule set, shared with the
# pre-commit hook, `make security-scan-go` and CI (issue #340). It installs
# the gosec version in .gosec-version itself (issue #347), so there is no
# not-installed branch.
print_section "1. Go Security Scanner (gosec)"
if "$(dirname "$0")/run-gosec.sh" 2>&1 | tee "$REPORT_DIR/gosec-report.txt"; then
echo -e "${GREEN}✓ gosec scan completed${NC}"
Expand Down
Loading
Loading