Skip to content

K2: scan a pull request's diff and comment the band-aid #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
henleda merged 2 commits into from
Jul 30, 2026
Merged

K2: scan a pull request's diff and comment the band-aid #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5a5b369
feat(ci): K2 scan a pull request's diff and comment the band-aid
henleda Jul 30, 2026
b33a7ab
fix(ci): make the moved G2 gate work without tenant credentials, and …
henleda Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions .github/actions/vpcopilot-scan/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
name: "Virtual Patch Copilot — PR scan"
description: >-
Scan a pull request's diff for vulnerabilities and comment the F5 Distributed Cloud virtual patch
that would hold each one closed while the code fix ships. Never touches an XC tenant.
author: "virtual-patch-copilot contributors"
branding:
icon: shield
color: blue

inputs:
repo-path:
description: "Directory within the checkout to scan (default: the repository root)."
required: false
default: "."
base:
description: >-
Branch the PR targets. The diff is taken against the MERGE BASE of this and the head, so
commits that landed on the base since the branch point are not counted as the PR's changes.
required: false
default: ""
min-severity:
description: "Report findings at or above this severity: critical, high, medium or low."
required: false
default: "high"
min-confidence:
description: "Drop verified findings below this confidence (0-1)."
required: false
default: "0.5"
max-files:
description: >-
Cap on changed files scanned. A PR diff is small and each file costs model calls, so this is
deliberately much lower than a full scan's default of 200.
required: false
default: "40"
comment:
description: "Post (or update) the review comment on the pull request."
required: false
default: "true"
fail-on-findings:
description: >-
Fail the check when anything is reported. Default false: the comment is the deliverable, and a
red check on a finding the team has decided to accept is how a useful bot gets switched off.
required: false
default: "false"
anthropic-api-key:
description: "Model credential for the scan. Pass a secret; never hardcode."
required: true
github-token:
description: "Token used to comment. The default GITHUB_TOKEN needs pull-requests: write."
required: false
default: ${{ github.token }}
python-version:
description: "Python to run under."
required: false
default: "3.12"
simulation-json:
description: >-
Optional path to a `simulation.json` from a REAL tenant run. Blast radius cannot be measured in
CI — it requires attaching a policy to a load balancer — so supply one here to have the
would-block numbers appear in the comment. Without it the comment says no measurement was made,
which is the honest answer rather than an implied all-clear.
required: false
default: ""

outputs:
reported:
description: "How many findings were reported at or above the threshold."
value: ${{ steps.review.outputs.reported }}
comment-path:
description: "Path to the rendered comment markdown."
value: ${{ steps.review.outputs.comment-path }}

runs:
using: composite
steps:
- name: Check the model credential is actually present
shell: bash
env:
KEY: ${{ inputs.anthropic-api-key }}
run: |
# `required: true` on an input is not enforced by the runner when the caller passes an empty
# value — and `secrets.ANTHROPIC_API_KEY` is the empty string when the secret is not set. The
# scan would then fail deep inside the pipeline, or worse, discover nothing and read as a
# clean review. Fail here, where the message can name the cause.
if [ -z "${KEY}" ]; then
{
echo "## ⚠️ The review did not run"
echo
echo "\`anthropic-api-key\` is empty — the repository secret is probably not set."
echo "**This is not a clean bill of health**: no code was analysed."
} >> "$GITHUB_STEP_SUMMARY"
echo "::error::anthropic-api-key is empty — set the ANTHROPIC_API_KEY repository secret"
exit 1
fi

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}

- name: Install vpcopilot
shell: bash
run: |
# This is a LOCAL action: it installs vpcopilot from the checkout it ships in, so the
# version reviewing a PR is the version in that PR. `github.action_path` is
# `.github/actions/vpcopilot-scan`, three levels below the root — resolved rather than passed
# to pip with `..` segments, which pip accepts inconsistently across versions.
root="$(cd "${{ github.action_path }}/../../.." && pwd)"
echo "installing vpcopilot from ${root}"
python -m pip install --quiet -e "${root}[deploy]"

- name: Check out enough history for a merge base
shell: bash
run: |
# `base` is documented as a BRANCH NAME, but `origin/main` is the natural thing to pass (it is
# the CLI's own default), and blindly prefixing would ask git for `origin/origin/main`. Strip
# a leading remote prefix so both forms work.
BASE_REF="${BASE_REF_RAW#origin/}"
echo "BASE_REF=${BASE_REF}" >> "$GITHUB_ENV"
# actions/checkout fetches a single commit by default, which leaves `git merge-base`
# with nothing to find. Deepen rather than demand fetch-depth: 0 in every caller.
git -C "${{ github.workspace }}" rev-parse --verify HEAD >/dev/null
if ! git -C "${{ github.workspace }}" merge-base HEAD "origin/${BASE_REF}" >/dev/null 2>&1; then
echo "deepening the checkout so a merge base exists"
git -C "${{ github.workspace }}" fetch --quiet --deepen=200 origin "${BASE_REF}" || true
git -C "${{ github.workspace }}" fetch --quiet origin "${BASE_REF}" || true
fi
env:
BASE_REF_RAW: ${{ inputs.base != '' && inputs.base || github.event.pull_request.base.ref || github.event.repository.default_branch }}

- name: Stage the simulation result, if one was supplied
if: inputs.simulation-json != ''
shell: bash
env:
SIM_JSON: ${{ inputs.simulation-json }}
run: |
# Through env, not interpolated into the command: an Actions expression inside `run:` is
# substituted as TEXT before bash sees it, so any input carrying shell metacharacters is a
# script-injection vector. These inputs come from the workflow author rather than a PR, but
# the safe form costs nothing and the habit is what matters.
#
# This comment deliberately does NOT spell that expression out. The runner parses expression
# syntax anywhere in a run block, comments included, so writing an empty one here failed the
# whole action to load with "An expression was expected" — which pyyaml cannot see, because
# it parses YAML and not Actions templates.
mkdir -p out-ci
cp "$SIM_JSON" out-ci/simulation.json
echo "blast-radius numbers will come from $SIM_JSON"

- name: Review the diff
id: review
shell: bash
working-directory: ${{ github.workspace }}
env:
ANTHROPIC_API_KEY: ${{ inputs.anthropic-api-key }}
GITHUB_TOKEN: ${{ inputs.github-token }}
BASE_REF_RAW: ${{ inputs.base != '' && inputs.base || github.event.pull_request.base.ref || github.event.repository.default_branch }}
PR_NUMBER: ${{ github.event.pull_request.number }}
# Every input reaches bash through the environment rather than a `${{ }}` expansion inside
# `run:`, which is substituted as TEXT before bash parses the line.
REPO_PATH: ${{ inputs.repo-path }}
MIN_SEVERITY: ${{ inputs.min-severity }}
MIN_CONFIDENCE: ${{ inputs.min-confidence }}
MAX_FILES: ${{ inputs.max-files }}
WANT_COMMENT: ${{ inputs.comment }}
FAIL_ON_FINDINGS: ${{ inputs.fail-on-findings }}
GH_REPO_SLUG: ${{ github.repository }}
run: |
set -o pipefail
# Self-contained rather than relying on the previous step's GITHUB_ENV: `origin/main` is the
# natural thing to pass for `base` (it is the CLI's own default) and would otherwise become
# `origin/origin/main`.
BASE_REF="${BASE_REF_RAW#origin/}"
args=(--repo "$REPO_PATH"
--base "origin/${BASE_REF}"
--out out-ci
--min-severity "$MIN_SEVERITY"
--min-confidence "$MIN_CONFIDENCE"
--max-files "$MAX_FILES"
--comment-out out-ci/pr-comment.md)
if [ "$WANT_COMMENT" = "true" ] && [ -n "${PR_NUMBER}" ]; then
args+=(--post --pr-repo "$GH_REPO_SLUG" --pr "${PR_NUMBER}")
fi
# ci-review exits 1 when it reports a finding, which is information rather than a failure —
# `fail-on-findings` decides whether the check goes red. Exit 2 is a real error and always
# fails.
set +e
vpcopilot ci-review "${args[@]}"
rc=$?
set -e
if [ "$rc" -ge 2 ]; then
echo "ci-review failed (exit $rc)"
exit "$rc"
fi
echo "reported=$([ "$rc" -eq 1 ] && echo 1 || echo 0)" >> "$GITHUB_OUTPUT"
echo "comment-path=out-ci/pr-comment.md" >> "$GITHUB_OUTPUT"
if [ "$rc" -eq 1 ] && [ "$FAIL_ON_FINDINGS" = "true" ]; then
echo "failing the check because fail-on-findings is true"
exit 1
fi
exit 0

- name: Summary
if: always()
shell: bash
env:
# `if: always()` means this also runs when the review CRASHED. Without knowing which, it
# wrote "no findings at or above the threshold" for a review that never completed — a clean
# bill of health for something that was never looked at, in the one place a human reads.
REVIEW_OUTCOME: ${{ steps.review.outcome }}
run: |
if [ -f out-ci/pr-comment.md ]; then
cat out-ci/pr-comment.md >> "$GITHUB_STEP_SUMMARY"
elif [ "$REVIEW_OUTCOME" = "success" ]; then
echo "No findings at or above the threshold — nothing was posted." >> "$GITHUB_STEP_SUMMARY"
else
{
echo "## ⚠️ The review did not complete"
echo
echo "\`vpcopilot ci-review\` exited \`${REVIEW_OUTCOME}\` and produced no comment."
echo "**This is not a clean bill of health** — the diff was not reviewed. See the step log."
} >> "$GITHUB_STEP_SUMMARY"
fi
83 changes: 83 additions & 0 deletions .github/workflows/pr-review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: pr-review

# K2 — scan the diff on a pull request and comment the proposed band-aid.
#
# Deliberately NOT part of `ci.yml`: that workflow is the test gate and must stay green, fast and
# free of model credentials. This one spends model calls and needs write access to comment, so it is
# its own workflow with its own permissions and can be disabled without touching the tests.
#
# `pull_request` (not `pull_request_target`) is the safe trigger: it runs with a read-only token
# against the merge commit and cannot be used by a fork to exfiltrate secrets. The consequence is
# that PRs from forks get no secrets and therefore no review — the right trade, and stated in
# docs/CI.md rather than discovered.

on:
pull_request:
# Scoped to the directory the action below actually SCANS, not to every code file in the repo.
# A `**/*.py` filter fired on any change — including this repo's own `src/vpcopilot/**` — and
# then scanned zero files, because `repo-path` points at the demo app. That spends a model
# credential and a runner to review nothing. Point both at the same place, or the trigger and
# the scan disagree about what this workflow is for.
paths:
- "bench/fixtures/nimbus-vuln-lab/app/src/app/api/**"
- ".github/workflows/pr-review.yml"
- ".github/actions/vpcopilot-scan/**"
workflow_dispatch:

permissions:
contents: read
pull-requests: write # to leave the review comment, and nothing else

concurrency:
# A pushed branch cancels its own in-flight review rather than racing two comments.
group: pr-review-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
review:
# Two reasons to skip rather than fail. A fork PR has no access to secrets, so the scan could not
# run; and a repository that has not set ANTHROPIC_API_KEY has nothing to run it with. Neither is
# something the PR author can fix, and a permanently red check is a check people learn to ignore.
# Skipping is safe here only because the action itself fails loudly on an empty key — so the
# combination cannot produce a green check for a review that did not happen.
if: >-
(github.event.pull_request.head.repo.full_name == github.repository
|| github.event_name == 'workflow_dispatch')
&& github.event.repository.fork == false
runs-on: ubuntu-latest
env:
# Read once at job level so both the skip note and the review step key off the same value.
MODEL_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
steps:
- name: Say so when there is no model credential to scan with
if: env.MODEL_KEY == ''
run: |
{
echo "## The PR review was skipped"
echo
echo "\`ANTHROPIC_API_KEY\` is not set for this repository, so there is nothing to scan"
echo "with. **This is not a clean bill of health** — the diff was not reviewed."
echo
echo "Set the secret, or run \`vpcopilot ci-review\` locally (see docs/CI.md)."
} >> "$GITHUB_STEP_SUMMARY"

- uses: actions/checkout@v4
if: env.MODEL_KEY != ''
with:
# `git merge-base` needs history on both sides; the default single-commit fetch has none.
fetch-depth: 0

- name: Review the diff
if: env.MODEL_KEY != ''
uses: ./.github/actions/vpcopilot-scan
with:
# The demo app this repo carries. Point this at your own source directory.
repo-path: bench/fixtures/nimbus-vuln-lab/app/src/app/api
min-severity: high
max-files: 40
comment: "true"
# The comment is the deliverable; a red check on an accepted finding is how a useful bot
# gets switched off.
fail-on-findings: "false"
anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
github-token: ${{ github.token }}
Loading
Loading