Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 154 additions & 4 deletions .github/workflows/go-generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
# It runs the commands that regenerate the code (go.sum, mocks, CRDs, ...) and fails
# when the result differs from what is committed, so a PR cannot merge with stale
# generated files. `git status --porcelain` also sees new untracked files.
# On a Renovate branch, it can also push the regenerated code (push-to-renovate-branches).

name: go-generated

Expand All @@ -22,22 +23,58 @@ on:
required: false
default: .
type: string
runs-on:
description: 'Runner label'
push-to-renovate-branches:
Comment thread
ezekiel-alexrod marked this conversation as resolved.
Resolved
description: 'On a renovate/* branch of this repository, push the regenerated code instead of only failing'
required: false
default: ubuntu-24.04
default: false
type: boolean
actions-app-id:
description: 'GitHub App ID used to push the regenerated code (usually vars.ACTIONS_APP_ID)'
required: false
default: ''
type: string
secrets:
GIT_ACCESS_TOKEN:
description: 'Token to fetch private github.com/scality modules'
required: false
ACTIONS_APP_PRIVATE_KEY:
description: 'Private key of the GitHub App, required with push-to-renovate-branches'
required: false

permissions:
contents: read

env:
# Push mode: a renovate/* branch of this repository, never a fork.
RENOVATE_PUSH: >-
${{ inputs.push-to-renovate-branches && github.event_name == 'pull_request'
&& startsWith(github.head_ref, 'renovate/')
&& github.event.pull_request.head.repo.full_name == github.repository }}

jobs:
# A job of its own: it reads the App key, so it must not run with the generators.
# An error only when a push is due. Elsewhere a warning, so a caller sees a missing App
# config early, while PRs that get no Actions secrets (Dependabot, forks) stay green.
validate-push-inputs:
if: inputs.push-to-renovate-branches
runs-on: ubuntu-26.04
steps:
- name: Validate inputs
env:
APP_ID: ${{ inputs.actions-app-id }}
HAS_KEY: ${{ secrets.ACTIONS_APP_PRIVATE_KEY != '' }}
Comment thread
ezekiel-alexrod marked this conversation as resolved.
Resolved
run: |
if [ -z "$APP_ID" ] || [ "$HAS_KEY" != "true" ]; then
msg="push-to-renovate-branches needs actions-app-id and the ACTIONS_APP_PRIVATE_KEY secret"
if [ "$RENOVATE_PUSH" = "true" ]; then echo "::error::$msg"; exit 1; fi
echo "::warning::$msg"
fi

go-generated:
runs-on: ${{ inputs.runs-on }}
runs-on: ubuntu-26.04
Comment thread
ezekiel-alexrod marked this conversation as resolved.
Resolved
outputs:
patch: ${{ steps.patch.outputs.patch }}
artifact: ${{ steps.patch.outputs.artifact }}
defaults:
run:
working-directory: ${{ inputs.working-directory }}
Expand All @@ -47,6 +84,7 @@ jobs:
with:
# The commands run repo code: keep the job token out of .git/config.
persist-credentials: false
ref: ${{ env.RENOVATE_PUSH == 'true' && github.event.pull_request.head.sha || '' }}
- name: Set up Go
uses: actions/setup-go@v7
with:
Expand All @@ -67,6 +105,32 @@ jobs:
env:
COMMANDS: ${{ inputs.commands }}
run: bash -euo pipefail -c "$COMMANDS"
# See "Renovate branches" in docs/go-generated.md for the design of the push.
- name: Save the regenerated code for the Renovate branch
id: patch
if: env.RENOVATE_PUSH == 'true'
run: |
outside=$(git status --porcelain -- ':/' ':!.')
if [ -n "$outside" ]; then
echo "::error::The commands changed files outside working-directory, nothing is pushed:"
echo "$outside"
exit 1
fi
git add -A .
git diff --cached --binary > "$RUNNER_TEMP/regenerated.patch"
if [ -s "$RUNNER_TEMP/regenerated.patch" ]; then
echo "patch=true" >> "$GITHUB_OUTPUT"
# Unique per call: a caller may run this workflow once per module in one run.
echo "artifact=go-generated-patch-${GITHUB_JOB}-$(date +%s%N)" >> "$GITHUB_OUTPUT"
fi
git reset -q
- name: Upload the regenerated code
if: steps.patch.outputs.patch == 'true'
uses: actions/upload-artifact@v7
with:
name: ${{ steps.patch.outputs.artifact }}
path: ${{ runner.temp }}/regenerated.patch
retention-days: 1
- name: Check that nothing changed
env:
COMMANDS: ${{ inputs.commands }}
Expand All @@ -78,3 +142,89 @@ jobs:
git diff
exit 1
fi

push-to-renovate-branch:
needs: go-generated
# The check above fails on purpose: the pushed commit starts a new run, which passes.
# The branch condition again, not only the output of the job that ran the generators:
# they can write to $GITHUB_ENV there. A job-level `if` cannot read env, so it is inlined.
if: >-
${{ !cancelled() && needs.go-generated.outputs.patch == 'true'
&& inputs.push-to-renovate-branches && github.event_name == 'pull_request'
&& startsWith(github.head_ref, 'renovate/')
&& github.event.pull_request.head.repo.full_name == github.repository }}
runs-on: ubuntu-26.04
steps:
- uses: actions/create-github-app-token@v3
id: app-token
with:
app-id: ${{ inputs.actions-app-id }}
private-key: ${{ secrets.ACTIONS_APP_PRIVATE_KEY }}
repositories: ${{ github.event.repository.name }}
permission-contents: write
- name: Checkout
uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha }}
token: ${{ steps.app-token.outputs.token }}
# The loop guard reads every commit of the branch since the base.
fetch-depth: 0
- name: Download the regenerated code
uses: actions/download-artifact@v7
with:
name: ${{ needs.go-generated.outputs.artifact }}
path: ${{ runner.temp }}
- name: Commit and push
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
APP_SLUG: ${{ steps.app-token.outputs.app-slug }}
BASE_REF: ${{ github.base_ref }}
HEAD_REF: ${{ github.head_ref }}
WORKING_DIRECTORY: ${{ inputs.working-directory }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
user_id=$(gh api "/users/${APP_SLUG}%5Bbot%5D" --jq .id)
email="${user_id}+${APP_SLUG}[bot]@users.noreply.github.com"
git apply --index "$RUNNER_TEMP/regenerated.patch"
# --no-renames: a move out of .github/ must show its old path too. In a variable:
# `| grep -q` can stop git diff early, and pipefail then hides the match.
names=$(git diff --cached --name-only --no-renames -z | tr '\0' '\n')
if printf '%s\n' "$names" | awk 'index($0, ".github/") == 1 {found=1} END {exit !found}'; then
echo "::error::The regenerated code changes .github/, it is not pushed."
exit 1
fi
# Checked again here: the first job ran the generators, which could fake its checks.
dir="${WORKING_DIRECTORY#./}"; dir="${dir%/}"
if [ -n "$dir" ] && [ "$dir" != "." ]; then
# A literal prefix, not a regex: a "." in working-directory must not match any char.
outside=$(git diff --cached --name-only --no-renames -z | tr '\0' '\n' |
awk -v p="${dir}/" 'index($0, p) != 1')
if [ -n "$outside" ]; then
echo "::error::The patch changes files outside ${dir}/, it is not pushed:"
echo "$outside"
exit 1
fi
fi
# Loop guard: files this App already regenerated on the branch, all modules together.
base=$(git merge-base HEAD "origin/${BASE_REF}")
done_files=$(git log --format='%H %ae' "${base}..HEAD" | awk -v e="$email" '$2 == e {print $1}' |
xargs -r -n1 git show --name-only --no-renames --format= | sort -u)
# In a variable: `| grep -q` can stop comm early, and pipefail then hides the match.
again=$(comm -12 <(git diff --cached --name-only --no-renames | sort) <(echo "$done_files"))
if [ -n "$again" ]; then
echo "::error::These files were already regenerated on this branch and still differ:"
echo "$again"
exit 1
fi
git -c core.hooksPath=/dev/null -c user.name="${APP_SLUG}[bot]" -c user.email="$email" \
commit -q -m "chore(deps): regenerate the code"
# Not forced: when the branch moved (Renovate, or another module's push job),
# the push is rejected and the new commit starts its own run.
if ! git -c core.hooksPath=/dev/null push origin "HEAD:refs/heads/${HEAD_REF}"; then
if [ "$(git ls-remote origin "refs/heads/${HEAD_REF}" | cut -f1)" != "$HEAD_SHA" ]; then
echo "::notice::${HEAD_REF} moved since this run started, nothing pushed."
exit 0
fi
exit 1
fi
echo "Pushed as ${APP_SLUG}[bot] <${email}>. Add this email to gitIgnoredAuthors in the Renovate config." >> "$GITHUB_STEP_SUMMARY"
6 changes: 6 additions & 0 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,14 @@ jobs:

go-generated:
uses: ./.github/workflows/go-generated.yaml
# Push mode only pushes on a renovate/* branch: here it checks the inputs and the
# condition, and stays a plain check.
secrets:
ACTIONS_APP_PRIVATE_KEY: ${{ secrets.ACTIONS_APP_PRIVATE_KEY }}
with:
working-directory: tests/go-generated
push-to-renovate-branches: true
actions-app-id: ${{ vars.ACTIONS_APP_ID }}

release-dry-run:
uses: ./.github/workflows/release.yaml
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ This is **scality/workflows**, a repository of reusable GitHub Actions workflows
| `lfs-warning.yaml` | Validates file sizes in PRs, warns about files not tracked by Git LFS |
| `release.yaml` | Computes the next semver off the last GA tag and pushes an annotated `v*` tag |
| `promote.yaml` | Creates a GitHub Release from a pushed `v*` tag (pre-release on hyphenated tags) |
| `go-generated.yaml` | Reruns the Go code generation (`go mod tidy`, `go generate`, ...) and fails when the result differs from what is committed |
| `go-generated.yaml` | Reruns the Go code generation (`go mod tidy`, `go generate`, ...) and fails when the result differs from what is committed; can push the regenerated code on Renovate branches |
| `claude-code-review.yml` | AI-powered PR review via Anthropic API (Workload Identity Federation), reports costs to `cost-center-review` |
| `claude-code-dependency-review.yml` | AI-powered review of Dependabot/Renovate bumps via Anthropic API (Workload Identity Federation), reports costs to `cost-center-review` |

Expand Down
62 changes: 61 additions & 1 deletion docs/go-generated.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,64 @@ jobs:
make manifests generate
```

## Renovate branches

When Renovate bumps a code generator (oapi-codegen, mockery, controller-tools), its PR
changes `go.mod` but not the generated code, so this check fails until someone runs the
commands and pushes on the Renovate branch. Renovate then stops updating that branch.

With `push-to-renovate-branches`, the workflow pushes the regenerated code itself on a
`renovate/*` branch of the repository. The check still fails on the Renovate commit, and
the pushed commit starts a new run that passes.

```yaml
jobs:
go-generated:
uses: scality/workflows/.github/workflows/go-generated.yaml@v2
secrets:
ACTIONS_APP_PRIVATE_KEY: ${{ secrets.ACTIONS_APP_PRIVATE_KEY }}
with:
push-to-renovate-branches: true
actions-app-id: ${{ vars.ACTIONS_APP_ID }}
```

Pass the secrets one by one, not with `secrets: inherit`: the commands can read
`GIT_ACCESS_TOKEN`, so only pass it when the repository needs private modules, and only
with a read-only token.

How it stays safe:

- The commands run with a read-only token. A second job, which never runs the code of
the repository (git hooks are disabled), applies the result and pushes it.
- The push uses a GitHub App token limited to the repository, because a push made with
`GITHUB_TOKEN` starts no workflow, so the required checks would never run.
- It only runs on a `renovate/*` branch of the repository, never on a fork.
- The pushed commit only holds changes under `working-directory`, never under `.github/`.
When the commands change files outside `working-directory`, nothing is pushed. The push
job checks this again, since the first job ran the generators.
With one call per module, each run pushes one module: the first push moves the branch,
the other calls push nothing, and the next run regenerates the remaining modules.
- It does not loop: when files the App already regenerated on the branch still differ,
the job fails instead of pushing again.
- It does not force-push. If Renovate moved the branch meanwhile, it pushes nothing: the
new commit starts its own run, which regenerates again.

Limits:

- On a Renovate branch, the commands run on the branch head, not on its merge with the
base branch. A base change that needs regeneration is caught on the next PR.
- The pushed code is what the generators wrote. A compromised generator could add code
there, so keep these PRs under human review: do not automerge a PR that holds a
regeneration commit.

The inputs are checked on every PR: a missing App ID or key fails the run when a push
is due, and only warns otherwise, so PRs that get no Actions secrets (Dependabot, forks)
stay green.

The commit author is the GitHub App. Add its email, printed in the run summary, to
`gitIgnoredAuthors` in the Renovate config: otherwise Renovate sees a foreign commit and
stops updating the branch.

Code generators run through `go generate` are best declared as `tool` in `go.mod`
(`go get -tool github.com/vektra/mockery/v3`): their version is then pinned and
updated like any other dependency.
Expand All @@ -42,10 +100,12 @@ updated like any other dependency.
| ------------------- | ------------------------------------ | ------------------------------------ |
| `commands` | Commands that regenerate the code | `go mod tidy` and `go generate ./...` |
| `working-directory` | Directory that holds the `go.mod` | `.` |
| `runs-on` | Runner label | `ubuntu-24.04` |
| `push-to-renovate-branches` | Push the regenerated code on a `renovate/*` branch | `false` |
| `actions-app-id` | GitHub App ID used for that push (usually `vars.ACTIONS_APP_ID`) | `''` |

## Secrets

| Name | Description |
| ------------------ | ------------------------------------------------------------------ |
| `GIT_ACCESS_TOKEN` | Optional, to fetch private `github.com/scality` modules. The commands can read it: use a read-only token |
| `ACTIONS_APP_PRIVATE_KEY` | Private key of the GitHub App, required with `push-to-renovate-branches` |
Loading