feat(fleet): re-run every app's gates weekly, so drift can turn a badge red - #507
Open
rubenvdlinde wants to merge 2 commits into
Open
feat(fleet): re-run every app's gates weekly, so drift can turn a badge red#507rubenvdlinde wants to merge 2 commits into
rubenvdlinde wants to merge 2 commits into
Conversation
added 2 commits
August 19, 2026 07:25
…ge red
ANSWERING A DIRECT QUESTION: no, the Friday merge routine would not have
caught this, and nothing else would either.
A GREEN BADGE IS A VERDICT ABOUT A MOMENT, NOT A STATE. The gates are
consumed at `@main`, so a stricter gate reaches every app the instant it
merges here — but a gate only produces a verdict when a workflow RUNS, and
app workflows trigger on `push` and `pull_request` only. An app nobody has
touched keeps the green badge it earned under the OLD gates. Tighten a gate
and the fleet is non-compliant and green simultaneously, until somebody
happens to push. The badge is not lying about the past; it is being read as
a claim about the present.
Measured 2026-08-19: of 21 swept apps, exactly ONE (pipelinq) had any
scheduled run at all. Twenty could only be re-measured by hand. For a fleet
that intends to be managed through ADRs and gates, that is the load-bearing
gap — the gates can be changed centrally and the change reaches nothing.
WHY IT IS CENTRAL, WHICH IS THE PART THAT IS EASY TO GET WRONG
`schedule:` cannot choose a branch. GitHub runs scheduled workflows from the
repository's DEFAULT branch and offers no ref input. The fleet's defaults are
split almost evenly — measured today, 11 apps default to `development` and 10
to `main`:
docudesk, launchpad, nldesign, opencatalogi, openconnector, openregister,
procest, softwarecatalog, zaakafhandelapp, nextcloud-app-template
Copying pipelinq's cron into every app would therefore measure `development`
on eleven and `main` on ten. `main` is stale-red across much of the fleet, so
those ten would go red for reasons that have nothing to do with drift. The
comment in pipelinq's own workflow — "a scheduled run always uses the default
branch's ref (development)" — is true FOR PIPELINQ and becomes false the
moment it is copied.
`workflow_dispatch` does take a ref, so the sweep dispatches centrally with
`--ref development` stated explicitly for every app.
WHAT IT REFUSES TO DO SILENTLY
- It asserts FLEET_DISPATCH_TOKEN before dispatching anything. Without a
cross-repo token every call would be refused and the job would report a
green sweep that measured nothing — the exact failure this exists to stop.
GITHUB_TOKEN cannot do this; it is repository-scoped.
- It COUNTS dispatches and fails when the count is short, naming the apps it
could not reach. A partial sweep that exits 0 tells the reader the fleet
was re-measured, and the apps that were not are indistinguishable from the
ones that passed.
- It refuses an empty app list, because a sweep over nothing also exits 0.
It grades nothing. Each app's own Quality Report decides; the sweep only
ensures a verdict exists that was computed against TODAY's gates.
fleet-apps.json is the machine-readable scope of record beside the human
app-health page. `planix` is listed as deprecated rather than omitted, so the
decision is visible — a name quietly absent is indistinguishable from a name
never added, which is how hrmq sat red for six days unseen.
NOT YET LIVE: the token has to be provisioned. The workflow fails loudly
until it is, which is the honest state to ship in.
The workflow dispatched every app's Code Quality and stopped there, on the
stated reasoning that "each app's own Quality Report decides, and the badges on
the app-health page show the outcome".
Measured 2026-08-19, that is false. Every badge on the app-health page carries
`&event=push`, and a sweep produces `workflow_dispatch` runs — shields.io does
not look at them at all, so the badge keeps showing the last PUSH verdict. An
app could fail every gate in the Friday sweep and the fleet page would stay
green: the exact silent failure this workflow was written to prevent, rebuilt
one layer down.
Positive control on pipelinq/development, so the filter is shown to be live
rather than assumed:
?branch=development&event=push -> newest: push, success
?branch=development -> newest: pull_request, FAILURE
Dropping the filter is not the fix — unfiltered, the newest run on a branch is
often a pull_request run of a merge ref, a verdict about a proposal rather than
about the branch.
So a `collect` job waits for the runs this sweep started and fails when any app
is not `success`, with a per-app table in the run summary. It identifies "this
sweep's run" by a timestamp stamped before the first dispatch, because
`gh workflow run` returns no run id and "the newest dispatch run" would happily
adopt one from last week. `cancelled`, `no run appeared` and `still running at
deadline` are each reported red — an app that produced no verdict is never
subtracted as though it passed.
A second badge column on `event=workflow_dispatch` is viable (an unswept app
renders grey `no status`, not a false red — verified against
nextcloud-app-template) but is deliberately deferred: every app already carries
unrelated MANUAL dispatch runs, openregister's newest being 2026-08-11 and
failed, so the column would open showing week-old verdicts as this Friday's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answering a direct question: no — the Friday merge routine would not have caught this, and nothing else would either.
A green badge is a verdict about a moment, not a state. The gates are consumed at
@main, so a stricter gate reaches every app the instant it merges here — but a gate only produces a verdict when a workflow runs, and app workflows trigger onpushandpull_requestonly. An app nobody has touched keeps the green badge it earned under the old gates. Tighten a gate and the fleet is non-compliant and green at the same time, until somebody happens to push.Measured 2026-08-19: of 21 swept apps, exactly one (pipelinq) had any scheduled run at all. For a fleet that intends to be managed through ADRs and gates, that is the load-bearing gap — the gates can be changed centrally and the change reaches nothing.
Why it is central, which is the part that is easy to get wrong
schedule:cannot choose a branch. GitHub runs scheduled workflows from the repo's default branch with no ref input. The fleet's defaults are split almost evenly — 11 default todevelopment, 10 tomain:So copying pipelinq's cron everywhere would measure
developmenton eleven andmainon ten — andmainis stale-red across much of the fleet, so those ten would go red for reasons unrelated to drift. The comment in pipelinq's own workflow ("a scheduled run always uses the default branch's ref (development)") is true for pipelinq and becomes false the moment it is copied.workflow_dispatchdoes take a ref, so the sweep dispatches centrally with--ref developmentstated explicitly.What it refuses to do silently
FLEET_DISPATCH_TOKENbefore dispatching. Without a cross-repo token every call would be refused and the job would report a green sweep that measured nothing — the exact failure this exists to stop.GITHUB_TOKENis repository-scoped and cannot.It grades nothing — each app's Quality Report decides. Its only job is to ensure a verdict exists that was computed against today's gates.
Not yet live
The token has to be provisioned: an org-level
FLEET_DISPATCH_TOKENwithactions: writeon every repo infleet-apps.json. The workflow fails loudly until it is, which is the honest state to ship in.