fix(safety): catch bare tracker ticket refs in the internal-leak scan - #12454
Merged
Conversation
Member
Author
|
This pull request is part of a Mergify stack:
|
Contributor
Merge Protections🟢 All 6 merge protections satisfied — ready to merge. Show 6 satisfied protections🟢 🤖 Continuous Integration
🟢 👀 Review Requirements
🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 🔎 Reviews
🟢 📕 PR description
🟢 🚦 Auto-queueWhen all merge protections are satisfied, this pull request will be queued automatically. |
There was a problem hiding this comment.
Pull request overview
This PR tightens the pnpm check:internal-leaks gate to detect bare internal tracker ticket keys (not only tracker URLs), and updates documentation + guidance so these leaks are less likely to ship unnoticed in prose and example values.
Changes:
- Add a new
ticket-refleak rule for explicit internal ticket prefixes (including lowercase branch-name form). - Extend the allow-directive syntax to support multiple rule IDs on a single line (comma-separated).
- Sweep the docs to remove the previously published bare ticket key from the Quarantine CLI example, and update contributor/skill guidance to flag bare keys.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/content/docs/test-insights/quarantine.mdx | Removes an internal ticket key from the CLI example --reason value. |
| scripts/check-internal-leaks.test.mjs | Adds test coverage for bare ticket refs, multi-rule allow directives, and false-positive avoidance cases. |
| scripts/check-internal-leaks.mjs | Implements ticket-ref rule and comma-separated allow directive parsing; updates usage text accordingly. |
| AGENTS.md | Updates internal-leak guidance to explicitly include bare tracker ticket keys. |
| .claude/skills/proofread-leaks/SKILL.md | Updates the leak-proofreading checklist to include bare tracker ticket references. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
jd
marked this pull request as ready for review
August 20, 2026 11:52
`pnpm check:internal-leaks` only ever saw a ticket as a *link*: its `internal-tracker` rule matches tracker hosts. A bare issue key in prose, in a code block or in an example value passed cleanly — and one did. The `mergify tests quarantines add` example on the quarantine page shipped with a `--reason` string naming an internal ticket, so it has been live on docs.mergify.com since #11681 and sits in this repository's public history. The scan reported that exact file clean, before and after. A bare key is also the case a reviewer is least likely to stop on: inside an example value it reads as a plausible sample rather than as a leak. What changed: - A `ticket-ref` rule for bare issue keys — the two live prefixes, plus the lowercase form a branch name carries. - Prefixes are listed explicitly instead of matching a generic `[A-Z]{3,}-\d+`. The generic shape fires on `AES-256`, `WCAG-2` and on a reader's own issue keys in a sample config, and that cost lands on every docs contributor; the miss cost of an explicit list falls only on a prefix somebody introduces deliberately, which is a deliberate act. - The allow directive now accepts several rule ids. One line can trip two rules — a tracker URL carries the key inside it — and such a line previously could not be allowed at all. - The corpus is swept: across the 143 scanned pages that example was the only occurrence. Its reason string is now "flaky under load, fix in progress", the same wording #12411 uses for the same line, so the two changes do not fight. That also removes the prose em dash the example carried inside a CLI string. - `AGENTS.md` and the `proofread-leaks` skill now list bare keys as well. The judgment layer had the same blind spot as the regex, which is why the line survived review too. Verified: the scan exits 1 against the pre-fix file and 0 against the tree, the full `pnpm test` suite and `pnpm check` are green, and the commit is green on its own as well as on top of the stack. One decision is deliberately left open: whether the already-published value needs scrubbing beyond this branch. It is a ticket prefix rather than a credential, and it is in the git history of a public repository as well as on the site, so removing it from the page is all this change claims to do. Reported as Mergifyio/ci-bot#368. MRGFY-8721 Change-Id: I847b80676f560fb73f319e2524be6d94207591e8
jd
force-pushed
the
devs/jd/jd/mrgfy-8721-docs-repo-internal-leak-scan-misses-bare-ticket-refs-and/catch-bare-tracker-ticket-refs-internal-leak-scan--847b8067
branch
from
August 20, 2026 12:03
3caaf1e to
dd28773
Compare
Member
Author
Revision history
|
JulianMaurin
approved these changes
Aug 20, 2026
AlexandreGaubert
approved these changes
Aug 20, 2026
Contributor
Merge Queue Status
This pull request spent 2 minutes 57 seconds in the queue, including 2 minutes 28 seconds running CI. Required conditions to merge
|
60 tasks
mergify
Bot
deleted the
devs/jd/jd/mrgfy-8721-docs-repo-internal-leak-scan-misses-bare-ticket-refs-and/catch-bare-tracker-ticket-refs-internal-leak-scan--847b8067
branch
August 20, 2026 13:56
62 tasks
mergify Bot
pushed a commit
that referenced
this pull request
Aug 21, 2026
The "Any CI (Mergify CLI)" block on the four monorepo scopes pages ends in
`mergify ci scopes-send --file scopes.json`. `--file` is a deprecated alias, kept
hidden from `--help` and from `public/cli-schema.json` since the CLI was ported
to Rust: it still works, and prints `Warning: --file is deprecated, use
--scopes-json instead.` on every run. So a reader copying the block gets a
deprecation warning for a flag they cannot look up.
`--scopes-json` is the right one, not `--scopes-file`: it reads the
`{"scopes": [...]}` object, which is exactly the shape every page's `jq` filter
writes, while `--scopes-file` expects one scope per line. Confirmed against the
CLI's own reader — it deserializes `scopes` plus an optional `all_scopes`, so the
examples need no change, and `--all` documented on the scopes page stays
consistent with it.
While in the block, the base and head refs now come from `--format json` piped
through `jq` instead of `awk '/^Base:/ {print $2}'` on the default output. The
text format is the human one with no stability promise; `json` and `shell` are
the two contracts meant for scripts. `jq` is already required by all four pages,
so this adds no dependency, and the variables the pages interpolate (`$BASE`,
`$HEAD`) are unchanged.
Checked the rest of the corpus: no other page repeats `--file`, and the scopes
page already documents `--scopes-json`. Nothing outside this repository generates
the snippet — the dashboard and the engine never emit this command.
Reported as Mergifyio/ci-bot#371.
MRGFY-8721
Depends-On: #12454
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pnpm check:internal-leaksonly ever saw a ticket as a link: itsinternal-trackerrule matches tracker hosts. A bare issue key in prose, in acode block or in an example value passed cleanly — and one did. The
mergify tests quarantines addexample on the quarantine page shipped with a--reasonstring naming an internal ticket, so it has been live ondocs.mergify.com since #11681 and sits in this repository's public history. The
scan reported that exact file clean, before and after.
A bare key is also the case a reviewer is least likely to stop on: inside an
example value it reads as a plausible sample rather than as a leak.
What changed:
ticket-refrule for bare issue keys — the two live prefixes, plus thelowercase form a branch name carries.
[A-Z]{3,}-\d+.The generic shape fires on
AES-256,WCAG-2and on a reader's own issue keysin a sample config, and that cost lands on every docs contributor; the miss
cost of an explicit list falls only on a prefix somebody introduces
deliberately, which is a deliberate act.
— a tracker URL carries the key inside it — and such a line previously could
not be allowed at all.
occurrence. Its reason string is now "flaky under load, fix in progress", the
same wording docs(agent): fix how detection describes test health, confidence and impact #12411 uses for the same line, so the two changes do not fight.
That also removes the prose em dash the example carried inside a CLI string.
AGENTS.mdand theproofread-leaksskill now list bare keys as well. Thejudgment layer had the same blind spot as the regex, which is why the line
survived review too.
Verified: the scan exits 1 against the pre-fix file and 0 against the tree,
the full
pnpm testsuite andpnpm checkare green, and the commit is greenon its own as well as on top of the stack.
One decision is deliberately left open: whether the already-published value
needs scrubbing beyond this branch. It is a ticket prefix rather than a
credential, and it is in the git history of a public repository as well as on
the site, so removing it from the page is all this change claims to do.
Reported as Mergifyio/ci-bot#368.
MRGFY-8721