feat(xtask): add fuzz coverage subcommand - #1733
Open
Greg Lamberson (glamberson) wants to merge 1 commit into
Open
feat(xtask): add fuzz coverage subcommand#1733Greg Lamberson (glamberson) wants to merge 1 commit into
Greg Lamberson (glamberson) wants to merge 1 commit into
Conversation
Coverage-guided fuzzing typically plateaus around 12 hours in (Liyanage
et al., ICSE 2023), and several of the workspace's fuzz targets have
run far longer than that without any coverage-feedback check. A target
can keep running for hours after its coverage stopped growing, hiding
bug-finding opportunity elsewhere, with nothing surfacing it.
Add `cargo xtask fuzz coverage [--target NAME]`: runs a target (or all
targets, matching `fuzz run`'s existing default) against its existing
corpus via `cargo fuzz coverage`, then renders a per-file line coverage
report via the toolchain's own llvm-cov, filtered to only the files
with nonzero coverage. The full report otherwise lists every source
file linked into the fuzz binary, most of which a given target never
reaches and are not useful to see.
llvm-cov and llvm-profdata are resolved from the pinned nightly
toolchain's own rustlib tree rather than a plain PATH lookup: `rustup
run` does not expose them, and whatever `llvm-cov` PATH does resolve to
can be an unrelated system-wide LLVM install with an incompatible
profile format, which is what a `rustup run {NIGHTLY_TOOLCHAIN} --
llvm-cov` invocation actually hit locally. `cargo fuzz coverage`
resolves llvm-profdata the same way internally for the same reason.
Also adds the `llvm-tools-preview` rustup component to `fuzz install`,
required to merge raw coverage profiles; without it `coverage` failed
partway through with a missing llvm-profdata error instead of failing
at install time where the other fuzzing prerequisites are handled.
`fuzz/coverage/`, the report's working directory, is gitignored
alongside the existing `corpus` and `artifacts` entries.
Auto-discovers targets via the existing `discover_targets()` used by
`run`/`list`/`corpus-min`, so new fuzz targets pick up coverage
reporting automatically.
Greg Lamberson (glamberson)
temporarily deployed
to
llm-providers
August 20, 2026 23:50 — with
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
al., ICSE 2023), and several targets in this workspace have run far longer
than that without any coverage-feedback check. A target can keep running
for hours after its coverage stopped growing, hiding bug-finding
opportunity elsewhere, with nothing surfacing it.
cargo xtask fuzz coverage [--target NAME]: runs a target (or alltargets, matching
fuzz run's existing default) against its existingcorpus via
cargo fuzz coverage, then renders a per-file line coveragereport via the toolchain's own
llvm-cov, filtered to only files withnonzero coverage. Auto-discovers targets via the existing
discover_targets()used byrun/list/corpus-min.llvm-cov/llvm-profdataare resolved from the pinned nightlytoolchain's own
rustlibtree rather than a plainPATHlookup: aPATHresolution can land on an unrelated system-wide LLVM install with an
incompatible profile format, which is what happened locally when testing
this.
cargo fuzz coverageresolvesllvm-profdatathe same wayinternally, for the same reason.
llvm-tools-previewrustup component tofuzz install, neededto merge raw coverage profiles. Without it,
coveragefails partwaythrough with a missing
llvm-profdataerror instead of at install time.fuzz/coverage/is gitignored alongside the existingcorpusandartifactsentries.Validation
cargo xtask check fmt/lints/tests/typos/locksall pass. Also ran thesubcommand end to end against a seeded
pdu_decodingcorpus: correctlynarrowed the full linked-binary report from ~60 source files down to the 49
the target actually touches.
Notes
Pure additive, no existing subcommand changes. The compiled fuzz binary's
path assumes the
x86_64-unknown-linux-gnutarget triple, matchingcargo fuzz coverage's own documented default rather than introducing a newlimitation.