Skip to content

feat(xtask): add fuzz coverage subcommand - #1733

Open
Greg Lamberson (glamberson) wants to merge 1 commit into
Devolutions:masterfrom
lamco-admin:feat/xtask-fuzz-coverage
Open

feat(xtask): add fuzz coverage subcommand#1733
Greg Lamberson (glamberson) wants to merge 1 commit into
Devolutions:masterfrom
lamco-admin:feat/xtask-fuzz-coverage

Conversation

@glamberson

Copy link
Copy Markdown
Contributor

Summary

  • Coverage-guided fuzzing typically plateaus around 12 hours in (Liyanage et
    al., ICSE 2023), and several targets in this workspace have run far longer
    than that without any coverage-feedback check. A target can keep running
    for hours after its coverage stopped growing, hiding bug-finding
    opportunity elsewhere, with nothing surfacing it.
  • Adds cargo xtask fuzz coverage [--target NAME]: runs a target (or all
    targets, matching fuzz run's existing default) against its existing
    corpus via cargo fuzz coverage, then renders a per-file line coverage
    report via the toolchain's own llvm-cov, filtered to only files with
    nonzero coverage. Auto-discovers targets via the existing
    discover_targets() used by run/list/corpus-min.
  • llvm-cov/llvm-profdata are resolved from the pinned nightly
    toolchain's own rustlib tree rather than a plain PATH lookup: a PATH
    resolution can land on an unrelated system-wide LLVM install with an
    incompatible profile format, which is what happened locally when testing
    this. cargo fuzz coverage resolves llvm-profdata the same way
    internally, for the same reason.
  • Adds the llvm-tools-preview rustup component to fuzz install, needed
    to merge raw coverage profiles. Without it, coverage fails partway
    through with a missing llvm-profdata error instead of at install time.
  • fuzz/coverage/ is gitignored alongside the existing corpus and
    artifacts entries.

Validation

cargo xtask check fmt/lints/tests/typos/locks all pass. Also ran the
subcommand end to end against a seeded pdu_decoding corpus: correctly
narrowed the full linked-binary report from ~60 source files down to the 49
the target actually touches.

Notes

Pure additive, no existing subcommand changes. The compiled fuzz binary's
path assumes the x86_64-unknown-linux-gnu target triple, matching cargo fuzz coverage's own documented default rather than introducing a new
limitation.

Coverage-guided fuzzing typically plateaus around 12 hours in (Liyanage
et al., ICSE 2023), and several of the workspace's fuzz targets have
run far longer than that without any coverage-feedback check. A target
can keep running for hours after its coverage stopped growing, hiding
bug-finding opportunity elsewhere, with nothing surfacing it.

Add `cargo xtask fuzz coverage [--target NAME]`: runs a target (or all
targets, matching `fuzz run`'s existing default) against its existing
corpus via `cargo fuzz coverage`, then renders a per-file line coverage
report via the toolchain's own llvm-cov, filtered to only the files
with nonzero coverage. The full report otherwise lists every source
file linked into the fuzz binary, most of which a given target never
reaches and are not useful to see.

llvm-cov and llvm-profdata are resolved from the pinned nightly
toolchain's own rustlib tree rather than a plain PATH lookup: `rustup
run` does not expose them, and whatever `llvm-cov` PATH does resolve to
can be an unrelated system-wide LLVM install with an incompatible
profile format, which is what a `rustup run {NIGHTLY_TOOLCHAIN} --
llvm-cov` invocation actually hit locally. `cargo fuzz coverage`
resolves llvm-profdata the same way internally for the same reason.

Also adds the `llvm-tools-preview` rustup component to `fuzz install`,
required to merge raw coverage profiles; without it `coverage` failed
partway through with a missing llvm-profdata error instead of failing
at install time where the other fuzzing prerequisites are handled.
`fuzz/coverage/`, the report's working directory, is gitignored
alongside the existing `corpus` and `artifacts` entries.

Auto-discovers targets via the existing `discover_targets()` used by
`run`/`list`/`corpus-min`, so new fuzz targets pick up coverage
reporting automatically.
@github-actions github-actions Bot added maintainer-required Maintainer review or intervention is required risk/low Self-contained change with no cross-crate behavioral effect scope/tooling Build, CI, release, or developer tooling size/S Size: up to 199 counted lines and 5 files; exceeds XS in either measure labels Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer-required Maintainer review or intervention is required risk/low Self-contained change with no cross-crate behavioral effect scope/tooling Build, CI, release, or developer tooling size/S Size: up to 199 counted lines and 5 files; exceeds XS in either measure

Development

Successfully merging this pull request may close these issues.

1 participant