Skip to content

quality/coverage: enable Rust coverage in the LLVM pipeline#772

Open
dcalavrezo-qorix wants to merge 3 commits into
mainfrom
dcalavrezo_llvm_cov_rust
Open

quality/coverage: enable Rust coverage in the LLVM pipeline#772
dcalavrezo-qorix wants to merge 3 commits into
mainfrom
dcalavrezo_llvm_cov_rust

Conversation

@dcalavrezo-qorix

Copy link
Copy Markdown
Until now the unified llvm-cov pipeline reported C++ only: the Ferrocene
toolchain registered for coverage runs did not declare llvm_cov, so
rules_rust never instrumented Rust crates, no .rs file was in the scope
allowlist, and justify.py did not scan Rust sources.

Toolchain:
- bump score_toolchains_rust to 0.9.2: its ferrocene extension now wires
  llvm-cov/llvm-profdata from the coverage-tools tarball into the
  generated rust_toolchain, which activates -Cinstrument-coverage under
  'bazel coverage'
- bump ferrocene_toolchain_builder coverage-tools to 1.3.0 (ships
  llvm-cov/llvm-profdata/llvm-cxxfilt built from the same LLVM as rustc)

coverage.bazelrc:
- -Cllvm-args=-runtime-counter-relocation: continuous-mode profiling
  (LLVM_PROFILE_CONTINUOUS_MODE) otherwise writes no .profraw for Rust
- -Clink-dead-code: keep unreferenced functions visible as uncovered
- -Zcoverage-options=branch: branch coverage instrumentation (unstable
  flag; works because Ferrocene is a nightly-based rolling build)

Pipeline:
- coverage_scope.bzl: CrateInfo branch collects rust_binary sources and
  executables; rust_library was already handled via CcInfo; remove
  leftover debug prints
- merger.py: resolve RUST_LLVM_PROFDATA (Bazel only sets LLVM_PROFDATA
  via the cc toolchain); pick up the Rust test binary from the coverage
  manifest (ELF detection), skipping external/ toolchain binaries
- reporter.py: normalize /proc/self/cwd/ covmap paths so the allowlist
  subtraction works with mixed C++ (absolute) and Rust (relative) paths;
  expand Rust rlib baseline archives into their .o members (llvm-cov
  rejects rlibs due to the leading lib.rmeta member); resolve
  llvm-cxxfilt via runfiles (demangles Rust v0/legacy and C++ symbols)
- justify.py: scan .rs files for COV_JUSTIFIED markers; skip bazel-*
  convenience symlinks
- quality/coverage/BUILD: add the Rust production crates to
  coverage_scope (mocks/test_support excluded); crate visibilities
  widened by //quality/coverage:__pkg__ only

Known limitations:
- com-api-concept-test and com-api-runtime-lola-tests are tagged
  'manual' and are skipped by wildcard coverage runs; their crates
  report as 0% baseline unless the tests are invoked explicitly
- lib.rs/error.rs/reloc.rs-style files without executable code carry no
  coverage regions and do not appear in reports (same as C++ headers)

Until now the unified llvm-cov pipeline reported C++ only: the Ferrocene
toolchain registered for coverage runs did not declare llvm_cov, so
rules_rust never instrumented Rust crates, no .rs file was in the scope
allowlist, and justify.py did not scan Rust sources.

Toolchain:
- bump score_toolchains_rust to 0.9.2: its ferrocene extension now wires
  llvm-cov/llvm-profdata from the coverage-tools tarball into the
  generated rust_toolchain, which activates -Cinstrument-coverage under
  'bazel coverage'
- bump ferrocene_toolchain_builder coverage-tools to 1.3.0 (ships
  llvm-cov/llvm-profdata/llvm-cxxfilt built from the same LLVM as rustc)

coverage.bazelrc:
- -Cllvm-args=-runtime-counter-relocation: continuous-mode profiling
  (LLVM_PROFILE_CONTINUOUS_MODE) otherwise writes no .profraw for Rust
- -Clink-dead-code: keep unreferenced functions visible as uncovered
- -Zcoverage-options=branch: branch coverage instrumentation (unstable
  flag; works because Ferrocene is a nightly-based rolling build)

Pipeline:
- coverage_scope.bzl: CrateInfo branch collects rust_binary sources and
  executables; rust_library was already handled via CcInfo; remove
  leftover debug prints
- merger.py: resolve RUST_LLVM_PROFDATA (Bazel only sets LLVM_PROFDATA
  via the cc toolchain); pick up the Rust test binary from the coverage
  manifest (ELF detection), skipping external/ toolchain binaries
- reporter.py: normalize /proc/self/cwd/ covmap paths so the allowlist
  subtraction works with mixed C++ (absolute) and Rust (relative) paths;
  expand Rust rlib baseline archives into their .o members (llvm-cov
  rejects rlibs due to the leading lib.rmeta member); resolve
  llvm-cxxfilt via runfiles (demangles Rust v0/legacy and C++ symbols)
- justify.py: scan .rs files for COV_JUSTIFIED markers; skip bazel-*
  convenience symlinks
- quality/coverage/BUILD: add the Rust production crates to
  coverage_scope (mocks/test_support excluded); crate visibilities
  widened by //quality/coverage:__pkg__ only

Known limitations:
- com-api-concept-test and com-api-runtime-lola-tests are tagged
  'manual' and are skipped by wildcard coverage runs; their crates
  report as 0% baseline unless the tests are invoked explicitly
- lib.rs/error.rs/reloc.rs-style files without executable code carry no
  coverage regions and do not appear in reports (same as C++ headers)
justify.py validates 'platforms' as mandatory (as documented in
llvm_cov/README.md), but the three gateway-* entries were added without
it. Because generate_coverage_html.sh tolerates justify failures, the
justification and effective-coverage processing has been silently
skipped since then. With the field added, the 8 justified lines resolve
again.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the unified llvm-cov coverage pipeline to include Rust crates (in addition to C++), by updating the Rust toolchain configuration, broadening the coverage scope to Rust sources/binaries, and enhancing the merger/reporter/justification tooling so Rust artifacts and paths are handled correctly.

Changes:

  • Enable Rust coverage instrumentation under bazel coverage via Ferrocene/toolchain updates and Rust-specific coverage.bazelrc flags.
  • Extend coverage scoping and collection to Rust binaries/crates (allowlist + baseline objects), and update merger/reporter to handle Rust-specific profdata/object discovery and path normalization.
  • Update justification scanning to include .rs sources and avoid descending into Bazel output symlinks.

Reviewed changes

Copilot reviewed 5 out of 14 changed files in this pull request and generated no comments.

Show a summary per file
File Description
score/mw/com/impl/rust/com-api/com-api-runtime-lola/BUILD Widens visibility to allow //quality/coverage to include this crate in coverage scoping.
score/mw/com/impl/rust/com-api/com-api-ffi-lola/BUILD Widens visibility for coverage scoping access.
score/mw/com/impl/rust/com-api/com-api-concept/BUILD Widens visibility for coverage scoping access.
score/mw/com/impl/plumbing/rust/BUILD Widens visibility for coverage scoping access to plumbing Rust crates.
quality/coverage/llvm_cov/reporter.py Adds Rust path normalization, rlib baseline expansion, and runfiles-based llvm-cxxfilt resolution.
quality/coverage/llvm_cov/merger.py Resolves llvm-profdata for Rust runs and detects Rust test executables from the manifest.
quality/coverage/llvm_cov/BUILD Adds llvm-cxxfilt as a data dependency for demangling support.
quality/coverage/justify.py Scans .rs files for justification markers and skips Bazel convenience symlink trees.
quality/coverage/coverage.bazelrc Adds Rust-specific rustc flags needed for usable coverage output (incl. continuous-mode profiling).
quality/coverage/coverage_scope.bzl Extends scope collection to handle rust_binary via CrateInfo and removes debug prints.
quality/coverage/coverage_justifications.yaml Adds explicit platforms fields to select justifications.
quality/coverage/BUILD Explicitly adds Rust production crates to the overall coverage scope.
MODULE.bazel.lock Updates module lock entries for the new Rust toolchain + Ferrocene coverage tools.
MODULE.bazel Bumps score_toolchains_rust and updates Ferrocene coverage-tools URL/SHA to a newer release.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@castler castler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great contribution. Thanks! Some comments from my side.

Comment thread quality/coverage/llvm_cov/merger.py Outdated
Comment thread quality/coverage/BUILD Outdated
Comment thread quality/coverage/coverage.bazelrc Outdated
Comment thread score/mw/com/impl/plumbing/rust/BUILD Outdated
Comment thread quality/coverage/llvm_cov/reporter.py Outdated
- coverage_scope: reference only the public Rust API crate
  (//score/mw/com/impl/rust/com-api/com-api); the scope aspect reaches
  all Rust production crates through its transitive deps. The resulting
  allowlist is identical (17 .rs files). Revert the visibility widening
  in the four crate BUILD files - no longer needed.
- coverage.bazelrc: use the list-typed extra_rustc_flags setting
  (replace semantics) instead of the repeatable extra_rustc_flag, and
  reset it to empty in the qnx_x86_64 section so the gcov-based QNX
  pipeline does not carry llvm covmap instrumentation options.
- merger.py: find_llvm_profdata() terminates directly with a detailed
  error (checked env vars and remedy) instead of returning empty and
  failing later in a subprocess call.
- reporter.py: find_cxxfilt() terminates with an error when the
  demangler is missing - it is a declared data dependency, so absence
  indicates a broken setup.

Validated: rust coverage run green (5 tests, 11 .rs files in LCOV,
rlib baseline expansion working); QNX config parses with the flag
reset applied.
Copilot AI review requested due to automatic review settings July 27, 2026 06:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 10 changed files in this pull request and generated no new comments.

@dcalavrezo-qorix
dcalavrezo-qorix requested a review from castler July 27, 2026 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

3 participants