test: extend downstream coverage compatibility checks by AlexanderLanin · Pull Request #47 · eclipse-score/coverage_tool

AlexanderLanin · 2026-10-26T13:14:01Z

Coverage tool changes should be checked against all current S-CORE consumers so integration problems are caught before reaching downstream repositories.

Add compatibility checks for inc_security_crypto, inc_someip_gateway, kyron, logging, persistency, and time alongside baselibs and lifecycle. A shared parametrized pytest target runs one consumer per CI job and retains separate reports. The checks require generated reports with measured coverage, while allowing temporary lockfile updates and leaving consumer-specific coverage thresholds to downstream CI.

Publish concise command summaries and retain full Bazel logs to make downstream failures easier to diagnose. Invalid UTF-8 in consumer output no longer turns a successful build into a test failure.

Fixes #41.

Check six more consumer repositories with one parametrized test target. Keep reports separate for each consumer and require measured coverage without enforcing consumer-specific coverage thresholds. Signed-off-by: Alexander Lanin <Alexander.Lanin@etas.com>

github-actions · 2026-10-26T13:14:59Z

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
2026/10/26 13:51:47 Downloading https://releases.bazel.build/8.6.0/release/bazel-8.6.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: 2d0edf59-394b-408d-80a6-4ffc6567c8dd
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (13 packages loaded, 10 targets configured)

Analyzing: target //:license-check (71 packages loaded, 10 targets configured)

Analyzing: target //:license-check (89 packages loaded, 10 targets configured)

Analyzing: target //:license-check (91 packages loaded, 10 targets configured)

Analyzing: target //:license-check (151 packages loaded, 2342 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3277 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3277 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3277 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3277 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3277 targets configured)

Analyzing: target //:license-check (160 packages loaded, 6521 targets configured)

Analyzing: target //:license-check (160 packages loaded, 6521 targets configured)

Analyzing: target //:license-check (170 packages loaded, 6579 targets configured)

Analyzing: target //:license-check (170 packages loaded, 6579 targets configured)

Analyzing: target //:license-check (173 packages loaded, 8466 targets configured)

Analyzing: target //:license-check (173 packages loaded, 8466 targets configured)

Analyzing: target //:license-check (176 packages loaded, 11921 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11949 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11949 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11949 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11949 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11949 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11949 targets configured)

INFO: Analyzed target //:license-check (179 packages loaded, 12076 targets configured).
[12 / 16] JavaToolchainCompileClasses external/rules_java+/toolchains/platformclasspath_classes; 0s disk-cache, processwrapper-sandbox ... (2 actions running)
[14 / 16] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar; 0s disk-cache, processwrapper-sandbox
INFO: Found 1 target...
Target //tools:license.check.license_check up-to-date:
  bazel-bin/tools/license.check.license_check
  bazel-bin/tools/license.check.license_check.jar
INFO: Elapsed time: 50.495s, Critical Path: 2.46s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/tools/license.check.license_check tools/formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

Copilot

🟡 Changes recommended

Nested Bazel commands still inherit the crypto consumer’s strict lockfile mode, which can block dependency resolution before coverage runs.

1 open issue

Pass lockfile update mode to nested Bazel commands · New

What changed in this PR

Expands downstream compatibility testing from two to eight S-CORE consumers, using a shared test to catch coverage-tool integration failures.

Changes:

Replaces separate consumer tests with one parametrized coverage test.
Runs each consumer in a separate CI job and retains consumer-specific reports.
Removes explicit strict lockfile flags and consumer-specific coverage thresholds while preserving measured-coverage checks.

File	Description
`tests/compatibility/test_lifecycle.py`	Removes the standalone lifecycle test.
`tests/compatibility/test_consumers.py`	Adds shared coverage checks for eight consumers.
`tests/compatibility/test_baselibs.py`	Removes the standalone baselibs test.
`tests/compatibility/BUILD`	Defines the shared parametrized test target.
`tests/compatibility/_consumer_workspace.py`	Separates retained reports by consumer and removes lockfile preparation.
`README.md`	Documents selecting an individual consumer.
`.github/workflows/compatibility_tests.yml`	Expands the CI matrix and updates report paths.

_{Review effort: Balanced}

github-actions · 2026-10-26T13:24:17Z

Documentation preview for this pull request is available at:
pr-47: https://eclipse-score.github.io/coverage_tool/pr-47/

AlexanderLanin · 2026-10-26T13:33:51Z

The time failure exposes an existing breakage in eclipse-score/time main on a fresh checkout. Dependency resolution aborts before any tests run because score_devcontainer v1.11.1 no longer matches its registry checksum:

Registry expects sha256-B9caGwH1Ff/asDuyQBEwE8KRRYIozv4NYwdwChraNwY=.
A fresh download returns sha256-XhiGdP4K7IVUADCqAxURtnK6wpFTt1JsL+Jn9Xl/Md4=; I verified this independently.

Failing compatibility job. The last successful coverage run on time/main also resolved v1.11.1 but restored a repository cache, which likely masks the broken download. The published archive/registry entry needs to be corrected upstream.

Consumer bazelrc files can enforce strict lockfile checks. Override that setting for both nested Bazel commands so the local coverage-tool override can update the temporary workspace lockfile. Signed-off-by: Alexander Lanin <Alexander.Lanin@etas.com>

Copilot

🔵 Needs a closer look

The six new downstream checks need successful integration CI results and human validation of their repository-specific environment requirements.

No open issues 🎉

1 resolved since last review

Pass lockfile update mode to nested Bazel commands

_{Review effort: Balanced}

Decode consumer output with replacement for invalid UTF-8 so successful builds do not fail while their output is read. Retain complete Bazel logs and publish concise command summaries with early errors and the log tail. Signed-off-by: Alexander Lanin <Alexander.Lanin@etas.com>

Copilot

🔵 Needs a closer look

The six new downstream pipelines depend on live repositories and toolchains, so their CI results need human confirmation.

No open issues 🎉

_{Review effort: Balanced}

AlexanderLanin requested review from MaximilianSoerenPollak, dcalavrezo-qorix and nradakovic as code owners October 26, 2026 13:14

Copilot AI balanced review requested due to automatic review settings October 26, 2026 13:14

github-project-automation Bot added this to INF - Infrastructure Community Oct 26, 2026

Copilot started reviewing on behalf of AlexanderLanin October 26, 2026 13:14

View session

Copilot AI reviewed Oct 26, 2026

View reviewed changes

Comment thread tests/compatibility/test_consumers.py Outdated

Copilot AI balanced review requested due to automatic review settings October 26, 2026 13:34

Copilot started reviewing on behalf of AlexanderLanin October 26, 2026 13:35

View session

Copilot AI reviewed Oct 26, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings October 26, 2026 13:51

Copilot started reviewing on behalf of AlexanderLanin October 26, 2026 13:52

View session

Copilot AI reviewed Oct 26, 2026

View reviewed changes

Navigation Menu

Repository navigation

test: extend downstream coverage compatibility checks - #47

test: extend downstream coverage compatibility checks#47
AlexanderLanin wants to merge 3 commits into
eclipse-score:mainfrom
etas-contrib:test/extend-downstream-consumers

AlexanderLanin commented Oct 26, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 26, 2026

Uh oh!

AlexanderLanin commented Oct 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

AlexanderLanin commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

License Check Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot commented Oct 26, 2026

Uh oh!

AlexanderLanin commented Oct 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AlexanderLanin commented Oct 26, 2026 •

edited

Loading

github-actions Bot commented Oct 26, 2026 •

edited

Loading