Skip to content

Dependency compatibility checker - #773

Merged
castler merged 8 commits into
eclipse-score:mainfrom
castler:dependency-compatibility-checker
Jul 27, 2026
Merged

Dependency compatibility checker#773
castler merged 8 commits into
eclipse-score:mainfrom
castler:dependency-compatibility-checker

Conversation

@castler

@castler castler commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@castler castler mentioned this pull request Jul 24, 2026
Closed
@castler
castler force-pushed the dependency-compatibility-checker branch from 6fd45d3 to 5c75493 Compare July 27, 2026 07:27
hoe-jo
hoe-jo reviewed Jul 27, 2026
View reviewed changes
Comment thread .github/workflows/test_dependency_version_combinations.yml

@hoe-jo hoe-jo Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not maintainable / really complex to understand what is done here.
What about move it to a python script which we could also test?

@castler castler Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly is to complex? What exactly you want to extract into a python script?

We want to utilize Github Actions matrix builds, if you test all combinations within one action, it will take ages and not run in a parallel?

Comment thread .github/workflows/test_dependency_version_combinations.yml

@hoe-jo hoe-jo Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I think we should include it in our docs build (valuable addition)

@castler castler Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can at some point, goal of this is to get this kind of checks starting.

castler and others added 2 commits July 27, 2026 10:24
@castler
feat(quality): add config schema for dependency compatibility checker
7d84f99 
Introduce the dependency_compatibility_checker under quality/ with its
configuration model and a default config.

- config_schema.py validates the config: the set of dependencies, the
  candidate versions per dependency, optional patches, and optional
  build flags, then expands them into the cartesian product of version
  combinations to test.
- config.yaml is the source of truth for the matrix: Bazel, rules_cc,
  rules_python and rules_rust versions, yielding the combinations the
  nightly job exercises against the root MODULE.bazel.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castler
feat(quality): add compatibility matrix generation
471ed0c 
generate_matrix.py reads the checker config and emits the list of
dependency version combinations as JSON, suitable for consumption as a
GitHub Actions matrix. Each entry carries the versions, patches and
build flags for one combination.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hoe-jo
hoe-jo reviewed Jul 27, 2026
View reviewed changes
Comment thread quality/dependency_compatibility_checker/module_rewriter.py

@hoe-jo hoe-jo Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Regex stuff is very brittle, either replace by some parser or enhance the test suite, both positive and negative cases, currently I guess a bracket inside a comment would break the tool

@castler
castler force-pushed the dependency-compatibility-checker branch from 5c75493 to 459c025 Compare July 27, 2026 09:34
castler
castler commented Jul 27, 2026
View reviewed changes
Comment thread quality/dependency_compatibility_checker/README.md Outdated
Comment on lines +21 to +22
See the design spec for the full rationale:
`docs/superpowers/specs/2026-07-24-dependency-compatibility-checker-design.md`.

@castler castler Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

Comment thread .github/workflows/test_dependency_version_combinations.yml

@castler castler Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly is to complex? What exactly you want to extract into a python script?

We want to utilize Github Actions matrix builds, if you test all combinations within one action, it will take ages and not run in a parallel?

Comment thread .github/workflows/test_dependency_version_combinations.yml

@castler castler Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can at some point, goal of this is to get this kind of checks starting.

@castler
castler marked this pull request as ready for review July 27, 2026 10:00
@castler
castler added this pull request to the merge queue Jul 27, 2026
Merged via the queue into eclipse-score:main with commit e8abf72 Jul 27, 2026
8 checks passed
@castler
castler deleted the dependency-compatibility-checker branch July 27, 2026 10:22
@github-project-automation github-project-automation Bot moved this from Backlog to Done in COM - Communication FT Jul 27, 2026
castler and others added 6 commits July 27, 2026 12:32
@castler
feat(quality): add MODULE.bazel rewriter
b8f0a6d 
module_rewriter.py rewrites the root MODULE.bazel for a single version
combination by appending single_version_override blocks that pin each
dependency to the version under test (and apply patches when configured).

Overrides are emitted in buildifier-canonical multi-line form and
separated by blank lines so the rewritten file passes the repository's
Starlark formatting checks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castler
feat(quality): add report generation
f0a512f 
generate_report.py aggregates the per-combination build/test results and
renders an HTML report (via report.html.j2) plus a machine-readable JSON
summary. Each combination is classified as green (fully compatible, only
differing build flags allowed), orange (compatible only with patches) or
red (incompatible).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castler
test(quality): add Bazel targets and unit tests for the checker
5798024 
Wire the dependency_compatibility_checker into Bazel with py_binary
targets for the matrix/rewriter/report entry points and py_test targets
for their unit tests. Tests cover config validation and combination
expansion, matrix JSON generation, MODULE.bazel rewriting (with fixtures
for plain, single_version_override and git_override files) and report
rendering/classification.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castler
docs(quality): add dependency compatibility checker README
b26cfd7 
Document the purpose of the checker, the config.yaml format, how to run
the tools locally via Bazel, and how the green/orange/red classification
is derived.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castler
ci: add nightly dependency version combination workflow
9c65f09 
Add a scheduled (nightly) + manually dispatchable workflow that drives
the dependency_compatibility_checker end to end:

- generate-matrix: expands config.yaml into the combination matrix.
- build-and-test: for each combination, rewrites MODULE.bazel, then runs
  bazel mod deps, bazel build //... and bazel test //..., recording the
  outcome and the stage that failed. Per-combination logs are uploaded as
  artifacts to aid debugging.
- report: aggregates the results into the HTML/JSON compatibility report
  and fails if any combination is red.

All bazel invocations pass --lockfile_mode=off because rewriting
MODULE.bazel invalidates the committed lockfile, which would otherwise
fail under the repository's --lockfile_mode=error default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castler
fix(visibility_guard): disable lockfile verification in nested query
459c025 
visibility_guard's parser runs a nested 'bazel query' to enumerate public
targets. Under the compatibility matrix the root MODULE.bazel is rewritten,
so the committed lockfile no longer matches and the query fails (its exit
code was ignored, yielding zero targets and a spurious golden diff).

Pass --lockfile_mode=off to the nested query so it resolves the module
graph regardless of lockfile staleness.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hoe-jo hoe-jo hoe-jo approved these changes
@LittleHuba LittleHuba Awaiting requested review from LittleHuba LittleHuba is a code owner
@crimson11 crimson11 Awaiting requested review from crimson11 crimson11 is a code owner
@bemerybmw bemerybmw Awaiting requested review from bemerybmw bemerybmw is a code owner
@limdor limdor Awaiting requested review from limdor limdor is a code owner

Assignees

No one assigned

Labels

None yet

Projects

COM - Communication FT
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@castler @hoe-jo