Skip to content

Add resolved-dependency override mechanism for DR-008 - #278

Draft
Subramanian-K812 wants to merge 7 commits into
eclipse-score:mainfrom
qorix-group:Subramanian-K812_resolve_override_mechanism
Draft

Add resolved-dependency override mechanism for DR-008#278
Subramanian-K812 wants to merge 7 commits into
eclipse-score:mainfrom
qorix-group:Subramanian-K812_resolve_override_mechanism

Conversation

@Subramanian-K812

Copy link
Copy Markdown
Contributor

DR-008: resolved-dependency resolve + override mechanism (PR 1 of 2)

Part of #264. This is the first of two PRs that split the original DR-008 Option 4
change. This PR adds only the mechanism; the follow-up
PR wires it into the test workflow.

What this PR does

Adds ResolvedDependencies — the mechanism DR-008 Option 4 needs to (1) resolve the full
integrated dependency set at the reference_integration root and (2) override those
resolved versions directly into an individual module.

  • Resolve (root): merges the override directives ref_int actually declares
    (MODULE.bazel + bazel_common/*.MODULE.bazel) with the post-MVS registry versions
    from bazel mod graph --output=json, and serializes the result to a single
    resolved_versions.json manifest (--export / from_mod_graph / to_file).
  • Override (into a module): scans a module's MODULE.bazel for every declared
    bazel_dep and appends a git_override / single_version_override for each one we
    have a resolved version for, so the module builds/tests against the resolved set
    (scan / overwrite). Injection is append-only and idempotent (marked with begin/end
    comments), always skips the module-under-test itself (the root is never overridden),
    always overwrites any dependency the module already declares an override for (ref_int's
    resolved version always wins), and is never committed back to module sources. A declared
    dependency with no entry in the resolved set is left to resolve on its own and logs a
    warning — this is expected to be effectively impossible once the resolved set comes from
    the full bazel mod graph (a superset of any module's own graph).

Files

File Change
scripts/known_good/resolved_dependencies.py New — the ResolvedDependencies mechanism (resolve + override) + CLI
scripts/known_good/tests/test_resolved_dependencies.py New — 18 self-contained unit tests (no cloned repos / Bazel needed)
scripts/known_good/BUILD Newpy_library :known_good, py_binary :resolve_deps (Bazel-runnable), and score_py_pytest :known_good_tests so the unit tests run under Bazel
scripts/tooling/BUILD Modifiedalias :resolve_deps exposing it as bazel run //scripts/tooling:resolve_deps

Follow-up (PR 2)

The two-stage test_and_docs.yml (Stage 1 resolves + exports the manifest; Stage 2 checks
out each module, overrides deps via this mechanism, and runs its UT + coverage; then
aggregates), the module-context quality_runners.py, the dependency pin bumps, and the
associated config all land in the follow-up PR, which is stacked on this one.

@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_resolve_override_mechanism branch from 6d78316 to fbc58a3 Compare July 3, 2026 08:31
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

Comment thread scripts/tooling/BUILD

# Alias: expose resolve_deps under //scripts/tooling for `bazel run //scripts/tooling:resolve_deps`.
alias(
name = "resolve_deps",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this alias? This is usually for backwards-compatibility but since this is new functionality it should not be necessary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alias under //scripts/tooling provides reachability alongside other entry points. The implementation itself remains in //scripts/known_good since that’s where its models and tests live. The pipeline continues to call //scripts/known_good:resolve_deps directly; the alias simply exposes it without duplicating the target.

"""Unit tests for ResolvedDependencies (DR-008 Option 4 dependency injection).

Self-contained: builds the resolved set from a temporary known_good.json and
overwrites a temporary module MODULE.bazel — no cloned repos or Bazel required.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this self-contained aspect. It allows to use this in an hermetic way.

However, the tradeoff is that we cannot parallelize the followup actions (build, test, docs, etc) via GitHub actions, can we?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, they can still run in parallel. Stage 1 resolves once and uploads resolved_versions.json as an artifact; each Stage 2 module job downloads that same artifact and does its own local override + build/test independently, still as a normal parallel matrix. The hermetic is only about the unit tests not needing real clones/bazel to validate scan()/overwrite() — it doesn't force any job serialization in the actual pipeline.

@Subramanian-K812
Subramanian-K812 force-pushed the Subramanian-K812_resolve_override_mechanism branch from e32b107 to cff77c1 Compare July 24, 2026 06:56

@OliverHeilwagen OliverHeilwagen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix transitive dependency pinning gap

Comment thread scripts/known_good/BUILD
)

# Runnable binary for the resolve + inject workflow.
# Stage 1 (export): bazel run //scripts/known_good:resolve_deps -- \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please extend description for Stage 1.
bazel mod graph --output json > graph.json is required as prerequisite.

In the script please make use of BUILD_WORKING_DIRECTORY or BUILD_WORKSPACE_DIRECTORY environment variable so the graph.json file can be found without requiring to be in data = ["..."]

Comment thread scripts/known_good/BUILD
# Runnable binary for the resolve + inject workflow.
# Stage 1 (export): bazel run //scripts/known_good:resolve_deps -- \
# --mod-graph graph.json --export artifacts/resolved_versions.json
# Stage 2 (inject): bazel run //scripts/known_good:resolve_deps -- \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please keep Stage 1 and Stage 2 in sync.
--resolved-deps artifacts/ or _resolved_deps in both.

import sys
from pathlib import Path

_HERE = Path(__file__).resolve().parent

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make use of BUILD_WORKING_DIRECTORY or BUILD_WORKSPACE_DIRECTORY environment variable instead.

version = _field(body, "version")
if name and version:
modules.append(Module(name=name, hash="", repo="", version=version))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add support for multiple_version_override .

for m in re.finditer(r'(archive_override|local_path_override)\(\s*module_name\s*=\s*"([^"]+)"', text):
unrepresentable.append(f"{m.group(2)} ({m.group(1)})")

graph = json.loads(Path(mod_graph_json).read_text())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check file existence before using it.

Comment on lines +481 to +482
override_files = [repo_root / "MODULE.bazel", *sorted((repo_root / "bazel_common").glob("*.MODULE.bazel"))]
override_files = [f for f in override_files if f.is_file()]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
override_files = [repo_root / "MODULE.bazel", *sorted((repo_root / "bazel_common").glob("*.MODULE.bazel"))]
override_files = [f for f in override_files if f.is_file()]
override_files = [
f for f in [repo_root / "MODULE.bazel", *sorted((repo_root / "bazel_common").glob("*.MODULE.bazel"))]
if f.is_file()
]

Comment on lines +493 to +496
if args.resolved_deps:
resolved = ResolvedDependencies.from_resolved_artifact(args.resolved_deps)
else:
resolved = ResolvedDependencies.from_known_good(args.known_good_path)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transitive dependency pinning gap

overwrite() only injects overrides for modules the module-under-test directly declares
via bazel_dep. Transitive dependencies silently fall through to MVS resolution, which may
pick a different version than what ref_int validated against.

Example:

ref_int (resolved: flatbuffers @ 25.12.19)
    └── communication (module under test)
            └── bazel_dep(name = "score_baselibs")   ← override injected ✅
                    └── flatbuffers            ← transitive, NO override injected ❌
                                                 MVS picks its own version

Fix: For every module in the resolved set that is not directly declared by the
module-under-test, inject both a bazel_dep stub and the override into the injection block:

# brings the transitive dep into the root graph so the override is valid
bazel_dep(name = "flatbuffers", version = "0.0.0")
git_override(module_name = "flatbuffers", commit = "...", remote = "...")

This also means from_known_good is insufficient for the inject mode, as it only carries
first-party score modules and has no transitive registry versions; --resolved-deps (from the
Stage-1 manifest) must be the required source for Stage 2 injection.

In Stage-1 we should also takeover and store the given graph.json as it is needed for Stage-2 to be aware of the necessary transitive overrides for a given MODULE.bazel file. Identification can be done via regex for module(name = "...").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants