Skip to content

feat(git): fully managed Git backend selectable via GITVERSION_GIT_BACKEND - #5084

Open
arturcic wants to merge 15 commits into
GitTools:mainfrom
arturcic:feature/managed-git
Open

feat(git): fully managed Git backend selectable via GITVERSION_GIT_BACKEND#5084
arturcic wants to merge 15 commits into
GitTools:mainfrom
arturcic:feature/managed-git

Conversation

@arturcic

Copy link
Copy Markdown
Member

Follow-up to #5075 (closed).

Implements the managed Git backend for the v7.0 dual-backend window (#5031, design: docs/design/managed-git-migration.md): a vendored, fully managed reader for all read/history operations paired with the git CLI for mutations and network, selectable at runtime alongside the existing libgit2 backend.

What's in the branch

Layered bottom-up, one commit per layer, each with its tests:

  • Object database — hash-agnostic GitObjectId, zlib/random-access streams, pack + idx v2 readers with delta application, bounded pack memory cache, multi-pack-index, loose objects, SHA-256 detection with a clear failure message (NOTICE attribution for code ported from Nerdbank.GitVersioning ManagedGit, MIT).
  • Parsers — commit (signatures, encoding with Latin-1 fallback), tree (streaming reads), annotated tags.
  • Refs & repository layout — loose/packed refs (peeled, symrefs, .lock skipping), walk-up discovery with .git-file indirection, worktrees/commondir, shallow detection, config parsing, reftable rejection.
  • Revision walker — libgit2-parity time/topo ordering with exact tie-break mechanics, mark-uninteresting limiting, paint-down-to-common merge-base, .git/shallow boundaries grafted parentless.
  • Diff & status — changed-paths tree diff in raw byte order, index v2–v4 reader, gitignore matching, working-tree status matching the libgit2 adapter's uncommitted-changes expression.
  • CLI mutator — plumbing-only parsing, ArgumentList invocation (LC_ALL=C, no terminal prompts), per-invocation auth headers, stderr classification mapped to the existing exceptions (Polly LockedFileException retry preserved).
  • Adapters & selectionManaged* implementations of the Core git abstractions over an immutable session snapshot (invalidated after mutations); GITVERSION_GIT_BACKEND=libgit2|managed resolved once in GitBackendSelector (unset ⇒ libgit2 in v7.0; unknown values fail fast); the resolved backend is logged at startup.

Behavior

  • v7.0 default is unchanged (libgit2) — the managed backend is opt-in for validation; v7.1 flips the default per the release plan.
  • With managed, git on PATH is required only for normalization/dynamic-repo scenarios; plain version calculation on a prepared checkout needs no git binary.

Validation

  • CI unit-test matrix runs every leg against both backends (git_backend dimension) on three OSes — the full integration suites assert exact SemVer strings over complex histories.
  • DualBackendParityTests: deep-equality on ref enumeration, order-sensitive walks, merge-base over commit pairs (criss-cross, equal timestamps), DiffPaths, UncommittedChangesCount.
  • build/parity-corpus.ps1 diffs gitversion /nocache /output json across backends on real-world repositories.

🤖 Generated with Claude Code

@arturcic
arturcic requested a review from asbjornu July 22, 2026 21:32
@arturcic
arturcic force-pushed the feature/managed-git branch 3 times, most recently from 17b3155 to c3d263d Compare July 23, 2026 23:53
@arturcic
arturcic force-pushed the main branch 2 times, most recently from 1ad796b to 6f8b3ab Compare July 24, 2026 10:58
@arturcic
arturcic force-pushed the feature/managed-git branch from c3d263d to 901fb35 Compare July 24, 2026 11:08
@arturcic

arturcic commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

@asbjornu @HHobeck hey guys would you mind to review this PR and actually give it a try? I really need your feedback, which I can address before we merge it for v7

@arturcic
arturcic force-pushed the feature/managed-git branch 3 times, most recently from a9ef63c to b13fd9f Compare July 30, 2026 21:07
@arturcic
arturcic force-pushed the feature/managed-git branch 2 times, most recently from 74c2e3b to ffdc489 Compare July 31, 2026 19:08
Research and phased plan for replacing LibGit2Sharp with a vendored
managed reader plus git CLI writes (GitTools#5031).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@arturcic
arturcic force-pushed the feature/managed-git branch from ffdc489 to 7c63cb1 Compare August 1, 2026 21:04
Copilot AI review requested due to automatic review settings August 1, 2026 21:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new fully managed Git backend (read/history in-process; mutations/network via git CLI) to the legacy src/ GitVersion stack, selectable at runtime via GITVERSION_GIT_BACKEND (defaulting to libgit2 for v7.0), and wires CI/documentation to support dual-backend parity validation.

Changes:

  • Introduces GitVersion.Git.Managed (managed ODB/pack/idx/midx, refs/repo layout, revwalk, tree diff, index/status, git-CLI mutator, Core adapter implementations) plus a dedicated test project.
  • Adds backend selection (GitBackendSelector) and integrates backend choice into CLI/module composition roots and logging.
  • Updates CI pipelines/docs/release notes to run/test both backends and document the new environment switch.

Reviewed changes

Copilot reviewed 117 out of 117 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/GitVersion.slnx Adds managed backend and its tests to the legacy solution.
src/GitVersion.MsBuild.Tests/GitVersion.MsBuild.Tests.csproj References managed backend from MSBuild tests.
src/GitVersion.Git.Managed/StreamExtensions.cs Adds stream helpers used by pack/object parsing.
src/GitVersion.Git.Managed/SafeFileHandleExtensions.cs Adds random-access read helper for pack/idx reads.
src/GitVersion.Git.Managed/Refs/GitReference.cs Defines internal reference model used by ref store/adapter.
src/GitVersion.Git.Managed/RandomAccessStream.cs Provides per-reader positioning over shared file handles.
src/GitVersion.Git.Managed/PublicAPI.Unshipped.txt Declares new public surface for the managed module.
src/GitVersion.Git.Managed/PublicAPI.Shipped.txt Initializes shipped API baseline for the new project.
src/GitVersion.Git.Managed/NOTICE.md Adds third-party attribution for ported components.
src/GitVersion.Git.Managed/History/GitRevisionWalkOptions.cs Defines walk options for managed revwalk.
src/GitVersion.Git.Managed/History/GitRevisionSortStrategies.cs Adds managed sort-strategy flags for revwalk parity.
src/GitVersion.Git.Managed/GitZLibStream.cs Implements zlib-wrapped deflate reader for git object data.
src/GitVersion.Git.Managed/GitVersionManagedGitModule.cs Registers managed backend services into DI.
src/GitVersion.Git.Managed/GitVersion.Git.Managed.csproj Adds the managed backend project and internals visibility.
src/GitVersion.Git.Managed/GitTreeStreamingReader.cs Fast tree scanning for node lookup without full parsing.
src/GitVersion.Git.Managed/GitTreeReader.cs Parses git tree objects into GitTree/GitTreeEntry.
src/GitVersion.Git.Managed/GitTree.cs Adds managed tree model used by diff/status code.
src/GitVersion.Git.Managed/GitTextDecoder.cs Implements git-like UTF-8/Latin-1 fallback decoding.
src/GitVersion.Git.Managed/GitTagReader.cs Parses annotated tag objects.
src/GitVersion.Git.Managed/GitTag.cs Adds annotated tag model.
src/GitVersion.Git.Managed/GitSignature.cs Parses and represents author/committer/tagger signatures.
src/GitVersion.Git.Managed/GitPackReader.cs Reads pack entries, including delta application.
src/GitVersion.Git.Managed/GitPackObjectType.cs Defines pack object type enum.
src/GitVersion.Git.Managed/GitPackMemoryCacheViewStream.cs Adds view streams over cached pack objects.
src/GitVersion.Git.Managed/GitPackMemoryCacheStream.cs Adds decompressed-object caching stream to enable random reads.
src/GitVersion.Git.Managed/GitPackMemoryCache.cs Adds bounded LRU cache for pack objects.
src/GitVersion.Git.Managed/GitPackIndexReader.cs Reads .idx v2 to map object IDs to pack offsets.
src/GitVersion.Git.Managed/GitPackDeltafiedStream.cs Implements delta instruction application stream (naming flagged).
src/GitVersion.Git.Managed/GitPackCache.cs Defines cache abstraction for pack-object caching.
src/GitVersion.Git.Managed/GitPack.cs Wraps pack+idx access and integrates caching.
src/GitVersion.Git.Managed/GitObjectTypes.cs Centralizes canonical object type strings.
src/GitVersion.Git.Managed/GitObjectStream.cs Reads loose objects (type/size header + zlib content).
src/GitVersion.Git.Managed/GitObjectStoreException.cs Defines managed object-store exception type.
src/GitVersion.Git.Managed/GitCommitReader.cs Parses commit objects while deferring expensive decoding.
src/GitVersion.Git.Managed/GitCommit.cs Adds commit model with lazy-decoded fields.
src/GitVersion.Git.Managed/FileHelpers.cs Adds file open helper with IO-failure handling.
src/GitVersion.Git.Managed/Diff/GitTreeDiff.cs Implements tree-vs-tree diff producing libgit2-parity path lists.
src/GitVersion.Git.Managed/DeltaStreamReader.cs Reads pack delta instructions.
src/GitVersion.Git.Managed/DeltaInstruction.cs Defines delta instruction representation.
src/GitVersion.Git.Managed/CommandLine/PullRequestBranchOperations.cs Adds PR-ref normalization logic for CLI mutator path.
src/GitVersion.Git.Managed/CommandLine/GitRemoteReference.cs Adds remote ref record for ls-remote-style output.
src/GitVersion.Git.Managed/CommandLine/GitCliExecutor.cs Implements safe git CLI invocation with env scrubbing/redaction.
src/GitVersion.Git.Managed/Adapter/RepositoryPathResolution.cs Adds path-resolution utilities for managed repo info.
src/GitVersion.Git.Managed/Adapter/ManagedTagCollection.cs Exposes tags via Core abstractions.
src/GitVersion.Git.Managed/Adapter/ManagedTag.cs Implements ITag over managed refs/object store.
src/GitVersion.Git.Managed/Adapter/ManagedRemoteCollection.cs Exposes remotes via Core abstractions (with CLI-backed mutations).
src/GitVersion.Git.Managed/Adapter/ManagedRemote.cs Implements IRemote and refspec exposure for managed backend.
src/GitVersion.Git.Managed/Adapter/ManagedRefSpecCollection.cs Wraps refspec enumeration.
src/GitVersion.Git.Managed/Adapter/ManagedRefSpec.cs Parses refspecs in a libgit2-compatible way.
src/GitVersion.Git.Managed/Adapter/ManagedReferenceCollection.cs Exposes references via Core abstractions, incl. glob enumeration.
src/GitVersion.Git.Managed/Adapter/ManagedReference.cs Implements IReference over managed reference model.
src/GitVersion.Git.Managed/Adapter/ManagedObjectId.cs Implements Core IObjectId over hash-agnostic IDs.
src/GitVersion.Git.Managed/Adapter/ManagedGitRepositoryInfo.cs Implements IGitRepositoryInfo for managed discovery/dynamic repos.
src/GitVersion.Git.Managed/Adapter/ManagedGitRepository.mutating.cs Implements CLI-backed mutating repository operations.
src/GitVersion.Git.Managed/Adapter/ManagedGitRepository.cs Implements managed IGitRepository with session invalidation model.
src/GitVersion.Git.Managed/Adapter/ManagedCommitCollection.cs Implements commit enumeration and filtering with managed revwalk.
src/GitVersion.Git.Managed/Adapter/ManagedCommit.cs Implements ICommit with managed diff-paths integration.
src/GitVersion.Git.Managed/Adapter/ManagedBranchCollection.cs Exposes branches via Core abstractions.
src/GitVersion.Git.Managed/Adapter/ManagedBranch.cs Implements IBranch over managed references.
src/GitVersion.Git.Managed/Adapter/DynamicRepositoryPath.cs Computes/chooses dynamic clone paths for remote URL scenarios.
src/GitVersion.Git.Managed.Tests/TreeTests.cs Adds managed ODB tree parsing/streaming reader tests.
src/GitVersion.Git.Managed.Tests/TempDirectory.cs Adds temp directory helper for managed backend tests.
src/GitVersion.Git.Managed.Tests/TagTests.cs Adds annotated tag parsing tests (loose/packed/nested).
src/GitVersion.Git.Managed.Tests/PullRequestBranchOperationsTests.cs Adds tests for PR-ref normalization candidate selection.
src/GitVersion.Git.Managed.Tests/PackedObjectTests.cs Adds tests for pack reading and deep delta chains.
src/GitVersion.Git.Managed.Tests/MultiPackIndexTests.cs Adds tests for multi-pack-index reading and lookups.
src/GitVersion.Git.Managed.Tests/GitVersion.Git.Managed.Tests.csproj Adds the managed-backend test project definition.
src/GitVersion.Git.Managed.Tests/GitTreeDiffTests.cs Adds libgit2 parity tests for tree diff path listing.
src/GitVersion.Git.Managed.Tests/GitTestRepository.cs Adds deterministic git-CLI-backed repo fixture for managed tests.
src/GitVersion.Git.Managed.Tests/GitRepositoryLayoutTests.cs Adds discovery/worktree/shallow/bare/reftable/sha256 tests.
src/GitVersion.Git.Managed.Tests/GitObjectIdTests.cs Adds tests for hash-agnostic object ID parsing/equality.
src/GitVersion.Git.Managed.Tests/GitIndexTests.cs Adds tests for index v2–v4 parsing and parity with git ls-files.
src/GitVersion.Git.Managed.Tests/GitConfigurationFileTests.cs Adds tests for config parsing/escaping behavior.
src/GitVersion.Git.Managed.Tests/DeltaStreamReaderTests.cs Adds unit tests for delta instruction decoding.
src/GitVersion.Git.Managed.Tests/AssemblyParallelizable.cs Enables fixture-level test parallelization for managed tests.
src/GitVersion.Core/GitVersion.Core.csproj Adds internals visibility for managed backend and its tests.
src/GitVersion.Core/Git/GitBackend.cs Introduces backend enum + environment-based selector.
src/GitVersion.Core.Tests/Helpers/GitVersionCoreTestModule.cs Selects managed vs libgit2 module in Core test composition.
src/GitVersion.Core.Tests/GitVersion.Core.Tests.csproj References managed backend from Core tests.
src/GitVersion.Core.Tests/Core/GitCliMutatorTests.cs Adds tests for CLI mutator behavior and error classification.
src/GitVersion.Core.Tests/Core/GitBackendSelectorTests.cs Adds tests for env-var backend resolution and fail-fast behavior.
src/GitVersion.App/GitVersionExecutor.cs Logs selected backend at startup.
src/GitVersion.App/GitVersion.App.csproj References managed backend from legacy CLI project.
src/GitVersion.App/CliHost.cs Selects backend module at runtime via GitBackendSelector.
src/.run/cli (version).run.xml Adds IDE run config for version command.
src/.run/cli (managed).run.xml Adds IDE run config for managed backend execution.
src/.run/cli (libgit2sharp).run.xml Adds IDE run config for libgit2 backend execution.
new-cli/GitVersion.Common/GitVersion.Common.csproj Adds Microsoft.Extensions.Logging using for source-linked files.
docs/input/docs/migration/v6-to-v7.md Documents backend selection and v7.0/v7.1 default behavior.
build/parity-corpus.md Documents parity corpus tool for real-world backend diffing.
build/common/Utilities/DockerContextExtensions.cs Forwards backend env var into docker runs.
BREAKING_CHANGES.md Documents selectable backend and managed backend behavioral notes.
.github/workflows/ci.yml Ensures CI runs for the managed-git feature branch.
.github/workflows/_unit_tests.yml Adds git_backend matrix dimension and forwards env var/flags.
.github/workflows/_docker.yml Runs docker tests per backend and forwards backend selection.
.github/workflows/_artifacts_windows.yml Runs artifacts tests sequentially for both backends (Windows).
.github/workflows/_artifacts_linux.yml Runs artifacts tests for both backends and forwards env var.
.github/actions/docker-test/action.yml Adds git_backend input and forwards env var into docker tests.

Comment on lines +142 to +146
var parentTree = commit.Parents.Count > 0
? store.GetCommit(commit.Parents[0]).Tree
: (GitObjectId?)null;

var actual = treeDiff.GetChangedPaths(parentTree, commit.Tree);
Comment on lines +5 to +10
/// <summary>
/// Reads data from a deltified object.
/// </summary>
/// <seealso href="https://git-scm.com/docs/pack-format#_deltified_representation"/>
internal sealed class GitPackDeltafiedStream : Stream
{
Comment on lines +131 to +135
this.diffPathsCache.GetOrAdd(commit.ObjectId, _ =>
{
GitObjectId? parentTreeId = commit.FirstParentId is { } parentId ? TryGetCommit(parentId)?.TreeId : null;
return TreeDiff.GetChangedPaths(parentTreeId, commit.TreeId);
});
Comment on lines +41 to +44
if (zlibHeader[0] != 0x78 || (zlibHeader[1] != 0x01 && zlibHeader[1] != 0x9C && zlibHeader[1] != 0x5E && zlibHeader[1] != 0xDA))
{
throw new GitObjectStoreException($"Invalid zlib header {zlibHeader[0]:X2} {zlibHeader[1]:X2}");
}
arturcic and others added 12 commits August 1, 2026 23:51
…primitives

New project (with NOTICE attribution for code ported from Nerdbank.GitVersioning
ManagedGit) and the raw building blocks: hash-agnostic GitObjectId, zlib and
random-access streams, text decoding with Latin-1 fallback, and file helpers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Packfile access with idx v2 lookup, ofs/ref delta stream application, a
bounded pack memory cache with refcounted views, and multi-pack-index support.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author/committer signatures with commit-encoding handling, streaming tree
reads for diff traversal, and annotated-tag target resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Single lookup facade across loose objects, packs, and the multi-pack index,
with SHA-256 repository detection failing fast with a clear message.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Loose and packed refs (peeled entries, symrefs, HEAD, lock-file skipping),
walk-up discovery with .git-file indirection, worktree and commondir
resolution, shallow detection, git config parsing, and reftable/objectformat
rejection with clear messages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Time/topological sorts with exact tie-break mechanics, first-parent walks,
mark-uninteresting limiting, paint-down-to-common merge-base selection, and
shallow boundaries grafted parentless from .git/shallow; unexplained missing
parents fail the walk.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Changed-paths tree diff aligned by raw byte order, .git/index v2-v4 reader
(v4 prefix compression), gitignore matching, and the working-tree status
walk matching the libgit2 adapter's uncommitted-changes expression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plumbing-only parsing, ArgumentList invocation with LC_ALL=C and terminal
prompts disabled, per-invocation auth headers, stderr classification mapped
to existing exceptions (LockedFileException retry preserved), and shared
pull-request tip matching with peeled-entry folding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Managed* adapters implement IGitRepository/IMutatingGitRepository and the
collection interfaces directly over the managed reader and CLI mutator, with
an immutable session snapshot invalidated after mutations. Backend selection
is centralized in GitBackendSelector: GITVERSION_GIT_BACKEND picks libgit2
(default when unset) or managed at every composition root, failing fast on
unrecognized values.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deep-equality parity assertions across both backends on adversarial fixtures,
plus a real-world corpus script diffing gitversion JSON output across
repositories and backends.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the git_backend dimension so every leg runs with libgit2 and managed,
publishing per-backend test summaries and coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GITVERSION_GIT_BACKEND usage, the v7.0 libgit2 default and v7.1 managed flip,
and the git-on-PATH requirement scoped to normalization scenarios.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
arturcic and others added 2 commits August 1, 2026 23:51
Add shared IntelliJ/Rider run configurations to facilitate debugging the CLI using both libgit2sharp and managed git backends.
Exercise both Git backends (libgit2 and managed) within the same job. The
docker smoke test and the Linux artifact tests run the two backends as
parallel steps (background: true + wait-all), each in an ephemeral --rm
container so the runs are independent. The in-process Windows artifact
tests share a single install directory, so their two backends run
sequentially.

Backend selection flows through GITVERSION_GIT_BACKEND: GetDockerRunSettings
forwards it into the test containers; _docker.yml passes it as the
git_backend input to the docker-test action; the Windows tests inherit it
in-process. Docker publishing is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 1, 2026 21:55
@arturcic
arturcic force-pushed the feature/managed-git branch from 7c63cb1 to c155458 Compare August 1, 2026 21:55
@sonarqubecloud

sonarqubecloud Bot commented Aug 1, 2026

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 118 out of 118 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

src/GitVersion.Git.Managed/Adapter/DynamicRepositoryPath.cs:33

  • targetUrl.Split(...)[^1].Replace(".git", string.Empty) removes every occurrence of ".git" (not just a trailing suffix) and also mishandles URLs ending with a slash. This can generate incorrect dynamic-repo directory names (and collisions) for some URLs.
    src/GitVersion.Git.Managed/CommandLine/GitCliExecutor.cs:31
  • The git CLI output/error streams are decoded using the process default encoding. Git may emit UTF-8 (e.g., non-ASCII ref names/paths in stderr), which will be garbled on Windows unless the encodings are explicitly set when redirecting output.

Comment on lines +17 to +21
this.refSpecs = new(
[
.. fetchRefSpecs.Select(IRefSpec (spec) => new ManagedRefSpec(spec, RefSpecDirection.Fetch)),
.. pushRefSpecs.Select(IRefSpec (spec) => new ManagedRefSpec(spec, RefSpecDirection.Push))
]);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants