Skip to content

check --repair: re-read only the packs the repair wrote, refs #8466 - #10379

Draft
mr-raj12 wants to merge 4 commits into
borgbackup:masterfrom
mr-raj12:check-finish-narrow-walk-8466
Draft

mr-raj12 wants to merge 4 commits into
borgbackup:masterfrom
mr-raj12:check-finish-narrow-walk-8466

Conversation

@mr-raj12

@mr-raj12 mr-raj12 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Description

Refs #8466, item 2 of #10318. Based on #10368, only the last commit is new.

With #10368, the checker and the repository share one chunk index, and put() / delete() update it for every pack they write. So ArchiveChecker.finish() no longer rebuilds the index from all packs after a repair that changed packs. It re-reads only the packs the repair wrote.

  • ArchiveChecker records the packs repair writes in written_packs: from the put() and flush() results, and from delete(), which now returns compact_pack()'s (new_pack_id, dropped_bytes). A pack rewritten again is replaced by its new pack.
  • create_archive_entry() flushes before Archives.create(), which flushes and discards the results.
  • verify_written_packs() walks each written pack with a validator and compares it with the index:
    • an index entry the pack does not hold: removed from the index
    • an object whose chunk id is not indexed: added to the index
    • a written pack that is missing: its index entries are removed
    • each of these is a finding
    • an object whose chunk id is already indexed (a superseded duplicate) is not a finding
  • If check() built the index without a validator (the key could not be read), finish() still rebuilds the index from all packs with a validator.
  • finish() drops the index in one place for both paths (invalidate_chunk_index() and self.chunks = None), before the full rebuild.
  • Repository.flush() returns the pack writer's flush results.

Tests (check_cmd_test.py):

  • finish() reads only the rewritten pack, the packs put() wrote, and the pack its own flush() stores.
  • Without the key, finish() rebuilds from all packs.
  • A defect chunk alone in its pack: delete() drops the pack and finish() re-reads no pack.
  • A rewritten pack holding an object no index entry names, a superseded duplicate, a wrong index entry, and a missing written pack.
  • test_check_holds_a_single_chunk_index: a case for the full rebuild in finish(), asserting the checker's index is freed first.

Tests (repository_test.py): flush() returns the stored objects, or None with nothing buffered or no pack writer.

Full test suite: 3093 passed, 1142 skipped.

Checklist

  • PR is against master (or maintenance branch if only applicable there)
  • New code has tests and docs where appropriate
  • Tests pass (run tox or the relevant test subset)
  • Commit messages are clean and reference related issues

…orgbackup#10364

The repository uses the index ArchiveChecker builds instead of loading or building a second one.
A check without --repair uses the index the repository check loaded. Tighten the comments.
Mark the index invalid before verify_data() deletes, flush before using the index, never store the checker's index at close().
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.94737% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 88.14%. Comparing base (3b905e0) to head (74fcb2b).
⚠️ Report is 35 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/borg/archive.py 98.90% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10379      +/-   ##
==========================================
+ Coverage   88.12%   88.14%   +0.02%     
==========================================
  Files         103      103              
  Lines       18928    18910      -18     
  Branches     2923     2928       +5     
==========================================
- Hits        16681    16669      -12     
- Misses       1556     1557       +1     
+ Partials      691      684       -7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

…kup#8466

finish() validates the written packs against the shared index instead of rebuilding it from all packs.
@mr-raj12
mr-raj12 force-pushed the check-finish-narrow-walk-8466 branch from 0a6e74a to 74fcb2b Compare September 16, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant