[core] Persist row ID reassignment plans and mark snapshots - #10008
Draft
leaves12138 wants to merge 2 commits into
Draft
leaves12138 wants to merge 2 commits into
leaves12138 wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Row-ID reassignment currently publishes a metadata-only OVERWRITE snapshot without a durable record of the mapping it applied. Extract the existing
AssignmentintoDataEvolutionRowIdAssignmentand add file serialization directly to it. The same assignment rewrites the manifests and is persisted before each commit attempt; there is no separate persisted-plan model.Record
row-id-reassign.planin snapshot properties, referencing a versioned, checksummed file undermanifest/. Persist the assigned row-ID bounds and absolute mappings by partition. Snapshot IDs and the manifest rewrite list remain in the commit context and are not serialized.Keep the marker local to the reassignment commit: ordinary manifest replacement, manifest compaction, and rollback must not inherit it. Include plan files in snapshot/tag retention and orphan cleanup, and remove plans from definitively rejected commit attempts. This PR adds persistence and readback only; compact conflict rebasing will follow separately.
Tests
Passed 78 tests on dev2 with Java 11, using the normal Maven profile (without
fast-build):