SS-322: prune keys referencing excluded columns during PG purification#37423
Merged
patrickwwbutler merged 2 commits intoJul 6, 2026
Merged
Conversation
Purification removed an excluded column from the stored table descriptor but left any key/constraint naming that column untouched. If the column (and its constraint) was later dropped upstream, determine_compatibility saw the stale key vanish from the upstream key set and halted the source as an incompatible schema change, even though the column was never ingested in the first place. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
martykulma
approved these changes
Jul 2, 2026
martykulma
left a comment
Contributor
There was a problem hiding this comment.
I would add a couple tests, but otherwise looks good!
- composite primary keys (PRIMARY KEY (A,B)) with A excluded
- multiple keys (PRIMARY KEY (A), UNIQUE (B)) with A excluded
Addresses marty's PR review comment: cover a composite PRIMARY KEY with one of its columns excluded (the whole key must be dropped, not left partially referencing a pruned column), and a table with independent PRIMARY KEY and UNIQUE constraints where only the PRIMARY KEY's column is excluded (only that key should be dropped). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Author
|
Added both test cases: a composite PRIMARY KEY |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
determine_compatibility's keys check saw the stale key vanish from the upstream key set and halted the source with "incompatible schema change", even though the column was never ingested.purify_source_exports, also drop any key whose columns reference an excluded column, so the descriptor doesn't carry a key that can never be re-verified once the excluded column disappears upstream.Test plan
test/pg-cdc/exclude-columns-key.tdcovering bothverify_schemacall sites (replication and snapshot) for a table that excludes its primary key column and later has that column dropped upstream.bin/mzcompose --find pg-cdc run cdc exclude-columns-key.tdpasses, including the post-fix Materialize restart sanity check.cargo check -p mz-sqlandbin/fmtare clean.🤖 Generated with Claude Code