Skip to content

SS-322: prune keys referencing excluded columns during PG purification#37423

Merged
patrickwwbutler merged 2 commits into
MaterializeInc:mainfrom
patrickwwbutler:patrick/ss-322
Jul 6, 2026
Merged

SS-322: prune keys referencing excluded columns during PG purification#37423
patrickwwbutler merged 2 commits into
MaterializeInc:mainfrom
patrickwwbutler:patrick/ss-322

Conversation

@patrickwwbutler

Copy link
Copy Markdown
Contributor

Summary

  • Postgres source purification pruned an excluded column from the stored table descriptor but left any key/constraint naming that column untouched.
  • If the excluded column (and its constraint) was later dropped upstream, determine_compatibility's keys check saw the stale key vanish from the upstream key set and halted the source with "incompatible schema change", even though the column was never ingested.
  • Fix: when pruning excluded columns in purify_source_exports, also drop any key whose columns reference an excluded column, so the descriptor doesn't carry a key that can never be re-verified once the excluded column disappears upstream.

Test plan

  • Added a regression test to test/pg-cdc/exclude-columns-key.td covering both verify_schema call sites (replication and snapshot) for a table that excludes its primary key column and later has that column dropped upstream.
  • bin/mzcompose --find pg-cdc run cdc exclude-columns-key.td passes, including the post-fix Materialize restart sanity check.
  • cargo check -p mz-sql and bin/fmt are clean.

🤖 Generated with Claude Code

Purification removed an excluded column from the stored table descriptor
but left any key/constraint naming that column untouched. If the column
(and its constraint) was later dropped upstream, determine_compatibility
saw the stale key vanish from the upstream key set and halted the source
as an incompatible schema change, even though the column was never
ingested in the first place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@patrickwwbutler patrickwwbutler requested a review from a team as a code owner July 2, 2026 17:46
@patrickwwbutler patrickwwbutler requested a review from a team July 2, 2026 17:47

@martykulma martykulma left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a couple tests, but otherwise looks good!

  • composite primary keys (PRIMARY KEY (A,B)) with A excluded
  • multiple keys (PRIMARY KEY (A), UNIQUE (B)) with A excluded

Addresses marty's PR review comment: cover a composite PRIMARY KEY with
one of its columns excluded (the whole key must be dropped, not left
partially referencing a pruned column), and a table with independent
PRIMARY KEY and UNIQUE constraints where only the PRIMARY KEY's column
is excluded (only that key should be dropped).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@patrickwwbutler

Copy link
Copy Markdown
Contributor Author

Added both test cases: a composite PRIMARY KEY (id, sub_id) with id excluded (verifies the whole key is dropped, not left partially referencing the pruned column), and a table with independent PRIMARY KEY (id) + UNIQUE (u) with id excluded (verifies only the key naming id is dropped, leaving UNIQUE (u) intact).

@patrickwwbutler patrickwwbutler enabled auto-merge (squash) July 6, 2026 20:47
@patrickwwbutler patrickwwbutler merged commit 0543b34 into MaterializeInc:main Jul 6, 2026
117 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants