fix: Scale semi/anti-join column stats by estimated row count#22762
Open
neilconway wants to merge 2 commits into
Open
fix: Scale semi/anti-join column stats by estimated row count#22762neilconway wants to merge 2 commits into
neilconway wants to merge 2 commits into
Conversation
github-actions
Bot
added
core
Contributor
Author
|
FYI @asolimando |
asolimando
approved these changes
Jun 4, 2026
Member
asolimando
left a comment
asolimando
left a comment
There was a problem hiding this comment.
LGTM, taking into consideration NullEquality makes the upper bound tighter and estimation more accurate
| /// "filter selectivity analysis"). | ||
| /// - Column statistics for inner/outer joins are simply combined from inputs | ||
| /// without adjusting for join selectivity (acknowledged in the code as | ||
| /// needing "filter selectivity analysis"). |
Member
There was a problem hiding this comment.
Q: is there an issue filed already? It's easier to discover pending issues in GitHub rather than code comments
| .enumerate() | ||
| .map(|(idx, stats)| { | ||
| let mut stats = stats.to_inexact(); | ||
| stats.null_count = if join_key_indices.contains(&idx) { |
Member
There was a problem hiding this comment.
Nit: Not a big problem as there won't be many items, but maybe we can use a HashSet here as it seems we are just using containment test.
Contributor
Author
There was a problem hiding this comment.
HashSet would make the set-containment usage more clear, but since it will be small in practice, I'm inclined to leave this as-is.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
This PR makes several related improvements/fixes to the stats code for semi- and anti-joins:
total_byte_sizefor semi/anti-join results, based on summing per-columnbyte_size, instead of always emittingAbsent. We still emit absent for other join types and if any of the per-columnbyte_sizevalues areAbsentNullEqualitysemantics, and use those for stats: underNullEqualsNothing, null join keys will never match (so we can returnExact(0)), whereas underNullEqualsNullwe consider nulls just like any other value.What changes are included in this PR?
Are these changes tested?
Yes; new tests added.
Are there any user-facing changes?
Some queries might get different plans.