Skip to content

Prune dictionary layouts for missing list values#8778

Open
JSap0914 wants to merge 1 commit into
vortex-data:developfrom
JSap0914:fix/dict-list-contains-pruning-3953
Open

Prune dictionary layouts for missing list values#8778
JSap0914 wants to merge 1 commit into
vortex-data:developfrom
JSap0914:fix/dict-list-contains-pruning-3953

Conversation

@JSap0914

Copy link
Copy Markdown

Rationale for this change

Dictionary layouts currently return the input pruning mask for list_contains, even when evaluating the expression against the dictionary values proves that no encoded row can match. This forces scans to fetch and evaluate the codes unnecessarily.

Fixes #3953

What changes are included in this PR?

  • Evaluate list_contains against the dictionary values during pruning.
  • Return an all-false mask when no dictionary value matches; otherwise preserve the input mask.
  • Verify both matching and non-matching cases and assert that pruning never requests code segments.

What APIs are changed? Are there any user-facing changes?

No public APIs change. Dictionary-backed scans can now skip code reads when membership is impossible.

Validation

  • cargo nextest run -p vortex-layout list_contains_pruning_uses_only_dictionary_values
  • cargo nextest run -p vortex-layout (134 passed)
  • cargo clippy -p vortex-layout --all-targets --all-features -- -D warnings
  • cargo +nightly fmt -p vortex-layout -- --check
  • git diff --check

AI assistance

This change was developed with agentic coding assistance and reviewed and tested by the contributor.

Signed-off-by: JSup <hanjisang0914@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dict Layout to short-circuit fetching codes for list_contains

1 participant