Skip to content

Fix join filter pushdown - #2249

Draft
ianton-ru wants to merge 3 commits into
antalya-26.6from
fix/join-filter-pushdown-through-rename
Draft

Fix join filter pushdown#2249
ianton-ru wants to merge 3 commits into
antalya-26.6from
fix/join-filter-pushdown-through-rename

Conversation

@ianton-ru

@ianton-ru ianton-ru commented Aug 21, 2026

Copy link
Copy Markdown

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix join filter pushdown

Documentation entry for user-facing changes

Solved #2245

Push left-only JOIN filters when column names do not match the join header

A left-only WHERE on count() of SELECT * … JOIN was not pushed under the JOIN. The filtered column often disappeared from the JOIN output (unused-column removal after count() of SELECT *), while the Filter DAG still referenced it — sometimes under an identifier name such as __table1.a rather than a. get_available_columns_for_filter required the name to appear in the JOIN header, so splitActionsForJOINFilterPushDown never saw it.

That is not Iceberg-specific. On MergeTree the same shape skipped Prewhere / index analysis on the left read. The filter still ran after the JOIN, so the result was correct but the left table was scanned without the predicate.

JoinStepLogical can also alias a side input (a) to a JOIN-output / filter name (__table1.a). Pushdown matches filter inputs against the available-column list, then fix_predicate_for_join_logical_step remaps aliases back to input names. The output alias has to be listed or the split never runs.

JOIN filter pushdown

In tryPushDownOverJoinStep:

  • A side column stays eligible if the Filter DAG still names it, even when it is missing from the JOIN output.
  • For JoinStepLogical, output actions that fromLeft() / fromRight() are added to that list (including aliases). The existing split and remap then push the predicate under the JOIN.

Covered by 04673_join_filter_pushdown_count_subquery.sql: MergeTree left + Memory right, count() of SELECT * … LEFT JOIN … WHERE foo.a < 40 (and the same with an extra (SELECT * FROM t_left) AS foo wrap). EXPLAIN actions = 1 must contain Prewhere.

icebergCluster file listing

icebergCluster (IStorageCluster) lists files on the initiator. The planner wraps the left cluster table so remotes do not get the JOIN (SELECT cols FROM icebergCluster). That wrap had no WHERE, so initiator listing stayed unfiltered even after JOIN pushdown.

Left-only WHERE / PREWHERE is copied onto the wrap with removeExpressionsThatDoNotDependOnTableIdentifiers (same helper as IStorageCluster::updateQueryWithJoinToSendIfNeeded). Wrap planning runs collectFiltersForAnalysis. Listing-only filters are attached on the wrap source without adding a FilterStep that would drop unused wrap columns.

Covered by test_cluster_join_filter_minmax_pruning.py: IcebergMinMaxIndexPrunedFiles for icebergS3Cluster with a plain WHERE, with JOIN … WHERE, and with outer count() of SELECT * … JOIN … WHERE.

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

ianton-ru and others added 3 commits August 21, 2026 17:52
…in header

Unused-column removal and `JoinStepLogical` aliases can hide a one-sided `WHERE` from `get_available_columns_for_filter`. Include those names so existing split and remap can push the predicate under the JOIN.

Co-authored-by: Cursor <cursoragent@cursor.com>
…r can prune files

Initiator listing runs on the wrap subquery (`SELECT cols FROM icebergCluster`), which previously had no WHERE. A left-only predicate on `count()` of `SELECT * … JOIN` never reached min/max file listing.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the duplicated WHERE walker and the PK-walk-through-JOIN remapping. Wrap listing still uses collectFiltersForAnalysis and tryAddClusterWrapFilter.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [4e1b754]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant