[xudong963]

Which issue does this PR close?

• No existing issue. Found while looking at optimizer time for queries joining many tables;

Rationale for this change

EliminateCrossJoin is quadratic (or worse) in the number of joined inputs, both for comma joins (FROM t0, t1, ..., tn WHERE t0.a = t1.a AND ...) and for explicit inner join chains (t0 JOIN t1 ON ... JOIN t2 ON ...). With 128 tables the rule alone takes ~0.6-0.7 s.

After flattening the join tree and reassembling it, the rule calls rewrite_children on the reassembled tree. The left child is the reassembled chain minus one input, so the rule flattens and rebuilds it again, and so on at every nesting level. Each rebuild also checks candidate join keys against the accumulated left schema, which grows with the number of inputs.

What changes are included in this PR?

• EliminateCrossJoin::rewrite: rewrite each flattened input once, before reassembling the joins, instead of calling rewrite_children on the reassembled tree. rewrite preserves the schema of its input, which is all that find_inner_join looks at, so the result is unchanged. Uncorrelated subqueries in the join keys of each reassembled join (previously visited by rewrite_children via map_uncorrelated_subqueries) are now rewritten explicitly.
• find_valid_equijoin_key_pair: check the narrower of the two schemas first. Looking up a column scans the schema, so a candidate key that does not belong to the join inputs is then usually rejected without scanning the wider one. For EliminateCrossJoin, the wider schema is the accumulated left input of the left-deep join tree, while most candidate keys don't reference the (narrower) right input at all. Other callers (ExtractEquijoinPredicate, LogicalPlanBuilder joins) may instead see a narrow left and a wide right input, e.g. t JOIN (t1 JOIN t2 ...), which is why the order depends on the schema widths rather than always checking the right schema first. check_all_columns_from_schema is side-effect free and never returns an error, so reordering the checks does not change the result.
• New eliminate_cross_join criterion benchmark (comma joins and inner join chains over 8-128 tables), and a find_valid_equijoin_key_pair benchmark covering a wide left and a wide right input.

Benchmark results (Apple M-series laptop, main @ c18e5d9 vs this PR):

cargo bench -p datafusion-optimizer --bench eliminate_cross_join

benchmark	main	this PR	change
comma_join/8	76.09 µs	28.90 µs	-62%
comma_join/32	4.09 ms	313.13 µs	-92%
comma_join/64	45.48 ms	1.24 ms	-97%
comma_join/128	579.64 ms	4.80 ms	-99%
inner_join/8	74.28 µs	26.82 µs	-64%
inner_join/32	4.04 ms	309.11 µs	-92%
inner_join/64	47.05 ms	1.21 ms	-97%
inner_join/128	586.04 ms	4.79 ms	-99%

cargo bench -p datafusion-expr --bench find_valid_equijoin_key_pair (a key that belongs to neither input, against a 3-column schema and a schema of the given width; "right first" is the order used by an earlier revision of this PR):

benchmark	main (left first)	right first	this PR (narrower first)
wide_left/16	136.83 ns	75.68 ns	80.82 ns
wide_left/128	477.35 ns	76.38 ns	80.05 ns
wide_left/1024	2.54 µs	76.67 ns	79.82 ns
wide_right/16	87.70 ns	133.76 ns	84.60 ns
wide_right/128	84.85 ns	477.28 ns	83.84 ns
wide_right/1024	87.73 ns	2.52 µs	86.50 ns

What is the testing strategy for this PR?

• Existing eliminate_cross_join unit tests, plus a new eliminate_cross_join_in_join_key_subquery test covering an uncorrelated subquery (containing an eliminable cross join) that ends up in a join key.
• New find_valid_equijoin_key_pair_inputs_of_different_widths unit test, covering matching, swapped and non-matching keys with both a narrower left and a narrower right input.
• The full sqllogictest suite passes (including the join plans in joins.slt and join.slt.part).
• Locally, I ran a randomized differential test comparing the previous and the new implementation on 3,000 generated plans (nested inner/cross/left joins, ON filters, AND/OR predicates and uncorrelated scalar subqueries in filters and join keys): the output plans, transformed flags and errors were identical. It is not included here because it needs a copy of the old implementation.

Are there any user-facing changes?

No, other than faster planning for queries with many joins. No public API changes.

🤖 Generated with Claude Code