Skip to content

HIVE-29457: HiveSortExchangePullUpConstantsRule doesn't remove consta… #6316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
soumyakanti3578 merged 3 commits into from
Feb 23, 2026
+51 −1
Merged

HIVE-29457: HiveSortExchangePullUpConstantsRule doesn't remove consta… #6316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e59ffa6
HIVE-29457: HiveSortExchangePullUpConstantsRule doesn't remove consta…
soumyakanti3578 Feb 12, 2026
14774e1
address review comments
soumyakanti3578 Feb 19, 2026
4f077f4
Remove unnecessary SORT BY clause
soumyakanti3578 Feb 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion ...c/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortPullUpConstantsRule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
import org.apache.calcite.tools.RelBuilder;
import org.apache.calcite.util.Pair;
import org.apache.calcite.util.mapping.Mappings;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelDistribution;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortExchange;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit;
Expand Down Expand Up @@ -94,9 +95,24 @@ private HiveSortExchangePullUpConstantsRule() {
@Override
protected void buildSort(RelBuilder relBuilder, HiveSortExchange sortNode, Mappings.TargetMapping mapping) {
List<RelFieldCollation> fieldCollations = applyToFieldCollations(sortNode.getCollation(), mapping);
RelDistribution distribution = sortNode.getDistribution().apply(mapping);
RelDistribution distribution = applyToDistribution(sortNode.getDistribution(), mapping);
relBuilder.sortExchange(distribution, RelCollations.of(fieldCollations));
}

private RelDistribution applyToDistribution(
RelDistribution distribution, Mappings.TargetMapping mapping) {
List<Integer> newKeys = new ArrayList<>();
for (int key : distribution.getKeys()) {
final int target = mapping.getTargetOpt(key);
if (target < 0) {
// It is a constant, we can ignore it
continue;
}
newKeys.add(target);
}

return new HiveRelDistribution(distribution.getType(), newKeys);
}
Comment on lines +102 to +115
Copy link
Member

@zabetak zabetak Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very similar to applyToFieldCollations it would be nice to see if we can refactor some of the commons parts together.

Copy link
Contributor Author

@soumyakanti3578 soumyakanti3578 Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried moving the for loop which checks if the key is present in the mapping to a new method. However, this doesn't really simplify applyToFieldCollations as we still need this loop: for (RelFieldCollation fc : relCollation.getFieldCollations()).

Maybe we can revisit this in a follow up.

}


Expand Down
6 changes: 6 additions & 0 deletions ql/src/test/queries/clientpositive/distribution_key_constant_value.q
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
CREATE TABLE test (col1 string, col2 string);

EXPLAIN CBO
SELECT col1, col2 FROM test
WHERE col2 = 'a'
DISTRIBUTE BY col1, col2;
28 changes: 28 additions & 0 deletions ql/src/test/results/clientpositive/llap/distribution_key_constant_value.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
PREHOOK: query: CREATE TABLE test (col1 string, col2 string)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@test
POSTHOOK: query: CREATE TABLE test (col1 string, col2 string)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@test
PREHOOK: query: EXPLAIN CBO
SELECT col1, col2 FROM test
WHERE col2 = 'a'
DISTRIBUTE BY col1, col2
PREHOOK: type: QUERY
PREHOOK: Input: default@test
#### A masked pattern was here ####
POSTHOOK: query: EXPLAIN CBO
SELECT col1, col2 FROM test
WHERE col2 = 'a'
DISTRIBUTE BY col1, col2
POSTHOOK: type: QUERY
POSTHOOK: Input: default@test
#### A masked pattern was here ####
CBO PLAN:
HiveSortExchange(distribution=[hash[0]], collation=[[]])
HiveProject(col1=[$0], col2=[CAST(_UTF-16LE'a':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"):VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
HiveFilter(condition=[=($1, _UTF-16LE'a')])
HiveTableScan(table=[[default, test]], table:alias=[test])