Skip to content

Commit 1d04cd8

Browse files
zhengruifenghuangxiaopingRD
authored andcommitted
[SPARK-54332][PYTHON][CONNECT] No need to attach PlanId in grouping column names in rollup/cube/groupingSets
### What changes were proposed in this pull request? A follow up of apache#52933, avoiding attaching plan id of column names in rollup/cube/groupingSets ### Why are the changes needed? to be consistent with classic: https://github.com/apache/spark/blob/e75ca577923f9f465eb06b4df814c00143fa41ea/sql/api/src/main/scala/org/apache/spark/sql/Dataset.scala#L1415-L1416 https://github.com/apache/spark/blob/e75ca577923f9f465eb06b4df814c00143fa41ea/sql/api/src/main/scala/org/apache/spark/sql/Dataset.scala#L1366-L1368 ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? ci ### Was this patch authored or co-authored using generative AI tooling? no Closes apache#53033 from zhengruifeng/no_plan_id_cube. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent f5d462f commit 1d04cd8

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

python/pyspark/sql/connect/dataframe.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -619,7 +619,7 @@ def rollup(self, *cols: "ColumnOrNameOrOrdinal") -> "GroupedData": # type: igno
619619
if isinstance(c, Column):
620620
_cols.append(c)
621621
elif isinstance(c, str):
622-
_cols.append(self[c])
622+
_cols.append(F.col(c))
623623
elif isinstance(c, int) and not isinstance(c, bool):
624624
if c < 1:
625625
raise PySparkIndexError(
@@ -649,7 +649,7 @@ def cube(self, *cols: "ColumnOrName") -> "GroupedData": # type: ignore[misc]
649649
if isinstance(c, Column):
650650
_cols.append(c)
651651
elif isinstance(c, str):
652-
_cols.append(self[c])
652+
_cols.append(F.col(c))
653653
elif isinstance(c, int) and not isinstance(c, bool):
654654
if c < 1:
655655
raise PySparkIndexError(
@@ -675,7 +675,7 @@ def groupingSets(
675675
if isinstance(c, Column):
676676
gset.append(c)
677677
elif isinstance(c, str):
678-
gset.append(self[c])
678+
gset.append(F.col(c))
679679
else:
680680
raise PySparkTypeError(
681681
errorClass="NOT_COLUMN_OR_STR",
@@ -691,7 +691,7 @@ def groupingSets(
691691
if isinstance(c, Column):
692692
gcols.append(c)
693693
elif isinstance(c, str):
694-
gcols.append(self[c])
694+
gcols.append(F.col(c))
695695
else:
696696
raise PySparkTypeError(
697697
errorClass="NOT_COLUMN_OR_STR",

0 commit comments

Comments
 (0)