[FLINK-40410][table-planner] Exclude VARIANT from constant folding - #28991
Open
manner wants to merge 1 commit into
Open
[FLINK-40410][table-planner] Exclude VARIANT from constant folding#28991manner wants to merge 1 commit into
manner wants to merge 1 commit into
Conversation
Collaborator
Comment on lines
+1041
to
+1044
| // Regression test for FLINK-40410: PARSE_JSON(f0) (VARIANT) precedes | ||
| // JSON_STRING(PARSE_JSON(f0)) (STRING) in a constant-folded projection. VARIANT | ||
| // is excluded from folding in ExpressionReducer, like ROW/ARRAY/MAP, and | ||
| // evaluated normally at runtime. |
Contributor
There was a problem hiding this comment.
nit: this comment narrates the bug/ticket story rather than stating the timeless invariant. The test name already establishes it's a regression test, so the JIRA reference and blow-by-blow of what the old bug looked like can move to the commit message. Something tighter reads better long-term, e.g.:
Suggested change
| // Regression test for FLINK-40410: PARSE_JSON(f0) (VARIANT) precedes | |
| // JSON_STRING(PARSE_JSON(f0)) (STRING) in a constant-folded projection. VARIANT | |
| // is excluded from folding in ExpressionReducer, like ROW/ARRAY/MAP, and | |
| // evaluated normally at runtime. | |
| // VARIANT must be excluded from constant folding, like ROW/ARRAY/MAP, | |
| // otherwise a constant-folded projection reads a stale slot for it. |
Contributor
There was a problem hiding this comment.
Actually, I'd just drop the comment
gustavodemorais
left a comment
Contributor
There was a problem hiding this comment.
Added one nit. Thanks for working and fixing this @manner
Comment on lines
+1041
to
+1044
| // Regression test for FLINK-40410: PARSE_JSON(f0) (VARIANT) precedes | ||
| // JSON_STRING(PARSE_JSON(f0)) (STRING) in a constant-folded projection. VARIANT | ||
| // is excluded from folding in ExpressionReducer, like ROW/ARRAY/MAP, and | ||
| // evaluated normally at runtime. |
Contributor
There was a problem hiding this comment.
Actually, I'd just drop the comment
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
This pull request excludes the VARIANT data type from constant folding VARIANT expressions. This previously resulted in a
ClassCastExceptionwhen running a query such as the following, where a constant expression followed a VARIANT expression in the projection.The
ExpressionReducer.scalainflink-table-plannerevaluates constant sub-expressions, so they get computed during compile-time and not during query run-time. It runs in two passes that must agree on which types get a slot in the intermediate (compile-time) result:Brief change log
Verifying this change
This change added tests and can be verified as follows:
JsonFunctionsITCase.javafor VARIANT, where constant folding is enabledDoes this pull request potentially affect one of the following parts:
@Public(Evolving): (yes / no)Documentation
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Sonnet 5