Commit ab9cc08
[SPARK-49635][SQL] Remove ANSI config suggestion in CAST error messages
### What changes were proposed in this pull request?
This PR improves CAST error messages to provide suggestions based on whether casts work in ANSI mode, non-ANSI mode, or try_cast():
1. Suggest try_cast() when the cast is valid but fails in ANSI mode and try_cast() supports it
2. Suggest disabling ANSI mode when the cast works in non-ANSI mode but try_cast() doesn't support it
3. Provide no suggestion when the cast is invalid in all modes
### Why are the changes needed?
In Spark 4.0.0, ANSI mode is now enabled by default. The previous error messages suggested users disable ANSI mode when encountering cast failures, which goes against the direction of the project. However, completely removing these suggestions would leave users without guidance when migrating code from Spark 3.x that relied on non-ANSI cast behavior.
### Does this PR introduce _any_ user-facing change?
Yes. Error messages for CAST operations that fail in ANSI mode now suggest using `try_cast()` instead of disabling ANSI mode, but only when `try_cast()` actually supports that cast operation.
**Before:**
```
cannot cast "ARRAY<INT>" to "ARRAY<BINARY>" with ANSI mode on.
If you have to cast ARRAY<INT> to ARRAY<BINARY>, you can set "spark.sql.ansi.enabled" as 'false'.
```
**After (for casts supported by try_cast):**
```
cannot cast "ARRAY<INT>" to "ARRAY<BINARY>".
To convert values from "ARRAY<INT>" to "ARRAY<BINARY>", you can use the functions `try_cast` instead.
```
**After (suggests disabling ANSI when try_cast doesn't support it):**
```
cannot cast "TIMESTAMP" to "BOOLEAN" with ANSI mode on.
This cast is not allowed in ANSI mode but works when "spark.sql.ansi.enabled" is 'false'.
```
**After (for casts not supported by try_cast):**
```
cannot cast "INT" to "BINARY".
```
### How was this patch tested?
- Added new test case for complex type casts (Array[Int] to Array[Binary]) that
demonstrates try_cast suggestion
- Added test case for TIMESTAMP → BOOLEAN cast that demonstrates config suggestion
- Updated existing test expectations in CastWithAnsiOnSuite to reflect the new
behavior
- All Cast-related unit tests pass
- Verified no regression in legacy (ANSI off) mode
- Manually tested with spark-shell to verify error messages
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53295 from qlong/SPARK-49635-remove-ansi-suggestion.
Authored-by: Qiegang Long <qlong@Qiegangs-MacBook-Pro.local>
Signed-off-by: Gengliang Wang <gengliang@apache.org>1 parent 8e75fc9 commit ab9cc08
File tree
2 files changed
+41
-2
lines changed- sql/catalyst/src
- main/scala/org/apache/spark/sql/catalyst/expressions
- test/scala/org/apache/spark/sql/catalyst/expressions
2 files changed
+41
-2
lines changedLines changed: 17 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
505 | 505 | | |
506 | 506 | | |
507 | 507 | | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
508 | 512 | | |
509 | 513 | | |
510 | 514 | | |
| |||
588 | 592 | | |
589 | 593 | | |
590 | 594 | | |
591 | | - | |
592 | | - | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
593 | 608 | | |
594 | 609 | | |
595 | 610 | | |
| |||
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
214 | 238 | | |
215 | 239 | | |
216 | 240 | | |
| |||
0 commit comments