Skip to content

fix(proto): preserve case_insensitive of SimilarTo across a round trip - #26832

Open
0xdeafbeef wants to merge 1 commit into
apache:mainfrom
0xdeafbeef:similar-to-proto-case-insensitive
Open

0xdeafbeef wants to merge 1 commit into
apache:mainfrom
0xdeafbeef:similar-to-proto-case-insensitive

Conversation

@0xdeafbeef

Copy link
Copy Markdown

Which issue does this PR close?

Found while fixing the SQL unparser's handling of SIMILAR TO in #26831; no separate issue.

Rationale for this change

After a logical plan round trip through datafusion-proto, a case-insensitive SimilarTo matches case-sensitively.
SimilarToNode had no field for case_insensitive.
The encoder ignored the flag, and the decoder passed false.
The physical planner runs an Expr::SimilarTo with case_insensitive: true as a case-insensitive regex match (RegexIMatch / RegexNotIMatch).
The SQL planner always creates Expr::SimilarTo with case_insensitive: false, so only expressions built in Rust carry true.

What changes are included in this PR?

SimilarToNode gets bool case_insensitive = 5, which to_proto writes and from_proto reads.
prost.rs and pbjson.rs are regenerated with the datafusion/proto-models/gen generator, and the generated diff touches only SimilarToNode.
The pbjson serializer writes caseInsensitive only when it is true, so the JSON of a case-sensitive SimilarTo does not change.
A payload without the field decodes as false, the same result as before this PR.

The other option is a separate message, as LIKE and ILIKE use LikeNode and ILikeNode.

What is the testing strategy for this PR?

roundtrip_similar_to in datafusion/proto/tests/cases/roundtrip_logical_plan.rs takes a case_insensitive parameter and gets two cases with true: similar_to(false, None, true) and similar_to(true, Some('X'), true).
Without the fix the test fails on the first of them: the decoded expression has case_insensitive: false.

I ran these checks locally, not in CI:

  • cargo test -p datafusion-proto --features json,avro: lib tests, doc-tests and proto_integration pass, except seven parquet source tests that need the parquet-testing submodule data, which the local checkout did not have.
    The same seven fail without this change.
  • cargo fmt --all -- --check, cargo clippy -p datafusion-proto -p datafusion-proto-models --all-targets -- -D warnings with and without --all-features, typos on the changed files, and prettier on the upgrade guide pass.

Are there any user-facing changes?

The generated SimilarToNode struct has a new public field, so code that builds it with a struct literal must set case_insensitive.
docs/source/library-user-guide/upgrading/56.0.0.md gets a section with a before/after struct literal.
This change needs the api change label.

… trip

`SimilarToNode` had no field for `case_insensitive`, so `datafusion-proto` encoded an `Expr::SimilarTo` without it and decoded it as `false`.
The physical planner runs a case-insensitive `SimilarTo` as a case-insensitive regex match, so after a round trip the expression matched case-sensitively.

`SimilarToNode` gets `bool case_insensitive = 5`.
A payload without the field decodes as `false`, as before.
@github-actions github-actions Bot added documentation Improvements or additions to documentation proto Related to proto crate labels Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant