Allow dialects that use `->` as an operator to support `LAMBDA` syntax by adriangb · Pull Request #2458 · apache/datafusion-sqlparser-rs

adriangb · 2026-08-25T23:29:27Z

Motivation

supports_lambda_functions() gates two different spellings of the same feature:

the arrow form, x -> x + 1
the LAMBDA keyword form, LAMBDA x : x + 1

Because they share one flag, a dialect cannot have one without the other. That
shuts out any dialect that already gives -> a meaning. PostgreSQL is the
obvious case: -> is JSON member access, so turning the flag on silently
reinterprets existing expressions rather than adding a capability.

Concretely, with the flag enabled, a -> 'b' no longer parses as a binary
operator. It parses as a lambda with parameter a and body 'b', because
parse_prefix treats any unreserved word followed by -> as a lambda
parameter. Note ->> is unaffected, so the breakage is partial and easy to
miss.

The LAMBDA keyword form has no such conflict: it does not claim ->.

Change

Adds Dialect::supports_lambda_keyword_syntax(), which gates only the LAMBDA
keyword form, and defaults to supports_lambda_functions().

No existing dialect changes behavior. Dialects that support the arrow form keep
both spellings; dialects that support neither still get neither. A dialect that
uses -> for something else can now override just the new method to get lambdas
without disturbing its operator.

No dialect shipped here opts in; this only makes the capability reachable.

Tests

Two tests in tests/sqlparser_custom_dialect.rs:

a dialect enabling only supports_lambda_keyword_syntax parses
lambda x : x + 1 while -> stays a BinaryOperator::Arrow
a dialect enabling only supports_lambda_functions still accepts both
spellings, pinning the defaulting behavior

Full suite, cargo fmt --check, and cargo clippy --all-targets --all-features
all pass.

adriangb · 2026-08-26T12:13:01Z

+    fn supports_lambda_keyword_syntax(&self) -> bool {
+        self.supports_lambda_functions()
+    }


One could argue for adding fn supports_lambda_arrow_syntax() as well, but I'd hold off until there is a concrete use case for enabling arrow syntax but not lambda syntax.

Arrow-only is already expressible by overriding supports_lambda_keyword_syntax to false, so I believe there is no need.

adriangb · 2026-08-26T12:35:21Z

+#[test]
+fn test_lambda_keyword_syntax_on_postgres_derivative() {


I'm split between this (the real regression test I want) and another test using a MyDialect in sqlparser_custom_dialect.rs that enables the two flags. Open to input.

Both seem worth keeping.

adriangb · 2026-08-30T15:02:02Z

@LucaCappelletti94 could I ask you to take a look at this change? Thanks!

LucaCappelletti94

Generally ok, just a missing Snowflake syntax test, and the missing documentation to lambda, and the wrong syntax in the example.

LucaCappelletti94 · 2026-08-30T15:25:29Z

+        panic!("expected `->` to stay a binary operator");
+    };
+    assert_eq!(&BinaryOperator::Arrow, op);
+}


I believe we should add a test for the Spark/Snowflake case (they only accept ->):

Suggested change

}

}

#[test]

fn custom_dialect_lambda_arrow_syntax_without_keyword() {

// Arrow lambdas stay on while the `LAMBDA` keyword spelling is off,

// as in engines like Spark and Snowflake.

#[derive(Debug)]

struct MyDialect {}

impl Dialect for MyDialect {

fn is_identifier_start(&self, ch: char) -> bool {

is_identifier_start(ch)

}

fn is_identifier_part(&self, ch: char) -> bool {

is_identifier_part(ch)

}

fn supports_lambda_functions(&self) -> bool {

true

}

fn supports_lambda_keyword_syntax(&self) -> bool {

false

}

}

let dialect = MyDialect {};

let sql = "SELECT transform(xs, x -> x + 1)";

assert_eq!(

sql,

&format!("{}", Parser::parse_sql(&dialect, sql).unwrap()[0])

);

assert!(Parser::parse_sql(&dialect, "SELECT transform(xs, lambda x : x + 1)").is_err());

}

Thanks, added but in sqlparser_custom_dialect.rs

LucaCappelletti94 · 2026-08-30T15:29:10Z

+    /// lambda functions, for example:
+    ///
+    /// ```sql
+    /// SELECT transform(array(1, 2, 3), LAMBDA x : x + 1); -- returns [2,3,4]


I believe transform(array(...), LAMBDA ...) is the wrong spelling, if you meant the DuckDB syntax it would be more like:

Suggested change

/// SELECT transform(array(1, 2, 3), LAMBDA x : x + 1); -- returns [2,3,4]

/// SELECT list_transform([1, 2, 3], lambda x : x + 1); -- returns [2, 3, 4]

Thanks, applied!

LucaCappelletti94 · 2026-08-30T15:29:49Z

+    /// and its derivatives, where `->` is JSON member access. Defaults to
+    /// [`Self::supports_lambda_functions`], so dialects supporting the `->`
+    /// spelling accept the `LAMBDA` spelling too unless they say otherwise.
+    fn supports_lambda_keyword_syntax(&self) -> bool {


Suggested change

fn supports_lambda_keyword_syntax(&self) -> bool {

///

/// See <https://duckdb.org/docs/stable/sql/functions/lambda>

fn supports_lambda_keyword_syntax(&self) -> bool {

Thanks, applied!

LucaCappelletti94 · 2026-08-30T15:30:41Z

+    fn supports_lambda_keyword_syntax(&self) -> bool {
+        self.supports_lambda_functions()
+    }


Arrow-only is already expressible by overriding supports_lambda_keyword_syntax to false, so I believe there is no need.

LucaCappelletti94 · 2026-08-30T15:31:26Z

+#[test]
+fn test_lambda_keyword_syntax_on_postgres_derivative() {


Both seem worth keeping.

Exercises the capability the way a downstream crate would: derive a dialect from PostgreSqlDialect with `supports_lambda_keyword_syntax` overridden, then check that `LAMBDA x : x + 1` parses while `->` and `->>` keep parsing as JSON member access rather than lambda parameters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Address review feedback: rather than parsing the `LAMBDA` spelling and the `->` operator as separate statements, parse one expression that uses both — a lambda whose body is a JSON access — which is the shape a PostgreSQL derivative actually cares about. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

All three changes are applied review suggestions from Luca Cappelletti. Co-Authored-By: Luca Cappelletti <7738570+LucaCappelletti94@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

adriangb · 2026-08-31T11:36:17Z

Thanks for the review @LucaCappelletti94!

@iffyio or @yoavcloud could one of you take a look at this PR that @LucaCappelletti94 has already reviewed and approved? Thanks!

iffyio

left a comment, its not clear to me what the PR is looking to solve for

iffyio · 2026-09-02T07:54:47Z

+    /// [`Self::supports_lambda_functions`], so dialects supporting the `->`
+    /// spelling accept the `LAMBDA` spelling too unless they say otherwise.
+    ///
+    /// See <https://duckdb.org/docs/stable/sql/functions/lambda>


Im not sure I understood the problem being solved for - the description mentions pg as an example, suggesting there is some ambiguous grammar in play but pg doesnt have lambda syntax to my knowledge?

is there an example syntax that issupported by a dialect and the parser doesnt cover?

adriangb · 2026-09-02T11:21:53Z

you cannot currently have a grammar that supports both postgres style json operators and lambda transforms because -> conflicts

adriangb commented Aug 26, 2026

View reviewed changes

Comment thread tests/sqlparser_derive_dialect.rs Outdated

LucaCappelletti94 suggested changes Aug 30, 2026

View reviewed changes

adriangb force-pushed the split-lambda-keyword-syntax branch from 2d11f84 to d994419 Compare August 30, 2026 16:08

adriangb requested a review from LucaCappelletti94 August 30, 2026 16:08

adriangb and others added 5 commits August 30, 2026 11:24

Allow dialects that use -> as an operator to support LAMBDA syntax

6753b2f

Remove redundant borrows in format! args to satisfy clippy 1.98

4c58d2e

Address review: DuckDB doc example and link, arrow-only dialect test

1d89c66

All three changes are applied review suggestions from Luca Cappelletti. Co-Authored-By: Luca Cappelletti <7738570+LucaCappelletti94@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

adriangb force-pushed the split-lambda-keyword-syntax branch from d994419 to 1d89c66 Compare August 30, 2026 16:24

LucaCappelletti94 approved these changes Aug 31, 2026

View reviewed changes

iffyio reviewed Sep 2, 2026

View reviewed changes

		#[test]
		fn test_lambda_keyword_syntax_on_postgres_derivative() {

-}
+}
+#[test]
+fn custom_dialect_lambda_arrow_syntax_without_keyword() {
+    // Arrow lambdas stay on while the `LAMBDA` keyword spelling is off,
+    // as in engines like Spark and Snowflake.
+    #[derive(Debug)]
+    struct MyDialect {}
+    impl Dialect for MyDialect {
+        fn is_identifier_start(&self, ch: char) -> bool {
+            is_identifier_start(ch)
+        }
+        fn is_identifier_part(&self, ch: char) -> bool {
+            is_identifier_part(ch)
+        }
+        fn supports_lambda_functions(&self) -> bool {
+            true
+        }
+        fn supports_lambda_keyword_syntax(&self) -> bool {
+            false
+        }
+    }
+    let dialect = MyDialect {};
+    let sql = "SELECT transform(xs, x -> x + 1)";
+    assert_eq!(
+        sql,
+        &format!("{}", Parser::parse_sql(&dialect, sql).unwrap()[0])
+    );
+    assert!(Parser::parse_sql(&dialect, "SELECT transform(xs, lambda x : x + 1)").is_err());
+}

	/// SELECT transform(array(1, 2, 3), LAMBDA x : x + 1); -- returns [2,3,4]
	/// SELECT list_transform([1, 2, 3], lambda x : x + 1); -- returns [2, 3, 4]

Uh oh!

Conversation

adriangb commented Aug 25, 2026

Motivation

Change

Tests

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adriangb commented Aug 30, 2026

Uh oh!

LucaCappelletti94 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adriangb commented Aug 31, 2026

Uh oh!

iffyio left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adriangb commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants