Skip to content

[SPARK-59628][SQL] Report a missing MATCH_CONDITION in ASOF JOIN instead of parsing ASOF as an alias - #58966

Open
AMC-hawk wants to merge 1 commit into
apache:masterfrom
AMC-hawk:SPARK-59628
Open

AMC-hawk wants to merge 1 commit into
apache:masterfrom
AMC-hawk:SPARK-59628

Conversation

@AMC-hawk

@AMC-hawk AMC-hawk commented Sep 22, 2026

Copy link
Copy Markdown

What changes were proposed in this pull request?

MATCH_CONDITION is required for an ASOF join, but ASOF is a non-reserved keyword, so it is also a legal table alias. When MATCH_CONDITION is missing, the ASOF branch of joinRelation cannot match, and the parser falls back to reading ASOF as the alias of the left relation:

SELECT * FROM t ASOF JOIN u ON t.a = u.a;
'Project [*]
+- 'Join Inner, ('t.a = 'u.a)
   :- 'SubqueryAlias asof
   :  +- 'UnresolvedRelation [t]
   +- 'UnresolvedRelation [u]

This PR makes that case fail with a clear error, in two parts:

  1. tableAlias no longer takes ASOF as an implicit alias when the next token is JOIN. This uses a semantic predicate, isAsofJoinAhead(), in the same way the grammar already uses isOperatorPipeStart(). An explicit AS is unaffected, so t AS asof JOIN u still aliases the relation, and asof remains usable as an identifier everywhere else.
  2. MATCH_CONDITION is made optional in asofJoinCriteria, so that omitting it reaches AstBuilder.withAsOfJoin, which raises the new ASOF_JOIN_MATCH_CONDITION_MISSING error (SQLSTATE 42601). It is still required: omitting it always fails. The check runs after the existing feature flag check, so with ASOF disabled users still see UNSUPPORTED_FEATURE.ASOF_JOIN first.

Part 1 alone would turn the silent inner join into a generic PARSE_SYNTAX_ERROR; part 2 makes the error name the missing clause, as the JIRA asks.

Making ASOF a reserved keyword would also fix the ambiguity, but would break every existing query that uses asof as an identifier. The predicate only refuses the one ambiguous shape, an implicit alias immediately before JOIN.

Why are the changes needed?

An ASOF join returns the single nearest right row for each left row. A plain inner join returns every matching row. Parsing a malformed ASOF join as an inner join therefore returns silently wrong results, often many more rows on time series data, with no error or warning.

The behavior was also inconsistent. Only the default join type with an unaliased left relation was misread; with an alias or with LEFT, the same mistake failed with a generic PARSE_SYNTAX_ERROR ... at or near 'ON' that did not mention MATCH_CONDITION.

This is one of the two parser stage defects tracked by SPARK-59626.

Does this PR introduce any user-facing change?

Yes. An ASOF join without MATCH_CONDITION now fails with a clear error in every form, instead of running as an inner join or failing with a generic syntax error. The ASOF JOIN syntax is disabled by default (spark.sql.join.asofJoin.enabled) and is unreleased.

With ASOF enabled:

  • SELECT * FROM t ASOF JOIN u ON t.a = u.a: before, ran as t AS asof INNER JOIN u. After, ASOF_JOIN_MATCH_CONDITION_MISSING.
  • SELECT * FROM t ASOF JOIN u: before, ran as t AS asof INNER JOIN u. After, ASOF_JOIN_MATCH_CONDITION_MISSING.
  • SELECT * FROM t x ASOF JOIN u ON t.a = u.a: before, PARSE_SYNTAX_ERROR at 'ON'. After, ASOF_JOIN_MATCH_CONDITION_MISSING.
  • SELECT * FROM t LEFT ASOF JOIN u ON t.a = u.a: before, PARSE_SYNTAX_ERROR at 'ON'. After, ASOF_JOIN_MATCH_CONDITION_MISSING.
[ASOF_JOIN_MATCH_CONDITION_MISSING] ASOF JOIN requires a MATCH_CONDITION clause, for example MATCH_CONDITION (left.timestamp >= right.timestamp). SQLSTATE: 42601

Unchanged: valid ASOF joins; t AS asof JOIN u ... and t `asof` JOIN u ..., which still alias the relation as asof; FROM t asof; and asof as a column name. A query that used asof as an implicit alias directly before JOIN must now write AS asof or quote it.

How was this patch tested?

New tests:

  • PlanParserSuite: asof join - missing match condition rejected covers five shapes (unaliased with ON, aliased with ON, USING, LEFT with no condition, no condition at all) and checks the error condition, SQLSTATE and query context. asof join - an alias named asof is still allowed when written with AS checks that t as asof join u ... and t asof still parse to an aliased relation.
  • join-asof-grammar.sql: three new golden queries for the unaliased case, the case with no join condition, and trades AS asof JOIN quotes ... returning rows. The existing aliased case now reports ASOF_JOIN_MATCH_CONDITION_MISSING instead of PARSE_SYNTAX_ERROR; this is the only change to existing expected output. Golden files were regenerated with SPARK_GENERATE_GOLDEN_FILES=1.

Existing tests, since tableAlias is used by almost every query:

build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.parser.*'      # 784 passed
build/sbt 'sql/testOnly org.apache.spark.sql.SQLQueryTestSuite'           # 845 passed
build/sbt 'core/testOnly org.apache.spark.SparkThrowableSuite'            # 38 passed

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Yes, Claude

…ead of parsing ASOF as an alias

MATCH_CONDITION is required for an ASOF join, but ASOF is a non-reserved
keyword. When MATCH_CONDITION was missing and the left relation had no
alias, the parser read ASOF as that alias, so
`SELECT * FROM t ASOF JOIN u ON t.a = u.a` silently ran as a plain inner
join of `t AS asof` with `u` and returned wrong results. With an alias on
the left relation the same mistake failed with a generic PARSE_SYNTAX_ERROR.

A table alias no longer takes ASOF implicitly when JOIN follows it, and
MATCH_CONDITION is optional in the grammar so that AstBuilder can report a
missing one by name with the new ASOF_JOIN_MATCH_CONDITION_MISSING error.
An alias named asof is still allowed when written as `t AS asof`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant