Skip to content

[SPARK-58519][SQL] Document UPDATE and DELETE FROM statements - #57725

Open
marcuslin123 wants to merge 2 commits into
apache:masterfrom
marcuslin123:codex/SPARK-58519-document-update-delete
Open

[SPARK-58519][SQL] Document UPDATE and DELETE FROM statements#57725
marcuslin123 wants to merge 2 commits into
apache:masterfrom
marcuslin123:codex/SPARK-58519-document-update-delete

Conversation

@marcuslin123

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add SQL reference pages for the UPDATE and DELETE FROM statements. Both pages document:

  • statement syntax
  • table aliases
  • dynamic table options
  • assignment or filtering parameters
  • conditional and whole-table examples

This PR also links the new pages from the SQL syntax index and adds them to the related statements
on the MERGE INTO page.

Why are the changes needed?

Spark supports UPDATE and DELETE FROM for Data Source V2 tables, but the SQL reference did not
have dedicated pages for either statement. Users therefore could not find their syntax, parameters,
or examples alongside the other DML statements.

Does this PR introduce any user-facing change?

No. This is a documentation-only change for existing SQL functionality.

How was this patch tested?

The focused parser tests passed:

build/sbt 'catalyst/testOnly *DDLParserSuite -- -z "delete from table" -z "update table"'

The run completed with 10 tests passed and no failures. The staged changes also pass
git diff --cached --check, and the new Markdown links and code fences were checked locally.

The full Jekyll build was not run because Bundler 2.4.22 is not installed in the environment.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-5) was used for general coding assistance.

Comment thread docs/sql-ref-syntax-dml-update.md Outdated
Comment on lines +58 to +61
* **SET column = expression [ , ... ]**

Assigns a value to one or more columns. Each value may be an expression or `DEFAULT`. A nested
field may be targeted by using a qualified column name.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SET column = expression entry says a value may be an expression or DEFAULT, which is accurate, but doesn't call out that the expression can be an uncorrelated subquery over a completely different table (e.g. SET salary = (SELECT max(salary) FROM other_table)). Worth adding a sentence here.

Suggested change
* **SET column = expression [ , ... ]**
Assigns a value to one or more columns. Each value may be an expression or `DEFAULT`. A nested
field may be targeted by using a qualified column name.
* **SET column = value [ , ... ]**
Specifies the columns to update and the values to assign to them. Each `value` is an
expression, typically referencing columns of the target table, but it may also be an
uncorrelated subquery over other tables. A comma separates each assignment.

Comment thread docs/sql-ref-syntax-dml-update.md Outdated
Comment on lines +53 to +56
* **WITH ( option_key = option_value [ , ... ] )**

Specifies dynamic table options for this `UPDATE` operation. These options are passed to the
data source connector when writing to the table. The supported options depend on the connector.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both WITH (option_key = option_value) entries (UPDATE and DELETE FROM) describe the clause generically but don't mention: (a) that these options apply to this statement only, without changing the table's persistent configuration, (b) that a key that isn't a valid identifier needs backtick-quoting, and (c) that options the connector doesn't recognize are silently ignored. All three are worth stating explicitly so users know what to expect.

Suggested change
* **WITH ( option_key = option_value [ , ... ] )**
Specifies dynamic table options for this `UPDATE` operation. These options are passed to the
data source connector when writing to the table. The supported options depend on the connector.
* **WITH ( key = val [ , ... ] )**
An optional list of dynamic table options passed to the Data Source V2 connector for this
statement only. The options are surfaced to the connector's row-level write, allowing
per-statement tuning (for example a write file size or an isolation level) without changing
the table configuration. Keys and values are treated as strings; a key that is not a valid
identifier can be quoted with backticks. Options that the connector does not recognize are
ignored.

Comment on lines +103 to +107
### Related Statements

* [DELETE FROM statement](sql-ref-syntax-dml-delete-from.html)
* [MERGE INTO statement](sql-ref-syntax-dml-merge-into.html)
* [SELECT statement](sql-ref-syntax-qry-select.html)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, link to INSERT statement as well?

@anuragmantri

Copy link
Copy Markdown
Contributor

@peter-toth, @szehon-ho - Could you take a look?

@szehon-ho szehon-ho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these -- the syntax all checks out. I verified the clause order against SqlBaseParser.g4 (DELETE FROM identifierReference tableAlias optionsClause? whereClause? and UPDATE identifierReference tableAlias optionsClause? setClause whereClause?, lines 750-751), so alias, then WITH (...), then SET / WHERE is right. The claims in the SET bullet are all backed by UpdateTableSuiteBase: alias-qualified assignment, nested field targeting, DEFAULT, and the uncorrelated scalar subquery. I also confirmed _data/menu-sql.yaml only links section anchors rather than individual DML pages, so the sql-ref-syntax.md update is all the index needs.

A few non-blocking comments below, mostly about making the examples reproducible the way the neighbouring DML pages do.

Comment on lines +53 to +59
* **WITH ( key = value [ , ... ] )**

Specifies an optional list of dynamic table options passed to the Data Source V2 connector for
this statement only. The options allow per-statement tuning without changing the table's
persistent configuration. Keys and values are treated as strings; a key that is not a valid
identifier can be quoted with backticks. Spark passes options through without validating their
names, and connectors may ignore options they do not recognize.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor things here.

First, consistency: #57724 documents this same WITH (...) clause on the INSERT, MERGE INTO, and SELECT pages in two sentences ("Specifies dynamic table options for this INSERT operation. These options are passed to the data source connector when writing to the table. The supported options depend on the connector."). With both PRs landing around the same time, readers will hit the same clause described at two different levels of detail depending on which page they land on. Worth either syncing the wording across all five pages, or describing it once -- the Row-Level DML section of sql-v2-data-sources.md that this page already links to is a plausible home -- and giving each page the one-line version plus a link.

Second, if the long form stays, it can lose some weight. The second sentence restates "for this statement only" from the first, and the last sentence makes the same point twice ("passes options through without validating their names" / "connectors may ignore options they do not recognize"). The backtick note is the part that really earns its place, since write.split-size in the example below is a parse error unquoted (propertyKey is identifier (DOT identifier)* | stringLit). One thing that might be worth having in place of "keys and values are treated as strings": keys are case-insensitive, since the options end up in a CaseInsensitiveStringMap (AstBuilder.resolveOptions). That is behavior a user cannot guess.


### Examples

The following examples assume that an `employees` table has already been created and populated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The neighbouring DML pages make their examples runnable end to end. MERGE INTO opens its Examples section with the initial SELECT * FROM target / SELECT * FROM source state and shows the resulting table under each example; INSERT TABLE creates students inline and shows the output after each insert.

Here there is no schema for employees and no results, so a reader cannot tell what status, department, last_active_date, or salary are, or check that they got the expected outcome. Could you add a small CREATE TABLE plus initial SELECT * block here, and a result under each example?

Comment on lines +88 to +90
UPDATE employees AS e
SET e.salary = e.salary * 1.05, e.status = 'reviewed'
WHERE e.department = 'Sales';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, and it mostly goes away once the example table has a stated schema: if salary is an INT, then salary * 1.05 is a DOUBLE, and ANSI store assignment permits numeric narrowing (Cast.canANSIStoreAssign allows any NumericType -> NumericType), so the 5% raise is silently truncated back to an INT. Giving salary a DECIMAL or DOUBLE type in the setup, or using an increment that stays integral, avoids demonstrating truncation by accident.

Comment on lines +52 to +58
* **WITH ( key = value [ , ... ] )**

Specifies an optional list of dynamic table options passed to the Data Source V2 connector for
this statement only. The options allow per-statement tuning without changing the table's
persistent configuration. Keys and values are treated as strings; a key that is not a valid
identifier can be quoted with backticks. Spark passes options through without validating their
names, and connectors may ignore options they do not recognize.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same bullet as on the UPDATE page -- whatever wording you settle on there, please keep these two in sync (and ideally with the INSERT / MERGE INTO / SELECT pages in #57724).


### Examples

The following examples assume that an `employees` table has already been created and populated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the UPDATE page: no schema for employees and no results under the examples, unlike MERGE INTO and INSERT TABLE.

Comment thread docs/sql-ref-syntax.md
* [INSERT TABLE](sql-ref-syntax-dml-insert-table.html)
* [INSERT OVERWRITE DIRECTORY](sql-ref-syntax-dml-insert-overwrite-directory.html)
* [MERGE INTO](sql-ref-syntax-dml-merge-into.html)
* [UPDATE](sql-ref-syntax-dml-update.html)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit on placement. The DDL list above is strictly alphabetical, while this DML list is not (INSERT TABLE precedes INSERT OVERWRITE DIRECTORY, and LOAD trails MERGE INTO), so putting DELETE FROM first and UPDATE between MERGE INTO and LOAD ends up arbitrary either way. I would either append UPDATE after LOAD or alphabetize the whole list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants