Skip to content

Spark: Support view schema binding - #17716

Draft
manuzhang wants to merge 8 commits into
apache:mainfrom
manuzhang:codex/spark42-view-schema-binding
Draft

Spark: Support view schema binding#17716
manuzhang wants to merge 8 commits into
apache:mainfrom
manuzhang:codex/spark42-view-schema-binding

Conversation

@manuzhang

Copy link
Copy Markdown
Member

Depends on #14984.

This draft is stacked on the Spark 4.2 support branch because spark/v4.2 is not yet present on main. The schema-binding feature is isolated in the top commit and will be rebased onto main after #14984 merges.

Changes

  • Route Spark 4.2 ALTER VIEW ... WITH SCHEMA <mode> commands for Iceberg views to a dedicated execution node.
  • Persist schema-binding changes through Iceberg's property update API so the operation does not replace SQL representations or create a new view version.
  • Invalidate cached view plans after changing the schema mode.
  • Include the schema mode in SHOW CREATE TABLE output when Spark's schema-binding feature is enabled.
  • Cover EVOLUTION and COMPENSATION behavior, query-column handling, SHOW CREATE output, and named/session catalog caches.

Verification

  • Focused TestViews coverage for SHOW CREATE and ALTER VIEW ... WITH SCHEMA.
  • Focused TestViewCatalogCache schema-mode coverage for named and session catalogs.
  • Spark 4.2 extensions Scalastyle, Spotless, and Checkstyle.
  • git diff --check.

AI Disclosure

  • Model: GPT-5 Codex
  • Platform/Tool: Codex
  • Human Oversight: partially reviewed
  • Prompt Summary: Split Spark 4.2 Iceberg view schema-binding execution, SHOW CREATE support, and tests into a dedicated follow-up pull request.

manuzhang and others added 8 commits August 13, 2026 21:34
Generated-by: Codex
Generated-by: Codex
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant