[AURON #2130] Implement native function of weekofyear#2131
Merged
slfan1989 merged 1 commit intoapache:masterfrom Apr 5, 2026
Merged
[AURON #2130] Implement native function of weekofyear#2131slfan1989 merged 1 commit intoapache:masterfrom
slfan1989 merged 1 commit intoapache:masterfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Implements native support for Spark-compatible weekofyear() to reduce Spark fallbacks and improve Spark SQL function parity in Auron’s native execution path.
Changes:
- Add Spark expression translation for
WeekOfYearto a native ext scalar function (Spark_WeekOfYear). - Implement
spark_weekofyear()in the native Rust function layer and register it in the ext-function dispatcher. - Add Spark-side integration coverage plus Rust unit test coverage for core ISO week semantics and null propagation.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala | Adds Spark WeekOfYear expression mapping to the native ext function Spark_WeekOfYear. |
| spark-extension-shims-spark/src/test/scala/org/apache/auron/AuronFunctionSuite.scala | Adds an integration test exercising weekofyear() via checkSparkAnswerAndOperator. |
| native-engine/datafusion-ext-functions/src/spark_dates.rs | Adds spark_weekofyear() implementation (ISO week) plus a unit test. |
| native-engine/datafusion-ext-functions/src/lib.rs | Registers Spark_WeekOfYear in the native ext function dispatcher. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
spark-extension-shims-spark/src/test/scala/org/apache/auron/AuronFunctionSuite.scala
Outdated
Show resolved
Hide resolved
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala
Outdated
Show resolved
Hide resolved
edd72ee to
54bd43c
Compare
16 tasks
slfan1989
approved these changes
Apr 1, 2026
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala
Outdated
Show resolved
Hide resolved
ShreyeshArangath
approved these changes
Apr 2, 2026
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala
Show resolved
Hide resolved
yew1eb
approved these changes
Apr 3, 2026
Signed-off-by: weimingdiit <weimingdiit@gmail.com>
b0855a5 to
eefcef5
Compare
Contributor
|
@weimingdiit Thanks for the contribution! @ShreyeshArangath @yew1eb Thanks for the review! |
Contributor
Author
|
@slfan1989 Thanks for your merge! @ShreyeshArangath @yew1eb Thanks for the review! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes ##2130
Rationale for this change
To improve compatibility with Spark SQL date functions, we should implement
weekofyear()with Spark-compatible semantics.Expected behavior
Function name:
weekofyear(expr)Return type:
INTWeek semantics:
Examples:
weekofyear('2009-07-30')->31weekofyear('2016-01-01')->53weekofyear('2017-01-01')->52Supports:
DATE,TIMESTAMP, and compatible string/date inputs consistent with existing date extraction functionsAdditional expectations:
NULLif input isNULLWhat changes are included in this PR?
This PR adds native support for the
weekofyear()function with Spark-compatible semantics.The following changes are included:
spark_weekofyear()in the expression layerWeekOfYearexpression support inNativeConvertersfor proper Spark -> native translationSpark_WeekOfYearin native function dispatchAre there any user-facing changes?
NO.
How was this patch tested?
spark_weekofyear()weekofyear('2009-07-30') = 31weekofyear('2016-01-01') = 53weekofyear('2017-01-01') = 52