-
Notifications
You must be signed in to change notification settings - Fork 4k
ARROW-11354: [Rust] Speed-up cast of dates and times (2-4x) #9297
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## master #9297 +/- ##
=======================================
Coverage 81.88% 81.89%
=======================================
Files 215 216 +1
Lines 52988 52983 -5
=======================================
- Hits 43391 43388 -3
+ Misses 9597 9595 -2
Continue to review full report at Codecov.
|
|
NOTE: this PR is 40 LOC change. The rest comes from the other PR. Please wait for the merge, this is only a draft to indicate what can be done with this. |
nevi-me
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for generalising this, and cleaning up the cast kernels. There were plenty of fruits hanging low there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could save a further few cycles here by specifying the null count, as it's known and trusted to be correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a great and simple example, and I wonder if the compiler is smart enough to optimise 2 commutative unary functions to perform as well as a single combined one.
Worth trying out some time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we call this something more general, like arity https://en.wikipedia.org/wiki/Arity
We'd probably want a binary function that takes 2 arguments, or maybe ternary if we're feeling cute enough to push it 😉.
EDIT: math_op is binary
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this was only unary, I gave it that name. Note that we can lift all binary arithmetics into a generic like this one and place it together.
I do not have strong feelings here, but it is the first time I hear about arity xD
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No worries, whoever adds a ternary function can change the name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would be backward incompatible, so it would be good to change it soon. I will change to arity. It is a fun name. Another option would be generics, but that is too generic. :)
This PR improves the performance of certain time / date casts by using the brand new API proposed in #9235 .
That API allows for a very fast execution of unary and infalible operations on primitive arrays, and this PR leverages that for cast operations that require a simple mathematical operation.