Repository navigation
Add a reusable caching array hasher - #26817
Open
Rich-T-kid wants to merge 2 commits into
Open
Rich-T-kid wants to merge 2 commits into
Rich-T-kid wants to merge 2 commits into
Conversation
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
2 times, most recently
from
October 27, 2026 18:46
ec3f074 to
093c6fa
Compare
Rich-T-kid
marked this pull request as ready for review
October 27, 2026 18:46
Rich-T-kid
marked this pull request as draft
October 27, 2026 18:47
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 27, 2026 18:58
093c6fa to
02f9d0a
Compare
Rich-T-kid
marked this pull request as ready for review
October 27, 2026 19:01
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 27, 2026 19:02
02f9d0a to
1b38458
Compare
Contributor
Author
|
@kumarUjjawal @jayzhan-synnada small change but should make the follow on PR's less bloated. |
Rich-T-kid
marked this pull request as draft
October 27, 2026 19:02
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 27, 2026 19:03
1b38458 to
92e9e15
Compare
Rich-T-kid
marked this pull request as ready for review
October 27, 2026 19:13
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
2 times, most recently
from
October 27, 2026 19:26
752283a to
4ede583
Compare
Rich-T-kid
changed the title
Rich-T-kid
marked this pull request as draft
October 27, 2026 19:27
Rich-T-kid
marked this pull request as ready for review
October 27, 2026 19:35
This was referenced Oct 27, 2026
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 28, 2026 02:25
4ede583 to
9f6f79f
Compare
Rich-T-kid
changed the title
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 28, 2026 04:12
9f6f79f to
2bbdcbc
Compare
Rich-T-kid
changed the title
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 28, 2026 04:15
2bbdcbc to
1e48c85
Compare
Rich-T-kid
changed the title
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 28, 2026 04:27
1e48c85 to
23f953d
Compare
Reuse dictionary value hashes while the values Arc is unchanged so operators do not rehash the same dictionary for every batch.
Rich-T-kid
force-pushed
the
scatter-dictionary-hashes
branch
from
October 28, 2026 04:52
1284065 to
0490700
Compare
Contributor
|
@Rich-T-kid I see you're still working on this, can you ping me when its ready for me to look at. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Part of #26780.
Rationale for this change
Hash operators repeatedly see dictionary batches that share one values array. Rehashing those values for every batch wastes work; only the new keys need to be scattered.
What changes are included in this PR?
Add
CachingArrayHasher. One hasher corresponds to one hash expression and owns the hash seed, a weak dictionary-values identity, and a reusable value-hash buffer. Ordinary arrays keep the existing hashing path. Dictionary arrays reuse their value hashes while the valuesArcis unchanged and refill the same allocation when it changes.dictionary_hashesexposes the current values and hashes for aggregation, andsizereports the cached allocation.What is the testing strategy for this PR?
One focused test covers the same-values-
Arccache hit with composite-key rehashing, dictionary replacement, allocation reuse, null dictionary values, and the non-dictionary fallback. The focused normal and collision-mode hash tests pass on the rebased head, along with workspace Clippy, formatting, and typo checks.Are there any user-facing changes?
No.