Optimize BSI CompareValue with bitmap set operations - #540
Open
perfloop-agent wants to merge 2 commits into
Open
Optimize BSI CompareValue with bitmap set operations#540perfloop-agent wants to merge 2 commits into
perfloop-agent wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rework CompareValue to compare bit slices as bitmap sets instead of materializing column-ID batches and probing each bit. Equality starts from a selective set plane, range queries compose lower and upper bitmap comparisons, and signed values are handled by separating sign slices. The parallelism argument remains for API compatibility, but comparisons now operate directly on bitmap slices rather than splitting IDs into worker batches.
Performance
On workload
large preloaded BSI equality query for value 55, median metricns/opchanged from 1005014636 to 45904728; paired median delta -959257346 (-95.4% of baseline; at least 19/20 confidence interval -1080414403 to -927111616 from 10 pairs).On workload
large preloaded BSI equality query for value 55, median metricB/opchanged from 115856362 to 12982819; paired median delta -102873544 (-88.8% of baseline; at least 19/20 confidence interval -103155793 to -102584776 from 10 pairs).On workload
large preloaded BSI equality query for value 55, median metricallocs/opchanged from 14063 to 5162; paired median delta -8901 (-63.3% of baseline; at least 19/20 confidence interval -9442 to -8354 from 10 pairs).Testing
Added a large-fixture benchmark and ground-truth coverage for all comparison operations, signed and unsigned values, subset inputs, run-optimized bitmaps, and result isolation. Ran the full Go suite, the BitSliceIndexing race test, formatting and conversion checks, the appengine build and test, and ARM/ARM64 compilation.
All 5 declared correctness checks passed.
Full verification record: https://app.perfloop.ai/t/oss/case_xw6sb2t5zr