Skip to content

Optimize BSI CompareValue with bitmap set operations - #540

Open
perfloop-agent wants to merge 2 commits into
RoaringBitmap:masterfrom
perfloop:perfloop-pr-open-0baz0qdwgn
Open

Optimize BSI CompareValue with bitmap set operations#540
perfloop-agent wants to merge 2 commits into
RoaringBitmap:masterfrom
perfloop:perfloop-pr-open-0baz0qdwgn

Conversation

@perfloop-agent

Copy link
Copy Markdown
Contributor

Summary

Rework CompareValue to compare bit slices as bitmap sets instead of materializing column-ID batches and probing each bit. Equality starts from a selective set plane, range queries compose lower and upper bitmap comparisons, and signed values are handled by separating sign slices. The parallelism argument remains for API compatibility, but comparisons now operate directly on bitmap slices rather than splitting IDs into worker batches.

Performance

On workload large preloaded BSI equality query for value 55, median metric ns/op changed from 1005014636 to 45904728; paired median delta -959257346 (-95.4% of baseline; at least 19/20 confidence interval -1080414403 to -927111616 from 10 pairs).

On workload large preloaded BSI equality query for value 55, median metric B/op changed from 115856362 to 12982819; paired median delta -102873544 (-88.8% of baseline; at least 19/20 confidence interval -103155793 to -102584776 from 10 pairs).

On workload large preloaded BSI equality query for value 55, median metric allocs/op changed from 14063 to 5162; paired median delta -8901 (-63.3% of baseline; at least 19/20 confidence interval -9442 to -8354 from 10 pairs).

Testing

Added a large-fixture benchmark and ground-truth coverage for all comparison operations, signed and unsigned values, subset inputs, run-optimized bitmaps, and result isolation. Ran the full Go suite, the BitSliceIndexing race test, formatting and conversion checks, the appengine build and test, and ARM/ARM64 compilation.

All 5 declared correctness checks passed.

Full verification record: https://app.perfloop.ai/t/oss/case_xw6sb2t5zr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant