Repository navigation
Bring the 9.1.0 changes to main and start 9.2.0-SNAPSHOT - #794
Merged
Merged
Conversation
- Check that the image holds the preamble. - For a non-empty image, check that it holds the total weight and the whole sketch array. An empty image still allocates the configured sketch array, as designed: a CountMin sketch does not grow, and is sized up front for the accuracy it needs. Adds AllocationTracker, a test helper that measures allocation by the current thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In FrequentLongsSketch.getInstance(MemorySegment) and FrequentItemsSketch.getInstance(MemorySegment, serDe): - Check that LG_MIN_MAP_SIZE <= lgCurMapSize <= lgMaxMapSize, and that activeItems is between 0 and the capacity of the current map, as C++ check_size() does. - Check, in long arithmetic, that the image holds the declared arrays. - Rebuild the sketch into a map just large enough for the active items. The map grows as needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Updatable (QuickSelectSketch) image: - Check 0 <= lgNomEntries <= 26 (lgK has the same maximum as Theta), lgCurrentCapacity <= max(lgNomEntries + 1, 5) (the table grows to at most twice the nominal entries, but never starts below 32 slots), and 0 <= lgResizeFactor <= 3. - Check that the count is non-negative, smaller than the table, and that the image holds at least 8 bytes per entry. - Check that the table is plausible for the count: either the starting size or at most 32 times the count. This keeps the exact table size of valid images. Compact image (current and legacy forms): - Check that the image holds the preamble, and that the count is non-negative and fits in the image at 8 bytes per entry. Constructors: the QuickSelectSketch constructors now reject nominal entries that are not positive or are above 2^26, as the builder and Theta already do. Before, UpdatableTupleSketch, TupleUnion and the Double, Integer and ArrayOfStrings sketches accepted them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Compact images (heapify and wrap): check that numValues is positive, the count is non-negative, and the image holds count keys and count * numValues values. - QuickSelect images (heapify and wrap, and the union, which contains one): check that numValues is positive, 0 <= lgNomEntries <= 26, lgCurrentCapacity <= max(lgNomEntries + 1, 5) (the table never starts below 32 slots; unions accept nominal entries below 16), 0 <= count <= capacity, and that the image holds the whole table, which the serialized form always includes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- checkSegIntegrity(), used for updatable, alpha and union images (heapify and wrap): check lgNomLongs <= 26, 0 <= lgArrLongs <= lgNomLongs + 1 and 0 <= curCount <= table size, and compute the required size in long arithmetic. - Compact images (heapify, and wrap of a MemorySegment or byte[]): check that curCount is non-negative and that the image holds the hashes. Images in hash-table form must hold the whole table. - Compressed images (serial version 4): check preLongs, entryBits and numEntriesBytes, and that the image holds numEntries * entryBits bits. - Intersection images (heapify and wrap): check 0 <= lgArrLongs <= 27, -1 <= curCount <= table size (-1 means no result yet), and that the image holds the table when curCount > 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkPreamble(), which heapify, wrap, writableWrap and HllUnion.heapify all call, now also checks: - that lgK is in its documented range of [4, 21]; - that a list count is no more than the list capacity; - that a set count is between 0 and K and set lgArr is at most lgK; - that an aux count is between 0 and K and aux lgArr is at most lgK. writableWrap() now validates the preamble before using lgK. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a non-empty image, check that the remaining bytes hold the bit array. An empty image still allocates the configured bit array, as designed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each item takes at least one byte (its type code), so check that the input can hold numItems items. PR 783 makes the same check in the UTF-8 and UTF-16 string SerDes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No serialized item takes less than one bit (booleans are packed 8 per byte), so QuantilesItemsSketch.heapify() now requires that n is non-negative and that the number of retained items is no more than 8 times the remaining image bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Check that k is even and in [4, 1024] before reading the compactors. - Check that 1 <= sectionSize <= k, numSections >= 1, and sectionSize * numSections <= k * k. Simulating the section growth rule for every legal k gives at most 0.96 * k * k. - Check that the count is non-negative and that the image holds the items. - Size each reconstructed buffer to the items it holds. The buffer grows by delta as needed, as it does after compaction. ReqCompactorTest compared the rebuilt buffer's capacity and backing array. It now compares the count, the sorted flag and the items. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Check the region counts against k: a sketch in sampling mode holds exactly k items, and one in warm-up holds at most k. - Check that the image holds the H-region weights. - Size the weight array to hCount, and the item, weight and mark lists to the items in the image. The ArrayLists grow as needed, so this does not change behavior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Compute the required image size in long arithmetic, and bounds-check each stream against the image. - Check that 0 <= numCoupons <= 64k (k rows of 64 columns), that 0 <= numSv <= numCoupons, and that numSv <= 32 * svLengthInts, since each surprising value takes at least one bit of the compressed stream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proost
approved these changes
Nov 6, 2026
The project's first ASF release was in 2019. This matches datasketches-cpp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the 9.1.0 release changes from
9.1.Xto main and starts 9.2.0 development.9.1.X.users@datasketches.apache.org..gitdirectory.Compared with
9.1.0-RC1, the only difference is the pom version.Please merge with a merge commit, not a squash, so the commits keep the same hashes as on
9.1.X.mvn clean verifypasses: 2385 tests, 0 failures.🤖 Generated with Claude Code