Repository navigation
Check KLL, REQ and Quantiles images when deserializing - #568
Merged
Merged
Conversation
…izing deserialize() used the serialized levels array without checking it. A level beyond the capacity, or a level that starts before the previous one, put level boundaries outside the items buffer, and the bytes overload read the levels array without checking the input size. Both overloads now check that k >= MIN_K, that there is at least one level, that the input holds the levels array, and that each level starts at or after the previous one and no later than the capacity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
deserialize() used the compactor header fields without checking them. A section size or number of sections of zero gave a nominal capacity of zero, so the buffer did not grow on the next update. The bytes overload checked 8 of the 20 header bytes. Large lg_weight and number of sections values made shifts undefined, and a non-empty sketch with no items read past the end of its only compactor. Both overloads now check: - k is even and in [MIN_K, MAX_K] - a non-empty sketch has at least one level, and exactly one with raw items, and a single level holds at least one item - compactor i has lg_weight i - the section size rounds to an even value in [MIN_K, k] - the number of sections is in [1, 64], and the raw section size, rounded to an integer, times the number of sections is at most k * k, as in datasketches-java The bytes overload checks the input holds the full compactor header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bytes overload of deserialize() read the unused 8-byte field of serial version 1 images without checking that the input holds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test relied on a real sketch shrinking its raw section size to 3. Setting the header tests the same image without depending on how far the section size shrinks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 42304902839Coverage increased (+0.06%) to 82.461%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
tisonkun
approved these changes
Nov 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings these commits from the 5.3.x branch to master, so development continues on the same checks that are in 5.3.1:
deserialize()checksk, the number of levels and the levels array.deserialize()checkskand the compactor headers.One commit is new here. The REQ test for the smallest section size now sets the
k= 6 header directly, so it no longer depends on how far a sketch's section size shrinks.Tested with all 17 test suites under ASan and UBSan.
🤖 Generated with Claude Code