You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The test_async_chunk_transform_matches_sync[bb] test was comparing
raw gzip-encoded bytes between async and sync paths. Since gzip
embeds a wall-clock timestamp in compressed output, two compressions
of the same data at different times produce different bytes even
though the data is identical.
The fix skips the raw byte comparison when a GzipCodec is present
in the codec chain. Round-trip fidelity is still verified by the
decode assertions that follow.
Thanks for this fix. I think skipping the comparison entirely isn't as good as comparing everything except the timestamp. You could add a new utility routine to the gzip module for taking a gzip encoded stream and returning a timestamp, remainder pair. We would then compare the remainder. That's my first idea but I'm sure there's an even better one.
a marginally better idea: add a private boolean function to the gzip module that compares two gzip streams for "everything but timestamp" equality. There's probably a name for this kind of comparison (comparing two outputs only on the basis of the parameters under control of the producers). internally, we can just compare bytestream a and bytestream b on everything except the four bytes starting at byte 4, which is the mtime field (if I am reading the gzip spec correctly).
Thanks for the suggestion. I updated the test to compare the gzip streams while ignoring only the 4-byte MTIME field, and added a focused test for the comparison.
❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
needs release notesAutomatically applied to PRs which haven't added release notes
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4254
The test_async_chunk_transform_matches_sync[bb] test was comparing
raw gzip-encoded bytes between async and sync paths. Since gzip
embeds a wall-clock timestamp in compressed output, two compressions
of the same data at different times produce different bytes even
though the data is identical.
The fix skips the raw byte comparison when a GzipCodec is present
in the codec chain. Round-trip fidelity is still verified by the
decode assertions that follow.