Skip to content

Keep the input in the result of Zstd.write_skippable_frame - #163

Open
sribalakumar wants to merge 1 commit into
SpringMT:mainfrom
sribalakumar:fix-write-skippable-frame-payload
Open

sribalakumar wants to merge 1 commit into
SpringMT:mainfrom
sribalakumar:fix-write-skippable-frame-payload

Conversation

@sribalakumar

Copy link
Copy Markdown

Fixes #161.

Change: write the skippable frame, then copy the input after it, and resize to frame size + input size. Also adds RB_GC_GUARD for both string arguments.

New specs (4 examples) in spec/zstd-skippable_frame_spec.rb, under describe 'write_skippable_frame':

  • returns the skippable frame followed by the input
  • keeps the compressed data decodable by Zstd.decompress
  • keeps the compressed data with a magic_variant (magic_variant: 15, checks both read_skippable_frame and Zstd.decompress)
  • accepts an empty input

Before

Current main, rspec spec/zstd-skippable_frame_spec.rb: 15 examples, 3 failures.

1) returns the skippable frame followed by the input
   expected: 204
        got: 19

2) keeps the compressed data decodable by Zstd.decompress
   RuntimeError: not a zstd frame (magic not found)

3) keeps the compressed data with a magic_variant
   RuntimeError: not a zstd frame (magic not found)

("accepts an empty input" passes before too, since there's no payload to lose.)

After

15 examples, 0 failures. Full suite: 95 examples, 0 failures.

The README example now works end to end: read_skippable_frame returns "sample data" and Zstd.decompress returns the original data.

Compatibility note

Anyone who worked around the old behaviour by concatenating the frame and the data themselves would now get the data twice, and Zstd.decompress would return the payload twice. If you'd rather not change behaviour, the alternative is to keep it as-is and correct the README instead — happy to switch to that approach if you'd prefer.

Separate thing noticed while testing (not fixed here)

While testing this, I noticed Zstd::StreamingDecompress#decompress returns an empty string when its input begins with a skippable frame — or an empty frame such as Zstd.compress("") — followed by real data; its loop stops at the first frame that produces no output. This is independent of this PR and also happens with a hand-built skippable frame, and Zstd.decompress and Zstd::StreamReader both handle it fine. Happy to open a separate issue for that if useful.

write_skippable_frame(input, data) allocated room for the input plus the
skippable frame, but returned only the frame: ZSTD_writeSkippableFrame
writes the frame from the start of the buffer and the result was then
truncated to the frame's size. The input was discarded, so the README
example's compressed_data_with_skippable_frame no longer contained the
compressed data, and Zstd.decompress on it raised "not a zstd frame".

Write the frame and then the input after it. A leading frame is what
read_skippable_frame expects, and Zstd.decompress skips skippable frames
to reach the data that follows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zstd.write_skippable_frame drops the input it's given

1 participant