Skip to content

Free the compression context when Zstd.compress raises - #162

Open
sribalakumar wants to merge 1 commit into
SpringMT:mainfrom
sribalakumar:fix-compress-ctx-leak
Open

sribalakumar wants to merge 1 commit into
SpringMT:mainfrom
sribalakumar:fix-compress-ctx-leak

Conversation

@sribalakumar

Copy link
Copy Markdown

This fixes the compress half of #160; the decompress half is handled separately by #148. It mirrors #148's approach, applied to the compress side.

Changes:

  1. Zstd.compress now runs its body under rb_ensure, so the ZSTD_CCtx is freed on every path — normal return, argument errors, and interrupts.
  2. set_compress_params and convert_compression_level used to free the context themselves before raising. With an rb_ensure also owning it that would double-free, so they now just raise and leave the context to its owner. (convert_compression_level's now-unused ctx parameter is dropped.)
  3. StreamingCompress#initialize now assigns sc->ctx before calling set_compress_params, so the object's free callback owns the context if it raises.

New specs (6 examples):

In spec/zstd-ruby_spec.rb, under "when it raises after creating the compression context":

  • raises ArgumentError for an unknown keyword
  • raises RangeError for a level that does not fit in an int
  • raises ArgumentError for a dict: that is neither a CDict nor a String
  • can be interrupted by Thread#raise while compressing

In spec/zstd-ruby-streaming-compress_spec.rb, under "initialize raising after creating the compression context":

  • raises ArgumentError for an unknown keyword
  • raises ArgumentError for a dict: that is neither a CDict nor a String

Leaks aren't visible from Ruby, so these specs pass both before and after this change on their own — what turns a leak into a failing test is the repo's rake spec:valgrind. That's how the before/after below was produced.

Before/after: rake spec:valgrind

Linux, Docker, ruby 3.4, run on just these 6 examples (SPEC_OPTS='-e "creating the compression context"'), "before" being current main's C code plus the new specs.

Before: 6 examples, 0 failures, but Valgrind fails the task with three "definitely lost" records:

5,288 bytes in 1 blocks — ZSTD_createCCtx <- rb_streaming_compress_initialize (streaming_compress.c:90)
10,576 bytes in 2 blocks — ZSTD_createCCtx <- rb_compress (zstdruby.c:19)
85,196,958 bytes (5,288 direct, 85,191,670 indirect) in 1 blocks — ZSTD_createCCtx <- rb_compress (zstdruby.c:19)   [the interrupted compression]

...followed by: "Valgrind reported errors (e.g. memory leak or use-after-free)"

After: 6 examples, 0 failures, Valgrind clean, task exits 0.

Before/after: macOS leaks --atExit

RUBY_FREE_AT_EXIT=1, 20 calls per scenario (baseline from Ruby itself is 1 leak / 192 bytes):

scenario before after
Zstd.compress level 19 of 32 MiB, interrupted 39 leaks / 1,533,665,472 bytes baseline
Zstd.compress unknown keyword 21 leaks / 123,072 bytes baseline
Zstd.compress level: 2**40 21 leaks / 123,072 bytes baseline
StreamingCompress.new(unknown: 1) 21 leaks / 123,072 bytes baseline
Zstd.compress / StreamingCompress.new with dict: 123 baseline baseline, and no double free

Full suite: 97 examples, 0 failures.

This composes with #148: the two branches merge cleanly, and with both applied the suite passes (99 examples, 0 failures) with every compress and decompress leak scenario back to baseline. With both applied, the full rake spec:valgrind run is also clean — 99 examples, 0 failures, no Valgrind errors — so the Valgrind job that has been failing on main since #152 would go green.

No API or behaviour change — the same inputs raise the same errors as before.

Zstd.compress frees its ZSTD_CCtx only after compression returns
normally, so any exception raised in between abandons the context. That
includes interrupts: zstd_compress releases the GVL, and when it
reacquires it Ruby delivers pending interrupts (Thread#raise, Timeout,
Thread#kill, Sidekiq shutdown) by raising out of the call. An interrupted
level-19 compression of a 32 MiB input leaks roughly 75-85 MB per call.

The same applies to argument errors raised after the context exists: an
unknown keyword or a level that does not fit in an int leaked the
context from both Zstd.compress and StreamingCompress#initialize.

Run the body of Zstd.compress under rb_ensure so the context is freed on
every path. set_compress_params and convert_compression_level used to
free the context themselves before raising; with an ensure also owning
it that would be a double free, so they now raise and leave it to the
owner. StreamingCompress#initialize therefore assigns sc->ctx before
calling set_compress_params, so the object's free callback owns it.

This mirrors the decompression-side change in SpringMT#148.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant