fix: close getObject response stream for buffering transformers - #524
Merged
Merged
Conversation
| * the response stream: buffering transformers do not need the connection left open (so the client | ||
| * closes the stream and avoids the leak), while streaming transformers do (the caller closes it). | ||
| */ | ||
| public class S3EncryptionClientGetObjectStreamCloseTest { |
Contributor
There was a problem hiding this comment.
This doesn't actually test the updated getObject method, no? Just properties about the SDK classes themselves? So I don't think this would actually fail without the fix
| // materialized result, so the stream must be closed here to release its buffers. Streaming | ||
| // transformers (e.g. toInputStream) return the stream for the caller to read and close. | ||
| if (!responseTransformer.needsConnectionLeftOpen()) { | ||
| joinFutureGet.close(); |
Contributor
There was a problem hiding this comment.
This doesn't close the stream if some exception is thrown in the try block, let's move this to a finally block
S3EncryptionClient.getObject never closed the ResponseInputStream from the toBlockingInputStream() pipeline, leaking Netty direct memory on every call with a buffering transformer (getObjectAsBytes, toFile). Close the stream when the transformer does not need the connection left open; streaming transformers (toInputStream) still return the stream for the caller to close. Fixes #518.
sharmabikram
force-pushed
the
shbikram/getobject-stream-leak
branch
from
September 26, 2026 22:55
42f58b4 to
55d84e2
Compare
Replace the SDK-contract test with one that drives S3EncryptionClient.getObject through a real S3AsyncClient backed by an in-memory transport, so it fails without the stream close: a transformer that stops reading early or throws must release the response stream, while toInputStream leaves it open for the caller. Log instead of throw when closing the stream fails, so a close failure does not mask the transformed result or the original exception.
ShubhamChaturvedi7
approved these changes
Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
close getObject response stream for buffering transformers. Fixes #518.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Check any applicable: