Skip to content

Use forwarding proxy mode for plaintext endpoints in CRT S3 client - #7321

Open
MingWangSong wants to merge 1 commit into
aws:masterfrom
MingWangSong:fix-crt-plaintext-endpoint-proxy-forwarding
Open

Use forwarding proxy mode for plaintext endpoints in CRT S3 client#7321
MingWangSong wants to merge 1 commit into
aws:masterfrom
MingWangSong:fix-crt-plaintext-endpoint-proxy-forwarding

Conversation

@MingWangSong

Copy link
Copy Markdown

Motivation and Context

Fixes #7320.

When an HTTP proxy is configured, the CRT-based S3 client always establishes the proxy connection through a CONNECT tunnel, even when endpointOverride uses the plaintext http:// scheme. Proxies that only support forwarding mode — a common corporate configuration, since tunnels defeat traffic inspection — reject these requests with:

Failed to send the request: Proxy-based connection establishment failed because the CONNECT call failed

The emitted request is self-contradictory: the client sends CONNECT host:80, so it clearly knows the target is plaintext (hence port 80), yet still negotiates a tunnel as if TLS were in use.

This is inconsistent with the rest of the SDK. Given identical configuration:

HTTP client http:// endpoint https:// endpoint
ApacheHttpClient (sync S3Client) GET http://host/... CONNECT host:443
CRT (S3AsyncClient.crtBuilder()) CONNECT host:80 CONNECT host:443

It is also inconsistent with the CRT client's own behaviour without a proxy, where an http:// endpoint is correctly reached in plaintext.

Root cause. aws-c-http derives the proxy connection type from whether TLS options were supplied for the main connection:

AWS_HPCT_HTTP_LEGACY = 0,
/* If tls options are provided (for the main connection) then treat the proxy as a tunneling proxy
   If tls options are not provided (for the main connection), then treat the proxy as a forwarding proxy */

HttpProxyOptions defaults connectionType to Legacy and CrtConfigurationUtils.resolveProxy never overrides it, so that rule always applies. Meanwhile S3NativeClientConfiguration builds a TlsContext unconditionally and S3CrtAsyncHttpClient hands it to the native client unconditionally — so the main connection always "has TLS options", regardless of the endpoint scheme. In effect, "the client holds a TLS context" is being treated as "this connection uses TLS".

Note that aws-crt-java's own HttpClientConnectionManager already guards this correctly:

boolean useTls = HTTPS.equals(uri.getScheme());
...
useTls && tlsContext != null ? tlsContext.getNativeHandle() : 0,

Modifications

S3NativeClientConfiguration now pins the proxy connection type to Forwarding when endpointOverride uses the http scheme, mirroring the scheme check the CRT already performs elsewhere.

  • No new public API — HttpProxyConnectionType.Forwarding and HttpProxyOptions.setConnectionType() are already public in aws-crt-java.
  • Only affects the combination proxy configured and plaintext endpointOverride. Every other path is untouched: https:// endpoints, endpoints without an override, and configurations without a proxy all keep resolving through Legacy exactly as before.
  • tlsContext handling is deliberately left alone. Not passing the TlsContext was evaluated first and does not work — aws-c-s3 still tunnels. The connection type has to be set explicitly.

Testing

Unit tests added to S3NativeClientConfigurationTest:

  • build_proxyConnectionType_followsEndpointScheme — parameterized over http://, HTTP:// (case-insensitivity), https://, and no endpoint override, asserting Forwarding only for the plaintext cases.
  • build_whenPlaintextEndpointAndNoProxy_shouldNotSetProxyOptions — guards the null-proxy path.

End-to-end verification against a stub proxy that records the first line it receives:

connectionType tlsContext First line sent
Legacy (before this change) passed CONNECT host:80
Legacy not passed CONNECT host:80
Forwarding (after this change) passed GET http://host:80/...

Confirmed Forwarding combined with a non-null TlsContext works and does not trigger the "configuration error" mentioned in the AWS_HPCT_HTTP_FORWARD doc comment — that refers to TLS on the tunnel destination, not to the client holding a context.

Also verified that https:// endpoints continue to tunnel, and that requests without a proxy are unaffected.

Client-level verification against the stub proxy, before and after the change:

Client endpoint before after
CRT http:// CONNECT host:80 GET http://host:80/bucket?list-type=2
CRT https:// CONNECT host:443 CONNECT host:443 (unchanged)
Netty http:// CONNECT host:80 CONNECT host:80 (unchanged — out of scope for this PR)

Downstream verification. Also verified through a real downstream consumer — Alluxio's S3 under-filesystem connector, which constructs its CRT client via S3AV2UnderFileSystem.createAmazonS3CRTAsync(...) from mount-level configuration. Running that code path unchanged, only swapping the SDK jar:

SDK endpoint first line sent by the connector
2.54.4 (unpatched) http:// CONNECT host:80
2.54.5-SNAPSHOT (patched) http:// GET http://host:80/bucket?list-type=2
2.54.5-SNAPSHOT (patched) https:// CONNECT host:443 (unchanged)

mvn install for services/s3 and its dependencies succeeds, S3NativeClientConfigurationTest passes (9 tests), and checkstyle:check reports 0 violations.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@MingWangSong
MingWangSong requested a review from a team as a code owner August 26, 2026 15:41
The CRT derives the proxy connection type from whether TLS options were
supplied for the main connection, not from the scheme actually in use.
Because S3NativeClientConfiguration always builds a TlsContext and
S3CrtAsyncHttpClient passes it to the native client unconditionally, a
plaintext http:// endpoint was still reached through a CONNECT tunnel.
Proxies that only support forwarding mode reject those requests.

Pin the proxy connection type to Forwarding when endpointOverride uses the
http scheme, mirroring the scheme check that the CRT's own
HttpClientConnectionManager already performs before handing a TLS context
to the native layer. Only the combination of a configured proxy and a
plaintext endpoint override is affected; https endpoints, endpoints without
an override, and proxy-less configurations continue to resolve through
Legacy as before.

Not passing the TlsContext was evaluated first and does not work: aws-c-s3
still tunnels, so the connection type has to be set explicitly.

Fixes aws#7320
@MingWangSong
MingWangSong force-pushed the fix-crt-plaintext-endpoint-proxy-forwarding branch from 93d67b6 to e8e6e08 Compare August 27, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3 async clients (CRT and Netty) always use CONNECT tunneling with a proxy, even for plaintext http:// endpoints

1 participant