Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: n/a
Summary
Adds HTTP proxy support to
AWSCRTHTTPClient. The client now respect the standard HTTP_PROXY, HTTPS_PROXY, NO_PROXY environment variables by default.Why?
Most tools respect HTTP_PROXY, HTTPS_PROXY and NO_PROXY which can be useful in corporate environments.
Problem example: aws/aws-sdk-python#18
Description of changes:
smithy_http.interfacesHTTPProxyConfig: a new frozen dataclass withhttp_proxy,https_proxyand
no_proxy.__post_init__using the formatscheme://[user[:password]@]host[:port]. If the scheme is missing it defaults tohttp, and if the port is missing it defaults to 80/443 based on the proxy's scheme. Percent-encoded credentials are decoded.socks5, bad port, missing host) raiseSmithyHTTPError. Error messages never include credentials.from_environment()reads the lowercase variable first and falls back to uppercase, treating empty values as unset. An unusable proxy URL in the environment is logged and ignored rather than raised, so a definedHTTPS_PROXY=socks5h://...meant for other tools doesn't stop the client from being created.proxy_for(destination)picks the proxy by destination scheme and returnsNonewhen the destination should connect directly.NO_PROXYsupports*, domain suffixes (a leading dot is optional),host:portentries that match only that port, and bracketed IPv6 entries. Matching is case-insensitive.__repr__should not show passwords.smithy_http.aio.crtAWSCRTHTTPClientConfiggainsproxy: HTTPProxyConfig | None. When it isNone, the client callsHTTPProxyConfig.from_environment().__init__and passed to the copies made in__deepcopy__. The environment isn't re-read for every operation (configs are deep-copied per operation).HttpProxyOptions:httpsdestinations use tunneling (CONNECT), so TLS stays end-to-end with the destination.httpdestinations use forwarding.https://proxy URLs get their own TLS connection options, with SNI set to the proxy host.Behaviour change
Clients created without an explicit
proxynow read the proxy environment variables. Anyone who has HTTP(S)_PROXY set but relied on smithy-python ignoring it will now send traffic through that proxy. To disable that, passAWSCRTHTTPClientConfig(proxy=HTTPProxyConfig())or setNO_PROXY.Tests
tests/unit/test_proxy.py(new) covers URL parsing, scheme selection,NO_PROXYmatching, reading from the environment (case handling, empty values, unusable URLs), and keeping credentials out ofrepr, error messages and logs.tests/unit/aio/test_crt.pychecks theHttpProxyOptionspassed to CRT for tunneling vs forwarding, Basic auth,NO_PROXYbypass, reading from the environment, and that an unusable environment proxy doesn't stop the client from being created. An autouse fixture keeps the developer's own proxy variables out of the tests.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.