Skip to content

support http_proxy(https_proxy) - #964

Open
shinespb wants to merge 1 commit into
smithy-lang:developfrom
shinespb:feat/respect-http_proxy
Open

shinespb wants to merge 1 commit into
smithy-lang:developfrom
shinespb:feat/respect-http_proxy

Conversation

@shinespb

Copy link
Copy Markdown

Issue #, if available: n/a

Summary

Adds HTTP proxy support to AWSCRTHTTPClient. The client now respect the standard HTTP_PROXY, HTTPS_PROXY, NO_PROXY environment variables by default.

Why?

Most tools respect HTTP_PROXY, HTTPS_PROXY and NO_PROXY which can be useful in corporate environments.
Problem example: aws/aws-sdk-python#18

Description of changes:

smithy_http.interfaces

  • HTTPProxyConfig: a new frozen dataclass with http_proxy, https_proxy
    and no_proxy.
    • Proxy URLs are parsed once in __post_init__ using the format scheme://[user[:password]@]host[:port]. If the scheme is missing it defaults to http, and if the port is missing it defaults to 80/443 based on the proxy's scheme. Percent-encoded credentials are decoded.
    • Invalid URLs (unsupported scheme such as socks5, bad port, missing host) raise SmithyHTTPError. Error messages never include credentials.
    • from_environment() reads the lowercase variable first and falls back to uppercase, treating empty values as unset. An unusable proxy URL in the environment is logged and ignored rather than raised, so a defined
      HTTPS_PROXY=socks5h://... meant for other tools doesn't stop the client from being created.
    • proxy_for(destination) picks the proxy by destination scheme and returns None when the destination should connect directly.
    • NO_PROXY supports *, domain suffixes (a leading dot is optional), host:port entries that match only that port, and bracketed IPv6 entries. Matching is case-insensitive.
    • __repr__ should not show passwords.

smithy_http.aio.crt

  • AWSCRTHTTPClientConfig gains proxy: HTTPProxyConfig | None. When it is None, the client calls HTTPProxyConfig.from_environment().
  • The proxy config is resolved once in __init__ and passed to the copies made in __deepcopy__. The environment isn't re-read for every operation (configs are deep-copied per operation).
  • New connections get CRT HttpProxyOptions:
    • https destinations use tunneling (CONNECT), so TLS stays end-to-end with the destination. http destinations use forwarding.
    • https:// proxy URLs get their own TLS connection options, with SNI set to the proxy host.
    • Credentials in the proxy URL enable Basic proxy auth.

Behaviour change

Clients created without an explicit proxy now read the proxy environment variables. Anyone who has HTTP(S)_PROXY set but relied on smithy-python ignoring it will now send traffic through that proxy. To disable that, pass AWSCRTHTTPClientConfig(proxy=HTTPProxyConfig()) or set NO_PROXY.

Tests

  • tests/unit/test_proxy.py (new) covers URL parsing, scheme selection, NO_PROXY matching, reading from the environment (case handling, empty values, unusable URLs), and keeping credentials out of repr, error messages and logs.
  • tests/unit/aio/test_crt.py checks the HttpProxyOptions passed to CRT for tunneling vs forwarding, Basic auth, NO_PROXY bypass, reading from the environment, and that an unusable environment proxy doesn't stop the client from being created. An autouse fixture keeps the developer's own proxy variables out of the tests.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@shinespb
shinespb requested a review from a team as a code owner November 18, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant