Repository navigation
Fix CONNECT adapters, net-http drift, and CI - #43
Draft
proxymeshai wants to merge 2 commits into
Draft
proxymeshai wants to merge 2 commits into
proxymeshai wants to merge 2 commits into
Conversation
HTTP.rb, RestClient, and Typhoeus send HTTPS CONNECT headers through RubyProxyHeaders::NetHTTP.request. The Net::HTTP patch delegates connect to the installed net-http so timeouted_connect and HTTPS proxies stay intact, and Faraday uses public adapter hooks. Specs cover a local mock proxy and net-http 0.3.2 through 0.10.0. CI adds Ruby 3.4 and a weekly latest-gems workflow. Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>
WebMock replaces Net::HTTP with a subclass created before patch!. A one-time prepend stuck to whichever class was current, so Ruby 3.4 lost proxy_connect_request_headers= after WebMock.disable!. patch! now installs the extension on both classes. Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. Auto-merge is off. No tag and no repository settings change.
1. HTTP.rb, RestClient, and Typhoeus
RubyProxyHeaders::HTTPGem,RestClient, andTyphoeuscalledRubyProxyHeaders::NetHTTP.request, which did not exist, so every HTTPS request raisedNoMethodError.NetHTTP.request/get/post/put/deletenow perform the HTTPS request on patchedNet::HTTPand return a response whoseproxy_response_headersare case-insensitive. All three adapters use that path for every HTTPS URL, including when no custom request headers are set, so the CONNECT response is readable. Non-HTTPS still uses the native client.Specs:
spec/connect_header_injection_spec.rb,spec/mock_proxy_integration_spec.rb, and the live harness cases intest/test_proxy_headers.rb. The mock proxy (test/mock_connect_proxy.py) echoesX-ProxyMesh-*request headers. Each example asserts the echo header arrived on CONNECT and thatX-ProxyMesh-IPis readable. Covered clients: Net::HTTP, HTTP.rb, RestClient, Typhoeus, Faraday, and HTTParty.2. Net::HTTP connect no longer copies upstream
The global patch no longer replaces
Net::HTTP#connectwith a copy of the pre-0.9 method. A thin prepend validates the CONNECT target and header values before any socket I/O, then callssuper. Upstream keepstimeouted_connectand@proxy_use_ssl.Header injection prepends
Net::BufferedIO#writeand only rewrites a buffer that starts withCONNECT. Response capture prependsNet::HTTPResponse.read_newand records headers only whileconnectis on the stack, so the origin response is not stored as the proxy response.https://proxy URLs setproxy_use_sslwhen that writer exists (net-http >= 0.5) and raiseArgumentErrorotherwise.WebMock replaces
Net::HTTPwith a subclass it created beforepatch!. A one-shot prepend stayed on whichever class was current, so afterWebMock.disable!Ruby 3.4 raisedNoMethodErrorforproxy_connect_request_headers=.patch!now installs the extension on both the original class and WebMock's stand-in.spec/net_http_compat_spec.rbruns against the installed net-http. Locally, on Ruby 3.2.3:timeouted_connectproxy_use_ssl)The TLS proxy checks that the first bytes are a TLS handshake (
0x16), thatX-ProxyMesh-Countryis inside the CONNECT request, and that the origin body andX-ProxyMesh-IPcome back. After the CONNECT response, net-http starts the origin handshake on the raw TCP socket (ruby/net-http#212); the lab proxy forwards those bytes instead of decrypting them.CI job
net-httpinunit_tests.ymlrepeats this on Ruby 3.2/3.3/3.4 with the versions above viagemfiles/net_http_matrix.gemfile.3. Faraday
FaradayAdapter::NetHttpno longer overrides privaterequest_with_wrapped_block(private in faraday-net_http 3.4 and 3.5). It uses publicnet_http_connectionandcall, then merges CONNECT response headers intoenv[:response_headers]with||=.spec/faraday_adapter_spec.rbasserts the private method is inherited and not redefined. The mock-proxy example runs on faraday-net_http 3.5.0 and readsX-ProxyMesh-IPand the country echo fromresponse.headers.4. Metadata and docs
Gemspec metadata:
homepage_uri,changelog_uri,bug_tracker_uri,rubygems_mfa_required. Packaged files arelib/,LICENSE,README.md,CHANGELOG.md,docs/**/*.md, and.yardopts.IMPLEMENTATION_PRIORITY.md,LIBRARY_RESEARCH.md, andDEFERRED.mdstay in the repo and are not in the gem.README covers HTTP.rb, Typhoeus, and RestClient, moves the version badge under the H1, and uses absolute links including the other language repos. Docs and marketing text install
gem 'ruby-proxy-headers'(the require path staysruby_proxy_headers). Faraday, HTTParty, and Excon guides now matchFaradayIntegration,ProxyHeadersConnectionAdapter, andExconIntegration.Owner settings, not changed from this PR. Suggested description:
Suggested topics:
net-http,connect,tunnel,excon,typhoeus.5. CI
unit_tests.ymlmatrix is Ruby 3.2, 3.3, and 3.4, plus the net-http job above.runs-on: ubuntu-24.04.latest_client_gems.yml: Mondays 06:00 UTC andworkflow_dispatch, Ruby 3.2/3.3/3.4,ubuntu-24.04, deletesGemfile.lock,bundle update, thenbundle exec rspec(includes the mock-proxy specs). NoPROXY_URLsecret..readthedocs.yamlis unchanged.Tests run here
Ruby 3.2.3, bundler 2.7.2. Full suite: 45 examples, 0 failures, 6 pending. The pending examples are the pre-existing
Connection.parse_proxy_url/build_auth_headergaps. Compat specs passed on net-http 0.3.2, 0.4.1, 0.5.0, 0.6.0, 0.9.1, and 0.10.0. Ruby 3.3 and 3.4 were not installed in this environment; those matrix rows run in CI. The first CI run failedrspec (3.4)on the WebMock class swap; that is fixed in the follow-up commit.Left as-is
0.2.3. No tag.:ssl_proxy_headersand still cannot read CONNECT response headers.CURLOPT_PROXYHEADERis still deferred; HTTPS goes throughNet::HTTP.Connection#parse_proxy_urlandbuild_auth_headerare still unimplemented (docs/core-api.mdstill describes that string constructor).docs/net-http.mdand the HTTP.rb / Typhoeus / RestClient guides match the wrappers; they were not rewritten beyond the Gemfile name.scheduleandworkflow_dispatchonly, so it does not run on this pull request.