Skip to content

Fix CONNECT adapters, net-http drift, and CI - #43

Draft
proxymeshai wants to merge 2 commits into
mainfrom
cursor/fix-connect-headers-53ba
Draft

proxymeshai wants to merge 2 commits into
mainfrom
cursor/fix-connect-headers-53ba

Conversation

@proxymeshai

@proxymeshai proxymeshai commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Draft. Auto-merge is off. No tag and no repository settings change.

1. HTTP.rb, RestClient, and Typhoeus

RubyProxyHeaders::HTTPGem, RestClient, and Typhoeus called RubyProxyHeaders::NetHTTP.request, which did not exist, so every HTTPS request raised NoMethodError.

NetHTTP.request / get / post / put / delete now perform the HTTPS request on patched Net::HTTP and return a response whose proxy_response_headers are case-insensitive. All three adapters use that path for every HTTPS URL, including when no custom request headers are set, so the CONNECT response is readable. Non-HTTPS still uses the native client.

Specs: spec/connect_header_injection_spec.rb, spec/mock_proxy_integration_spec.rb, and the live harness cases in test/test_proxy_headers.rb. The mock proxy (test/mock_connect_proxy.py) echoes X-ProxyMesh-* request headers. Each example asserts the echo header arrived on CONNECT and that X-ProxyMesh-IP is readable. Covered clients: Net::HTTP, HTTP.rb, RestClient, Typhoeus, Faraday, and HTTParty.

2. Net::HTTP connect no longer copies upstream

The global patch no longer replaces Net::HTTP#connect with a copy of the pre-0.9 method. A thin prepend validates the CONNECT target and header values before any socket I/O, then calls super. Upstream keeps timeouted_connect and @proxy_use_ssl.

Header injection prepends Net::BufferedIO#write and only rewrites a buffer that starts with CONNECT . Response capture prepends Net::HTTPResponse.read_new and records headers only while connect is on the stack, so the origin response is not stored as the proxy response.

https:// proxy URLs set proxy_use_ssl when that writer exists (net-http >= 0.5) and raise ArgumentError otherwise.

WebMock replaces Net::HTTP with a subclass it created before patch!. A one-shot prepend stayed on whichever class was current, so after WebMock.disable! Ruby 3.4 raised NoMethodError for proxy_connect_request_headers=. patch! now installs the extension on both the original class and WebMock's stand-in.

spec/net_http_compat_spec.rb runs against the installed net-http. Locally, on Ruby 3.2.3:

net-http plaintext CONNECT + custom header timeouted_connect TLS-to-proxy round trip
0.3.2 pass skipped (socket opened inline) skipped (no proxy_use_ssl)
0.4.1 pass skipped skipped
0.5.0 pass skipped pass
0.6.0 pass skipped pass
0.9.1 pass pass pass
0.10.0 pass pass pass

The TLS proxy checks that the first bytes are a TLS handshake (0x16), that X-ProxyMesh-Country is inside the CONNECT request, and that the origin body and X-ProxyMesh-IP come back. After the CONNECT response, net-http starts the origin handshake on the raw TCP socket (ruby/net-http#212); the lab proxy forwards those bytes instead of decrypting them.

CI job net-http in unit_tests.yml repeats this on Ruby 3.2/3.3/3.4 with the versions above via gemfiles/net_http_matrix.gemfile.

3. Faraday

FaradayAdapter::NetHttp no longer overrides private request_with_wrapped_block (private in faraday-net_http 3.4 and 3.5). It uses public net_http_connection and call, then merges CONNECT response headers into env[:response_headers] with ||=. spec/faraday_adapter_spec.rb asserts the private method is inherited and not redefined. The mock-proxy example runs on faraday-net_http 3.5.0 and reads X-ProxyMesh-IP and the country echo from response.headers.

4. Metadata and docs

Gemspec metadata: homepage_uri, changelog_uri, bug_tracker_uri, rubygems_mfa_required. Packaged files are lib/, LICENSE, README.md, CHANGELOG.md, docs/**/*.md, and .yardopts. IMPLEMENTATION_PRIORITY.md, LIBRARY_RESEARCH.md, and DEFERRED.md stay in the repo and are not in the gem.

README covers HTTP.rb, Typhoeus, and RestClient, moves the version badge under the H1, and uses absolute links including the other language repos. Docs and marketing text install gem 'ruby-proxy-headers' (the require path stays ruby_proxy_headers). Faraday, HTTParty, and Excon guides now match FaradayIntegration, ProxyHeadersConnectionAdapter, and ExconIntegration.

Owner settings, not changed from this PR. Suggested description:

Send and read custom proxy headers on HTTPS CONNECT for Net::HTTP, Faraday, HTTParty, Excon, HTTP.rb, Typhoeus, and RestClient

Suggested topics: net-http, connect, tunnel, excon, typhoeus.

5. CI

  • unit_tests.yml matrix is Ruby 3.2, 3.3, and 3.4, plus the net-http job above. runs-on: ubuntu-24.04.
  • New latest_client_gems.yml: Mondays 06:00 UTC and workflow_dispatch, Ruby 3.2/3.3/3.4, ubuntu-24.04, deletes Gemfile.lock, bundle update, then bundle exec rspec (includes the mock-proxy specs). No PROXY_URL secret.
  • .readthedocs.yaml is unchanged.

Tests run here

Ruby 3.2.3, bundler 2.7.2. Full suite: 45 examples, 0 failures, 6 pending. The pending examples are the pre-existing Connection.parse_proxy_url / build_auth_header gaps. Compat specs passed on net-http 0.3.2, 0.4.1, 0.5.0, 0.6.0, 0.9.1, and 0.10.0. Ruby 3.3 and 3.4 were not installed in this environment; those matrix rows run in CI. The first CI run failed rspec (3.4) on the WebMock class swap; that is fixed in the follow-up commit.

Left as-is

  • Version stays 0.2.3. No tag.
  • Excon can send :ssl_proxy_headers and still cannot read CONNECT response headers.
  • Native Typhoeus CURLOPT_PROXYHEADER is still deferred; HTTPS goes through Net::HTTP.
  • Mechanize is still deferred.
  • Connection#parse_proxy_url and build_auth_header are still unimplemented (docs/core-api.md still describes that string constructor).
  • docs/net-http.md and the HTTP.rb / Typhoeus / RestClient guides match the wrappers; they were not rewritten beyond the Gemfile name.
  • The weekly latest-gems workflow is schedule and workflow_dispatch only, so it does not run on this pull request.
Open in Web Open in Cursor 

cursoragent and others added 2 commits October 23, 2026 04:08
HTTP.rb, RestClient, and Typhoeus send HTTPS CONNECT headers through
RubyProxyHeaders::NetHTTP.request. The Net::HTTP patch delegates connect
to the installed net-http so timeouted_connect and HTTPS proxies stay
intact, and Faraday uses public adapter hooks. Specs cover a local mock
proxy and net-http 0.3.2 through 0.10.0. CI adds Ruby 3.4 and a weekly
latest-gems workflow.

Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>
WebMock replaces Net::HTTP with a subclass created before patch!.
A one-time prepend stuck to whichever class was current, so Ruby 3.4
lost proxy_connect_request_headers= after WebMock.disable!. patch! now
installs the extension on both classes.

Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants