Skip to content

fix(egress): bound idle CONNECT tunnels after hijack - #591

Open
walsykerem wants to merge 3 commits into
kernel:mainfrom
walsykerem:egress-tunnel-leak-problem
Open

walsykerem wants to merge 3 commits into
kernel:mainfrom
walsykerem:egress-tunnel-leak-problem

Conversation

@walsykerem

@walsykerem walsykerem commented Nov 21, 2026 •

Copy link
Copy Markdown

The egress proxy terminates HTTPS by hijacking each client socket and then starting its own TLS handshake. Once a connection was hijacked, the HTTP server's 15 second header timeout stopped applying, and the handshake had no deadline at all. A client that sent a CONNECT and then went quiet kept that goroutine, its file descriptor and its handshake buffers for as long as it held the socket open. Forty eight silent CONNECTs measured 48 stuck goroutines.

The fix gives the setup phase a 3 second bound. The deadline is set right after the 200 response and cleared once the handshake succeeds. The request loop arms a 3 second idle deadline on each pass, so a tunnel that stops sending is closed instead of held. Writing one response gets a separate 60 second bound, which leaves slow but genuine transfers working. Nothing else moved: refusals, routing and header injection behave as before.

Same test after the change: 48 tunnels, zero extra goroutines after five seconds. A long lived stream is only reaped when it actually goes quiet past the bound.


Note

Medium Risk
Touches core CONNECT/TLS proxy I/O timing; mis-tuned deadlines could drop legitimate slow clients or leave edge cases, but scope is localized to hijacked tunnels.

Overview
Fixes a goroutine and connection leak on hijacked HTTPS CONNECT tunnels, where the egress proxy no longer inherits http.Server’s read header timeout after Hijack.

The CONNECT path now enforces 3s idle bounds on TLS setup (deadline until handshake completes), on each wait for the next tunneled request, and on request-body reads via a new connectBodyReader that refreshes the deadline as bytes arrive. Response writes get a separate 60s write deadline so slow but active transfers are not cut off by the idle limit.

Refusals, upstream routing, and header injection are unchanged; only post-hijack I/O timing behavior is new.

Reviewed by Cursor Bugbot for commit 97507d7. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/egressproxy/service.go Resolved
The post-hijack read deadline that bounds the tunnel header read stayed in
force during RoundTrip, so a request body streamed over more than 3s failed
even while the client was actively uploading. Refresh the deadline on every
successful body read and keep the 3s bound when no data arrives, so idle and
stalled tunnels are still reaped.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9eadabd. Configure here.

Comment thread lib/egressproxy/service.go Resolved
Comment thread lib/egressproxy/service.go Resolved
The bound was armed before the origin connection was established, so a slow
dial or TLS handshake could expire it before the first body read; arm it when
each body read starts instead. Wrapping http.NoBody also turned bodyless
requests into chunked uploads, so leave it unwrapped.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant