Skip to content

fix: Close the connection before reporting a stream error, and keep the reconnect loop alive - #95

Merged
jsonbailey merged 1 commit into
mainfrom
jb/gem-retry-hardening
Oct 15, 2026
Merged

jsonbailey merged 1 commit into
mainfrom
jb/gem-retry-hardening

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

BEGIN_COMMIT_OVERRIDE
fix: Close the connection before passing a stream error to the error handler
fix: Pass at most one error to the error handler for each connection attempt
fix: Keep reconnecting when closing the previous connection fails
fix: Prevent a reconnect delay of NaN when the retry interval is zero
END_COMMIT_OVERRIDE

Summary

Fixes found while reviewing the Ruby server SDK's RETRY work (ruby-server-sdk#448). That SDK waits inside on_error before it reconnects, which exposed these paths in the reconnect loop:

  • Close before reporting. run_stream called on_error before reset_http. A handler that blocks kept the dead connection open for the whole wait, for example after a read timeout. The connection is now closed first.
  • One error per attempt. A failure in reset_http was sent to on_error as a second error for the same attempt. It is now logged at warn.
  • The worker cannot stop silently. close_connection at the top of the loop was outside any rescue, so a raise ended the worker thread and the client never reconnected. It is now logged and the loop continues.
  • No NaN in the backoff. With a Float 0.0 base (for example after a server retry: 0), 0.0 * 2**n became NaN after about 1024 attempts, and [NaN, max].min raised on every later attempt. A zero base now returns no delay, and the attempt count stops growing at the maximum.
  • Two comments named StreamingHTTPConnection, which was removed in 2.0.0. They now name HTTP::Response.

No API change.

Testing

  • New specs for each change. Each fails when its fix is reverted.
  • bundle exec rspec spec: 182 examples, 0 failures. bundle exec rubocop: no offenses.

Note

Overview
Hardens the SSE client reconnect loop so cleanup and error reporting happen in a safer order, and fixes backoff edge cases that could break long-running reconnects.

In run_stream, the connection is now closed (reset_http) before on_error runs, so blocking error handlers do not keep a dead socket open. Close failures are handled via log_close_errors: they are logged only (not sent to on_error), and close_connection at the start of each loop is wrapped the same way so a close raise no longer kills the worker thread. Backoff#next_interval returns 0 when the base or max interval is non-positive (avoids NaN after many attempts with a 0.0 base from retry: 0), and stops incrementing the attempt counter once the delay hits the max ceiling.

New specs cover close-before-error ordering, a single error per attempt when close fails, worker survival, and backoff behavior. Comments now refer to HTTP::Response instead of the removed StreamingHTTPConnection. No public API changes.

Reviewed by Cursor Bugbot for commit dedf9f8. Bugbot is set up for automated code reviews on this repo. Configure here.

…he reconnect loop alive

- Close the failed connection before calling on_error, so a handler that
  blocks does not hold the dead connection open.
- Log a failure to close a connection instead of reporting a second error
  for the same attempt, and do not let it stop the worker thread.
- Return no delay for a zero base interval and stop counting attempts at
  the maximum, so the backoff can no longer compute NaN.
@jsonbailey
jsonbailey merged commit 4b59dcc into main Oct 15, 2026
13 checks passed
@jsonbailey
jsonbailey deleted the jb/gem-retry-hardening branch October 15, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants