Fix stale nginx upstream keepalive connections causing broken pipe errors#674
Open
WeiQuan0605 wants to merge 1 commit into
Open
Conversation
…rors Add keepalive_timeout=15s to nginx upstream block so nginx evicts idle connections before Puma closes them (persistent_timeout=20s), preventing writev() EPIPE errors on stale connections.
stephanme
reviewed
Jul 22, 2026
|
|
||
| upstream cloud_controller { | ||
| server unix:/var/vcap/data/cloud_controller_ng/cloud_controller.sock; | ||
| keepalive 32; |
Member
There was a problem hiding this comment.
The default was:
keepalive 32 local;
Not sure if we want to change this (removing local). But if we want to change the value, then we would maybe set it to the number of parallel requests that ccng can handle (puma workers * threads per worker).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A short explanation of the proposed change:
Add keepalive_timeout=15s to nginx upstream block so nginx evicts idle connections before Puma closes them (persistent_timeout=20s), preventing writev() EPIPE errors on stale connections.
An explanation of the use cases your change solves
This change solves the case where nginx reuses an idle upstream connection to Puma that Puma has already closed. Setting keepalive_timeout=15s, shorter than Puma's persistent_timeout=20s, ensures nginx always evicts idle connections from its pool before Puma closes them. The stale connection window is eliminated entirely.
Links to any other associated PRs
I have viewed signed and have submitted the Contributor License Agreement
I have made this pull request to the
developbranchI have run CF Acceptance Tests on bosh lite