DOC-6996 Document the go-redis pipeline connection pool [PARKED] - #3854
Open
andy-stark-redis wants to merge 2 commits into
Open
DOC-6996 Document the go-redis pipeline connection pool [PARKED]#3854andy-stark-redis wants to merge 2 commits into
andy-stark-redis wants to merge 2 commits into
Conversation
Add a Connection pooling section to the go-redis production usage page covering the dedicated pipeline pool that go-redis PR #3959 makes default-on, and point the two pipeline pages at it. The ticket framed this as expanding the autopipeline paragraph at lines 157-160. That turned out to be the wrong home. Pipeline() and TxPipeline() both route through withPipelineConn (redis.go:1753-1765, 1807), so the new pool serves hand-built pipelines and MULTI/EXEC too, not just automatic pipelining -- and autopipeline.md carries an "experimental feature" banner that would have wrongly colored default-on pooling behavior. produsage.md already owns Timeouts and Retries, so sizing advice belongs there; the Go guide had no pooling prose at all before this. The ticket's "Not parked: the change is already merged upstream" is a merged-vs-released conflation. v9.22.0 shipped 2026-08-03, #3959 merged 2026-08-24, master is 15 commits ahead of the tag and no release contains it. Ran the negative check to be sure rather than reasoning from dates: on v9.22.0, PoolStats().PipelineStats is nil by default AND stays nil with PipelinePoolSize:10 alone, because the released version only builds the pool when a buffer field is set. So the whole section describes behavior no reader can observe yet, and the version line is a deliberate vX.Y.Z placeholder with an HTML TODO. Compiling the PipelineStats snippet was necessary but not sufficient. Running it against master + Redis 8.8 is what confirmed the claims: pipeline pool non-nil and holding zero connections when idle, a plain command touching only the main pool, both Pipeline() and TxPipeline() using the pipeline pool, PipelinePoolSize:-1 making PipelineStats nil, and 60 concurrent pipelines capping the pool at 10. The fallback claim did NOT reproduce on that burst (Timeouts stayed 0, main pool untouched) -- fast batches never hold a connection past the 100ms wait. It took PipelinePoolSize:1 plus BLPOP batches to force it: Timeouts=5 and five batches on the main pool. DEBUG SLEEP is unavailable on the local server, so BLPOP on a never-populated key is the way to hold a pipeline connection open. Also fixed a pre-existing broken anchor in the same checklist: #seamless-client-experience never matched the "Smart client handoffs" heading. Deliberately not done: no connect.md change for client-side caching. Pipeline connections now skip CLIENT TRACKING (redis.go:942-949), but pipelined commands never consulted or populated the cache in v9.22.0 either, so nothing a reader can observe changed. URL query params deferred by decision -- documenting them drags in ParseURL coverage the Go guide has never had. Upstream doc bug worth reporting: osscluster.go:148 still says the pool is created "only when PipelineReadBufferSize or PipelineWriteBufferSize is set", stale after #3959 and contradicting ring.go:154. Cluster nodes are built via clOpt.NewClient() (osscluster.go:518) with PipelinePoolSize passed through, so they do get pipeline pools -- which is also why the ceiling arithmetic multiplies per node. Learned: merged != released, and a saturation claim needs saturation forced -- a 60-pipeline burst never triggered the main-pool fallback that PipelinePoolSize:1 plus BLPOP did. Constraint: the PipelineStats snippet must keep `if ps := stats.PipelineStats; ps != nil` -- the field is *internal/pool.Stats, so type inference is the only form that compiles outside the module, not a style preference. Constraint: produsage.md carries no page-level bannerText; the pipeline pool's version requirement is stated in-section so the released Health checks, Retries and Timeouts sections are not mislabeled as unreleased. Rejected: expanding autopipeline.md lines 157-160 as the ticket suggested | files default-on pooling under that page's experimental-feature banner and leaves hand-built Pipeline()/TxPipeline() readers with nothing Gaps: the 64 KiB buffer defaults and the RESP3 minimum clamp are read from pipelinePoolOptions, not observed at runtime; the Limiter-charged-once behavior is likewise source-only. Recheck: replace the vX.Y.Z placeholder and delete the DOC-6996 HTML comment in produsage.md when a non-prerelease go-redis tag contains bd0cea4. Directive: verify DefaultPipelinePoolSize (10), DefaultPipelineBufferSize (64 KiB) and DefaultPipelinePoolTimeout (100ms) against the shipping tag before merging -- all three are quoted as bare numbers in the prose and table. Ticket: DOC-6996 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Contributor
Contributor
🧠 Redis MemoryFound 6 related items from repository history (1 new this commit):
Memory updated at 48490fe |
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the dedicated pipeline connection pool that
go-redis#3959 makes default-on.
Adds a
### Connection poolingsection to the go-redis Production usage page — the Goguide had no pooling prose anywhere before this — and points the two pipeline pages at it:
content/develop/clients/go/produsage.md— new section, plus a checklist entry. Also fixes apre-existing broken anchor in that checklist (
#seamless-client-experiencenever matched the"Smart client handoffs" heading).
content/develop/clients/go/autopipeline.md— the paragraph that listed the three pipelinefields inline now links to the new section.
content/develop/clients/go/transpipe.md— one sentence noting that hand-built pipelines andtransactions use the pool too.
The ticket proposed expanding the autopipeline paragraph instead. That was the wrong home:
Pipeline()andTxPipeline()both route throughwithPipelineConn, so the pool serveshand-built pipelines and MULTI/EXEC as well as automatic pipelining, and
autopipeline.mdcarries an "experimental feature" banner that would have wrongly colored default-on behavior.
Warning
DO NOT MERGE. #3959 is merged to
masterbut not in any release. v9.22.0 shipped2026-08-03; #3959 merged 2026-08-24. On v9.22.0 there is no pipeline pool by default, so this
section describes behavior no reader can observe yet. The version line is a deliberate
vX.Y.Zplaceholder.Park manifest
Ticket: DOC-6996
Parked at: 2026-08-25
Trigger to pick up: go-redis merge commit
bd0cea4is an ancestor of a non-prerelease go-redis tag (a beta does not satisfy this)Labels:
parked,do not merge yetPinned sources (state observed at park time)
state: closed,merged: true,merged_at: 2026-08-24T11:42:41Z,head_sha: ae9488a650b7b4b23f7b67b71da7bb0431bb62e4,merge_commit_sha: bd0cea4ea87b81452618aa0704a808cd7b129215,base: master,milestone: nullgh api repos/redis/go-redis/pulls/3959 --jq '{state, merged, head_sha: .head.sha, base: .base.ref, milestone: .milestone.title, merge_commit_sha}'masteris 15 commits ahead of v9.22.0; no tag containsbd0cea4gh api repos/redis/go-redis/releases --jq '.[0:6][] | "\(.tag_name)\t\(.published_at)\t\(.prerelease)"'Trigger test (mechanical). Base is
master, so the release is cut from the branch the PRlanded on — no integration-branch indirection to watch:
At park time, against v9.22.0:
{"status":"behind","ahead_by":0,"behind_by":15}. The trigger ismet when a non-prerelease tag returns
behind_by: 0.Observed shape the page assumes
Semantics — confidence HIGH. Unusually high for a parked page, because #3959 is already
merged and these were verified at runtime, not read off a diff. Compiled and ran the documented
snippet against
go-redis v9.22.1-0.20260824140911-18837034a1b9(master, post-merge) and Redis8.8.0:
PoolStats().PipelineStatsnon-nilTotalConns=0before any pipelineTotalConnsstayed 0, main went to 1Pipeline()andTxPipeline()both use the poolPipelinePoolSize: -1opts outPipelineStats == nil, pipeline ran on main poolTotalConnsat 10PipelinePoolSize: 1+BLPOPbatches:Timeouts=5, 5 batches on the main pool. Note a plain 60-pipeline burst did not reproduce this (Timeouts=0) — fast batches never hold a connection past the 100 ms waitPipelineStatsnil by default and nil withPipelinePoolSize: 10alone, since the released version only builds the pool when a buffer field is setpipelinePoolOptions,redis.go:628MinIdleConnsforced to 0;MaxActiveConnsnot inherited; ceiling =MaxActiveConns + PipelinePoolSizepipelinePoolOptions, andOptions.initatoptions.go:629-644Limitercharged once across the fallbackwithPipelineConn,redis.go:1268-1292PipelineStatsforClusterClient/RingclOpt.NewClient()(osscluster.go:518) withPipelinePoolSizepassed through (osscluster.go:481); folding atosscluster.go:1548,ring.go:707Identifiers — confidence MEDIUM. Merged rather than in-review, so less exposed than the
usual parked page, but DOC-6832 is the precedent: on that page the semantics survived to release
while nearly every name had moved. Tick these off one at a time; locations are given because
"never existed here" and "renamed" look identical in a snapshot.
Public, named on the page:
Options.PipelinePoolSize—options.go:278Options.PipelineReadBufferSize—options.go:227Options.PipelineWriteBufferSize—options.go:246Options.PoolSize,MinIdleConns,MaxActiveConns,PoolTimeout,ReadBufferSize,WriteBufferSize— pre-existing, named in the new prosePoolStats.PipelineStats— declared oninternal/pool.Stats(internal/pool/pool.go), surfaced throughtype PoolStats pool.Stats; assigned inClient.PoolStats()atredis.go:2356-2361Stats.TotalConns/.Hits/.Misses/.Timeouts—internal/pool/pool.go, used in the snippetLimiter— pre-existing public type, named in the fallback paragraphValues quoted as bare numbers in the prose and table (each must be re-read at the shipping tag):
DefaultPipelinePoolSize= 10 —options.go:481DefaultPipelineBufferSize= 64 KiB (64 * 1024) —options.go:490DefaultPipelinePoolTimeout= 100 ms —options.go:504Internal, deliberately not named on the page but load-bearing for its claims — if these move,
the behavior described may have changed even when public names are stable:
pipelinePoolOptions—redis.go:628withPipelineConn—redis.go:1217if opt.PipelinePoolSize >= 0gate inNewClient—redis.go, and mirrored inOptions.initatoptions.go:634isPipelinePoolConnCSC guard —redis.go:948Re-check checklist
Highest risk first.
vX.Y.Zplaceholder inprodusage.mdwith the shipping version and delete the<!--DOC-6996: ...-->HTML comment. The page is wrong as written until this is done.DefaultPipelinePoolSize10,DefaultPipelineBufferSize64 KiB,DefaultPipelinePoolTimeout100 ms) at the shipping tag. They are quoted as bare numbers in both the prose and the table, so a tuned default silently falsifies the page.TotalConns0, bothPipeline()andTxPipeline()on the pool,PipelinePoolSize: -1→ nil, burst capped at the default size, and the forced-saturation fallback (PipelinePoolSize: 1+BLPOP, expectTimeouts> 0 and the main pool used).if ps := stats.PipelineStats; ps != nil.PipelineStatsis*internal/pool.Stats, so inference is the only form available to a reader outside the module — if the type is ever exported, the snippet can be simplified but does not have to be.MinIdleConnsforced to 0,MaxActiveConnsnot inherited, theMaxActiveConns + PipelinePoolSizeceiling, and the Limiter-charged-once claim. (fromGaps:)ClusterClientandRingagainst a real cluster — that each node gets a pool and thatPipelineStatsaggregates. Read from source only; the page tells readers to budget the connection ceiling per node, which is the costliest thing on the page to get wrong.pipeline_pool_size,pipeline_read_buffer_size,pipeline_write_buffer_size,options.go:941-943) are still deferred by choice. Excluded here because documenting them pulls inParseURLcoverage the Go guide has never had.CLIENT TRACKING(redis.go:942-949), but pipelined commands did not consult or populate the cache on v9.22.0 either, soconnect.mdwas left alone. If the shipping release changes the cached-command path, that call needs revisiting.produsage.mdshould still have no page-levelbannerText. Deliberate: the page's Health checks, Retries and Timeouts sections are long-released, and a page banner would mislabel them. The version requirement is stated in-section instead.Upstream report worth filing
osscluster.go:148still documents the pipeline pool as "created only whenPipelineReadBufferSizeorPipelineWriteBufferSizeis set", which is stale after #3959 andcontradicts
ring.go:154. Not blocking this PR — the docs follow the code, not the comment — butworth a note to go-redis.
On unpark, then
Run
/unpark <this PR>. It reconciles the docs against the settled source and takes the PRthrough the normal
/reflect→/finalizepipeline./finalizeis deferred until then — theepisodic
Recheck:/Gaps:/Directive:trailers on this branch's commit are what unparkreads, and finalizing now would squash them away. The
do not merge yetguard holds until/finalizecompletes.Note
Low Risk
Documentation-only changes with no runtime or security impact; the only caveat is the parked placeholder version until go-redis releases the pipeline pool behavior.
Overview
Adds go-redis production guidance for connection pooling, including the default-on pipeline connection pool from go-redis PR #3959, and wires the pipeline docs to that single section instead of repeating option names inline.
On Production usage (
produsage.md), a new Connection pooling checklist item and section explain the main pool (PoolSize,MinIdleConns,MaxActiveConns,PoolTimeout) and the separate pipeline pool (PipelinePoolSize, buffer sizes), fallback to the main pool under saturation, per-node behavior for cluster/ring, and monitoring viaPoolStats().PipelineStats. A version note uses avX.Y.Zplaceholder until the feature ships in a release. The checklist link for smart client handoffs is corrected from a broken#seamless-client-experienceanchor to#smart-client-handoffs.Automatic pipelining and Pipelines/transactions now briefly state that batches use the pipeline pool and link to the new section rather than listing the three
Pipeline*fields in autopipeline alone.Reviewed by Cursor Bugbot for commit 48490fe. Bugbot is set up for automated code reviews on this repo. Configure here.