Skip to content

DOC-6996 Document the go-redis pipeline connection pool [PARKED] - #3854

Open
andy-stark-redis wants to merge 2 commits into
mainfrom
DOC-6996-go-redis-pipeline-pool
Open

DOC-6996 Document the go-redis pipeline connection pool [PARKED]#3854
andy-stark-redis wants to merge 2 commits into
mainfrom
DOC-6996-go-redis-pipeline-pool

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Documents the dedicated pipeline connection pool that
go-redis#3959 makes default-on.

Adds a ### Connection pooling section to the go-redis Production usage page — the Go
guide had no pooling prose anywhere before this — and points the two pipeline pages at it:

  • content/develop/clients/go/produsage.md — new section, plus a checklist entry. Also fixes a
    pre-existing broken anchor in that checklist (#seamless-client-experience never matched the
    "Smart client handoffs" heading).
  • content/develop/clients/go/autopipeline.md — the paragraph that listed the three pipeline
    fields inline now links to the new section.
  • content/develop/clients/go/transpipe.md — one sentence noting that hand-built pipelines and
    transactions use the pool too.

The ticket proposed expanding the autopipeline paragraph instead. That was the wrong home:
Pipeline() and TxPipeline() both route through withPipelineConn, so the pool serves
hand-built pipelines and MULTI/EXEC as well as automatic pipelining, and autopipeline.md
carries an "experimental feature" banner that would have wrongly colored default-on behavior.

Warning

DO NOT MERGE. #3959 is merged to master but not in any release. v9.22.0 shipped
2026-08-03; #3959 merged 2026-08-24. On v9.22.0 there is no pipeline pool by default, so this
section describes behavior no reader can observe yet. The version line is a deliberate
vX.Y.Z placeholder.

Park manifest

Ticket: DOC-6996
Parked at: 2026-08-25
Trigger to pick up: go-redis merge commit bd0cea4 is an ancestor of a non-prerelease go-redis tag (a beta does not satisfy this)
Labels: parked, do not merge yet

Pinned sources (state observed at park time)

Source State at park time Re-fetch
redis/go-redis#3959 — "chore(pipeline): Create pipeline pool even without specific buffer passed" state: closed, merged: true, merged_at: 2026-08-24T11:42:41Z, head_sha: ae9488a650b7b4b23f7b67b71da7bb0431bb62e4, merge_commit_sha: bd0cea4ea87b81452618aa0704a808cd7b129215, base: master, milestone: null gh api repos/redis/go-redis/pulls/3959 --jq '{state, merged, head_sha: .head.sha, base: .base.ref, milestone: .milestone.title, merge_commit_sha}'
go-redis releases — newest non-prerelease is v9.22.0 (2026-08-03), which predates the merge master is 15 commits ahead of v9.22.0; no tag contains bd0cea4 gh api repos/redis/go-redis/releases --jq '.[0:6][] | "\(.tag_name)\t\(.published_at)\t\(.prerelease)"'

Trigger test (mechanical). Base is master, so the release is cut from the branch the PR
landed on — no integration-branch indirection to watch:

gh api "repos/redis/go-redis/compare/bd0cea4ea87b81452618aa0704a808cd7b129215...<tag>" \
  --jq '{status, ahead_by, behind_by}'

At park time, against v9.22.0: {"status":"behind","ahead_by":0,"behind_by":15}. The trigger is
met when a non-prerelease tag returns behind_by: 0.

Observed shape the page assumes

Semantics — confidence HIGH. Unusually high for a parked page, because #3959 is already
merged and these were verified at runtime, not read off a diff. Compiled and ran the documented
snippet against go-redis v9.22.1-0.20260824140911-18837034a1b9 (master, post-merge) and Redis
8.8.0:

Claim How verified
Pipeline pool exists by default, no option set RunPoolStats().PipelineStats non-nil
Idle pipeline pool holds zero connections RunTotalConns=0 before any pipeline
A plain command uses only the main pool Run — pipeline TotalConns stayed 0, main went to 1
Pipeline() and TxPipeline() both use the pool Run — pipeline pool hits/misses moved for both
PipelinePoolSize: -1 opts out RunPipelineStats == nil, pipeline ran on main pool
Default size is 10 Run — 60 concurrent pipelines capped TotalConns at 10
Saturation waits, then runs on the main pool Run — forced with PipelinePoolSize: 1 + BLPOP batches: Timeouts=5, 5 batches on the main pool. Note a plain 60-pipeline burst did not reproduce this (Timeouts=0) — fast batches never hold a connection past the 100 ms wait
Not released: no pool by default on v9.22.0 Run (negative check)PipelineStats nil by default and nil with PipelinePoolSize: 10 alone, since the released version only builds the pool when a buffer field is set
Buffers default to max(regular, 64 KiB); RESP3 minimum clamp Source onlypipelinePoolOptions, redis.go:628
MinIdleConns forced to 0; MaxActiveConns not inherited; ceiling = MaxActiveConns + PipelinePoolSize Source onlypipelinePoolOptions, and Options.init at options.go:629-644
Limiter charged once across the fallback Source onlywithPipelineConn, redis.go:1268-1292
Per-node pool and folded PipelineStats for ClusterClient / Ring Source only — nodes built via clOpt.NewClient() (osscluster.go:518) with PipelinePoolSize passed through (osscluster.go:481); folding at osscluster.go:1548, ring.go:707

Identifiers — confidence MEDIUM. Merged rather than in-review, so less exposed than the
usual parked page, but DOC-6832 is the precedent: on that page the semantics survived to release
while nearly every name had moved. Tick these off one at a time; locations are given because
"never existed here" and "renamed" look identical in a snapshot.

Public, named on the page:

  • Options.PipelinePoolSizeoptions.go:278
  • Options.PipelineReadBufferSizeoptions.go:227
  • Options.PipelineWriteBufferSizeoptions.go:246
  • Options.PoolSize, MinIdleConns, MaxActiveConns, PoolTimeout, ReadBufferSize, WriteBufferSize — pre-existing, named in the new prose
  • PoolStats.PipelineStats — declared on internal/pool.Stats (internal/pool/pool.go), surfaced through type PoolStats pool.Stats; assigned in Client.PoolStats() at redis.go:2356-2361
  • Stats.TotalConns / .Hits / .Misses / .Timeoutsinternal/pool/pool.go, used in the snippet
  • Limiter — pre-existing public type, named in the fallback paragraph

Values quoted as bare numbers in the prose and table (each must be re-read at the shipping tag):

  • DefaultPipelinePoolSize = 10options.go:481
  • DefaultPipelineBufferSize = 64 KiB (64 * 1024) — options.go:490
  • DefaultPipelinePoolTimeout = 100 msoptions.go:504

Internal, deliberately not named on the page but load-bearing for its claims — if these move,
the behavior described may have changed even when public names are stable:

  • pipelinePoolOptionsredis.go:628
  • withPipelineConnredis.go:1217
  • the if opt.PipelinePoolSize >= 0 gate in NewClientredis.go, and mirrored in Options.init at options.go:634
  • isPipelinePoolConn CSC guard — redis.go:948

Re-check checklist

Highest risk first.

  • Replace the vX.Y.Z placeholder in produsage.md with the shipping version and delete the <!--DOC-6996: ...--> HTML comment. The page is wrong as written until this is done.
  • Re-read the three default values (DefaultPipelinePoolSize 10, DefaultPipelineBufferSize 64 KiB, DefaultPipelinePoolTimeout 100 ms) at the shipping tag. They are quoted as bare numbers in both the prose and the table, so a tuned default silently falsifies the page.
  • Re-run the runtime probe against the released tag, not master: default pool non-nil, idle TotalConns 0, both Pipeline() and TxPipeline() on the pool, PipelinePoolSize: -1 → nil, burst capped at the default size, and the forced-saturation fallback (PipelinePoolSize: 1 + BLPOP, expect Timeouts > 0 and the main pool used).
  • Confirm the snippet still compiles with if ps := stats.PipelineStats; ps != nil. PipelineStats is *internal/pool.Stats, so inference is the only form available to a reader outside the module — if the type is ever exported, the snippet can be simplified but does not have to be.
  • Verify the source-only rows above that no runtime check covered: the 64 KiB buffer resolution and RESP3 clamp, MinIdleConns forced to 0, MaxActiveConns not inherited, the MaxActiveConns + PipelinePoolSize ceiling, and the Limiter-charged-once claim. (from Gaps:)
  • Verify the per-node claims for ClusterClient and Ring against a real cluster — that each node gets a pool and that PipelineStats aggregates. Read from source only; the page tells readers to budget the connection ceiling per node, which is the costliest thing on the page to get wrong.
  • Check whether the URL query params (pipeline_pool_size, pipeline_read_buffer_size, pipeline_write_buffer_size, options.go:941-943) are still deferred by choice. Excluded here because documenting them pulls in ParseURL coverage the Go guide has never had.
  • Re-check the client-side caching decision. Pipeline connections skip CLIENT TRACKING (redis.go:942-949), but pipelined commands did not consult or populate the cache on v9.22.0 either, so connect.md was left alone. If the shipping release changes the cached-command path, that call needs revisiting.
  • Confirm produsage.md should still have no page-level bannerText. Deliberate: the page's Health checks, Retries and Timeouts sections are long-released, and a page banner would mislabel them. The version requirement is stated in-section instead.

Upstream report worth filing

osscluster.go:148 still documents the pipeline pool as "created only when
PipelineReadBufferSize or PipelineWriteBufferSize is set", which is stale after #3959 and
contradicts ring.go:154. Not blocking this PR — the docs follow the code, not the comment — but
worth a note to go-redis.

On unpark, then

Run /unpark <this PR>. It reconciles the docs against the settled source and takes the PR
through the normal /reflect/finalize pipeline. /finalize is deferred until then — the
episodic Recheck: / Gaps: / Directive: trailers on this branch's commit are what unpark
reads, and finalizing now would squash them away. The do not merge yet guard holds until
/finalize completes.


Note

Low Risk
Documentation-only changes with no runtime or security impact; the only caveat is the parked placeholder version until go-redis releases the pipeline pool behavior.

Overview
Adds go-redis production guidance for connection pooling, including the default-on pipeline connection pool from go-redis PR #3959, and wires the pipeline docs to that single section instead of repeating option names inline.

On Production usage (produsage.md), a new Connection pooling checklist item and section explain the main pool (PoolSize, MinIdleConns, MaxActiveConns, PoolTimeout) and the separate pipeline pool (PipelinePoolSize, buffer sizes), fallback to the main pool under saturation, per-node behavior for cluster/ring, and monitoring via PoolStats().PipelineStats. A version note uses a vX.Y.Z placeholder until the feature ships in a release. The checklist link for smart client handoffs is corrected from a broken #seamless-client-experience anchor to #smart-client-handoffs.

Automatic pipelining and Pipelines/transactions now briefly state that batches use the pipeline pool and link to the new section rather than listing the three Pipeline* fields in autopipeline alone.

Reviewed by Cursor Bugbot for commit 48490fe. Bugbot is set up for automated code reviews on this repo. Configure here.

Add a Connection pooling section to the go-redis production usage page
covering the dedicated pipeline pool that go-redis PR #3959 makes
default-on, and point the two pipeline pages at it.

The ticket framed this as expanding the autopipeline paragraph at lines
157-160. That turned out to be the wrong home. Pipeline() and
TxPipeline() both route through withPipelineConn (redis.go:1753-1765,
1807), so the new pool serves hand-built pipelines and MULTI/EXEC too,
not just automatic pipelining -- and autopipeline.md carries an
"experimental feature" banner that would have wrongly colored
default-on pooling behavior. produsage.md already owns Timeouts and
Retries, so sizing advice belongs there; the Go guide had no pooling
prose at all before this.

The ticket's "Not parked: the change is already merged upstream" is a
merged-vs-released conflation. v9.22.0 shipped 2026-08-03, #3959 merged
2026-08-24, master is 15 commits ahead of the tag and no release
contains it. Ran the negative check to be sure rather than reasoning
from dates: on v9.22.0, PoolStats().PipelineStats is nil by default AND
stays nil with PipelinePoolSize:10 alone, because the released version
only builds the pool when a buffer field is set. So the whole section
describes behavior no reader can observe yet, and the version line is a
deliberate vX.Y.Z placeholder with an HTML TODO.

Compiling the PipelineStats snippet was necessary but not sufficient.
Running it against master + Redis 8.8 is what confirmed the claims:
pipeline pool non-nil and holding zero connections when idle, a plain
command touching only the main pool, both Pipeline() and TxPipeline()
using the pipeline pool, PipelinePoolSize:-1 making PipelineStats nil,
and 60 concurrent pipelines capping the pool at 10. The fallback claim
did NOT reproduce on that burst (Timeouts stayed 0, main pool
untouched) -- fast batches never hold a connection past the 100ms wait.
It took PipelinePoolSize:1 plus BLPOP batches to force it: Timeouts=5
and five batches on the main pool. DEBUG SLEEP is unavailable on the
local server, so BLPOP on a never-populated key is the way to hold a
pipeline connection open.

Also fixed a pre-existing broken anchor in the same checklist:
#seamless-client-experience never matched the "Smart client handoffs"
heading.

Deliberately not done: no connect.md change for client-side caching.
Pipeline connections now skip CLIENT TRACKING (redis.go:942-949), but
pipelined commands never consulted or populated the cache in v9.22.0
either, so nothing a reader can observe changed. URL query params
deferred by decision -- documenting them drags in ParseURL coverage the
Go guide has never had.

Upstream doc bug worth reporting: osscluster.go:148 still says the pool
is created "only when PipelineReadBufferSize or PipelineWriteBufferSize
is set", stale after #3959 and contradicting ring.go:154. Cluster nodes
are built via clOpt.NewClient() (osscluster.go:518) with
PipelinePoolSize passed through, so they do get pipeline pools -- which
is also why the ceiling arithmetic multiplies per node.

Learned: merged != released, and a saturation claim needs saturation forced -- a 60-pipeline burst never triggered the main-pool fallback that PipelinePoolSize:1 plus BLPOP did.
Constraint: the PipelineStats snippet must keep `if ps := stats.PipelineStats; ps != nil` -- the field is *internal/pool.Stats, so type inference is the only form that compiles outside the module, not a style preference.
Constraint: produsage.md carries no page-level bannerText; the pipeline pool's version requirement is stated in-section so the released Health checks, Retries and Timeouts sections are not mislabeled as unreleased.
Rejected: expanding autopipeline.md lines 157-160 as the ticket suggested | files default-on pooling under that page's experimental-feature banner and leaves hand-built Pipeline()/TxPipeline() readers with nothing
Gaps: the 64 KiB buffer defaults and the RESP3 minimum clamp are read from pipelinePoolOptions, not observed at runtime; the Limiter-charged-once behavior is likewise source-only.
Recheck: replace the vX.Y.Z placeholder and delete the DOC-6996 HTML comment in produsage.md when a non-prerelease go-redis tag contains bd0cea4.
Directive: verify DefaultPipelinePoolSize (10), DefaultPipelineBufferSize (64 KiB) and DefaultPipelinePoolTimeout (100ms) against the shipping tag before merging -- all three are quoted as bare numbers in the prose and table.
Ticket: DOC-6996
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andy-stark-redis andy-stark-redis added do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead. labels Aug 25, 2026
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

DOC-6996

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

🧠 Redis Memory

Found 6 related items from repository history (1 new this commit):

Memory updated at 48490fe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant