client cache: group-level configuration (MultiGroupOptions.ClientCache) and Prefixes as RedisKey

design/client-side-cache.md

@@ -1,4 +1,4 @@
-# Client-side caching in v4: the design as built
+# Client-side caching in v4: the design as built
 
 Status: current as of 2026-10-12 (`a22ba168`; shipped in `4.0.127-alpha`). This is the record of **why** the cache is
 the way it is - each mechanism with what was decided, what was rejected, and the evidence - gathered from the
@@ -186,6 +186,16 @@ invalidation, validating against a key that changed); the cross-index.
   The rule is that the cache is never staler than the member it answers for. Server invalidations and flushes stay
   per member. Rejected: flushing on every switch - with equal weights a group picks by latency, so the active
   member can change on any health pass, and a flush would throw away a healthy member's cache each time.
+- **In a group the cache is configured once, on `MultiGroupOptions.ClientCache` (2026-10-27), and a member's own
+  `ConfigurationOptions.ClientCache` is refused.** Members hold separate caches and the group answers from the
+  active one, so members that cached differently - or one that did not cache at all - would change what callers
+  get on every failover; inheriting one definition is how they agree, and allowing a per-member override would be
+  allowing them to disagree. Plumbed like the group's circuit breaker: alongside the configuration, never written
+  into it (a sentinel member gets a clone, since its cache is built with the connection). A sentinel connection
+  itself never builds a cache. `CacheOptions.Prefixes` is now `IReadOnlyList<RedisKey>` rather than strings -
+  keys are bytes, and a prefix that is not valid UTF-8 had no spelling - rendered to a private byte snapshot
+  once (a key's whole rendering, context prefix included), which both the matcher and the handshake's `PREFIX`
+  arguments read; nothing of the caller's list or arrays is kept, and the getter hands out copies.
 - **An inactive member is kept honest exactly as an active one is.** Group members always run
   `HeartbeatConsistencyChecks`, so each endpoint is pinged every heartbeat whether or not it is active; a half-open
   tracking connection trips the dead-socket rule (commands timing out, nothing received for 4x the timeout), and

docs/ClientSideCaching.md

@@ -37,7 +37,7 @@ Requirements:
 
 The cache is configured in code only - it is not part of the connection string, because its settings are correctness choices that deserve to be read before they are set.
 
-There is **one cache per multiplexer**, shared by every database, every `IDatabase`, and every context made from it.
+There is **one cache per multiplexer**, shared by every database, every `IDatabase`, and every context made from it. On a [connection group](Failover) each member has a cache of its own and the options are set once, on `MultiGroupOptions.ClientCache` rather than on the members - see [Connection groups](#connection-groups).
 
 What it covers
 ---
@@ -106,7 +106,7 @@ Controlling it
 |---|---|---|
 | `Enabled` | `true` | Whether the cache exists. Setting `ClientCache` at all is what turns caching on; this lets you keep the options and switch it off. |
 | `TrackingMode` | `Default` (= `Tracking` today, or `TrackingByPrefix` when `Prefixes` are set) | How the server decides which changes to announce, and whether reads are cached by default (`Tracking`, `Broadcast`), only inside `Prefixes` (`TrackingByPrefix`), or only when a call asks (`TrackingOptIn`) - see below. |
-| `Prefixes` | none (all keys) | Which keys to cache. Setting any selects `TrackingByPrefix`, where the list decides for calls that pass no flag; under `Broadcast` it is also sent to the server to filter what is announced. |
+| `Prefixes` | none (all keys) | Which keys to cache, as `RedisKey` values: strings, raw bytes, or a key made with `WithKeyPrefix` (taken whole). Copied when set; changing your list afterwards changes nothing. Setting any selects `TrackingByPrefix`, where the list decides for calls that pass no flag; under `Broadcast` it is also sent to the server to filter what is announced. |
 | `MaxBytes` | unbounded | A memory budget for cached replies. Entries that have not been read since they were stored are evicted, oldest first, to get back under it - see [Eviction](#eviction). |
 | `MaxEntries` | unbounded | A limit on the number of entries - use it alongside `MaxBytes` when replies are small and numerous. |
 | `MaxPayloadBytes` | 1 MiB | The largest single reply that may be cached. |
@@ -193,7 +193,7 @@ Under `MaxBytes`, replies are stored in large slabs in the order they arrive and
 ### Tracking modes and prefixes
 
 - **`Tracking`** (the default without `Prefixes`): the server remembers which keys this client read and announces only those. Precise - invalidation traffic scales with what this client holds, not with how busy the server is - but the server holds a table per client, and evicts from it (announcing what it drops) under pressure, which costs a refill here. `Prefixes` cannot be combined with it. Negotiated as `CLIENT TRACKING ON OPTOUT`: every read is tracked and cacheable unless a call says otherwise with `CommandFlags.CacheOptOut`, which also tells the server not to track that read (a `CLIENT CACHING NO` goes out as one unit with it).
-- **`TrackingByPrefix`** (the default when `Prefixes` are set): `Prefixes` decide for the calls that pass no flag. A read whose keys all fall under a prefix is treated as if it had said `CommandFlags.CacheOptIn`: it goes out behind `CLIENT CACHING YES`, the server tracks it, and it is stored on its first miss. Any other read goes to the server as it would without a cache. The flags still win: `CacheOptIn` on a read outside the prefixes caches it, `CacheOptOut` on a read inside them does not. Negotiated as `CLIENT TRACKING ON OPTIN`; the list is **not** sent to the server (the server only accepts `PREFIX` with `BCAST`), so the server tracks exactly the keys this client holds. The price is an extra frame and reply per miss inside the prefixes, which is the traffic the cache exists to shrink. Prefixes must not overlap one another (`"app:"` and `"app:user:"` together are rejected), and they are compared with the key as sent - so if you use `WithKeyPrefix`, include that prefix.
+- **`TrackingByPrefix`** (the default when `Prefixes` are set): `Prefixes` decide for the calls that pass no flag. A read whose keys all fall under a prefix is treated as if it had said `CommandFlags.CacheOptIn`: it goes out behind `CLIENT CACHING YES`, the server tracks it, and it is stored on its first miss. Any other read goes to the server as it would without a cache. The flags still win: `CacheOptIn` on a read outside the prefixes caches it, `CacheOptOut` on a read inside them does not. Negotiated as `CLIENT TRACKING ON OPTIN`; the list is **not** sent to the server (the server only accepts `PREFIX` with `BCAST`), so the server tracks exactly the keys this client holds. The price is an extra frame and reply per miss inside the prefixes, which is the traffic the cache exists to shrink. Prefixes are `RedisKey` values, so a byte prefix is as good as a string one, and a key made with `WithKeyPrefix` can be given as it is. They must not overlap one another (`"app:"` and `"app:user:"` together are rejected), and they are compared with the key as sent - so if you build keys with `WithKeyPrefix`, the prefix must include that.
 - **`TrackingOptIn`**: opt-in by flag alone. Negotiated as `CLIENT TRACKING ON OPTIN`: the server tracks nothing, and nothing is cached, unless a call says so with `CommandFlags.CacheOptIn` - that read goes out behind `CLIENT CACHING YES`, is tracked, and is stored on its first miss. Every other read goes to the server as it would without a cache. For "cache these few hot keys and leave everything else alone" when the keys are not a prefix; the price is an extra frame and an extra reply per opted-in miss. `Prefixes` cannot be combined with it - that is `TrackingByPrefix`.
 - **`Broadcast`**: the server announces every change to every key matching your `Prefixes`, whoever made it, and keeps no record of what you read. Cheap for the server; the cost is on the client, which hears about keys it never cached, and that cost is set by how busy the server is rather than by anything this client does - which is why prefixes no longer select this mode by default. **Set `Prefixes`** if you choose it: with none, this connection hears about every write to every key in the instance. Keys outside the prefixes are simply not cached, and here the list is a hard boundary: the server was never asked about those keys, so no flag can opt one in. The per-call flags keep only their local meaning: the server has no per-client table to opt in to or out of.
 
@@ -223,6 +223,26 @@ These behave as a write from this process does: the entries are dropped at once,
 
 The same object carries the cache's counters - `Count`, `Bytes`, `Stored`, `Evicted`, `Promoted`, `Expired`, `Refreshes`, `ServedStale` - cumulative since the connection was made. There is no hit counter, because a hit deliberately records nothing; see [Is it working?](#is-it-working) for the measurement that cannot be fooled.
 
+### Connection groups
+
+With a [connection group](Failover) (geo-failover, Active-Active), the cache is configured once, on `MultiGroupOptions.ClientCache`, and every member inherits it; a member whose own `ConfigurationOptions.ClientCache` is set is refused when the group connects. Each member still needs `Protocol = RedisProtocol.Resp3`.
+
+```csharp
+MultiGroupOptions options = new MultiGroupOptions.Builder
+{
+    ClientCache = new CacheOptions { Prefixes = ["app:"], MaxBytes = 64 * 1024 * 1024 },
+};
+await using var conn = await ConnectionMultiplexer.ConnectGroupAsync(members, options);
+```
+
+What that gives you:
+
+- **One cache per member, not one for the group.** A cached entry is only as good as the invalidation stream behind it, and that stream belongs to one member's connections. So each member keeps its own cache, and a read is served from, and stored into, the cache of whichever member is active at that moment. Nothing is copied or flushed when the active member changes: the member you left keeps its cache, still invalidated by its own server, ready for when you come back.
+- **Your own writes reach every member.** A write made while member B is active reaches A's server only by replication, so without help A's cache could, after a failback, hand you the value you had just replaced. To prevent that, a write from this process invalidates the key in every member's cache. A then misses and asks its server, which may still be catching up; that is ordinary replication lag, and an uncached read would see the same. Server-sent invalidations, and the flush that follows a lost connection, stay with their own member.
+- **The members must agree.** That is why the setting lives on the group: a member that cached differently, or not at all, would change what callers get on every failover.
+
+`GetClientCache()` on a group returns the active member's cache at the time of the call. `Invalidate` and `InvalidateAll` through it reach every member, as a write does; its counters describe that member only.
+
 ### Per call, and per context
 
 - **`CommandFlags.CacheOptOut`** - this call neither reads from nor writes to the cache, and in `Tracking` mode the server is told not to track it:

docs/Configuration.md

@@ -161,7 +161,7 @@ Additional code-only options:
 - HealthCheck (`HealthCheck`) - Default: `null`
   - **Client-side geographic failover.** An *active* health check used when the connection is a member of a connection group; when `null`, the group-level `MultiGroupOptions.HealthCheck` is used. See [Client-side geographic failover](Failover)
 - ClientCache (`CacheOptions`) - Default: `null`
-  - Enables client-side caching of reads, kept current by server-assisted invalidation; requires RESP3. `null` means no cache. See [Client-side caching](ClientSideCaching)
+  - Enables client-side caching of reads, kept current by server-assisted invalidation; requires RESP3. `null` means no cache. Not for a connection group member: there the cache is set once, on `MultiGroupOptions.ClientCache`, and a member that sets its own is refused. See [Client-side caching](ClientSideCaching)
 
 Tokens in the configuration string are comma-separated; any without an `=` sign are assumed to be redis server endpoints. Endpoints without an explicit port will use 6379 if ssl is not enabled, and 6380 if ssl is enabled.
 Tokens starting with `$` are taken to represent command maps, for example: `$config=cfg`.

docs/Failover.md

@@ -86,10 +86,22 @@ Because the policies are immutable, there is no question of whether a change "ta
 | `FailbackHealthCheck` | `MultiGroupOptions.FailbackHealthCheck` | `ConnectionGroupMember.FailbackHealthCheck` |
 | `Weight` | *(none)* | `ConnectionGroupMember.Weight` |
 | `SkipInitialHealthCheck` | *(none)* | `ConnectionGroupMember.SkipInitialHealthCheck` |
+| `ClientCache` | `MultiGroupOptions.ClientCache` | *(none; a member's `ConfigurationOptions.ClientCache` must be left unset, or the group refuses it)* |
 
 Resolution is always "member override, else group default". Resolving a group default never writes back into your `ConfigurationOptions` - you can safely reuse one `ConfigurationOptions` for a group member and for an unrelated `Connect` without the group's policies leaking across.
 
 `ConfigurationOptions` carries only the two availability settings that are meaningful for a **single** connection - `CircuitBreaker` and `RetryPolicy`. `HealthCheck` is a group-only concept and lives on `ConnectionGroupMember`.
+
+[Client-side caching](ClientSideCaching) is the one setting that goes the other way: on a group it is set **only** at the group level, and every member inherits it. Each member keeps a cache of its own (the invalidations that keep it honest arrive on that member's connections), and the group serves reads from whichever member is active, so members must cache alike or a failover would change what callers get. A member whose own `ConfigurationOptions.ClientCache` is set is refused. Every member needs `Protocol = RedisProtocol.Resp3`, as any cached connection does.
+
+```csharp
+MultiGroupOptions options = new MultiGroupOptions.Builder
+{
+    ClientCache = new CacheOptions { Prefixes = ["app:"], MaxBytes = 64 * 1024 * 1024 },
+};
+```
+
+See [Connection groups](ClientSideCaching#connection-groups) in the caching documentation for what happens to the caches across a failover.
 The library automatically selects the best available endpoint based on:
 
 1. **Availability** - Connected endpoints are always preferred over disconnected ones

src/StackExchange.Redis/Availability/MultiGroupMultiplexer.cs

@@ -550,7 +550,7 @@ internal static async Task<IConnectionGroup> ConnectAsync(
 
                     // note the resolved circuit-breaker is passed *alongside* the configuration rather than
                     // written into it; see ConnectionMultiplexer.GroupCircuitBreaker
-                    pending[i] = ConnectionMultiplexer.ConnectGroupMemberAsync(config, log, members[i].ResolveCircuitBreaker(options));
+                    pending[i] = ConnectionMultiplexer.ConnectGroupMemberAsync(config, log, members[i].ResolveCircuitBreaker(options), options.ClientCache);
                 }
 
                 for (int i = 0; i < pending.Length; i++)
@@ -593,6 +593,8 @@ internal static async Task<IConnectionGroup> ConnectAsync(
             /// </remarks>
             private static void ValidateConsistentConfiguration(ConnectionGroupMember[] members)
             {
+                for (var i = 0; i < members.Length; i++) DemandNoMemberCache(members[i], i);
+
                 if (members.Length < 2) return;
 
                 var first = members[0].Configuration;
@@ -621,6 +623,27 @@ private static void ValidateConsistentConfiguration(ConnectionGroupMember[] memb
                 static string Describe(int? database) => database is { } value ? value.ToString() : "unset";
             }
 
+            /// <summary>
+            /// A member's cache comes from the group, never from the member: refuse a member that brings its own.
+            /// </summary>
+            /// <remarks>
+            /// Each member holds a cache of its own, and the group answers reads from the active member's, so a
+            /// member that cached differently from the others - or not at all - would change what callers get on
+            /// every failover. Inheriting one <see cref="MultiGroupOptions.ClientCache"/> is how members agree;
+            /// allowing a per-member override would be allowing them to disagree.
+            /// </remarks>
+            private static void DemandNoMemberCache(ConnectionGroupMember member, int index)
+            {
+                if (member.Configuration.ClientCache is not null)
+                {
+                    throw new ArgumentException(
+                        $"ConfigurationOptions.ClientCache must not be set on a connection group member (member {index}, "
+                        + $"'{member.Name}'); set MultiGroupOptions.ClientCache instead, which every member inherits. "
+                        + "The group serves reads from whichever member is active, so members must cache alike.",
+                        nameof(member));
+                }
+            }
+
             private readonly MultiGroupOptions _options;
 
             public MultiGroupOptions Options => _options;
@@ -1550,11 +1573,12 @@ private void OnConnectionChanged(
             public async Task AddAsync(ConnectionGroupMember member, TextWriter? log = null)
             {
                 // connect
+                DemandNoMemberCache(member, _members.Length);
                 member.Init(_members.Length);
                 member.Configuration.AbortOnConnectFail = false; // members are gated by health-checks, not connect-fail
                 member.Configuration.HeartbeatConsistencyChecks = true;
                 var muxer = await ConnectionMultiplexer.ConnectGroupMemberAsync(
-                    member.Configuration, log, member.ResolveCircuitBreaker(_options)).ConfigureAwait(false);
+                    member.Configuration, log, member.ResolveCircuitBreaker(_options), _options.ClientCache).ConfigureAwait(false);
                 member.SetMultiplexer(muxer);
 
                 // unless told otherwise, run an initial health-check so a healthy member can be selected immediately;