Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
0f3ae13
Toy server: support the maintenance-notification opt-in, and sending …
mgravell Aug 24, 2026
8109d4b
Maintenance notifications: opt in, and receive (D2) (#3193)
mgravell Sep 1, 2026
3d9b3a6
D8: document the notifications, and make the diagnostic it recommends…
mgravell Sep 1, 2026
b44132a
Ask for a moving-endpoint-type, and find the successor field was neve…
mgravell Sep 1, 2026
901f067
Derive moving-endpoint-type per connection
mgravell Sep 1, 2026
fb7160a
Use a named successor directly, instead of hoping DNS agrees
mgravell Sep 1, 2026
7d8f805
Default to deriving the moving-endpoint-type
mgravell Sep 1, 2026
c007915
Cover mTLS, which also proves the TLS half of the derivation
mgravell Sep 1, 2026
8f6577f
Apply the half-window reconnect rule where it is the only tool
mgravell Sep 2, 2026
22ebea4
Serialise the fault-injector tier, and stop bounding it with an HTTP …
mgravell Sep 2, 2026
b5e6061
Sweep the databases a killed scenario leaves behind
mgravell Sep 2, 2026
e9830ac
D9: a soak for the state that only misbehaves with repetition
mgravell Sep 2, 2026
c733b1c
Prove relaxed timeouts rescue a command, and attribute one that fails
mgravell Sep 2, 2026
3e975df
Measure how long a completion stays retained
mgravell Sep 2, 2026
d033d9c
Key the handoff count on distinct MOVING sequences, not one overall
mgravell Sep 2, 2026
e3ad81c
A catch-up completion opens no relaxed window
mgravell Sep 2, 2026
75fa3e5
Read the retention probe from the log, not the relaxed window
mgravell Sep 2, 2026
3572a70
Stop claiming parity with go-redis on the post-event tail
mgravell Sep 2, 2026
51d22f0
Say that the connect-time rule is completions only
mgravell Sep 2, 2026
acd8309
Retention measured to three hours, not ninety minutes
mgravell Sep 2, 2026
8a38780
Attribute a fault to a window that has just closed, and warn on a mis…
mgravell Sep 3, 2026
c246f9f
Do not raise an event for a replayed completion
mgravell Sep 3, 2026
7cc75fe
Run the destructive scenarios, supervised
mgravell Sep 3, 2026
b1aea16
Document the destructive gate in the tier README
mgravell Sep 3, 2026
366c9fe
Prove a refusing node is retired, and correct why it was in doubt
mgravell Sep 2, 2026
13bb7d3
Put the fault-injector tier in the solution
mgravell Sep 2, 2026
5d603ae
Re-read the topology when an endpoint only ever refuses
mgravell Sep 2, 2026
f0de7f4
One black-hole tunnel, not one per test class
mgravell Sep 3, 2026
956ce53
Document the connect-failure topology refresh
mgravell Sep 3, 2026
31c96d0
Merge main: keep both sets of unshipped public API entries
mgravell Sep 3, 2026
fa6b032
Docs: lead with the server-native notifications, and name the feature
mgravell Sep 3, 2026
eb8c4a0
Soften the SequenceId wording
mgravell Sep 3, 2026
a9d3f1c
Validate endpoint retirement against a real node removal
mgravell Sep 3, 2026
f7b1f97
cluster_failure ends the deployment; assert clean degradation, not re…
mgravell Sep 3, 2026
1a4c858
Warn that a stale environment directory passes every check
mgravell Sep 3, 2026
16adfe3
Re-read the topology on a long timer as a backstop
mgravell Sep 8, 2026
377abfa
Say what ConfigCheckSeconds actually checks
mgravell Sep 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<CodeAnalysisRuleset>$(MSBuildThisFileDirectory)Shared.ruleset</CodeAnalysisRuleset>
<MSBuildWarningsAsMessages>NETSDK1069</MSBuildWarningsAsMessages>
<!-- SER002/SER003/SER006 retired (Redis 8.4/8.6/8.8 features no longer experimental); IDs reserved, see Experiments.cs -->
<NoWarn>$(NoWarn);NU5105;NU1507;SER001;SER004;SER005;SER007;SER008;SER009</NoWarn>
<NoWarn>$(NoWarn);NU5105;NU1507;SER001;SER004;SER005;SER007;SER008;SER009;SER010</NoWarn>
<PackageReleaseNotes>https://github.com/StackExchange/StackExchange.Redis/releases</PackageReleaseNotes>
<PackageProjectUrl>https://seredis.dev/</PackageProjectUrl>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
Expand Down
1 change: 1 addition & 0 deletions StackExchange.Redis.slnx
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
<Project Path="tests/RedisConfigs/RedisConfigs.csproj" />
<Project Path="tests/RESPite.Tests/RESPite.Tests.csproj" />
<Project Path="tests/StackExchange.Redis.Benchmarks/StackExchange.Redis.Benchmarks.csproj" />
<Project Path="tests/StackExchange.Redis.FaultInjector.Tests/StackExchange.Redis.FaultInjector.Tests.csproj" />
<Project Path="tests/StackExchange.Redis.Tests/StackExchange.Redis.Tests.csproj" />
</Folder>
<Folder Name="/toys/">
Expand Down
100 changes: 99 additions & 1 deletion docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,8 @@ The `ConfigurationOptions` object has a wide range of properties, all of which a
| connectRetry={int} | `ConnectRetry` | `3` | The number of times to repeat connect attempts during initial `Connect` |
| connectTimeout={int} | `ConnectTimeout` | `5000` | Timeout (ms) for connect operations |
| configChannel={string} | `ConfigurationChannel` | `__Booksleeve_MasterChanged` | Broadcast channel name for communicating configuration changes |
| configCheckSeconds={int} | `ConfigCheckSeconds` | `60` | Time (seconds) to check configuration. This serves as a keep-alive for interactive sockets, if it is supported. |
| configCheckSeconds={int} | `ConfigCheckSeconds` | `60` | Time (seconds) between re-checks of each connected server's replication role, via `INFO replication`; also acts as a keep-alive for interactive sockets. Not a topology re-read: see `topologyRefreshSeconds` |
| topologyRefreshSeconds={int} | `TopologyRefreshSeconds` | `1800` | Time (seconds) between unprompted topology re-reads, or `0` to never do so. Jittered by up to 30 seconds. |
| defaultDatabase={int} | `DefaultDatabase` | `null` | Default database index, from `0` to `databases - 1` |
| keepAlive={int} | `KeepAlive` | `-1` | Time (seconds) at which to send a message to help keep sockets alive (60 sec default) |
| tcpKeepAlive={bool} | `TcpKeepAlive` | `true` | Enables TCP keep-alive when appropriate (endpoint- and platform-dependent) |
Expand All @@ -100,6 +101,11 @@ The `ConfigurationOptions` object has a wide range of properties, all of which a
| setlib={bool} | `SetClientLibrary` | `true` | Whether to attempt to use `CLIENT SETINFO` to set the library name/version on the connection |
| protocol={string} | `Protocol` | `null` | Redis protocol to use; see section below |
| highIntegrity={bool} | `HighIntegrity` | `false` | High integrity (incurs overhead) sequence checking on every command; see section below |
| defaults={string} | `Defaults` | `null` | Selects a named defaults provider; see section below |
| maintNotifications={string} | `MaintenanceNotifications` | `disabled` | Whether to ask servers for maintenance notifications; see section below |
| maintRelaxedTimeout={int} | `MaintenanceRelaxedTimeout` | `10` | **Seconds** that command timeouts are relaxed to during announced maintenance; see section below |
| maintRelaxedWindowMax={int} | `MaintenanceRelaxedWindowMax` | `30` | **Seconds** a relaxed window may last at most; see section below |
| maintPostEventRelaxed={int} | `MaintenancePostEventRelaxedDuration` | `20` | **Seconds** timeouts stay relaxed after maintenance completes; see section below |

Additional code-only options:
- LoggerFactory (`ILoggerFactory`) - Default: `null`
Expand Down Expand Up @@ -260,6 +266,45 @@ Both options can be customized or disabled (set to `""`), via the `.Configuratio

These settings are also used by the `IServer.MakeMaster()` method, which can set the tie-breaker in the database and broadcast the configuration change message. The configuration message can also be used separately to primary/replica changes simply to request all nodes to refresh their configurations, via the `ConnectionMultiplexer.PublishReconfigure` method.

## Refreshing the topology after repeated connect failures

An endpoint that refuses every connection is evidence that what the client believes about the deployment may
be wrong, so after **three consecutive** failed connection attempts to the same endpoint, the client re-reads
the topology - the same refresh a `MOVED` or a configuration announcement would have caused, jittered and
coalesced in the same way.

This closes a real gap rather than a theoretical one. Every other path that re-reads the topology needs
somebody *else* to notice first: a redirect from a reachable node, a peer's configuration announcement, or a
maintenance notification. The internal flag that drives a refresh-on-failure is only set once a connection has
been *established*, so an endpoint that has never connected - because it was replaced while the client was
running, or was already gone at startup - could be retried indefinitely with nobody to say otherwise. Measured
in the field: a client dialled three removed nodes for around 37 hours.

The re-read is rate-limited to `configCheckSeconds` (default 60), deliberately reusing the knob that already
means "how often may we re-read configuration" rather than adding one. That restraint is what makes it safe:
a permanently dead endpoint, times a retry loop, times every client in a fleet would otherwise be a great
many topology reads, so a dead endpoint prompts at most one re-read per interval until something changes.

Note that `configCheckSeconds` on its own is *not* a periodic topology refresh - it drives an
`INFO replication` on an established connection, which is a replication-role check. This is the path that
notices an endpoint nobody can reach.

### ...and the backstop for one nobody can fault

Repeated connect failures cover an endpoint that refuses or never finishes a handshake. What they cannot cover
is an endpoint that is *reachable*, answers a handshake, and is no longer part of the deployment: a re-bound
port now serving something else produces no failure, no redirect, and nothing announced, so no event-driven
path asks the question.

`topologyRefreshSeconds` is the answer to that, and only that: every 30 minutes by default, the client
re-reads the topology whether or not anything appears to be wrong. Two things keep it cheap. The interval is
long, and each client picks its own phase within a 30-second jitter on every cycle, so a fleet started
together does not stay in step. Set it to `0` to turn it off.

It is deliberately a backstop rather than the mechanism. Topology is normally learned from something
happening - a redirect, an announcement, a maintenance notification, a connection failing - and those react in
seconds where this reacts in minutes.

## ReconnectRetryPolicy

StackExchange.Redis automatically tries to reconnect in the background when the connection is lost for any reason. It keeps retrying until the connection has been restored. It would use ReconnectRetryPolicy to decide how long it should wait between the retries.
Expand Down Expand Up @@ -287,6 +332,59 @@ config.ReconnectRetryPolicy = new LinearRetry(5000);
//6 5000
```

## Defaults providers

Some settings have sensible values that depend on *what you are connecting to* rather than on what you want, so
the library keeps them in a provider and consults it for anything you have not set explicitly. Providers are
normally chosen automatically by looking at the endpoints - an `*.redis.cache.windows.net` host selects the
Azure provider, and so on - and you can select one explicitly instead:

```
myserver:6379,defaults=enterprise
```

The names are `azure`, `amr` (Azure Managed Redis), `rediscloud` and `enterprise` (a self-managed Redis
Enterprise deployment). The last one exists because it cannot be detected: a self-managed cluster has whatever
DNS its operator gave it, so there is nothing to recognize. It is also the right choice for a hosted deployment
reached behind private DNS, a CNAME or a proxy, where the endpoint no longer looks like what it is.

A provider chosen explicitly appears in `ToString()`; one that was merely inferred from the endpoints does not,
since writing it out would turn a guess into a decision. Custom providers (assigned in code, via
`ConfigurationOptions.Defaults`, or registered with `DefaultOptionsProvider.AddProvider`) work exactly as
before; they can additionally be named in a configuration string if they override `Name`.

## Maintenance notifications

Server-native maintenance notifications - *smart client handoffs*, also called *hitless upgrades* - are configured with the keys below; [ServerMaintenanceEvent](ServerMaintenanceEvent) is the guide to what they do.

Redis Enterprise and Redis Cloud can warn a connected client *before* a disruptive event - a shard migration, a
failover, or an endpoint being replaced - so the client can act ahead of it rather than discover it by way of a
broken connection. This requires RESP3, and the client asks for it per connection:

| Mode | Behaviour |
| --- | --- |
| `disabled` | Never ask. The default, because most servers have never heard of the request. |
| `auto` | Ask, and carry on if the server refuses or the connection ends up RESP2. Safe against a mixture of servers, and what most callers want. |
| `enabled` | **Required**: ask, and reject the connection unless notifications are live. Only point this at a deployment you know supports them. |

Note that `enabled` means *required*, not merely *on* - it is the cross-client name for that mode, and it will
refuse a connection that cannot deliver notifications, including any RESP2 connection. The `amr`, `rediscloud`
and `enterprise` providers select `auto` for you, so on those deployments you need not set anything.

While a disruption has been announced, command timeouts are relaxed - raised to `maintRelaxedTimeout`, never
lowered, so a caller with a more generous timeout keeps it. The window ends when the server says the disruption
finished, and then stays relaxed for `maintPostEventRelaxed` longer, because completion is exactly when every
other client that received the same notification comes back. `maintRelaxedWindowMax` bounds a window whose
closing notification never arrives. Only *command* timeouts are affected: keep-alive and connection-failure
detection are untouched, so a server that dies mid-maintenance is still noticed on the usual schedule.

These durations are in **seconds**, unlike every other timeout here, because those are the units the
cross-client specification names for them.

Receiving a notification raises `ConnectionMultiplexer.ServerMaintenanceEvent` with a `PushMaintenanceEvent`,
and a timeout or connection fault that happened during an announced disruption carries a `MaintenanceType`
saying so.

## Redis protocol

RESP3 is a newer protocol (available on v6 servers and above) which allows (among other changes) pub/sub messages to be communicated on the *same* connection - which can be very
Expand Down
9 changes: 5 additions & 4 deletions docs/Resp3.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# RESP3 and StackExchange.Redis
# RESP3 and StackExchange.Redis

RESP2 and RESP3 are evolutions of the Redis protocol, with RESP3 existing from Redis server version 6 onwards (v7.2+ for Redis Enterprise). The main differences are:

1. RESP3 can carry out-of-band / "push" messages on a single connection, where-as RESP2 requires a separate connection for out-of-band (pub/sub) messages
- this single connection can be of huge benefit in high-usage servers, as it halves the number of connections required
2. RESP3 supports *additional* out-of-band messages that cannot be expressed in RESP2, which allows advanced features such as "smart client handoffs" (a family of
server maintenance notifications)
- these features (not yet implemented in SE.Redis) allow for greater stability in complex deployments
2. RESP3 supports *additional* out-of-band messages that cannot be expressed in RESP2, which allows advanced features such as "smart client handoffs" (also called
"hitless upgrades"; a family of server maintenance notifications)
- these features allow for greater stability in complex deployments, and are implemented in SE.Redis: see
[ServerMaintenanceEvent](ServerMaintenanceEvent) - note they are RESP3-only, so a connection that ends up on RESP2 does not get them
3. RESP3 can (when appropriate) convey additional semantic meaning about returned payloads inside the same result structure
- this is *mostly* relevant to client libraries that do not explicitly interpret the results before exposing to the user, so this does not directly impact SE.Redis itself,
but it is relevant to consumers of SE.Redis that use Lua scripts or ad-hoc commands
Expand Down
Loading
Loading