Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,14 +185,19 @@ client = create_async_redis_client(settings, metrics=metrics)
# - myapp_redis_connection_errors_total{error_type}
```

Prometheus registers a metric name once per process, so a second `RedisMetrics(prefix="myapp")`
raises `ValueError`. `get_redis_metrics(prefix="myapp")` returns the one instance per prefix
instead, which is what a test suite that rebuilds its container per test wants.

### With Dishka DI

```python
from dishka import make_async_container
from redis_client_kit.providers import AsyncRedisProvider
from redis_client_kit.providers import AsyncRedisProvider, PrometheusRedisMetricsProvider

container = make_async_container(
AsyncRedisProvider(),
AsyncRedisProvider(provide_default_metrics=False),
PrometheusRedisMetricsProvider(), # RedisMetrics when settings.metrics_enabled, else None
SettingsProvider(), # Your settings provider
)

Expand Down
60 changes: 37 additions & 23 deletions docs/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ raises `AttributeError` on the first one missing.
| `response` | `RedisResponseSettings` | all defaults |
| `key_prefix` | `str` | **required** — never read by this library |
| `health_check_interval` | `int | None`, `ge=0` | `30` |
| `metrics_enabled` | `bool` | `False` — never read by this library |
| `metrics_enabled` | `bool` | `False` — read only by `PrometheusRedisMetricsProvider` |

| Group | Field | Default | Notes |
|---|---|---|---|
Expand Down Expand Up @@ -225,8 +225,8 @@ The rest lives one import deeper.
| `redis_client_kit.protocols` | — | `RedisMetricsProtocol` |
| `redis_client_kit.utils` | — | the three exported helpers, plus `mask_redis_kwargs(kwargs)` for logging and `WRITE_PROBE_TTL_S`, the write probe's expiry in seconds |
| `redis_client_kit.settings` | `settings` | `BaseRedisSettings`, `RedisConnectionSettings`, `RedisClusterSettings`, `RedisPoolSettings`, `RedisRetrySettings`, `RedisSSLSettings`, `RedisResponseSettings` |
| `redis_client_kit.metrics` | `metrics` | `RedisMetrics`, `REDIS_COMMAND_DURATION_BUCKETS` |
| `redis_client_kit.providers` | `providers` | `AsyncRedisProvider(check_health_on_startup=True, provide_default_metrics=True)` |
| `redis_client_kit.metrics` | `metrics` | `RedisMetrics`, `get_redis_metrics`, `REDIS_COMMAND_DURATION_BUCKETS` |
| `redis_client_kit.providers` | `providers` | `AsyncRedisProvider(check_health_on_startup=True, provide_default_metrics=True)`, `PrometheusRedisMetricsProvider(prefix=None)` |

Each optional module raises `ImportError` at import time when its extra is missing, naming
the extra. The root package imports none of them.
Expand All @@ -250,30 +250,34 @@ Buckets are `REDIS_COMMAND_DURATION_BUCKETS` — `(0.0001, 0.0005, 0.001, 0.005,
argument of `execute_command`, upper-cased — it is unbounded cardinality only if you send
unbounded command names.

`get_redis_metrics(prefix=None)` returns the one `RedisMetrics` per prefix on the default
registry, creating it on the first call and handing the same instance back afterwards;
`""` and `None` are the same unprefixed instance. Use it wherever the collector may be
asked for twice in one process — a container rebuilt per test, above all — since a second
`RedisMetrics()` with the same prefix raises (rule 18).

### Dishka

```python
from dishka import Provider, Scope, make_async_container, provide

from redis_client_kit import AsyncRedisClient
from redis_client_kit.config import RedisSettingsProtocol
from redis_client_kit.metrics import RedisMetrics
from redis_client_kit.protocols import RedisMetricsProtocol
from redis_client_kit.providers import AsyncRedisProvider
from redis_client_kit.providers import AsyncRedisProvider, PrometheusRedisMetricsProvider
from redis_client_kit.settings import BaseRedisSettings

class AppProvider(Provider):
scope = Scope.APP

@provide
def settings(self) -> RedisSettingsProtocol:
return BaseRedisSettings(key_prefix="myapp")

@provide
def metrics(self) -> RedisMetricsProtocol | None: # this exact annotation
return RedisMetrics(prefix="myapp")
return BaseRedisSettings(key_prefix="myapp", metrics_enabled=True)

container = make_async_container(AsyncRedisProvider(), AppProvider()) # this order
container = make_async_container(
AsyncRedisProvider(provide_default_metrics=False),
PrometheusRedisMetricsProvider(prefix="myapp"),
AppProvider(),
)
client = await container.get(AsyncRedisClient)
```

Expand All @@ -284,7 +288,14 @@ registers:
| Argument | Default | Effect |
|---|---|---|
| `check_health_on_startup` | `True` | pings Redis before yielding the client, and raises when it does not answer; `False` registers the factory that yields immediately |
| `provide_default_metrics` | `True` | provides `RedisMetricsProtocol | None` as `None` so a container without metrics resolves; `False` leaves that type to your own provider |
| `provide_default_metrics` | `True` | provides `RedisMetricsProtocol | None` as `None` so a container without metrics resolves; `False` leaves that type to another provider |

`PrometheusRedisMetricsProvider(*, prefix=None)` is that other provider: `Scope.APP`,
requests `RedisSettingsProtocol` and nothing else, provides `RedisMetricsProtocol | None`
as `get_redis_metrics(prefix)` when `settings.metrics_enabled` and as `None` otherwise.
With `metrics_enabled` on it needs the `metrics` extra, and raises `ImportError` naming it
when the collector is resolved. A collector of your own is a provider of the same key,
registered in its place.

See rules 15 to 17.

Expand Down Expand Up @@ -320,10 +331,11 @@ See rules 15 to 17.
from a bad command is raised on the first try. The delay is
`min(backoff_cap, backoff_base * 2**failures)` with no jitter, so the first retry waits
exactly `backoff_base`.
7. **`key_prefix` and `metrics_enabled` are declared and never read.** `key_prefix` is
required by `BaseRedisSettings` and used by nothing in this package;
`metrics_enabled=True` does not turn on instrumentation. Passing `metrics=` to the
factory does, and it is the only thing that does.
7. **`key_prefix` is declared and never read; `metrics_enabled` is read by one thing.**
`key_prefix` is required by `BaseRedisSettings` and used by nothing in this package.
`metrics_enabled` is read only by `PrometheusRedisMetricsProvider`; the factory ignores
it, so `metrics_enabled=True` without that provider turns nothing on. Passing
`metrics=` to the factory does, and outside Dishka it is the only thing that does.
8. **`BaseRedisSettings` is grouped and forbids extras.** `BaseRedisSettings(host="…")`
raises `ValidationError: Extra inputs are not permitted`. Pass
`connection=RedisConnectionSettings(host="…")`.
Expand Down Expand Up @@ -360,8 +372,9 @@ See rules 15 to 17.
Dishka the last provider to claim a type wins — put it second and your metrics are
silently dropped, leaving an uninstrumented client.
`AsyncRedisProvider(provide_default_metrics=False)` registers no default, so order
stops mattering. Either way the annotation on your factory must be exactly
`RedisMetricsProtocol | None`; `RedisMetricsProtocol` is a different key.
stops mattering. This holds for `PrometheusRedisMetricsProvider` as much as for a
provider of your own; for your own, the annotation on the factory must be exactly
`RedisMetricsProtocol | None` — `RedisMetricsProtocol` is a different key.
16. **The provider registers one client factory, chosen at construction.**
`AsyncRedisProvider()` registers `get_redis_with_health_check()`;
`AsyncRedisProvider(check_health_on_startup=False)` registers `get_redis()` instead.
Expand All @@ -375,7 +388,8 @@ See rules 15 to 17.
block startup.
18. **A `RedisMetrics` instance owns global Prometheus names.** Building a second one with
the same prefix raises a duplicate-timeseries `ValueError` from the default registry.
Build one per process and inject it.
That is Prometheus, not this package: build one per process and inject it, or ask
`get_redis_metrics(prefix)` and get the same instance back on every call.
19. **Cluster clients record no pool statistics.** Single-node clients, async and sync,
report `redis_pool_size` and `redis_pool_checked_out` from the pool's own containers
before every command; `InstrumentedRedisCluster` reports neither. Command counts,
Expand Down Expand Up @@ -427,12 +441,12 @@ client = redis.asyncio.Redis(**build_base_redis_kwargs(settings, asyncio=True),
```

```python
# WRONG — metrics_enabled does nothing, and the client is never instrumented
# WRONG — the factory never reads metrics_enabled, and the client is never instrumented
settings = BaseRedisSettings(key_prefix="myapp", metrics_enabled=True)
client = create_async_redis_client(settings)

# RIGHT
client = create_async_redis_client(settings, metrics=RedisMetrics(prefix="myapp"))
# RIGHT — pass the collector; only PrometheusRedisMetricsProvider reads the flag, and only in Dishka
client = create_async_redis_client(settings, metrics=get_redis_metrics(prefix="myapp"))
```

```python
Expand Down
40 changes: 40 additions & 0 deletions docs/guide/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,23 @@ client = create_async_redis_client(settings, metrics=metrics)
# - myapp_redis_connection_errors_total{error_type}
```

### One Instance Per Prefix

Prometheus registers a metric name once per registry, so a second
`RedisMetrics(prefix="myapp")` on the default registry raises
`ValueError: Duplicated timeseries in CollectorRegistry` — what a test suite runs into
when it builds a container per test. `get_redis_metrics(prefix=None)` caches one instance
per prefix and hands it back on every later call:

```python
from redis_client_kit.metrics import get_redis_metrics

metrics = get_redis_metrics(prefix="myapp")
assert get_redis_metrics(prefix="myapp") is metrics
```

The [Dishka provider](#with-metrics) calls the getter for you.

### Metrics Configuration

```python
Expand Down Expand Up @@ -149,6 +166,29 @@ contact Redis at all.

### With Metrics

`PrometheusRedisMetricsProvider` provides `RedisMetricsProtocol | None` — the key
`AsyncRedisProvider` reads — as `get_redis_metrics(prefix)` when `settings.metrics_enabled`
is on and `None` when it is off, so the client is instrumented exactly when the settings
say so:

```python
from redis_client_kit.providers import AsyncRedisProvider, PrometheusRedisMetricsProvider

container = make_async_container(
AsyncRedisProvider(provide_default_metrics=False),
PrometheusRedisMetricsProvider(), # or PrometheusRedisMetricsProvider(prefix="myapp")
SettingsProvider(),
)
```

With `metrics_enabled` on it needs the `metrics` extra; without it, resolving the
collector raises `ImportError` naming the extra. The collector is the one
`get_redis_metrics(prefix)` returns, so a container built per test never registers the
same series twice.

To wire a collector of your own — the `PrometheusRedisMetrics` above, say — provide the
same key yourself:

```python
from redis_client_kit.protocols import RedisMetricsProtocol

Expand Down
6 changes: 4 additions & 2 deletions docs/guide/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,7 @@ class MySettings:
ssl: MySSL
response: MyResponse
health_check_interval: int = 30
metrics_enabled: bool = False

# Use it
settings = MySettings(
Expand All @@ -374,8 +375,9 @@ settings = MySettings(
client = create_async_redis_client(settings)
```

Every attribute the protocols name has to be there: the factory reads all of them and
raises `AttributeError` on the first one missing. The protocols are not
Every attribute the protocols name has to be there: the factory reads all of them but
`metrics_enabled`, which only `PrometheusRedisMetricsProvider` reads, and raises
`AttributeError` on the first one missing. The protocols are not
`@runtime_checkable`, so `isinstance(settings, RedisSettingsProtocol)` raises `TypeError`.

## Configuration Best Practices
Expand Down
1 change: 1 addition & 0 deletions redis_client_kit/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,4 @@ class RedisSettingsProtocol(Protocol):
ssl: RedisSSLProtocol
response: RedisResponseProtocol
health_check_interval: int | None
metrics_enabled: bool
4 changes: 2 additions & 2 deletions redis_client_kit/metrics/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@
if not HAS_PROMETHEUS:
raise ImportError("prometheus-client not installed. Install redis-client-kit[metrics] to use metrics.")

from .redis import REDIS_COMMAND_DURATION_BUCKETS, RedisMetrics
from .redis import REDIS_COMMAND_DURATION_BUCKETS, RedisMetrics, get_redis_metrics

__all__ = ["REDIS_COMMAND_DURATION_BUCKETS", "RedisMetrics"]
__all__ = ["REDIS_COMMAND_DURATION_BUCKETS", "RedisMetrics", "get_redis_metrics"]
32 changes: 31 additions & 1 deletion redis_client_kit/metrics/redis.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,20 @@ class RedisMetrics:
>>> metrics = RedisMetrics(prefix="myapp")
>>> metrics.record_command("GET", "success", 0.001)
>>> metrics.record_pool_stats(pool_size=10, pool_checked_out=3)

Prometheus registers a metric name once per registry, so a second instance with the
same prefix raises ``ValueError``; ``get_redis_metrics`` hands back the first one instead.
"""

def __init__(self, prefix: str | None = None) -> None:
"""Initialize Redis Prometheus metrics.

Args:
prefix: Optional metric name prefix (e.g., "myapp" -> "myapp_redis_pool_size")

Raises:
ValueError: If a metric of the same name is already registered on the default
registry -- see ``get_redis_metrics`` for the second instance.
"""
metric_prefix = f"{prefix}_" if prefix else ""

Expand Down Expand Up @@ -95,4 +102,27 @@ def record_pool_stats(self, pool_size: int, pool_checked_out: int) -> None:
self.pool_checked_out.set(float(pool_checked_out))


__all__ = ["REDIS_COMMAND_DURATION_BUCKETS", "RedisMetrics"]
_REDIS_METRICS_CACHE: dict[str | None, RedisMetrics] = {}


def get_redis_metrics(prefix: str | None = None) -> RedisMetrics:
"""Get (or lazily create) the cached ``RedisMetrics`` for a prefix, on the default registry.

Caching by prefix is what lets a container be rebuilt -- a test suite does it per test --
without Prometheus refusing the second registration of the same series.

Args:
prefix: The metric name prefix the instance was, or is, created with; ``""`` and
``None`` are the same unprefixed instance.

Returns:
The one instance for that prefix.
"""
key = prefix or None
metrics = _REDIS_METRICS_CACHE.get(key)
if metrics is None:
metrics = _REDIS_METRICS_CACHE[key] = RedisMetrics(prefix=key)
return metrics


__all__ = ["REDIS_COMMAND_DURATION_BUCKETS", "RedisMetrics", "get_redis_metrics"]
3 changes: 2 additions & 1 deletion redis_client_kit/providers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
if not HAS_DISHKA:
raise ImportError("dishka not installed. Install redis-client-kit[providers] to use AsyncRedisProvider.")

from .metrics import PrometheusRedisMetricsProvider
from .redis import AsyncRedisProvider

__all__ = ["AsyncRedisProvider"]
__all__ = ["AsyncRedisProvider", "PrometheusRedisMetricsProvider"]
44 changes: 44 additions & 0 deletions redis_client_kit/providers/metrics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
"""Dishka provider for the Prometheus Redis metrics collector."""

from ..config import RedisSettingsProtocol
from ..protocols import RedisMetricsProtocol
from ._deps import Provider, Scope, provide


class PrometheusRedisMetricsProvider(Provider): # type: ignore[misc]
"""Dishka provider for the collector ``AsyncRedisProvider`` records into.

Provides ``RedisMetricsProtocol | None`` -- the key ``AsyncRedisProvider`` reads -- as
``get_redis_metrics(prefix)`` when ``settings.metrics_enabled`` and ``None`` otherwise,
so the client is instrumented exactly when the settings say so. Register it with
``AsyncRedisProvider(provide_default_metrics=False)``, or after ``AsyncRedisProvider()``,
since the last provider of a type wins.

The collector comes from ``get_redis_metrics``: one instance per prefix on the default
registry, so a container rebuilt per test never asks Prometheus to register the same
series twice. Resolving it with metrics on needs the ``metrics`` extra; without it the
import raises ``ImportError`` naming the extra.
"""

scope = Scope.APP # type: ignore[misc]

def __init__(self, *, prefix: str | None = None) -> None:
"""Remember the prefix the collector is created with.

Args:
prefix: Metric name prefix (``"myapp"`` gives ``myapp_redis_pool_size``)
"""
super().__init__()
self._prefix = prefix

@provide
def get_metrics(self, redis_settings: RedisSettingsProtocol) -> RedisMetricsProtocol | None:
"""Provide the collector when ``metrics_enabled`` is on, ``None`` otherwise."""
if not redis_settings.metrics_enabled:
return None
from ..metrics import get_redis_metrics # noqa: PLC0415 - lazy: needs the [metrics] extra

return get_redis_metrics(self._prefix)


__all__ = ["PrometheusRedisMetricsProvider"]
1 change: 1 addition & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,5 +71,6 @@ def mock_redis_settings() -> MagicMock:
settings.ssl = ssl
settings.response = response
settings.health_check_interval = 30
settings.metrics_enabled = False

return settings
Loading