Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
9c9ff05
fix(cachekitio): bound the writer's L1 copy by the PUT's Fresh-For an…
27Bslash6 Nov 1, 2026
9321c2a
perf(cache): cut the write-path freshness check off the miss path
27Bslash6 Nov 1, 2026
a6d04bf
fix: address coderabbit review — fork take-over before a declined L1 …
27Bslash6 Nov 1, 2026
1f8cb4a
fix: time the writer's L1 bound from the PUT's receipt, through set o…
27Bslash6 Nov 1, 2026
cd1e60c
docs: scope the no-expiry claim to where no plan maximum applied
27Bslash6 Nov 1, 2026
9a48058
fix: bound the writer's L1 copy from receipt on every path, through h…
27Bslash6 Nov 1, 2026
ede8541
fix: report write deadlines through composing wrappers, keep a 2xx's …
27Bslash6 Nov 1, 2026
811cf63
fix: bound a write-behind copy when its write fails after the 2xx; ke…
27Bslash6 Nov 1, 2026
909c0d1
perf: open the write-deadline list only for a handler whose backend c…
27Bslash6 Nov 1, 2026
d454b28
fix: find slotted wrappers, and search breadth first so a large sibli…
27Bslash6 Nov 1, 2026
1509bf1
fix: write-deadline search runs no user code, reaches containers, and…
27Bslash6 Nov 1, 2026
f2f6d18
fix: write-deadline search counts every read, ends on cycles, reads c…
27Bslash6 Nov 1, 2026
b8a843c
fix: one breadth-first walk with one read counter that fails open whe…
27Bslash6 Nov 1, 2026
060c74a
fix: keep the shallowest depth per object, read each type once, read …
27Bslash6 Nov 1, 2026
ee6a9f5
fix: skip only the root's own flag, so a container root's items count
27Bslash6 Nov 1, 2026
ee94b40
fix: charge each value once and every slot read in the write-deadline…
27Bslash6 Nov 1, 2026
50566d9
fix: charge a set slot once, when the walk reads its value
27Bslash6 Nov 1, 2026
25cfd6d
perf: read the cached write-deadline answer with an attribute load on…
27Bslash6 Nov 1, 2026
dac1d4d
fix: read a container subclass's own attributes as well as its items
27Bslash6 Nov 1, 2026
f8a07c8
Merge remote-tracking branch 'origin/main' into agent/winston/778b453…
27Bslash6 Nov 1, 2026
0684264
test: state the 90-day cap's wire value beside the rounding case
27Bslash6 Nov 1, 2026
a623b78
test: give the fail-open stub the reader's three parameters
27Bslash6 Nov 1, 2026
0382339
fix: bound a write-behind copy as the 2xx is read, and find backends …
27Bslash6 Nov 2, 2026
f92ceb3
test: assert the probes' preconditions ran (slow parse, the raise aft…
27Bslash6 Nov 2, 2026
9f14a32
fix: record a bounded deferred copy's expiry, and read every class na…
27Bslash6 Nov 2, 2026
5a99eeb
fix: yield a type's class values again from any shallower read of it
27Bslash6 Nov 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .secrets.baseline

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ Setting `CACHEKIT_MASTER_KEY` instead of passing `master_key=` means every prese
>
> 🔒 `@cache.secure` forces `integrity_checking=True` — passing `integrity_checking=False` raises `ConfigurationError` at decoration, including as an override next to `config=DecoratorConfig.secure(...)`. Its encryption is fixed the same way: `encryption=` raises `ConfigurationError` on `@cache.secure` and next to `config=DecoratorConfig.secure(...)`, so pass `fail_closed=` and the other encryption options to the preset itself. `@cache.secure` also rejects `config=`; the RORO form is `@cache(config=DecoratorConfig.secure(...))`.
>
> **Default TTL** follows the cross-SDK [intent-preset spec](https://github.com/cachekit-io/protocol/blob/main/spec/intent-presets.md#default-ttl) (`@cache.io` 3600 s) — the same numbers as cachekit-rs and cachekit-ts. `ttl=` overrides it; `ttl=None` is the explicit never-expire opt-in ([details](docs/configuration.md#intent-presets)).
> **Default TTL** follows the cross-SDK [intent-preset spec](https://github.com/cachekit-io/protocol/blob/main/spec/intent-presets.md#default-ttl) (`@cache.io` 3600 s) — the same numbers as cachekit-rs and cachekit-ts. `ttl=` overrides it; `ttl=None` is the explicit no-TTL opt-in: never-expire on Redis, Memcached and File, while CachekitIO stores the entry for your plan's maximum TTL and keeps it forever only on a plan without a maximum, or where none applied when the entry was written ([details](docs/configuration.md#intent-presets), [plan maximums](docs/backends/cachekitio.md#ttl-range-and-plan-maximum)).
>
> **Cross-process L1 eviction** is not a preset feature: every preset's `invalidate_cache()` deletes from L1 and L2 alike, and evicting other processes' L1 copies is one process-wide opt-in, `CACHEKIT_INVALIDATION_LISTENER_ENABLED` ([L1 invalidation](docs/features/l1-invalidation.md#cross-process-l1-eviction)).
>
Expand Down
4 changes: 2 additions & 2 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ def your_function(args):

#### Core Parameters

- **`ttl`** (`int | None`, default: `None` on bare `@cache`; each preset sets its own — see the [preset matrix](configuration.md#intent-presets)) - Cache time-to-live in seconds. `None` = no expiration; on a preset that is an explicit opt-in.
- **`ttl`** (`int | None`, default: `None` on bare `@cache`; each preset sets its own — see the [preset matrix](configuration.md#intent-presets)) - Cache time-to-live in seconds. `None` = no TTL: no expiration on Redis, Memcached and File; one year in L1-only mode (`backend=None`); on CachekitIO the server stores the entry for your plan's maximum TTL ([plan maximums](backends/cachekitio.md#ttl-range-and-plan-maximum)). On a preset it is an explicit opt-in.
- **`namespace`** (`str | None`, default: `None`) - Cache key prefix for organization
- **`serializer`** (`str | SerializerProtocol`, default: `"default"`) - Serializer name (`"default"`, `"std"`, `"auto"`, `"arrow"`, `"orjson"`) or `SerializerProtocol` instance
- **`integrity_checking`** (`bool`, default: `True`; any non-bool raises `ConfigurationError`) - Enable xxHash3-64 checksums for corruption detection (non-cryptographic — detects bit rot and storage bugs, NOT tampering; tamper resistance requires encryption). Behaviour when a writer and reader disagree on this flag: [cross-config reads](serializers/auto.md#cross-config-reads-integrity_checking-mismatch)
Expand Down Expand Up @@ -222,7 +222,7 @@ def get_exchange_rates():

- Inherits all production-grade reliability features: circuit breaker, backpressure, full monitoring
- L1 in-memory cache is enabled — hot data is served from process memory without an HTTP round-trip (each L1 hit deserializes the stored bytes)
- Default `ttl=3600`; with that default the stale-while-revalidate window is also 3600 s. `ttl=None` disables both expiry and SWR.
- Default `ttl=3600`; with that default the stale-while-revalidate window is also 3600 s. `ttl=None` sends no TTL and disables SWR: the server stores the entry for your plan's maximum TTL, and never expires it only on a plan without a maximum, or where none applied when the entry was written ([plan maximums](backends/cachekitio.md#ttl-range-and-plan-maximum)).
- Standard `ttl`, `namespace`, `serializer`, and other `@cache(...)` kwargs are all supported as overrides

### Health Check Methods
Expand Down
38 changes: 38 additions & 0 deletions docs/backends/cachekitio.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,44 @@ async with backend.acquire_lock("my-key", timeout=30.0, blocking_timeout=5.0) as
The `@cache` decorators use this automatically on cache miss for **async**
functions — see [Distributed Locking](../features/distributed-locking.md).

## TTL Range and Plan Maximum

A TTL is 1 to 7,776,000 seconds (90 days), and `ttl + stale_ttl` stays within the same 7,776,000.
The decorator rejects a larger `ttl + stale_ttl` at decoration time. Your plan then sets the
longest an entry is kept, its stale window included:

| Plan | Maximum TTL |
| :--- | :--- |
| Free | 86,400 s (24 hours) |
| Startup | 2,592,000 s (30 days) |
| Professional | 7,776,000 s (90 days) |
| Enterprise | None |

A longer TTL is clamped to the maximum, not rejected, and the stale window gets only what the
maximum leaves after the fresh TTL. A write without a TTL (`ttl=None`) is stored for the plan
maximum. An entry never expires only on a plan without a maximum, or where none applied when
it was written: a write whose plan the service could not determine is stored as stated, and an
entry written before the maximum applied keeps its timing until its next write. `refresh_ttl` is
clamped the same way.

The server reports the result in the `PUT` response's `X-CacheKit-Fresh-For`, and the decorator
bounds its in-process (L1) copy of the value it wrote by it: the copy lives at most
`min(ttl, fresh_for)` from the response, so a clamped write is never served from L1 past the
plan maximum. A malformed or `0` value keeps no L1 copy; a server that sends no header leaves the
copy at `ttl`. The `PATCH` response of `refresh_ttl` does not report the applied TTL, so it never
lengthens an L1 copy. A `CachekitIOBackend` subclass that overrides `set` keeps this bound as long
as its override calls `super().set(...)` on the same thread. The same holds for a wrapper or proxy
backend whose `set` calls a `CachekitIOBackend`'s `set` on the same thread, provided the decorator
can find that backend at its first L2 write: in an instance or class attribute (`__slots__` included), or in a dict,
list, tuple or set there (a router or a shard ring), at most three wrappers down. A wrapper too
large to search is assumed to hold one. A wrapper that holds its backend only in a closure,
or creates it on first use (even into an attribute: the search runs before the first write and its
answer is kept), is not found: its L1 copies keep `ttl`, and the decorator logs this once per
wrapper class. Hold the backend in an attribute from construction to get the bound. A wrapper that delegates on another thread (a timeout wrapper with its own executor) must
run the call under `contextvars.copy_context().run`; otherwise its L1 copies keep `ttl` and ignore
Fresh-For. A subclass that writes any other way gives the decorator no response to read, so it
keeps no L1 copy of that write.

## TTL Inspection (async only)

```python notest
Expand Down
6 changes: 3 additions & 3 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,14 +197,14 @@ def report():

Rules and behavior:

- Requires a positive `ttl`; `ttl + stale_ttl` is capped at 2,592,000 s (30 days). Violations raise `ConfigurationError` at decoration time.
- Requires a positive `ttl`; `ttl + stale_ttl` is capped at 7,776,000 s (90 days, the protocol's TTL range). Violations raise `ConfigurationError` at decoration time. A plan whose [maximum TTL](backends/cachekitio.md#ttl-range-and-plan-maximum) is lower clamps the write server-side instead of rejecting it: the fresh TTL is kept first and the stale window gets what is left.
- **CachekitIO only, known at decoration time** — `@cache.io` or an explicit `backend=CachekitIOBackend()`. Other backends have no read-side freshness signal and raise `ConfigurationError` if `stale_ttl` is set; so does a CachekitIO backend resolved lazily from `CACHEKIT_API_KEY` under another preset (the remaining-freshness bound below still applies to its reads).
- Concurrent stale hits trigger at most one revalidation: per-process dedup plus (async functions) a non-blocking distributed lease on the backend's lock. Contested = serve stale, don't wait.
- At most 32 revalidations per decorated function run at once. A stale hit that finds the limit reached serves stale without revalidating, and a later hit retries. An async revalidation stranded on a stopped event loop (sync code that calls `loop.run_until_complete` and leaves the loop stopped, or closes it without cancelling its tasks) gives up its turn 30 seconds after it was scheduled and is cancelled, so a stale hit on another loop can revalidate that key again. If its loop resumes, it stores nothing, even when your function swallows the cancellation and returns. A write it had already sent before its loop stopped is not recalled, and can land after a newer revalidation's.
- A failed background recompute never reaches the caller: the entry keeps serving stale until its hard eviction bound, after which the next call takes the ordinary synchronous miss path. It logs a WARNING `SWR revalidation failed`, as does a revalidation skipped because the call's arguments cannot be deep-copied (`SWR revalidation skipped`) or one that could not be scheduled. It names the function by the same digest as the L1-only refresh WARNING. Each fires at most once a minute per function and carries the count since the last one; the occurrences in between log at DEBUG.
- The background recompute runs with a **snapshot of the caller's `contextvars`** (contextvar-based tenant extraction works), but outside the request otherwise — don't rely on other request-scoped resources (open sessions, connections) inside functions that enable SWR.
- Stale values are never written to the L1 in-memory cache, and stale reads still count as cache **hits** for metered-misses billing.
- On the CachekitIO backend, every read (SWR-configured or not) also carries the server's remaining freshness (`X-CacheKit-Fresh-For`, [protocol spec](https://github.com/cachekit-io/protocol/blob/main/spec/saas-api.md#remaining-freshness)): an L2 hit backfilled into L1 lives at most `min(ttl, remaining)` locally (with `ttl=None`, L1's own 300-second default lifetime, capped by `remaining`), so a value read near the end of its server-side freshness window is never served fresh from L1 past the server's bound. Pre-signal servers omit the header and behavior is unchanged.
- On the CachekitIO backend, every read (SWR-configured or not) also carries the server's remaining freshness (`X-CacheKit-Fresh-For`, [protocol spec](https://github.com/cachekit-io/protocol/blob/main/spec/saas-api.md#remaining-freshness)): an L2 hit backfilled into L1 lives at most `min(ttl, remaining)` locally (with `ttl=None`, L1's own 300-second default lifetime, capped by `remaining`), so a value read near the end of its server-side freshness window is never served fresh from L1 past the server's bound. Pre-signal servers omit the header and behavior is unchanged. The same bound applies to the copy a call keeps of a value it just wrote: the `PUT` response carries `X-CacheKit-Fresh-For` too, so that copy lives at most `min(ttl, fresh_for)` from the response (shorter when the server clamped the TTL to the plan maximum), a malformed or `0` value keeps no L1 copy, and an absent header leaves it at `ttl`.

### File Backend Environment Variables

Expand Down Expand Up @@ -392,7 +392,7 @@ def secure_function():
| `secure()` | 600 s | ❌ | 100 MB | Function runs, no error | AES-256-GCM encryption required; refuses `backend=None`, so no L1-only SWR |
| `io()` | 3600 s | ✓ | 100 MB | Function runs, no error | CachekitIO managed SaaS backend (closed beta — [request access](https://cachekit.io)); past-TTL [SWR](#stale-while-revalidate-stale_ttl) default-on (`stale_ttl = ttl`) |

**Default TTL** is fixed by the cross-SDK [intent-preset spec](https://github.com/cachekit-io/protocol/blob/main/spec/intent-presets.md#default-ttl) so a `production` entry expires at the same moment in Python, Rust and TypeScript; `dev()` / `test()` are SDK-local, Python-only presets and take `minimal`'s 300 s. Pass `ttl=<seconds>` to override, or `ttl=None` to opt in to never-expire explicitly. The spec forbids a process-wide TTL override, so there is no `CACHEKIT_DEFAULT_TTL` (the name is reserved and ignored). Two consequences of a finite default: an entry this process writes also lives in its L1 for the same TTL (previously L1's own 300 s when no `ttl` was set), and the presets' `swr_enabled` / `stale_ttl` features — which need a positive `ttl` — are now active without an explicit `ttl=`.
**Default TTL** is fixed by the cross-SDK [intent-preset spec](https://github.com/cachekit-io/protocol/blob/main/spec/intent-presets.md#default-ttl) so a `production` entry expires at the same moment in Python, Rust and TypeScript; `dev()` / `test()` are SDK-local, Python-only presets and take `minimal`'s 300 s. Pass `ttl=<seconds>` to override, or `ttl=None` to opt in to no TTL explicitly: never-expire, except on CachekitIO, where the server applies your plan's [maximum TTL](backends/cachekitio.md#ttl-range-and-plan-maximum). The spec forbids a process-wide TTL override, so there is no `CACHEKIT_DEFAULT_TTL` (the name is reserved and ignored). Two consequences of a finite default: an entry this process writes also lives in its L1 for the same TTL (previously L1's own 300 s when no `ttl` was set), and the presets' `swr_enabled` / `stale_ttl` features — which need a positive `ttl` — are now active without an explicit `ttl=`.

**Invalidation is not a preset feature.** Every preset's `invalidate_cache()` deletes from L1 and L2 alike. Evicting other processes' L1 copies is the process-wide `CACHEKIT_INVALIDATION_LISTENER_ENABLED` ([Environment Variables](#environment-variables)), off by default and set by no preset ([Cross-Process L1 Eviction](features/l1-invalidation.md#cross-process-l1-eviction)). On the tenant-scoped Redis backend, an entry with `ttl=None` or a TTL above 7 days outlives the key registry's tracking set, so call `invalidate_cache()` before you retire such a function (**Set lifetime** under [Whole-Function Invalidation](features/l1-invalidation.md#whole-function-invalidation)).

Expand Down
2 changes: 1 addition & 1 deletion docs/features/l1-invalidation.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ Things to know:

**Tenant scope:** with the tenant-scoped Redis backend (env auto-detection, or `RedisBackendProvider(...).get_shared_backend()`), each tenant's entries live under its own `t:{tenant}:` prefix. `invalidate_cache()` — with or without arguments — deletes only the L2 entries of the tenant set in `tenant_context` for the calling context (`default` when none is set); other tenants' entries stay cached and tracked. L1 is not tenant-scoped: within a process, all tenants share one L1 entry per cache key, so a tenant can be served the value another tenant cached, and `invalidate_cache()` evicts that entry for every tenant. The exception is an encrypted cache: its AAD [binds the `t:{tenant}:` prefix](zero-knowledge-encryption.md#cache-key-binding), so an L1 entry another tenant cached fails authentication for this one, is skipped as a miss, and the read goes on to L2. Disable L1 on functions whose results differ by tenant: `@cache(..., l1_enabled=False)`, or with a preset `@cache.production(..., l1_enabled=False)`, which keeps the preset's other L1 settings.

**Writes in flight.** An invalidation does not stop a write that began before it. A call that missed the cache before an invalidation still stores its result after it, and so does a `@cache.io` [`stale_ttl`](../configuration.md#stale-while-revalidate-stale_ttl) refresh. The value it stores was computed no earlier than the call started, and it expires by its TTL. With `ttl=None` that is never for an L2 entry, and one year for an entry in L1-only mode. With a backend, the store is two steps, L2 then L1, so an invalidation that arrives between them, from this process or [announced](#cross-process-l1-eviction) by another, can leave the value in this process's L1 alone. Likewise, a read that fetched the old value from L2 just before an invalidation can copy it into this process's L1 just after. Either copy expires by its L1 TTL. In L1-only mode a background SWR refresh never lands after an invalidation: it is dropped when the entry it refreshes has been removed. A call that starts after an invalidation in this process never joins a miss that started before it ([in-process single-flight](distributed-locking.md#in-process-single-flight-before-the-lock)): it runs its own.
**Writes in flight.** An invalidation does not stop a write that began before it. A call that missed the cache before an invalidation still stores its result after it, and so does a `@cache.io` [`stale_ttl`](../configuration.md#stale-while-revalidate-stale_ttl) refresh. The value it stores was computed no earlier than the call started, and it expires by its TTL. With `ttl=None` that is never for an L2 entry (on CachekitIO, your plan's [maximum TTL](../backends/cachekitio.md#ttl-range-and-plan-maximum)), and one year for an entry in L1-only mode. With a backend, the store is two steps, L2 then L1, so an invalidation that arrives between them, from this process or [announced](#cross-process-l1-eviction) by another, can leave the value in this process's L1 alone. Likewise, a read that fetched the old value from L2 just before an invalidation can copy it into this process's L1 just after. Either copy expires by its L1 TTL. In L1-only mode a background SWR refresh never lands after an invalidation: it is dropped when the entry it refreshes has been removed. A call that starts after an invalidation in this process never joins a miss that started before it ([in-process single-flight](distributed-locking.md#in-process-single-flight-before-the-lock)): it runs its own.

### Upgrading: Custom `key=` Keys Carry the Function's Identity

Expand Down
10 changes: 10 additions & 0 deletions src/cachekit/backends/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,21 @@

from collections.abc import Callable, Iterable
from contextlib import AbstractAsyncContextManager
from contextvars import ContextVar
from typing import Any, BinaryIO, Optional, Protocol, runtime_checkable

# Re-export BackendError for convenience (public API)
from cachekit.backends.errors import BackendError # noqa: F401

# Write freshness (API-103). A backend with ``_reports_write_deadline = True`` appends to this list, on each
# successful ``set`` while the decorator holds one open, ``(received, fresh_until)``: the ``time.time()`` its
# 2xx arrived, and the instant the server's freshness for the written value runs out (``received`` plus the
# remaining freshness the response reports, ``math.inf`` when it reports none). The decorator bounds its L1
# copy of the value from them (``cache_handler._written_copy_expiry``). A list, not a value: a write run by
# ``asyncio.to_thread`` sees a copy of the context, which shares the list but not a rebinding. Private, like
# ``_set_tracked``: not part of the backend contract, and free to change.
_WRITE_DEADLINES: ContextVar[Optional[list[tuple[float, float]]]] = ContextVar("cachekit_write_deadlines", default=None)


@runtime_checkable
class BaseBackend(Protocol):
Expand Down
Loading
Loading