diff --git a/changelog.d/20261031_lab-12077.md b/changelog.d/20261031_lab-12077.md new file mode 100644 index 0000000..5ae6804 --- /dev/null +++ b/changelog.d/20261031_lab-12077.md @@ -0,0 +1,33 @@ +### SaaS API — L1 hit-latency telemetry and the telemetry switch + +- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines three + optional headers that report how long calls served from L1 take, as the caller sees them: + `X-CacheKit-L1-Latency-Buckets` (sixteen fixed buckets doubling from 128 ns to about + 2.1 ms, the last one up to 60 s; their sum is the number of hits timed), + `X-CacheKit-L1-Latency-Sum-Ns`, and `X-CacheKit-L1-Latency-One-In` (the sampling + denominator N, so readers weight each report). Values are counted per report, not per + session, kept per CachekitIO backend instance, taken atomically and never re-sent once an + attempt may have been delivered. An SDK may time a random sample of calls, and sends no + latency headers from a runtime whose clock cannot resolve an L1 hit, such as Cloudflare + Workers ([API-79](spec/saas-api.md#api-79)–[API-87](spec/saas-api.md#api-87)). +- [`spec/saas-api.md` § Server handling](spec/saas-api.md#server-handling) (new): the server + validates the latency and session headers as independent sets, never rejects a request over + one, drops valid latency sets only at random and records each kept set's probability, weights + tenants equally (or caps them) in cross-tenant aggregates, and never lets an optional metrics + header change billing or a rate-limit verdict + ([API-88](spec/saas-api.md#api-88)–[API-93](spec/saas-api.md#api-93)). +- [`spec/saas-api.md` § Telemetry switch](spec/saas-api.md#telemetry-switch) (new): every SDK + exposes an optional `telemetry` setting and reads `CACHEKIT_TELEMETRY`, once, at backend + construction; either one turned off stops every optional metrics header. `X-CacheKit-L1-Status` + and any `User-Agent` continue. Default: on + ([API-94](spec/saas-api.md#api-94)–[API-96](spec/saas-api.md#api-96)). +- [`spec/saas-api.md` § Optional Metrics Headers](spec/saas-api.md#optional-metrics-headers): + the headers are client-reported observability, sent only to the configured CachekitIO API + ([API-77](spec/saas-api.md#api-77)). +- [`spec/saas-api.md` § L1 hit rate](spec/saas-api.md#l1-hit-rate) (new) pins + `X-CacheKit-L1-Hit-Rate` to `L1-Hits / (L1-Hits + L2-Hits + Misses)` + ([API-78](spec/saas-api.md#api-78)). **Changes cachekit-py**, which sends + `L1-Hits / (L1-Hits + L2-Hits)`. +- [`sdk-feature-matrix.md` § Observability](sdk-feature-matrix.md#observability): rows for the + latency headers and the telemetry switch, unshipped in every SDK, and the Python hit-rate + difference on the session-headers row. diff --git a/conformance/coverage.md b/conformance/coverage.md index 60c5074..5abf115 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -25,14 +25,14 @@ How ids are assigned, and what each status means: [README.md](README.md#coverage | [`spec/file-backend-format.md`](../spec/file-backend-format.md) | 6 | 1 | 3 | 0 | 2 | | [`spec/intent-presets.md`](../spec/intent-presets.md) | 55 | 0 | 19 | 0 | 36 | | [`spec/interop-mode.md`](../spec/interop-mode.md) | 36 | 5 | 23 | 4 | 4 | -| [`spec/saas-api.md`](../spec/saas-api.md) | 76 | 0 | 15 | 0 | 61 | +| [`spec/saas-api.md`](../spec/saas-api.md) | 96 | 0 | 15 | 0 | 81 | | [`spec/wire-format.md`](../spec/wire-format.md) | 24 | 3 | 18 | 3 | 0 | | SDK | Covered | Partial | Uncovered | Gap | n/a | | :--- | ---: | ---: | ---: | ---: | ---: | -| cachekit-py | 12 | 86 | 5 | 69 | 47 | -| cachekit-ts | 9 | 78 | 7 | 59 | 66 | -| cachekit-rs | 4 | 69 | 3 | 55 | 88 | +| cachekit-py | 12 | 86 | 5 | 83 | 53 | +| cachekit-ts | 9 | 78 | 7 | 73 | 72 | +| cachekit-rs | 4 | 69 | 3 | 69 | 94 | Not indexed yet: [`spec/interop-v2.md`](../spec/interop-v2.md). @@ -274,6 +274,26 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-74](../spec/saas-api.md#api-74) | *SSRF Protection*: All SDKs **MUST** enforce SSRF protection when accepting the API URL. The following rules apply: | **Gap:** No vector holds an API URL, so nothing here checks HTTPS-only, the listed private ranges or the hostname allowlist. | gap | gap | gap | | [API-75](../spec/saas-api.md#api-75) | *Counting outcomes*: Plain read. A value read **MUST** count one outcome: an L1 hit when the in-memory layer served it, an L2 hit when a backend read did, or a miss when … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a plain read twice, or counts a read that failed at the backend; only the SDK's own counter tests do. | gap | gap | gap | | [API-76](../spec/saas-api.md#api-76) | *Counting outcomes*: … caching wrapper (the cachekit-py decorators, cachekit-ts `wrap()`, cachekit-rs `#[cachekit]`) **MUST** count one outcome, however many reads it makes. A call that runs the wrapped function counts one miss, before its … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do. | gap | gap | gap | +| [API-77](../spec/saas-api.md#api-77) | *Optional Metrics Headers*: … them only while telemetry is on (Telemetry switch), and only from its CachekitIO backend: an SDK **MUST NOT** send them, or a report built for them, to any host but the CachekitIO API that backend is configured with. A Redis, … | **Gap:** No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or a report built for them, through a Redis, Memcached or File backend or to any host but its configured CachekitIO API. | gap | gap | gap | +| [API-78](../spec/saas-api.md#api-78) | *L1 hit rate*: An SDK **MUST** compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same … | **Gap:** No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), sends it with other than three decimal places, or sends other than 0.000 for all-zero counters. | gap | gap | gap | +| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … start of the wrapped function. Only work that runs after the hand-back is outside it. An SDK **MUST** add to a report only the durations of calls it counts as an L1 hit and that return a value. It MUST NOT add the … | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report; only the SDK's own tests do. | gap | gap | gap | +| [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … add to a report only the durations of calls it counts as an L1 hit and that return a value. It **MUST NOT** add the duration of any of these: | **Gap:** As API-79, for a client that adds a joined in-process call, an existence check or a read made inside a wrapped call. | gap | gap | gap | +| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It **MUST** decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call … | **Gap:** As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, lets the key, arguments, position, outcome or duration sway the draw, or draws with a probability other than the 1/N it sends. | gap | gap | gap | +| [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers. | gap | gap | gap | +| [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: … that backend's requests, so one backend's hits never ride another's requests. A process that forks **MUST** start the child with an empty accumulator, so that the parent's hits are not reported twice. | **Gap:** No vector reaches a forked process, so nothing here catches a child that inherits and re-sends its parent's accumulated report. | gap | gap | gap | +| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … sum test is part of the atomic step below, so a racing request never attaches all-zero buckets. It **MUST NOT** attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report … | **Gap:** No vector covers request headers, so nothing here catches a client that attaches a latency report to a health request. | gap | gap | gap | +| [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: … request, which the server does not record. When an SDK attaches a report to a request, it **MUST** take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values. | gap | gap | gap | +| [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … that carries it may have reached the server, the report is consumed whatever the outcome: the SDK **MUST NOT** add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an … | **Gap:** No vector covers request headers, so nothing here catches a client that adds a consumed report back to its accumulator, or whose own or its HTTP library's retries resend it after an attempt that may have been delivered. | gap | gap | gap | +| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and these rules: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, a duration in the wrong bucket, or a duration above 60 s. | gap | gap | gap | +| [API-88](../spec/saas-api.md#api-88) | *Server handling*: The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | +| [API-89](../spec/saas-api.md#api-89) | *Server handling*: … headers fail any of the server's checks, and the reverse. Within the latency set, the server **MUST** drop all three latency values when any of these holds: a header is missing or malformed; the sum is 0; the One-In … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that records a latency set with a missing or malformed header, a sum of 0, a One-In outside 1 to 1,000,000, or a sum its buckets cannot produce. | n/a | n/a | n/a | +| [API-90](../spec/saas-api.md#api-90) | *Server handling*: … most the sum of each count times the bucket's bound (60,000,000,000 ns for bucket 16). The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | +| [API-91](../spec/saas-api.md#api-91) | *Server handling*: … its response status, except `GET /v1/cache/health`. A server that records valid latency sets **MUST** keep or drop each one only by a random draw, whose probability may vary by tenant, time or load, for example to stay … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops latency sets deterministically, such as once a write budget is spent, or keeps them without recording the probability it kept each one. | n/a | n/a | n/a | +| [API-92](../spec/saas-api.md#api-92) | *Server handling*: … every other client together. An aggregate of latency sets across tenants, or one shown publicly, **MUST** weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. | Binds the server.
**Gap:** How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-93](../spec/saas-api.md#api-93) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting, or let the value of `X-CacheKit-L1-Status` change … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read, including whether billing reads X-CacheKit-L1-Status, is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-94](../spec/saas-api.md#api-94) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, or ignores CACHEKIT_TELEMETRY. | gap | gap | gap | +| [API-95](../spec/saas-api.md#api-95) | *Telemetry switch*: An SDK **MUST** read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend … | **Gap:** As API-94, for a client that lets a configuration loader fill the setting from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction. | gap | gap | gap | +| [API-96](../spec/saas-api.md#api-96) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header. | gap | gap | gap | Marked not-a-requirement in `spec/saas-api.md`: diff --git a/conformance/requirements.json b/conformance/requirements.json index 16a81fa..e9e7199 100644 --- a/conformance/requirements.json +++ b/conformance/requirements.json @@ -1521,7 +1521,7 @@ }, "spec/saas-api.md": { "prefix": "API", - "next": 77, + "next": 97, "requirements": { "API-1": { "section": "Authentication", @@ -2110,6 +2110,106 @@ "section": "Counting outcomes", "binds": "sdk", "gap": "No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do." + }, + "API-77": { + "section": "Optional Metrics Headers", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or a report built for them, through a Redis, Memcached or File backend or to any host but its configured CachekitIO API." + }, + "API-78": { + "section": "L1 hit rate", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), sends it with other than three decimal places, or sends other than 0.000 for all-zero counters." + }, + "API-79": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report; only the SDK's own tests do." + }, + "API-80": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "As API-79, for a client that adds a joined in-process call, an existence check or a read made inside a wrapped call." + }, + "API-81": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, lets the key, arguments, position, outcome or duration sway the draw, or draws with a probability other than the 1/N it sends." + }, + "API-82": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers." + }, + "API-83": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector reaches a forked process, so nothing here catches a child that inherits and re-sends its parent's accumulated report." + }, + "API-84": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that attaches a latency report to a health request." + }, + "API-85": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values." + }, + "API-86": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that adds a consumed report back to its accumulator, or whose own or its HTTP library's retries resend it after an attempt that may have been delivered." + }, + "API-87": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, a duration in the wrong bucket, or a duration above 60 s." + }, + "API-88": { + "section": "Server handling", + "binds": "server", + "gap": "No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected." + }, + "API-90": { + "section": "Server handling", + "binds": "server", + "gap": "As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed." + }, + "API-91": { + "section": "Server handling", + "binds": "server", + "gap": "No public vector exercises the server, so nothing here catches one that drops latency sets deterministically, such as once a write budget is spent, or keeps them without recording the probability it kept each one." + }, + "API-92": { + "section": "Server handling", + "binds": "server", + "gap": "How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe." + }, + "API-93": { + "section": "Server handling", + "binds": "server", + "gap": "Which inputs a server's billing and rate-limit verdicts read, including whether billing reads X-CacheKit-L1-Status, is server code no public vector or SDK test can observe." + }, + "API-94": { + "section": "Telemetry switch", + "binds": "sdk", + "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, or ignores CACHEKIT_TELEMETRY." + }, + "API-95": { + "section": "Telemetry switch", + "binds": "sdk", + "gap": "As API-94, for a client that lets a configuration loader fill the setting from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction." + }, + "API-96": { + "section": "Telemetry switch", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header." + }, + "API-89": { + "section": "Server handling", + "binds": "server", + "gap": "No public vector exercises the server, so nothing here catches one that records a latency set with a missing or malformed header, a sum of 0, a One-In outside 1 to 1,000,000, or a sum its buckets cannot produce." } }, "retired": {} diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 9c92bbf..14c7bd2 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -229,7 +229,9 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Metrics | ✅ Live `prometheus_client` counters / gauges / histograms on the default registry; no HTTP exposition helper | ✅ `cachekit-rs` 0.8.0+ — live counters ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). `CacheKit::stats()` returns `L1Stats`, plus `l1_entry_count()` and `circuit_state()`; no Prometheus exposition (LAB-521) | ✅ Node — live Prometheus metrics via the optional `prom-client` peer dep, custom registry supported, warns once and no-ops if absent (LAB-517) · ❌ Workers — `prom-client` is CI-excluded from the edge bundle¹ | ❌ | | Structured logging | ✅ JSON ring-buffer logger with sensitive-data masking (`logging.py`) | ✅ `cachekit-rs` 0.8.0+ — `tracing` events ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Opt-in `tracing` cargo feature: one `debug` event per get/set/delete, carrying `key_hash` (unkeyed Blake2b-128 of the storage key: a correlator, not a redaction), plus breaker transitions, to the host's subscriber (LAB-521) | ⚠️ Pluggable error-logger hook (`setLogger`, default `console.error`) — a logging seam, not a structured logger (LAB-517) | ❌ | | Distributed tracing (OTel) | ❌ The span-shaped API accepts spans and discards them (`NoOpSpan` in `FeatureOrchestrator.create_span`, `decorators/orchestrator.py`) — instrumentation built against it silently produces nothing | ❌ | ❌ | ❌ | -| SaaS telemetry headers (`X-CacheKit-L1-*`) | ✅ Auto. The decorators count one outcome per call ([Counting outcomes](spec/saas-api.md#counting-outcomes)) | ✅ `cachekit-rs` 0.8.0+ ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Auto-wired from live counters; an explicit `.metrics_provider()` still wins (LAB-521) · ⚠️ through 0.13.0 a `#[cachekit]` call can count up to about 51 misses ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-rs#196](https://github.com/cachekit-io/cachekit-rs/pull/196) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins · ⚠️ through npm 0.2.1 a `wrap()` call can count two outcomes or none ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-ts#265](https://github.com/cachekit-io/cachekit-ts/pull/265) | ❌ | +| SaaS session telemetry headers (`X-CacheKit-Session-*`, `-L1-Hits`, `-L2-Hits`, `-Misses`, `-L1-Hit-Rate`) | ✅ Auto. The decorators count one outcome per call ([Counting outcomes](spec/saas-api.md#counting-outcomes)) · ⚠️ sends `X-CacheKit-L1-Hit-Rate` as L1-Hits / (L1-Hits + L2-Hits), not the share of all counted operations [API-78](spec/saas-api.md#api-78) pins | ✅ `cachekit-rs` 0.8.0+ ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Auto-wired from live counters; an explicit `.metrics_provider()` still wins (LAB-521) · ⚠️ through 0.13.0 a `#[cachekit]` call can count up to about 51 misses ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-rs#196](https://github.com/cachekit-io/cachekit-rs/pull/196) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins · ⚠️ through npm 0.2.1 a `wrap()` call can count two outcomes or none ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-ts#265](https://github.com/cachekit-io/cachekit-ts/pull/265) | ❌ | +| SaaS L1 hit-latency headers (`X-CacheKit-L1-Latency-*`) | ❌ Specified ([L1 hit latency](spec/saas-api.md#l1-hit-latency)), not shipped. Backed-mode L1 hits are timed only for the local Prometheus histogram | ❌ Specified, not shipped. No L1 timer; never on the `workers` feature, whose clock cannot time an L1 hit ([API-82](spec/saas-api.md#api-82)) | ❌ Specified, not shipped. No L1 timer; never on the Workers entry, whose clock cannot time an L1 hit ([API-82](spec/saas-api.md#api-82)) | ❌ | +| Telemetry switch (`telemetry` setting, `CACHEKIT_TELEMETRY`) | ❌ Specified ([Telemetry switch](spec/saas-api.md#telemetry-switch)), not shipped: no switch; the session headers ride every request made inside a decorated call | ❌ Specified, not shipped: the session headers are always sent | ❌ Specified, not shipped: the session headers are always sent | ❌ | --- diff --git a/spec/saas-api.md b/spec/saas-api.md index e4a179f..267f104 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -517,7 +517,9 @@ Authorization: Bearer ck_live_xxx ### Optional Metrics Headers -SDKs SHOULD send cache metrics headers for rate limiting and observability: +The optional metrics headers are client-reported telemetry, for observability only. An SDK sends them only while telemetry is on ([Telemetry switch](#telemetry-switch)), and only from its CachekitIO backend: an SDK MUST NOTAPI-77 send them, or a report built for them, to any host but the CachekitIO API that backend is configured with. A Redis, Memcached or File backend, or an L1-only configuration, sends no telemetry anywhere. Metrics an SDK exposes inside the user's own process, such as Prometheus counters or a `stats()` call, are outside these rules. While telemetry is on, SDKs SHOULD send the session headers on every request, and the latency headers whenever they have a report to send ([L1 hit latency](#l1-hit-latency)). + +**Session headers.** The counters are cumulative for the session ([Counting outcomes](#counting-outcomes)). | Header | Type | Description | | :--- | :--- | :--- | @@ -526,9 +528,15 @@ SDKs SHOULD send cache metrics headers for rate limiting and observability: | `X-CacheKit-L1-Hits` | integer | Count of L1 (in-memory) cache hits | | `X-CacheKit-L2-Hits` | integer | Count of L2 (backend) cache hits | | `X-CacheKit-Misses` | integer | Count of cache misses | -| `X-CacheKit-L1-Hit-Rate` | float | L1 hit rate (0.000 to 1.000, 3 decimal places) | +| `X-CacheKit-L1-Hit-Rate` | decimal | Share of counted operations L1 served, `0.000` to `1.000` ([L1 hit rate](#l1-hit-rate)) | -`X-CacheKit-L1-Status` — see [Required Headers](#required-headers). +**Latency headers.** The values cover one report, not the session ([L1 hit latency](#l1-hit-latency)). + +| Header | Type | Description | +| :--- | :--- | :--- | +| `X-CacheKit-L1-Latency-Buckets` | 16 integers | The report's timed L1 hits, counted into sixteen fixed duration buckets. Their sum is the number of hits timed | +| `X-CacheKit-L1-Latency-Sum-Ns` | integer | The sum of those hits' durations, in nanoseconds | +| `X-CacheKit-L1-Latency-One-In` | integer | N: each call was timed with probability 1/N | #### Counting outcomes @@ -539,6 +547,83 @@ SDKs SHOULD send cache metrics headers for rate limiting and observability: A stale serve counts one hit by the layer that served it (a backend stale-window `GET` is an L2 hit, an L1 stale serve an L1 hit); its background revalidation counts nothing, though it runs the wrapped function ([API-63](#api-63)). +#### L1 hit rate + +An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same request, rounded to three decimal places (either way on an exact tie), and send `0.000` while all three counters are 0. The value is written as `0.` or `1.` followed by exactly three digits, and never exceeds `1.000`. It is the share of counted operations that L1 served, the figure the documentation reports. The share of hits alone, `L1-Hits / (L1-Hits + L2-Hits)`, is not this header. A server or reader that aggregates requests SHOULD compute the rate from the three counters rather than read this header. + +#### L1 hit latency + +**What is timed.** A duration is one cache call counted as an L1 hit ([API-75](#api-75), [API-76](#api-76)), as its caller sees it. The clock starts at the first SDK instruction the call runs: for a coroutine or an async function, its first step, not its creation. It stops in the same synchronous step that produces the value the SDK hands back, never in a continuation or callback that runs after it. The interval includes all work the SDK does before it hands the value back: deriving the cache key, the L1 lookup, any decryption and deserialization, and any revalidation work an L1 stale serve runs before the value returns, such as the synchronous start of the wrapped function. Only work that runs after the hand-back is outside it. An SDK MUSTAPI-79 add to a report only the durations of calls it counts as an L1 hit and that return a value. It MUST NOTAPI-80 add the duration of any of these: + +- a wrapped call that joined another in-process call for the same key, because its duration is the other call's work; +- an existence check; +- a read made inside a wrapped call, such as the `get` a wrapper makes on the SDK's own public API, which [API-76](#api-76) does not count either. + +The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. + +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. + +**Clock.** An SDK MUST NOTAPI-82 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: + +| SDK | Clock | +| :--- | :--- | +| Python | `time.perf_counter_ns()`, where `time.get_clock_info("perf_counter").resolution` is 1e-7 or less | +| Rust | `std::time::Instant`, on any target but `wasm32` | +| TypeScript on Node.js | `process.hrtime.bigint()` | + +Any other clock qualifies only where the SDK's documentation names it and its resolution. Cloudflare Workers has no qualifying clock: its clock advances only on I/O, so an in-memory hit would read as 0 ns. An SDK that cannot tell which runtime it is on treats its clock as not qualifying, and sends no latency headers at all, never zeros. On a qualifying clock with a 100 ns tick, a fast hit can read 0 ns; that is a measurement, and it counts. + +**Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the previous report. An SDK keeps one report accumulator per CachekitIO backend instance. It times a call into the accumulator of the backend that call's cache is configured with, and attaches the report only to that backend's requests, so one backend's hits never ride another's requests. A process that forks MUSTAPI-83 start the child with an empty accumulator, so that the parent's hits are not reported twice. + +An SDK attaches a report only when its duration sum is at least 1; a report whose sum is 0 is not attached and stays in the accumulator. The sum test is part of the atomic step below, so a racing request never attaches all-zero buckets. It MUST NOTAPI-84 attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report to a request, it MUSTAPI-85 take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and the buckets and the sum always describe the same hits. Separate atomic swaps of each field do not satisfy this; one structure taken under a lock, or swapped as a unit, does. It sends the three latency headers together. A report rides one attempt. Once an attempt that carries it may have reached the server, the report is consumed whatever the outcome: the SDK MUST NOTAPI-86 add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an attempt the HTTP layer knows was never delivered, such as a refused connection, may be retried with the same report. An SDK whose HTTP layer resends requests that may have been delivered, without re-running the code that attaches the report, does not use that layer's retries for a request that carries one. Hits timed after the client's last request are never reported; there is no background reporter. + +Per-report values were chosen over the session counters' cumulative form for two reasons. A reader adds them across requests, sessions and clients as they stand, while cumulative values need each session's previous value to difference. And they need no per-session state: a server may discard a session-headers sample whose counters are lower than an earlier sample for the same Session-ID, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. A consumed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. Reports are lost when their request fails or their process exits first, and those losses are not independent of load, so the latency data is a sample of the hits, not a census. A report carries no measurement time: its hits happened at some point since the previous report, so a reader that groups reports by time groups them by arrival. + +**Encoding.** No mean or percentile goes on the wire, because neither can be added across reports. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-87 send values that meet this grammar and these rules: + +- `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. A duration above 60,000,000,000 ns (60 s) is discarded, not counted, so that the server's range check always has an upper limit. The latency data therefore describes L1 hits of 60 s or less. A longer one, such as a hit held up by a slow synchronous revalidation, is outside it, and no figure built from the data covers it. +- `X-CacheKit-L1-Latency-Sum-Ns`: the sum of the same hits' durations, each in whole nanoseconds rounded down; at least 1. +- `X-CacheKit-L1-Latency-One-In`: N, the backend instance's sampling denominator, from 1 to 1,000,000. + +| Bucket | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| *d* (ns) at most | 128 | 256 | 512 | 1,024 | 2,048 | 4,096 | 8,192 | 16,384 | + +| Bucket | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| *d* (ns) at most | 32,768 | 65,536 | 131,072 | 262,144 | 524,288 | 1,048,576 | 2,097,152 | 60,000,000,000 | + +Each bucket holds the durations above the previous bucket's bound, up to and including its own; bucket 1 holds everything up to 128 ns, 0 included. Example: with N = 1, hits of 90, 300, 300 and 40,000 ns are sent as buckets `1,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0`, sum `40690` and one-in `1`. + +A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to about 2.1 ms cover the sub-microsecond hits the SDKs target, the tens of microseconds an encrypted L1 hit on a large value can take, and the pauses above them; a reader estimates a percentile by interpolating within the bucket that holds it. Bucket 16 spans 2,097,153 ns to 60 s, so a percentile that falls in it is reported only as above 2,097,152 ns. A change to the bounds needs a new header name. + +#### Server handling + +The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-88 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail any of the server's checks, and the reverse. Within the latency set, the server MUSTAPI-89 drop all three latency values when any of these holds: a header is missing or malformed; the sum is 0; the One-In value is outside 1 to 1,000,000; or the sum is outside the range its buckets allow, which is at least the sum, over the buckets, of each count times the bucket's lowest duration (the previous bucket's bound plus 1, and 0 for bucket 1), and at most the sum of each count times the bucket's bound (60,000,000,000 ns for bucket 16). The server MUST NOTAPI-90 answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid latency set on every authenticated request it admits, whatever its response status, except `GET /v1/cache/health`. A server that records valid latency sets MUSTAPI-91 keep or drop each one only by a random draw, whose probability may vary by tenant, time or load, for example to stay within a write budget, and record with each kept set the probability p it was kept (1 when it keeps every one). It never drops sets deterministically once a budget is spent: requests that carry no session headers are not a random subset of clients, so a set dropped without a recorded probability biases what remains. A reader weights each kept set by N / p, times any sampling interval its storage layer applies. It does that arithmetic, like the range check, in IEEE 754 doubles or exact integers: N × Sum-Ns, and a count times 60,000,000,000, can overflow a 64-bit integer at grammatical values. + +Authentication proves who holds a key, not that its measurements are true: one set of sixteen grammatical, consistent values can claim more hits than every other client together. An aggregate of latency sets across tenants, or one shown publicly, MUSTAPI-92 weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. + +The server MUST NOTAPI-93 let an optional metrics header change billing or rate limiting, or let the value of `X-CacheKit-L1-Status` change billing: all of them are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. `X-CacheKit-L1-Status` may change rate limiting, as [API-1](#api-1) describes; headers that carry an operation's own inputs, such as `X-CacheKit-TTL`, are outside this rule. + +#### Telemetry switch + +Every SDK MUSTAPI-94 let its user turn telemetry off through both of these controls, which mean the same in every SDK: + +- a setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend. It is an optional boolean that is unset unless the user sets it (Python `None`, TypeScript `undefined`, Rust `None`): an SDK does not give it a default of true or false; +- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). The SDK reads the process environment wherever the runtime has one. On Cloudflare Workers it reads `CACHEKIT_TELEMETRY` from both `process.env` and the Worker's bindings (`env` imported from `cloudflare:workers`, or the bindings it is constructed with); off from either wins. Only an SDK on a runtime with neither a process environment nor bindings documents that only the setting applies there. + +An SDK MUSTAPI-95 read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend instance's lifetime. No configuration loader fills the `telemetry` setting from the environment, and `CACHEKIT_TELEMETRY` is parsed only by this rule, never by a loader's own boolean parsing or precedence: a loader that lets a code argument override the environment would let on win over off. + +`CACHEKIT_TELEMETRY` is read case-insensitively, after trimming surrounding white space: `false`, `0`, `off` and `no` are off; `true`, `1`, `on` and `yes` are on; unset or empty is unset. Any other value is off, and the SDK SHOULD warn once: a user who set the variable meant to change something, and off is the safe reading. Each control is then on, off or unset, and they combine like this: + +| Setting \ Environment | unset | on | off | +| :--- | :--- | :--- | :--- | +| **unset** | the default: on | on | off | +| **on** | on | on | off | +| **off** | off | off | off | + +With telemetry off, an SDK MUST NOTAPI-96 send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Buckets`, `X-CacheKit-L1-Latency-Sum-Ns` or `X-CacheKit-L1-Latency-One-In`. With telemetry off, an SDK SHOULD NOT accumulate a latency report. Besides the headers each operation needs, two continue: `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session; and a `User-Agent` that may name the SDK and its version, its runtime, and its HTTP library and that library's version. + --- ## Error Handling @@ -586,6 +671,7 @@ SDKs should classify errors for circuit breaker integration: | `CACHEKIT_MAX_RETRIES` | No | `3` | Max retry attempts | | `CACHEKIT_CONNECTION_POOL_SIZE` | No | `10` | HTTP connection pool size | | `CACHEKIT_ALLOW_CUSTOM_HOST` | No | `false` | Allow non-standard API hostnames | +| `CACHEKIT_TELEMETRY` | No | unset (on) | Turns the optional metrics headers off; values and precedence in [Telemetry switch](#telemetry-switch) | ### SSRF Protection