From 777fdde17f0601cd5b10baf76a92b20af2e9ec20 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 20:05:37 +1100 Subject: [PATCH 1/7] feat(spec): L1 hit-latency telemetry headers and the SDK telemetry switch Defines three optional per-report headers carrying a count, a nanosecond sum and ten fixed bucket counts for calls served from L1, the telemetry setting and CACHEKIT_TELEMETRY that turn the optional metrics headers off, one L1-Hit-Rate formula, and server handling that keeps the latency and session header sets independent. Requirement ids API-77 to API-91. --- changelog.d/20261031_lab-12077.md | 26 +++++++++++ conformance/coverage.md | 23 +++++++-- conformance/requirements.json | 77 ++++++++++++++++++++++++++++++- sdk-feature-matrix.md | 4 +- spec/saas-api.md | 62 +++++++++++++++++++++++-- 5 files changed, 183 insertions(+), 9 deletions(-) create mode 100644 changelog.d/20261031_lab-12077.md diff --git a/changelog.d/20261031_lab-12077.md b/changelog.d/20261031_lab-12077.md new file mode 100644 index 0000000..c24f045 --- /dev/null +++ b/changelog.d/20261031_lab-12077.md @@ -0,0 +1,26 @@ +### SaaS API — L1 hit-latency telemetry and the telemetry switch + +- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines three + optional headers that report how long calls served from L1 take, as the caller sees them: + `X-CacheKit-L1-Latency-Count`, `X-CacheKit-L1-Latency-Sum-Ns` and + `X-CacheKit-L1-Latency-Buckets` (ten fixed buckets doubling from 128 ns to 32.768 µs, plus + an open last bucket). Values are counted per report, not per session; a report is sent at + most once. An SDK may time a sample of calls, and sends no latency headers on a runtime + whose clock cannot resolve an L1 hit, such as Cloudflare Workers + ([API-79](spec/saas-api.md#api-79)–[API-86](spec/saas-api.md#api-86)). +- [`spec/saas-api.md` § Telemetry switch](spec/saas-api.md#telemetry-switch) (new): every SDK + exposes a `telemetry` setting and reads `CACHEKIT_TELEMETRY`; either one turned off stops + every optional metrics header. `X-CacheKit-L1-Status` is still sent. Default: on + ([API-90](spec/saas-api.md#api-90), [API-91](spec/saas-api.md#api-91)). +- [`spec/saas-api.md` § Optional Metrics Headers](spec/saas-api.md#optional-metrics-headers): + the headers are client-reported observability, sent only by the CachekitIO backend + ([API-77](spec/saas-api.md#api-77)); the server never uses them for billing or rate limiting + and never rejects a request over one, and a bad latency set never drops the session headers, + or the reverse ([API-87](spec/saas-api.md#api-87)–[API-89](spec/saas-api.md#api-89)). +- [`spec/saas-api.md` § L1 hit rate](spec/saas-api.md#l1-hit-rate) (new) pins + `X-CacheKit-L1-Hit-Rate` to `L1-Hits / (L1-Hits + L2-Hits + Misses)` + ([API-78](spec/saas-api.md#api-78)). **Changes cachekit-py**, which sends + `L1-Hits / (L1-Hits + L2-Hits)`. +- [`sdk-feature-matrix.md` § Observability](sdk-feature-matrix.md#observability): rows for the + latency headers and the telemetry switch, unshipped in every SDK, and the Python hit-rate + difference on the telemetry-headers row. diff --git a/conformance/coverage.md b/conformance/coverage.md index 60c5074..e061b94 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -25,14 +25,14 @@ How ids are assigned, and what each status means: [README.md](README.md#coverage | [`spec/file-backend-format.md`](../spec/file-backend-format.md) | 6 | 1 | 3 | 0 | 2 | | [`spec/intent-presets.md`](../spec/intent-presets.md) | 55 | 0 | 19 | 0 | 36 | | [`spec/interop-mode.md`](../spec/interop-mode.md) | 36 | 5 | 23 | 4 | 4 | -| [`spec/saas-api.md`](../spec/saas-api.md) | 76 | 0 | 15 | 0 | 61 | +| [`spec/saas-api.md`](../spec/saas-api.md) | 91 | 0 | 15 | 0 | 76 | | [`spec/wire-format.md`](../spec/wire-format.md) | 24 | 3 | 18 | 3 | 0 | | SDK | Covered | Partial | Uncovered | Gap | n/a | | :--- | ---: | ---: | ---: | ---: | ---: | -| cachekit-py | 12 | 86 | 5 | 69 | 47 | -| cachekit-ts | 9 | 78 | 7 | 59 | 66 | -| cachekit-rs | 4 | 69 | 3 | 55 | 88 | +| cachekit-py | 12 | 86 | 5 | 81 | 50 | +| cachekit-ts | 9 | 78 | 7 | 71 | 69 | +| cachekit-rs | 4 | 69 | 3 | 67 | 91 | Not indexed yet: [`spec/interop-v2.md`](../spec/interop-v2.md). @@ -274,6 +274,21 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-74](../spec/saas-api.md#api-74) | *SSRF Protection*: All SDKs **MUST** enforce SSRF protection when accepting the API URL. The following rules apply: | **Gap:** No vector holds an API URL, so nothing here checks HTTPS-only, the listed private ranges or the hostname allowlist. | gap | gap | gap | | [API-75](../spec/saas-api.md#api-75) | *Counting outcomes*: Plain read. A value read **MUST** count one outcome: an L1 hit when the in-memory layer served it, an L2 hit when a backend read did, or a miss when … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a plain read twice, or counts a read that failed at the backend; only the SDK's own counter tests do. | gap | gap | gap | | [API-76](../spec/saas-api.md#api-76) | *Counting outcomes*: … caching wrapper (the cachekit-py decorators, cachekit-ts `wrap()`, cachekit-rs `#[cachekit]`) **MUST** count one outcome, however many reads it makes. A call that runs the wrapped function counts one miss, before its … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do. | gap | gap | gap | +| [API-77](../spec/saas-api.md#api-77) | *Optional Metrics Headers*: … them only while telemetry is on (Telemetry switch), and only from its CachekitIO backend: an SDK **MUST NOT** send them, or the measurements they carry, through any other backend or to any other destination. A Redis, Memcached … | **Gap:** No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or their values, through a Redis, Memcached or File backend or anywhere other than the CachekitIO API. | gap | gap | gap | +| [API-78](../spec/saas-api.md#api-78) | *L1 hit rate*: An SDK **MUST** compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same … | **Gap:** No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), or sends it with other than three decimal places or a value other than 0.000 for all-zero counters. | gap | gap | gap | +| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … happens after the hand-back, such as an event loop scheduling the caller's continuation. An SDK **MUST** time only calls it counts as an L1 hit. It MUST NOT time a wrapped call that joined another in-process call for the … | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits or misses into the L1 latency report; only the SDK's own tests do. | gap | gap | gap | +| [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … loop scheduling the caller's continuation. An SDK MUST time only calls it counts as an L1 hit. It **MUST NOT** time a wrapped call that joined another in-process call for the same key, whose duration is the other call's work, or … | **Gap:** As API-79, for a client that times a wrapped call that joined another in-process call, or an existence check. | gap | gap | gap | +| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It **MUST** decide whether to time a call before it knows the call's outcome, and that decision MUST NOT depend on the key, the … | **Gap:** As API-79, for a client that decides to time a call only after it knows the call was an L1 hit. | gap | gap | gap | +| [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: … hit. It MUST decide whether to time a call before it knows the call's outcome, and that decision **MUST NOT** depend on the key, the arguments, the outcome or the duration. A per-client counter (every Nth call) or a random draw … | **Gap:** As API-79, for a client whose sampling decision depends on the key, the arguments, the outcome or the duration. | gap | gap | gap | +| [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers from a runtime whose clock does not advance during synchronous execution or resolves coarser than 100 ns, such as Cloudflare Workers. | gap | gap | gap | +| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … the hits timed since the SDK's previous report. When an SDK attaches a report to a request, it **MUST** take the values and reset its accumulators in one atomic step, so that two concurrent requests never carry the same … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent requests carry the same timed hit, or that sends cumulative rather than per-report values. | gap | gap | gap | +| [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: … request carries only what has been timed since, and a report whose request fails is lost: the SDK **MUST NOT** add its values back. Hits timed after the client's last request are never reported; there is no background reporter. | **Gap:** No vector covers request headers, so nothing here catches a client that re-sends a report on a retry, or adds a failed request's values back to its accumulators. | gap | gap | gap | +| [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and the rules below: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, a count of 0, buckets other than ten, buckets that do not sum to the count, or a duration in the wrong bucket. | gap | gap | gap | +| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches a server that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | +| [API-88](../spec/saas-api.md#api-88) | *L1 hit latency*: … header, or buckets that do not sum to the count, drops all three latency values. The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed. It MUST NOT use the … | Binds the server.
**Gap:** As API-87, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | +| [API-89](../spec/saas-api.md#api-89) | *L1 hit latency*: … NOT answer a request with an error because an optional metrics header is missing or malformed. It **MUST NOT** use the optional metrics headers as input to billing or to rate limiting: they are client-reported, and only … | Binds the server.
**Gap:** Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-90](../spec/saas-api.md#api-90) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on. | gap | gap | gap | +| [API-91](../spec/saas-api.md#api-91) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it. | gap | gap | gap | Marked not-a-requirement in `spec/saas-api.md`: diff --git a/conformance/requirements.json b/conformance/requirements.json index 16a81fa..626eae8 100644 --- a/conformance/requirements.json +++ b/conformance/requirements.json @@ -1521,7 +1521,7 @@ }, "spec/saas-api.md": { "prefix": "API", - "next": 77, + "next": 92, "requirements": { "API-1": { "section": "Authentication", @@ -2110,6 +2110,81 @@ "section": "Counting outcomes", "binds": "sdk", "gap": "No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do." + }, + "API-77": { + "section": "Optional Metrics Headers", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or their values, through a Redis, Memcached or File backend or anywhere other than the CachekitIO API." + }, + "API-78": { + "section": "L1 hit rate", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), or sends it with other than three decimal places or a value other than 0.000 for all-zero counters." + }, + "API-79": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits or misses into the L1 latency report; only the SDK's own tests do." + }, + "API-80": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "As API-79, for a client that times a wrapped call that joined another in-process call, or an existence check." + }, + "API-81": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "As API-79, for a client that decides to time a call only after it knows the call was an L1 hit." + }, + "API-82": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "As API-79, for a client whose sampling decision depends on the key, the arguments, the outcome or the duration." + }, + "API-83": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends latency headers from a runtime whose clock does not advance during synchronous execution or resolves coarser than 100 ns, such as Cloudflare Workers." + }, + "API-84": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers concurrent requests, so nothing here catches a client whose concurrent requests carry the same timed hit, or that sends cumulative rather than per-report values." + }, + "API-85": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that re-sends a report on a retry, or adds a failed request's values back to its accumulators." + }, + "API-86": { + "section": "L1 hit latency", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, a count of 0, buckets other than ten, buckets that do not sum to the count, or a duration in the wrong bucket." + }, + "API-87": { + "section": "L1 hit latency", + "binds": "server", + "gap": "No public vector exercises the server, so nothing here catches a server that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected." + }, + "API-88": { + "section": "L1 hit latency", + "binds": "server", + "gap": "As API-87, for a server that answers a request with an error because an optional metrics header is missing or malformed." + }, + "API-89": { + "section": "L1 hit latency", + "binds": "server", + "gap": "Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe." + }, + "API-90": { + "section": "Telemetry switch", + "binds": "sdk", + "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on." + }, + "API-91": { + "section": "Telemetry switch", + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it." } }, "retired": {} diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 9c92bbf..9c400d3 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -229,7 +229,9 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Metrics | ✅ Live `prometheus_client` counters / gauges / histograms on the default registry; no HTTP exposition helper | ✅ `cachekit-rs` 0.8.0+ — live counters ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). `CacheKit::stats()` returns `L1Stats`, plus `l1_entry_count()` and `circuit_state()`; no Prometheus exposition (LAB-521) | ✅ Node — live Prometheus metrics via the optional `prom-client` peer dep, custom registry supported, warns once and no-ops if absent (LAB-517) · ❌ Workers — `prom-client` is CI-excluded from the edge bundle¹ | ❌ | | Structured logging | ✅ JSON ring-buffer logger with sensitive-data masking (`logging.py`) | ✅ `cachekit-rs` 0.8.0+ — `tracing` events ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Opt-in `tracing` cargo feature: one `debug` event per get/set/delete, carrying `key_hash` (unkeyed Blake2b-128 of the storage key: a correlator, not a redaction), plus breaker transitions, to the host's subscriber (LAB-521) | ⚠️ Pluggable error-logger hook (`setLogger`, default `console.error`) — a logging seam, not a structured logger (LAB-517) | ❌ | | Distributed tracing (OTel) | ❌ The span-shaped API accepts spans and discards them (`NoOpSpan` in `FeatureOrchestrator.create_span`, `decorators/orchestrator.py`) — instrumentation built against it silently produces nothing | ❌ | ❌ | ❌ | -| SaaS telemetry headers (`X-CacheKit-L1-*`) | ✅ Auto. The decorators count one outcome per call ([Counting outcomes](spec/saas-api.md#counting-outcomes)) | ✅ `cachekit-rs` 0.8.0+ ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Auto-wired from live counters; an explicit `.metrics_provider()` still wins (LAB-521) · ⚠️ through 0.13.0 a `#[cachekit]` call can count up to about 51 misses ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-rs#196](https://github.com/cachekit-io/cachekit-rs/pull/196) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins · ⚠️ through npm 0.2.1 a `wrap()` call can count two outcomes or none ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-ts#265](https://github.com/cachekit-io/cachekit-ts/pull/265) | ❌ | +| SaaS telemetry headers (`X-CacheKit-L1-*`) | ✅ Auto. The decorators count one outcome per call ([Counting outcomes](spec/saas-api.md#counting-outcomes)) · ⚠️ sends `X-CacheKit-L1-Hit-Rate` as L1-Hits / (L1-Hits + L2-Hits), not the share of all counted operations [API-78](spec/saas-api.md#api-78) pins | ✅ `cachekit-rs` 0.8.0+ ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Auto-wired from live counters; an explicit `.metrics_provider()` still wins (LAB-521) · ⚠️ through 0.13.0 a `#[cachekit]` call can count up to about 51 misses ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-rs#196](https://github.com/cachekit-io/cachekit-rs/pull/196) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins · ⚠️ through npm 0.2.1 a `wrap()` call can count two outcomes or none ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-ts#265](https://github.com/cachekit-io/cachekit-ts/pull/265) | ❌ | +| SaaS L1 hit-latency headers (`X-CacheKit-L1-Latency-*`) | ❌ Specified ([L1 hit latency](spec/saas-api.md#l1-hit-latency)), not shipped. Backed-mode L1 hits are timed only for the local Prometheus histogram | ❌ Specified, not shipped. No L1 timer; never on the `workers` feature, whose clock cannot time an L1 hit ([API-83](spec/saas-api.md#api-83)) | ❌ Specified, not shipped. No L1 timer; never on the Workers entry, whose clock cannot time an L1 hit ([API-83](spec/saas-api.md#api-83)) | ❌ | +| Telemetry switch (`telemetry` setting, `CACHEKIT_TELEMETRY`) | ❌ Specified ([Telemetry switch](spec/saas-api.md#telemetry-switch)), not shipped: the session headers are always sent | ❌ Specified, not shipped: the session headers are always sent | ❌ Specified, not shipped: the session headers are always sent | ❌ | --- diff --git a/spec/saas-api.md b/spec/saas-api.md index e4a179f..83da945 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -517,7 +517,9 @@ Authorization: Bearer ck_live_xxx ### Optional Metrics Headers -SDKs SHOULD send cache metrics headers for rate limiting and observability: +The optional metrics headers are client-reported telemetry, for observability only. An SDK sends them only while telemetry is on ([Telemetry switch](#telemetry-switch)), and only from its CachekitIO backend: an SDK MUST NOTAPI-77 send them, or the measurements they carry, through any other backend or to any other destination. A Redis, Memcached or File backend, or an L1-only configuration, sends no telemetry anywhere. While telemetry is on, SDKs SHOULD send the session headers on every request, and the latency headers whenever they have a report to send ([L1 hit latency](#l1-hit-latency)). + +**Session headers.** The counters are cumulative for the session ([Counting outcomes](#counting-outcomes)). | Header | Type | Description | | :--- | :--- | :--- | @@ -526,9 +528,17 @@ SDKs SHOULD send cache metrics headers for rate limiting and observability: | `X-CacheKit-L1-Hits` | integer | Count of L1 (in-memory) cache hits | | `X-CacheKit-L2-Hits` | integer | Count of L2 (backend) cache hits | | `X-CacheKit-Misses` | integer | Count of cache misses | -| `X-CacheKit-L1-Hit-Rate` | float | L1 hit rate (0.000 to 1.000, 3 decimal places) | +| `X-CacheKit-L1-Hit-Rate` | decimal | Share of counted operations L1 served, `0.000` to `1.000` ([L1 hit rate](#l1-hit-rate)) | + +**Latency headers.** The values cover one report, not the session ([L1 hit latency](#l1-hit-latency)). -`X-CacheKit-L1-Status` — see [Required Headers](#required-headers). +| Header | Type | Description | +| :--- | :--- | :--- | +| `X-CacheKit-L1-Latency-Count` | integer | Number of L1 hits timed in this report | +| `X-CacheKit-L1-Latency-Sum-Ns` | integer | Sum of their durations, in nanoseconds | +| `X-CacheKit-L1-Latency-Buckets` | 10 integers | Their durations, counted into ten fixed buckets | + +`X-CacheKit-L1-Status` is not one of these headers: it is required on every `ck_sdk_` request, whatever the telemetry setting. See [Required Headers](#required-headers). #### Counting outcomes @@ -539,6 +549,51 @@ SDKs SHOULD send cache metrics headers for rate limiting and observability: A stale serve counts one hit by the layer that served it (a backend stale-window `GET` is an L2 hit, an L1 stale serve an L1 hit); its background revalidation counts nothing, though it runs the wrapped function ([API-63](#api-63)). +#### L1 hit rate + +An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same request, rounded to three decimal places (either way on an exact tie), and send `0.000` while all three counters are 0. The value is written as `0.` or `1.` followed by exactly three digits, and never exceeds `1.000`. It is the share of counted operations that L1 served, the figure the documentation reports. The share of hits alone, `L1-Hits / (L1-Hits + L2-Hits)`, is not this header. A reader that aggregates rows computes the rate from the three counters, not from this header. + +#### L1 hit latency + +**What is timed.** A duration is one cache call counted as an L1 hit ([API-75](#api-75), [API-76](#api-76)), as its caller sees it: from the call's entry into the SDK's public API (the wrapped call, or the read method) to the point where the SDK hands the value back (its return, or the resolution of its promise). It includes deriving the cache key, the L1 lookup, and any decryption and deserialization the SDK does on an L1 hit. It excludes whatever happens after the hand-back, such as an event loop scheduling the caller's continuation. An SDK MUSTAPI-79 time only calls it counts as an L1 hit. It MUST NOTAPI-80 time a wrapped call that joined another in-process call for the same key, whose duration is the other call's work, or an existence check. An L1 stale serve is timed like any L1 hit; the background revalidation it starts is not part of the call. + +The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. + +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, and that decision MUST NOTAPI-82 depend on the key, the arguments, the outcome or the duration. A per-client counter (every Nth call) or a random draw qualifies. A call chosen for timing that is not an L1 hit is discarded. The sample rate is the SDK's choice and is not sent: `X-CacheKit-L1-Latency-Count` says how many hits a report covers, so a reader divides by it, never by `X-CacheKit-L1-Hits`. + +**Clock.** An SDK MUST NOTAPI-83 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. Where it cannot tell whether its runtime's clock qualifies, the clock does not qualify. Cloudflare Workers does not qualify: its clock advances only on I/O, so an in-memory hit would read as 0 ns. Neither does a browser that coarsens its timers. On such a runtime the SDK sends no latency headers at all, never zeros. A 0 ns reading on a qualifying clock is a measurement, and is sent. + +**Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the SDK's previous report. When an SDK attaches a report to a request, it MUSTAPI-84 take the values and reset its accumulators in one atomic step, so that two concurrent requests never carry the same timed hit. It sends the three latency headers together, and only when the count is at least 1. A report is sent once. A retry of the request carries only what has been timed since, and a report whose request fails is lost: the SDK MUST NOTAPI-85 add its values back. Hits timed after the client's last request are never reported; there is no background reporter. + +Per-report values were chosen over the session counters' cumulative form for three reasons. An analytics store sums them across rows, sessions and clients as they stand, while cumulative values need each session's previous row to difference. They need no per-session state on the server, so they stay correct whatever a Session-ID covers: the server drops a session-headers row whose counters went backwards, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. And a lost report loses only its samples; a loss does not depend on the latencies it held, so the distribution stays unbiased. A failed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. + +**Encoding.** Every latency value is a count or a sum, so a reader adds values across rows; no mean or percentile goes on the wire, because neither can be summed. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-86 send values that meet this grammar and the rules below: + +- `X-CacheKit-L1-Latency-Count`: the number of timed hits in the report, at least 1. +- `X-CacheKit-L1-Latency-Sum-Ns`: the sum of their durations, each in whole nanoseconds, rounded down. +- `X-CacheKit-L1-Latency-Buckets`: exactly ten integers separated by `,`, with no white space. The i-th counts the timed hits whose duration *d*, in whole nanoseconds, falls in bucket i of the table. The ten integers sum to the count. + +| Bucket | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| *d* (ns) | ≤ 128 | ≤ 256 | ≤ 512 | ≤ 1,024 | ≤ 2,048 | ≤ 4,096 | ≤ 8,192 | ≤ 16,384 | ≤ 32,768 | > 32,768 | + +Each bucket holds the durations above the previous bucket's bound, up to and including its own. Example: hits of 90, 300, 300 and 40,000 ns are sent as count `4`, sum `40690` and buckets `1,0,2,0,0,0,0,0,0,1`. + +A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to 32.768 µs cover the sub-microsecond hits the SDKs target and the slower hits of an interpreted runtime or an encrypted L1; the open last bucket holds the pauses. A reader estimates a percentile by interpolating within the bucket that holds it. A change to the bounds needs a new header name. + +**Server handling.** The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-87 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail validation, including the server's per-session counter check, and the reverse. Within the latency set, a missing or malformed header, or buckets that do not sum to the count, drops all three latency values. The server MUST NOTAPI-88 answer a request with an error because an optional metrics header is missing or malformed. It MUST NOTAPI-89 use the optional metrics headers as input to billing or to rate limiting: they are client-reported, and only `X-CacheKit-L1-Status`, which [API-1](#api-1) requires, takes part in classifying a request. + +#### Telemetry switch + +Every SDK MUSTAPI-90 let its user turn telemetry off through both of these controls, which mean the same in every SDK: + +- a boolean setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend; +- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). + +With telemetry off, an SDK MUST NOTAPI-91 send any optional metrics header: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Count`, `X-CacheKit-L1-Latency-Sum-Ns` or `X-CacheKit-L1-Latency-Buckets`. It still sends `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session. With telemetry off, an SDK SHOULD NOT time L1 hits at all. The switch governs only what an SDK sends to CachekitIO; local metrics such as Prometheus counters or a `stats()` call are unaffected. + +Each control is on, off or unset. Telemetry is on when neither control is off, and either one is on or both are unset. The default, with both unset, is **on**. When the controls disagree, off wins, so an operator's environment turns off telemetry that code turned on, and code turns off telemetry the environment turned on. `CACHEKIT_TELEMETRY` is read case-insensitively, after trimming surrounding white space: `false`, `0`, `off` and `no` are off; `true`, `1`, `on` and `yes` are on; unset or empty is unset. Any other value is off, and the SDK SHOULD warn once: a user who set the variable meant to change something, and off is the safe reading. + --- ## Error Handling @@ -586,6 +641,7 @@ SDKs should classify errors for circuit breaker integration: | `CACHEKIT_MAX_RETRIES` | No | `3` | Max retry attempts | | `CACHEKIT_CONNECTION_POOL_SIZE` | No | `10` | HTTP connection pool size | | `CACHEKIT_ALLOW_CUSTOM_HOST` | No | `false` | Allow non-standard API hostnames | +| `CACHEKIT_TELEMETRY` | No | unset (on) | `false` turns off the optional metrics headers ([Telemetry switch](#telemetry-switch)) | ### SSRF Protection From 8d3aefc59f9d73c153f6148f0667ecbad515fbe2 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 20:17:27 +1100 Subject: [PATCH 2/7] fix(spec): tighten L1 latency reports, sampling, clocks and the switch Drop the redundant count header and widen the buckets to sixteen, up to about 2.1 ms. Require a per-call random sampling draw, name the qualifying clocks, keep one report per backend instance taken as a unit, keep a report off every retry, and make the telemetry setting an unset-by-default option read once at construction, with off winning over on. --- changelog.d/20261031_lab-12077.md | 29 ++++++------ conformance/coverage.md | 39 +++++++-------- conformance/requirements.json | 41 +++++++++------- spec/saas-api.md | 79 +++++++++++++++++++++---------- 4 files changed, 111 insertions(+), 77 deletions(-) diff --git a/changelog.d/20261031_lab-12077.md b/changelog.d/20261031_lab-12077.md index c24f045..55eda31 100644 --- a/changelog.d/20261031_lab-12077.md +++ b/changelog.d/20261031_lab-12077.md @@ -1,22 +1,23 @@ ### SaaS API — L1 hit-latency telemetry and the telemetry switch -- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines three +- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines two optional headers that report how long calls served from L1 take, as the caller sees them: - `X-CacheKit-L1-Latency-Count`, `X-CacheKit-L1-Latency-Sum-Ns` and - `X-CacheKit-L1-Latency-Buckets` (ten fixed buckets doubling from 128 ns to 32.768 µs, plus - an open last bucket). Values are counted per report, not per session; a report is sent at - most once. An SDK may time a sample of calls, and sends no latency headers on a runtime - whose clock cannot resolve an L1 hit, such as Cloudflare Workers - ([API-79](spec/saas-api.md#api-79)–[API-86](spec/saas-api.md#api-86)). + `X-CacheKit-L1-Latency-Buckets` (sixteen fixed buckets doubling from 128 ns to about + 2.1 ms, the last one open; their sum is the number of hits timed) and + `X-CacheKit-L1-Latency-Sum-Ns`. Values are counted per report, not per session, kept per + CachekitIO backend instance, taken atomically and sent on exactly one attempt. An SDK may + time a random sample of calls, and sends no latency headers from a runtime whose clock + cannot resolve an L1 hit, such as Cloudflare Workers + ([API-79](spec/saas-api.md#api-79)–[API-87](spec/saas-api.md#api-87)). - [`spec/saas-api.md` § Telemetry switch](spec/saas-api.md#telemetry-switch) (new): every SDK - exposes a `telemetry` setting and reads `CACHEKIT_TELEMETRY`; either one turned off stops - every optional metrics header. `X-CacheKit-L1-Status` is still sent. Default: on - ([API-90](spec/saas-api.md#api-90), [API-91](spec/saas-api.md#api-91)). + exposes an optional `telemetry` setting and reads `CACHEKIT_TELEMETRY`; either one turned off + stops every optional metrics header. `X-CacheKit-L1-Status` is still sent. Default: on + ([API-91](spec/saas-api.md#api-91), [API-92](spec/saas-api.md#api-92)). - [`spec/saas-api.md` § Optional Metrics Headers](spec/saas-api.md#optional-metrics-headers): - the headers are client-reported observability, sent only by the CachekitIO backend - ([API-77](spec/saas-api.md#api-77)); the server never uses them for billing or rate limiting - and never rejects a request over one, and a bad latency set never drops the session headers, - or the reverse ([API-87](spec/saas-api.md#api-87)–[API-89](spec/saas-api.md#api-89)). + the headers are client-reported observability, sent only to the configured CachekitIO API + ([API-77](spec/saas-api.md#api-77)). The server never reads them for billing or rate + limiting and never rejects a request over one, and a bad latency set never drops the session + headers, or the reverse ([API-88](spec/saas-api.md#api-88)–[API-90](spec/saas-api.md#api-90)). - [`spec/saas-api.md` § L1 hit rate](spec/saas-api.md#l1-hit-rate) (new) pins `X-CacheKit-L1-Hit-Rate` to `L1-Hits / (L1-Hits + L2-Hits + Misses)` ([API-78](spec/saas-api.md#api-78)). **Changes cachekit-py**, which sends diff --git a/conformance/coverage.md b/conformance/coverage.md index e061b94..d2c4b0e 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -25,14 +25,14 @@ How ids are assigned, and what each status means: [README.md](README.md#coverage | [`spec/file-backend-format.md`](../spec/file-backend-format.md) | 6 | 1 | 3 | 0 | 2 | | [`spec/intent-presets.md`](../spec/intent-presets.md) | 55 | 0 | 19 | 0 | 36 | | [`spec/interop-mode.md`](../spec/interop-mode.md) | 36 | 5 | 23 | 4 | 4 | -| [`spec/saas-api.md`](../spec/saas-api.md) | 91 | 0 | 15 | 0 | 76 | +| [`spec/saas-api.md`](../spec/saas-api.md) | 92 | 0 | 15 | 0 | 77 | | [`spec/wire-format.md`](../spec/wire-format.md) | 24 | 3 | 18 | 3 | 0 | | SDK | Covered | Partial | Uncovered | Gap | n/a | | :--- | ---: | ---: | ---: | ---: | ---: | -| cachekit-py | 12 | 86 | 5 | 81 | 50 | -| cachekit-ts | 9 | 78 | 7 | 71 | 69 | -| cachekit-rs | 4 | 69 | 3 | 67 | 91 | +| cachekit-py | 12 | 86 | 5 | 82 | 50 | +| cachekit-ts | 9 | 78 | 7 | 72 | 69 | +| cachekit-rs | 4 | 69 | 3 | 68 | 91 | Not indexed yet: [`spec/interop-v2.md`](../spec/interop-v2.md). @@ -274,21 +274,22 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-74](../spec/saas-api.md#api-74) | *SSRF Protection*: All SDKs **MUST** enforce SSRF protection when accepting the API URL. The following rules apply: | **Gap:** No vector holds an API URL, so nothing here checks HTTPS-only, the listed private ranges or the hostname allowlist. | gap | gap | gap | | [API-75](../spec/saas-api.md#api-75) | *Counting outcomes*: Plain read. A value read **MUST** count one outcome: an L1 hit when the in-memory layer served it, an L2 hit when a backend read did, or a miss when … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a plain read twice, or counts a read that failed at the backend; only the SDK's own counter tests do. | gap | gap | gap | | [API-76](../spec/saas-api.md#api-76) | *Counting outcomes*: … caching wrapper (the cachekit-py decorators, cachekit-ts `wrap()`, cachekit-rs `#[cachekit]`) **MUST** count one outcome, however many reads it makes. A call that runs the wrapped function counts one miss, before its … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do. | gap | gap | gap | -| [API-77](../spec/saas-api.md#api-77) | *Optional Metrics Headers*: … them only while telemetry is on (Telemetry switch), and only from its CachekitIO backend: an SDK **MUST NOT** send them, or the measurements they carry, through any other backend or to any other destination. A Redis, Memcached … | **Gap:** No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or their values, through a Redis, Memcached or File backend or anywhere other than the CachekitIO API. | gap | gap | gap | -| [API-78](../spec/saas-api.md#api-78) | *L1 hit rate*: An SDK **MUST** compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same … | **Gap:** No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), or sends it with other than three decimal places or a value other than 0.000 for all-zero counters. | gap | gap | gap | -| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … happens after the hand-back, such as an event loop scheduling the caller's continuation. An SDK **MUST** time only calls it counts as an L1 hit. It MUST NOT time a wrapped call that joined another in-process call for the … | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits or misses into the L1 latency report; only the SDK's own tests do. | gap | gap | gap | -| [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … loop scheduling the caller's continuation. An SDK MUST time only calls it counts as an L1 hit. It **MUST NOT** time a wrapped call that joined another in-process call for the same key, whose duration is the other call's work, or … | **Gap:** As API-79, for a client that times a wrapped call that joined another in-process call, or an existence check. | gap | gap | gap | -| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It **MUST** decide whether to time a call before it knows the call's outcome, and that decision MUST NOT depend on the key, the … | **Gap:** As API-79, for a client that decides to time a call only after it knows the call was an L1 hit. | gap | gap | gap | -| [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: … hit. It MUST decide whether to time a call before it knows the call's outcome, and that decision **MUST NOT** depend on the key, the arguments, the outcome or the duration. A per-client counter (every Nth call) or a random draw … | **Gap:** As API-79, for a client whose sampling decision depends on the key, the arguments, the outcome or the duration. | gap | gap | gap | -| [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers from a runtime whose clock does not advance during synchronous execution or resolves coarser than 100 ns, such as Cloudflare Workers. | gap | gap | gap | -| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … the hits timed since the SDK's previous report. When an SDK attaches a report to a request, it **MUST** take the values and reset its accumulators in one atomic step, so that two concurrent requests never carry the same … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent requests carry the same timed hit, or that sends cumulative rather than per-report values. | gap | gap | gap | -| [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: … request carries only what has been timed since, and a report whose request fails is lost: the SDK **MUST NOT** add its values back. Hits timed after the client's last request are never reported; there is no background reporter. | **Gap:** No vector covers request headers, so nothing here catches a client that re-sends a report on a retry, or adds a failed request's values back to its accumulators. | gap | gap | gap | -| [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and the rules below: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, a count of 0, buckets other than ten, buckets that do not sum to the count, or a duration in the wrong bucket. | gap | gap | gap | -| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches a server that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | -| [API-88](../spec/saas-api.md#api-88) | *L1 hit latency*: … header, or buckets that do not sum to the count, drops all three latency values. The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed. It MUST NOT use the … | Binds the server.
**Gap:** As API-87, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | -| [API-89](../spec/saas-api.md#api-89) | *L1 hit latency*: … NOT answer a request with an error because an optional metrics header is missing or malformed. It **MUST NOT** use the optional metrics headers as input to billing or to rate limiting: they are client-reported, and only … | Binds the server.
**Gap:** Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | -| [API-90](../spec/saas-api.md#api-90) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on. | gap | gap | gap | -| [API-91](../spec/saas-api.md#api-91) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it. | gap | gap | gap | +| [API-77](../spec/saas-api.md#api-77) | *Optional Metrics Headers*: … them only while telemetry is on (Telemetry switch), and only from its CachekitIO backend: an SDK **MUST NOT** send them, or a report built for them, to any host but the CachekitIO API that backend is configured with. A Redis, … | **Gap:** No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or a report built for them, through a Redis, Memcached or File backend or to any host but its configured CachekitIO API. | gap | gap | gap | +| [API-78](../spec/saas-api.md#api-78) | *L1 hit rate*: An SDK **MUST** compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same … | **Gap:** No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), sends it with other than three decimal places, or sends other than 0.000 for all-zero counters. | gap | gap | gap | +| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … revalidation an L1 stale serve triggers; the revalidation's own work is outside it. An SDK **MUST** time only calls it counts as an L1 hit and that return a value. It MUST NOT time any of these: | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits, misses or failed calls into the latency report, or starts or stops the clock outside the call; only the SDK's own tests do. | gap | gap | gap | +| [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … work is outside it. An SDK MUST time only calls it counts as an L1 hit and that return a value. It **MUST NOT** time any of these: | **Gap:** As API-79, for a client that times a joined in-process call, an existence check or a read made inside a wrapped call. | gap | gap | gap | +| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It **MUST** decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call … | **Gap:** As API-79, for a client that decides to time a call only after it knows the outcome, or samples by a counter rather than a per-call random draw. | gap | gap | gap | +| [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: … draw for each call with a fixed probability (a probability of 1 times every call). That decision **MUST NOT** depend on the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times … | **Gap:** As API-79, for a client whose sampling decision depends on the key, the arguments, the call's position in a sequence, the outcome or the duration. | gap | gap | gap | +| [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers. | gap | gap | gap | +| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … is on treats its clock as not qualifying, and sends no latency headers at all, never zeros. An SDK **MUST NOT** send a report whose duration sum is 0: on a qualifying clock that is a frozen clock, not a measurement. | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency report whose duration sum is 0. | gap | gap | gap | +| [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: When an SDK attaches a report to a request, it **MUST** take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values. | gap | gap | gap | +| [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … once that attempt is sent, whatever its outcome. A report whose attempt fails is lost: the SDK **MUST NOT** add its values back, or let a retry carry them. That includes a retry an HTTP library makes on its own: an SDK whose … | **Gap:** No vector covers request headers, so nothing here catches a client that adds a failed attempt's report back to its accumulator, or whose own or its HTTP library's retries resend a report. | gap | gap | gap | +| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and these rules: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, or a duration in the wrong bucket. | gap | gap | gap | +| [API-88](../spec/saas-api.md#api-88) | *L1 hit latency*: … The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | +| [API-89](../spec/saas-api.md#api-89) | *L1 hit latency*: … latency set, a missing or malformed header, or a sum of 0, drops both latency values. The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | +| [API-90](../spec/saas-api.md#api-90) | *L1 hit latency*: … SHOULD record a valid latency set on any authenticated request, whatever its response status. It **MUST NOT** read the optional metrics headers for billing or for rate limiting: they are client-reported. Only … | Binds the server.
**Gap:** Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-91](../spec/saas-api.md#api-91) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on. | gap | gap | gap | +| [API-92](../spec/saas-api.md#api-92) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header on that backend's requests or another's, or drops X-CacheKit-L1-Status with it. | gap | gap | gap | Marked not-a-requirement in `spec/saas-api.md`: diff --git a/conformance/requirements.json b/conformance/requirements.json index 626eae8..25436bf 100644 --- a/conformance/requirements.json +++ b/conformance/requirements.json @@ -1521,7 +1521,7 @@ }, "spec/saas-api.md": { "prefix": "API", - "next": 92, + "next": 93, "requirements": { "API-1": { "section": "Authentication", @@ -2114,77 +2114,82 @@ "API-77": { "section": "Optional Metrics Headers", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or their values, through a Redis, Memcached or File backend or anywhere other than the CachekitIO API." + "gap": "No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or a report built for them, through a Redis, Memcached or File backend or to any host but its configured CachekitIO API." }, "API-78": { "section": "L1 hit rate", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), or sends it with other than three decimal places or a value other than 0.000 for all-zero counters." + "gap": "No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), sends it with other than three decimal places, or sends other than 0.000 for all-zero counters." }, "API-79": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits or misses into the L1 latency report; only the SDK's own tests do." + "gap": "No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits, misses or failed calls into the latency report, or starts or stops the clock outside the call; only the SDK's own tests do." }, "API-80": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client that times a wrapped call that joined another in-process call, or an existence check." + "gap": "As API-79, for a client that times a joined in-process call, an existence check or a read made inside a wrapped call." }, "API-81": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client that decides to time a call only after it knows the call was an L1 hit." + "gap": "As API-79, for a client that decides to time a call only after it knows the outcome, or samples by a counter rather than a per-call random draw." }, "API-82": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client whose sampling decision depends on the key, the arguments, the outcome or the duration." + "gap": "As API-79, for a client whose sampling decision depends on the key, the arguments, the call's position in a sequence, the outcome or the duration." }, "API-83": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends latency headers from a runtime whose clock does not advance during synchronous execution or resolves coarser than 100 ns, such as Cloudflare Workers." + "gap": "No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers." }, "API-84": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers concurrent requests, so nothing here catches a client whose concurrent requests carry the same timed hit, or that sends cumulative rather than per-report values." + "gap": "No vector covers request headers, so nothing here catches a client that sends a latency report whose duration sum is 0." }, "API-85": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that re-sends a report on a retry, or adds a failed request's values back to its accumulators." + "gap": "No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values." }, "API-86": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, a count of 0, buckets other than ten, buckets that do not sum to the count, or a duration in the wrong bucket." + "gap": "No vector covers request headers, so nothing here catches a client that adds a failed attempt's report back to its accumulator, or whose own or its HTTP library's retries resend a report." }, "API-87": { "section": "L1 hit latency", - "binds": "server", - "gap": "No public vector exercises the server, so nothing here catches a server that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected." + "binds": "sdk", + "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, or a duration in the wrong bucket." }, "API-88": { "section": "L1 hit latency", "binds": "server", - "gap": "As API-87, for a server that answers a request with an error because an optional metrics header is missing or malformed." + "gap": "No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected." }, "API-89": { "section": "L1 hit latency", "binds": "server", - "gap": "Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe." + "gap": "As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed." }, "API-90": { + "section": "L1 hit latency", + "binds": "server", + "gap": "Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe." + }, + "API-91": { "section": "Telemetry switch", "binds": "sdk", - "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on." + "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on." }, - "API-91": { + "API-92": { "section": "Telemetry switch", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it." + "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header on that backend's requests or another's, or drops X-CacheKit-L1-Status with it." } }, "retired": {} diff --git a/spec/saas-api.md b/spec/saas-api.md index 83da945..922e67b 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -517,7 +517,7 @@ Authorization: Bearer ck_live_xxx ### Optional Metrics Headers -The optional metrics headers are client-reported telemetry, for observability only. An SDK sends them only while telemetry is on ([Telemetry switch](#telemetry-switch)), and only from its CachekitIO backend: an SDK MUST NOTAPI-77 send them, or the measurements they carry, through any other backend or to any other destination. A Redis, Memcached or File backend, or an L1-only configuration, sends no telemetry anywhere. While telemetry is on, SDKs SHOULD send the session headers on every request, and the latency headers whenever they have a report to send ([L1 hit latency](#l1-hit-latency)). +The optional metrics headers are client-reported telemetry, for observability only. An SDK sends them only while telemetry is on ([Telemetry switch](#telemetry-switch)), and only from its CachekitIO backend: an SDK MUST NOTAPI-77 send them, or a report built for them, to any host but the CachekitIO API that backend is configured with. A Redis, Memcached or File backend, or an L1-only configuration, sends no telemetry anywhere. Metrics an SDK exposes inside the user's own process, such as Prometheus counters or a `stats()` call, are outside these rules. While telemetry is on, SDKs SHOULD send the session headers on every request, and the latency headers whenever they have a report to send ([L1 hit latency](#l1-hit-latency)). **Session headers.** The counters are cumulative for the session ([Counting outcomes](#counting-outcomes)). @@ -534,9 +534,8 @@ The optional metrics headers are client-reported telemetry, for observability on | Header | Type | Description | | :--- | :--- | :--- | -| `X-CacheKit-L1-Latency-Count` | integer | Number of L1 hits timed in this report | -| `X-CacheKit-L1-Latency-Sum-Ns` | integer | Sum of their durations, in nanoseconds | -| `X-CacheKit-L1-Latency-Buckets` | 10 integers | Their durations, counted into ten fixed buckets | +| `X-CacheKit-L1-Latency-Buckets` | 16 integers | The report's timed L1 hits, counted into sixteen fixed duration buckets. Their sum is the number of hits timed | +| `X-CacheKit-L1-Latency-Sum-Ns` | integer | The sum of those hits' durations, in nanoseconds | `X-CacheKit-L1-Status` is not one of these headers: it is required on every `ck_sdk_` request, whatever the telemetry setting. See [Required Headers](#required-headers). @@ -551,48 +550,76 @@ A stale serve counts one hit by the layer that served it (a backend stale-window #### L1 hit rate -An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same request, rounded to three decimal places (either way on an exact tie), and send `0.000` while all three counters are 0. The value is written as `0.` or `1.` followed by exactly three digits, and never exceeds `1.000`. It is the share of counted operations that L1 served, the figure the documentation reports. The share of hits alone, `L1-Hits / (L1-Hits + L2-Hits)`, is not this header. A reader that aggregates rows computes the rate from the three counters, not from this header. +An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same request, rounded to three decimal places (either way on an exact tie), and send `0.000` while all three counters are 0. The value is written as `0.` or `1.` followed by exactly three digits, and never exceeds `1.000`. It is the share of counted operations that L1 served, the figure the documentation reports. The share of hits alone, `L1-Hits / (L1-Hits + L2-Hits)`, is not this header. A server or reader that aggregates requests SHOULD compute the rate from the three counters rather than read this header. #### L1 hit latency -**What is timed.** A duration is one cache call counted as an L1 hit ([API-75](#api-75), [API-76](#api-76)), as its caller sees it: from the call's entry into the SDK's public API (the wrapped call, or the read method) to the point where the SDK hands the value back (its return, or the resolution of its promise). It includes deriving the cache key, the L1 lookup, and any decryption and deserialization the SDK does on an L1 hit. It excludes whatever happens after the hand-back, such as an event loop scheduling the caller's continuation. An SDK MUSTAPI-79 time only calls it counts as an L1 hit. It MUST NOTAPI-80 time a wrapped call that joined another in-process call for the same key, whose duration is the other call's work, or an existence check. An L1 stale serve is timed like any L1 hit; the background revalidation it starts is not part of the call. +**What is timed.** A duration is one cache call counted as an L1 hit ([API-75](#api-75), [API-76](#api-76)), as its caller sees it. The clock starts at the first SDK instruction the call runs: for a coroutine or an async function, its first step, not its creation. It stops in the same synchronous step that produces the value the SDK hands back, never in a continuation or callback that runs after it. The interval includes deriving the cache key, the L1 lookup, any decryption and deserialization the SDK does on an L1 hit, and starting any background revalidation an L1 stale serve triggers; the revalidation's own work is outside it. An SDK MUSTAPI-79 time only calls it counts as an L1 hit and that return a value. It MUST NOTAPI-80 time any of these: + +- a wrapped call that joined another in-process call for the same key, because its duration is the other call's work; +- an existence check; +- a read made inside a wrapped call, such as the `get` a wrapper makes on the SDK's own public API, which [API-76](#api-76) does not count either. The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. -**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, and that decision MUST NOTAPI-82 depend on the key, the arguments, the outcome or the duration. A per-client counter (every Nth call) or a random draw qualifies. A call chosen for timing that is not an L1 hit is discarded. The sample rate is the SDK's choice and is not sent: `X-CacheKit-L1-Latency-Count` says how many hits a report covers, so a reader divides by it, never by `X-CacheKit-L1-Hits`. +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with a fixed probability (a probability of 1 times every call). That decision MUST NOTAPI-82 depend on the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A thread-local generator keeps the draw off any shared cache line. A call chosen for timing that is not an L1 hit is discarded. The probability is the SDK's choice and is not sent: the bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. + +**Clock.** An SDK MUST NOTAPI-83 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: + +| SDK | Clock | +| :--- | :--- | +| Python | `time.perf_counter_ns()`, where `time.get_clock_info("perf_counter").resolution` is 1e-7 or less | +| Rust | `std::time::Instant`, on any target but `wasm32` | +| TypeScript on Node.js | `process.hrtime.bigint()` | +| Go | the monotonic reading of `time.Now()`, through `time.Since` | + +Any other clock qualifies only where the SDK's documentation names it and its resolution. Cloudflare Workers has no qualifying clock: its clock advances only on I/O, so an in-memory hit would read as 0 ns. Neither has a browser that coarsens its timers. An SDK that cannot tell which runtime it is on treats its clock as not qualifying, and sends no latency headers at all, never zeros. An SDK MUST NOTAPI-84 send a report whose duration sum is 0: on a qualifying clock that is a frozen clock, not a measurement. -**Clock.** An SDK MUST NOTAPI-83 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. Where it cannot tell whether its runtime's clock qualifies, the clock does not qualify. Cloudflare Workers does not qualify: its clock advances only on I/O, so an in-memory hit would read as 0 ns. Neither does a browser that coarsens its timers. On such a runtime the SDK sends no latency headers at all, never zeros. A 0 ns reading on a qualifying clock is a measurement, and is sent. +**Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the previous report. An SDK keeps one report accumulator per CachekitIO backend instance. It times a call into the accumulator of the backend that call's cache is configured with, and attaches the report only to that backend's requests, so one backend's hits never ride another's requests. A process that forks starts the child with an empty accumulator. -**Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the SDK's previous report. When an SDK attaches a report to a request, it MUSTAPI-84 take the values and reset its accumulators in one atomic step, so that two concurrent requests never carry the same timed hit. It sends the three latency headers together, and only when the count is at least 1. A report is sent once. A retry of the request carries only what has been timed since, and a report whose request fails is lost: the SDK MUST NOTAPI-85 add its values back. Hits timed after the client's last request are never reported; there is no background reporter. +When an SDK attaches a report to a request, it MUSTAPI-85 take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and the buckets and the sum always describe the same hits. Separate atomic swaps of each field do not satisfy this; one structure taken under a lock, or swapped as a unit, does. It sends the two latency headers together, and only when at least one hit was timed. A report rides exactly one attempt and is consumed once that attempt is sent, whatever its outcome. A report whose attempt fails is lost: the SDK MUST NOTAPI-86 add its values back, or let a retry carry them. That includes a retry an HTTP library makes on its own: an SDK whose HTTP layer can resend a request without re-running the code that attaches the report must not use that layer's retries for a request that carries one. Hits timed after the client's last request are never reported; there is no background reporter. If adding a duration would push the sum or a bucket count past the bound in Encoding, the SDK discards that duration. -Per-report values were chosen over the session counters' cumulative form for three reasons. An analytics store sums them across rows, sessions and clients as they stand, while cumulative values need each session's previous row to difference. They need no per-session state on the server, so they stay correct whatever a Session-ID covers: the server drops a session-headers row whose counters went backwards, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. And a lost report loses only its samples; a loss does not depend on the latencies it held, so the distribution stays unbiased. A failed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. +Per-report values were chosen over the session counters' cumulative form for two reasons. A reader adds them across requests, sessions and clients as they stand, while cumulative values need each session's previous value to difference. And they need no per-session state: a server may discard a session-headers sample whose counters are lower than an earlier sample for the same Session-ID, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. A failed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. Reports are lost when their request fails or their process exits first, and those losses are not independent of load, so the latency data is a sample of the hits, not a census. -**Encoding.** Every latency value is a count or a sum, so a reader adds values across rows; no mean or percentile goes on the wire, because neither can be summed. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-86 send values that meet this grammar and the rules below: +**Encoding.** Every latency value is a count or a sum, so a reader adds values across requests; no mean or percentile goes on the wire, because neither can be added. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-87 send values that meet this grammar and these rules: -- `X-CacheKit-L1-Latency-Count`: the number of timed hits in the report, at least 1. -- `X-CacheKit-L1-Latency-Sum-Ns`: the sum of their durations, each in whole nanoseconds, rounded down. -- `X-CacheKit-L1-Latency-Buckets`: exactly ten integers separated by `,`, with no white space. The i-th counts the timed hits whose duration *d*, in whole nanoseconds, falls in bucket i of the table. The ten integers sum to the count. +- `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. +- `X-CacheKit-L1-Latency-Sum-Ns`: the sum of the same hits' durations, each in whole nanoseconds rounded down; at least 1. -| Bucket | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | -| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | -| *d* (ns) | ≤ 128 | ≤ 256 | ≤ 512 | ≤ 1,024 | ≤ 2,048 | ≤ 4,096 | ≤ 8,192 | ≤ 16,384 | ≤ 32,768 | > 32,768 | +| Bucket | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| *d* (ns) at most | 128 | 256 | 512 | 1,024 | 2,048 | 4,096 | 8,192 | 16,384 | -Each bucket holds the durations above the previous bucket's bound, up to and including its own. Example: hits of 90, 300, 300 and 40,000 ns are sent as count `4`, sum `40690` and buckets `1,0,2,0,0,0,0,0,0,1`. +| Bucket | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| *d* (ns) at most | 32,768 | 65,536 | 131,072 | 262,144 | 524,288 | 1,048,576 | 2,097,152 | no bound | -A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to 32.768 µs cover the sub-microsecond hits the SDKs target and the slower hits of an interpreted runtime or an encrypted L1; the open last bucket holds the pauses. A reader estimates a percentile by interpolating within the bucket that holds it. A change to the bounds needs a new header name. +Each bucket holds the durations above the previous bucket's bound, up to and including its own; bucket 1 holds everything up to 128 ns, 0 included. Example: hits of 90, 300, 300 and 40,000 ns are sent as buckets `1,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0` and sum `40690`. -**Server handling.** The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-87 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail validation, including the server's per-session counter check, and the reverse. Within the latency set, a missing or malformed header, or buckets that do not sum to the count, drops all three latency values. The server MUST NOTAPI-88 answer a request with an error because an optional metrics header is missing or malformed. It MUST NOTAPI-89 use the optional metrics headers as input to billing or to rate limiting: they are client-reported, and only `X-CacheKit-L1-Status`, which [API-1](#api-1) requires, takes part in classifying a request. +A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to about 2.1 ms cover the sub-microsecond hits the SDKs target, the tens of microseconds an encrypted L1 hit on a large value can take, and the pauses above them; a reader estimates a percentile by interpolating within the bucket that holds it. A percentile that falls in bucket 16 has no upper bound, so a reader reports it only as above 2,097,152 ns. A change to the bounds needs a new header name. + +**Server handling.** The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-88 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail any of its checks, and the reverse. Within the latency set, a missing or malformed header, or a sum of 0, drops both latency values. The server MUST NOTAPI-89 answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid latency set on any authenticated request, whatever its response status. It MUST NOTAPI-90 read the optional metrics headers for billing or for rate limiting: they are client-reported. Only `X-CacheKit-L1-Status`, which [API-1](#api-1) requires, may be read by rate limiting. #### Telemetry switch -Every SDK MUSTAPI-90 let its user turn telemetry off through both of these controls, which mean the same in every SDK: +Every SDK MUSTAPI-91 let its user turn telemetry off through both of these controls, which mean the same in every SDK: + +- a setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend. It is an optional boolean that is unset unless the user sets it (Python `None`, TypeScript `undefined`, Rust `None`, a nil Go `*bool`): an SDK does not give it a default of true or false; +- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). Where the runtime has no process environment, as on Cloudflare Workers, the SDK reads it from the environment bindings it is constructed with; an SDK that has neither documents that only the setting applies there. + +The SDK reads both controls once, when it constructs the backend; they hold for that backend instance's lifetime. It parses `CACHEKIT_TELEMETRY` only by the rule below, never through a configuration loader's own boolean parsing or precedence, so that off wins even where that loader would let a code argument override the environment. + +With telemetry off, an SDK MUST NOTAPI-92 send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Buckets` or `X-CacheKit-L1-Latency-Sum-Ns`. Its counts and timings reach no other backend's requests either. It still sends `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session. With telemetry off, an SDK SHOULD NOT accumulate a latency report. -- a boolean setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend; -- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). +`CACHEKIT_TELEMETRY` is read case-insensitively, after trimming surrounding white space: `false`, `0`, `off` and `no` are off; `true`, `1`, `on` and `yes` are on; unset or empty is unset. Any other value is off, and the SDK SHOULD warn once: a user who set the variable meant to change something, and off is the safe reading. Each control is then on, off or unset, and they combine like this: -With telemetry off, an SDK MUST NOTAPI-91 send any optional metrics header: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Count`, `X-CacheKit-L1-Latency-Sum-Ns` or `X-CacheKit-L1-Latency-Buckets`. It still sends `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session. With telemetry off, an SDK SHOULD NOT time L1 hits at all. The switch governs only what an SDK sends to CachekitIO; local metrics such as Prometheus counters or a `stats()` call are unaffected. +| Setting \ Environment | unset | on | off | +| :--- | :--- | :--- | :--- | +| **unset** | the default: on | on | off | +| **on** | on | on | off | +| **off** | off | off | off | -Each control is on, off or unset. Telemetry is on when neither control is off, and either one is on or both are unset. The default, with both unset, is **on**. When the controls disagree, off wins, so an operator's environment turns off telemetry that code turned on, and code turns off telemetry the environment turned on. `CACHEKIT_TELEMETRY` is read case-insensitively, after trimming surrounding white space: `false`, `0`, `off` and `no` are off; `true`, `1`, `on` and `yes` are on; unset or empty is unset. Any other value is off, and the SDK SHOULD warn once: a user who set the variable meant to change something, and off is the safe reading. +Off wins: either control turns telemetry off, whatever the other says, so an operator's environment can turn off telemetry that code turned on, and the reverse. --- @@ -641,7 +668,7 @@ SDKs should classify errors for circuit breaker integration: | `CACHEKIT_MAX_RETRIES` | No | `3` | Max retry attempts | | `CACHEKIT_CONNECTION_POOL_SIZE` | No | `10` | HTTP connection pool size | | `CACHEKIT_ALLOW_CUSTOM_HOST` | No | `false` | Allow non-standard API hostnames | -| `CACHEKIT_TELEMETRY` | No | unset (on) | `false` turns off the optional metrics headers ([Telemetry switch](#telemetry-switch)) | +| `CACHEKIT_TELEMETRY` | No | unset (on) | `false`, `0`, `off`, `no` or any unrecognised value turns off the optional metrics headers; `true`, `1`, `on`, `yes` leave them on, unless the `telemetry` setting is off ([Telemetry switch](#telemetry-switch)) | ### SSRF Protection From 80c9a065b99136b7b0240b200a7b0350c5ab8114 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 20:54:57 +1100 Subject: [PATCH 3/7] fix(spec): weight L1 latency reports, harden the switch and server rules Send the sampling denominator with every report, so readers can weight pooled data; time all work before the cached value returns; keep a zero-sum report in the accumulator; never attach a report to a health request; scope the no-resend rule to attempts that may have been delivered; reset the accumulator in a forked child. Give the switch's parse and precedence rules their own requirement. Move server rules to their own section: independent validation, random-only dropping with recorded probability, per-tenant weighting of cross-tenant aggregates, and no billing or rate-limit input. Drop the Go clock row and fix the Python matrix cell. --- changelog.d/20261031_lab-12077.md | 34 +++++++++++++--------- conformance/coverage.md | 39 +++++++++++++------------ conformance/requirements.json | 47 ++++++++++++++++++++----------- sdk-feature-matrix.md | 6 ++-- spec/saas-api.md | 43 +++++++++++++++------------- 5 files changed, 99 insertions(+), 70 deletions(-) diff --git a/changelog.d/20261031_lab-12077.md b/changelog.d/20261031_lab-12077.md index 55eda31..d226681 100644 --- a/changelog.d/20261031_lab-12077.md +++ b/changelog.d/20261031_lab-12077.md @@ -1,27 +1,33 @@ ### SaaS API — L1 hit-latency telemetry and the telemetry switch -- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines two +- [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines three optional headers that report how long calls served from L1 take, as the caller sees them: `X-CacheKit-L1-Latency-Buckets` (sixteen fixed buckets doubling from 128 ns to about - 2.1 ms, the last one open; their sum is the number of hits timed) and - `X-CacheKit-L1-Latency-Sum-Ns`. Values are counted per report, not per session, kept per - CachekitIO backend instance, taken atomically and sent on exactly one attempt. An SDK may - time a random sample of calls, and sends no latency headers from a runtime whose clock - cannot resolve an L1 hit, such as Cloudflare Workers - ([API-79](spec/saas-api.md#api-79)–[API-87](spec/saas-api.md#api-87)). + 2.1 ms, the last one open; their sum is the number of hits timed), + `X-CacheKit-L1-Latency-Sum-Ns`, and `X-CacheKit-L1-Latency-One-In` (the sampling + denominator N, so readers weight each report). Values are counted per report, not per + session, kept per CachekitIO backend instance, taken atomically and never re-sent once an + attempt may have been delivered. An SDK may time a random sample of calls, and sends no + latency headers from a runtime whose clock cannot resolve an L1 hit, such as Cloudflare + Workers ([API-79](spec/saas-api.md#api-79)–[API-87](spec/saas-api.md#api-87)). +- [`spec/saas-api.md` § Server handling](spec/saas-api.md#server-handling) (new): the server + validates the latency and session headers as independent sets, never rejects a request over + one, drops latency sets only at random and records each kept set's probability, weights + tenants equally (or caps them) in cross-tenant aggregates, and never lets an optional metrics + header change billing or a rate-limit verdict + ([API-88](spec/saas-api.md#api-88)–[API-92](spec/saas-api.md#api-92)). - [`spec/saas-api.md` § Telemetry switch](spec/saas-api.md#telemetry-switch) (new): every SDK - exposes an optional `telemetry` setting and reads `CACHEKIT_TELEMETRY`; either one turned off - stops every optional metrics header. `X-CacheKit-L1-Status` is still sent. Default: on - ([API-91](spec/saas-api.md#api-91), [API-92](spec/saas-api.md#api-92)). + exposes an optional `telemetry` setting and reads `CACHEKIT_TELEMETRY`, once, at backend + construction; either one turned off stops every optional metrics header. `X-CacheKit-L1-Status` + and `User-Agent` are still sent. Default: on + ([API-93](spec/saas-api.md#api-93)–[API-95](spec/saas-api.md#api-95)). - [`spec/saas-api.md` § Optional Metrics Headers](spec/saas-api.md#optional-metrics-headers): the headers are client-reported observability, sent only to the configured CachekitIO API - ([API-77](spec/saas-api.md#api-77)). The server never reads them for billing or rate - limiting and never rejects a request over one, and a bad latency set never drops the session - headers, or the reverse ([API-88](spec/saas-api.md#api-88)–[API-90](spec/saas-api.md#api-90)). + ([API-77](spec/saas-api.md#api-77)). - [`spec/saas-api.md` § L1 hit rate](spec/saas-api.md#l1-hit-rate) (new) pins `X-CacheKit-L1-Hit-Rate` to `L1-Hits / (L1-Hits + L2-Hits + Misses)` ([API-78](spec/saas-api.md#api-78)). **Changes cachekit-py**, which sends `L1-Hits / (L1-Hits + L2-Hits)`. - [`sdk-feature-matrix.md` § Observability](sdk-feature-matrix.md#observability): rows for the latency headers and the telemetry switch, unshipped in every SDK, and the Python hit-rate - difference on the telemetry-headers row. + difference on the session-headers row. diff --git a/conformance/coverage.md b/conformance/coverage.md index d2c4b0e..311d6d0 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -25,14 +25,14 @@ How ids are assigned, and what each status means: [README.md](README.md#coverage | [`spec/file-backend-format.md`](../spec/file-backend-format.md) | 6 | 1 | 3 | 0 | 2 | | [`spec/intent-presets.md`](../spec/intent-presets.md) | 55 | 0 | 19 | 0 | 36 | | [`spec/interop-mode.md`](../spec/interop-mode.md) | 36 | 5 | 23 | 4 | 4 | -| [`spec/saas-api.md`](../spec/saas-api.md) | 92 | 0 | 15 | 0 | 77 | +| [`spec/saas-api.md`](../spec/saas-api.md) | 95 | 0 | 15 | 0 | 80 | | [`spec/wire-format.md`](../spec/wire-format.md) | 24 | 3 | 18 | 3 | 0 | | SDK | Covered | Partial | Uncovered | Gap | n/a | | :--- | ---: | ---: | ---: | ---: | ---: | -| cachekit-py | 12 | 86 | 5 | 82 | 50 | -| cachekit-ts | 9 | 78 | 7 | 72 | 69 | -| cachekit-rs | 4 | 69 | 3 | 68 | 91 | +| cachekit-py | 12 | 86 | 5 | 83 | 52 | +| cachekit-ts | 9 | 78 | 7 | 73 | 71 | +| cachekit-rs | 4 | 69 | 3 | 69 | 93 | Not indexed yet: [`spec/interop-v2.md`](../spec/interop-v2.md). @@ -276,20 +276,23 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-76](../spec/saas-api.md#api-76) | *Counting outcomes*: … caching wrapper (the cachekit-py decorators, cachekit-ts `wrap()`, cachekit-rs `#[cachekit]`) **MUST** count one outcome, however many reads it makes. A call that runs the wrapped function counts one miss, before its … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do. | gap | gap | gap | | [API-77](../spec/saas-api.md#api-77) | *Optional Metrics Headers*: … them only while telemetry is on (Telemetry switch), and only from its CachekitIO backend: an SDK **MUST NOT** send them, or a report built for them, to any host but the CachekitIO API that backend is configured with. A Redis, … | **Gap:** No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or a report built for them, through a Redis, Memcached or File backend or to any host but its configured CachekitIO API. | gap | gap | gap | | [API-78](../spec/saas-api.md#api-78) | *L1 hit rate*: An SDK **MUST** compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same … | **Gap:** No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), sends it with other than three decimal places, or sends other than 0.000 for all-zero counters. | gap | gap | gap | -| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … revalidation an L1 stale serve triggers; the revalidation's own work is outside it. An SDK **MUST** time only calls it counts as an L1 hit and that return a value. It MUST NOT time any of these: | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits, misses or failed calls into the latency report, or starts or stops the clock outside the call; only the SDK's own tests do. | gap | gap | gap | -| [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … work is outside it. An SDK MUST time only calls it counts as an L1 hit and that return a value. It **MUST NOT** time any of these: | **Gap:** As API-79, for a client that times a joined in-process call, an existence check or a read made inside a wrapped call. | gap | gap | gap | -| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It **MUST** decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call … | **Gap:** As API-79, for a client that decides to time a call only after it knows the outcome, or samples by a counter rather than a per-call random draw. | gap | gap | gap | -| [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: … draw for each call with a fixed probability (a probability of 1 times every call). That decision **MUST NOT** depend on the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times … | **Gap:** As API-79, for a client whose sampling decision depends on the key, the arguments, the call's position in a sequence, the outcome or the duration. | gap | gap | gap | -| [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers. | gap | gap | gap | -| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … is on treats its clock as not qualifying, and sends no latency headers at all, never zeros. An SDK **MUST NOT** send a report whose duration sum is 0: on a qualifying clock that is a frozen clock, not a measurement. | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency report whose duration sum is 0. | gap | gap | gap | -| [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: When an SDK attaches a report to a request, it **MUST** take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values. | gap | gap | gap | -| [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … once that attempt is sent, whatever its outcome. A report whose attempt fails is lost: the SDK **MUST NOT** add its values back, or let a retry carry them. That includes a retry an HTTP library makes on its own: an SDK whose … | **Gap:** No vector covers request headers, so nothing here catches a client that adds a failed attempt's report back to its accumulator, or whose own or its HTTP library's retries resend a report. | gap | gap | gap | -| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and these rules: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, or a duration in the wrong bucket. | gap | gap | gap | -| [API-88](../spec/saas-api.md#api-88) | *L1 hit latency*: … The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | -| [API-89](../spec/saas-api.md#api-89) | *L1 hit latency*: … latency set, a missing or malformed header, or a sum of 0, drops both latency values. The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | -| [API-90](../spec/saas-api.md#api-90) | *L1 hit latency*: … SHOULD record a valid latency set on any authenticated request, whatever its response status. It **MUST NOT** read the optional metrics headers for billing or for rate limiting: they are client-reported. Only … | Binds the server.
**Gap:** Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | -| [API-91](../spec/saas-api.md#api-91) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on. | gap | gap | gap | -| [API-92](../spec/saas-api.md#api-92) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header on that backend's requests or another's, or drops X-CacheKit-L1-Status with it. | gap | gap | gap | +| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … start of the wrapped function. Only work that runs after the hand-back is outside it. An SDK **MUST** add to a report only the durations of calls it counts as an L1 hit and that return a value. It MUST NOT add the … | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report, or starts or stops the clock outside the call; only the SDK's own tests do. | gap | gap | gap | +| [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … add to a report only the durations of calls it counts as an L1 hit and that return a value. It **MUST NOT** add the duration of any of these: | **Gap:** As API-79, for a client that adds a joined in-process call, an existence check or a read made inside a wrapped call. | gap | gap | gap | +| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … from 1 to 1,000,000 that it fixes for the backend instance's lifetime (N = 1 times every call). It **MUST** decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call, … | **Gap:** As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, or lets the key, arguments, position, outcome or duration sway the draw. | gap | gap | gap | +| [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers. | gap | gap | gap | +| [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: … that backend's requests, so one backend's hits never ride another's requests. A process that forks **MUST** start the child with an empty accumulator, so that the parent's hits are not reported twice. | **Gap:** No vector reaches a forked process, so nothing here catches a child that inherits and re-sends its parent's accumulated report. | gap | gap | gap | +| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … sum is at least 1; a report whose sum is 0 is not attached and stays in the accumulator. It **MUST NOT** attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report … | **Gap:** No vector covers request headers, so nothing here catches a client that attaches a latency report to a health request. | gap | gap | gap | +| [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: … request, which the server does not record. When an SDK attaches a report to a request, it **MUST** take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values. | gap | gap | gap | +| [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … that carries it may have reached the server, the report is consumed whatever the outcome: the SDK **MUST NOT** add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an … | **Gap:** No vector covers request headers, so nothing here catches a client that adds a consumed report back to its accumulator, or whose own or its HTTP library's retries resend it after an attempt that may have been delivered. | gap | gap | gap | +| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and these rules: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, or a duration in the wrong bucket. | gap | gap | gap | +| [API-88](../spec/saas-api.md#api-88) | *Server handling*: The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | +| [API-89](../spec/saas-api.md#api-89) | *Server handling*: … 1; and, when bucket 16 is 0, at most the sum of each count times the bucket's bound). The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | +| [API-90](../spec/saas-api.md#api-90) | *Server handling*: … authenticated request, whatever its response status. A server that records valid latency sets **MUST** keep or drop each one only by a random draw, never by a cap such as a daily budget, and record with each kept set the … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops latency sets by a daily budget or other non-random cap, or keeps them without recording the probability it kept each one. | n/a | n/a | n/a | +| [API-91](../spec/saas-api.md#api-91) | *Server handling*: … every other client together. An aggregate of latency sets across tenants, or one shown publicly, **MUST** weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. | Binds the server.
**Gap:** How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-92](../spec/saas-api.md#api-92) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-93](../spec/saas-api.md#api-93) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting or ignores CACHEKIT_TELEMETRY. | gap | gap | gap | +| [API-94](../spec/saas-api.md#api-94) | *Telemetry switch*: An SDK **MUST** read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend … | **Gap:** As API-93, for a client that defaults the setting to true or false, lets a configuration loader fill it from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction. | gap | gap | gap | +| [API-95](../spec/saas-api.md#api-95) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it. | gap | gap | gap | Marked not-a-requirement in `spec/saas-api.md`: diff --git a/conformance/requirements.json b/conformance/requirements.json index 25436bf..fb9d862 100644 --- a/conformance/requirements.json +++ b/conformance/requirements.json @@ -1521,7 +1521,7 @@ }, "spec/saas-api.md": { "prefix": "API", - "next": 93, + "next": 96, "requirements": { "API-1": { "section": "Authentication", @@ -2124,32 +2124,32 @@ "API-79": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector reaches an SDK's timing, so nothing here catches a client that times L2 hits, misses or failed calls into the latency report, or starts or stops the clock outside the call; only the SDK's own tests do." + "gap": "No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report, or starts or stops the clock outside the call; only the SDK's own tests do." }, "API-80": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client that times a joined in-process call, an existence check or a read made inside a wrapped call." + "gap": "As API-79, for a client that adds a joined in-process call, an existence check or a read made inside a wrapped call." }, "API-81": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client that decides to time a call only after it knows the outcome, or samples by a counter rather than a per-call random draw." + "gap": "As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, or lets the key, arguments, position, outcome or duration sway the draw." }, "API-82": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client whose sampling decision depends on the key, the arguments, the call's position in a sequence, the outcome or the duration." + "gap": "No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers." }, "API-83": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers." + "gap": "No vector reaches a forked process, so nothing here catches a child that inherits and re-sends its parent's accumulated report." }, "API-84": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends a latency report whose duration sum is 0." + "gap": "No vector covers request headers, so nothing here catches a client that attaches a latency report to a health request." }, "API-85": { "section": "L1 hit latency", @@ -2159,37 +2159,52 @@ "API-86": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that adds a failed attempt's report back to its accumulator, or whose own or its HTTP library's retries resend a report." + "gap": "No vector covers request headers, so nothing here catches a client that adds a consumed report back to its accumulator, or whose own or its HTTP library's retries resend it after an attempt that may have been delivered." }, "API-87": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, or a duration in the wrong bucket." + "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, or a duration in the wrong bucket." }, "API-88": { - "section": "L1 hit latency", + "section": "Server handling", "binds": "server", "gap": "No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected." }, "API-89": { - "section": "L1 hit latency", + "section": "Server handling", "binds": "server", "gap": "As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed." }, "API-90": { - "section": "L1 hit latency", + "section": "Server handling", "binds": "server", - "gap": "Which inputs a server's billing and rate limiting read is server code no public vector or SDK test can observe." + "gap": "No public vector exercises the server, so nothing here catches one that drops latency sets by a daily budget or other non-random cap, or keeps them without recording the probability it kept each one." }, "API-91": { + "section": "Server handling", + "binds": "server", + "gap": "How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe." + }, + "API-92": { + "section": "Server handling", + "binds": "server", + "gap": "Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe." + }, + "API-93": { "section": "Telemetry switch", "binds": "sdk", - "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, ignores CACHEKIT_TELEMETRY, lets on win over off, or reads an unrecognised value as on." + "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting or ignores CACHEKIT_TELEMETRY." }, - "API-92": { + "API-94": { + "section": "Telemetry switch", + "binds": "sdk", + "gap": "As API-93, for a client that defaults the setting to true or false, lets a configuration loader fill it from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction." + }, + "API-95": { "section": "Telemetry switch", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header on that backend's requests or another's, or drops X-CacheKit-L1-Status with it." + "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it." } }, "retired": {} diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 9c400d3..14c7bd2 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -229,9 +229,9 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Metrics | ✅ Live `prometheus_client` counters / gauges / histograms on the default registry; no HTTP exposition helper | ✅ `cachekit-rs` 0.8.0+ — live counters ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). `CacheKit::stats()` returns `L1Stats`, plus `l1_entry_count()` and `circuit_state()`; no Prometheus exposition (LAB-521) | ✅ Node — live Prometheus metrics via the optional `prom-client` peer dep, custom registry supported, warns once and no-ops if absent (LAB-517) · ❌ Workers — `prom-client` is CI-excluded from the edge bundle¹ | ❌ | | Structured logging | ✅ JSON ring-buffer logger with sensitive-data masking (`logging.py`) | ✅ `cachekit-rs` 0.8.0+ — `tracing` events ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Opt-in `tracing` cargo feature: one `debug` event per get/set/delete, carrying `key_hash` (unkeyed Blake2b-128 of the storage key: a correlator, not a redaction), plus breaker transitions, to the host's subscriber (LAB-521) | ⚠️ Pluggable error-logger hook (`setLogger`, default `console.error`) — a logging seam, not a structured logger (LAB-517) | ❌ | | Distributed tracing (OTel) | ❌ The span-shaped API accepts spans and discards them (`NoOpSpan` in `FeatureOrchestrator.create_span`, `decorators/orchestrator.py`) — instrumentation built against it silently produces nothing | ❌ | ❌ | ❌ | -| SaaS telemetry headers (`X-CacheKit-L1-*`) | ✅ Auto. The decorators count one outcome per call ([Counting outcomes](spec/saas-api.md#counting-outcomes)) · ⚠️ sends `X-CacheKit-L1-Hit-Rate` as L1-Hits / (L1-Hits + L2-Hits), not the share of all counted operations [API-78](spec/saas-api.md#api-78) pins | ✅ `cachekit-rs` 0.8.0+ ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Auto-wired from live counters; an explicit `.metrics_provider()` still wins (LAB-521) · ⚠️ through 0.13.0 a `#[cachekit]` call can count up to about 51 misses ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-rs#196](https://github.com/cachekit-io/cachekit-rs/pull/196) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins · ⚠️ through npm 0.2.1 a `wrap()` call can count two outcomes or none ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-ts#265](https://github.com/cachekit-io/cachekit-ts/pull/265) | ❌ | -| SaaS L1 hit-latency headers (`X-CacheKit-L1-Latency-*`) | ❌ Specified ([L1 hit latency](spec/saas-api.md#l1-hit-latency)), not shipped. Backed-mode L1 hits are timed only for the local Prometheus histogram | ❌ Specified, not shipped. No L1 timer; never on the `workers` feature, whose clock cannot time an L1 hit ([API-83](spec/saas-api.md#api-83)) | ❌ Specified, not shipped. No L1 timer; never on the Workers entry, whose clock cannot time an L1 hit ([API-83](spec/saas-api.md#api-83)) | ❌ | -| Telemetry switch (`telemetry` setting, `CACHEKIT_TELEMETRY`) | ❌ Specified ([Telemetry switch](spec/saas-api.md#telemetry-switch)), not shipped: the session headers are always sent | ❌ Specified, not shipped: the session headers are always sent | ❌ Specified, not shipped: the session headers are always sent | ❌ | +| SaaS session telemetry headers (`X-CacheKit-Session-*`, `-L1-Hits`, `-L2-Hits`, `-Misses`, `-L1-Hit-Rate`) | ✅ Auto. The decorators count one outcome per call ([Counting outcomes](spec/saas-api.md#counting-outcomes)) · ⚠️ sends `X-CacheKit-L1-Hit-Rate` as L1-Hits / (L1-Hits + L2-Hits), not the share of all counted operations [API-78](spec/saas-api.md#api-78) pins | ✅ `cachekit-rs` 0.8.0+ ([cachekit-rs#81](https://github.com/cachekit-io/cachekit-rs/pull/81)). Auto-wired from live counters; an explicit `.metrics_provider()` still wins (LAB-521) · ⚠️ through 0.13.0 a `#[cachekit]` call can count up to about 51 misses ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-rs#196](https://github.com/cachekit-io/cachekit-rs/pull/196) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins · ⚠️ through npm 0.2.1 a `wrap()` call can count two outcomes or none ([API-76](spec/saas-api.md#api-76)); fix: [cachekit-ts#265](https://github.com/cachekit-io/cachekit-ts/pull/265) | ❌ | +| SaaS L1 hit-latency headers (`X-CacheKit-L1-Latency-*`) | ❌ Specified ([L1 hit latency](spec/saas-api.md#l1-hit-latency)), not shipped. Backed-mode L1 hits are timed only for the local Prometheus histogram | ❌ Specified, not shipped. No L1 timer; never on the `workers` feature, whose clock cannot time an L1 hit ([API-82](spec/saas-api.md#api-82)) | ❌ Specified, not shipped. No L1 timer; never on the Workers entry, whose clock cannot time an L1 hit ([API-82](spec/saas-api.md#api-82)) | ❌ | +| Telemetry switch (`telemetry` setting, `CACHEKIT_TELEMETRY`) | ❌ Specified ([Telemetry switch](spec/saas-api.md#telemetry-switch)), not shipped: no switch; the session headers ride every request made inside a decorated call | ❌ Specified, not shipped: the session headers are always sent | ❌ Specified, not shipped: the session headers are always sent | ❌ | --- diff --git a/spec/saas-api.md b/spec/saas-api.md index 922e67b..f6e6a46 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -536,6 +536,7 @@ The optional metrics headers are client-reported telemetry, for observability on | :--- | :--- | :--- | | `X-CacheKit-L1-Latency-Buckets` | 16 integers | The report's timed L1 hits, counted into sixteen fixed duration buckets. Their sum is the number of hits timed | | `X-CacheKit-L1-Latency-Sum-Ns` | integer | The sum of those hits' durations, in nanoseconds | +| `X-CacheKit-L1-Latency-One-In` | integer | N: each call was timed with probability 1/N | `X-CacheKit-L1-Status` is not one of these headers: it is required on every `ck_sdk_` request, whatever the telemetry setting. See [Required Headers](#required-headers). @@ -554,7 +555,7 @@ An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1 #### L1 hit latency -**What is timed.** A duration is one cache call counted as an L1 hit ([API-75](#api-75), [API-76](#api-76)), as its caller sees it. The clock starts at the first SDK instruction the call runs: for a coroutine or an async function, its first step, not its creation. It stops in the same synchronous step that produces the value the SDK hands back, never in a continuation or callback that runs after it. The interval includes deriving the cache key, the L1 lookup, any decryption and deserialization the SDK does on an L1 hit, and starting any background revalidation an L1 stale serve triggers; the revalidation's own work is outside it. An SDK MUSTAPI-79 time only calls it counts as an L1 hit and that return a value. It MUST NOTAPI-80 time any of these: +**What is timed.** A duration is one cache call counted as an L1 hit ([API-75](#api-75), [API-76](#api-76)), as its caller sees it. The clock starts at the first SDK instruction the call runs: for a coroutine or an async function, its first step, not its creation. It stops in the same synchronous step that produces the value the SDK hands back, never in a continuation or callback that runs after it. The interval includes all work the SDK does before it hands the value back: deriving the cache key, the L1 lookup, any decryption and deserialization, and any revalidation work an L1 stale serve runs before the value returns, such as the synchronous start of the wrapped function. Only work that runs after the hand-back is outside it. An SDK MUSTAPI-79 add to a report only the durations of calls it counts as an L1 hit and that return a value. It MUST NOTAPI-80 add the duration of any of these: - a wrapped call that joined another in-process call for the same key, because its duration is the other call's work; - an existence check; @@ -562,29 +563,29 @@ An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1 The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. -**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with a fixed probability (a probability of 1 times every call). That decision MUST NOTAPI-82 depend on the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A thread-local generator keeps the draw off any shared cache line. A call chosen for timing that is not an L1 hit is discarded. The probability is the SDK's choice and is not sent: the bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It times each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime (N = 1 times every call). It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call, never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates every L1 hit's latency, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. -**Clock.** An SDK MUST NOTAPI-83 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: +**Clock.** An SDK MUST NOTAPI-82 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: | SDK | Clock | | :--- | :--- | | Python | `time.perf_counter_ns()`, where `time.get_clock_info("perf_counter").resolution` is 1e-7 or less | | Rust | `std::time::Instant`, on any target but `wasm32` | | TypeScript on Node.js | `process.hrtime.bigint()` | -| Go | the monotonic reading of `time.Now()`, through `time.Since` | -Any other clock qualifies only where the SDK's documentation names it and its resolution. Cloudflare Workers has no qualifying clock: its clock advances only on I/O, so an in-memory hit would read as 0 ns. Neither has a browser that coarsens its timers. An SDK that cannot tell which runtime it is on treats its clock as not qualifying, and sends no latency headers at all, never zeros. An SDK MUST NOTAPI-84 send a report whose duration sum is 0: on a qualifying clock that is a frozen clock, not a measurement. +Any other clock qualifies only where the SDK's documentation names it and its resolution. Cloudflare Workers has no qualifying clock: its clock advances only on I/O, so an in-memory hit would read as 0 ns. An SDK that cannot tell which runtime it is on treats its clock as not qualifying, and sends no latency headers at all, never zeros. On a qualifying clock with a 100 ns tick, a fast hit can read 0 ns; that is a measurement, and it counts. -**Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the previous report. An SDK keeps one report accumulator per CachekitIO backend instance. It times a call into the accumulator of the backend that call's cache is configured with, and attaches the report only to that backend's requests, so one backend's hits never ride another's requests. A process that forks starts the child with an empty accumulator. +**Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the previous report. An SDK keeps one report accumulator per CachekitIO backend instance. It times a call into the accumulator of the backend that call's cache is configured with, and attaches the report only to that backend's requests, so one backend's hits never ride another's requests. A process that forks MUSTAPI-83 start the child with an empty accumulator, so that the parent's hits are not reported twice. -When an SDK attaches a report to a request, it MUSTAPI-85 take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and the buckets and the sum always describe the same hits. Separate atomic swaps of each field do not satisfy this; one structure taken under a lock, or swapped as a unit, does. It sends the two latency headers together, and only when at least one hit was timed. A report rides exactly one attempt and is consumed once that attempt is sent, whatever its outcome. A report whose attempt fails is lost: the SDK MUST NOTAPI-86 add its values back, or let a retry carry them. That includes a retry an HTTP library makes on its own: an SDK whose HTTP layer can resend a request without re-running the code that attaches the report must not use that layer's retries for a request that carries one. Hits timed after the client's last request are never reported; there is no background reporter. If adding a duration would push the sum or a bucket count past the bound in Encoding, the SDK discards that duration. +An SDK attaches a report only when its duration sum is at least 1; a report whose sum is 0 is not attached and stays in the accumulator. It MUST NOTAPI-84 attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report to a request, it MUSTAPI-85 take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and the buckets and the sum always describe the same hits. Separate atomic swaps of each field do not satisfy this; one structure taken under a lock, or swapped as a unit, does. It sends the three latency headers together. A report rides one attempt. Once an attempt that carries it may have reached the server, the report is consumed whatever the outcome: the SDK MUST NOTAPI-86 add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an attempt the HTTP layer knows was never delivered, such as a refused connection, may be retried with the same report. An SDK whose HTTP layer resends requests that may have been delivered, without re-running the code that attaches the report, does not use that layer's retries for a request that carries one. Hits timed after the client's last request are never reported; there is no background reporter. -Per-report values were chosen over the session counters' cumulative form for two reasons. A reader adds them across requests, sessions and clients as they stand, while cumulative values need each session's previous value to difference. And they need no per-session state: a server may discard a session-headers sample whose counters are lower than an earlier sample for the same Session-ID, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. A failed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. Reports are lost when their request fails or their process exits first, and those losses are not independent of load, so the latency data is a sample of the hits, not a census. +Per-report values were chosen over the session counters' cumulative form for two reasons. A reader adds them across requests, sessions and clients as they stand, while cumulative values need each session's previous value to difference. And they need no per-session state: a server may discard a session-headers sample whose counters are lower than an earlier sample for the same Session-ID, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. A consumed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. Reports are lost when their request fails or their process exits first, and those losses are not independent of load, so the latency data is a sample of the hits, not a census. A report carries no measurement time: its hits happened at some point since the previous report, so a reader that groups reports by time groups them by arrival and says so. -**Encoding.** Every latency value is a count or a sum, so a reader adds values across requests; no mean or percentile goes on the wire, because neither can be added. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-87 send values that meet this grammar and these rules: +**Encoding.** No mean or percentile goes on the wire, because neither can be added across reports. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-87 send values that meet this grammar and these rules: - `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. - `X-CacheKit-L1-Latency-Sum-Ns`: the sum of the same hits' durations, each in whole nanoseconds rounded down; at least 1. +- `X-CacheKit-L1-Latency-One-In`: N, the backend instance's sampling denominator, from 1 to 1,000,000. | Bucket | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | @@ -594,22 +595,26 @@ Per-report values were chosen over the session counters' cumulative form for two | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | *d* (ns) at most | 32,768 | 65,536 | 131,072 | 262,144 | 524,288 | 1,048,576 | 2,097,152 | no bound | -Each bucket holds the durations above the previous bucket's bound, up to and including its own; bucket 1 holds everything up to 128 ns, 0 included. Example: hits of 90, 300, 300 and 40,000 ns are sent as buckets `1,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0` and sum `40690`. +Each bucket holds the durations above the previous bucket's bound, up to and including its own; bucket 1 holds everything up to 128 ns, 0 included. Example: with N = 1, hits of 90, 300, 300 and 40,000 ns are sent as buckets `1,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0`, sum `40690` and one-in `1`. A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to about 2.1 ms cover the sub-microsecond hits the SDKs target, the tens of microseconds an encrypted L1 hit on a large value can take, and the pauses above them; a reader estimates a percentile by interpolating within the bucket that holds it. A percentile that falls in bucket 16 has no upper bound, so a reader reports it only as above 2,097,152 ns. A change to the bounds needs a new header name. -**Server handling.** The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-88 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail any of its checks, and the reverse. Within the latency set, a missing or malformed header, or a sum of 0, drops both latency values. The server MUST NOTAPI-89 answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid latency set on any authenticated request, whatever its response status. It MUST NOTAPI-90 read the optional metrics headers for billing or for rate limiting: they are client-reported. Only `X-CacheKit-L1-Status`, which [API-1](#api-1) requires, may be read by rate limiting. +#### Server handling -#### Telemetry switch +The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-88 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail any of the server's checks, and the reverse. Within the latency set, any of these drops all three latency values: a missing or malformed header, a sum of 0, or a sum outside the range its buckets allow (at least the sum, over the buckets, of each count times the bucket's lowest duration, which is the previous bucket's bound plus 1 and 0 for bucket 1; and, when bucket 16 is 0, at most the sum of each count times the bucket's bound). The server MUST NOTAPI-89 answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid latency set on every authenticated request, whatever its response status. A server that records valid latency sets MUSTAPI-90 keep or drop each one only by a random draw, never by a cap such as a daily budget, and record with each kept set the probability it was kept (1 when it keeps every one). A reader weights each kept set by that probability's inverse and by the set's N: requests that carry no session headers are not a random subset of clients, so a set dropped without a recorded probability biases what remains. + +Authentication proves who holds a key, not that its measurements are true: one set of sixteen grammatical, consistent values can claim more hits than every other client together. An aggregate of latency sets across tenants, or one shown publicly, MUSTAPI-91 weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. -Every SDK MUSTAPI-91 let its user turn telemetry off through both of these controls, which mean the same in every SDK: +The server MUST NOTAPI-92 let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. Of the `X-CacheKit-*` request headers, only `X-CacheKit-L1-Status`, which [API-1](#api-1) requires, may change a rate-limit verdict or limit, and none changes billing. + +#### Telemetry switch -- a setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend. It is an optional boolean that is unset unless the user sets it (Python `None`, TypeScript `undefined`, Rust `None`, a nil Go `*bool`): an SDK does not give it a default of true or false; -- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). Where the runtime has no process environment, as on Cloudflare Workers, the SDK reads it from the environment bindings it is constructed with; an SDK that has neither documents that only the setting applies there. +Every SDK MUSTAPI-93 let its user turn telemetry off through both of these controls, which mean the same in every SDK: -The SDK reads both controls once, when it constructs the backend; they hold for that backend instance's lifetime. It parses `CACHEKIT_TELEMETRY` only by the rule below, never through a configuration loader's own boolean parsing or precedence, so that off wins even where that loader would let a code argument override the environment. +- a setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend. It is an optional boolean that is unset unless the user sets it (Python `None`, TypeScript `undefined`, Rust `None`): an SDK does not give it a default of true or false; +- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). The SDK reads the process environment wherever the runtime has one, including Cloudflare Workers' `process.env` where it is populated; otherwise it reads the environment bindings it is constructed with. Only an SDK on a runtime with neither documents that only the setting applies there. -With telemetry off, an SDK MUST NOTAPI-92 send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Buckets` or `X-CacheKit-L1-Latency-Sum-Ns`. Its counts and timings reach no other backend's requests either. It still sends `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session. With telemetry off, an SDK SHOULD NOT accumulate a latency report. +An SDK MUSTAPI-94 read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend instance's lifetime. No configuration loader fills the `telemetry` setting from the environment, and `CACHEKIT_TELEMETRY` is parsed only by this rule, never by a loader's own boolean parsing or precedence: a loader that lets a code argument override the environment would let on win over off. `CACHEKIT_TELEMETRY` is read case-insensitively, after trimming surrounding white space: `false`, `0`, `off` and `no` are off; `true`, `1`, `on` and `yes` are on; unset or empty is unset. Any other value is off, and the SDK SHOULD warn once: a user who set the variable meant to change something, and off is the safe reading. Each control is then on, off or unset, and they combine like this: @@ -619,7 +624,7 @@ With telemetry off, an SDK MUST NOTAPI-92 send any option | **on** | on | on | off | | **off** | off | off | off | -Off wins: either control turns telemetry off, whatever the other says, so an operator's environment can turn off telemetry that code turned on, and the reverse. +With telemetry off, an SDK MUST NOTAPI-95 send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Buckets`, `X-CacheKit-L1-Latency-Sum-Ns` or `X-CacheKit-L1-Latency-One-In`. With telemetry off, an SDK SHOULD NOT accumulate a latency report. Two headers continue: `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session; and `User-Agent`, which names the SDK and its version, and, where the SDK sends one, its HTTP library and that library's version. --- @@ -668,7 +673,7 @@ SDKs should classify errors for circuit breaker integration: | `CACHEKIT_MAX_RETRIES` | No | `3` | Max retry attempts | | `CACHEKIT_CONNECTION_POOL_SIZE` | No | `10` | HTTP connection pool size | | `CACHEKIT_ALLOW_CUSTOM_HOST` | No | `false` | Allow non-standard API hostnames | -| `CACHEKIT_TELEMETRY` | No | unset (on) | `false`, `0`, `off`, `no` or any unrecognised value turns off the optional metrics headers; `true`, `1`, `on`, `yes` leave them on, unless the `telemetry` setting is off ([Telemetry switch](#telemetry-switch)) | +| `CACHEKIT_TELEMETRY` | No | unset (on) | Turns the optional metrics headers off; values and precedence in [Telemetry switch](#telemetry-switch) | ### SSRF Protection From 4ad34dd04b2d6b60303d236ee2d6b411664b67b3 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 21:42:56 +1100 Subject: [PATCH 4/7] fix(spec): bound latency sets and correct server and switch statements Validate the sampling denominator, cap a timed duration at 60 s so the range check always has an upper bound, give the server's in-set drop rule its own requirement, let server sampling vary by tenant, time or load but never drop deterministically, exclude denied and health requests from the record SHOULD, read the switch's variable from both process.env and bindings on Workers, keep the sum test inside the atomic take, and correct the billing and User-Agent statements. --- changelog.d/20261031_lab-12077.md | 10 ++++----- conformance/coverage.md | 31 ++++++++++++++------------- conformance/requirements.json | 35 ++++++++++++++++++------------- spec/saas-api.md | 28 ++++++++++++------------- 4 files changed, 54 insertions(+), 50 deletions(-) diff --git a/changelog.d/20261031_lab-12077.md b/changelog.d/20261031_lab-12077.md index d226681..5ae6804 100644 --- a/changelog.d/20261031_lab-12077.md +++ b/changelog.d/20261031_lab-12077.md @@ -3,7 +3,7 @@ - [`spec/saas-api.md` § L1 hit latency](spec/saas-api.md#l1-hit-latency) (new) defines three optional headers that report how long calls served from L1 take, as the caller sees them: `X-CacheKit-L1-Latency-Buckets` (sixteen fixed buckets doubling from 128 ns to about - 2.1 ms, the last one open; their sum is the number of hits timed), + 2.1 ms, the last one up to 60 s; their sum is the number of hits timed), `X-CacheKit-L1-Latency-Sum-Ns`, and `X-CacheKit-L1-Latency-One-In` (the sampling denominator N, so readers weight each report). Values are counted per report, not per session, kept per CachekitIO backend instance, taken atomically and never re-sent once an @@ -12,15 +12,15 @@ Workers ([API-79](spec/saas-api.md#api-79)–[API-87](spec/saas-api.md#api-87)). - [`spec/saas-api.md` § Server handling](spec/saas-api.md#server-handling) (new): the server validates the latency and session headers as independent sets, never rejects a request over - one, drops latency sets only at random and records each kept set's probability, weights + one, drops valid latency sets only at random and records each kept set's probability, weights tenants equally (or caps them) in cross-tenant aggregates, and never lets an optional metrics header change billing or a rate-limit verdict - ([API-88](spec/saas-api.md#api-88)–[API-92](spec/saas-api.md#api-92)). + ([API-88](spec/saas-api.md#api-88)–[API-93](spec/saas-api.md#api-93)). - [`spec/saas-api.md` § Telemetry switch](spec/saas-api.md#telemetry-switch) (new): every SDK exposes an optional `telemetry` setting and reads `CACHEKIT_TELEMETRY`, once, at backend construction; either one turned off stops every optional metrics header. `X-CacheKit-L1-Status` - and `User-Agent` are still sent. Default: on - ([API-93](spec/saas-api.md#api-93)–[API-95](spec/saas-api.md#api-95)). + and any `User-Agent` continue. Default: on + ([API-94](spec/saas-api.md#api-94)–[API-96](spec/saas-api.md#api-96)). - [`spec/saas-api.md` § Optional Metrics Headers](spec/saas-api.md#optional-metrics-headers): the headers are client-reported observability, sent only to the configured CachekitIO API ([API-77](spec/saas-api.md#api-77)). diff --git a/conformance/coverage.md b/conformance/coverage.md index 311d6d0..fead22a 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -25,14 +25,14 @@ How ids are assigned, and what each status means: [README.md](README.md#coverage | [`spec/file-backend-format.md`](../spec/file-backend-format.md) | 6 | 1 | 3 | 0 | 2 | | [`spec/intent-presets.md`](../spec/intent-presets.md) | 55 | 0 | 19 | 0 | 36 | | [`spec/interop-mode.md`](../spec/interop-mode.md) | 36 | 5 | 23 | 4 | 4 | -| [`spec/saas-api.md`](../spec/saas-api.md) | 95 | 0 | 15 | 0 | 80 | +| [`spec/saas-api.md`](../spec/saas-api.md) | 96 | 0 | 15 | 0 | 81 | | [`spec/wire-format.md`](../spec/wire-format.md) | 24 | 3 | 18 | 3 | 0 | | SDK | Covered | Partial | Uncovered | Gap | n/a | | :--- | ---: | ---: | ---: | ---: | ---: | -| cachekit-py | 12 | 86 | 5 | 83 | 52 | -| cachekit-ts | 9 | 78 | 7 | 73 | 71 | -| cachekit-rs | 4 | 69 | 3 | 69 | 93 | +| cachekit-py | 12 | 86 | 5 | 83 | 53 | +| cachekit-ts | 9 | 78 | 7 | 73 | 72 | +| cachekit-rs | 4 | 69 | 3 | 69 | 94 | Not indexed yet: [`spec/interop-v2.md`](../spec/interop-v2.md). @@ -276,23 +276,24 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-76](../spec/saas-api.md#api-76) | *Counting outcomes*: … caching wrapper (the cachekit-py decorators, cachekit-ts `wrap()`, cachekit-rs `#[cachekit]`) **MUST** count one outcome, however many reads it makes. A call that runs the wrapped function counts one miss, before its … | **Gap:** No vector covers request headers, so nothing here catches a client that counts a wrapped call that runs the function as anything but one miss, a joined in-process caller as anything but one L1 hit, or a miss on each single-flight re-poll or lock re-check; only the SDK's own counter tests do. | gap | gap | gap | | [API-77](../spec/saas-api.md#api-77) | *Optional Metrics Headers*: … them only while telemetry is on (Telemetry switch), and only from its CachekitIO backend: an SDK **MUST NOT** send them, or a report built for them, to any host but the CachekitIO API that backend is configured with. A Redis, … | **Gap:** No vector covers request headers, so nothing here catches a client that sends the optional metrics headers, or a report built for them, through a Redis, Memcached or File backend or to any host but its configured CachekitIO API. | gap | gap | gap | | [API-78](../spec/saas-api.md#api-78) | *L1 hit rate*: An SDK **MUST** compute `X-CacheKit-L1-Hit-Rate` as `L1-Hits / (L1-Hits + L2-Hits + Misses)` from the counter values on the same … | **Gap:** No vector covers request headers, so nothing here catches a client that computes X-CacheKit-L1-Hit-Rate as L1-Hits / (L1-Hits + L2-Hits), sends it with other than three decimal places, or sends other than 0.000 for all-zero counters. | gap | gap | gap | -| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … start of the wrapped function. Only work that runs after the hand-back is outside it. An SDK **MUST** add to a report only the durations of calls it counts as an L1 hit and that return a value. It MUST NOT add the … | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report, or starts or stops the clock outside the call; only the SDK's own tests do. | gap | gap | gap | +| [API-79](../spec/saas-api.md#api-79) | *L1 hit latency*: … start of the wrapped function. Only work that runs after the hand-back is outside it. An SDK **MUST** add to a report only the durations of calls it counts as an L1 hit and that return a value. It MUST NOT add the … | **Gap:** No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report; only the SDK's own tests do. | gap | gap | gap | | [API-80](../spec/saas-api.md#api-80) | *L1 hit latency*: … add to a report only the durations of calls it counts as an L1 hit and that return a value. It **MUST NOT** add the duration of any of these: | **Gap:** As API-79, for a client that adds a joined in-process call, an existence check or a read made inside a wrapped call. | gap | gap | gap | -| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … from 1 to 1,000,000 that it fixes for the backend instance's lifetime (N = 1 times every call). It **MUST** decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call, … | **Gap:** As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, or lets the key, arguments, position, outcome or duration sway the draw. | gap | gap | gap | +| [API-81](../spec/saas-api.md#api-81) | *L1 hit latency*: … rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It **MUST** decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call … | **Gap:** As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, lets the key, arguments, position, outcome or duration sway the draw, or draws with a probability other than the 1/N it sends. | gap | gap | gap | | [API-82](../spec/saas-api.md#api-82) | *L1 hit latency*: Clock. An SDK **MUST NOT** send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and … | **Gap:** No vector covers request headers, so nothing here catches a client that sends latency headers timed with a clock that does not advance during synchronous execution or resolves coarser than 100 ns, such as on Cloudflare Workers. | gap | gap | gap | | [API-83](../spec/saas-api.md#api-83) | *L1 hit latency*: … that backend's requests, so one backend's hits never ride another's requests. A process that forks **MUST** start the child with an empty accumulator, so that the parent's hits are not reported twice. | **Gap:** No vector reaches a forked process, so nothing here catches a child that inherits and re-sends its parent's accumulated report. | gap | gap | gap | -| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … sum is at least 1; a report whose sum is 0 is not attached and stays in the accumulator. It **MUST NOT** attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report … | **Gap:** No vector covers request headers, so nothing here catches a client that attaches a latency report to a health request. | gap | gap | gap | +| [API-84](../spec/saas-api.md#api-84) | *L1 hit latency*: … sum test is part of the atomic step below, so a racing request never attaches all-zero buckets. It **MUST NOT** attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report … | **Gap:** No vector covers request headers, so nothing here catches a client that attaches a latency report to a health request. | gap | gap | gap | | [API-85](../spec/saas-api.md#api-85) | *L1 hit latency*: … request, which the server does not record. When an SDK attaches a report to a request, it **MUST** take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and … | **Gap:** No vector covers concurrent requests, so nothing here catches a client whose concurrent attempts carry the same timed hit, whose buckets and sum describe different hits, or that sends cumulative rather than per-report values. | gap | gap | gap | | [API-86](../spec/saas-api.md#api-86) | *L1 hit latency*: … that carries it may have reached the server, the report is consumed whatever the outcome: the SDK **MUST NOT** add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an … | **Gap:** No vector covers request headers, so nothing here catches a client that adds a consumed report back to its accumulator, or whose own or its HTTP library's retries resend it after an attempt that may have been delivered. | gap | gap | gap | -| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and these rules: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, or a duration in the wrong bucket. | gap | gap | gap | +| [API-87](../spec/saas-api.md#api-87) | *L1 hit latency*: … (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK **MUST** send values that meet this grammar and these rules: | **Gap:** No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, a duration in the wrong bucket, or a duration above 60 s. | gap | gap | gap | | [API-88](../spec/saas-api.md#api-88) | *Server handling*: The server validates the latency headers as one set and the session headers as another. It **MUST NOT** drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected. | n/a | n/a | n/a | -| [API-89](../spec/saas-api.md#api-89) | *Server handling*: … 1; and, when bucket 16 is 0, at most the sum of each count times the bucket's bound). The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | -| [API-90](../spec/saas-api.md#api-90) | *Server handling*: … authenticated request, whatever its response status. A server that records valid latency sets **MUST** keep or drop each one only by a random draw, never by a cap such as a daily budget, and record with each kept set the … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops latency sets by a daily budget or other non-random cap, or keeps them without recording the probability it kept each one. | n/a | n/a | n/a | -| [API-91](../spec/saas-api.md#api-91) | *Server handling*: … every other client together. An aggregate of latency sets across tenants, or one shown publicly, **MUST** weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. | Binds the server.
**Gap:** How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe. | n/a | n/a | n/a | -| [API-92](../spec/saas-api.md#api-92) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | -| [API-93](../spec/saas-api.md#api-93) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting or ignores CACHEKIT_TELEMETRY. | gap | gap | gap | -| [API-94](../spec/saas-api.md#api-94) | *Telemetry switch*: An SDK **MUST** read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend … | **Gap:** As API-93, for a client that defaults the setting to true or false, lets a configuration loader fill it from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction. | gap | gap | gap | -| [API-95](../spec/saas-api.md#api-95) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it. | gap | gap | gap | +| [API-89](../spec/saas-api.md#api-89) | *Server handling*: … headers fail any of the server's checks, and the reverse. Within the latency set, the server **MUST** drop all three latency values when any of these holds: a header is missing or malformed; the sum is 0; the One-In … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that records a latency set with a missing or malformed header, a sum of 0, a One-In outside 1 to 1,000,000, or a sum its buckets cannot produce. | n/a | n/a | n/a | +| [API-90](../spec/saas-api.md#api-90) | *Server handling*: … most the sum of each count times the bucket's bound (60,000,000,000 ns for bucket 16). The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | +| [API-91](../spec/saas-api.md#api-91) | *Server handling*: … its response status, except `GET /v1/cache/health`. A server that records valid latency sets **MUST** keep or drop each one only by a random draw, whose probability may vary by tenant, time or load, for example to stay … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops latency sets deterministically, such as once a write budget is spent, or keeps them without recording the probability it kept each one. | n/a | n/a | n/a | +| [API-92](../spec/saas-api.md#api-92) | *Server handling*: … every other client together. An aggregate of latency sets across tenants, or one shown publicly, **MUST** weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. | Binds the server.
**Gap:** How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-93](../spec/saas-api.md#api-93) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-94](../spec/saas-api.md#api-94) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, or ignores CACHEKIT_TELEMETRY. | gap | gap | gap | +| [API-95](../spec/saas-api.md#api-95) | *Telemetry switch*: An SDK **MUST** read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend … | **Gap:** As API-94, for a client that lets a configuration loader fill the setting from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction. | gap | gap | gap | +| [API-96](../spec/saas-api.md#api-96) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header. | gap | gap | gap | Marked not-a-requirement in `spec/saas-api.md`: diff --git a/conformance/requirements.json b/conformance/requirements.json index fb9d862..2f6ac14 100644 --- a/conformance/requirements.json +++ b/conformance/requirements.json @@ -1521,7 +1521,7 @@ }, "spec/saas-api.md": { "prefix": "API", - "next": 96, + "next": 97, "requirements": { "API-1": { "section": "Authentication", @@ -2124,7 +2124,7 @@ "API-79": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report, or starts or stops the clock outside the call; only the SDK's own tests do." + "gap": "No vector reaches an SDK's timing, so nothing here catches a client that adds L2 hits, misses or failed calls to the latency report; only the SDK's own tests do." }, "API-80": { "section": "L1 hit latency", @@ -2134,7 +2134,7 @@ "API-81": { "section": "L1 hit latency", "binds": "sdk", - "gap": "As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, or lets the key, arguments, position, outcome or duration sway the draw." + "gap": "As API-79, for a client that decides to time a call after it knows the outcome, samples by a counter, lets the key, arguments, position, outcome or duration sway the draw, or draws with a probability other than the 1/N it sends." }, "API-82": { "section": "L1 hit latency", @@ -2164,47 +2164,52 @@ "API-87": { "section": "L1 hit latency", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, or a duration in the wrong bucket." + "gap": "No vector covers request headers, so nothing here catches a client that sends a latency value outside the integer grammar, other than sixteen buckets, all-zero buckets, a sum of 0, a one-in value outside 1 to 1,000,000, a duration in the wrong bucket, or a duration above 60 s." }, "API-88": { "section": "Server handling", "binds": "server", "gap": "No public vector exercises the server, so nothing here catches one that drops the session headers because a latency header is malformed, or the latency headers because the session headers are missing, malformed or rejected." }, - "API-89": { + "API-90": { "section": "Server handling", "binds": "server", "gap": "As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed." }, - "API-90": { + "API-91": { "section": "Server handling", "binds": "server", - "gap": "No public vector exercises the server, so nothing here catches one that drops latency sets by a daily budget or other non-random cap, or keeps them without recording the probability it kept each one." + "gap": "No public vector exercises the server, so nothing here catches one that drops latency sets deterministically, such as once a write budget is spent, or keeps them without recording the probability it kept each one." }, - "API-91": { + "API-92": { "section": "Server handling", "binds": "server", "gap": "How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe." }, - "API-92": { + "API-93": { "section": "Server handling", "binds": "server", "gap": "Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe." }, - "API-93": { + "API-94": { "section": "Telemetry switch", "binds": "sdk", - "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting or ignores CACHEKIT_TELEMETRY." + "gap": "No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, or ignores CACHEKIT_TELEMETRY." }, - "API-94": { + "API-95": { "section": "Telemetry switch", "binds": "sdk", - "gap": "As API-93, for a client that defaults the setting to true or false, lets a configuration loader fill it from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction." + "gap": "As API-94, for a client that lets a configuration loader fill the setting from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction." }, - "API-95": { + "API-96": { "section": "Telemetry switch", "binds": "sdk", - "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header, or drops X-CacheKit-L1-Status with it." + "gap": "No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header." + }, + "API-89": { + "section": "Server handling", + "binds": "server", + "gap": "No public vector exercises the server, so nothing here catches one that records a latency set with a missing or malformed header, a sum of 0, a One-In outside 1 to 1,000,000, or a sum its buckets cannot produce." } }, "retired": {} diff --git a/spec/saas-api.md b/spec/saas-api.md index f6e6a46..fa785f4 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -538,8 +538,6 @@ The optional metrics headers are client-reported telemetry, for observability on | `X-CacheKit-L1-Latency-Sum-Ns` | integer | The sum of those hits' durations, in nanoseconds | | `X-CacheKit-L1-Latency-One-In` | integer | N: each call was timed with probability 1/N | -`X-CacheKit-L1-Status` is not one of these headers: it is required on every `ck_sdk_` request, whatever the telemetry setting. See [Required Headers](#required-headers). - #### Counting outcomes `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits` and `X-CacheKit-Misses` are cumulative for the session. Each counted operation adds one to exactly one of them, so their sum is the number of operations counted: @@ -563,7 +561,7 @@ An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1 The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. -**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It times each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime (N = 1 times every call). It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call, never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates every L1 hit's latency, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates every L1 hit's latency, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. **Clock.** An SDK MUST NOTAPI-82 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: @@ -577,13 +575,13 @@ Any other clock qualifies only where the SDK's documentation names it and its re **Reports.** The latency values are counted per report, not per session: each report covers the hits timed since the previous report. An SDK keeps one report accumulator per CachekitIO backend instance. It times a call into the accumulator of the backend that call's cache is configured with, and attaches the report only to that backend's requests, so one backend's hits never ride another's requests. A process that forks MUSTAPI-83 start the child with an empty accumulator, so that the parent's hits are not reported twice. -An SDK attaches a report only when its duration sum is at least 1; a report whose sum is 0 is not attached and stays in the accumulator. It MUST NOTAPI-84 attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report to a request, it MUSTAPI-85 take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and the buckets and the sum always describe the same hits. Separate atomic swaps of each field do not satisfy this; one structure taken under a lock, or swapped as a unit, does. It sends the three latency headers together. A report rides one attempt. Once an attempt that carries it may have reached the server, the report is consumed whatever the outcome: the SDK MUST NOTAPI-86 add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an attempt the HTTP layer knows was never delivered, such as a refused connection, may be retried with the same report. An SDK whose HTTP layer resends requests that may have been delivered, without re-running the code that attaches the report, does not use that layer's retries for a request that carries one. Hits timed after the client's last request are never reported; there is no background reporter. +An SDK attaches a report only when its duration sum is at least 1; a report whose sum is 0 is not attached and stays in the accumulator. The sum test is part of the atomic step below, so a racing request never attaches all-zero buckets. It MUST NOTAPI-84 attach a report to a `GET /v1/cache/health` request, which the server does not record. When an SDK attaches a report to a request, it MUSTAPI-85 take every value and reset the accumulator in one atomic step, so that no two attempts carry the same timed hit and the buckets and the sum always describe the same hits. Separate atomic swaps of each field do not satisfy this; one structure taken under a lock, or swapped as a unit, does. It sends the three latency headers together. A report rides one attempt. Once an attempt that carries it may have reached the server, the report is consumed whatever the outcome: the SDK MUST NOTAPI-86 add its values back, or send them on any later attempt, including a retry its HTTP library makes on its own. Only an attempt the HTTP layer knows was never delivered, such as a refused connection, may be retried with the same report. An SDK whose HTTP layer resends requests that may have been delivered, without re-running the code that attaches the report, does not use that layer's retries for a request that carries one. Hits timed after the client's last request are never reported; there is no background reporter. -Per-report values were chosen over the session counters' cumulative form for two reasons. A reader adds them across requests, sessions and clients as they stand, while cumulative values need each session's previous value to difference. And they need no per-session state: a server may discard a session-headers sample whose counters are lower than an earlier sample for the same Session-ID, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. A consumed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. Reports are lost when their request fails or their process exits first, and those losses are not independent of load, so the latency data is a sample of the hits, not a census. A report carries no measurement time: its hits happened at some point since the previous report, so a reader that groups reports by time groups them by arrival and says so. +Per-report values were chosen over the session counters' cumulative form for two reasons. A reader adds them across requests, sessions and clients as they stand, while cumulative values need each session's previous value to difference. And they need no per-session state: a server may discard a session-headers sample whose counters are lower than an earlier sample for the same Session-ID, which happens whenever two clients that share one Session-ID interleave their requests, and per-report values have nothing to go backwards. A consumed report is not re-sent because the server may have recorded it before the request failed, and nothing deduplicates a report sent twice. Reports are lost when their request fails or their process exits first, and those losses are not independent of load, so the latency data is a sample of the hits, not a census. A report carries no measurement time: its hits happened at some point since the previous report, so a reader that groups reports by time groups them by arrival. **Encoding.** No mean or percentile goes on the wire, because neither can be added across reports. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-87 send values that meet this grammar and these rules: -- `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. +- `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. A duration above 60,000,000,000 ns (60 s) is discarded, not counted: no L1 hit takes that long, and the bound gives the server's range check an upper limit. - `X-CacheKit-L1-Latency-Sum-Ns`: the sum of the same hits' durations, each in whole nanoseconds rounded down; at least 1. - `X-CacheKit-L1-Latency-One-In`: N, the backend instance's sampling denominator, from 1 to 1,000,000. @@ -593,28 +591,28 @@ Per-report values were chosen over the session counters' cumulative form for two | Bucket | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | -| *d* (ns) at most | 32,768 | 65,536 | 131,072 | 262,144 | 524,288 | 1,048,576 | 2,097,152 | no bound | +| *d* (ns) at most | 32,768 | 65,536 | 131,072 | 262,144 | 524,288 | 1,048,576 | 2,097,152 | 60,000,000,000 | Each bucket holds the durations above the previous bucket's bound, up to and including its own; bucket 1 holds everything up to 128 ns, 0 included. Example: with N = 1, hits of 90, 300, 300 and 40,000 ns are sent as buckets `1,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0`, sum `40690` and one-in `1`. -A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to about 2.1 ms cover the sub-microsecond hits the SDKs target, the tens of microseconds an encrypted L1 hit on a large value can take, and the pauses above them; a reader estimates a percentile by interpolating within the bucket that holds it. A percentile that falls in bucket 16 has no upper bound, so a reader reports it only as above 2,097,152 ns. A change to the bounds needs a new header name. +A mean alone would hide the tail that a garbage-collection pause or a contended lock adds, and a percentile computed by a client cannot be combined with another client's. Buckets that double from 128 ns to about 2.1 ms cover the sub-microsecond hits the SDKs target, the tens of microseconds an encrypted L1 hit on a large value can take, and the pauses above them; a reader estimates a percentile by interpolating within the bucket that holds it. Bucket 16 spans 2,097,153 ns to 60 s, so a percentile that falls in it is reported only as above 2,097,152 ns. A change to the bounds needs a new header name. #### Server handling -The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-88 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail any of the server's checks, and the reverse. Within the latency set, any of these drops all three latency values: a missing or malformed header, a sum of 0, or a sum outside the range its buckets allow (at least the sum, over the buckets, of each count times the bucket's lowest duration, which is the previous bucket's bound plus 1 and 0 for bucket 1; and, when bucket 16 is 0, at most the sum of each count times the bucket's bound). The server MUST NOTAPI-89 answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid latency set on every authenticated request, whatever its response status. A server that records valid latency sets MUSTAPI-90 keep or drop each one only by a random draw, never by a cap such as a daily budget, and record with each kept set the probability it was kept (1 when it keeps every one). A reader weights each kept set by that probability's inverse and by the set's N: requests that carry no session headers are not a random subset of clients, so a set dropped without a recorded probability biases what remains. +The server validates the latency headers as one set and the session headers as another. It MUST NOTAPI-88 drop either set because the other is missing, malformed or rejected: a latency set is recorded when the session headers fail any of the server's checks, and the reverse. Within the latency set, the server MUSTAPI-89 drop all three latency values when any of these holds: a header is missing or malformed; the sum is 0; the One-In value is outside 1 to 1,000,000; or the sum is outside the range its buckets allow, which is at least the sum, over the buckets, of each count times the bucket's lowest duration (the previous bucket's bound plus 1, and 0 for bucket 1), and at most the sum of each count times the bucket's bound (60,000,000,000 ns for bucket 16). The server MUST NOTAPI-90 answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid latency set on every authenticated request it admits, whatever its response status, except `GET /v1/cache/health`. A server that records valid latency sets MUSTAPI-91 keep or drop each one only by a random draw, whose probability may vary by tenant, time or load, for example to stay within a write budget, and record with each kept set the probability p it was kept (1 when it keeps every one). It never drops sets deterministically once a budget is spent: requests that carry no session headers are not a random subset of clients, so a set dropped without a recorded probability biases what remains. A reader weights each kept set by N / p, times any sampling interval its storage layer applies. It does that arithmetic, like the range check, in IEEE 754 doubles or exact integers: N × Sum-Ns, and a count times 60,000,000,000, can overflow a 64-bit integer at grammatical values. -Authentication proves who holds a key, not that its measurements are true: one set of sixteen grammatical, consistent values can claim more hits than every other client together. An aggregate of latency sets across tenants, or one shown publicly, MUSTAPI-91 weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. +Authentication proves who holds a key, not that its measurements are true: one set of sixteen grammatical, consistent values can claim more hits than every other client together. An aggregate of latency sets across tenants, or one shown publicly, MUSTAPI-92 weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. -The server MUST NOTAPI-92 let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. Of the `X-CacheKit-*` request headers, only `X-CacheKit-L1-Status`, which [API-1](#api-1) requires, may change a rate-limit verdict or limit, and none changes billing. +The server MUST NOTAPI-93 let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. `X-CacheKit-L1-Status` ([API-1](#api-1)) is not an optional metrics header; this rule does not cover it. #### Telemetry switch -Every SDK MUSTAPI-93 let its user turn telemetry off through both of these controls, which mean the same in every SDK: +Every SDK MUSTAPI-94 let its user turn telemetry off through both of these controls, which mean the same in every SDK: - a setting named `telemetry` (in the SDK's own case convention) on the configuration of its CachekitIO backend. It is an optional boolean that is unset unless the user sets it (Python `None`, TypeScript `undefined`, Rust `None`): an SDK does not give it a default of true or false; -- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). The SDK reads the process environment wherever the runtime has one, including Cloudflare Workers' `process.env` where it is populated; otherwise it reads the environment bindings it is constructed with. Only an SDK on a runtime with neither documents that only the setting applies there. +- the environment variable `CACHEKIT_TELEMETRY` ([Environment Variables](#environment-variables)). The SDK reads the process environment wherever the runtime has one. On Cloudflare Workers it reads `CACHEKIT_TELEMETRY` from both `process.env` and the Worker's bindings (`env` imported from `cloudflare:workers`, or the bindings it is constructed with); off from either wins. Only an SDK on a runtime with neither a process environment nor bindings documents that only the setting applies there. -An SDK MUSTAPI-94 read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend instance's lifetime. No configuration loader fills the `telemetry` setting from the environment, and `CACHEKIT_TELEMETRY` is parsed only by this rule, never by a loader's own boolean parsing or precedence: a loader that lets a code argument override the environment would let on win over off. +An SDK MUSTAPI-95 read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend instance's lifetime. No configuration loader fills the `telemetry` setting from the environment, and `CACHEKIT_TELEMETRY` is parsed only by this rule, never by a loader's own boolean parsing or precedence: a loader that lets a code argument override the environment would let on win over off. `CACHEKIT_TELEMETRY` is read case-insensitively, after trimming surrounding white space: `false`, `0`, `off` and `no` are off; `true`, `1`, `on` and `yes` are on; unset or empty is unset. Any other value is off, and the SDK SHOULD warn once: a user who set the variable meant to change something, and off is the safe reading. Each control is then on, off or unset, and they combine like this: @@ -624,7 +622,7 @@ An SDK MUSTAPI-94 read both controls once, when it constr | **on** | on | on | off | | **off** | off | off | off | -With telemetry off, an SDK MUST NOTAPI-95 send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Buckets`, `X-CacheKit-L1-Latency-Sum-Ns` or `X-CacheKit-L1-Latency-One-In`. With telemetry off, an SDK SHOULD NOT accumulate a latency report. Two headers continue: `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session; and `User-Agent`, which names the SDK and its version, and, where the SDK sends one, its HTTP library and that library's version. +With telemetry off, an SDK MUST NOTAPI-96 send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, `X-CacheKit-L1-Hits`, `X-CacheKit-L2-Hits`, `X-CacheKit-Misses`, `X-CacheKit-L1-Hit-Rate`, `X-CacheKit-L1-Latency-Buckets`, `X-CacheKit-L1-Latency-Sum-Ns` or `X-CacheKit-L1-Latency-One-In`. With telemetry off, an SDK SHOULD NOT accumulate a latency report. Besides the headers each operation needs, two continue: `X-CacheKit-L1-Status`, which [API-1](#api-1) requires on every `ck_sdk_` request and which describes that one request, not the session; and a `User-Agent` that may name the SDK and its version, its runtime, and its HTTP library and that library's version. --- From 22c6715aaa7d6f1db8818ed8eb480b79b615d5dd Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 21:46:56 +1100 Subject: [PATCH 5/7] fix(spec): keep L1-Status out of billing in the server rule --- conformance/coverage.md | 2 +- conformance/requirements.json | 2 +- spec/saas-api.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/conformance/coverage.md b/conformance/coverage.md index fead22a..9aefd65 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -290,7 +290,7 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-90](../spec/saas-api.md#api-90) | *Server handling*: … most the sum of each count times the bucket's bound (60,000,000,000 ns for bucket 16). The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | | [API-91](../spec/saas-api.md#api-91) | *Server handling*: … its response status, except `GET /v1/cache/health`. A server that records valid latency sets **MUST** keep or drop each one only by a random draw, whose probability may vary by tenant, time or load, for example to stay … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops latency sets deterministically, such as once a write budget is spent, or keeps them without recording the probability it kept each one. | n/a | n/a | n/a | | [API-92](../spec/saas-api.md#api-92) | *Server handling*: … every other client together. An aggregate of latency sets across tenants, or one shown publicly, **MUST** weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. | Binds the server.
**Gap:** How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe. | n/a | n/a | n/a | -| [API-93](../spec/saas-api.md#api-93) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-93](../spec/saas-api.md#api-93) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting, or let `X-CacheKit-L1-Status` change billing: all of … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read, including whether billing reads X-CacheKit-L1-Status, is server code no public vector or SDK test can observe. | n/a | n/a | n/a | | [API-94](../spec/saas-api.md#api-94) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, or ignores CACHEKIT_TELEMETRY. | gap | gap | gap | | [API-95](../spec/saas-api.md#api-95) | *Telemetry switch*: An SDK **MUST** read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend … | **Gap:** As API-94, for a client that lets a configuration loader fill the setting from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction. | gap | gap | gap | | [API-96](../spec/saas-api.md#api-96) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header. | gap | gap | gap | diff --git a/conformance/requirements.json b/conformance/requirements.json index 2f6ac14..e9e7199 100644 --- a/conformance/requirements.json +++ b/conformance/requirements.json @@ -2189,7 +2189,7 @@ "API-93": { "section": "Server handling", "binds": "server", - "gap": "Which inputs a server's billing and rate-limit verdicts read is server code no public vector or SDK test can observe." + "gap": "Which inputs a server's billing and rate-limit verdicts read, including whether billing reads X-CacheKit-L1-Status, is server code no public vector or SDK test can observe." }, "API-94": { "section": "Telemetry switch", diff --git a/spec/saas-api.md b/spec/saas-api.md index fa785f4..5d4aa99 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -603,7 +603,7 @@ The server validates the latency headers as one set and the session headers as a Authentication proves who holds a key, not that its measurements are true: one set of sixteen grammatical, consistent values can claim more hits than every other client together. An aggregate of latency sets across tenants, or one shown publicly, MUSTAPI-92 weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. -The server MUST NOTAPI-93 let an optional metrics header change billing or rate limiting: they are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. `X-CacheKit-L1-Status` ([API-1](#api-1)) is not an optional metrics header; this rule does not cover it. +The server MUST NOTAPI-93 let an optional metrics header change billing or rate limiting, or let `X-CacheKit-L1-Status` change billing: all of them are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. `X-CacheKit-L1-Status` may change rate limiting, as [API-1](#api-1) describes; headers that carry an operation's own inputs, such as `X-CacheKit-TTL`, are outside this rule. #### Telemetry switch From da7c2075d9517382fa137d1c3bfd6c4d749e5e45 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 21:54:18 +1100 Subject: [PATCH 6/7] fix(spec): state that latency data covers L1 hits of 60 s or less --- spec/saas-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/saas-api.md b/spec/saas-api.md index 5d4aa99..598d032 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -561,7 +561,7 @@ An SDK MUSTAPI-78 compute `X-CacheKit-L1-Hit-Rate` as `L1 The whole call is timed, rather than the L1 lookup alone, for three reasons. It is what a user pays for a hit, and what the SDKs' performance targets measure. It is comparable across SDKs whose L1 layers store different representations: an SDK that decodes or decrypts on every L1 hit would look faster than it is if only the lookup were timed. And it needs no instrumentation inside the L1 store. -**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates every L1 hit's latency, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. +**Sampling.** An SDK MAY time a sample of calls rather than every call, so that two clock reads do not add measurably to a sub-microsecond hit. It MUSTAPI-81 decide whether to time a call before it knows the call's outcome, by an independent pseudo-random draw for each call with probability 1/N, where N is a whole number from 1 to 1,000,000 that it fixes for the backend instance's lifetime and sends as `X-CacheKit-L1-Latency-One-In` (N = 1 times every call); never by the key, the arguments, the call's position in a sequence, the outcome or the duration. A counter that times every Nth call does not qualify: a caller that reads key A and then key B on every request would have only A's hits timed. A call chosen for timing that is not an L1 hit is discarded. Every report carries N, so a reader weights each report by N: summing buckets and sums multiplied by N estimates the latency of every L1 hit of 60 s or less, and SDKs with different N pool without the one that times more calls dominating. The bucket counts say how many hits a report covers, so a reader divides by their sum, never by `X-CacheKit-L1-Hits`. **Clock.** An SDK MUST NOTAPI-82 send the latency headers unless it timed with a monotonic clock that advances during synchronous execution and resolves 100 ns or finer. These clocks qualify: @@ -581,7 +581,7 @@ Per-report values were chosen over the session counters' cumulative form for two **Encoding.** No mean or percentile goes on the wire, because neither can be added across reports. Each integer is `0`, or a digit `1`–`9` followed by at most 15 further ASCII digits, and is at most 9,007,199,254,740,991 (2^53 − 1, the largest integer a JavaScript number or an IEEE 754 double holds exactly). An SDK MUSTAPI-87 send values that meet this grammar and these rules: -- `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. A duration above 60,000,000,000 ns (60 s) is discarded, not counted: no L1 hit takes that long, and the bound gives the server's range check an upper limit. +- `X-CacheKit-L1-Latency-Buckets`: exactly sixteen integers separated by `,`, with no white space, at least one of them not 0. The i-th counts the timed hits whose duration *d*, in whole nanoseconds rounded down, falls in bucket i of the table. A duration above 60,000,000,000 ns (60 s) is discarded, not counted, so that the server's range check always has an upper limit. The latency data therefore describes L1 hits of 60 s or less. A longer one, such as a hit held up by a slow synchronous revalidation, is outside it, and no figure built from the data covers it. - `X-CacheKit-L1-Latency-Sum-Ns`: the sum of the same hits' durations, each in whole nanoseconds rounded down; at least 1. - `X-CacheKit-L1-Latency-One-In`: N, the backend instance's sampling denominator, from 1 to 1,000,000. From b82bb15cd8cb7fc2a87359480868ddcdb3fafa41 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 31 Oct 2026 22:00:36 +1100 Subject: [PATCH 7/7] fix(spec): scope the L1-Status billing rule to its value --- conformance/coverage.md | 2 +- spec/saas-api.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/conformance/coverage.md b/conformance/coverage.md index 9aefd65..5abf115 100644 --- a/conformance/coverage.md +++ b/conformance/coverage.md @@ -290,7 +290,7 @@ Marked not-a-requirement in `spec/intent-presets.md`: | [API-90](../spec/saas-api.md#api-90) | *Server handling*: … most the sum of each count times the bucket's bound (60,000,000,000 ns for bucket 16). The server **MUST NOT** answer a request with an error because an optional metrics header is missing or malformed, and SHOULD record a valid … | Binds the server.
**Gap:** As API-88, for a server that answers a request with an error because an optional metrics header is missing or malformed. | n/a | n/a | n/a | | [API-91](../spec/saas-api.md#api-91) | *Server handling*: … its response status, except `GET /v1/cache/health`. A server that records valid latency sets **MUST** keep or drop each one only by a random draw, whose probability may vary by tenant, time or load, for example to stay … | Binds the server.
**Gap:** No public vector exercises the server, so nothing here catches one that drops latency sets deterministically, such as once a write budget is spent, or keeps them without recording the probability it kept each one. | n/a | n/a | n/a | | [API-92](../spec/saas-api.md#api-92) | *Server handling*: … every other client together. An aggregate of latency sets across tenants, or one shown publicly, **MUST** weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. | Binds the server.
**Gap:** How a server's cross-tenant or public latency aggregate weights tenants is server code no public vector or SDK test can observe. | n/a | n/a | n/a | -| [API-93](../spec/saas-api.md#api-93) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting, or let `X-CacheKit-L1-Status` change billing: all of … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read, including whether billing reads X-CacheKit-L1-Status, is server code no public vector or SDK test can observe. | n/a | n/a | n/a | +| [API-93](../spec/saas-api.md#api-93) | *Server handling*: The server **MUST NOT** let an optional metrics header change billing or rate limiting, or let the value of `X-CacheKit-L1-Status` change … | Binds the server.
**Gap:** Which inputs a server's billing and rate-limit verdicts read, including whether billing reads X-CacheKit-L1-Status, is server code no public vector or SDK test can observe. | n/a | n/a | n/a | | [API-94](../spec/saas-api.md#api-94) | *Telemetry switch*: Every SDK **MUST** let its user turn telemetry off through both of these controls, which mean the same in every SDK: | **Gap:** No vector reaches an SDK's configuration, so nothing here catches a client that lacks the telemetry setting, defaults it to true or false, or ignores CACHEKIT_TELEMETRY. | gap | gap | gap | | [API-95](../spec/saas-api.md#api-95) | *Telemetry switch*: An SDK **MUST** read both controls once, when it constructs the backend, and resolve them exactly as below, for that backend … | **Gap:** As API-94, for a client that lets a configuration loader fill the setting from the environment, lets on win over off, reads an unrecognised value as on, or re-reads the controls after construction. | gap | gap | gap | | [API-96](../spec/saas-api.md#api-96) | *Telemetry switch*: With telemetry off, an SDK **MUST NOT** send any optional metrics header on that backend's requests: `X-CacheKit-Session-ID`, `X-CacheKit-Session-Start`, … | **Gap:** No vector covers request headers, so nothing here catches a client that, with telemetry off, still sends a session or latency header. | gap | gap | gap | diff --git a/spec/saas-api.md b/spec/saas-api.md index 598d032..267f104 100644 --- a/spec/saas-api.md +++ b/spec/saas-api.md @@ -603,7 +603,7 @@ The server validates the latency headers as one set and the session headers as a Authentication proves who holds a key, not that its measurements are true: one set of sixteen grammatical, consistent values can claim more hits than every other client together. An aggregate of latency sets across tenants, or one shown publicly, MUSTAPI-92 weight each tenant equally or cap each tenant's share, and never sums raw sets across tenants. -The server MUST NOTAPI-93 let an optional metrics header change billing or rate limiting, or let `X-CacheKit-L1-Status` change billing: all of them are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. `X-CacheKit-L1-Status` may change rate limiting, as [API-1](#api-1) describes; headers that carry an operation's own inputs, such as `X-CacheKit-TTL`, are outside this rule. +The server MUST NOTAPI-93 let an optional metrics header change billing or rate limiting, or let the value of `X-CacheKit-L1-Status` change billing: all of them are client-reported. Rate limiting means the admit-or-deny verdict on a request and the limit applied to it. `X-CacheKit-L1-Status` may change rate limiting, as [API-1](#api-1) describes; headers that carry an operation's own inputs, such as `X-CacheKit-TTL`, are outside this rule. #### Telemetry switch