From c00f41879bc51f95c456b596ef632c2a6114aaf7 Mon Sep 17 00:00:00 2001 From: Flotapponnier Date: Sat, 19 Sep 2026 23:39:57 +0200 Subject: [PATCH] hotfix(prod) bench 002 bridge-quote-latency: the headline was a random Prometheus cell (queries filtered on bridge only over a rule that keeps chain, region and amount; prom.scalar took result[0]); every query now aggregates with sum by (le) / sum / avg, region pinned to EU-West, meme route and execution-path quotes excluded in the rules (cherry-pick of dcd91e85, spec and rules only) Production read Mobula p50 31 / p90 57 / p99 4172 ms and Near Intents p99 108 below p90 4700 on 2026-09-19: a different cell on every render. Verified on the VPS Prometheus after the change: one series, Mobula 43.7 / 345.8 / 982.6 ms, mean 118.3 (the previous mean query floored a per-second rate to 1 and printed ms per second). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HJgbZCqjR4nvCfcJSzofbw --- benchmarks/bridge-quote-latency.yml | 134 +++++++++--------- .../recording_rules/ocb_bridge_quote.yml | 32 +++++ 2 files changed, 99 insertions(+), 67 deletions(-) create mode 100644 infrastructure/monitoring/prometheus/recording_rules/ocb_bridge_quote.yml diff --git a/benchmarks/bridge-quote-latency.yml b/benchmarks/bridge-quote-latency.yml index 9de6245f3..a177381be 100644 --- a/benchmarks/bridge-quote-latency.yml +++ b/benchmarks/bridge-quote-latency.yml @@ -43,9 +43,9 @@ seo_intro: | number every public roundup ignores. how many milliseconds the API burns before the user sees a price. We hammer the same USDC routes (Solana, Base, Arbitrum) against every bridge from the EU-West (Paris) - origin, every five minutes, on three notional sizes. The US-East and - Singapore origins are temporarily paused, so the live numbers reflect - EU-West only. p50, p90 and p99 are computed via Prometheus + origin, every five minutes, on three notional sizes. The headline is + pinned to EU-West (a US-East probe also runs and feeds the region + breakdown; Singapore is down). p50, p90 and p99 are computed via Prometheus `histogram_quantile` over the 24-hour window, so the leaderboard reflects sustained API performance, not a lucky cold-start. The same harness records failure mode for every request (quote_failed, @@ -65,7 +65,7 @@ methodology: - "Routes: USDC pairs spanning Solana, Base and Arbitrum." - "Notional sizes: $5, $50, $300 per quote." - "Cadence: full sweep (4 routes × 3 amounts × N bridges) every 5 minutes for 24 hours." - - "Regions: eu-west only at present. The us-east and sgp probes are temporarily paused, so the live figures reflect the EU-West (Paris) origin; the harness is built to probe three origins simultaneously and the other two tabs return when those probes resume." + - "Regions: every figure on this page is pinned to the EU-West (Paris) origin. A us-east probe also runs and its cells stay in Prometheus for the region breakdown, but they are not pooled into the headline; the sgp probe is down. Execution-path quotes ($3 and $30 tickets) and the R4 meme route quoted by the same loop are excluded from every cell." - "Histogram buckets: 10, 25, 50, 100, 200, 500, 1000, 2000, 5000, 10000 ms. Sub-50ms buckets added 2026-07-13 so solver-cached fast paths (Near Intents on HyperCore in particular) surface their true p50 instead of being floored at the interpolation of the (0, 50] bucket." - "Failures (quote_failed, execution_failed, unsupported route) excluded from latency aggregates and counted toward success rate." @@ -122,22 +122,22 @@ providers: tag: Aggregator + intent layer formula: "Median wall-clock ms over 24h for Mobula's quote API to return a usable USDC route, sampled every 5 minutes across 4 routes and 3 notionals from the EU-West (Paris) origin." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="mobula"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="mobula"} - success: ocb:bridge_quote_success:avg24h{bridge="mobula"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="mobula"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="mobula"}[1h]))) regions: - region: eu-west - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="eu-west"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="mobula", region="eu-west"}[1h]))) - region: us-east - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="us-east"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="us-east", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="mobula", region="us-east"}[1h]))) - region: sgp - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="sgp"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="mobula", region="sgp", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="mobula", region="sgp"}[1h]))) - slug: relay @@ -145,22 +145,22 @@ providers: tag: Cross-chain relay formula: "Median wall-clock ms over 24h for Relay's solver-quote API to return a usable USDC route, sampled every 5 minutes across 4 routes and 3 notionals from the EU-West (Paris) origin." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="relay"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="relay"} - success: ocb:bridge_quote_success:avg24h{bridge="relay"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="relay"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="relay"}[1h]))) regions: - region: eu-west - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="eu-west"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="relay", region="eu-west"}[1h]))) - region: us-east - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="us-east"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="us-east", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="relay", region="us-east"}[1h]))) - region: sgp - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="sgp"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="relay", region="sgp", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="relay", region="sgp"}[1h]))) - slug: lifi @@ -168,22 +168,22 @@ providers: tag: Aggregator formula: "Median wall-clock ms over 24h for LI.FI's route-search API to return a usable USDC route across underlying bridges, sampled every 5 minutes across 4 routes and 3 notionals from the EU-West (Paris) origin." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="lifi"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="lifi"} - success: ocb:bridge_quote_success:avg24h{bridge="lifi"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="lifi"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="lifi"}[1h]))) regions: - region: eu-west - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="eu-west"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="lifi", region="eu-west"}[1h]))) - region: us-east - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="us-east"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="us-east", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="lifi", region="us-east"}[1h]))) - region: sgp - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="sgp"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="lifi", region="sgp", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="lifi", region="sgp"}[1h]))) - slug: debridge @@ -191,22 +191,22 @@ providers: tag: Native-token fees formula: "Median wall-clock ms over 24h for deBridge's direct-protocol quote API to return a single-route USDC quote, sampled every 5 minutes across 4 routes and 3 notionals from the EU-West (Paris) origin." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="debridge"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="debridge"} - success: ocb:bridge_quote_success:avg24h{bridge="debridge"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="debridge"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="debridge"}[1h]))) regions: - region: eu-west - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="eu-west"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="debridge", region="eu-west"}[1h]))) - region: us-east - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="us-east"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="us-east", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="debridge", region="us-east"}[1h]))) - region: sgp - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="sgp"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="debridge", region="sgp", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="debridge", region="sgp"}[1h]))) - slug: near-intents @@ -214,22 +214,22 @@ providers: tag: Intent layer (NEAR) formula: "Median wall-clock ms over 24h for Near Intents 1Click API to return a usable USDC quote via the solver auction bus, sampled every 5 minutes across the supported routes and 3 notionals from the EU-West (Paris) origin." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="near-intents"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="near-intents"} - success: ocb:bridge_quote_success:avg24h{bridge="near-intents"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="near-intents"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="near-intents"}[1h]))) regions: - region: eu-west - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="eu-west"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="near-intents", region="eu-west"}[1h]))) - region: us-east - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="us-east"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="us-east", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="near-intents", region="us-east"}[1h]))) - region: sgp - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="sgp"}) + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="near-intents", region="sgp", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="near-intents", region="sgp"}[1h]))) - slug: across @@ -237,12 +237,12 @@ providers: tag: Intent layer formula: "Median wall-clock ms over 24h for Across's Swap API to return a usable USDC quote across supported corridors (Sol to Base, Base to Arb) from the EU-West origin. Arb to Sol excluded: no destination swaps to Solana." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="across"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="across"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="across"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="across"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="across"} - success: ocb:bridge_quote_success:avg24h{bridge="across"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="across"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="across", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="across"}[1h]))) - slug: squid @@ -250,12 +250,12 @@ providers: tag: Aggregator (Axelar) formula: "Median wall-clock ms for Squid Router's v2 API to return a best-output USDC route, p50 over 24h, EVM corridors only (Base to Arb). Solana source excluded: API returns server error." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="squid"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="squid"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="squid"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="squid"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="squid"} - success: ocb:bridge_quote_success:avg24h{bridge="squid"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="squid"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="squid", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="squid"}[1h]))) - slug: socket @@ -263,10 +263,10 @@ providers: tag: Aggregator formula: "Median wall-clock ms for Socket Protocol's v2 API to return a best-output USDC route (via Across, Stargate, CCTP), p50 over 24h, EVM corridors only (Base to Arb)." queries: - p50: histogram_quantile(0.50, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="socket"}) - p90: histogram_quantile(0.90, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="socket"}) - p99: histogram_quantile(0.99, ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="socket"}) - mean: ocb:bridge_quote_latency_ms_sum:rate24h{bridge="socket"} / ocb:bridge_quote_latency_ms_count:rate24h{bridge="socket"} - success: ocb:bridge_quote_success:avg24h{bridge="socket"} - sample_size: ocb:bridge_quote_latency_ms_count:increase24h{bridge="socket"} + p50: histogram_quantile(0.50, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"})) + p90: histogram_quantile(0.90, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"})) + p99: histogram_quantile(0.99, sum by (le) (ocb:bridge_quote_latency_ms_bucket:rate24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"})) + mean: sum(ocb:bridge_quote_latency_ms_sum:rate24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"}) / clamp_min(sum(ocb:bridge_quote_latency_ms_count:rate24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"}), 1e-9) + success: avg(ocb:bridge_quote_success:avg24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"}) + sample_size: round(sum(ocb:bridge_quote_latency_ms_count:increase24h{bridge="socket", region="eu-west", amount_usd=~"5|50|300"})) series: histogram_quantile(0.50, sum by (le) (rate(bridge_quote_latency_ms_bucket{bridge="socket"}[1h]))) diff --git a/infrastructure/monitoring/prometheus/recording_rules/ocb_bridge_quote.yml b/infrastructure/monitoring/prometheus/recording_rules/ocb_bridge_quote.yml new file mode 100644 index 000000000..e16df402f --- /dev/null +++ b/infrastructure/monitoring/prometheus/recording_rules/ocb_bridge_quote.yml @@ -0,0 +1,32 @@ +# Recording rules for the bridge QUOTE benches (002 bridge-quote-latency, +# 003 bridge-fee reads raw gauges). Deployed on the VPS Prometheus as +# /opt/ocb/prom-rules/histogram_recording_rules.yml (the file was VPS-only +# until 2026-09-19; this is the source of truth now). +# +# The rules keep (bridge, region, chain, amount_usd) so the chain tabs can +# filter; the SPEC queries must aggregate with sum by (le) / sum() / avg() +# over what is left, or prom.scalar takes result[0] and the headline is a +# random cell (audit 2026-09-19: p99 below p50 on the live page). +# +# Excluded at the source: +# - the R4 meme route (TRUMP on Solana to BRETT on Base) quoted by the +# same loop, which pooled a cross-asset swap into the USDC corridors; +# - execution-path quotes, which the harness now observes in +# bridge_exec_quote_latency_ms (amount_usd 3 and 30 kept out here for +# the samples recorded before that change). +groups: + - name: bridge_quote_recording + interval: 15m + rules: + - record: ocb:bridge_quote_latency_ms_bucket:rate24h + expr: sum by (bridge, region, chain, amount_usd, le) (rate(bridge_quote_latency_ms_bucket{from_token!="6p6xgHyF7AeE6TZkSmFsko444wqoP15icUSqi2jfGiPN", amount_usd!~"3|30"}[24h])) + - record: ocb:bridge_quote_latency_ms_sum:rate24h + expr: sum by (bridge, region, chain, amount_usd) (rate(bridge_quote_latency_ms_sum{from_token!="6p6xgHyF7AeE6TZkSmFsko444wqoP15icUSqi2jfGiPN", amount_usd!~"3|30"}[24h])) + - record: ocb:bridge_quote_latency_ms_count:rate24h + expr: sum by (bridge, region, chain, amount_usd) (rate(bridge_quote_latency_ms_count{from_token!="6p6xgHyF7AeE6TZkSmFsko444wqoP15icUSqi2jfGiPN", amount_usd!~"3|30"}[24h])) + - record: ocb:bridge_quote_latency_ms_count:increase24h + expr: sum by (bridge, region, chain, amount_usd) (increase(bridge_quote_latency_ms_count{from_token!="6p6xgHyF7AeE6TZkSmFsko444wqoP15icUSqi2jfGiPN", amount_usd!~"3|30"}[24h])) + - record: ocb:bridge_quote_success:avg24h + expr: avg by (bridge, region, chain, amount_usd) (avg_over_time(bridge_quote_success{from_token!="6p6xgHyF7AeE6TZkSmFsko444wqoP15icUSqi2jfGiPN", amount_usd!~"3|30"}[24h])) + - record: ocb:bridge_quote_latency_ms_bucket:rate24h_provider + expr: sum by (provider, region, chain, amount_usd, le) (label_replace(rate(bridge_quote_latency_ms_bucket{from_token!="6p6xgHyF7AeE6TZkSmFsko444wqoP15icUSqi2jfGiPN", amount_usd!~"3|30"}[24h]), "provider", "$1", "bridge", "(.+)"))