feat(asap-tools): pooled and dashboard-style random workloads on ASAPQuery's RQE frontend

[milindsrivastava1997]

Stacked on #907 (batch driver, collation and plots, time-ordered raw ClickHouse, dashboards K, Alibaba replay, x2 variant). It merges main for the RQE stack (#887–890). Review the commits from 556e2cd on; the earlier ones are #907's.

Summary

• Pooled random workloads (random_workloads.py --pooled, set seed-862-pooled): the same random choices as the base set.
  • Quantiles are pooled: one quantile over all of a group's samples.
  • Sum top-k becomes topk(k, sum by (region, service, job) (...)).
  • PromQL can't express a pooled quantile, so Prometheus/VictoriaMetrics run the stand-in quantile by (G) (phi, quantile_over_time(phi, data[W])).
  • ClickHouse (raw and MV) computes the pooled quantile exactly from the RQE.
• Dashboard-style workloads (--dashboard, set seed-862-dashboard, the final eval template). Ten pooled queries per workload:
  • a sum roll-up chain at every grouping;
  • a p50/p90/p99 panel by one grouping;
  • pooled quantiles at the other two groupings;
  • pooled top-k by service and by microservice.
  • Windows run from 1 s to 20 m. No 1 s sums: PromQL's closed window counts one more edge sample. No 1 s quantiles by microservice: below the accuracy curves' 1,000 samples.
• ASAPQuery on its RQE frontend: a pooled workload's e2e config sets query_engine.frontend=rqe with a side file mapping each query's text to its tree (e2e.rqe_exprs.json). Prometheus runs the text. sketch_bench_e2e_configs.py reads a tree-only query's text from client_promql.
• Validity (collate_batch.py): in a pooled or dashboard set, quantiles are checked against raw ClickHouse's exact pooled answers. Prometheus/VictoriaMetrics stand-ins are left unchecked.
• Batch driver: the Prometheus arm forces query_engine.frontend=promql, since its config's sketchdb mode reads trees.
• Frozen sets with saved plans (expr-only, from eval: export pooled quantiles and top-k from random manifests sketch-bench#230): seed-862-pooled and seed-862-dashboard, 10 workloads each.

Results so far (pooled w01, synthetic, all 5 systems valid)

System	CPU-s	Peak mem	p95
ASAPQuery	1,296	0.35 GiB	0.05 s
ClickHouse+MV	1,868	2.64 GiB	0.30 s
raw ClickHouse	7,262	1.97 GiB	1.21 s
VictoriaMetrics	10,462	1.73 GiB	1.59 s
Prometheus	12,663	1.97 GiB	3.89 s

The PromQL stand-in is 11–304% off the exact pooled quantile; ASAPQuery's pooled DDSketch is about 5% off.

Validation

• python3 -m unittest discover -s tests -p "test_*.py" in asap-tools/experiments: 269 pass. New tests cover:
  • pooled manifests pairing with base;
  • pooled quantile and top-k SQL;
  • the dashboard template's rules;
  • pooled pilot configs (RQE frontend, side file, VictoriaMetrics without it);
  • checked-query filtering;
  • the Prometheus arm's frontend override.
• The prometheus-client tests pass (33). The merge with main resolved a conflict in main_prometheus_client.py: the RQE protocol and the pool size are both kept.
• Re-collating base-final-1015 under the new checks gives the same 50 valid runs and ratios.

🤖 Generated with Claude Code

https://claude.ai/code/session_015KzRxtgjFfFJeyDy8fN2CB