[GordonYuanyc]

Closes #643. Closes #644.

Why

ASAPPlanner pins asap_sketchlib at 3f44cf1c. asap_sketchlib main (de15d66,
ProjectASAP/asap_sketchlib#242) removes the types the executor's kernels wrap:
the portable HllSketch, KllSketch, DdSketch, CountMinSketch and
HydraKllSketch, and WeightedFrequency. It also renames the UnivMon, Hydra
and HLL APIs. Staying on the old pin keeps the Planner on wrapper types that
no longer exist upstream.

What

• The workspace pins asap_sketchlib main, rev = "de15d664264a628e63bd5a67494aa53a62791e9f".
• HLL, KLL, DDSketch, Count-Min and KMV kernels run on the native sketchlib
  types (Use the asap_sketchlib sketch types #643).
• The keyed top-k state (WeightedFrequency) is now a Planner type over f64
  CountMinSketchWithHeap / CountSketchWithHeap, or plain CountMinSketch /
  CountSketch at capacity 0. The weight and identity checks live in Planner.
• HydraKLL is a Hydra with one key column over HydraCounter::Kll cells (HydraKLL on asap_sketchlib Hydra #644).
• Planner's sizing is unchanged: Count-Min depth up to 32, Count Sketch up to
  255, UnivMon L2 up to 19 rows. sketchlib rows past 20 now hash independently,
  and UnivMon accepts any row count that fits its 128-bit key hash.

Results change where the native sketches differ from the wrappers they replace:

	Before	After
Plain Count-Min weight ≤ 0	skipped	added (a negative weight subtracts)
Plain Count-Min hashing	FastPath	regular path, one seed per row
DDSketch quantile q of n values (quantile_over_time and friends)	0-based order statistic ⌊q(n−1)⌋	get_value_at_quantile(q): rank ⌈q·n⌉, as KLL already reads; q = 0 and q = 1 give the exact min and max. Neither interpolates
HLL cardinality	f64 estimate (HllVariant::Regular)	classic estimator, rounded down to an integer
KLL memory charge	fixed 4608 + 32k bytes (11,008 at k = 200)	Kll::memory_bytes, which grows as items arrive
Plain Count-Min memory charge	flat 16 KiB	CountMinSketch::memory_bytes (struct + rows × cols × 8)
HydraKLL memory charge	flat 32 KiB (state), rows × cols × 4096 (updater)	struct + every cell's Kll::memory_bytes
KMV memory charge	KMV::approx_memory_bytes	Kmv::memory_bytes
Top-k heap retention	every update rescored all candidates and kept the top capacity	sketchlib's heap: an update rescores only its own key, and a new key replaces the smallest held count when its estimate is larger; merge rescores both heaps' keys against the merged matrix. rows() reads every held key at its current estimate, as before
Top-k row order on equal estimates	by the identity's bincode bytes	by its TupleKey bytes
Weighted frequency overflow	an update or merge refused when a counter would leave f64	refused when the running Σ|weight| would pass f64::MAX, so Count Sketch now refuses large cancelling weights it used to accept
HydraKLL cells	HydraKllSketch's own hashing	Hydra's shared seed list, so keys land in other cells, within the same error bounds

How

• HLL. sketchlib fixes the precision in the register type, so
  hll_sketch.rs declares one HyperLogLogImpl<Classic, HllRegistersP{p}> per
  precision 4..=18 with impl_hll_registers!. A private Registers enum holds
  whichever one was built. States at different precisions do not merge.
• Weighted frequency. An identity tuple is encoded as one TupleKey
  (-0.0 and 0.0 encode the same), hashed with DataInput::Bytes, and the
  heap keeps those bytes. Planner checks what sketchlib does not: finite
  weights (nonnegative for Count-Min), finite Float64 identities, an odd Count
  Sketch depth, and a total weight magnitude that stays within f64. Shapes are
  checked with CountMinSketch::check_shape / CountSketch::check_shape.
• HydraKLL. Hydra::with_schema(rows, cols, ["key"], HydraCounter::Kll(..)), as HydraKLL on asap_sketchlib Hydra #644 names it.
  The key's joined label values are the single key column. A quantile is the
  median over rows of the cells' KLL quantiles.
• The keyed group-state template GroupStates::Keyed::empty is boxed because
  the heap makes it much larger than the other group-state kinds (clippy
  large_enum_variant).

Key code interfaces added / modified

Links are at the branch head be071a6 and resolve once it is pushed.

crates/executor/src/summary_kernels/weighted_frequency.rs

/// Which matrix a weighted frequency state keeps. The type stays; it is now
/// declared here.
pub enum FrequencyAlgorithm {      // was the re-export asap_sketchlib::FrequencyAlgorithm
    /// Count-Min: nonnegative weights, minimum over rows.
    Cms,                           // unchanged
    /// Count Sketch: signed weights, median over an odd number of rows.
    CountSketch,                   // unchanged
}

/// The shapes a state is built with: positive width and depth, odd for Count
/// Sketch. Capability checks call it before a state is built.
pub fn check_shape(algorithm: FrequencyAlgorithm, width: usize, depth: usize) -> Result<(), Error>; // new

/// Keyed top-k (or, at capacity 0, plain) Float64 frequency state. The type
/// stays; it no longer derives Serialize / Deserialize.
#[derive(Clone)]                   // was Clone, Debug, Serialize, Deserialize
pub struct WeightedFrequency { /* private fields */ }

impl WeightedFrequency {
    /// The state's algorithm.
    pub fn algorithm(&self) -> FrequencyAlgorithm;                       // unchanged
    /// `(width, depth, capacity)`, compared against a declared family.
    pub fn shape(&self) -> (usize, usize, usize);                        // unchanged
    /// Builds an empty state; fails for a shape `check_shape` refuses.
    pub fn new(algorithm: FrequencyAlgorithm, width: usize, depth: usize, capacity: usize) -> Result<Self, Error>; // unchanged
    /// Adds `weight` to an identity tuple; an error, and the state unchanged,
    /// for a refused weight or identity, or a total weight past f64::MAX.
    pub fn update(&mut self, values: &[Value], weight: f64) -> Result<(), Error>; // unchanged
    /// Point estimate for one identity tuple.
    pub fn point_estimate(&self, values: &[Value]) -> Result<f64, Error>; // unchanged
    /// Up to `n` held identity tuples, each followed by its current estimate.
    pub fn rows(&self, n: usize) -> Vec<Vec<Value>>;                     // unchanged
    /// The depth × width counters, row by row (tests and diagnostics).
    pub fn counters(&self) -> &[f64];                                    // new
    /// Merges two states of one algorithm and shape, leaving both unchanged.
    pub fn merge(&self, other: &Self) -> Result<Self, Error>;            // new (was only AggregateCore::merge_with)
}

crates/executor/src/summary_kernels/hll_sketch.rs

/// The precisions an HLL state can be built at; the capability check reads it.
pub const HLL_PRECISIONS: std::ops::RangeInclusive<u8> = 4..=18;    // new

/// HyperLogLog distinct-count state. The type stays.
pub struct HllSketchAccumulator {
    /// The registers at the built precision.
    inner: Registers,              // was `pub inner: asap_sketchlib::HllSketch`
}

impl HllSketchAccumulator {
    /// Builds an empty state; fails for a precision outside HLL_PRECISIONS.
    pub fn new(precision: u8) -> Result<Self, KernelError>;  // was new(variant: HllVariant, precision: u32) -> Self
    /// The built precision.
    pub fn precision(&self) -> u8;                           // new
    /// Observes one item, hashed by its bytes.
    pub fn update(&mut self, item: &[u8]);                   // new
    /// One rank per register.
    pub fn registers(&self) -> &[u8];                        // new
    /// Empties the state in place, keeping its precision.
    pub fn clear(&mut self);                                 // new
}

crates/executor/src/summary_kernels/count_min_sketch.rs,
crates/executor/src/summary_kernels/factory.rs

/// Plain Count-Min state. The type stays.
pub struct CountMinSketchAccumulator {
    /// The f64 regular-path matrix.
    pub inner: asap_sketchlib::CountMinF64,  // was asap_sketchlib::CountMinSketch (portable)
}

impl CountMinSketchAccumulator {
    /// Builds an empty state; fails for a zero dimension.
    pub fn new(rows: usize, cols: usize) -> Result<Self, KernelError>; // was new(row_num, col_num) -> Self
    /// Adds `weight` to `item`, whatever its sign.
    pub fn update(&mut self, item: &str, weight: f64);                // new
    /// Estimated frequency of one item.
    pub fn query_key(&self, key: &KeyByLabelValues) -> f64;           // unchanged
}

impl CmsAccumulatorUpdater {
    /// The factory's Count-Min updater; fails for a shape the state refuses.
    pub fn new(rows: usize, cols: usize) -> Result<Self, String>;     // was new(row_num, col_num) -> Self
}

crates/executor/src/summary_kernels/hydra_kll.rs,
datasketches_kll.rs,
dd_sketch.rs,
kmv.rs

/// HydraKLL state: per-key quantiles in one shared grid. The type stays.
pub struct HydraKllSketchAccumulator {
    /// A rows × cols Hydra with one key column and KLL cells.
    pub inner: asap_sketchlib::Hydra,        // was asap_sketchlib::HydraKllSketch
}
impl HydraKllSketchAccumulator {
    /// Builds an empty grid of `k`-parameter KLL cells.
    pub fn new(rows: usize, cols: usize, k: u16) -> Self; // unchanged (parameters were row_num, col_num)
}

/// KLL state. The type stays.
pub struct DatasketchesKLLAccumulator {
    /// The native KLL sketch.
    pub inner: asap_sketchlib::Kll,          // was asap_sketchlib::KllSketch
}

/// DDSketch state. The type stays.
pub struct DDSketchAccumulator {
    /// The native DDSketch.
    pub inner: asap_sketchlib::DdSketch,     // was the portable asap_sketchlib::DdSketch
}

/// KMV state. The type stays.
pub struct KmvAccumulator {
    /// The native KMV sketch.
    pub inner: asap_sketchlib::Kmv,          // was asap_sketchlib::KMV
}

Usage. For topk(2, sum by (job) (rate(data_total[5m]))) with
data_total declared a counter, plan selection picks a Count-Min-with-heap
candidate. The executor's keyed build (StateColumnUpdater) then makes one
WeightedFrequency per group. These numbers come from running the code at
be071a6 in a scratch test:

// The state the keyed build makes for CmsWithHeap { width: 272, depth: 5, heap_size: 4 }.
let mut state = WeightedFrequency::new(FrequencyAlgorithm::Cms, 272, 5, 4)?;
// Each sample's `job` label is the identity; its reset-corrected rate is the weight.
state.update(&[Value::Utf8("api".into())], 3.5)?;
state.update(&[Value::Utf8("db".into())], 1.25)?;
state.update(&[Value::Utf8("api".into())], 0.5)?;
// What topk reads: [["api", 4.0], ["db", 1.25]].
let top = state.rows(2);
// A negative weight is refused for Count-Min and leaves the state unchanged:
// Err(Operator("weighted frequency requires finite weights; CMS additionally requires nonnegative weights")).
let refused = state.update(&[Value::Utf8("api".into())], -1.0);
// What the memory limit charges (AggregateCore::approx_memory_bytes): 12,055 bytes,
// 11,360 for the same shape at capacity 0.
let charged = state.approx_memory_bytes();
// Planner's Count-Min sizing goes up to depth 32, past ASAPv1's 20 rows; such a
// state builds, updates and reads, since nothing encodes it.
let deep = WeightedFrequency::new(FrequencyAlgorithm::Cms, 272, 32, 4)?;

Before this PR

The executor wrapped asap_sketchlib's portable types and WeightedFrequency.
None of them exist on asap_sketchlib main, so the Planner could not move its
pin past 3f44cf1c.

After this PR

The Planner builds and tests against asap_sketchlib main. Every kernel runs on
a native sketchlib type, and the keyed top-k state's weight rules are Planner
code. For four values 1, 2, 3, 4 at α = 0.01, the DDSketch kernel now returns
1, ~1.994, 4, 4 at q = 0, 0.5, 0.9, 1 (dd_sketch::quantile_reads_rank_ceil_q_n).
Before, q = 0.9 read the third value. An HLL at precision 12 over 1,000
distinct items reads 998, an integer (hll_sketch::cardinality_is_an_integer).

Evidence

• Execution example: the Usage block's numbers come from a scratch test run
  at be071a6. The DDSketch and HLL figures above are asserted by the tests
  they name.
• Screenshots, performance measurements: not applicable.

Verification

• cargo fmt --all -- --check: clean.
• cargo clippy --workspace --all-targets --all-features --locked -- -D warnings: clean.
• cargo check -p asap-planner --tests --locked: clean.
• cargo test --workspace --locked --no-fail-fast: 1,772 passed, 0 failed, 34 ignored.
• tools/verify_metricsql_external_consumer.sh, tools/verify_metricsql_vendored_baseline.py: pass.

New tests:

• count_min_sketch::non_positive_weights_are_added: a zero weight adds nothing and a negative one subtracts. It fails with the old skip (5.0, expected 3.0).
• hll_sketch::precision_range_and_mismatched_merge: precisions 4..=18 build 2^p registers, 3 and 19 fail, and different precisions do not merge.
• hydra_kll::quantiles_per_key_survive_merge: a key's quantile reads its own values, and a merge keeps both inputs'.
• count_min_sketch::merge_rejects_a_different_shape: states with different rows or columns do not merge.
• datasketches_kll::merge_rejects_a_different_k: KLL states with different k do not merge.
• dd_sketch::quantile_reads_rank_ceil_q_n: the quantile at q reads rank ⌈q·n⌉, and q = 0 / q = 1 read the exact min / max (1, ~1.994, 4, 4 for 1..4).
• hll_sketch::cardinality_is_an_integer: the cardinality has no fractional part (998 for 1,000 items at p = 12).
• weighted_frequency::{signed_weights_and_typed_identities, invalid_updates_leave_the_state_unchanged, signed_zero_and_shape_rules}: Count Sketch ranks signed estimates and typed identities do not collide; refused weights and identities leave the state unchanged; signed zeros are one identity; even Count Sketch depths and zero dimensions are refused; depth 32 Count-Min and depth 255 Count Sketch build.
• weighted_frequency::deep_states_update_merge_and_read: Count-Min with a heap at 272×32×4 and Count Sketch at 64×21 (with and without a heap), all deeper than ASAPv1's 20 rows, build, update, merge and read.

Changed expectations:

• executor::univmon::adoption_rejects_terminal_or_invalid_sketches: the misshapen state claims 26 rows of 16 columns (130 hash bits), was 21 rows. sketchlib now builds 21.
• estimators::univmon::univmon_past_twenty_rows_is_not_hash_independent → univmon_rows_are_bounded_by_the_hash_budget: 21 rows × 32 columns (126 bits) is hash independent with an L2 guarantee; 22 × 32 (132 bits) is not. It was "21 rows is never independent".
• stored_pane_memory_limit: each state takes 1,100 samples, was 1. Native KLL allocates its buffer as items arrive, so it needs those samples to be charged about 11 kB again. The limits and assertions are unchanged.
• promql_numeric_regressions::checked_ratio_must_not_certify_cross_zero_interpolation: values [-2, -1, 1.011, 2], was [-1, 1.011]. The native sketch reads min and max exactly, so the counterexample has to interpolate interior order statistics. The test's reference PromQL interpolation is computed by a test helper from the sketch's order statistics, because the portable quantile_interpolated is gone. The executor itself does not interpolate; it answers quantile_over_time with get_value_at_quantile(q), rank ⌈q·n⌉, as it does for KLL. The assertion is unchanged.
• promql_to_post_asap::ddsketch_ratio_bound_holds_for_signed_pinned_sketch_evaluations: same interpolation helper; the bound is unchanged.
• state_column_updater and the row-reference harness: a weighted frequency state's fingerprint is its counters plus rows(usize::MAX) (held keys at current estimates). It was the serde JSON of the old state, which also held each candidate's stored score. Stored heap counts are no longer compared; counters and every held key's current estimate still are.

Architectural decisions

• WeightedFrequency stays a Planner type over sketchlib matrices and heaps.
  Its weight and identity rules belong to the executor's input contract;
  sketchlib only provides the sketches.
• HLL picks a register type per precision through a macro-generated enum. A
  runtime-precision HLL would have to be a new sketchlib type.

Limitations and follow-up

• Top-k retention follows sketchlib's heap: a held key's stored count is
  refreshed only when that key is updated or the state is merged. Those stale
  counts decide which key is evicted; rows() rescores every held key, so the
  estimates it returns are current.
• The DDSketch and KLL relative-α guarantee covers the value at one rank, not
  PromQL's interpolated quantile. Measured on
  [1, 1, 1, 30, 100] at α = 0.0049, q0.9 / q0.5 = 99.1 against the exact
  interpolated 72. The old ⌊q(n−1)⌋ rule gave 30, so the gap predates this PR.
• Keyed top-k memory charging calls approx_memory_bytes twice per keyed
  update, and each call walks the heap, O(heap).
• HydraKLL's approx_memory_bytes walks every cell, O(rows · cols).
• Weighted frequency states deeper than 20 rows are never ASAPv1-encoded here;
  sketchlib's to_asapv1 refuses them.
• HydraCMS cells stay CountMinSketch<Vector2D<i32>, FastPath>, unchanged.

Human review — do not complete with an agent

• The MVP boundary is correct.
• New conceptual layers or public interfaces are necessary.
• The before/after description matches the intended product behavior.
• Human reviewer:
• Decision and rationale:

🤖 Generated with Claude Code

[zzylol]

I added four commits on top of this branch so that it also covers #785. None of the existing commits were changed.

1. 3eb91cdc: merges origin/main (feat: declare non-negative gauges so per-series quantile sketches compose through avg (#772) #780, feat(planner): make the SQL and MetricsQL frontends optional cargo features #782, feat: InstallablePlan describes every stored summary through its SummaryCoverage (#781) #787). There were no conflicts.
2. 2b564237: caps UnivMon at 20 rows in the executor's builder, adoption and capability check, and in the Planner's hash_independent. Main's UnivMon::check_shape accepts 21 rows, but the ASAPv1 encoder refuses more than 20, so a deployment could never store such a state.
3. 201febfd: KLL and weighted-frequency memory charges counted the struct twice: size_of::<Self>() plus memory_bytes(), which already includes it. A new test pins each charge to the library's count.
4. 11535835: moves the pin from de15d66 to the 0.4.0 release commit eb04e7f. The library code is the same; only the version, CHANGELOG and README differ.

cargo test --workspace --locked: 1,787 passed, 0 failed, 34 ignored.