fix(query-engine): range queries expand keys_query and merge same-timestamp buckets - #582
Conversation
…estamp buckets (#580) execute_range_query_pipeline never read keys_query, so dual-population metrics returned nothing over a range. It also collapsed same-start- timestamp buckets in a HashMap, dropping all but one when a Sliding aggregation legitimately returns more than one bucket per window (#567/#570). Mirrors the keys fetch/merge/expand pattern already used by execute_and_merge_store_queries/collect_results_separate_keys.
|
Code review findings on
|
…fetch/merge (#582 review) - execute_range_query_pipeline now iterates the merged keys map (like collect_results_separate_keys) and fails the whole query on an unresolvable key set, instead of silently continue-ing past it. - Extracted fetch_and_merge_keys, shared by the instant and range paths, and pass the real do_merge flag through instead of hardcoding true for range. - merge_precomputed_outputs's spatial (non-merge) branch now warns and merges instead of asserting exactly 1 precompute per key, since do_merge=false no longer guarantees that under a range query's widened fetch (and won't once Sliding needs merging there too).
|
Follow-up finding on the latest push (
This causes two different failure modes:
None of the new tests in |
|
Correction on the DeltaSetAggregator part of the finding above. `merge_accumulators` in `delta_set_aggregator_accumulator.rs` does correctly reconcile add/remove deltas across `[0, T]` — a key that's added then removed cancels out. So it's not "picks up every key that ever existed" as I said. The real trigger is simpler: `get_keys()` (delta_set_aggregator_accumulator.rs:251-258) returns `None` for the entire accumulator if `removed` is non-empty post-merge, instead of returning `added.difference(&removed)`. Since `removed` is non-empty as soon as any key has ever been removed across the metric's full history, this fires on ordinary label churn, not just stale/ancient keys. Pre-existing (since #342), not introduced by this PR — but #582's new hard-fail on missing keys (mod.rs:1547) turns this common case into a full range-query failure instead of the previous silent skip. |
Summary
execute_range_query_pipelinenever readkeys_query, so dual-population metrics (separate value/key aggregations) returned an empty result over a range instead of the expanded key set.HashMap<u64, _>, silently dropping all but one when a Sliding aggregation legitimately returns more than one bucket per window (asap-query-engine: convert binary PromQL instant queries from DataFusion to native execution #567/fix(query-engine): merge all sliding-window buckets per key instead of taking first #570).execute_and_merge_store_queries/collect_results_separate_keys).Fixes #580.
Test plan
native_range_query_tests.rs: RED before the fix, GREEN after — dual-population single-step, dual-population multi-step, sliding 2-bucket collision, sliding 3-bucket collision, sliding single-bucket regression guard.cargo test -p query_engine_rust --lib— 552 passed, 0 failed.cargo clippy -p query_engine_rust --lib --tests— clean.