Repository navigation
Pass 2: window-composition rule (tumbling windows) — blocked on #511 #518
Description
Goal
Implement the window-composition rule of logical Pass 2 in #509 (docs/design_docs/proposals/planner-layering.md, Example 3 Pattern B), starting with tumbling windows. Computations with the same summary input data share one tumbling-window summary. Each query's window is answered by merging the tumbling windows it covers. The tumbling length must divide both the query window and the evaluation interval. For example, [5m] evaluated every 1m uses 1-minute KLLs and merges 5 of them.
Blocked on
#511 (unified operators). Today a summary's window lives inside its input (SummaryAgg { KeepPreAsap(TimeRange{w, …}) }), so the window is part of the state's identity. A clean window-composition rule needs KeepPreAsap removed first, which is part of the #511 implementation.
Current state on main (107ab28)
- The evaluation interval isn't visible to planning. It exists only in
RepeatedDemand(crates/types/src/workload.rs). Pass 1/2 (search_workload_with_targets) receive only (id, expr, accuracy). - The window framework (tumbling/sliding/EH) is always
Nonein production.selected_window_frameworkcomes only from provider evidence, which only tests register (summary_maintenance_cost/model.rs,bind_window_framework_candidate). - The planner never builds a
SummaryMerge. Its doc says "inserted by a deployment's stage allocator". The physical planner already compilesSummaryMergeas union + merge. - Queries with different windows never share. feat: share identical summary producers across queries after Pass 1 #515 shares only structurally identical
SummaryAggs, and the window is inside the child.different_producers_are_not_shared(crates/planner/tests/summary_sharing.rs) covers this. select_shared_panes(pane_sharing.rs) is dead code. It has no callers outside its own tests and duplicates feat: share identical summary producers across queries after Pass 1 #515's class pricing.- Merging is lossless for KLL (fixed k) and for exact Sum/Count/Min/Max. Rate/Increase need counter-reset handling across window boundaries. Offline empirical evidence rejects merges.
Proposed design (to revisit after #511; recommendations, not final)
- Merge count: give
SummaryMergethe query window, so it merges W/g tumbling windows. No new node type. Drop the "stage allocator only" note. - Tumbling length g: represent it in the state's input window so feat: share identical summary producers across queries after Pass 1 #515 identity applies unchanged, and set
selected_window_framework = Tumblingfrom the rule itself. Exact representation depends on docs: define unified operators and SQL/PromQL scalar boundaries #511's post-KeepPreAsapIR. - Evaluation interval: pass
RepeatedDemandinto Pass 1/2. g = gcd of the windows and evaluation intervals, or of the windows alone when no interval is known. - Phase/origin: the planner records only g. Window origin and phase belong to physical planning (stage 2).
- Delete
select_shared_panes. - Different accuracy across siblings: reuse the strictest-consumer sizing from Pass 2 rule 2. Window accuracy is
Exactfor mergeable families only. - Families: start with KLL and exact Sum/Count/Min/Max. Exclude Rate/Increase and non-mergeable intents.
- A single repeating query (Pattern B) gets a tumbling candidate through the same path, with a group of one. The independent candidate is kept, and selection compares the two.
- Candidate growth: one tumbling candidate per maximal sibling group, not every subset.
Tests to add
quantile_over_time(0.99, lat[5m])every 1m plusquantile_over_time(0.5, lat[15m])every 5m. Expect one shared 1-minute state, a per-querySummaryMerge, and independent candidates still present.- Single-query Pattern B.
- Update
different_producers_are_not_shared, using costs under which sharing loses.
Later
Sliding windows and Exponential Histograms. EH also needs query-time materialization and the summary subtract/delete design, which is a TODO in #509.
Activity
zzylol commentedon Oct 15, 2026
Status: in progress. This is being implemented as the tumbling-pane Pass 2 rule. It builds on #696 (merged: the family-capability table and executor tumbling pane merges). The PR that adds the rule will close this issue.
zzylol commentedon Oct 15, 2026
Implemented by #713 (merged): the tumbling-pane window-composition rule in Pass 2 (per-query pane width gcd(lookback, cadence), a compact PaneChain node, guarantees from the family-capability merge facts). A group-wide pane width and long-window segments are tracked with the window-grid work (#522).