Skip to content

Query-time summary build reads unstored ingestion-time rows and still validates #738

Closed

Description

@zzylol

Problem, by example

Take a DAG where the rows below a summary build only exist at ingestion time, nobody stores them, and the build itself runs at query time:

SummaryAgg(Kll)                 timing: QueryTime,     NotMaterialized
  └─ <rows node>                timing: IngestionTime  (fixed, e.g. by its own consumer), NotMaterialized
       └─ Scan(latency_ms)

Today: timing validation accepts this plan. The SummaryAgg.child edge rule in crates/types/src/ir/timing.rs (validate_asap) accepts ExecutionDataState::INGESTION_ROWS whatever the build's own timing. It only rejects a query-time value under an ingestion-time build:

match state_of(child) {
    ExecutionDataState::INGESTION_ROWS | ExecutionDataState::QUERY_ROWS => {}   // accepted at any timing
    ...
}
if timing == IngestionTime && avail.timing == QueryTime { return Err(QueryTimeValueUnderIngestionSummary { .. }) }

At query time, nothing holds those ingestion-time rows: they were not stored (NotMaterialized), and the query-time build cannot see the ingestion stream. The plan validates but cannot be executed.

Expected: the same rule a NonASAP consumer already follows (edge-rule table, last row). A query-time consumer reads either rows at its own timing, or a MatAtIngestion input's stored output. A query-time SummaryAgg over unstored ingestion-time rows is rejected with a clear error.

Cause

The SummaryAgg.child / PaneChain.child rule has no "stored or same timing" check for rows. The NonASAP rule has one.

Proposal

In validate_asap, refuse SummaryAgg / PaneChain at QueryTime whose child is INGESTION_ROWS from a node that is not MatAtIngestion. Add a new ExecutionDataStateError variant naming the edge, and add the row to the edge-rule table.

Rejecting this changes which plans validate. Before doing it, check:

  • whether Stage 2 ever produces such an assignment; it should not, because its options keep a build and its rows on the same timing;
  • which tests build such plans by hand.

Scope / acceptance

  • A unit test with the DAG above: validation fails with the new error. The same DAG with the rows node MatAtIngestion validates.
  • Examples 1–4 select the same plans at the same costs.

Status / related

Deferred by decision on 2026-10-16. Found in the #667 timing review and listed as row 14 of #737's nit table. Related: #730 (raw-data retention), #717 (Stage 2 materialization).

Activity

  1. added
    bindingLogical-to-physical binding and implementation selection
    bugSomething isn't working
    schemaSchema, type, key, and nullability contracts
    on Oct 17, 2026
  2. added 2 commits that reference this issue on Oct 18, 2026
    c342271
    376a2b3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bindingLogical-to-physical binding and implementation selectionbugSomething isn't workingschemaSchema, type, key, and nullability contracts

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions