Skip to content

fix(scheduler): skip intervals that overlap unready parent intervals - #6106

Open
breken-ai wants to merge 1 commit into
SQLMesh:mainfrom
breken-ai:fix/interval-diff-unready-parent
Open

breken-ai wants to merge 1 commit into
SQLMesh:mainfrom
breken-ai:fix/interval-diff-unready-parent

Conversation

@breken-ai

Copy link
Copy Markdown

Description

When an upstream model's signal marks an interval as not ready, the scheduler is supposed to hold back every downstream interval that overlaps it. interval_diff in sqlmesh/core/scheduler.py does this, but it only works when the unready interval starts at or before the candidate interval. If the candidate interval is larger and starts first, it takes the index_b += 1 branch. That branch moves past the unready interval and leaves the candidate in place.

This happens whenever a model has a coarser interval unit than a parent that has a signal. Example: a daily model reading from an hourly model whose signal reports 2023-01-01 10:00 and 2023-01-03 05:00 as not ready.

  • Jan 1: compared against the Jan 3 unready hour instead of the Jan 1 one, so it is treated as ready. The daily model runs over a day that is missing an hour of upstream data, and the interval is recorded as done, so later runs don't fix it.
  • Jan 4: fully ready, but it is dropped by the check after the loop, which compares it with the interval that caused the overlap.
>>> interval_diff([(0, 24), (24, 48), (48, 72), (72, 96)], [(10, 11), (53, 54)])
[(0, 24), (24, 48)]   # expected [(24, 48), (72, 96)]
>>> interval_diff([(0, 24), (24, 48)], [(10, 11)])
[]                    # expected [(24, 48)]

The fix skips the candidate interval on any overlap and keeps the unready interval, because it can also overlap the next candidate. Once the unready intervals run out, every remaining candidate starts after the last one, so all of them are kept. The uninterrupted (depends-on-past) early return is unchanged.

Test Plan

  • Added two cases to test_interval_diff for a large interval that contains a smaller unready interval.
  • Added test_signals_unready_hourly_parent_blocks_daily_child: a daily INCREMENTAL_BY_TIME_RANGE model downstream of an hourly model with a signal that holds back two hours. On main the child's batches are [(2023-01-01, 2023-01-03)], which evaluates Jan 1 and skips Jan 4. With the fix they are [(2023-01-02, 2023-01-03), (2023-01-04, 2023-01-05)].
  • Both new tests fail on main (263723f) and pass with the fix.
  • A randomized comparison against a brute-force overlap filter (20k cases, including uninterrupted=True) matches.
  • pytest tests/core/test_scheduler.py tests/core/test_snapshot.py: 159 passed.
  • tests/core/test_plan.py, test_plan_evaluator.py, test_context.py and tests/core/integration show the same results with and without the change. The only failures are dbt tests that need dbt, which isn't installed in my environment.
  • ruff check, ruff format --check and mypy sqlmesh/core/scheduler.py are clean for the changed files.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

I ran ruff and mypy directly rather than the full pre-commit suite. I ran the core scheduler, snapshot, plan and integration suites rather than the full make fast-test.

This fix was found and written with AI assistance (Claude).

`interval_diff` advanced past an unready interval whenever the
candidate interval started before it. When a daily model depends on an
hourly model whose signal marks an hour as not ready, the day containing
that hour was compared against the next unready interval instead, so the
daily model was evaluated over incomplete upstream data and the interval
was recorded as done. Ready intervals after the last overlap were also
dropped.

Always skip the candidate interval on overlap and keep every remaining
interval once the unready intervals are exhausted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant