Skip to content

feat(firestore): add over() frame override on WindowFunction - #19148

Open
ohmayr wants to merge 1 commit into
feat/firestore-window-3-range-framefrom
feat/firestore-window-4-over
Open

ohmayr wants to merge 1 commit into
feat/firestore-window-3-range-framefrom
feat/firestore-window-4-over

Conversation

@ohmayr

@ohmayr ohmayr commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Goal

Add per-accumulator window frame overrides via WindowFunction.over(frame), building on DocumentWindowFrame (#19117) and RangeWindowFrame (#19119).

Background: Per-Accumulator Window Frames

Setting frame=... on WindowSpec applies one default frame to every accumulator in an AddWindowFields stage. Often, however, a single partition and sort order needs to evaluate multiple accumulators over different frames—for example, computing a 20-year trailing count alongside a 3-document sliding window of nearby titles in one pass:

pipeline.add_window_fields(
    WindowSpec(sort=Field.of("published").ascending()),
    Count()
    .over(RangeWindowFrame(20, WindowBound.CURRENT))
    .as_("books_within_20_years"),
    Field.of("title")
    .array_agg()
    .over(DocumentWindowFrame(1, 1))
    .as_("nearby_titles"),
)

On the wire, fn.over(frame) encodes as a function call Value(function_value={"name": "over", "args": [fn._to_pb(), Value(map_value={"fields": frame._to_dict()})]}).

Summary of Changes

  • Add WindowFunction.over(frame: DocumentWindowFrame | RangeWindowFrame) -> WindowFunction in google/cloud/firestore_v1/pipeline_expressions.py (inherited by AggregateFunction, so all aggregate functions support .over(...)).
  • Add an internal _WindowFrameExpression(Expression) @dataclass adapter (mirroring _PipelineValueExpression) so FunctionExpression stays generic and WindowFunction requires no custom dunder or protobuf overrides.
  • Add unit tests in tests/unit/v1/test_pipeline_expressions.py covering repr, equality, type hierarchy (not hasattr(Field.of("rating"), "over")), and protobuf serialization for both document and range frames.
  • Add testAddWindowFields - accumulator-level over() frames and testAddWindowFields - error on rank() with over() frame to tests/system/pipeline_e2e/window.yaml.

What We Will Follow Up On

Verification

  • Unit Tests: pytest tests/unit/v1/test_pipeline_stages.py tests/unit/v1/test_pipeline_expressions.py tests/unit/v1/test_pipeline.py tests/unit/v1/test_async_pipeline.py (475 passed; 0 missed lines or branches across all added lines in pipeline_expressions.py, 100% coverage on pipeline_types.py).
  • System Tests: pytest tests/system/test_pipeline_acceptance.py -k window.yaml (proto serialization check in default CI mode, plus live sync/async execution against test-firestore.sandbox.googleapis.com:443: 16 passed).
  • Linter & Typecheck: mypy, ruff check --select I, ruff format --check, and flake8 google tests all pass cleanly.

Add WindowFunction.over(frame) to allow per-accumulator document or
range window frame overrides inside an AddWindowFields stage.

Wrap the frame argument in an internal _WindowFrameExpression dataclass
adapter (mirroring _PipelineValueExpression) so FunctionExpression stays
generic and WindowFunction requires no custom dunder or protobuf
overrides. Add unit tests and pipeline_e2e/window.yaml system tests
covering accumulator-level frames and backend rejection of
Rank().over(...).
@ohmayr
ohmayr requested a review from a team as a code owner November 20, 2026 00:50
class WindowFunction(FunctionExpression):
"""A base class for window functions that operate over a window of documents."""

def over(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chosen Design: Define over(frame) directly on WindowFunction (inherited by AggregateFunction, which subclasses WindowFunction), returning a WindowFunction and letting the backend reject invalid pairings like Rank().over(frame) (verified in window.yaml).

Alternatives Considered & Why Rejected:

  • Placing over(frame) only on AggregateFunction: Excludes upcoming non-aggregate window functions that do accept frames (e.g., first_value, last_value) or requires a third intermediate class solely for framing.
  • Client-side rejection of Rank().over(frame): Duplicates backend validation and risks drifting as window function support evolves; aligns with Node.js and Java.

@ohmayr
ohmayr added this pull request to stack #19118 November 20, 2026 00:51


@dataclass
class _WindowFrameExpression(Expression):

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chosen Design: Wrap frame in a small @dataclass Expression adapter (_WindowFrameExpression, mirroring _PipelineValueExpression directly above) so over() constructs WindowFunction("over", [self, _WindowFrameExpression(frame)], repr_function=...). This keeps FunctionExpression untouched (params: Sequence[Expression]), avoids importing WindowSpec here, gives WindowFunction zero dunder/_to_pb boilerplate, and lets the backend reject chained .over(f1).over(f2) (over(over(...))) rather than silently overwriting f1.

Alternatives Considered & Why Rejected:

  • Passing WindowSpec(frame=frame) and widening FunctionExpression.params: Leaks window-stage configuration types into the generic FunctionExpression base class.
  • Storing self._frame on WindowFunction with custom __init__, __repr__, __eq__, _to_pb: Adds four overrides to WindowFunction solely for over(), and risks silently dropping f1 if chained.
  • Dedicated _OverExpression(WindowFunction) subclass: Still introduces an internal class while requiring manual __init__, __eq__, and _to_pb instead of reusing FunctionExpression + @dataclass.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the over method to the WindowFunction class in pipeline_expressions.py, enabling the configuration of window frames (DocumentWindowFrame or RangeWindowFrame) for window functions. It also implements an internal _WindowFrameExpression wrapper to serialize these frames as expressions, and adds comprehensive unit and system tests to validate the new functionality. There are no review comments, so I have no feedback to provide.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant