Skip to content

chore(firestore): prep for firestore pipelines GA #16549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
daniel-sanche wants to merge 3 commits into
base: main
Choose a base branch
from
+38 −40
Draft

chore(firestore): prep for firestore pipelines GA #16549

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0d398dd
remove preview warnings
daniel-sanche Apr 2, 2026
91a69c1
exposed options in raw_stage
daniel-sanche Apr 3, 2026
0e723a8
fix typo
daniel-sanche Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions packages/google-cloud-firestore/google/cloud/firestore_v1/base_pipeline.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -459,7 +459,12 @@ def unnest(
"""
return self._append(stages.Unnest(field, alias, options))

def raw_stage(self, name: str, *params: Expression) -> "_BasePipeline":
def raw_stage(
self,
name: str,
*params: Expression,
options: dict[str, Expression | Value] | None = None,
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The type hint Value is used here but it does not appear to be imported in this file. If Value is a type alias defined in google.cloud.firestore_v1.pipeline_expressions, it should be imported alongside Expression to avoid a NameError at runtime, especially if type hints are evaluated.

) -> "_BasePipeline":
"""
Adds a stage to the pipeline by specifying the stage name as an argument. This does not offer any
type safety on the stage params and requires the caller to know the order (and optionally names)
Expand All @@ -477,11 +482,12 @@ def raw_stage(self, name: str, *params: Expression) -> "_BasePipeline":
Args:
name: The name of the stage.
*params: A sequence of `Expression` objects representing the parameters for the stage.
options: An optional dictionary of stage options.

Returns:
A new Pipeline object with this stage appended to the stage list
"""
return self._append(stages.RawStage(name, *params))
return self._append(stages.RawStage(name, *params, options=options or {}))

def offset(self, offset: int) -> "_BasePipeline":
"""
Expand Down
25 changes: 2 additions & 23 deletions packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_expressions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
.. warning::
**Preview API**: Firestore Pipelines is currently in preview and is
subject to potential breaking changes in future releases.
"""

from __future__ import annotations

Expand Down Expand Up @@ -188,7 +183,7 @@ class Expression(ABC):
"""Represents an expression that can be evaluated to a value within the
execution of a pipeline.

Expressionessions are the building blocks for creating complex queries and
Expressions are the building blocks for creating complex queries and
transformations in Firestore pipelines. They can represent:

- **Field references:** Access values from document fields.
Expand Down Expand Up @@ -938,7 +933,7 @@ def is_absent(self) -> "BooleanExpression":
>>> Field.of("email").is_absent()

Returns:
A new `BooleanExpressionession` representing the isAbsent operation.
A new `BooleanExpression` representing the isAbsent operation.
"""
return BooleanExpression("is_absent", [self])

Expand Down Expand Up @@ -1832,10 +1827,6 @@ def array_agg(self) -> "Expression":
`None`. The order of elements in the output array is not stable and
shouldn't be relied upon.

This API is provided as a preview for developers and may change based
on feedback that we receive. Do not use this API in a production
environment.

Example:
>>> # Collect all values of field 'color' into an array
>>> Field.of("color").array_agg()
Expand All @@ -1854,10 +1845,6 @@ def array_agg_distinct(self) -> "Expression":
`None`. The order of elements in the output array is not stable and
shouldn't be relied upon.

This API is provided as a preview for developers and may change based
on feedback that we receive. Do not use this API in a production
environment.

Example:
>>> # Collect distinct values of field 'color' into an array
>>> Field.of("color").array_agg_distinct()
Expand All @@ -1872,10 +1859,6 @@ def first(self) -> "Expression":
"""Creates an aggregation that finds the first value of an expression
across multiple stage inputs.

This API is provided as a preview for developers and may change based
on feedback that we receive. Do not use this API in a production
environment.

Example:
>>> # Select the first value of field 'color'
>>> Field.of("color").first()
Expand All @@ -1890,10 +1873,6 @@ def last(self) -> "Expression":
"""Creates an aggregation that finds the last value of an expression
across multiple stage inputs.

This API is provided as a preview for developers and may change based
on feedback that we receive. Do not use this API in a production
environment.

Example:
>>> # Select the last value of field 'color'
>>> Field.of("color").last()
Expand Down
5 changes: 0 additions & 5 deletions packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_result.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
.. warning::
**Preview API**: Firestore Pipelines is currently in preview and is
subject to potential breaking changes in future releases.
"""

from __future__ import annotations

Expand Down
5 changes: 0 additions & 5 deletions packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
.. warning::
**Preview API**: Firestore Pipelines is currently in preview and is
subject to potential breaking changes in future releases.
"""

from __future__ import annotations

Expand Down
5 changes: 0 additions & 5 deletions packages/google-cloud-firestore/google/cloud/firestore_v1/pipeline_stages.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
.. warning::
**Preview API**: Firestore Pipelines is currently in preview and is
subject to potential breaking changes in future releases.
"""

from __future__ import annotations

Expand Down
14 changes: 14 additions & 0 deletions packages/google-cloud-firestore/tests/unit/v1/test_async_pipeline.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,3 +448,17 @@ def test_async_pipeline_aggregate_with_groups():
assert isinstance(result_ppl.stages[0], stages.Aggregate)
assert list(result_ppl.stages[0].groups) == [Field.of("author")]
assert list(result_ppl.stages[0].accumulators) == [Field.of("title")]


def test_async_pipeline_raw_stage_with_options():
from google.cloud.firestore_v1.base_vector_query import Field
from google.cloud.firestore_v1.pipeline_stages import RawStage
Comment on lines +454 to +455
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is generally better practice to place imports at the top of the file rather than inside a test function, unless there is a specific reason to defer the import (e.g., avoiding circular dependencies). Also, consider if Field should be imported from google.cloud.firestore_v1.pipeline_expressions for consistency with the pipeline API being tested.

References
  1. PEP 8: Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. (link)


start_ppl = _make_async_pipeline()
result_ppl = start_ppl.raw_stage(
"stage_name", Field.of("n"), options={"key": "val"}
)
assert len(start_ppl.stages) == 0
assert len(result_ppl.stages) == 1
assert isinstance(result_ppl.stages[0], RawStage)
assert result_ppl.stages[0].options == {"key": "val"}
14 changes: 14 additions & 0 deletions packages/google-cloud-firestore/tests/unit/v1/test_pipeline.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -437,3 +437,17 @@ def test_pipeline_aggregate_with_groups():
assert isinstance(result_ppl.stages[0], stages.Aggregate)
assert list(result_ppl.stages[0].groups) == [Field.of("author")]
assert list(result_ppl.stages[0].accumulators) == [Field.of("title")]


def test_pipeline_raw_stage_with_options():
from google.cloud.firestore_v1.base_vector_query import Field
from google.cloud.firestore_v1.pipeline_stages import RawStage
Comment on lines +443 to +444
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the async pipeline tests, these imports should ideally be moved to the top of the file. Additionally, ensure that importing Field from base_vector_query is intentional and consistent with how it's used in the pipeline implementation.

References
  1. PEP 8: Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. (link)


start_ppl = _make_pipeline()
result_ppl = start_ppl.raw_stage(
"stage_name", Field.of("n"), options={"key": "val"}
)
assert len(start_ppl.stages) == 0
assert len(result_ppl.stages) == 1
assert isinstance(result_ppl.stages[0], RawStage)
assert result_ppl.stages[0].options == {"key": "val"}
Loading