Skip to content

perf(ado.net): fetch multiple rows at a time#779

Merged
olavloite merged 1 commit into
mainfrom
fetch-multiple-rows
Jun 30, 2026
Merged

perf(ado.net): fetch multiple rows at a time#779
olavloite merged 1 commit into
mainfrom
fetch-multiple-rows

Conversation

@olavloite

Copy link
Copy Markdown
Collaborator

Fetch multiple rows at a time when reading data from a query.

@olavloite olavloite requested review from a team as code owners June 26, 2026 17:49

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces batching and prefetching support for streaming rows in the Spanner ADO.NET provider, updating connection and streaming methods to pass a batch size and buffering fetched rows. However, several critical issues were identified in the review. First, clearing the buffered rows at the start of NextResultSet and NextResultSetAsync can lead to silent data loss, and the provider needs to correctly track and defer the HasMoreResults signal when rows are buffered. Second, the removal of the DrainStream helper and its calls in Server.cs must be reverted, as failing to drain the redirected standard output and error streams of the child process will cause it to block and hang indefinitely when the OS pipe buffer fills up.

Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcServer/Server.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcServer/Server.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcServer/Server.cs Outdated
Fetch multiple rows at a time when reading data from a query.
@olavloite olavloite force-pushed the fetch-multiple-rows branch from 216fc83 to 2901edf Compare June 29, 2026 11:49
@olavloite

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces row prefetching and batching capabilities to the streaming rows implementation in the Spanner ADO.NET gRPC client. It adds a new ContinueStreamingRequest protobuf message to specify batch sizes and implements a buffered queue (_bufferedRows) in StreamingRows to cache pre-fetched rows. Feedback on these changes highlights an opportunity to optimize the row enqueuing logic by returning the first row directly and only enqueuing the remainder. Additionally, a bug was identified in NextResultSetAsync where a synchronous call to ContinueStreaming is used instead of its asynchronous counterpart, which would block the async thread.

Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
Comment thread spanner-ado-net/spanner-ado-net/SpannerLibGrpcImpl/StreamingRows.cs
@olavloite olavloite requested a review from surbhigarg92 June 29, 2026 12:04
@olavloite olavloite merged commit cbdc157 into main Jun 30, 2026
24 checks passed
@olavloite olavloite deleted the fetch-multiple-rows branch June 30, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants