Skip to content

chore(spanner): error on truncated or malformed result streams - #19164

Open
olavloite wants to merge 1 commit into
mainfrom
spanner-error-on-malformed-streams
Open

olavloite wants to merge 1 commit into
mainfrom
spanner-error-on-malformed-streams

Conversation

@olavloite

Copy link
Copy Markdown
Contributor

Previously, streaming query results silently ignored trailing partial data if a stream ended mid-row, while waiting for the next part of a chunked value, or with missing column metadata. This could cause callers to receive truncated results without any error.

Note that the above problems should never happen as long as Spanner adheres to its own contract. This is therefore purely a hardening for potential data corruption.

Now both the sync and async streaming result sets detect these malformed terminations and raise an internal server error across all consumption methods (iteration, single-row lookups, and dictionary conversion), keeping the failure sticky if the result set is read again.

Includes symmetric sync/async unit tests and end-to-end mock server tests covering truncated chunks, incomplete rows, and missing row metadata.

@olavloite
olavloite requested a review from a team as a code owner November 21, 2026 06:51

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust stream termination validation for both synchronous and asynchronous streamed result sets, ensuring that sticky terminal errors are recorded and raised if a stream ends mid-chunk, mid-row, or with invalid metadata. It also adds comprehensive test coverage for these scenarios. The review feedback correctly points out that one_or_none() should check and raise self._error at the beginning of the method in both implementations to prevent silent failures when a stream has previously encountered a terminal error.

Previously, streaming query results silently ignored trailing partial
data if a stream ended mid-row, while waiting for the next part of a
chunked value, or with missing column metadata. This could cause callers
to receive truncated results without any error.

Note that the above problems should never happen as long as Spanner
adheres to its own contract. This is therefore purely a hardening
for potential data corruption.

Now both the sync and async streaming result sets detect these malformed
terminations and raise an internal server error across all consumption
methods (iteration, single-row lookups, and dictionary conversion),
keeping the failure sticky if the result set is read again.

Includes symmetric sync/async unit tests and end-to-end mock server
tests covering truncated chunks, incomplete rows, and missing row
metadata.
@olavloite
olavloite force-pushed the spanner-error-on-malformed-streams branch from 704ad29 to 7f5e1e6 Compare November 21, 2026 07:02
@olavloite

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces validation for stream termination in both the synchronous and asynchronous streamed result sets (streamed.py). It ensures that an InternalServerError is raised and sticky-cached if a stream ends unexpectedly (e.g., mid-chunk, mid-row, with missing metadata, or with empty chunked values). Comprehensive unit and mock server tests have been added to cover these invalid termination scenarios, and existing tests were updated accordingly. I have no feedback to provide as there are no review comments.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant