Skip to content

feat(spanner-driver): integrate native spannerlib-node wrapper and implement type system - #9141

Merged
surbhigarg92 merged 3 commits into
mainfrom
spanner-driver-native-bridge
Aug 17, 2026
Merged

feat(spanner-driver): integrate native spannerlib-node wrapper and implement type system#9141
surbhigarg92 merged 3 commits into
mainfrom
spanner-driver-native-bridge

Conversation

@surbhigarg92

@surbhigarg92 surbhigarg92 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

This PR integrates the native spannerlib-node wrapper into @google-cloud/spanner-driver and implements the complete PostgreSQL-compatible data type codec and parsing system following standard node-postgres (pg) conventions. It enables connecting to Cloud Spanner PostgreSQL-dialect databases, executing parameterized queries, decoding all Cloud Spanner scalar and array data types, streaming rows/events, managing transactions (BEGIN, COMMIT, ROLLBACK), and pooling connections.

Key Changes

1. Native Bridge Integration (spannerlib-node)

Connection & Pool Bridge: Integrated Connection, Pool, and Rows from spannerlib-node in src/lib/native.ts
Execution & Teardown Lifecycle: Implemented connection handshake, sequential query task queueing, and safe resource closure in Client and Pool.

2. Comprehensive Type System & Codecs

  • Protobuf to PostgreSQL Catalog OID Mapping: Implemented mapSpannerTypeToPgOid in src/lib/pg/types.ts , mapping all Spanner TypeCodes and TypeAnnotationCodes to standard numeric PostgreSQL OIDs.
  • Type Parsers: Built default text parsers converting Spanner raw values to native JavaScript types (string, number, boolean, Date, Buffer, object, Array).
  • TypeOverrides Registry: Added scoped and hierarchical TypeOverrides support matching pg.types for custom parser overrides per client or query.
  • ORM .toPostgres() Serialization: Added automatic parameter serialization for objects implementing .toPostgres() (e.g. Knex/ORMs) in src/lib/pg/utilities.ts

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates the native CGO bridge (spannerlib-node) into the Spanner driver, enabling query execution, connection pooling, and a custom type parser system (TypeOverrides and Codec) compatible with node-postgres. Feedback focuses on preventing resource leaks by wrapping result set processing and connection creation in appropriate try...finally and try...catch blocks. Additionally, it is recommended to relax the numeric OID checks in TypeOverrides to support GoogleSQL string descriptors and to improve array element type inference when the first element is null.

Comment thread handwritten/spanner-driver/src/lib/client.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/client.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts
Comment thread handwritten/spanner-driver/src/lib/codec.ts Outdated
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch 6 times, most recently from 7a2a255 to 5b37e00 Compare August 13, 2026 15:57
@surbhigarg92 surbhigarg92 changed the title Spanner driver native bridge feat(spanner-driver): integrate native spannerlib-node wrapper and implement type system Aug 13, 2026
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch from 5b37e00 to ae110fb Compare August 13, 2026 17:19
@surbhigarg92

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates the native Go CGO bridge (spannerlib-node) into the Spanner Node.js driver, enabling native connection management, query execution, parameter serialization, and custom type parsing. The reviewer provided valuable feedback pointing out several critical issues, including a potential native resource leak during concurrent client teardown, a crash risk with invalid Date parameters, and a TypeError that breaks GoogleSQL queries. Additionally, the reviewer highlighted architectural concerns regarding nested native pools and memory buffering during streaming, a naive array parser that fails on commas, a regression in connect() compatibility, and a packaging issue with local dependency paths.

Comment thread handwritten/spanner-driver/src/lib/client.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/codec.ts
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts
Comment thread handwritten/spanner-driver/src/lib/client.ts
Comment thread handwritten/spanner-driver/package.json
Comment thread handwritten/spanner-driver/src/lib/client.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/client.ts
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch 2 times, most recently from e117502 to 7ffb333 Compare August 14, 2026 05:48
@surbhigarg92
surbhigarg92 marked this pull request as ready for review August 14, 2026 06:07
@surbhigarg92
surbhigarg92 requested a review from a team as a code owner August 14, 2026 06:07
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch 6 times, most recently from 7cb6ac0 to 1582dc1 Compare August 14, 2026 10:41
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts Outdated
Comment thread handwritten/spanner-driver/src/lib/codec.ts
Comment thread handwritten/spanner-driver/src/lib/client.ts
Comment thread handwritten/spanner-driver/src/lib/client.ts
Comment thread handwritten/spanner-driver/src/lib/codec.ts
Comment thread handwritten/spanner-driver/src/lib/codec.ts Outdated
Comment thread handwritten/spanner-driver/README.md Outdated
Comment thread handwritten/spanner-driver/src/lib/pg/types.ts Outdated
Comment thread handwritten/spanner-driver/system-test/driver.ts
@surbhigarg92

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates the native CGO bridge (spannerlib-node) into the Cloud Spanner Node.js driver, adding transaction support, connection pooling, a custom type system, and parameter serialization. It also introduces a Codec utility for protobuf wire format mapping, updates the documentation, and adds comprehensive unit and system tests. The review feedback highlights several critical issues: a potential crash in TypeOverrides.getTypeParser when handling non-numeric OIDs, silent query failures when nativeConnection is missing, unhandled errors in the finally block of _executeOnNativeConnection, the use of dynamic require() paths in commented-out code, and a relative file: path in optionalDependencies that will break external installations.

Comment thread handwritten/spanner-driver/src/lib/pg/types.ts
Comment thread handwritten/spanner-driver/src/lib/client.ts
Comment thread handwritten/spanner-driver/src/lib/client.ts
Comment thread handwritten/spanner-driver/src/lib/native.ts Outdated
Comment thread handwritten/spanner-driver/package.json Outdated
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch from 7dc6e9f to 36e43a5 Compare August 17, 2026 09:03
…plement type system

- Integrate Client and Pool directly with native CGO/N-API bindings via spannerlib-node.
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch 4 times, most recently from ec38890 to 5f146fb Compare August 17, 2026 09:27
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch from 5f146fb to a8ad609 Compare August 17, 2026 09:28
@surbhigarg92
surbhigarg92 force-pushed the spanner-driver-native-bridge branch from a8ad609 to 728139f Compare August 17, 2026 09:37
Comment on lines +76 to +81
if (Number.isInteger(val)) {
return {
valueProto: {stringValue: String(val)},
typeProto: {code: TypeCode.INT64},
};
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worry about how this will work in real life. If an application tries to insert the value 1.0 into a FLOAT64 column, what will happen then? Do we have an end-to-end test for that?

Comment on lines +82 to +85
return {
valueProto: {numberValue: val},
typeProto: {code: TypeCode.FLOAT64},
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also risky. What if the application is trying to insert the value 1.2 into a float32 column. Does Spanner accept this?

Also, we probably need to manually check if the value is NaN, Inf or -Inf. Those values should also be sent as strings to Spanner.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, my question about float32 can probably be ignored. I see now that all data types are removed before the query parameters are sent to Spanner, so the float64 vs float32 problem should not be there.

@surbhigarg92
surbhigarg92 merged commit 4c4bbdc into main Aug 17, 2026
43 checks passed
@surbhigarg92
surbhigarg92 deleted the spanner-driver-native-bridge branch August 17, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants