Skip to content

Refactor GreptimeDB Python SDK to separate gRPC and PostgreSQL backends - #2

Merged
paomian merged 2 commits into
mainfrom
pg-copy-from-stdin
Nov 2, 2026
Merged

paomian merged 2 commits into
mainfrom
pg-copy-from-stdin

Conversation

@paomian

@paomian paomian commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator
  • Updated documentation to clarify backend usage and import paths.
  • Removed automatic backend selection; users must explicitly import the desired backend.
  • Simplified error handling by consolidating GreptimeError into a single implementation.
  • Enhanced type handling in the gRPC and PostgreSQL modules to ensure compatibility with shared schema.
  • Removed legacy native module and associated imports.
  • Added tests to verify that both backends correctly utilize the shared schema and handle errors appropriately.
  • Ensured that the public schema is consistent across both backends.

- Updated documentation to clarify backend usage and import paths.
- Removed automatic backend selection; users must explicitly import the desired backend.
- Simplified error handling by consolidating GreptimeError into a single implementation.
- Enhanced type handling in the gRPC and PostgreSQL modules to ensure compatibility with shared schema.
- Removed legacy native module and associated imports.
- Added tests to verify that both backends correctly utilize the shared schema and handle errors appropriately.
- Ensured that the public schema is consistent across both backends.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the refactor separating explicit backend selection from shared schema and error types.

No open issues 🎉

What changed in this PR

This PR separates backend selection from the SDK’s shared schema. Callers explicitly import the gRPC or PostgreSQL client, while both backends use the same public column types and GreptimeError.

Changes:

  • Remove automatic backend selection and the legacy native import shim.
  • Adapt PostgreSQL and Rust bindings to consume shared Python schema and error types.
  • Update tests, examples, and documentation for explicit backend imports.
File Description
tests/​test_types.py Require public Column objects in PostgreSQL validation.
tests/​test_shared_schema.py Add shared schema and error checks for both backends.
tests/​test_postgres.py Update validation calls and strengthen import isolation checks.
tests/​test_postgres_schema_compat.py Remove obsolete native-schema compatibility tests.
tests/​test_integration.py Update imports and check bulk writer column types.
tests/​test_convert.py Use explicit gRPC and native plugin imports.
tests/​test_backends.py Verify root exports and backend import isolation.
README.zh-CN.md Document explicit imports and refresh reference links.
README.md Mirror backend import guidance and reference updates.
python/​greptimedb_ingester/​postgres.py Use shared columns with explicit SQL type mappings.
python/​greptimedb_ingester/​grpc.py Limit exports to backend-specific APIs.
python/​greptimedb_ingester/​_types.py Clarify shared schema ownership and simplify error creation.
python/​greptimedb_ingester/​_native.py Remove the legacy native import shim.
python/​greptimedb_ingester/​_errors.py Define one backend-independent GreptimeError.
python/​greptimedb_ingester/​__init__.py Remove automatic client selection and backend exports.
packages/​grpc/​src/​types.rs Convert between shared Python columns and internal Rust types.
packages/​grpc/​src/​lib.rs Stop registering native schema and error classes.
packages/​grpc/​src/​error.rs Construct the shared Python error from Rust.
packages/​grpc/​src/​bulk.rs Return public Python columns from the writer getter.
examples/​insert_example.py Import Client explicitly from gRPC.
examples/​bulk_example.py Import Client and WriteOptions from gRPC.
examples/​bench_write.py Separate shared schema and gRPC imports.
docs/​usage.md Update examples and backend selection guidance.
AGENTS.md Document explicit imports and prohibit legacy shims.

Review effort: Balanced

- Update `.gitignore` to ignore all files in the `docs` directory.
- Modify README files to reference the new PG getting started guide.
- Refactor decimal handling in gRPC to improve precision and scale validation.
- Change `exponent` type in `DecimalParts` from `i32` to `i64` for better compatibility.
- Enhance error handling in `raise` function to ensure retriable status is correctly set.
- Introduce `checked_duration` function to validate duration inputs.
- Add comprehensive tests for input validation and error handling in gRPC and PostgreSQL clients.
- Ensure schema validation is robust against forged inputs and maintains integrity across operations.
- Update PostgreSQL client to handle connection errors gracefully and invalidate cached schemas when necessary.
@paomian
paomian merged commit 0e0a510 into main Nov 2, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants