feat(grpc-web): Pyodide/WASM grpc-web transport for the async client

[g-despot]

What & why

Lets the async client run inside Pyodide/WebAssembly (marimo notebooks, browser, WASM workers), where grpcio has no wheel and sockets don't exist. gRPC is re-routed over grpc-web (fetch), REST over the browser's fetch — against Weaviate core's native /v1/grpc-web endpoint (default-on since 1.38.3).

This is the trimmed successor of #2056 (kept open for reference): only what the WASM goal actually needs. The general fixes that were bundled there now ship separately:

• fix(connect): token refresh lifecycle, backoff and error-taxonomy follow-ups #2140 — token refresh lifecycle fixes. This PR is stacked on it (base branch fix/token-refresh-lifecycle): the async OIDC refresher must be an asyncio task because threads cannot start under Pyodide.
• fix(batch): surface background stream failures on exit instead of returning partial results #2141 — batch stream exit semantics.

Quickstart (Pyodide / marimo)

import micropip
await micropip.install(["weaviate-client", "weaviate-client-web"])  # until release: wheels built from this branch

import weaviate  # a bare import bootstraps weaviate_client_web under Emscripten

client = weaviate.use_async_with_custom(
    http_host="weaviate.example.com", http_port=443, http_secure=True,
    grpc_host="weaviate.example.com", grpc_port=443, grpc_secure=True,
)
await client.connect()

Under Emscripten all three async helpers (use_async_with_local / _weaviate_cloud / _custom) pin gRPC to the REST endpoint under /v1/grpc-web themselves — the same contract as the TS @weaviate/web client's webify(). There is no grpc_path_prefix parameter on the helpers: pass gRPC arguments equal to the HTTP ones; anything else is discarded with a Con006 warning. A grpc-web transcoder on a separate endpoint needs hand-built ConnectionParams(..., grpc_path_prefix=...). Off Emscripten nothing changes: the helpers build exactly the params they always did (pinned by test_helper_params_off_emscripten_are_unchanged).

Key pieces

packages/web/ — companion distribution weaviate-client-web (unchanged from #2056): _shim.py (pure-Python grpc module shim), _channel.py (GrpcWebChannel: unary calls, metadata → headers, grpc-web status → AioRpcError, grpc-timeout encoding, 404/405 diagnostics), _framing.py, _sender.py (pyfetch, plus an httpx sender for CPython testing), _httpx_fetch.py (REST over JS fetch), README.

Base client (no-ops off Emscripten):

• weaviate/__init__.py — platform-guarded bootstrap: a bare import weaviate soft-imports the companion; a missing companion raises an install hint.
• setup.cfg — grpcio ; sys_platform != "emscripten".
• connect/base.py — grpc_path_prefix on ConnectionParams (port collision allowed with a prefix); _check_grpc_web_usable runs once, at client construction (sync client or missing shim fails fast); grpc-web.path_prefix channel option for the shim.
• connect/helpers.py — _webify(): the async helpers route gRPC to the REST endpoint under Emscripten; Con006 when a caller's gRPC endpoint is discarded.
• connect/v4.py — sync client rejected at construction under Emscripten (a 3-line check in _ConnectionBase.__init__); _ping_grpc passes the gRPC error through so the diagnostics can say what went wrong; _check_package_version ignores OSError (fetch blocked by a page CSP).
• exceptions.py — WeaviateGRPCUnavailableError gains the gRPC code/details and a grpc-web branch (no firewall/port advice; a 404 names the two real causes: server < 1.38.3 or a wrong prefix).
• collections/batch/async_.py — batch.stream() fails fast under grpc-web, pointing to insert_many() (bidi streaming is impossible over fetch).
• embedded.py — explicit error under Emscripten. proto/v1/__init__.py — grpcio version fallback when dist metadata is absent (Emscripten only), drift-pinned by proto_test.

Dropped vs #2056 (and why)

• The _deadline / _Deadlines plumbing across v4.py and both batch modules: the web package already treats non-finite timeouts as "no deadline" in both transports (_encode_timeout, _abort_signal_ms); the one CPython effect (Thread.join(inf) in the sync batch-stream shutdown wait) is a separate one-line fix, not included here.
• The ConnectionSync.__init__ signature override (→ 3 lines in the base constructor) and the duplicate channel-level _check_grpc_web_usable call.
• Tests that pinned nothing or duplicated each other (test_no_helper_takes_a_grpc_path_prefix, the Emscripten __get_timeout test, the channel-level reject pair, the trivial platform-passes tests, …).

CI / tests

grpc-web-tests (package suite, 3.10–3.14) and pyodide-e2e (real Pyodide under Node against core-native /v1/grpc-web, Weaviate 1.39.0) jobs; ruff / flake8 / pyright cover packages/web; build-and-publish needs both. Base-client tests: test/test_wasm_compat.py, test/test_connection_params.py, proto_test. Locally: 176 tests across those + packages/web/tests pass; pyright clean for the whole project.

Deferred

Publish/lockstep versioning for weaviate-client-web; once on PyPI, weaviate-client gains weaviate-client-web ; sys_platform == "emscripten", collapsing the WASM install to micropip.install("weaviate-client").

🤖 Generated with Claude Code

https://claude.ai/code/session_01GUNU7QgDr9MmFZnjKY9zFN

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Secrets	0   0   0   0	View in Orca

[Copilot]

Pull request overview

Adds Pyodide/WASM support to the async client through a companion grpc-web and browser-fetch transport.

Changes:

• Adds the weaviate-client-web grpc-web transport and shim.
• Integrates automatic WASM routing, diagnostics, and unsupported-feature guards.
• Adds unit and Pyodide end-to-end CI coverage.

Reviewed changes

Copilot reviewed 32 out of 34 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
weaviate/warnings.py	Adds endpoint-override warning.
weaviate/proto/v1/__init__.py	Adds Emscripten grpcio fallback.
weaviate/exceptions.py	Adds grpc-web diagnostics.
weaviate/embedded.py	Rejects embedded mode under WASM.
weaviate/connect/v4.py	Integrates grpc-web lifecycle and errors.
weaviate/connect/helpers.py	Routes async helpers through grpc-web.
weaviate/connect/base.py	Adds grpc-web connection parameters.
weaviate/collections/batch/async_.py	Rejects streaming batches over grpc-web.
weaviate/__init__.py	Bootstraps the companion package.
test/test_wasm_compat.py	Tests WASM-specific behavior.
test/test_connection_params.py	Tests grpc-web connection configuration.
setup.cfg	Excludes grpcio under Emscripten.
pyrightconfig.json	Type-checks the web package.
proto_test/test_proto.py	Tests grpcio fallback compatibility.
packages/web/tests/test_transport.py	Tests grpc-web transport behavior.
packages/web/tests/test_single_import.py	Tests automatic bootstrap.
packages/web/tests/test_shim_install.py	Tests shim installation.
packages/web/tests/test_httpx_fetch.py	Tests fetch-based REST transport.
packages/web/tests/test_framing.py	Tests grpc-web framing.
packages/web/tests/conftest.py	Configures package test imports.
packages/web/src/weaviate_client_web/py.typed	Marks the package as typed.
packages/web/src/weaviate_client_web/_shim.py	Implements the grpc API shim.
packages/web/src/weaviate_client_web/_sender.py	Implements HTTP senders.
packages/web/src/weaviate_client_web/_httpx_fetch.py	Implements fetch-backed HTTPX transport.
packages/web/src/weaviate_client_web/_framing.py	Implements grpc-web framing.
packages/web/src/weaviate_client_web/_channel.py	Implements the grpc-web channel.
packages/web/src/weaviate_client_web/__init__.py	Exposes and bootstraps the package.
packages/web/README.md	Documents usage and limitations.
packages/web/pyproject.toml	Defines companion package metadata.
ci/pyodide-e2e/run.mjs	Runs tests inside Pyodide.
ci/pyodide-e2e/package.json	Pins the Pyodide runtime.
ci/pyodide-e2e/e2e.py	Exercises WASM client workflows.
.gitignore	Ignores web-package build artifacts.
.github/workflows/main.yaml	Adds grpc-web and Pyodide CI jobs.

Suppressed comments (1)

weaviate/connect/helpers.py:538

• This value comparison cannot tell an omitted argument from an explicitly supplied grpc_port=50051 (which the helper's own examples use). Under WASM that explicitly requested endpoint is still replaced by the REST port, but Con006 is suppressed, contrary to the documented promise that caller-supplied ports are warned about. Preserve argument presence with a sentinel default, then resolve the omitted value to 50051.

            grpc_chosen_by_caller=grpc_port != _LOCAL_GRPC_PORT_DEFAULT,

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[tsmith023]

Why is this import within the function instead of at the top of the file?

[tsmith023]

Ah I see it's so that the import is lazy

This makes me wonder, should we instead provide grpc-web functionality using the extras syntax, i.e.

pip install weaviate-client[grpc-web]

so that pyodide and cpython users don't conflict?

The default installation pip install weaviate-client would ship as it is right now and the extras would bundle this extra weaviate_client_web dep into it. wdyt?