Skip to content

Document reverse proxy header buffers for the sync websocket #309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LeoAnt02 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Document reverse proxy header buffers for the sync websocket #309

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions contents/docs/self-host.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -791,6 +791,29 @@ The replication-manager serves Zero's internal replication protocol. Keep it beh

The external load balancer for view-syncers must support websockets, and can use the health check at `/keepalive` to verify view-syncers are healthy. The replication-manager should also have a `/keepalive` health check, but that check should run through private infrastructure rather than a public load balancer.

### Reverse Proxy Header Buffers

During connection, the Zero client sends its entire client schema (and auth token, if any) URL-encoded in the `Sec-WebSocket-Protocol` header, and the view-syncer echoes this header back in the `101 Switching Protocols` response. The header grows with your schema — an app with a few dozen synced tables and columns can exceed 4 KB.

Reverse proxies buffer upstream response headers, and the defaults can be too small. nginx's `proxy_buffer_size` defaults to 4 KB; once the handshake outgrows it, nginx fails **every** sync connection with `502 Bad Gateway` (`upstream sent too big header` in the nginx error log). Nothing appears in `zero-cache` logs — the response dies at the proxy — and the client only sees a generic websocket failure. If sync worked and then broke after adding tables or columns to your schema, check for this.

Raise the buffer to fix it:

```nginx
# nginx
proxy_buffer_size 16k;
```

```yaml
# Kubernetes ingress-nginx annotation
nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"
```

<Note heading="Request headers too" slug="request-header-buffers">
The same header is also large on the way in. nginx's request-side limit (`large_client_header_buffers`, default 8 KB) is more generous, but a very large schema combined with a long auth JWT and cookies can eventually reach it as well. Other proxies and load balancers have equivalent limits.

</Note>

### Sticky Sessions

View syncers are designed to be disposable, but since they keep hydrated query pipelines in memory, it's important to try to keep clients connected to the same instance.
Expand Down