You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Return 503 when the database drops a request's connection #905
When Postgres ends a session, for example after the API's 60-second idle-in-transaction limit or during a restart or failover, the next query fails and the API returns a 500 and sends a Slack alert. The client can't tell that retrying will work. This issue returns 503 with Retry-After for a dropped connection instead, and keeps the Slack alert.
Steps to reproduce
Open a session with SET idle_in_transaction_session_timeout = '500ms'.
Run a query, wait 1.5 seconds, run another.
Observed
sqlalchemy.exc.OperationalError: server closed the connection unexpectedly, with no SQLSTATE (err.orig.pgcode is None) and err.connection_invalidated set. CatchAllErrorMiddleware returns 500.
Fix
In lib/middleware/errors.py, treat a DBAPIError with connection_invalidated as a 503 with Retry-After: 5.
Unlike the database timeouts, keep sending the Slack alert: a dropped connection can also mean an outage.
Acceptance criteria
A route whose connection is closed by the server returns 503 with Retry-After.
The Slack alert still fires for it.
Other database errors still return 500.
Background
Postgres reports this timeout as SQLSTATE 25P03, but it closes the connection as it does, so the client only sees the dropped connection. Checked with psycopg2 against a local Postgres.
Activity