fix: stop Postgres connection exhaustion (serialized sync views, duplicate Keycloak calls, blind health check)

[saqibmanan]

Fixes #134. Three related changes, smallest blast radius first.

Root cause

Django runs sync views under ASGI via sync_to_async(thread_sensitive=True), which executes them on one shared thread per process. With a single uvicorn worker, exactly one sync request is processed at a time regardless of how many arrive.

Measured on dev — 12 concurrent POSTs to /api/auth/keycloak/login/:

4s, 7s, 10s, 14s, 17s, 20s, 23s, 26s, 29s, 32s, 36s

Near-perfect ~3s increments. They were queued behind each other, not running concurrently.

That queue is what exhausted Postgres. Every in-flight request holds a database connection while it waits its turn — 20 concurrent calls took the connection count from 6 to 26, one per request. Deep enough queues hit max_connections (100), and Postgres began refusing:

FATAL:  sorry, too many clients already

Which is why the same fault produced three different symptoms: requests past nginx's 60s timeout became 504s, refused connections became 500s, and the deploy failed because manage.py could not get a connection either.

/health/ stayed green through all of it.

1. Multiple workers (Dockerfile)

Workers are processes, so N workers give N concurrent sync requests. The work is I/O-bound on Keycloak, so this helps well past the box's 2 CPUs.

--limit-concurrency is the backstop: in-flight requests cap at workers x limit (60), under max_connections with headroom. Excess gets a fast 503 rather than queueing until the database runs out. Shedding load is recoverable; exhausting connections takes the deploy pipeline with it. Both env-tunable.

2. Three Keycloak round-trips per login → one

KeycloakLoginView introspected the token twice (once inside validate_token, again for roles/orgs), and validate_token also called userinfo. On this deployment the client lacks the scope for userinfo, so it returns 403 every time and falls through to a branch that rebuilds the same fields from the introspection it already had.

validate_token now accepts an introspection the caller already holds, and skips userinfo when introspection already carries sub plus an identifier. The userinfo path stays for deployments where introspection is sparse.

Beyond latency: the request holds a connection for its whole duration, so every round-trip removed is connection-hold time removed.

3. Health check that can actually fail

The DB probe ran SELECT 1 on the request's own connection — already established, so it answers however saturated the server is. It could not fail for the condition taking the service down.

Now also opens a fresh connection and closes it. The two failure modes are independent, and only the second catches exhaustion.

Verification

Measurements above are all from dev before the change. Post-deploy verification (concurrency, connection ceiling, health check behaviour) to follow on this PR.