feat: PostgreSQL backend - #611
Conversation
…down Two independent fixes, neither dependent on the PostgreSQL work in tale#605. **Concurrent first logins leave the instance with no owner** `findOrCreateUser` inserted the new user, then counted rows in a separate statement, and promoted to owner only when the count was exactly 1. Two people signing in at the same moment interleave at the `await` boundaries, so both can observe a table that already holds two rows. Neither promotes itself, and the instance ends up with no owner and nobody able to administer it. The guard now lives inside the UPDATE, so the check and the write are one statement. This also removes the `count(*)` projection entirely. Note the condition tests for the absence of an owner rather than for a single user row. The two agree on a fresh install, and only the former is decidable in a single statement. **The database handle was never closed** Nothing disposed the database, so a graceful shutdown left the SQLite handle open with its journal un-finalized. `closeDbClient` is registered first in the disposer list so it runs last, after everything that might still query during shutdown. Adds a regression test that fails on the previous implementation (it produced zero owners, not two). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for tale#605. No behaviour change, no new dependency, still SQLite only. `createDbClient` now returns a `HeadplaneDb` — the drizzle client, the tables it can execute against, its dialect, and a `dispose()`. Callers reach tables through the bundle instead of importing the schema module directly, so the schema travels with the connection that can execute it rather than being fixed at import time. That is the single thing standing between here and a second dialect. `auth.ts` and `hp-agent.ts` unpack it once (`const { client: db, tables } = opts.db`), which keeps every query body identical apart from `users` becoming `tables.users`. No new layer, no indirection at the call sites. `NodeSQLiteDatabase` no longer appears outside `db/client.server.ts`, so adding Postgres later touches the client module and the schema, not the query code. `wrapSqliteClient` is exported so the unit test helper builds in-memory databases through the same path the application uses, rather than a parallel one. The dispose logic moves inside the bundle, replacing the standalone `closeDbClient` added in the previous commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
No issue link: both were found while scoping tale#605 rather than reported, and this branch does not close it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for tale#605, and a small feature on its own: the database can now live somewhere other than `server.data_path`, which is useful when data_path sits on a volume chosen for cache-like data. Omitting the block reproduces the historical location exactly, so upgrading changes nothing and no existing database moves. `createDbClient` now takes a resolved `DatabaseConfig` discriminated union rather than a bare path, so adding a dialect adds a member instead of changing the signature. Scoped to `type: "sqlite"` deliberately. Accepting `"postgres"` here while the client cannot open one would ship config that validates and then fails at runtime; the union widens in the PR that can actually honour it. Note `server.database.path` is a plain `string`, not `string.lower` like the older path fields. That keyword rewrites the value it validates, so a path containing a capital letter silently resolves somewhere else on a case-sensitive filesystem. There is a test pinning this. The same issue affects `data_path` and the other `string.lower` paths today — happy to open a separate issue for that if you'd like it fixed. Adds docs/configuration/database.md, covering the move procedure and what is actually lost if the database is (the answer is roles and sessions, recoverable but tedious). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the implementation half of tale#605. SQLite stays the default; omitting `server.database` changes nothing for an existing install. **A design concession I want to flag before you read the diff.** The plan was for the query layer to be typed against a union of the two dialects' clients, so both would be checked by the compiler. That does not work here. TypeScript cannot call a method on a union of generic signatures — every `.select()` and `.insert()` fails with "none of those signatures are compatible with each other". The only union that compiles is one containing `any`, which would erase type checking from every query in the codebase. So queries stay typed against the SQLite client and the PostgreSQL client is cast to it once, in `createPostgresClient`. The alternative was a repository port with a separate implementation per dialect, which is the large refactor you have said you do not want, for a three-table schema. What makes the cast safe is not the type system but `tests/integration/db`, which runs the same query code against both engines: SQLite in memory and a real PostgreSQL 17 under testcontainers. 25 tests, covering the divergences that types would not have caught anyway — timestamps surfacing as `Date` from both an integer epoch and a timestamptz, JSON round-tripping as an object from both text and jsonb, nullable columns, upserts, and the owner bootstrap. The API surface those queries use is identical on both dialects, and I checked that none of the SQLite-only methods (`.get()`, `.all()`, `.run()`) appear in shared query code. The cast is documented where it happens. **Owner bootstrap under READ COMMITTED.** The single-statement guard added earlier is atomic under SQLite's serialized writes but not under PostgreSQL: two concurrent first logins update different rows, take no conflicting row locks, both see no owner, and both commit. The PostgreSQL schema carries a partial unique index on `role = 'owner'` as the real guarantee, with a test asserting it bites. It is created unconditionally because PostgreSQL support is new — no existing database could already hold two owners. The SQLite schema deliberately does not get it: that migration would fail to apply on any install where the race has already fired. Also: the migrator differs by dialect (SQLite synchronous, PostgreSQL returning a promise), and a failed migration drains the pool before rethrowing so a failed start does not hang the process. Connection strings are redacted before they reach the log. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Finishes tale#605. Without this, switching `server.database.type` silently starts you over: every user is recreated on next sign-in with the default role, and whoever signs in first becomes the owner. The docs previously said as much, which is a poor answer for anyone with roles worth keeping. pnpm exec tsx scripts/db-copy.ts \ --from /var/lib/headplane/hp_persist.db \ --to postgres://headplane@10.0.0.5:5432/headplane Carries users, roles, Headscale links, live sessions and cached node info. Signed-in users stay signed in, since sessions come across and the cookie secret is unchanged. Expired sessions are dropped rather than copied. Three things it refuses to do quietly: - Copying into a target that already holds users, which would otherwise fail partway through on a primary key collision and leave it half-written. `--allow-nonempty` opts in. - Copying a source with more than one owner. PostgreSQL enforces a single owner via the partial unique index, and a constraint violation midway is a much worse message than naming the offending accounts up front. Reachable on a database hand-repaired around the owner-bootstrap race. - Printing the connection string with its password in it. `--dry-run` reports what would move without writing. Kept as a standalone script rather than a subcommand so it adds nothing to the server bundle or the startup path. It exports `copyDatabase` so the tests drive it directly; six integration tests run it against real PostgreSQL, and I also ran the documented command end to end to check the CLI wrapper, the refusal paths and their exit codes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
ℹ️ No critical issues — this is a clean, well-documented feature with strong cross-dialect test coverage. One inline nit and a couple of minor observations below.
Reviewed changes
- Atomic owner bootstrap (
fix(auth)) —findOrCreateUsernow promotes the first user with a single conditionalUPDATEguarded bynot exists, closing the concurrent-first-login race that could leave an instance ownerless, andHeadplaneDb.dispose()releases the SQLite handle (or drains the pg pool) on shutdown. HeadplaneDbbundle (refactor(db)) —createDbClientreturns{ client, tables, dialect, dispose };auth.tsandhp-agent.tsunpack it once, so query bodies only changeusers→tables.users.server.databaseconfig (feat(config)) — new block for sqlitepath/ postgresurlorhost/name/user,ssl_mode,max_connections; omitting it preserves the historical<data_path>/hp_persist.db.- PostgreSQL backend (
feat(db)) —schema.postgres.ts, migrations underdrizzle/postgres/, and a partial unique index enforcing a single owner. - SQLite→Postgres copy helper (
feat(db)) —scripts/db-copy.tswith--dry-run/--allow-nonempty, plus contract and copy test suites running against real PostgreSQL 17.
ℹ️ Concurrent first login can surface a 500 on PostgreSQL
The single-owner guarantee is correct: the partial unique index on role = 'owner' makes two owners impossible, and the conditional UPDATE handles the SQLite case. The remaining rough edge is that under a genuinely simultaneous first pair of logins on PostgreSQL, the NOT EXISTS guard evaluates before the other racer has committed, so both UPDATEs attempt to promote — and the index makes the loser throw 23505, which findOrCreateUser does not catch. The result is a one-off failed login for one of the two users, not data corruption; they retry and become a member. This is a once-per-install-lifetime race, so leaving it as-is is defensible, but it is worth a conscious decision.
Technical details
# Losing owner-bootstrap racer errors on PostgreSQL
## Affected sites
- app/server/web/auth.ts — the conditional owner-promotion UPDATE in findOrCreateUser does not tolerate a unique_violation losing the race
## Required outcome
- Decide whether a concurrent-first-login should surface as a 500 (current) or resolve cleanly (e.g. catch 23505 and treat it as "someone else became owner")
## Open questions for the human
- Is failing the losing racer acceptable given the race requires two simultaneous first logins on a brand-new instance?
ℹ️ Nitpicks
scripts/db-copy.ts--dry-runstill runspgMigrateon the target and creates the tables before short-circuiting, so the "without writing anything" wording indocs/configuration/database.mdis slightly inaccurate — it writes schema, just no rows.
DeepSeek Pro (free via Pullfrog for OSS) | 𝕏
| * be discovered by running against that dialect. | ||
| */ | ||
| type Exact<A extends B, B extends C, C = A> = A; | ||
| export type AssertUserRowsMatch = Exact<typeof postgresSchema.users.$inferSelect, HeadplaneUser>; |
There was a problem hiding this comment.
AssertUserRowsMatch only pins users. The HeadplaneTables doc comment (lines 33–36) says it pins "the two schemas" to identical row shapes, but authSessions and hostInfo are only guarded by the contract tests — a drift in either (e.g. authSessions.expires_at inferring string instead of Date) compiles clean and fails only at runtime. Consider adding equivalent Exact assertions for AuthSessionRecord and HostInfoRecord (both already exported from schema.ts), or narrowing the doc comment.

Closes #605.
Adds PostgreSQL as an alternative to the embedded SQLite database. SQLite stays
the default and the zero-config path. Omitting
server.databasechangesnothing for an existing install.
Six commits, each reviewable on its own:
fix(auth): two bugs found while scoping this, unrelated to PostgreSQL.Concurrent first logins could leave the instance with no owner (the count
ran as a statement separate from the insert, so both racers saw two rows and
neither promoted itself), and nothing ever closed the database handle. The
regression test fails on the old code.
refactor(db):createDbClientreturns the connection bundled with itsschema and dialect.
auth.tsandhp-agent.tsunpack it once, so every querybody is unchanged apart from
usersbecomingtables.users.docs(changelog)feat(config):server.database, which also lets the SQLite file livesomewhere other than
data_path.feat(db): the PostgreSQL backend, migrations, and the contract tests.feat(db): a SQLite -> PostgreSQL copy helper.One thing to look at first
The plan was for the query layer to be typed against a union of both dialects'
clients. That does not compile: TypeScript cannot call a method on a union of
generic signatures, and every
.select()fails with "none of those signaturesare compatible with each other". The only union that compiles contains
any,which would erase type checking from every query in the codebase.
So queries stay typed against the SQLite client and the PostgreSQL client is
cast to it once, in
createPostgresClient, with the reasoning written at thedefinition. The alternative was a repository port with one implementation per
dialect. The large refactor you have said you do not want, for a three-table
schema.
What makes the cast safe is
tests/integration/db, which runs the same querycode against both engines: SQLite in memory, and real PostgreSQL 17 under
testcontainers (already a devDependency, already used by the docker and dex
suites). 31 tests, covering the divergences types would not have caught anyway.
Timestamps surfacing as
Datefrom both an integer epoch and atimestamptz,JSON round-tripping as an object from both
textandjsonb, nullable columns,upserts, ownership transfer.
I also checked that no SQLite-only method (
.get(),.all(),.run()) appearsin shared query code, and said so in a comment where the cast lives.
If you would rather have compiler-proven safety than tested safety, say so. But
that means the repository port, and I did not want to spring that on you.
Owner bootstrap under READ COMMITTED
The single-statement guard from commit 1 is atomic under SQLite's serialized
writes. It is not under PostgreSQL: two concurrent first logins update
different rows, take no conflicting row locks, both see no owner, and both
commit. The PostgreSQL schema carries a partial unique index on
role = 'owner'as the real guarantee, with a test asserting it actually rejects a second owner.
It is created unconditionally because PostgreSQL support is new. No existing
database could already hold two owners. SQLite deliberately does not get it:
that migration would fail to apply on any install where the race already fired.
Notes
drizzle/*migrations are untouched; PostgreSQL lives indrizzle/postgres/.promise). A failed migration drains the pool before rethrowing so a failed
start does not hang the process.
server.database.password_pathfollows the existing_pathsecret convention.pg.docs/configuration/database.mdis new. The database was undocumented.Testing
pnpm run typecheck,pnpm run lintcleanpnpm test:unit-> 210 passingpnpm vitest run --project integration:db-> 31 passing against PostgreSQL 17Not included
docs/NixOS-options.mdneeds regenerating fromnix/options.nix(I added theserver.databasesubmodule there). It is built bynixosOptionsDocand I do nothave nix available, so I left the generated file alone rather than hand-editing
build output. Happy for you to regenerate, or I can if you tell me the command
you use.