docs: how to adopt partitioning on an existing PostgreSQL install - #1058
docs: how to adopt partitioning on an existing PostgreSQL install#1058AlexeyShalaev wants to merge 2 commits into
Conversation
The event-delivery-log guide points at managing partitions yourself once PostgreSQL is running at volume, but an existing install has every row in events_default and attempts_default, and two things follow from that schema regardless of tool: the rows have to be moved before the partitions covering them can exist, and every retention drop takes ACCESS EXCLUSIVE because a DEFAULT partition rules out DETACH CONCURRENTLY. Adds both caveats and a worked adoption path for pg_partman and for pg-partsmith. Both were run against the current migrations on PostgreSQL 17 with eight months of rows seeded into the DEFAULT partition. The pg_partman order matters: create_parent creates its own events_default, so the existing one is renamed before registration, and partition_data_proc cannot be used before the table is in part_config. Refs hookdeck#1027
There was a problem hiding this comment.
🟡 Changes recommended
One operational statement about ACCESS EXCLUSIVE locking is materially incomplete (it blocks reads as well as writes) and should be corrected for accuracy.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR expands the PostgreSQL retention guidance in the self-hosting event/delivery log docs by documenting how to adopt monthly range partitions on an already-running Outpost database where all existing data currently lives in the DEFAULT partitions.
Changes:
- Adds a callout explaining two schema-driven caveats when adopting partition lifecycle tooling on an existing install (data remains in DEFAULT until moved; retention operations require short exclusive locking).
- Adds a worked pg_partman adoption sequence (detach/rename existing DEFAULT, register parent, move rows, then configure/schedule maintenance).
- Adds an alternative pg-partsmith recipe for managed Postgres environments where installing extensions isn’t possible.
File summaries
| File | Description |
|---|---|
| docs/content/self-hosting/guides/event-delivery-log.mdoc | Adds operator-focused guidance and recipes for adopting and maintaining Postgres partitions for retention. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| {% callout %} | ||
| **Existing rows live in the DEFAULT partition.** Every row an existing install has written is in `events_default` and `attempts_default`. Creating monthly partitions doesn't move those rows, and PostgreSQL refuses to attach a partition for a month whose rows are still in the DEFAULT partition. Adopting partitioning on a running install means moving the existing rows first, not only creating partitions going forward. | ||
|
|
||
| **Dropping a partition takes an `ACCESS EXCLUSIVE` lock on the table.** PostgreSQL allows `DETACH PARTITION ... CONCURRENTLY` only when the parent has no DEFAULT partition, and Outpost's tables always have one. Each retention drop briefly blocks writes to that table. The lock is short, a catalog change rather than a data rewrite, but it isn't lock-free. |
|
@alexluong following up on this one — you asked for it in #1027, and I know a docs PR is an easy thing to slip past a week of feature work. It's still one file and still applies cleanly to main. If the pg-partsmith half is what gives you pause, say so and I'll cut it. The two caveats and the pg_partman ordering are the part I'd want in the docs regardless, and they stand on their own without it. |
Follows up on #1027, where @alexluong asked for this as a docs PR.
The retention section already suggests managing partitions yourself with something like pg_partman once PostgreSQL is running at volume. This adds what an operator needs to know before doing that on an existing install, and a worked path for two tools.
The two caveats come from Outpost's schema, not from any tool. Every row an existing install has written lives in
events_default/attempts_default, and PostgreSQL won't attach a partition for a month whose rows are still in the DEFAULT partition, so adoption means moving rows first rather than only creating partitions going forward. And because the tables always carry a DEFAULT partition,DETACH PARTITION ... CONCURRENTLYis unavailable, so each retention drop takes a shortACCESS EXCLUSIVEon the table. Both are stated plainly so nobody plans a maintenance window around a lock-free drop that isn't.Both recipes were run against the current
internal/migrator/migrations/postgres/*.up.sqlon PostgreSQL 17, with eight months of rows seeded intoevents_default. The pg_partman one took two attempts to get right, which is why it's worth having in the docs:create_parentcreates its ownevents_default, so it collides unless the existing one is renamed first, andpartition_data_procrefuses to run before the table is inpart_config. The order in the snippet is the one that ends with the DEFAULT partition empty, 13 monthly partitions, andrun_maintenancehealthy. The pg-partsmith one uses the published image and Outpost's ownPOSTGRES_URL.Scope: one page, no nav or redirect changes, one
{% callout %}from the supported component list. Voice checked against.agents/skills/hookdeck-outpost-docs-format.I maintain pg-partsmith, which is why I was reading this code in the first place. If you'd rather the page mention only pg_partman, I'm happy to trim it to that; the two caveats and the pg_partman ordering are the part I'd most want to survive.