Skip to content

Commit 43dd1a7

Browse files
committed
feat: require pgmq 1.5.1+ and remove compatibility layer (#301)
## Summary This PR removes the pgmq 1.4.x compatibility layer and requires pgmq 1.5.1 or higher. It eliminates deprecated functions, removes backported SQL code, and leverages new PGMQ features including message headers support. **⚠️ BREAKING CHANGE:** This version requires pgmq 1.5.0 or higher and will NOT work with pgmq 1.4.x. ## Changes ### PGMQ Version Requirement - **Minimum version:** pgmq 1.5.0+ (tested with 1.5.1) - **Migration guard:** Added compatibility check in migration that fails early with clear error message if pgmq < 1.5.0 is detected - **Affected packages:** `@pgflow/core`, `@pgflow/edge-worker` ### Removed Compatibility Layer #### Backported Functions Removed - **`pgflow.read_with_poll`** (68 lines) - Removed backport, now uses native `pgmq.read_with_poll` directly - **`pgflow.create_realtime_partition`** - Removed deprecated partitioning function - **Test file removed:** `supabase/tests/realtime/create_realtime_partition.test.sql` (83 lines) #### Function Updates - **`pgflow.set_vt_batch`** - Updated to return `headers` column from pgmq 1.5.0+: - Changed return type from `SETOF` to `TABLE` with explicit column definitions - Now returns: `msg_id`, `read_ct`, `enqueued_at`, `vt`, `message`, **`headers`** - Added comprehensive pgTAP test for headers handling ### Migration **File:** `pkgs/core/supabase/migrations/20251102201302_pgflow_upgrade_pgmq_1_5_1.sql` The migration includes: 1. **Compatibility check** - Verifies `pgmq.message_record` has `headers` column 2. **Clear error message** - Guides users to upgrade pgmq if version is incompatible 3. **Function replacement** - Drops and recreates `set_vt_batch` with new signature ### Code Updates #### TypeScript - **`PgflowSqlClient.ts`** - Changed queue polling to use `pgmq.read_with_poll` instead of `pgflow.read_with_poll` - **`Queue.ts`** - Updated queue reading logic - **Database types** - Regenerated to reflect new function signatures and removed deprecated functions - **Test fixtures** - Added `headers` field to message record types #### SQL Tests Updated 11 test files to remove calls to deprecated functions: - Removed `pgflow.create_realtime_partition()` calls from all realtime tests - Simplified test setup by using native PGMQ functions - Added new test: `supabase/tests/set_vt_batch/headers_handling.test.sql` ### Documentation - [**README.md**](http://README.md) - Enhanced documentation with: - Clarified root vs dependent map step behavior - Improved formatting and examples - Added edge case documentation - Better explanation of map step input handling - **BREAKING CHANGE notice** in changeset explaining: - Version requirement - Migration path for Supabase users - Self-hosting upgrade requirements ### Migration Management Skill - Updated `.claude/skills/migration-management/SKILL.md` with additional troubleshooting guidance ## Migration Guide ### For Supabase Users Recent Supabase versions include pgmq 1.5.0+ by default. Simply upgrade pgflow: ```bash # Upgrade will apply the migration automatically pnpm nx migrate @pgflow/core ``` The migration will verify pgmq compatibility and fail with a clear message if your pgmq version is too old. ### For Self-Hosted Users **Before upgrading pgflow:** 1. Verify your pgmq version: 2. If `extversion < 1.5.0`, upgrade pgmq first: 3. Then upgrade pgflow: ## Benefits - **Simplified codebase:** Removed 200+ lines of compatibility code - **Native PGMQ features:** Direct access to new pgmq 1.5.0+ capabilities - **Message headers:** Support for metadata propagation (foundation for future features) - **Cleaner tests:** Removed deprecated function calls across test suite - **Better error messages:** Migration fails early with actionable guidance ## Verified Fixes This PR verifies that upstream issues in Supabase Realtime have been resolved: - supabase/realtime#1369 - The Janitor's 10-minute startup delay that caused silent failures with `realtime.send()` after database resets has been fixed in recent Supabase versions - With the fix in place, pgflow no longer needs the `create_realtime_partition` workaround function that manually created partitions before tests - Removal of `pgflow.create_realtime_partition` and its test file confirms the upstream fix is stable and working as expected ## Testing - All pgTAP tests updated and passing - New test added for headers handling - Migration tested on both compatible (1.5.1) and incompatible (1.4.4) versions - Edge worker integration tests updated with headers support ## Dependencies - **Requires:** PR #300 (PostgreSQL 17 and Supabase CLI 2.54.11 upgrade) - **Breaking:** Incompatible with pgmq < 1.5.0 ## Related Files **Changed:** - 37 files changed, 629 insertions(+), 926 deletions(-) - Net reduction of ~300 lines of code **Key files:** - Migration: `pkgs/core/supabase/migrations/20251102201302_pgflow_upgrade_pgmq_1_5_1.sql` - Schema: `pkgs/core/schemas/0110_function_set_vt_batch.sql` - Client: `pkgs/core/src/PgflowSqlClient.ts` - Worker: `pkgs/edge-worker/src/queue/Queue.ts`
1 parent 38d9434 commit 43dd1a7

File tree

55 files changed

+663
-1068
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+663
-1068
lines changed

.changeset/pgmq-version-bump.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
'@pgflow/core': minor
3+
'@pgflow/edge-worker': minor
4+
---
5+
6+
BREAKING CHANGE: This version requires pgmq 1.5.0 or higher and will NOT work with pgmq 1.4.x.
7+
8+
The code now depends on schema changes introduced in pgmq 1.5.0 (specifically the headers column in message_record type). The compatibility layer that allowed pgflow to work with pgmq 1.4.x has been removed.
9+
10+
If you are using Supabase, pgmq 1.5.0+ is included by default in recent versions. If you are self-hosting, you must upgrade pgmq to version 1.5.0 or higher before upgrading pgflow.

0 commit comments

Comments
 (0)