diff --git a/docs/ext-compat.md b/docs/ext-compat.md new file mode 100644 index 00000000..47346b91 --- /dev/null +++ b/docs/ext-compat.md @@ -0,0 +1,119 @@ +# pg_durable Extension Compatibility + +Compatibility evaluation of pg_durable against PostgreSQL extensions available on [Azure Database for PostgreSQL – Flexible Server](https://learn.microsoft.com/en-us/azure/postgresql/extensions/concepts-extensions-versions), plus Citus. + +## Key pg_durable characteristics affecting compatibility + +| Characteristic | Detail | +|---|---| +| **Schemas** | Owns `df` and `duroxide` schemas; no public-schema pollution | +| **Background worker** | Single BGW (`pg_durable_worker`), requires `shared_preload_libraries` and a superuser role | +| **SQL execution** | BGW executes workflow SQL via **sqlx** (async TCP connections), **not** SPI. Two connection contexts: (1) worker role for df/duroxide state management, (2) submitting user's role for workflow SQL execution | +| **Hooks** | None — no executor, planner, utility, or parse hooks | +| **RLS** | Enabled (not forced) on `df.instances`, `df.nodes`, `df.vars` | +| **Operators** | `~>`, `\|=>`, `&`, `\|`, `?>`, `!>`, `@>` — all on `text` operands | +| **Custom types / AMs** | None | +| **GUCs** | `pg_durable.worker_role`, `pg_durable.database` (postmaster context) | +| **LISTEN/NOTIFY** | Used by duroxide long-polling in BGW (own channels) | +| **WAL / replication** | No custom WAL resources; standard heap tables only | + +## Compatibility ratings + +- **Full** — Works out of the box, no known issues. +- **Full\*** — Works, but a specific interaction is worth knowing about (see notes). +- **Partial** — Functional, but some features don't compose or need workarounds. +- **Risky** — Likely to break or require significant integration work. + +## Compatibility table + +| Extension | HorizonDB | Compat | What's most likely to go wrong | Fix effort | +|---|---|---|---|---| +| **address_standardizer** | Yes | Full | Nothing — pure data-type/function extension, separate schema. | — | +| **address_standardizer_data_us** | Yes | Full | Nothing — geocoding reference data, no runtime interaction. | — | +| **age** (Apache AGE) | Yes | Full | Nothing — graph database extension adding `ag_catalog` schema, `agtype` type, and Cypher query language via executor hooks. pg_durable has no hooks, so no conflict. BGW sqlx connections load AGE if it's in `shared_preload_libraries`, so `df.sql()` can execute `cypher()` graph queries. | — | +| **amcheck** | Yes | Full | Nothing — read-only B-tree verification functions. | — | +| **anon** (PostgreSQL Anonymizer) | No | Full* | anon's dynamic masking uses security labels and views. No conflict, but masking rules won't apply inside BGW sqlx connections unless the masking role/policy is explicitly configured there. | Low — configure anon policies for the BGW role if needed. | +| **azure** | Yes | Full | Nothing — Azure platform integration extension (managed identity, metrics). Internal extension loaded via `shared_preload_libraries`, no schema or hook conflicts. | — | +| **azure_ai** | Yes | Full | Nothing — callable functions for Azure AI services. Can be used inside `df.sql()`. | — | +| **azure_local_ai** | No | Full | Nothing — local ML inference functions. | — | +| **azure_storage** | Yes | Full | Nothing — Azure Blob access functions. Can be used inside `df.sql()`. | — | +| **bloom** | Yes | Full | Nothing — alternative index access method, transparent to pg_durable. | — | +| **btree_gin** | Yes | Full | Nothing — adds GIN operator classes for scalar types. | — | +| **btree_gist** | Yes | Full | Nothing — adds GiST operator classes for scalar types. | — | +| **Citus** | No | Partial | **df.\* and duroxide.\* tables must NOT be distributed.** Citus shards distributed tables across worker nodes, but the pg_durable BGW runs only on the coordinator and expects local tables. Distributing `df.instances` or `df.nodes` will break instance tracking. RLS policies may not propagate correctly to Citus workers. However, workflow SQL (`df.sql()`) **can** query distributed tables normally. Also: Citus's `citus.main_db` BGW will compete for resources. | Medium — ensure df/duroxide schemas are excluded from distribution (`SELECT citus_add_local_table_to_metadata()`). No code changes needed if tables stay local. | +| **citext** | Yes | Full | Nothing — case-insensitive text type, orthogonal. | — | +| **cube** | Yes | Full | Nothing — geometric data type, no runtime overlap. | — | +| **dblink** | Yes | Full | Nothing — cross-database queries. Can be called from `df.sql()`. | — | +| **dict_int** | Yes | Full | Nothing — text search dictionary. | — | +| **dict_xsyn** | Yes | Full | Nothing — text search dictionary. | — | +| **earthdistance** | Yes | Full | Nothing — distance calculation functions (depends on cube). | — | +| **file_fdw** | Yes | Full* | Foreign data wrapper for server-side flat files. `df.sql()` can query file_fdw foreign tables. The BGW executes workflow SQL as the submitting user, so that user must have access to the foreign server definition. File paths must be accessible to the PostgreSQL server process. | Low — ensure foreign server/user mappings exist for roles that submit workflows. | +| **fuzzystrmatch** | Yes | Full | Nothing — string matching functions. | — | +| **hstore** | Yes | Full | Nothing — key-value data type. | — | +| **hypopg** | Yes | Full* | hypopg uses planner hooks to inject hypothetical indexes. Since pg_durable's BGW uses sqlx (separate connections), hypothetical indexes will **not** affect workflow SQL execution — they only apply to the session that created them. This is expected behavior, not a bug. | — | +| **intagg** | Yes | Full | Nothing — integer aggregation functions. | — | +| **intarray** | Yes | Full | Nothing — integer array operators. Defines `&` and `\|` on `int[]`, not on `text` — no operator conflict. | — | +| **isn** | Yes | Full | Nothing — ISN data types (ISBN, ISSN, etc.). | — | +| **lo** | Yes | Full | Nothing — large object management trigger. | — | +| **ltree** | Yes | Full | Nothing — hierarchical label type. | — | +| **orafce** | No | Full | Nothing — Oracle compatibility functions and packages. | — | +| **orion_storage** | Yes | Full | Nothing — Azure internal storage-layer extension loaded via `shared_preload_libraries`. No schema or hook conflicts with pg_durable. | — | +| **pageinspect** | Yes | Full | Nothing — low-level page inspection functions. | — | +| **pg_availability** | Yes | Full | Nothing — Azure internal availability monitoring extension loaded via `shared_preload_libraries`. No schema or hook conflicts with pg_durable. | — | +| **pg_buffercache** | Yes | Full | Nothing — read-only shared buffer introspection. | — | +| **pg_cron** | Yes | Full* | Both use background workers and require `shared_preload_libraries`. No direct conflict — different BGWs, different schemas, different GUCs. `pg_cron` can schedule `df.start()` calls, making them complementary. Minor concern: both BGWs maintain their own connection pools, so under very high load they compete for `max_connections` slots. | None — works well as a combination. | +| **pg_diskann** | Yes | Full | Nothing — DiskANN-based vector index access method. Transparent to pg_durable; `df.sql()` queries using DiskANN indexes work normally via the standard planner/executor in BGW sqlx connections. | — | +| **pg_failover_slots** | No | Full | Nothing — replication slot management, different layer. | — | +| **pg_freespacemap** | Yes | Full | Nothing — FSM inspection functions. | — | +| **pg_fts** | Yes | Full | Nothing — Azure enhanced full-text search extension. No hook or schema conflicts. `df.sql()` can execute full-text search queries. | — | +| **pg_hint_plan** | No | Full* | pg_hint_plan uses planner hooks to inject query hints. Since pg_durable's BGW executes SQL via **sqlx** (regular TCP connections, not SPI), pg_hint_plan **will** be loaded in those connections (if in `shared_preload_libraries` or `session_preload_libraries`). Comment-based hints embedded in `df.sql()` queries **will work**. The BGW executes workflow SQL as the submitting user, so `pg_hint_plan.hints` table-based hints **do apply** if configured for that user. The only limitation is that the BGW session is not the user's interactive session, so session-level hint state does not carry over. | — | +| **pg_partman** | No | Full* | Works. Users could partition `df.instances` by `created_at` for large deployments. pg_partman's BGW (for auto-maintenance) coexists with pg_durable's BGW. Minor consideration: partitioning `df.nodes` requires care since the BGW queries by `instance_id`. | Low — ensure partition keys match BGW query patterns. | +| **pg_partman_bgw** | Yes | Full* | Background worker component of pg_partman for automatic partition maintenance. Both pg_durable and pg_partman_bgw run independent BGWs via `shared_preload_libraries` that coexist without conflict. Minor concern: both maintain connection pools competing for `max_connections` slots under heavy load. Users could partition `df.instances` by `created_at` for large deployments; partitioning `df.nodes` requires care since the BGW queries by `instance_id`. | Low — ensure partition keys match BGW query patterns. Monitor resource usage under high concurrency. | +| **pg_prewarm** | Yes | Full | Nothing — buffer prewarming utility. | — | +| **pg_qs** | Yes | Full* | Azure internal query store extension using executor hooks to capture query statistics. No hook conflict with pg_durable. BGW sqlx connections are regular client connections — workflow SQL queries **will** be captured by query store, useful for monitoring workflow performance. | — | +| **pg_repack** | Yes | Full* | Works. Can repack `df.instances` and `df.nodes` to reclaim space. Takes a brief exclusive lock during the swap phase, which may momentarily block `df.start()` or BGW updates. | None — standard operational consideration. | +| **pg_squeeze** | No | Full* | Same as pg_repack — concurrent table reorganization with brief lock at swap. | None. | +| **pg_stat_statements** | Yes | Full* | Uses executor hooks to capture statistics. pg_durable has no hooks, so no conflict. BGW sqlx connections are regular client connections — their queries **will** appear in `pg_stat_statements`. This is actually useful for monitoring workflow SQL performance. Workflow SQL queries appear under the submitting user's role; df state management queries appear under the worker role. | — | +| **pg_surgery** | Yes | Full | Nothing — administrative tool for repairing corrupted heap tuples. No runtime interaction with pg_durable. | — | +| **pg_trgm** | Yes | Full | Nothing — trigram-based text similarity functions and GIN/GiST ops. | — | +| **pg_visibility** | Yes | Full | Nothing — visibility map inspection functions. | — | +| **pgaudit** | Yes | Full* | pgaudit uses executor/utility hooks to log SQL statements. No hook conflict with pg_durable. BGW sqlx connections are regular client connections, so pgaudit **will** audit workflow SQL execution (logged under the submitting user's role). df state management is logged under the worker role. This is a feature, not a bug — full audit trail of workflow activity. | — | +| **pgcrypto** | Yes | Full | Nothing — cryptographic functions. Can be used inside `df.sql()`. | — | +| **pglogical** | No | Partial | pglogical performs logical replication of DML changes. `df.instances`, `df.nodes`, and duroxide state tables will be replicated to subscribers. **Risk: if pg_durable is installed on both publisher and subscriber, the subscriber's BGW will attempt to execute replicated instances**, causing duplicate workflow execution. The `duroxide` schema tables must be replicated atomically with `df.*` tables. | Medium — disable pg_durable BGW on replicas (set `shared_preload_libraries` without pg_durable, or set an unused `pg_durable.database`). Need operational discipline. | +| **pgms_stats** | Yes | Full | Nothing — Azure internal statistics collection extension loaded via `shared_preload_libraries`. No schema or hook conflicts with pg_durable. | — | +| **pgms_wait_sampling** | Yes | Full* | Azure wait event sampling extension with its own BGW loaded via `shared_preload_libraries`. No hook conflicts. pg_durable BGW wait events will be sampled — useful for diagnosing workflow execution bottlenecks. Multiple BGWs coexist without issues. | — | +| **pgrowlocks** | Yes | Full | Nothing — row-level lock inspection functions. | — | +| **pgstattuple** | Yes | Full | Nothing — tuple-level statistics functions. | — | +| **plpgsql** | No | Full | Nothing — PL/pgSQL is the default procedural language. `df.sql()` can call PL/pgSQL functions. `df.if()` condition queries return PL/pgSQL-compatible boolean-ish results. | — | +| **plv8** | No | Full | Nothing — V8 JavaScript procedural language. `df.sql()` can call PLV8 functions. | — | +| **postgis** | Yes | Full | Nothing — spatial types and functions. `df.sql()` can run spatial queries. | — | +| **postgis_raster** | Yes | Full | Nothing — raster type support for PostGIS. | — | +| **postgis_sfcgal** | Yes | Full | Nothing — SFCGAL-backed 3D geometry functions. | — | +| **postgis_tiger_geocoder** | Yes | Full | Nothing — US Census TIGER geocoder, data-only extension. | — | +| **postgis_topology** | Yes | Full | Nothing — topology type and functions for PostGIS. | — | +| **postgres_fdw** | No | Full* | Foreign data wrappers work. `df.sql()` can query foreign tables. The BGW executes workflow SQL as the submitting user, so FDW user mappings for that user apply directly. | Low — ensure FDW user mappings exist for roles that submit workflows. | +| **seg** | Yes | Full | Nothing — floating-point interval data type, no runtime overlap. | — | +| **semver** | No | Full | Nothing — semantic versioning data type. | — | +| **spi** | Yes | Full | Nothing — trigger-based utility functions (autoinc, moddatetime, etc.). No hooks or schema conflicts. Triggers fire normally on tables modified by `df.sql()`. | — | +| **sslinfo** | Yes | Full | Nothing — SSL certificate info functions. Note: BGW sqlx connections are local TCP, so `ssl_is_used()` will return false in BGW-executed queries. | — | +| **tablefunc** | Yes | Full | Nothing — crosstab and other table functions. Can be used in `df.sql()`. | — | +| **tcn** | Yes | Full | Nothing — triggered change notifications (fires NOTIFY on table changes). Uses NOTIFY on different channels than duroxide. No conflict. | — | +| **timescaledb** | No | Full* | TimescaleDB uses BGWs (for compression, retention, continuous aggregates) and planner/executor hooks. No hook conflict since pg_durable uses none. BGW sqlx connections load TimescaleDB if it's in `shared_preload_libraries`, so `df.sql()` queries against hypertables work correctly — TimescaleDB's planner hooks will fire and route queries to chunks. Multiple BGWs coexist (resource competition under heavy load). | None — both extensions' BGWs coexist. Monitor resource usage under high concurrency. | +| **tsm_system_rows** | Yes | Full | Nothing — TABLESAMPLE method. | — | +| **tsm_system_time** | Yes | Full | Nothing — TABLESAMPLE method. | — | +| **unaccent** | Yes | Full | Nothing — text search dictionary for accent removal. | — | +| **uuid-ossp** | Yes | Full | Nothing — UUID generation functions. | — | +| **vector** (pgvector) | Yes | Full | Nothing — vector data type and similarity search. `df.sql()` can run vector queries. Useful combo: durable pipelines that do embedding lookups. | — | +| **wal2json** | Yes | Full* | Logical decoding output plugin producing JSON from WAL changes. pg_durable uses standard heap tables, so DML on `df.*` and `duroxide.*` tables will appear in wal2json output. No runtime conflict. Note: if wal2json output is consumed to replay changes into another pg_durable instance, duplicate workflow execution could occur (same risk as pglogical). | Low — do not replay `df`/`duroxide` schema changes into another pg_durable-enabled database. | +| **xml2** | Yes | Full | Nothing — XML processing functions. | — | + +## Summary + +Out of all evaluated extensions, only two have meaningful compatibility considerations: + +| Extension | Issue | Severity | +|---|---|---| +| **Citus** | df/duroxide tables must stay local (not distributed) | Medium — operational constraint, no code fix needed | +| **pglogical** | Replicated instances may trigger duplicate BGW execution on subscriber | Medium — must disable BGW on replicas | + +Everything else works without issues. The core reason is that pg_durable has a minimal PostgreSQL footprint: no hooks, no custom types, no custom WAL, no custom access methods. The `df` and `duroxide` schemas are fully self-contained. The BGW uses standard PostgreSQL connections (via sqlx), which means hook-based extensions like pg_stat_statements, pgaudit, and TimescaleDB naturally interoperate with workflow SQL execution. diff --git a/docs/feature-compat.md b/docs/feature-compat.md new file mode 100644 index 00000000..304d0764 --- /dev/null +++ b/docs/feature-compat.md @@ -0,0 +1,49 @@ +# Azure PostgreSQL Flexible Server — Feature Compatibility Analysis + +**Date:** 2026-03-24 + +## Architecture Summary (relevant to compatibility) + +pg_durable is a `shared_preload_libraries` extension with a background worker (BGW). +All state lives in WAL-logged tables across two schemas (`df.*` and `duroxide.*`) in a single database. +The BGW connects back to PostgreSQL via **TCP/IP** (`127.0.0.1`) using `sqlx`—it does **not** use SPI or shared memory. +The BGW starts at `BgWorkerStartTime::RecoveryFinished` and auto-restarts after 5 seconds on failure. +Activity connections are also TCP, authenticated as the worker role (default `azuresu`) with `SET ROLE` to the submitting user. +No passwords are included in the connection string; local `trust` or `peer` authentication is assumed. + +## Compatibility Matrix + +| Feature | Compat | Risk Summary | Effort | +|---------|--------|--------------|--------| +| **Automated Backups** | Complete | All state is WAL-logged; no out-of-band files. Backup captures a fully consistent snapshot. | None | +| **Point-in-Time Recovery (PITR)** | Complete | Restores the database to a consistent mid-execution state. Duroxide replays orchestration history and retries in-flight activities (at-least-once semantics, by design). No data loss. | None | +| **High Availability (zone-redundant)** | Complete | Standby receives all WAL. On failover the new primary starts the BGW after recovery (`RecoveryFinished`). BGW reconnects to `127.0.0.1` on the new host. In-flight TCP connections die; duroxide retries activities automatically. Brief unavailability (~5 s BGW restart + failover time). | None | +| **Read Replicas** | Partial | BGW does **not** start on replicas (`RecoveryFinished` never fires on a hot standby). Read-only queries (`df.status()`, `df.result()`) work. Write calls (`df.start()`) error because the replica is read-only. This is the correct, expected behavior. | Low — document that `df.start()` is primary-only. Expose a read-only `df.status()` helper if needed. | +| **Customer-Managed Keys (CMK)** | Complete | CMK provides transparent disk-level encryption. pg_durable uses only regular heap tables and WAL; encryption is invisible to the extension. | None | +| **Major Version Upgrade** | Partial | Azure uses `pg_upgrade` internally. Extension data (`df.*`, `duroxide.*` tables, indexes, RLS policies) survives the upgrade. The main constraint is that the `.so` must be recompiled per PG major version (pgrx ABI). Azure must ship the matching `.so` for each supported version. Duroxide migrations are idempotent (`ApplyAll`), so the BGW re-initializes cleanly. | Medium — build and test the extension against each target PG major version. Verify `pg_upgrade --check` passes with pg_durable installed. | +| **Logical Replication** | Partial | `df.*` tables can be published, but replicating to a second cluster that also runs a BGW would cause duplicate activity execution and queue contention. Useful for read-only analytics replicas only. | Medium — if needed, add publication/subscription guidance and document that the target must **not** run a BGW. | +| **Connection Pooling (PgBouncer)** | Complete | The built-in PgBouncer proxy (port 6432) is on the client-facing path only. The BGW connects directly to PG (port 5432) via `127.0.0.1`, bypassing PgBouncer entirely. User sessions calling DSL functions use SPI (in-process), not pooled connections. | None | +| **Azure AD / Entra ID Auth** | Partial | The BGW connection string contains no password (`postgres://azuresu@127.0.0.1:…`). It relies on `trust` or `peer` for the loopback connection. If Azure enforces password/SCRAM for all connections (including loopback from the BGW), authentication will fail. Activity connections (`connect_as_user`) also use password-less options with just the username. | Medium — add optional password/token support to the connection builder (e.g., read `PGPASSWORD` or integrate Azure managed-identity token acquisition). | +| **Virtual Network / Private Endpoint** | Complete | BGW connects to `127.0.0.1` (loopback). No external network calls are made by the extension itself. Future HTTP activity support would need outbound rules. | None | +| **Storage Auto-grow** | Complete | Transparent to the extension. Table bloat from `duroxide.history` may accelerate storage growth under heavy workloads but is managed by normal VACUUM/autovacuum. | None | +| **Monitoring (Azure Metrics / pg_stat)** | Partial | BGW connections appear as regular backends in `pg_stat_activity`. `df.status()` provides per-instance monitoring. However, no Azure-native metrics integration exists (no custom metrics emitted to Azure Monitor). | Low — emit metrics to `pg_stat_user_tables` or a custom stats view; Azure-side integration would need a metrics extension or external exporter. | +| **Extension Allowlisting** | Blocked | pg_durable is not on the Azure PG Flexible Server allowlist today. It also requires `shared_preload_libraries` (a server-level config that demands a restart), which Azure only supports for a curated set of extensions (e.g., `pg_cron`, `pg_stat_statements`). The control file sets `superuser = true` and `trusted = false`. | High — requires Azure PG team onboarding: add to allowlist, configure `shared_preload_libraries` support, allocate the `azuresu` worker role, and configure `pg_hba.conf` for password-less loopback. | + +## Key Architectural Properties + +| Property | Value | Why it matters | +|----------|-------|----------------| +| State storage | WAL-logged heap tables only | Survives backup, PITR, HA failover, `pg_upgrade` | +| Shared memory | None (`enable_shmem_access(None)`) | No cross-process state to lose on failover | +| BGW start time | `RecoveryFinished` | Correct: starts on primary after crash recovery, does **not** start on standbys/replicas | +| BGW restart delay | 5 seconds | Brief gap after crash/failover; acceptable | +| Connection method | TCP via `sqlx` to `127.0.0.1` | Survives failover (new host, same loopback); bypasses PgBouncer; but requires local auth trust | +| Authentication | Username only, no password | Works with `trust`/`peer` `pg_hba.conf` rules; needs work for password-mandatory environments | +| Duroxide recovery | Replay from `duroxide.history` | Crash-safe; in-flight activities retried (at-least-once) | + +## Recommendations + +1. **Extension onboarding** is the critical-path blocker. Engage the Azure PG team early to add `pg_durable` to `shared_preload_libraries` support and the extension allowlist. +2. **Authentication hardening**: Add optional password / managed-identity token support to `postgres_connection_string()` and `connect_as_user()` for environments that do not allow password-less loopback. +3. **Major version upgrade testing**: Add a CI job that runs `pg_upgrade --check` (and a full upgrade) with pg_durable installed to catch ABI or catalog issues before release. +4. **Read replica documentation**: Document that `df.start()` is primary-only and that `df.status()`/`df.result()` work on read replicas for monitoring.