Skip to content

Phase 0: PostgreSQL 16 + Apache AGE 1.6.0 + pgvector 0.8.2 compatibility spike - #1

Merged
noelsaw1 merged 4 commits into
mainfrom
marathon/reflex-hybrid-analytics-starter-2026-07-21
Jul 21, 2026
Merged

Phase 0: PostgreSQL 16 + Apache AGE 1.6.0 + pgvector 0.8.2 compatibility spike#1
noelsaw1 merged 4 commits into
mainfrom
marathon/reflex-hybrid-analytics-starter-2026-07-21

Conversation

@noelsaw1

Copy link
Copy Markdown
Contributor

Summary

Phase 0 of the Reflex Hybrid Analytics Starter build plan — the PostgreSQL 16 + Apache AGE + pgvector compatibility spike that gates everything downstream.

  • Multi-stage docker/postgres/Dockerfile building PostgreSQL 16.14 + AGE 1.6.0 + pgvector 0.8.2 from source-pinned, SHA-verified checkouts.
  • Init SQL (extensions, least-privilege roles, healthcheck) + smoke SQL for relational / vector / AGE / combined.
  • Python psycopg/SQLAlchemy smoke tests (tests/test_database_smoke.py), scripts/verify-extensions, compose.test.yaml, a minimal database-image.yml CI workflow, and docs/architecture/database.md + docs/upgrade-matrix.md.
  • The marathon plan (PROJECT/2-WORKING/PROJECT-marathon/) that produced this was QA'd to Approved over 4 headless-Codex relay rounds before firing.

Result — honest status: 3/4 gate, one known limitation

Check Result
Image builds
age + vector extensions coexist in one PG16 DB ✅ (age 1.6.0, vector 0.8.2)
Container starts healthy
Relational smoke (CRUD + rollback)
Vector smoke (nearest-neighbour ordering)
AGE pooled-connection re-init (the plan's #1 named risk) passestest_age_pool_checkout_reinitializes_session
Combined vector→graph query ❌ fails

The one failure (test_combined_vector_candidate_feeds_curated_age_query, and the corresponding age-smoke.sql/combined-smoke.sql) is a known AGE limitation, not a compatibility gap: cypher()'s third argument must be a literal, so Cypher node labels / relationship types can't be bind parameters. The generated query tried to parameterize them. Fix is a curated Cypher template with literal labels — already the approach the plan calls for; not yet applied here.

Test plan

  • docker compose -f compose.test.yaml build postgres — succeeds
  • docker compose -f compose.test.yaml up -d postgres — healthy
  • psql -c '\dx' — confirms age 1.6.0 + vector 0.8.2 both present
  • docker/postgres/test/relational-smoke.sql — passes
  • docker/postgres/test/vector-smoke.sql — passes
  • docker/postgres/test/age-smoke.sql — fails (known limitation above)
  • docker/postgres/test/combined-smoke.sql — fails (same root cause)
  • uv run pytest -m 'database or integration' — 3 passed, 1 failed (same known failure)

Next

Rewrite the combined query's Cypher without parameterized labels, re-run the gate for a clean 4/4, then release Phase 1.

🤖 Generated with Claude Code

noelsaw1 and others added 4 commits July 21, 2026 00:42
7-phase serial chain (p0→p6) with per-phase briefs, swarm/concurrency
analysis, and a Phase-0 swarm-preflight contract. Reviewed to Approved
via a 4-round headless Codex relay (all blockers resolved): phase
write-sets widened to match briefs, pytest gate standardized, swarm
analysis corrected to an honest post-P0 per-phase decomposition, and
p0's scope widened to permit its success + failure evidence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Built via a headless Codex marathon turn (autonomously fired, per
operator's prior authorization), then hand-verified against the exit
gate: image builds; both extensions coexist in one PostgreSQL 16
database (age 1.6.0, vector 0.8.2); the container starts healthy.

Gate result: 3/4 pytest markers pass, including the plan's single
highest-risk item — an AGE session survives connection-pool checkout
and reinitializes correctly (test_age_pool_checkout_reinitializes_session).

The remaining failure (test_combined_vector_candidate_feeds_curated_age_query)
and the corresponding docker/postgres/test/{age,combined}-smoke.sql are
a known limitation, not a compatibility gap: AGE's cypher() forbids a
parameterized 3rd argument, so labels/relationship types can't be bind
params — a curated Cypher template is needed, per the plan's own
guidance on this exact restriction. Next: rewrite that one query
without parameterized labels and re-run the gate for a clean pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@noelsaw1
noelsaw1 merged commit 7c1ce8e into main Jul 21, 2026
1 check failed
noelsaw1 added a commit that referenced this pull request Jul 21, 2026
phases/reflex-hybrid-analytics-starter-p0--p0/{RELAY.md,ESCALATION.md}
were auto-committed by marathon-drive.sh's own turn protocol and
landed in main via PR #1 unintentionally — they're build-tooling
transcripts, not application deliverables. Untrack them and ignore
phases/, relay-system/, and .tick/ going forward so future marathon
phases don't repeat this.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
noelsaw1 added a commit that referenced this pull request Jul 21, 2026
…ests

The PR #1 merge landed Phase 0 at an honest 3/4 pytest pass with one
known failure. Closing it out surfaced four real, independently
confirmed AGE/SQLAlchemy findings (not one, as originally scoped):

1. SQLAlchemy's text() treats any ":word" as its own bind-parameter
   marker, colliding with Cypher's ":Label" syntax. Fixed by escaping
   literal colons (\:Document).
2. cypher()'s 1st argument (graph name) must be a literal in the query
   text, never a bind parameter of any kind -- AGE resolves it at parse
   time to look up label/table OIDs, before a genuine protocol param is
   bound. create_graph()/drop_graph() are plain functions and accept a
   bind param fine; only cypher()'s own graph-name argument doesn't.
3. cypher()'s 3rd argument (its native parameter map) must be a real
   Postgres Param/Const, not a value computed per-row in the same
   statement (a CROSS JOIN LATERAL column, or even a non-correlated
   scalar subquery). Fixed in Python with two round trips: fetch the
   candidate client-side, pass it as a genuine top-level bind param.
   Fixed in SQL with PL/pgSQL EXECUTE format() -- genuine dynamic SQL,
   exactly what PROJECT.md's own database section anticipated.
4. count(*) aggregated directly over a cypher() result (even via a CTE)
   is unreliable for graph data created earlier in the same session --
   returns 0 rows where an identical non-aggregate SELECT correctly
   returns 1. Root cause not fully isolated; PL/pgSQL's aggregate-into-
   variable form is reliable and used throughout instead.

Findings 2 and 3 both bear on PROJECT.md Phase 5's "one CTE, no
round-trip" bridge design -- verify there whether that goal is
achievable as stated before committing to it.

Also fixed an unrelated pre-existing bug in scripts/verify-extensions:
every psql call runs INSIDE the container via `compose exec`, but was
passed the HOST-mapped port (55432) instead of the container's own
(5432) -- meaning verify-extensions could never have passed as
originally written, independent of any AGE-level fix.

Also fixed age-smoke.sql's CREATE: three comma-separated patterns
don't reuse nodes from earlier patterns in the same clause (Cypher
CREATE never matches existing nodes), so the KNOWS edge was
(harmlessly, but confusingly) connecting two accidental duplicate
nodes rather than the two named people. Chained CREATE clauses with
shared variables fix this.

Verified: uv run pytest -m 'database or integration' -> 4 passed.
./scripts/verify-extensions -> all 4 SQL smoke tests pass, extension
check passes, "Phase 0 extension smoke checks passed."

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant