Temporary database for realm server tests by IanCal · Pull Request #4073 · cardstack/boxel

IanCal · 2026-02-26T13:57:10Z

Instead of storing test databases alongside the development database, this spins up a new container. It will create a template with all the migrations already applied and save this if it doesn't exist (takes a few seconds) and then will boot up a postgres container with the new data.

The immediate impact of this is that tests will no longer need to run a slow "removing test dbs" step, and tests will be a bit faster.

This is an initial step however to using templates as cached pre-seeded databases for much faster realm server tests.

…slice1

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 882cf707cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-26T14:04:52Z

packages/realm-server/tests/helpers/index.ts

+      `CREATE DATABASE ${quotePgIdentifier(database)} TEMPLATE ${quotePgIdentifier(
+        migratedTestDatabaseTemplate,
+      )}`,
+    );


Retry clone when test DB name already exists

This CREATE DATABASE ... TEMPLATE ... path fails immediately on already exists, but test DB names are still randomly generated (prepareTestDB), so long realm-server runs can hit birthday-collision duplicates and now abort instead of recovering. In environments running many tests/modules, this introduces intermittent failures; handle duplicate-name errors by regenerating a name and retrying the clone.

Useful? React with 👍 / 👎.

This matches our current setup, it can be managed at a later date

packages/realm-server/tests/scripts/prepare-test-pg.sh

github-actions · 2026-02-26T14:24:25Z

Host Test Results

1 files ± 0 1 suites ±0 2h 29m 17s ⏱️ + 55m 27s
1 896 tests +17 1 865 ✅ + 1 15 💤 ±0 0 ❌ ± 0 16 🔥 +16
1 911 runs +17 1 864 ✅ - 15 15 💤 ±0 16 ❌ +16 16 🔥 +16

For more details on these errors, see this check.

Results for commit 79ce1ce. ± Comparison against base commit 61ceac3.

This pull request removes 1 and adds 18 tests. Note that renamed tests count towards both.

Chrome ‑ Acceptance | code submode | create-file tests > when user has permissions to both test realms: new file button has options to create card def, field def, card instance, and text files

Chrome ‑ Acceptance | code submode tests > single realm: clicking "metadata" format for a non-card file shows metadata panel with JSON-API content
Chrome ‑ Acceptance | code submode tests > single realm: non-card file preview shows "metadata" format option and not "edit"
Chrome ‑ Acceptance | code submode tests > single realm: switching away from "metadata" format hides the metadata panel
Chrome ‑ Acceptance | code submode | create-file tests > when user has permissions to both test realms: can upload a file via the New menu
Chrome ‑ Acceptance | code submode | create-file tests > when user has permissions to both test realms: cancelling upload file picker does not cause errors
Chrome ‑ Acceptance | code submode | create-file tests > when user has permissions to both test realms: new file button has options to create card def, field def, card instance, text files, and upload file
Chrome ‑ Acceptance | code submode | file-tree tests > when the user lacks write permissions: file tree does not show context menu in read-only realm
Chrome ‑ Acceptance | code submode | file-tree tests: can cancel delete from file tree context menu
Chrome ‑ Acceptance | code submode | file-tree tests: can delete a file from file tree context menu
Chrome ‑ Acceptance | code submode | file-tree tests: can delete a file via right-click in file tree
…

♻️ This comment has been updated with latest results.

habdelra · 2026-02-26T18:03:28Z

I think we also use a temp DB for the isolated realm server in the matrix tests, no? don't we still need the cleanup test db's script for that?

Copilot

Pull request overview

This PR updates realm-server’s test infrastructure to use an ephemeral Postgres container seeded from a cached, pre-migrated PGDATA snapshot, avoiding per-run cleanup of test databases and reducing test startup time.

Changes:

Added scripts to build/cache a seeded Postgres data snapshot and to start/stop a dedicated test Postgres container from that snapshot.
Updated tests to clone per-test databases from a migrated template DB instead of running migrations via autoMigrate.
Removed the legacy remove-test-dbs.sh cleanup step and updated test scripts to use the new container-based flow.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
packages/realm-server/tests/scripts/wait-for-container-pg.sh	Adds readiness + SQL round-trip checks for Postgres containers.
packages/realm-server/tests/scripts/test-pg-config.sh	Centralizes container names, ports, and cache paths for test Postgres.
packages/realm-server/tests/scripts/stop-test-pg.sh	Stops/removes the test Postgres container.
packages/realm-server/tests/scripts/start-test-pg.sh	Starts the test Postgres container from a seeded PGDATA tar on tmpfs.
packages/realm-server/tests/scripts/prepare-test-pg.sh	Computes migration fingerprint; rebuilds seed snapshot if needed; starts test Postgres.
packages/realm-server/tests/scripts/create_seeded_db.sh	Builds a seeded DB cluster by running migrations and snapshotting PGDATA.
packages/realm-server/tests/scripts/boot_preseeded.sh	Container entrypoint to unpack seeded PGDATA into tmpfs and start Postgres with tuned settings.
packages/realm-server/tests/queue-test.ts	Switches to template-cloned DB setup via `createTestPgAdapter()`.
packages/realm-server/tests/helpers/index.ts	Adds template-clone DB creation logic and `createTestPgAdapter()` helper.
packages/realm-server/tests/billing-test.ts	Switches to template-cloned DB setup via `createTestPgAdapter()`.
packages/realm-server/tests/.test-pg-cache/.gitignore	Ensures the seed cache directory contents are ignored by git.
packages/realm-server/scripts/remove-test-dbs.sh	Removes the old Docker-based “drop all test_db_*” cleanup script.
packages/realm-server/package.json	Updates test scripts to run `prepare-test-pg.sh` and use the new test Postgres port.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/realm-server/tests/scripts/start-test-pg.sh

Copilot · 2026-02-26T18:07:47Z

packages/realm-server/tests/scripts/start-test-pg.sh

+while [ "$attempt" -le "$max_attempts" ]; do
+  if cid="$(start_container 2>&1)"; then
+    break
+  fi


cid="$(start_container 2>&1)" merges stderr into the container id on success if Docker prints warnings, and it also means $cid may contain an error string rather than an id. Since $cid is later passed to wait-for-container-pg.sh for docker inspect/logs, a non-id value can cause misleading failures. Capture stdout (container id) separately from stderr, and pass a stable identifier (container name or real id) to the wait/log helpers.

packages/realm-server/tests/scripts/create_seeded_db.sh

Copilot · 2026-02-26T18:07:47Z

packages/realm-server/tests/helpers/index.ts

+import { Client as PgClient } from 'pg';

 const testRealmURL = new URL('http://127.0.0.1:4444/');
 const testRealmHref = testRealmURL.href;
+const migratedTestDatabaseTemplate = 'boxel_migrated_template';


This file imports pg directly (Client as PgClient). @cardstack/realm-server doesn't list pg / @types/pg as direct deps, so this relies on a transitive dependency from another workspace package, which can break if the dependency graph changes. Add pg (and @types/pg if needed) as a direct devDependency of @cardstack/realm-server, or avoid the direct pg import by exposing this functionality via @cardstack/postgres.

Copilot · 2026-02-26T18:07:48Z

packages/realm-server/package.json

+    "test": "./tests/scripts/prepare-test-pg.sh && LOG_LEVELS=\"*=error,prerenderer-chrome=silent,pg-adapter=warn,realm:requests=warn${LOG_LEVELS:+,}${LOG_LEVELS}\" NODE_NO_WARNINGS=1 PGPORT=55436 STRIPE_WEBHOOK_SECRET=stripe-webhook-secret STRIPE_API_KEY=stripe-api-key qunit --require ts-node/register/transpile-only tests/index.ts",
+    "test-module": "./tests/scripts/prepare-test-pg.sh && LOG_LEVELS=\"*=error,prerenderer-chrome=silent,pg-adapter=warn,realm:requests=warn${LOG_LEVELS:+,}${LOG_LEVELS}\" NODE_NO_WARNINGS=1 PGPORT=55436 STRIPE_WEBHOOK_SECRET=stripe-webhook-secret STRIPE_API_KEY=stripe-api-key qunit --require ts-node/register/transpile-only --module ${TEST_MODULE} tests/index.ts",


The updated test / test-module scripts start (and potentially rebuild) a Postgres container but never stop it after the test run. This can leave a running container and an occupied port on developer machines when tests fail or are interrupted. Consider adding a trap/finally cleanup in the shell script, or chaining a ./tests/scripts/stop-test-pg.sh step (or a posttest script) to ensure the container is removed reliably.

IanCal · 2026-02-28T16:52:45Z

I think we also use a temp DB for the isolated realm server in the matrix tests, no? don't we still need the cleanup test db's script for that?

Good point, I notice we never call it from that flow so those dbs would only have been removed when someone ran the realm server tests. I'll either bring it back for now (and move it to the matrix tests) or also have the matrix tests use an ephemeral test postgres - they don't need persistence either (and ideally the whole snapshot/restore side would be pretty good there, particularly later as we add in support for storing snapshots across test runs.

IanCal added 4 commits February 26, 2026 12:56

Start a postgres container just for the realm server tests

0613b8e

SImplify setup test postgres, fix waits

40b2069

Use the pre-migrated template db

768e1b2

Merge remote-tracking branch 'origin/main' into realm-server-test-pg-…

882cf70

…slice1

chatgpt-codex-connector bot reviewed Feb 26, 2026

View reviewed changes

IanCal added 2 commits February 26, 2026 14:56

Broader supported checksumming

9f41827

Safer starting test postgres after killing the container

7de7dd6

habdelra requested a review from Copilot February 26, 2026 18:01

Copilot started reviewing on behalf of habdelra February 26, 2026 18:01 View session

Copilot AI reviewed Feb 26, 2026

View reviewed changes

Bind just to localhost and add debug info on port conflicts

79ce1ce

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Temporary database for realm server tests#4073

Temporary database for realm server tests#4073
IanCal wants to merge 7 commits intomainfrom
realm-server-test-pg-slice1

IanCal commented Feb 26, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Feb 26, 2026

Uh oh!

IanCal Feb 26, 2026

Uh oh!

Uh oh!

github-actions bot commented Feb 26, 2026 •

edited

Loading

Uh oh!

habdelra commented Feb 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

IanCal commented Feb 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		"test": "./tests/scripts/prepare-test-pg.sh && LOG_LEVELS=\"*=error,prerenderer-chrome=silent,pg-adapter=warn,realm:requests=warn${LOG_LEVELS:+,}${LOG_LEVELS}\" NODE_NO_WARNINGS=1 PGPORT=55436 STRIPE_WEBHOOK_SECRET=stripe-webhook-secret STRIPE_API_KEY=stripe-api-key qunit --require ts-node/register/transpile-only tests/index.ts",
		"test-module": "./tests/scripts/prepare-test-pg.sh && LOG_LEVELS=\"*=error,prerenderer-chrome=silent,pg-adapter=warn,realm:requests=warn${LOG_LEVELS:+,}${LOG_LEVELS}\" NODE_NO_WARNINGS=1 PGPORT=55436 STRIPE_WEBHOOK_SECRET=stripe-webhook-secret STRIPE_API_KEY=stripe-api-key qunit --require ts-node/register/transpile-only --module ${TEST_MODULE} tests/index.ts",

Conversation

IanCal commented Feb 26, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

IanCal Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Host Test Results

Uh oh!

habdelra commented Feb 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

IanCal commented Feb 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Feb 26, 2026 •

edited

Loading