Add caching to realm server tests by IanCal · Pull Request #4075 · cardstack/boxel

IanCal · 2026-02-26T15:36:30Z

This PR adds to a previous one which added a temporary db for realm server tests.

Here we add database caching. optionally a test can use cached helpers. When you do, the realm contents & parameters are hashed and used as a cache key. If a template db doesn't exist for this setup, we first create one, run all our indexing and then store it. After, we can boot up a realm very quickly, within a few hundred ms.

TODO:

Use the caching in more tests. Currently it's just in one module as a proof of concept.

…slice1

…using one shared prerenderer

…st-pg-slice2

github-actions · 2026-02-26T15:55:13Z

Host Test Results

1 files ± 0 1 suites ±0 1h 46m 53s ⏱️ + 3m 24s
1 896 tests +10 1 881 ✅ +11 15 💤 ±0 0 ❌ ±0
1 911 runs +10 1 896 ✅ +12 15 💤 ±0 0 ❌ - 1

Results for commit ebdcba3. ± Comparison against base commit 847b545.

♻️ This comment has been updated with latest results.

Copilot

Pull request overview

This PR speeds up realm-server’s test suite by switching tests to cached realm/DB fixtures backed by Postgres template databases, and adds scripts to bootstrap a pre-seeded Postgres container for fast DB cloning.

Changes:

Introduce cached permissioned realm/realms test helpers that build and reuse template DBs keyed by realm contents/parameters.
Add Docker-based scripts to prepare/start/stop a seeded Postgres test container and update package.json test scripts to use it.
Update many tests to use setupPermissionedRealmCached / setupPermissionedRealmsCached and createTestPgAdapter().

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/realm-server/tests/types-endpoint-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/server-endpoints/stripe-webhook-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/server-endpoints/stripe-session-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/server-endpoints/queue-status-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/server-endpoints/index-responses-test.ts	Switches published realm setup to cached helper with `realmURL` in options.
packages/realm-server/tests/server-endpoints/helpers.ts	Uses cached realm setup in shared server-endpoints test harness.
packages/realm-server/tests/search-prerendered-test.ts	Switches multiple nested fixtures to cached helper.
packages/realm-server/tests/scripts/wait-for-container-pg.sh	Adds readiness checks for Postgres-in-container startup.
packages/realm-server/tests/scripts/test-pg-config.sh	Defines test Postgres container/cache configuration variables.
packages/realm-server/tests/scripts/stop-test-pg.sh	Adds script to stop/remove the test Postgres container.
packages/realm-server/tests/scripts/start-test-pg.sh	Adds script to start test Postgres from seeded PGDATA tar into tmpfs.
packages/realm-server/tests/scripts/prepare-test-pg.sh	Adds script to (re)build seed tar when migrations change, then start container.
packages/realm-server/tests/scripts/create_seeded_db.sh	Adds script to build seeded PGDATA tar by running migrations in a seed container.
packages/realm-server/tests/scripts/boot_preseeded.sh	Adds entrypoint wrapper to untar seeded PGDATA and run Postgres with test-tuned settings.
packages/realm-server/tests/realm-endpoints/user-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints/search-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints/publishability-test.ts	Switches single- and multi-realm fixtures to cached helpers.
packages/realm-server/tests/realm-endpoints/permissions-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints/mtimes-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints/lint-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints/info-test.ts	Switches multiple nested fixtures to cached helper.
packages/realm-server/tests/realm-endpoints/directory-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints/dependencies-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/realm-endpoints-test.ts	Switches realm fixture setup to cached helper; threads prerenderer into `createRealm` calls.
packages/realm-server/tests/realm-auth-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/queue-test.ts	Switches DB setup to `createTestPgAdapter()` and removes concurrent migration timing concerns.
packages/realm-server/tests/publish-unpublish-realm-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/prerendering-test.ts	Switches multi-realm fixture setup to cached helper.
packages/realm-server/tests/prerender-server-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/indexing-test.ts	Switches multi-realm fixture setup to cached helper.
packages/realm-server/tests/helpers/index.ts	Adds Postgres cloning helpers, template-db caching, and cached realm/realms setup wrappers.
packages/realm-server/tests/file-watcher-events-test.ts	Switches realm fixture setup to cached helper with `realmURL` option.
packages/realm-server/tests/definition-lookup-test.ts	Switches multi-realm fixture setup to cached helper.
packages/realm-server/tests/card-source-endpoints-test.ts	Switches realm fixture setup to cached helper with `realmURL` option across modules.
packages/realm-server/tests/card-endpoints-test.ts	Switches realm and multi-realm fixtures to cached helpers with `realmURL` option.
packages/realm-server/tests/card-dependencies-endpoint-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/billing-test.ts	Switches DB setup to `createTestPgAdapter()` and makes PgAdapter import type-only.
packages/realm-server/tests/atomic-endpoints-test.ts	Switches realm fixture setup to cached helper.
packages/realm-server/tests/.test-pg-cache/.gitignore	Ignores seeded PGDATA tar/fingerprint cache directory contents.
packages/realm-server/scripts/remove-test-dbs.sh	Removes legacy test DB cleanup script.
packages/realm-server/package.json	Updates test scripts to run `prepare-test-pg.sh` and use new PGPORT.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-26T18:12:09Z

packages/realm-server/tests/helpers/index.ts

+  let client = new PgClient(pgAdminConnectionConfig());
+  try {
+    await client.connect();
+    await client.query(
+      `CREATE DATABASE ${quotePgIdentifier(database)} TEMPLATE ${quotePgIdentifier(
+        templateDatabaseName,
+      )}`,
+    );


cloneTestDBFromTemplate always runs CREATE DATABASE ... TEMPLATE ... for the target database name, but it doesn’t handle the case where that database already exists (e.g. a rare prepareTestDB() name collision during a large test run, or a prior failed run leaving the DB behind). This can cause intermittent failures with “database already exists”. Consider making prepareTestDB() generate stronger-unique names (pid + counter/uuid), or have cloneTestDBFromTemplate DROP DATABASE IF EXISTS / retry on “already exists” before creating the clone.

Copilot · 2026-02-26T18:12:10Z

packages/realm-server/tests/helpers/index.ts

+        (
+          '${publishedRealmId}',
+          '@user:localhost',
+          'http://example.localhost/source',
+          '${resolvedRealmURL.href}'
+        )`,


The SQL for inserting into published_realms is built via string interpolation (including resolvedRealmURL.href). Even in tests this can break on URLs containing quotes and is inconsistent with the rest of the file’s parameterized execute usage. Prefer passing values via bindings (or at least escaping) to keep the helper robust.

Suggested change

(

'${publishedRealmId}',

'@user:localhost',

'http://example.localhost/source',

'${resolvedRealmURL.href}'

)`,

($1, $2, $3, $4)`,

[

publishedRealmId,

'@user:localhost',

'http://example.localhost/source',

resolvedRealmURL.href,

],

Copilot · 2026-02-26T18:12:10Z

packages/realm-server/tests/helpers/index.ts

+  if (!entry) {
+    return;
+  }
+  entry.refs--;


releasePermissionedRealmTemplate decrements refs but never deletes the cache entry or drops the corresponding template database when refs reaches 0. Since useTemplateCache defaults to true across many tests, this can lead to an unbounded number of template DBs and Map entries over long runs/watch mode. Either remove the unused ref counting, or perform cleanup (delete entry + drop template DB) when refs hits 0 (or during suite teardown).

Suggested change

entry.refs--;

entry.refs--;

if (entry.refs <= 0) {

permissionedRealmTemplateCache.delete(cacheKey);

let templateDatabaseName = templateDatabaseNameForCacheKey(cacheKey);

try {

await dropDatabase(templateDatabaseName);

} catch {

// best-effort cleanup

}

}

Copilot · 2026-02-26T18:12:10Z

packages/realm-server/tests/types-endpoint-test.ts

@@ -70,7 +70,7 @@ module(basename(__filename), function () {
      resetCatalogRealms();
    });

-    setupPermissionedRealm(hooks, {
+    setupPermissionedRealmCached(hooks, {


This module still has a hooks.afterEach that calls closeServer(testRealmHttpServer) and resetCatalogRealms(), but setupPermissionedRealmCached (via setupPermissionedRealm) already performs server teardown and catalog reset in its own afterEach/after hook. Double-closing an HTTP server can throw (ERR_SERVER_NOT_RUNNING) and make cleanup flaky—consider removing the manual afterEach or limiting it to any extra servers created by this test.

IanCal added 10 commits February 26, 2026 12:56

Start a postgres container just for the realm server tests

0613b8e

SImplify setup test postgres, fix waits

40b2069

Use the pre-migrated template db

768e1b2

Merge remote-tracking branch 'origin/main' into realm-server-test-pg-…

882cf70

…slice1

Move prerendering to be the responsibility of the caller, default to …

51e0948

…using one shared prerenderer

Broader supported checksumming

9f41827

Safer starting test postgres after killing the container

7de7dd6

Merge branch 'realm-server-test-pg-slice1' into boxel-realm-server-te…

2d7b8f4

…st-pg-slice2

Add template based realm caching

f204f40

Add caching to a single test module

dcedbac

IanCal added 4 commits February 26, 2026 16:04

Roll caching out to more tests

70083b7

Setup at url was not very helpful it just moved where a parameter was.

b476f9b

Fix import

b1d5a7d

Cache setup multiple realms

8232cba

habdelra requested a review from Copilot February 26, 2026 18:04

Copilot started reviewing on behalf of habdelra February 26, 2026 18:04 View session

Copilot AI reviewed Feb 26, 2026

View reviewed changes

IanCal added 2 commits February 26, 2026 19:57

Restore non-ccahed version for indexing test

0f8adaf

Remove teardown that's not required

ebdcba3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add caching to realm server tests#4075

Add caching to realm server tests#4075
IanCal wants to merge 16 commits intomainfrom
boxel-realm-server-test-pg-slice2

IanCal commented Feb 26, 2026

Uh oh!

github-actions bot commented Feb 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

Copilot AI Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-  entry.refs--;
+  entry.refs--;
+  if (entry.refs <= 0) {
+    permissionedRealmTemplateCache.delete(cacheKey);
+    let templateDatabaseName = templateDatabaseNameForCacheKey(cacheKey);
+    try {
+      await dropDatabase(templateDatabaseName);
+    } catch {
+      // best-effort cleanup
+    }
+  }

Conversation

IanCal commented Feb 26, 2026

Uh oh!

github-actions bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Host Test Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Feb 26, 2026 •

edited

Loading