Lay the groundwork for durable execution

.github/workflows/ci.yml

@@ -21,6 +21,34 @@ jobs:
           fetch-depth: 0
       - run: sh scripts/check-repository-hygiene.sh
 
+  schema-migrations:
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:16-alpine
+        env:
+          POSTGRES_USER: roughstack
+          POSTGRES_PASSWORD: migration_test_only
+          POSTGRES_DB: roughstack_migration_test
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U roughstack -d roughstack_migration_test"
+          --health-interval 3s --health-timeout 3s --health-retries 20
+    env:
+      DATABASE_URL: postgresql://roughstack:migration_test_only@127.0.0.1:5432/roughstack_migration_test
+      SHADOW_DATABASE_URL: postgresql://roughstack:migration_test_only@127.0.0.1:5432/postgres
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 22
+          cache: npm
+      - run: npm ci
+      - run: npx prisma migrate deploy
+      - run: npx prisma migrate diff --from-migrations prisma/migrations --to-schema prisma/schema.prisma --exit-code
+      - run: npx prisma migrate diff --from-config-datasource --to-schema prisma/schema.prisma --exit-code
+
   web:
     runs-on: ubuntu-latest
     steps:

docs/execution-plane-implementation.md

@@ -0,0 +1,61 @@
+# Execution plane implementation
+
+This is an optional implementation plan for an instance that accepts untrusted
+public submissions, not a prerequisite for releasing the open-source project.
+The migration repair and versioned execution contracts in the current slice
+also improve reproducibility for local and self-hosted use. Do not start the
+queue, worker, or hosted sandbox work without a concrete deployment need and a
+written estimate of operational time, recurring cost, and simpler alternatives.
+This plan is not permission to accept untrusted public submissions. The current
+request handler still runs work inside the web process with a local Docker or
+synthetic backend.
+
+## Goal and invariants
+
+One accepted submission must create one durable execution job. A worker may
+retry an attempt, but only one terminal result may be published. A web restart
+must not lose queued work, and a worker crash must not leave a lease stuck
+forever. A run must be reproducible from its recorded source hash, challenge
+version, workload seed and parameters, judge policy, runtime image, and backend.
+
+The worker, not participant code, owns resource limits and sandbox cleanup.
+Only a verified, immutable arena bundle may be used for an official run. Stub
+results remain development-only and must never be presented as official scores.
+
+## Change order
+
+1. **Repair the database migration baseline.** The original migration
+   predates the current `Difficulty` and `Language` enums, `Reply`, and several
+   `Challenge` and `Submission` fields. The follow-up migration maps old
+   difficulty values (`BEGINNER` to `EASY`, `INTERMEDIATE` to `MEDIUM`, and
+   `ADVANCED`/`EXPERT` to `HARD`), converts legacy description and interface
+   text to blocks, and preserves submissions. Disposable Postgres tests passed
+   for both populated legacy rows and a database already at the `db push`
+   schema; CI now replays and compares migrations on every PR. Never reset a
+   user's database to hide drift.
+2. Define versioned job input and result-provenance contracts. Inputs snapshot
+   the challenge settings and workload variant at acceptance time. Arena jobs
+   pin an arena version and manifest digest. Hash the source without embedding
+   duplicate source code in the job payload.
+3. Commit a submission and its unique job in one database transaction. Keep
+   the old in-process executor behind an explicit development-only switch
+   during transition; no request may return success before the job commits.
+4. Build a worker that atomically claims a due job with a lease and fencing
+   token, heartbeats while running, and reconciles expired leases. Retried
+   attempts cannot overwrite a newer or terminal result.
+5. Resolve verified arena bundles by digest, then add a production sandbox
+   adapter with enforced CPU, memory, process, wall-clock, output and network
+   policy. Reconcile and destroy execution environments after every outcome.
+6. Exercise restarts, concurrent workers, lease expiry, duplicate delivery,
+   malformed bundles, and sandbox cleanup in integration tests. Add queue and
+   failure telemetry before enabling public traffic.
+
+## Current slice
+
+The first implementation slice repairs migration parity and defines the
+versioned input/provenance contract with tests. It is intentionally not wired
+into submission acceptance yet. No existing run changes behavior in this slice.
+
+Databases created earlier with `prisma db push` but without entries in
+`_prisma_migrations` need an operator-reviewed baseline procedure before
+`prisma migrate deploy`; do not run the initial migration against those tables.

docs/production-readiness.md

@@ -1,10 +1,30 @@
-# Production readiness roadmap
-
-Roughstack has a coherent learning experience, a deterministic arena contract,
-and a strong local development path. It is not ready to accept untrusted public
-traffic yet. The work below is ordered by dependency and launch risk.
-
-## Current product boundary
+# Project roadmap and optional hosted readiness
+
+Roughstack is an open-source project, not a hosted product by default. Its first
+promise is that contributors can understand, run, test, and extend the learning
+experience without buying infrastructure or maintaining a public service.
+Hosting an instance that accepts untrusted public submissions is a separate
+decision. The hosted-readiness work below is not a prerequisite for releasing
+the project or for trusted local development.
+
+## Default project priorities
+
+1. Make a clean checkout reproducible: documented setup, versioned migrations,
+   challenge packages that do not require an undocumented sibling checkout,
+   and CI that checks the same path contributors use.
+2. Keep challenge statements, reference solutions, test cases, and local run
+   results correct and reviewable. Label synthetic results as development-only.
+3. Document the trust boundary of local code execution. Never claim that a
+   local Docker runner safely accepts code from arbitrary internet users.
+4. Make contributions sustainable with clear repository ownership, small
+   reviewable changes, tests, and release notes.
+
+Before adding a paid service or a substantial workstream, record the concrete
+problem, the cheapest workable alternative, expected recurring cost and
+maintainer time, and a measurable criterion for adopting it. Prefer local or
+self-hosted components until an actual deployment need justifies more.
+
+## Current implementation boundary
 
 The browser experience is functional: visitors can browse challenges, read the
 educational material, edit code, submit a solution, inspect results, discuss an
@@ -25,7 +45,7 @@ The current server remains a development deployment:
 - The application has no production observability contract, incident signals,
   or service-level objectives.
 
-## P0: required before a public launch
+## Optional hosted track: required before accepting untrusted public submissions
 
 ### 1. Build the durable execution plane
 
@@ -115,7 +135,7 @@ Exit criteria:
 - Dependency, image, and secret scanning run in CI.
 - A release has a software bill of materials and signed artifacts.
 
-## P1: required for a strong learning product
+## Optional learning-experience improvements
 
 ### Guided progression
 
@@ -145,16 +165,18 @@ and explicit completion state. Complete keyboard and screen-reader audits for
 the editor, split panes, diagrams, tables, and result visualizations. Test reduced
 motion, zoom, narrow screens, and high-contrast modes.
 
-## P2: growth and ecosystem
+## Optional ecosystem ideas
 
 - Public profiles with opt-in visibility and verified achievements.
 - Organization classrooms, private cohorts, and assignment progress.
 - A documented arena authoring kit with local preview and contract validation.
 - Challenge versioning, deprecation, moderation, and an editorial review queue.
-- Product analytics based on explicit events and a documented retention policy.
-- Billing only after execution cost, abuse, refund, and quota models are known.
+- Privacy-conscious usage measurement only if it answers a demonstrated project
+  question; document retention before collecting events.
+- Billing is out of scope for the open-source project. Revisit only if a
+  separately operated hosted service is deliberately proposed.
 
-## Product patterns reviewed
+## Learning-platform patterns reviewed
 
 This ordering borrows proven patterns without copying another product's surface:
 
@@ -171,7 +193,12 @@ This ordering borrows proven patterns without copying another product's surface:
   show the value of grouped test cases, submission analysis, code replay, progress
   views, filters, profiles, security settings, and focus modes at scale.
 
-## Recommended implementation order
+## Order if the optional hosted track is chosen
+
+The [execution-plane implementation plan](execution-plane-implementation.md)
+tracks one possible hosted-readiness path and its migration prerequisite. The
+migration repair and versioned execution contracts are useful independently;
+the queue, worker, and public sandbox are not the default next milestone.
 
 1. Define the immutable execution job and result provenance schema.
 2. Publish and verify arena bundles independently of the platform image.

prisma.config.ts

@@ -16,5 +16,8 @@ export default defineConfig({
     // do not touch the database, such as `prisma generate` during a type-check in CI,
     // do not fail merely because DATABASE_URL is absent.
     url: process.env.DATABASE_URL ?? "",
+    // Migration parity checks replay committed SQL into a disposable database.
+    // Never point this at a database that contains user or development data.
+    shadowDatabaseUrl: process.env.SHADOW_DATABASE_URL,
   },
 });

prisma/migrations/20261016183000_align_current_schema/migration.sql

@@ -0,0 +1,98 @@
+-- Bring the original migration baseline to the current application schema.
+-- The data conversions are deliberately explicit: existing challenges and
+-- submissions must survive an upgrade from the original public schema.
+
+DO $$
+BEGIN
+  IF EXISTS (
+    SELECT 1 FROM pg_enum e
+    JOIN pg_type t ON t.oid = e.enumtypid
+    WHERE t.typname = 'Difficulty' AND e.enumlabel = 'BEGINNER'
+  ) THEN
+    CREATE TYPE "Difficulty_next" AS ENUM ('EASY', 'MEDIUM', 'HARD');
+    ALTER TABLE "Challenge" ALTER COLUMN "difficulty" TYPE "Difficulty_next"
+      USING (CASE "difficulty"::text
+        WHEN 'BEGINNER' THEN 'EASY'
+        WHEN 'INTERMEDIATE' THEN 'MEDIUM'
+        ELSE 'HARD'
+      END)::"Difficulty_next";
+    ALTER TYPE "Difficulty" RENAME TO "Difficulty_old";
+    ALTER TYPE "Difficulty_next" RENAME TO "Difficulty";
+    DROP TYPE "Difficulty_old";
+  END IF;
+END $$;
+
+ALTER TYPE "Language" ADD VALUE IF NOT EXISTS 'C';
+ALTER TYPE "Language" ADD VALUE IF NOT EXISTS 'CPP';
+ALTER TYPE "Language" ADD VALUE IF NOT EXISTS 'JAVA';
+ALTER TYPE "Language" ADD VALUE IF NOT EXISTS 'RUST';
+
+ALTER TABLE "Challenge"
+  ADD COLUMN IF NOT EXISTS "blocks" JSONB,
+  ADD COLUMN IF NOT EXISTS "ladder" TEXT,
+  ADD COLUMN IF NOT EXISTS "pack" TEXT NOT NULL DEFAULT 'ssd',
+  ADD COLUMN IF NOT EXISTS "task" TEXT,
+  ADD COLUMN IF NOT EXISTS "taskParams" JSONB,
+  ADD COLUMN IF NOT EXISTS "tier" INTEGER NOT NULL DEFAULT 1;
+
+DO $$
+BEGIN
+  IF EXISTS (
+    SELECT 1 FROM information_schema.columns
+    WHERE table_schema = 'public' AND table_name = 'Challenge'
+      AND column_name = 'description'
+  ) THEN
+    EXECUTE 'UPDATE "Challenge" SET "blocks" = jsonb_build_array('
+      || 'jsonb_build_object(''kind'', ''prose'', ''md'', "description")) '
+      || 'WHERE "blocks" IS NULL';
+  END IF;
+  IF EXISTS (
+    SELECT 1 FROM information_schema.columns
+    WHERE table_schema = 'public' AND table_name = 'Challenge'
+      AND column_name = 'interfaceDoc'
+  ) THEN
+    EXECUTE 'UPDATE "Challenge" SET "blocks" = "blocks" || '
+      || 'jsonb_build_array(jsonb_build_object(''kind'', ''callout'', '
+      || '''tone'', ''note'', ''title'', ''Interface'', ''md'', "interfaceDoc")) '
+      || 'WHERE "interfaceDoc" IS NOT NULL AND "blocks" IS NOT NULL';
+  END IF;
+END $$;
+
+-- Earlier challenges used their slug as the execution task. Keep the slug as
+-- a conservative ladder key until an editor assigns a richer progression.
+UPDATE "Challenge" SET "task" = "slug" WHERE "task" IS NULL;
+UPDATE "Challenge" SET "ladder" = "slug" WHERE "ladder" IS NULL;
+ALTER TABLE "Challenge"
+  ALTER COLUMN "blocks" SET NOT NULL,
+  ALTER COLUMN "task" SET NOT NULL,
+  ALTER COLUMN "ladder" SET NOT NULL,
+  DROP COLUMN IF EXISTS "description",
+  DROP COLUMN IF EXISTS "interfaceDoc";
+
+ALTER TABLE "Submission" ADD COLUMN IF NOT EXISTS "sessionId" TEXT;
+
+CREATE TABLE IF NOT EXISTS "Reply" (
+  "id" TEXT NOT NULL,
+  "challengeId" TEXT NOT NULL,
+  "sessionId" TEXT NOT NULL,
+  "authorLabel" TEXT NOT NULL,
+  "body" TEXT NOT NULL,
+  "isSpoiler" BOOLEAN NOT NULL DEFAULT false,
+  "isHidden" BOOLEAN NOT NULL DEFAULT false,
+  "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+  CONSTRAINT "Reply_pkey" PRIMARY KEY ("id")
+);
+
+CREATE INDEX IF NOT EXISTS "Challenge_ladder_tier_idx" ON "Challenge"("ladder", "tier");
+CREATE INDEX IF NOT EXISTS "Submission_sessionId_createdAt_idx" ON "Submission"("sessionId", "createdAt");
+CREATE INDEX IF NOT EXISTS "Reply_challengeId_createdAt_idx" ON "Reply"("challengeId", "createdAt");
+CREATE INDEX IF NOT EXISTS "Reply_sessionId_createdAt_idx" ON "Reply"("sessionId", "createdAt");
+
+DO $$
+BEGIN
+  IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'Reply_challengeId_fkey') THEN
+    ALTER TABLE "Reply" ADD CONSTRAINT "Reply_challengeId_fkey"
+      FOREIGN KEY ("challengeId") REFERENCES "Challenge"("id")
+      ON DELETE CASCADE ON UPDATE CASCADE;
+  END IF;
+END $$;