OpenMind v2 Phase 1: core extraction and tool-first foundation

[HelloThisWorld]

Turns OpenMind from a UI-centric application into a headless engineering knowledge runtime that the CLI, the MCP server, the FastAPI app and the tests all drive through one shared bootstrap.

This is an architectural foundation. It deliberately implements none of the later v2 enterprise knowledge features — see Deferred.

CLI · MCP · FastAPI · Tests
          ↓
   OpenMindRuntime   (idempotent: dirs → db → migrate → services)
          ↓
   Application services   (workspace / ingest / job / export / health)
          ↓
   Existing deterministic implementation → SQLite / vector store / filesystem

Design note: docs/v2/phase-1-core-foundation.md.

What changed

Runtime and services. New openmind/runtime.py composition root with an idempotent, ordered bootstrap. New services/ (workspace, ingest, job, export, health, container), domain/ (typed errors + boundary types), and ports/ — three narrow protocols, each with a real test seam: the workspace repository, the job reader/engine, and a clock that makes bounded waits testable in milliseconds. Services never import FastAPI.

The most consequential fix: jobs.start_worker() previously ran only under the FastAPI lifespan, so a job enqueued from any other process sat queued forever. Worker startup is now an explicit, opt-in runtime call — which is what makes ingest --wait work at all, while keeping doctor/status/export thread-free.

Database migrations. Replaces the untracked CREATE TABLE IF NOT EXISTS block with a checksummed, transactional runner and a schema_migrations ledger. A legacy database is baselined, never recreated. Editing an already-applied migration fails loudly. DDL is transactional via explicit BEGIN/COMMIT — sqlite3 would otherwise run it in autocommit, where it survives a rollback. No Alembic; the rationale is in docs/database-migrations.md.

CLI. python -m openmind.cli: doctor, init, add, ingest, status, export, mcp serve, serve. Under --json, exactly one object on stdout with diagnostics on stderr. Exit codes 0–5 stable and documented (docs/cli.md). serve stays loopback-only unless --allow-non-loopback is passed explicitly.

Adapters. Project, path, template, ingest, job, lifecycle and health operations in main.py now go through the services. mcp_server.create_mcp_server(runtime) makes MCP startup testable, and importing the module no longer opens a database.

Compatibility

Surface	Status
REST	55 routes registered; 29 probed against main via a git worktree — zero status-code differences, zero keys removed. 14 differences, all additive.
MCP	Nine tools, same names/arguments/responses. python -m openmind.mcp_server unchanged.
Artifact schema 1.x	Frozen at 1.1.0. Export verified to run with numpy/chroma/fastapi/mcp/pydantic all absent.
Skill bridge	Protocol unchanged; still computes in-memory and never touches the app database.
Existing databases	Legacy DB baselined; project/job/file-index/cases/kv rows asserted intact.

Two intentional changes:

• /api/health gains version and schema_version. The full doctor report sits behind ?diagnostics=1 — including it unconditionally measured 0.8 ms → 2291 ms per call, which is wrong for a liveness endpoint.
• DELETE /projects/{id}/paths now returns 404 for an unknown project. That route was demonstrably broken: run against main it raises ResponseValidationError (a 500), because the handler returned None from a -> Dict[str, Any] signature.

Verification

Baseline before any change: 18/18 verify scripts green, artifact contract 31/31.

Final: python scripts/run_acceptance.py --all → 24 passed, 0 failed, 0 skipped. New suites add 374 checks (migrations 49, runtime 31, services 97, CLI 112, adapters 85). Every acceptance-criteria command exits 0 from a clean data dir; python -m openmind.artifacts and python -m openmind.skill_bridge still work.

The new acceptance runner isolates OPENMIND_DATA_DIR per script and forces offline embeddings with no egress. Two integrity properties: a skipped core script counts as a failure, and every tests/verify_*.py must be registered in the manifest, so a new test cannot silently go unrun while the suite reports green.

Defects found and fixed during the work — each caught by a test rather than assumed away:

1. Migrations applied in the given order rather than numeric order.
2. openmind --quiet doctor silently not quiet: argparse parents= shares Action objects, so a subparser default clobbered the global flag. A first fix using set_defaults() failed for the same reason.
3. The CLI printed two JSON objects on the --wait timeout path, and init --ingest reported ok:true after a failed ingest.
4. services/__init__.py eager imports pulled vectorstore → numpy/chromadb into export, which would have broken the dependency-free CI job. Now lazy, and asserted by simulating the absent dependencies.
5. A race between get_runtime() and reset_runtime(); the acceptance runner reporting a stderr warning instead of the test verdict; two unused imports.

Reviewer notes

• CI has not actually run. The workflow YAML parses and every step's logic was exercised locally, but GitHub Actions could not be executed from the dev environment — this PR is the first real run.
• main.py is still ~950 lines. Ask, glossary-enrichment, /source and graph routes deliberately still call their modules directly: they are thin and adding a service layer there would be indirection without a seam.
• Migration checksums cover source text, so a comment-only edit to an applied migration will fail an existing database. Intended, documented, and still likely to surprise someone.
• Inherited and not fixed (the job engine is explicitly out of scope this phase): the read-then-write dedupe race in jobs.enqueue_*, and tests reaching into jobs._deleting / _shutdown / _recover_on_restart.
• verify_delete_responsive is excluded from the CI gate — it asserts sub-2-second API latency while a delete drains, which is flaky on a shared runner. The reason is recorded in the manifest and it runs under --all.
• The optional skill-verification job is workflow_dispatch-only and does nothing unless AGENT_SKILL_VERIFICATION_REF is set to a pinned owner/repo@ref; it reports "not configured" rather than passing silently.

Deferred

Enterprise Asset/Revision/Claim/Relation models, the Knowledge Graph, PDF/DOCX/XLSX, COBOL/JCL, cloud providers, requirement-to-code traceability, conflict detection, branch overlays, webhooks, Titan Mind, new graph/chat UI, autonomous code modification, TypeScript rewrite, monorepo conversion, Neo4j, Bundle 2.0, typed worker pool. Extension points exist (migration runner, ports/, ServiceContainer, CLI subcommand table); no placeholder code pretends otherwise.

Version is 1.1.0-dev, not 2.0.0 — labelling this build 2.0.0 would be a false claim.