OpenMind v2 Phase 2: Workspace Asset, Revision, Segment & Evidence foundation

[HelloThisWorld]

What & why

Phase 2 builds the canonical engineering-content foundation every later v2 capability needs. It transforms the storage model from paths + file index + vector chunks into the beginning of the v2 canonical model:

Workspace
└── Asset            one logical engineering object (a source/config file)
    └── Revision     an immutable observation of that file's exact bytes
        ├── Segment  a stable structural unit (Java type/method/ctor, or line-range)
        └── Evidence source-locatable, recoverable from the immutable snapshot

Chroma becomes a retrieval projection; the durable source of content identity is SQLite plus an immutable SHA-256 content-addressed blob store. This is a data-model foundation, not a feature release — no document parsing, Claim/Relation tables, Knowledge Graph or traceability (all still deferred).

Built directly on the Phase 1 boundaries (runtime, service container, migration runner, typed errors, CLI/FastAPI/MCP adapters); Phase 1 is not rewritten.

Schema (migration v0003_asset_model)

• New tables assets / asset_revisions / segments / evidence + indexes; FK cascade to projects; (asset_id, content_hash) intentionally not unique so an A→B→A revert is representable.
• v0001/v0002 untouched (frozen, checksummed). Head 2 → 3. A Phase 1 database upgrades additively with no data loss; existing files backfill their Assets on the next ingest without re-embedding.

Highlights

• Immutable content store (content_store.py): atomic writes, blob reuse, SHA-256 identity, explicit corruption detection; DB stores only the hash.
• Shared, deterministic segmentation (segmentation.py) that leaves rag.chunk_file byte-for-byte unchanged — retrieval, chunk ids and the search response cannot regress.
• Transactional ingestion: db.commit_revision is the single writer; no partial revision ever becomes current. The revision commits before the file_index row, so a crash in that window re-indexes the file instead of stranding a stale Asset (fix from the adversarial review below).
• Incremental: unchanged re-ingest = 0 revisions and 0 embed calls (asserted with an embedding-call spy); one change = one new revision (old superseded, still queryable); removed files keep history; reappearing keys reactivate the same Asset.
• Filtered ingests (asset add / ingest --path) skip whole-project artifact rebuilds and scope the prune — fixing a latent bug where a subtree ingest corrupted the glossary from a partial file set. Full ingest is unchanged.
• Adapters: AssetService (runtime.assets), CLI asset group + status counts, additive read-only REST under /projects/{id}/…, four additive read-only MCP tools. The nine core MCP tools, all REST routes and the /projects naming are unchanged.
• Lifecycle: terminate clears Asset rows + blobs (keeps workspace + paths); delete cascades + blobs die with the data dir. Delete-drain / janitor-race fixes preserved.

Measured incremental behavior (fixture, offline embedder)

Scenario	Result
initial ingest	10 assets / revisions / segments / evidence / blobs, 1 embed batch
unchanged re-ingest	0 new revisions, 0 embed calls
single-file change	1 new revision; old superseded + still queryable; historical evidence intact
revert A→B→A	3 revisions; final reuses the original content blob
file removal	Asset marked removed; revisions/segments/evidence/blobs preserved
legacy backfill	10 Assets recreated, 0 embed calls

Compatibility

REST routes ✓ · MCP 9 core tools ✓ · .openmind schema 1.1.0 (frozen; Asset model not exported yet) ✓ · skill bridge ✓ · glossary / structure / RAG search ✓ · existing databases migrate without loss ✓. Runtime version → 1.2.0-dev (single constant); build is not labelled 2.0.0.

Testing

Four new acceptance scripts registered in scripts/run_acceptance.py and gated in CI: verify_content_store, verify_asset_model, verify_asset_cli, verify_asset_adapters. Core suite now 28 scripts, all green (baseline was 24); local-tier delete-responsiveness 8/8. CI full gate + cross-platform smoke expanded with content-store, migration-to-v0003 and asset CLI/evidence checks. No paid API; offline hashing embedder throughout.

Reviewer notes

An adversarial review (5 dimensions → per-finding refutation) was run over the diff. It confirmed exactly one real defect — the file_index-before-Asset commit ordering — which is fixed in this branch (revision now commits first). Two non-reachable findings were hardened anyway (unique content-store temp name; evidence path-containment check). A regression test locks in the invariant "every file_index entry has an in-sync active Asset."

Design doc: docs/v2/phase-2-asset-model.md.