Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/local-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,25 @@ jobs:
E2E_BINARY: ${{ runner.temp }}/memory-bank-cli
LEGACY_SOURCE: ${{ github.workspace }}/.fixtures/legacy-source
run: bash scripts/e2e-source-format.sh

- name: Check out reviewed component producer
uses: actions/checkout@v7
with:
repository: dapi/memory-bank
ref: f695db6a703e5409f10c9988e6460b41068fe30c
fetch-depth: 0
path: .fixtures/component-source
persist-credentials: false
- name: Verify component contracts and transaction fixtures
env:
MEMORY_BANK_COMPONENT_SOURCE: ${{ github.workspace }}/.fixtures/component-source
MEMORY_BANK_LEGACY_SOURCE: ${{ github.workspace }}/.fixtures/legacy-source
run: |
go test ./...
go vet ./...
- name: Verify actual-binary component acceptance
env:
E2E_BINARY: ${{ runner.temp }}/memory-bank-cli
MEMORY_BANK_COMPONENT_SOURCE: ${{ github.workspace }}/.fixtures/component-source
MEMORY_BANK_LEGACY_SOURCE: ${{ github.workspace }}/.fixtures/legacy-source
run: python3 scripts/e2e-components.py
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,16 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht

## [Unreleased]

- Install schema-2 component presets (`core`, `docs`, `full`, `legacy`) and additive
adapters on Linux/macOS, with composition-aware README and AGENTS blocks.
- Create base documents independently from explicit flow adoption; bind adopted
documents to immutable contract bundles and support evidence-backed transitions
and identity-preserving moves.
- Migrate the pinned legacy source through deterministic previews, explicit
classification/ownership resolutions and exact plan-digest consent.
- Validate component state before mutations and retain durable recovery journals,
including prepared-draft snapshots, when rollback or cleanup cannot finish.

- Gate installation sources by a pinned legacy compatibility list or strict source-format
declaration; reject unknown formats before planning or changing downstream files.
- Add the JSON `capabilities --require` handshake for installation entrypoints.
Expand Down
25 changes: 24 additions & 1 deletion docs/component-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ Baseline `env -u GOROOT go test ./...` passes all packages on 2026-09-07.

## Gates and realization

CLI-01 uses the independently reviewed [bridge plan](source-format-bridge.md). CLI-02/03 wait for the shared component design gate. CLI code, tests
CLI-01 follows the independently reviewed [bridge plan](source-format-bridge.md).
The shared design and W2 execution-plan gates are complete; CLI-02/03 are implemented in
[PR 64](https://github.com/dapi/memory-bank-cli/pull/64), stacked on bridge PR 63. CLI code, tests
and evidence stay here. The slice is independently verifiable with synthetic source fixtures;
the template PR supplies the final cross-repository source integration.

Expand All @@ -39,3 +41,24 @@ findings. Unexecuted checks are not evidence.
Prepare the bridge commit and record its actual binary identity before component support.
Component source must wait for supporting CLI. No live downstream mutation, merge or release
publication belongs to this task. Related PRs state the required release order explicitly.

The reviewed [component runtime plan](component-runtime-plan.md) owns W2 CLI execution
sequencing. CTR-01/ADR-002 and the shared Solution Ready gate were accepted before implementation.

## Implemented verification surfaces

`internal/contracts` validates portable paths, deterministic wire encodings, frozen bundles,
registry continuity, reference relocation and base/flow rules. `internal/ownership` integrates
those contracts with composition, document operations, migration previews, resolution plans
and the existing pinned transaction writer. A prepared `--from` draft receives a durable
private snapshot for recovery; it remains an unchanged read precondition through cleanup.

The local acceptance run passes the complete Go suite and vet, 28 existing E2Es, source-format
fixtures and `scripts/e2e-components.py` against the actual producer. CI repeats these checks
using immutable producer and legacy source commits. Specific fixtures include all six base
types remaining unadopted, registry/marker/identity/type tampering, renderer-1 upgrade,
ambiguous migration resolution, unchanged legacy finding multisets, selector transition
rollback, adapter closure and complete recovery with restored read inputs.

The PR records the final immutable review revisions and CI links. No merge, release or live
migration is claimed by this implementation checkpoint.
169 changes: 169 additions & 0 deletions docs/component-runtime-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# Component and document runtime plan

This is CLI #62's implementation plan for W2, owned by memory-bank-cli. It imports
[CTR-01](https://github.com/dapi/memory-bank/blob/feat/141-component-adoption/docs/components.md)
and the parent issue's acceptance contract. The template feature owns payload declarations,
base templates and flow wrappers; this file owns CLI implementation sequencing only.

Status: active. Shared Solution Ready is satisfied by the reviewed c22294b → b94560c
contract chain (2026-09-07T01:14:17Z). This execution plan passed independent review at
acbfb32, and the bridge checkpoint is complete. Bridge checkpoint: ready PR #63 at 3b434fd93678c36447d10d4f308a39ce5d74b040, with
required CI, actual-binary canary and independent functional/simplification reviews clean. No release or live downstream migration is included.

## Grounding and boundaries

Grounded CLI baseline ac7101c307e65566787bdb32a1bdad40b9a8b995 plus W1 a0811c4:

- ownership/source.go verifies pinned regular Git blobs; W1's source_format.go classifies them.
- ownership/update.go owns run, buildPlan, mutation preconditions and applyAtomicallyPinned.
- ownership/lock.go reads schema 0/1 and validates ownership/digests/modes.
- ownership/resolution_plan.go revalidates sources in PlanPull and ApplyResolutionPlan.
- doctor/governance.go currently applies metadata and feature lifecycle checks type-wide.
- cli/cli.go dispatches init/pull/doctor/lint and keeps update reserved for the executable.
- agentinstructions/block.go preserves bytes outside the managed instruction block.

Add a pure internal/contracts package for manifest/rule decoding and frozen validation semantics.
It must not import ownership, doctor or CLI. Ownership uses contracts during transaction preflight;
doctor/lint use the same validator, avoiding divergent activation rules. The existing legacy
validator remains available for legacy installations. Schema-2 installations use explicit adoption.

## Steps

1. **Composition** — internal/contracts/manifest.go and ownership/components.go validate the whole
source inventory, resolve preset/adapter dependencies, filter payload and compose README/AGENTS.
Extend Lock/Options and strict lock decoding for schema 2 without upgrading ordinary legacy
source pulls. Unknown fields/paths/capabilities, missing inventory, cycles and removals fail
before writes. Component-aware PlanPull and ApplyResolutionPlan are required in W2:
both use the same composed transaction plan and validate selection, state and current source.
Apply regenerates the entire plan before writes, so a saved pre-bridge plan receives no
exemption. A matching-identity old-format plan against a component source is a direct
negative fixture. Resolution planning is complete only after successful component preview,
application, stale-file/lock/source rejection and no-mutation tests pass.
2. **Frozen contracts** — internal/contracts/rules.go and engine.go decode base types and immutable
bundles. Each bundle embeds DNA, base and extension rules, plus the exact engine ID/digest.
The trusted implementation embeds its immutable behavior artifact and a positive/negative
corpus. Published bundles and engine behavior are never modified in place. A live DNA/base
change cannot affect an adopted document's verdict. Missing or changed required bundles fail
before payload mutation. Extensions cannot weaken their embedded base requirements.
3. **Adoption integrity** — ownership/adoption.go owns the registry snapshot, identity/path/type
reconciliation and checksum binding to lock. Scan regular Markdown only under memory-bank/ for orphan projection
fields, excluding .repo, dna, flows, templates, document-types, prompts and declared managed
template assets. Do not inspect unrelated Markdown outside that root. Reject unsafe
symlinks/aliases rather than following them. Recorded targets must resolve exactly once. Base documents have no record or marker.
Schema-2 full/legacy always have a registry, even when empty; schema-0/1 installations
remain on the old validator and have no registry requirement before explicit migration.
Missing/corrupt schema-2 state is an error, never
an invitation to recreate it. Read/validate both source and installed contracts.
4. **Document operations** — ownership/documents.go and cli/documents.go implement create, adopt,
transition and move using the same handle-relative transaction engine. Report a dry-run plan;
check applicable old/new gates and explicit evidence references; commit document, registry,
history and lock together. Same adoption/move is idempotent. Unsupported detach/delete or
transitions reject before writes. Base creation in full does not adopt implicitly. Legacy-flow
creation explicitly resolves a per-document compatibility ID rather than adding to a selector.
5. **Legacy migration** — ownership/component_migration.go verifies the source-specific supported
legacy map, ownership and local drift; freezes the prior document identities in selector
snapshots; and previews the creation/validation semantics change. Apply requires both explicit
--migrate-components and the current --migration-plan-digest. The digest binds old lock, source,
resolution map, observed document bytes/modes and proposed mutations; deterministic snapshot
identities/history ensure repeated preview is stable. Unsupported versions and incomplete,
incompatible or ambiguous owner maps conflict. Migration preserves existing invalid legacy
verdicts, while identity/integrity constraints still must hold. Selector transition atomically
adds an exclusion plus a new per-document record, with no implicit precedence.
6. **Validation entrypoints** — doctor and lint check component state, dependencies, base documents,
adoption and navigation; core/docs do not require absent Flows. Source-profile projection is
explicitly distinguished from a downstream with missing lock. Preflight validates the resulting
tree, including derived_from, Markdown paths, embedded frontmatter and priming manifest paths.
Existing project-owned content is preserved by pull; scaffold ownership transfers on creation.
Migration alone allows the same pre-existing legacy validation findings: compare multisets
of stable identity, finding code, rule ID and subject before/after under the frozen engine.
Any added or removed legacy finding, identity/integrity/path failure or new navigation violation blocks apply.
Normal validation still reports the preserved errors. Test invalid legacy migration succeeds
while an added violation fails without writes; ordinary pulls get no blanket exemption.

Selection precedence is CTR-01's contract: fresh init without flags chooses legacy; a
flagless schema-2 pull preserves the locked preset/components/adapters exactly. An explicit
preset resolves together with retained/new adapters and dependencies, then rejects removal
of any installed component. Adapter flags are additions, never replacement. No selection
flag opts a schema-0/1 installation into component migration. With a legacy source, any
component-selection flag is rejected before writes; it is never silently ignored. With a
component source and schema-0/1 lock, selection flags without explicit migration consent
also reject before writes. The flagless case rejects identically: any schema-0/1 lock plus
a component source requires --migrate-components, including unattended pull with no preset
or adapter flags. It never implicitly changes validation semantics. A direct flagless
fixture checks byte/mode/lock preservation. Direct negative fixtures cover both source formats and preserve
every downstream byte/mode and lock. Tests repeat flagless pulls for
every preset and adapter variant and assert unchanged selection and no automatic Flows.

Migration preview is `pull --migrate-components --dry-run --json` (with explicit source
inputs and optional --migration-resolution FILE). It writes no downstream state and returns
migration_plan_digest plus the exact proposed changes/semantics. Apply passes that digest
back as `pull --migrate-components --migration-plan-digest DIGEST` with the same source and
resolution input. Apply regenerates the preview and rejects changed observations or a stale
digest before writes. Neither unattended mode nor --preset legacy replaces this consent.

CLI flags: init/pull --preset NAME, repeated --adapter NAME; pull --migrate-components,
--migration-plan-digest DIGEST and --migration-resolution FILE. Document commands use --type,
--path, --contract, optional --from for prepared creation, --to, --id (required for move), --dry-run and repeatable --evidence REF as applicable. An explicit
--legacy-flow chooses the installation's pinned compatibility contract. No adapter removal,
uninstall, contract composition, automatic adoption, arbitrary code execution or global service
is introduced. Exact serialized fields and encoding rules are owned by the shared
[CTR-01 wire format](https://github.com/dapi/memory-bank/blob/feat/141-component-adoption/docs/component-wire-format.md).
Go types and producer/consumer fixtures implement it; semantic changes return to design review.

## Durable recovery extension

The consolidated CTR-01 recovery predicate requires internal/ownership/component_recovery.go
and a component-only hook in the existing transaction engine. Before mutation, persist and
sync a versioned staging journal binding every observed/target path, before/after bytes and
modes, numbered backup mapping and created directories. Use a prepared/committed journal
state: first sync existing target-file contents and staged replacements, then sync the
prepared file, its staging directory and repository parent before target mutation.
The existing writer renames originals into numbered backups; it does not copy write-target originals before mutation. The optional read-only --from
input has a separate durable inputs/000000 snapshot, bound together with every ancestor
directory state through cleanup. Sync both directories after each original rename and before
installing its replacement, preserving the already synced original inode at target or backup.
The journal also records directory before/after existence and modes for complete restoration; after all replacements and lock-last, sync changed files and directories, then
atomically persist/sync the committed journal and staging directory. A crash before that
last durable marker is ambiguous and requires restoring the complete before state. No new
file writer or automatic rollback replay is introduced. Before subsequent component planning, retained staging blocks writes unless
its complete before state has been restored; committed cleanup retries instead check complete
after state and integrity. Unknown journals fail closed. The repository owner performs manual restoration using the
journal's exact path-to-backup mapping and before observations: restore originals from
numbered backups or a trusted pre-operation backup, restore modes, remove originally absent
targets and created empty directories, and preserve concurrent edits separately. Recovery
checking never edits target files; it checks the restored mixed-state fixture against all
observations before cleaning staging and permitting ordinary preflight. Tests inject rollback/cleanup failure and verify that restoring only the lock is
insufficient, complete restoration permits cleanup/re-entry and retry is idempotent.

## Verification and failure boundaries

Go contract/transaction fixtures cover every parent acceptance class: preset/default/adapter
matrix; repeated init/pull; docs-to-full and scaffold preservation; malicious paths and symlinks;
base vs adopted feature; registry/marker/identity/type/path tampering; frozen bundle/DNA/base/engine
drift and missing historical bundle; migration opt-in and stale digest; ambiguous moved documents
with valid/invalid owner maps; legacy selector exclusion plus rollback; fresh and migrated legacy
creation; unsupported transitions; failure during staged writes and concurrent changed lock.
Fixtures retain a byte/mode snapshot of the old tree and an external sentinel for negative paths.

Run `env -u GOROOT go test ./...`, `env -u GOROOT go vet ./...`, hermetic ownership E2E, and a
real component binary against the exact template candidate commit. Keep a separately built bridge
binary at the reviewed W1 commit; its real-binary fixture must reject the component candidate.
A pre-bridge binary is tested through the template's minimum-capability entrypoint. Direct
pre-bridge execution on a component source remains explicitly unsupported by the parent issue.

Before each implementation step, read its grounded owner files; update this plan when the exact
surface changes. Independent code-converge review uses a clean author commit, explicit baseline
and --max-cycles 0 so the run cannot fix, checkpoint or publish reviewed changes. Author fixes and
commits findings, then re-runs the review. Separate final code and simplification passes must be
clean. CLI and template PRs retain an explicit bridge-first release dependency; publication tags
are assigned by the release owner after review, not invented as already available binaries.

## Re-evaluated execution boundary

After five artifact review iterations, the implementation keeps one imported wire owner,
a bounded memory-bank document scan and an explicit digest-producing preview command.
The source-format bridge remains a separate delivery checkpoint; this plan does not
advertise component capability until its complete operation matrix is implemented.
Producer/consumer fixtures must cover CTR-01 selector grouping/IDs, context-root derivation,
canonical registry bytes and exact legacy finding multiset equality. Shared contract review
and this execution-plan review are separate gates; both completion checkpoints are recorded above.
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@ go 1.21
require (
golang.org/x/sys v0.17.0
golang.org/x/term v0.17.0
golang.org/x/text v0.14.0
gopkg.in/yaml.v3 v3.0.1
)
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.17.0 h1:mkTF7LCd6WGJNL3K1Ad7kwxNfYAW6a8a8QqtMblp/4U=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
Expand Down
Loading
Loading