Skip to content

feat(knowledge): retain completed write history - #200

Merged
drewstone merged 7 commits into
mainfrom
feat/retained-write-history
Sep 10, 2026
Merged

drewstone merged 7 commits into
mainfrom
feat/retained-write-history

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Knowledge writes currently delete before/after snapshots when a transaction finishes. This loses the evidence needed to recover consolidated pages. This opt-in API preserves terminal transaction manifests and existing snapshots under a symlink-safe .agent-knowledge/history/ path and makes a lost finish acknowledgement idempotent. Default callers retain existing cleanup behavior. The Lab can opt in for its persistent research namespace without imposing a curator, merge policy, or per-note optimizer workflow.

Verification: focused file-transaction suite 25 passed / 2 skipped; typecheck src and contracts passed; lint passed; check-version-bump passed with major 16.0.0; API surface regenerated and current. No research data or credentials changed.

@drewstone

Copy link
Copy Markdown
Contributor Author

I am preparing a separate compatibility patch from current main: Knowledge 15.0.3 will admit the published Eval 0.180.0 after packed-consumer and official optimizer checks.

The shared metadata overlap is package.json and CHANGELOG.md; this patch does not change the retained-write-history implementation or its API surface. PR #200 keeps ownership of that work and its 16.0.0 release. Its next rebase should retain the widened Eval peer range and the 15.0.3 changelog entry beneath 16.0.0.

@drewstone

Copy link
Copy Markdown
Contributor Author

Probed this branch for Discovery's collaboration research (discovery#169): retained history keeps the bytes a concurrent overwrite destroys, and it names neither the writer nor the conflict.

What was run. A throwaway vitest file in a worktree at this PR's head d64100b, over a temporary store under /tmp that is deleted afterwards. No real store, run, or session content. Two tool sets with distinct actorIds and retainHistory: true: A writes v0, A reads it, B replaces it with a counterexample, A writes its edit of the version it read.

STALE WRITE RESULT {"written":["pages/bound.md"],"warnings":[]}
FINAL PAGE "---\nid: bound\n---\nv0: upper bound 12 (seed). Confirmed for n<=6 (director A).\n"
HISTORY TRANSACTIONS 3 MANIFEST KEYS ["createdAt","entries","kind","pagesDirectory","purpose","retainHistory","transactionId"]
B BYTES RETAINED true
ANY MANIFEST NAMES A WRITER false
 ✓ PR #200: stale overwrite is retained in history but unattributed and unflagged

What that shows. The retention works: B's counterexample survives under .agent-knowledge/history/<transactionId> after A overwrote it, where on 15.0.2 it survives nowhere (file-transaction.ts:425 deletes the directory). That is a real improvement and it is the reason the two gaps below are now worth closing — the bytes exist to attribute and to compare against.

The manifest keys are the whole story for the rest: no actor, and nothing that marks the third transaction as an overwrite of a version its writer never read. So the history answers "what changed" and not "who changed it" or "was this a conflict".

Two asks, filed at their own issues rather than as scope on this PR.

  1. fix(knowledge): a knowledge_record write records no author — pass the tools' existing actorId into the write transaction #206 — carry the tools' existing actorId (knowledge-tools.ts:45) into the write transaction's manifest. The manifest is exactly the file this PR retains, so one optional field beside recoveryOwner makes the retained history attributable at no extra cost. If it lands together with this PR, the 16.0.0 history is attributable from its first release rather than needing a second migration.
  2. fix(knowledge): a write from an out-of-date read silently erases a peer's page — knowledge_read returns no digest and knowledge_record checks none #205knowledge_read returns the page digest and knowledge_record refuses a block whose declared digest no longer matches, under the lock the write already holds. That is the check that would have refused A's third write above instead of retaining it.

Neither is a request to change this PR's implementation. Retained history without them is still strictly better than deletion; with them, a history/ directory becomes a usable record of who disagreed with whom, which is what N directors writing one namespace need.

Findings and the full probe source: https://github.com/tangle-network/discovery/blob/195852cc3a723511d13e6302791f08da08eeb7fb/docs/research/169-collaboration-primitives.md

@drewstone
drewstone merged commit dd9a50a into main Sep 10, 2026
2 checks passed
@drewstone
drewstone deleted the feat/retained-write-history branch September 10, 2026 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant