Skip to content

feat(agent-memory): Ed25519 WitnessSigner for the TARL ledger's witness chain - #989

Draft
ruvnet wants to merge 4 commits into
mainfrom
claude/focused-darwin-tg4de6
Draft

ruvnet wants to merge 4 commits into
mainfrom
claude/focused-darwin-tg4de6

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Nightly Research: Closing the ADR-134 §9 WitnessSigner Gap

Hypothesis: Given the TARL ledger's existing FNV-1a witness chain and its own documented ADR-134 WitnessSigner gap, when witness records are Ed25519-signed either per-record or via an amortized batch-tail-only strategy (reusing rvf-types's existing Ed25519 primitive, zero new dependency), then batch-tail signing should reduce mean per-operation latency substantially relative to per-record signing, while both strategies retain identical detection of a "diligent" forgery (a fully self-consistent, forward-recomputed chain edit) that the existing unsigned chain-walk alone does not detect — subject to zero false negatives/positives across the full test matrix, with every number from a real cargo run --release execution.

This closes a gap named twice in this codebase (ops.rs's and ledger.rs's own tamper-evidence notes) and once more in the 2026-09-05 nightly's Next Research (item 4: "wire an Ed25519 WitnessSigner... so eviction receipts are signed, not just hash-chained").

Architecture

SignedWitnessSink<S: WitnessSink> wraps any inner sink and Ed25519-signs either every record (PerRecord) or the tail record of every batch_size-record run (BatchTail), domain-separated by purpose. verify_signed_chain combines the existing unsigned chain walk with signature checks cross-bound to each signed record's current content — the piece that actually defeats a diligent forgery. Reuses rvf-types::ed25519 (already an unconditional dependency of this crate, ADR-320) — zero new Cargo dependencies.

Files changed

  • crates/ruvector-agent-memory/src/witness_signing.rs (new module, 6 unit tests)
  • crates/ruvector-agent-memory/src/ledger.rs (TransactionalLedger::into_witness_sink)
  • crates/ruvector-agent-memory/src/lib.rs (module wiring + re-exports)
  • crates/ruvector-agent-memory/examples/witness_signing_bench.rs (benchmark)
  • docs/research/nightly/2026-09-16-witness-signer-agent-memory/{README,gist}.md
  • docs/adr/ADR-346-witness-signer-tarl-ledger.md + docs/adr/INDEX.md

Benchmark command

cargo test -p ruvector-agent-memory --lib witness_signing
cargo run --release -p ruvector-agent-memory --example witness_signing_bench

Real benchmark results (one cargo run --release execution, N=20,000 entries, 40,000 witness records)

Variant Mean latency/op Throughput Signatures Correctness
baseline (unsigned) 1.22µs 800,197 ops/s 0 PASS
PerRecord 137.42µs 7,275 ops/s 40,000 PASS
BatchTail{16} 9.65µs 103,206 ops/s 2,500 PASS
BatchTail{64} 3.77µs 262,868 ops/s 625 PASS
BatchTail{256} 2.01µs 489,882 ops/s 157 PASS

Diligent-forgery rejection (constructed by editing one interior record and consistently recomputing the chain forward — passes the unsigned verify_chain(), confirming the crate's own documented gap is real): both PerRecord and BatchTail{64} correctly reject it via verify_signed_chain.

Acceptance result: ACCEPT

Gate Threshold Measured Result
Honest-chain correctness 5/5 variants 5/5 PASS
Diligent-forgery rejection 2/2 strategies 2/2 PASS
Amortization (BatchTail{64} vs PerRecord) ≥5x lower latency 36.4x PASS
Signature-count exactness 40000/batch_size exact PASS
No pre-existing test regressed 28/28 pass 28/28 PASS

cargo test -p ruvector-agent-memory --lib: 34 passed, 0 failed (28 pre-existing + 6 new). cargo clippy -p ruvector-agent-memory --all-targets: clean. cargo fmt --check: clean.

Darwin / Flywheel / MetaHarness

No ruvector harness or flywheel/darwin CLI is installed in this environment (npx ruvector harness doctor --json → "could not determine executable to run"); npx metaharness --help resolves to a real, installed external-repo scaffolding tool, not an in-repo orchestrator for this workflow. Per the nightly process's own instruction not to assume tooling exists, this run did not fabricate Darwin generations or Flywheel gate calls — the two-parameter design space (signing strategy × batch size) was swept directly and exhaustively (16/64/256) instead of via evolutionary search. Full disclosure is in the research README's "MetaHarness/Flywheel/Darwin Role" sections.

Security review

  • No new cryptographic primitive — reuses ed25519-dalek via the crate's existing rvf-types dependency.
  • Domain-separated (crate-specific tag + purpose byte) so a per-record signature can never be replayed as a batch-tail signature.
  • verify_signed_chain's cross-check (signed chain_hash vs. the log's actual current content at that sequence) is the security-load-bearing step; documented as such, with the failure mode of omitting it spelled out.
  • Explicitly not claimed: this work does not evaluate the crate's own pre-existing "~2^32 work" FNV-1a second-preimage estimate — attempting that live risked reporting an unverified cryptanalytic figure, so it's flagged as open future work (Next Research item 2) rather than guessed at.
  • Key lifecycle management (generation/rotation/storage) is out of scope, matching the equivalent disclaimer in ruvector-retrieval-receipt::signing.

Main limitations

  • Single-threaded benchmark only (ledger is single-threaded by construction, unchanged).
  • No WASM/embedded-hardware measurement (a twice-already-deferred question for the sibling ruvector-retrieval-receipt crate).
  • No fault-injection measurement of BatchTail's larger blast radius if the signer crashes mid-batch (reasoned qualitatively).
  • FNV-1a preimage resistance is explicitly out of scope (see above).

Production recommendation

Promote as an opt-in capability (no default behavior change — every existing TransactionalLedger caller is unaffected). Recommend BatchTail (tuned batch size) for throughput-sensitive deployments, PerRecord where immediate per-record signature availability matters more than throughput. Composes for free with the existing witnessed_compaction eviction-witness path (ADR-345) with no additional code.

Research document / ADR / gist

  • Full methodology + evidence: docs/research/nightly/2026-09-16-witness-signer-agent-memory/README.md
  • Standalone technical write-up: docs/research/nightly/2026-09-16-witness-signer-agent-memory/gist.md
  • Architecture decision record: docs/adr/ADR-346-witness-signer-tarl-ledger.md

🤖 Generated with claude-flow

https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ


Generated by Claude Code

claude and others added 4 commits September 16, 2026 07:26
Closes the ADR-134 SS9 WitnessSigner gap named in ledger.rs/ops.rs's own
tamper-evidence notes and the 2026-09-05 nightly's Next Research item 4:
the FNV-1a witness chain is tamper-evident against accidental corruption
only, not a log-writing adversary.

Adds witness_signing::SignedWitnessSink<S: WitnessSink>, a decorator that
Ed25519-signs records either per-record or via an amortized batch-tail
strategy, reusing the crate's existing rvf-types Ed25519 primitive
(ADR-320) -- no new dependency. verify_signed_chain combines the existing
unsigned chain walk with signature checks cross-bound to each signed
record's current content, which is what actually detects a diligent,
fully self-consistent forgery that the unsigned walk alone cannot.

TransactionalLedger gains into_witness_sink() to recover a wrapped
sink's signed spans after a run.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ
Compares unsigned baseline vs PerRecord vs BatchTail{16,64,256} over a
20,000-entry (40,000 witness record) synthetic workload, release build,
real wall-clock timing, deterministic signing key. Also constructs a
diligent-forgery scenario (edit one record, recompute the chain forward
consistently) and confirms both signing strategies reject it while the
unsigned chain walk alone does not.

Run: cargo run --release -p ruvector-agent-memory --example witness_signing_bench

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ
Full hypothesis, methodology, raw benchmark evidence, acceptance gates,
failure modes, explicit scope boundary on the FNV-1a second-preimage
question, ecosystem-fit analysis, and next-research items for the
2026-09-16 nightly run (docs/research/nightly/2026-09-16-witness-signer-agent-memory).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ
Records the accepted decision, evidence summary, alternatives considered,
security scope, and rollback plan for witness_signing.rs. Updates the ADR
index (highest allocated number, new row).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01Gpkdh8ATrH62owAUHQhRUQ

ruvnet commented Sep 16, 2026

Copy link
Copy Markdown
Owner Author

CI status on d68df81c: every check passed except Tests (core-and-rest), which was cancelled after stalling ~4 hours mid-compile on ruvector-temporal-tensor-wasm — a crate this PR's diff does not touch (this PR only adds ruvector-agent-memory::witness_signing, which builds/tests/lints clean locally, including via cargo run --release for the benchmark). This is not this PR's failure: the same Workspace CI workflow's 5 most recent runs on main itself (runs #1435-#1440, e.g. 35054767523) all completed with conclusion cancelled too, with no change on this branch involved. I re-ran the failed job (rerun_failed_jobs on run 35068616234) rather than guessing at a code fix, since there's no diff-attributable root cause to fix. All other checks (Clippy, Rustfmt, Cargo check, Security audit, ADR numbering guard, Workspace membership guard, and every other Tests (*) job) are green.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants