Skip to content

research(agent-memory): canonical-cactus-cut forgetting — mixed result, evidence retained (ADR-346) - #976

Draft
ruvnet wants to merge 5 commits into
mainfrom
claude/focused-darwin-b5f2ef
Draft

ruvnet wants to merge 5 commits into
mainfrom
claude/focused-darwin-b5f2ef

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Hypothesis

ADR-345 (docs/research/nightly/2026-09-05-mincut-gated-forgetting) rejected a mincut-gated agent-memory compaction policy because its min-cut backend (RuVectorGraphAnalyzer::partition()) was measured non-deterministic (50% degenerate results) and slow (1,800–2,700x a scalar baseline). This run attacks that specific bottleneck instead of picking a new topic: ruvector-mincut separately ships a canonical feature (CactusGraph::canonical_cut(), deterministic dense Stoer-Wagner) that nobody had measured against ADR-345's own rejection criteria.

Given the identical corpus ADR-345 used (6 clusters x 12 memories + 12 bridge
memories, 32-dim, k-NN k=8 cosine>=0.05),

when compacting to 50% using CactusGatedForgetting-Soft/Hard (backed by
CactusGraph::canonical_cut() instead of RuVectorGraphAnalyzer::partition())
vs. baseline CoherencePolicy and vs. ADR-345's own MincutGatedForgetting,

then (a) the cactus backend is deterministic, (b) compaction stays within
20x the scalar baseline (tighter than ADR-345's 100x bar), and (c) bridge
survival improves >=15pp over baseline while recall stays within 2pp,

subject to 100% tamper-detection on the reused eviction-witness chain.

Architecture

ruvector-agent-memory::graph_forget_cactus::CactusGatedForgetting mirrors ADR-345's MincutGatedForgetting field-for-field; the only change is the min-cut backend used to compute the boundary vertex set. New default-off feature: mincut-forget-cactus = ["mincut-forget", "ruvector-mincut/canonical"].

flowchart TD
    A[84-memory corpus] --> B["k-NN cosine graph (k=8)"]
    B --> C1["RuVectorGraphAnalyzer::partition() (ADR-345)"]
    B --> C2["CactusGraph::canonical_cut() (this PR)"]
    C1 --> D1[boundary set] --> E1[MincutGatedForgetting]
    C2 --> D2[boundary set] --> E2[CactusGatedForgetting]
    E1 --> F[select_survivors] --> G[compact_witnessed]
    E2 --> F
Loading

Files changed

  • crates/ruvector-agent-memory/src/graph_forget_cactus.rs — new policy, 3 unit tests
  • crates/ruvector-agent-memory/{Cargo.toml,src/lib.rs} — new mincut-forget-cactus feature
  • crates/ruvector-agent-memory/examples/cactus_{gated_forgetting_bench,determinism_probe,scaling_probe,seed_sensitivity_probe}.rs — reproducible benchmarks
  • docs/adr/ADR-346-canonical-cactus-cut-forgetting.md, docs/adr/INDEX.md (regenerated via scripts/adr-index.mjs)
  • docs/research/nightly/2026-09-10-canonical-cactus-forgetting/{README.md,gist.md}

Benchmark command & real results

cargo run --release -p ruvector-agent-memory --example cactus_determinism_probe --features mincut-forget-cactus
# 50 trials, 0% degenerate, 100% identical partition (ADR-345: 50% degenerate)

cargo run --release -p ruvector-agent-memory --example cactus_gated_forgetting_bench --features mincut-forget-cactus
# CoherenceWeighted (baseline)        16.7% survival  100.0% recall      41us
# MincutGatedForgetting-Soft (ADR-345)16.7%           100.0%        151802us
# CactusGatedForgetting-Soft (new)     8.3%           100.0%          1632us  (93x faster than ADR-345's backend)
# Tamper detection: 20/20
# => REJECT (survival-gap and 20x-speed criteria both fail)

cargo run --release -p ruvector-agent-memory --example cactus_seed_sensitivity_probe --features mincut-forget-cactus
# old(mincut) gap: mean=-3.3pp std=7.6pp, 0/10 seeds meet the 15pp bar
# new(cactus) gap: mean=-4.2pp std=8.5pp, 0/10 seeds meet the 15pp bar

cargo test --release -p ruvector-agent-memory --features mincut-forget-cactus: 34/34 pass. Default-feature build/test unaffected (verified separately).

Acceptance result: REJECT (mixed, evidence-backed)

  • (a) Determinism — CONFIRMED: 100% identical output vs. ADR-345's 50% degenerate rate.
  • (b) Speed — CONFIRMED vs. old backend, FAILED vs. pre-registered bar: 85–93x faster than RuVectorGraphAnalyzer at the same corpus, but still 39–40x the scalar baseline (bar was 20x).
  • (c) Bridge-survival benefit — FALSIFIED, and more convincingly than ADR-345 ever tested: neither backend beat the scalar baseline at the pre-registered seed, and a 10-seed sensitivity sweep found 0/10 seeds meeting the 15pp bar for either backend (mean gap ≈ 0, std ≈ 8pp — statistical noise, not a real effect). ADR-345's original single positive seed looks like a favorable-seed artifact in hindsight.

Root cause: a single global min-cut on a many-cluster graph finds one structurally weakest point in the entire graph, not all of the semantically-engineered bridge vertices a multi-cluster corpus constructs. Making the computation faster and deterministic doesn't change what it's fundamentally computing.

Darwin / Flywheel / MetaHarness

Re-verified not available as in-repo, callable orchestration (same finding as ADR-345, nine days apart): npx metaharness --help resolves to a generic project scaffolding CLI, not in-repo research orchestration; npx ruvector harness doctor --json has no such executable. Roles were performed serially in one session; this PR's documents are the durable evidence record in place of a Flywheel store.

Security review

No new attack surface: pure in-memory graph computation, no I/O, no new unsafe code. Reused EvictionWitnessChain re-verified at 20/20 tamper detections, unchanged from ADR-134/ADR-345.

Main limitations

  • Cactus backend's dense Stoer-Wagner still scales roughly cubically (25ms at n=100 → 8.2s at n=800), so despite the large constant-factor win it doesn't reach the ~2,000-memory scale ADR-345 originally wanted.
  • Only the Soft policy variant was swept across the 10 additional seeds (kept runtime bounded); Hard was only checked at the single pre-registered seed.
  • No comparison against tree_packing::canonical_mincut_fast (Gomory-Hu) or local/community-cut alternatives.

Production recommendation

Do not enable. Keep mincut-forget-cactus as an opt-in, default-off experimental feature alongside ADR-345's mincut-forget, retained as (1) a validated fact that ruvector-mincut's canonical feature is correct/fast/deterministic for future reuse, and (2) stronger, multi-seed evidence that "protect the global min-cut boundary" is not a viable signal for this eviction task — closing that path so a future nightly run doesn't reopen it without new information. The concrete next experiment (community/local cuts) is recorded in ADR-346's Open Questions and the research README's Next Research section.

Research document / ADR / gist

  • ADR: docs/adr/ADR-346-canonical-cactus-cut-forgetting.md
  • Research report: docs/research/nightly/2026-09-10-canonical-cactus-forgetting/README.md
  • Gist: docs/research/nightly/2026-09-10-canonical-cactus-forgetting/gist.md

🤖 Generated with claude-flow

https://claude.ai/code/session_01EREM53aWt5uo42K2d5NAEE


Generated by Claude Code

claude and others added 5 commits September 10, 2026 07:31
Attacks the rejection root cause of ADR-345's MincutGatedForgetting:
swaps RuVectorGraphAnalyzer::partition() (measured non-deterministic,
50% degenerate results; 1800-2700x baseline latency) for
ruvector-mincut's canonical feature (CactusGraph::canonical_cut(),
deterministic dense Stoer-Wagner), behind a new default-off
mincut-forget-cactus feature.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01EREM53aWt5uo42K2d5NAEE
…getting

Direct, same-process comparison against ADR-345's own policies on the
identical 84-memory corpus, plus determinism and scaling probes
comparable to ADR-345's mincut_determinism_probe/mincut_scaling_probe.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01EREM53aWt5uo42K2d5NAEE
…ting

Informational follow-up (not a redefinition of the pre-registered
acceptance test): characterizes whether the single-seed survival-gap
result generalizes. It does not: 0/10 additional seeds meet the 15pp
bar for either min-cut backend.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01EREM53aWt5uo42K2d5NAEE
Full methodology, raw benchmark output, and honest mixed acceptance
result (determinism and speed confirmed; bridge-survival benefit
falsified across 11 seeds) for the 2026-09-10 nightly run.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01EREM53aWt5uo42K2d5NAEE
Regenerated docs/adr/INDEX.md via scripts/adr-index.mjs.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01EREM53aWt5uo42K2d5NAEE

ruvnet commented Sep 10, 2026

Copy link
Copy Markdown
Owner Author

CI status: pre-existing, unrelated hang in Tests (core-and-rest)

One check on this PR's head (224d037f) is red: Tests (core-and-rest) (Workspace CI / ci.yml) ran its "Run tests (core-and-rest)" step for ~4 hours (07:33–11:33 UTC) before being force-cancelled, almost certainly by a job-level timeout-minutes. Every other check on this PR (49/50 check runs) passed, including Clippy, Clippy (deny warnings), Rustfmt, Security audit, cargo audit, cargo deny, all platform builds, and every other Tests (*) shard.

This is not caused by this PR's diff. Evidence:

  1. This PR only touches crates/ruvector-agent-memory (new module + 4 examples, behind a new default-off feature) and docs. cargo test --release -p ruvector-agent-memory --features mincut-forget-cactus and the default-feature build both complete in seconds locally, with 34/34 and 31/31 tests passing respectively.
  2. Checking the last 5 Workspace CI runs on main itself (before this branch existed) via the Actions API: all 5 have overall conclusion cancelled, going back to and including commit 097493818 (Cleanup/hardening: fix five open correctness/security defects, adopt ADR-341 invariants #933) — which is itself titled "fix... a deadlock... nondeterministic under the unseeded test RNG; burned a 3h52m CI job" for ruvector-delta-index. That commit's own CI run also shows cancelled, and every run after it does too. That known fix is already an ancestor of this PR's base (edaffffb3), so it's already inherited here — but the hang pattern clearly persists past it, meaning there's a distinct, still-unresolved nondeterministic hang somewhere in the core-and-rest test group, unrelated to ruvector-agent-memory.

No fix exists yet to port: I don't have a specific failing-test lead beyond what's above (the job's own log ends mid-hang with no assertion/panic — it was hard-killed by timeout, not a normal test failure), and I don't have visibility into which specific test in that broad shard is hanging this time. Widening this PR's scope to bisect an unrelated, pre-existing, intermittent multi-crate test hang isn't appropriate here.

I'm not spending the one permitted re-run on this: the base-branch evidence above (5/5 recent main runs cancelled the same way, including the fix commit itself) already establishes this as a systemic, reproducing-every-time issue rather than a one-off flake, so a re-run would very likely just cost another ~4 hours for the same result.

Recommendation: this PR's own changes are fully green (build, tests, clippy, fmt, security, ADR/workspace guards all pass); the one red check reflects a pre-existing, repo-wide CI infrastructure issue that predates this branch and should be tracked/fixed independently (worth its own issue — likely a lock-reentrancy or resource-contention hang similar in shape to the one #933 already fixed once, recurring elsewhere in the core-and-rest shard).


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants