Skip to content

fix(ruvector-mincut): deterministic minimum-cut witness materialization (ADR-347) - #969

Draft
ruvnet wants to merge 4 commits into
mainfrom
claude/focused-darwin-1asj60
Draft

ruvnet wants to merge 4 commits into
mainfrom
claude/focused-darwin-1asj60

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Summary

Nightly research run (2026-09-07) directly following up on the 2026-09-05
nightly (ADR-345, "Mincut-Gated Forgetting"), which rejected a
ruvector-agent-memory compaction policy for two independent reasons and
left one as an explicit open question: "What specifically causes the
measured non-determinism inside ruvector-mincut... instance construction
order, witness materialization, or something else?"

This PR answers it. WitnessHandle::materialize_partition()
(crates/ruvector-mincut/src/instance/witness.rs) computed the complement
side V \ U as (0..=membership.max()).filter(not in U) -- deriving the
vertex universe from the found cut set's own highest member rather than
the graph's actual vertex set. Since a minimum cut's small side essentially
never contains the graph's globally-highest vertex ID, every vertex
numbered above it was silently dropped from both returned sides. This
was the dominant cause of ADR-345's ~50% "empty/unusable" measurement;
reproduction on this checkout (with a corrected probe graph -- see ADR-347's
"A trap in the original probe") measured 99-100%.

Two fixes, both default-on, no feature flag:

  • Fix A: new WitnessHandle::materialize_partition_within(universe),
    and RuVectorGraphAnalyzer::partition() switched to call it with the
    graph's real vertices() instead of guessing a universe.
  • Fix B: sort the HashSet-derived vertex/seed orderings in
    BoundedInstance's tie-breaking paths (brute_force_min_cut,
    search_for_cuts), removing RandomState hash-seed randomization as a
    source of which tied-optimal witness gets returned.

Not in scope: ADR-345's other, independent rejection cause (latency
scaling at small n) is unaffected and remains open -- see ADR-347's "Open
Questions" for the natural next nightly topic.

Note on ADR numbering: originally filed as ADR-346; renumbered to
ADR-347 after discovering open PR #965 independently claimed ADR-346 first
(filed a day earlier). See the numbering-fix commit on this branch.

Architecture

See ADR-347
for full hypothesis, decision, and rationale, and the
nightly research report
(plus gist
and raw transcripts) for the complete narrative and evidence.

Benchmark command

TRIALS=200 cargo run --release -p ruvector-mincut --example determinism_probe

Three fixed topologies (n=19 brute-force path, n=21 and n=85 LocalKCut-oracle
path), 200 fresh-analyzer trials each, isolated via git stash across
baseline / fix-A-only / fix-A+B states of the identical probe binary.

Real benchmark results

Topology Baseline (main) Fix A only Fix A + Fix B
n=19, empty/degenerate 200/200 (100%) 0/200 (0%) 0/200 (0%)
n=19, distinct partitions/200 0 (all degenerate) 2 1
n=21, empty/degenerate 200/200 (100%) 0/200 (0%) 0/200 (0%)
n=21, distinct partitions/200 0 2 1
n=85, empty/degenerate 198/200 (99%) 0/200 (0%) 0/200 (0%)
n=85, distinct partitions/200 2 2 1

Fix A alone eliminates all empty/degenerate results (confirming it as the
dominant defect). Fix B closes the remaining gap to full determinism --
exactly one partition observed across all 600 post-fix trials. Latency is
unaffected by either fix beyond noise (largest delta: n=19's
1236.9ms → 1167.6ms avg/call, a 5.6% improvement attributable to sort
overhead being smaller than search-order variance, not a claimed
optimization -- this PR's scope is correctness, not speed).

Full raw transcripts (including the interim diagnostic that caught a bug in
this run's own first-draft probe graph):
raw-runs.txt.

Acceptance result

ACCEPT. All of this PR's stated hard gates were met: 0% empty/degenerate
on all three topologies, exactly 1 distinct partition per topology across
200 trials, and no regression in either crate's test suite.

Darwin result

Not applicable this run -- a root-cause bug-fix investigation with exactly
one correct fix per identified defect, not a bounded-parameter-search
problem. See ADR-347 / research README for detail.

Flywheel result

No ruvector harness flywheel CLI surface was resolvable in this checkout
(npm error could not determine executable to run). ADR-347 and the
nightly research report serve as this run's evidence trail, in the same
spirit as ADR-345's, and explicitly carry forward the unresolved latency
question for the next nightly in this lineage.

Security review

No new cryptographic primitive. materialize_partition_within is a pure
function over already-public data (WitnessHandle::contains); it does not
change what a witness exposes, only how completely a caller reconstructs
both sides. Fixing silent vertex loss is itself a hardening for any future
witness-auditing code that expects .partition()'s two sides to sum to the
graph's vertex count.

Test evidence

  • cargo test --release -p ruvector-mincut --lib: 515 passed, 0 failed
    (5 pre-existing, unrelated ignores).
  • cargo test --release -p ruvector-mincut --doc: 27 passed, 0 failed
    (includes the new materialize_partition_within doctest).
  • cargo test --release -p ruvector-agent-memory --features mincut-forget
    (the downstream consumer from ADR-345): 63 passed, 0 failed across
    lib + 3 integration test binaries.
  • cargo clippy --release -p ruvector-mincut --lib: no new warnings.
  • cargo fmt -p ruvector-mincut -- --check: clean.

Main limitations

  • Evidence is limited to three synthetic topologies and BoundedInstance's
    two internal code paths; real (non-synthetic) embedding graphs at larger
    scale were not tested this run.
  • Latency scaling (the n=19 case's ~1.2s/call, matching ADR-345's "outlier
    69s at n=19" finding qualitatively) is not fixed here and remains
    open -- this PR only confirms neither fix worsens it.
  • The agentic feature's parallel query path was not touched or exercised.

Production recommendation

Ship as-is: this is a default-on bug fix in already-shipped public API
(RuVectorGraphAnalyzer::partition()), not an experimental addition. Every
existing and future caller (CommunityDetector, GraphPartitioner, and any
out-of-tree consumer) gets a complete, deterministic result automatically,
with no migration required.

Files changed

  • crates/ruvector-mincut/src/instance/witness.rs -- new
    materialize_partition_within, strengthened docs, 2 new tests.
  • crates/ruvector-mincut/src/integration/mod.rs -- partition() fixed;
    new test_partition_deterministic_and_complete regression test.
  • crates/ruvector-mincut/src/instance/bounded.rs -- sorted tie-breaking
    in brute_force_min_cut / search_for_cuts.
  • crates/ruvector-mincut/examples/determinism_probe.rs -- new,
    reproducible probe backing this PR's evidence.
  • docs/adr/ADR-347-deterministic-mincut-witness-materialization.md -- new.
  • docs/adr/INDEX.md -- new entry, counters updated.
  • docs/research/nightly/2026-09-07-deterministic-mincut-witness/ -- new
    (README.md, gist.md, raw-runs.txt).

Next research

ADR-345's Open Question #1 (does DynamicMinCut/ClusterHierarchy, used
directly instead of MinCutWrapper's bounded-instance sweep, avoid the
latency scaling this PR left untouched?) is the natural next nightly in
this lineage -- see ADR-347's "Open Questions" and the research README's
"Next research" for the full list.


🤖 Generated with claude-flow

https://claude.ai/code/session_01CgVaCy8r8yHCWpKASuEi5v

claude and others added 4 commits September 7, 2026 08:02
WitnessHandle::materialize_partition() derived its complement-side vertex
universe from the found cut set's own maximum vertex ID rather than the
graph's real vertex set, silently dropping every higher-numbered vertex
from both returned sides whenever the (typically small) cut set didn't
happen to contain the graph's highest ID -- nearly always. This was the
dominant cause of the ~50% "empty/unusable" RuVectorGraphAnalyzer::partition()
results flagged as an open question by the 2026-09-05 nightly (ADR-345);
reproduction on this checkout measured 99-100%.

Add WitnessHandle::materialize_partition_within(universe) and switch
RuVectorGraphAnalyzer::partition() to call it with the graph's actual
vertex set. Also sort the HashSet-derived vertex/seed orderings in
BoundedInstance's tie-breaking paths (brute_force_min_cut,
search_for_cuts), removing RandomState-seeded hash order as a source of
which tied-optimal witness gets returned.

Adds crates/ruvector-mincut/examples/determinism_probe.rs, the
reproducible probe backing this fix's evidence (see ADR-346 and
docs/research/nightly/2026-09-07-deterministic-mincut-witness/).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01CgVaCy8r8yHCWpKASuEi5v
Records the hypothesis, root-cause analysis, before/after evidence, and
decision for the ruvector-mincut fix in the prior commit. Directly
answers ADR-345's Open Question #2. Regenerates the ADR index entry.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01CgVaCy8r8yHCWpKASuEi5v
Full methodology, raw benchmark transcripts (baseline / fix-A-only /
fix-A-plus-B across three topologies), ecosystem-fit analysis, and a
standalone gist for the deterministic-mincut-witness nightly run.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01CgVaCy8r8yHCWpKASuEi5v
Open PR #965 ("structural-time keyframe retention for agent memory")
independently claimed ADR-346 first (filed 2026-09-06, a day before this
PR). Following this repo's established collision-resolution convention
(see ADR-341's own renumbering history), the later claim renumbers.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01CgVaCy8r8yHCWpKASuEi5v
@ruvnet ruvnet changed the title fix(ruvector-mincut): deterministic minimum-cut witness materialization (ADR-346) fix(ruvector-mincut): deterministic minimum-cut witness materialization (ADR-347) Sep 7, 2026

ruvnet commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

CI status note (not this PR's failure): the Tests (core-and-rest) check on this PR's head (8f7d328d5) shows cancelled after running ~4 hours before being killed. This is not introduced by this PR's diff — the same "Workspace CI" workflow shows the identical cancelled conclusion on main itself for every one of the last 5 consecutive pushes (runs 33985573600 / ADR-345 merge, 33981845489 / ADR-344, 33978199725 / ADR-343, 33974331920 / ADR-342, 33970595850 / #933), spanning before and after PR #933's own deadlock fix (which explicitly mentions "burned a 3h52m CI job" as the symptom it was trying to fix — the pattern evidently persists). I don't see an open PR or fix addressing this specific timeout, so I'm not porting anything speculative. All other 49 checks on this PR pass, including Cargo check, Clippy, Rustfmt, ADR numbering guard, and the ruvector-mincut-relevant test shards. Standing down on this one check as pre-existing infra rather than widening this PR's scope to chase it; will keep watching in case a fix lands upstream to port.


Generated by Claude Code

@ruvnet ruvnet left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dream exact-head review — INCONCLUSIVE

The candidate addresses both identified causes: HashSet-derived traversal is sorted, and partition materialization now receives the graph's actual vertex universe. The 30-trial regression is a useful deterministic falsification test. Native, WASM-dedup, regression, supply-chain, formatting, and clippy workflows pass.

Required Workspace CI was cancelled, so the frozen native/WASM/workspace acceptance gate is incomplete. Please obtain a green exact-head Workspace CI result and retain the complete-partition assertion on sparse/non-contiguous vertex sets. No performance or cross-backend parity claim is promoted until that final gate passes.

ruvnet commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Re: the review's request for a green exact-head Workspace CI result — pulled the actual job log for Tests (core-and-rest) (job 101736522536) rather than just re-asserting it's pre-existing. It's a compile-time hang, not a test hang: the last output is Compiling ruvector-temporal-tensor-wasm v2.3.0 at 12:12:37Z, then total silence until GitHub's own cancellation at 16:09:12Z (##[error]The operation was canceled.) — cargo nextest never got past building the workspace, so no test body ever ran. That's consistent with the identical pattern on main across the last 5+ pushes I cited earlier: a workspace-wide compile hang unrelated to this PR's 4-file diff.

Since it died before any test body ran, I've used this PR's one re-run allowance: triggered rerun_failed_jobs on the run (34120280378) rather than re-asserting a guess. Watching for the result — will report back with either a green run or, if it hangs identically again, treat that as confirmation this is a real, reproducible workspace-wide build issue (not flake) and dig into which specific crate's compilation is actually stuck rather than re-running further.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants