Skip to content

fix(mincut): deterministic, non-degenerate RuVectorGraphAnalyzer::partition() - #979

Draft
ruvnet wants to merge 4 commits into
mainfrom
claude/focused-darwin-4kpt0q
Draft

ruvnet wants to merge 4 commits into
mainfrom
claude/focused-darwin-4kpt0q

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

Nightly research run (autonomous). Root-causes and fixes the
non-determinism in ruvector_mincut::RuVectorGraphAnalyzer::partition()
that the 2026-09-05 nightly (docs/research/nightly/2026-09-05-mincut-gated-forgetting,
ADR-345)
measured (15/30 calls returning empty/degenerate on a byte-identical graph)
but did not locate.

Two independent bugs, both fixed, no public API signature changes:

  1. DynamicGraph::vertices()/edges() iterated a DashMap directly.
    DashMap's default hasher is randomly seeded per instance, so its
    iteration order is not stable across process runs even for identical
    insertions — downstream tie-breaking (BoundedInstance's seed selection
    and bitmask-to-vertex assignment) silently picked a different valid cut
    on every run.
  2. WitnessHandle::materialize_partition() inferred the graph's vertex
    range from max(U) (the cut side's own membership) instead of the
    graph's real vertex count, so the complement V \ U came back truncated
    or empty whenever U didn't happen to contain the graph's
    highest-numbered vertex — the common case, not an edge case. This was
    the direct, deterministic cause of most empty results; fixing bug 1
    alone (see commit history) made the symptom measurably worse before
    bug 2 was found, which is itself a notable finding about diagnosing
    compound bugs from a single aggregate metric.

Fix: sort vertices()/edges() at the DynamicGraph read boundary;
sort the HashSet-derived vertex lists BoundedInstance uses for
tie-breaking; have RuVectorGraphAnalyzer::partition() build both cut
sides from the graph's real vertex list via witness.contains() instead of
materialize_partition().

Eighteen crates depend on ruvector-mincut directly (including WASM/Node
bindings, ruvector-agent-memory, prime-radiant, cognitum-gate-kernel,
mcp-brain-server); all get this correctness fix for free on next rebuild
with zero migration cost.

Explicitly out of scope: the separately-documented latency scaling
problem (77ms–11.4s across 50–400 vertices) is untouched — this is a
correctness fix, not a performance fix.

Evidence

Reused the prior nightly's own unmodified reproduction script
(crates/ruvector-agent-memory/examples/mincut_determinism_probe.rs) for
an honest before/after comparison:

# Prior nightly (documented): 15/30 (50%) empty/degenerate
# This run, two independent 30-trial runs:
trials=30 elapsed=33.55s avg_per_call=1118.3ms empty_or_degenerate=0 (0%) bridge_detected_as_boundary=30 (100%)
trials=30 elapsed=33.88s avg_per_call=1129.5ms empty_or_degenerate=0 (0%) bridge_detected_as_boundary=30 (100%)

New regression test (crates/ruvector-mincut/tests/determinism_tests.rs)
asserts a strictly stronger property — byte-identical partitions across 30
repeated calls, not just "non-empty":

running 3 tests
test graph_vertices_and_edges_are_sorted ... ok
test partition_is_never_degenerate_on_connected_graph ... ok
test partition_is_stable_across_repeated_calls ... ok
test result: ok. 3 passed; 0 failed

Full pre-existing ruvector-mincut suite (lib + all 10 integration test
files): 644 passed, 0 failed, 5 pre-existing ignored.

Downstream sanity check — ruvector-agent-memory's graph_forget tests
(the crate/feature that originally surfaced this bug, mincut-forget
feature): 3 passed, 0 failed.

cargo clippy --release -p ruvector-mincut --lib --tests -- -D warnings:
clean (no new warnings).

Files changed

  • crates/ruvector-mincut/src/graph/mod.rs — sort vertices()/edges()
  • crates/ruvector-mincut/src/instance/bounded.rs — sort seed/tie-break
    vertex lists; deterministic witness seed
  • crates/ruvector-mincut/src/instance/witness.rs — doc comment only
    (documents materialize_partition()'s scope limitation)
  • crates/ruvector-mincut/src/integration/mod.rspartition() uses
    graph.vertices() + witness.contains() instead of
    materialize_partition()
  • crates/ruvector-mincut/tests/determinism_tests.rs — new regression test
  • docs/adr/ADR-346-deterministic-mincut-witness-partition.md — new ADR
  • docs/adr/INDEX.md — regenerated via node scripts/adr-index.mjs
  • docs/research/nightly/2026-09-11-mincut-partition-determinism/README.md
    — full nightly research report
  • docs/research/nightly/2026-09-11-mincut-partition-determinism/gist.md
    — standalone technical write-up

Benchmark commands (reproducible)

cargo build --release -p ruvector-agent-memory \
  --example mincut_determinism_probe --features mincut-forget
TRIALS=30 ./target/release/examples/mincut_determinism_probe

cargo test --release -p ruvector-mincut --lib --tests

Acceptance result

ACCEPT — see ADR-346
and the nightly README
for the full evidence table, falsification criteria, and limitations
(only the <20-vertex brute-force path was end-to-end re-benchmarked;
concurrent-mutation and WASM/Node binding smoke tests are flagged as
follow-up, not done this run).

MetaHarness / Flywheel / Darwin capability discovery

Re-verified per the nightly process's own rule — unchanged from the prior
nightly: npx metaharness --help resolves to a generic project-scaffolding
CLI not wired into this repo; npx ruvector harness doctor --json fails
(no such CLI package installed). No Darwin evolutionary search was run —
this was a targeted root-cause diagnosis of an already-reported bug, not a
parameter search over a candidate population.

Security / governance

No security-relevant surface changed (no new I/O, no trust-boundary or
signing-path changes). No schema, wire-format, or persisted-data changes.

Limitations / follow-up (see ADR-346 "Open Questions")

  • search_for_cuts's seed-ordering fix (the >=20-vertex LocalKCut path)
    was applied by the same reasoning but not independently re-benchmarked at
    that size this session.
  • WASM/Node bindings not rebuilt/smoke-tested this run.
  • No concurrent-mutation stress test.
  • Does not re-attempt ADR-345's MincutGatedForgetting acceptance
    benchmark (that rejection has a second, independent, unaddressed latency
    cause).

🤖 Generated with claude-flow

https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT


Generated by Claude Code

claude and others added 4 commits September 11, 2026 07:41
RuVectorGraphAnalyzer::partition() returned an empty/degenerate result in
~50% of repeated calls against a byte-identical graph (first measured by
the 2026-09-05 nightly run, root cause left unresolved). Two independent
bugs:

- DynamicGraph::vertices()/edges() iterated a DashMap directly. DashMap's
  default hasher is randomly seeded per instance, so iteration order isn't
  stable across process runs even for identical insertions. Downstream
  tie-breaking (BoundedInstance's seed selection and bitmask-to-vertex
  assignment) silently picked a different valid cut on every run.
- WitnessHandle::materialize_partition() inferred the graph's vertex range
  from max(U) (the cut side's own membership) instead of the graph's real
  vertex count, so V \ U came back truncated or empty whenever U didn't
  happen to contain the graph's highest-numbered vertex — the common case.

Fix: sort vertices()/edges() at the DynamicGraph read boundary, sort the
HashSet-derived vertex lists BoundedInstance uses for tie-breaking, and
have RuVectorGraphAnalyzer::partition() build both cut sides from the
graph's real vertex list via witness.contains() instead of
materialize_partition(). No public API signature changes.

Verified against the exact reproduction script from the 2026-09-05 run:
0/60 empty results across two 30-trial runs (was 15/30, 50%), plus a new
regression test asserting byte-identical partitions across repeated calls.
Full ruvector-mincut suite: 644 passed, 0 failed, 5 pre-existing ignored.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Records the decision, evidence, alternatives considered, and open
questions for the ruvector-mincut determinism fix. Regenerated
docs/adr/INDEX.md via scripts/adr-index.mjs.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Full methodology, root-cause diagnostic narrative (including the
intermediate finding that fixing only the DashMap-ordering bug made the
symptom look worse before the second, independent witness-partition bug
was found), benchmark results, limitations, and next research for the
2026-09-11 nightly run.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT
Fixes the Rustfmt CI check on PR #979.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01VFTtb2ZrTNkmpWVQfuiKkT

ruvnet commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

CI status: Tests (core-and-rest-heavy) failed on the current head (f891db8), but it's not this PR's failure.

The job ran 3547 tests across the workspace; exactly one failed:
ruvllm reasoning_bank::tests::test_trajectory_recording. ruvllm's
reasoning_bank module is not touched by this PR (which only changes
crates/ruvector-mincut/* source/tests and docs — no Cargo.toml/
Cargo.lock changes, so no dependency-version mechanism could connect the
two either). The failure immediately follows a test flagged SLOW [>600.000s] in the same job's log
(ruvector-nervous-system::retrieval_quality pattern_separation_collision_rate, which did pass at 809.9s), consistent
with timing/resource-contention flakiness in a long-running shared job
rather than a real regression.

Every other check on this head is green: Rustfmt, Clippy, Cargo check, and
all other test shards (core-platform, vector-index, core-and-rest,
core-and-rest-wasm, core-and-rest-examples, ruvix, rvagent,
ml-research-rest, research-nightly, plus all repo policy/security
guards) passed.

I don't have a fix to port — I did not touch ruvllm and found no open PR
addressing this specific test. I'll re-run the failed job once (per this
session's flake-confirmation budget) once the current workflow run
finishes (a re-run attempt while jobs are still in flight was rejected by
GitHub with "workflow is already running"), and will report back if it's
still red on retry.


Generated by Claude Code

ruvnet commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

Second failure, also not this PR's: Tests (core-and-rest) failed with a
Rust compile error, not a test failure:

error[E0275]: overflow evaluating the requirement `&mut Vec<u8>: std::io::Write`
  = help: consider increasing the recursion limit by adding a `#[recursion_limit = "8192"]` attribute to your crate (`ruvector_filter`)
error: could not compile `ruvector-filter` (lib test) due to 1 previous error

ruvector-filter is not a dependent of ruvector-mincut and this PR
touches no Cargo.toml/Cargo.lock/ruvector-filter source. It's also not
newly fragile: crates/ruvector-filter/src/lib.rs already carries
#[recursion_limit = "4096"] on main as a prior workaround for this
exact serde-derive/serde_json::Serializer trait-resolution recursion —
this run just tipped back over that limit, unrelated to anything in this
diff.

I don't have a fix to port (no open PR found addressing it, and bumping
ruvector-filter's own recursion limit would be scope creep on this PR).
Both failed jobs from this run
(Tests (core-and-rest-heavy)'s ruvllm flake and this one) have been
re-queued via rerun_failed_jobs — this is this session's one
flake-confirmation re-run for this workflow run. Will report back once
they complete; if either is still red, that confirms it's a real,
pre-existing base-branch issue rather than transient flakiness, and I'll
keep watching without pushing an out-of-scope fix.


Generated by Claude Code

ruvnet commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

Re-run results:

  • Tests (core-and-rest-heavy)success on re-run. Confirms the
    ruvllm reasoning_bank::tests::test_trajectory_recording failure was
    transient, not a real regression.
  • Tests (core-and-rest)cancelled, not fixed or re-failed the same
    way. Its log shows ruvector-filter actually compiled cleanly this time
    (no recurrence of the E0275 recursion-limit error), but the job then hung
    for ~4 hours after starting ruvector-temporal-tensor-wasm and was
    cancelled (The operation was canceled at 14:31, after last progress at
    10:34). The original failed attempt also spent ~2.7 hours between that
    same compile step and its error. Neither ruvector-filter nor
    ruvector-temporal-tensor-wasm is touched by this PR (still only
    crates/ruvector-mincut/* + docs), so this reads as a pre-existing,
    multi-hour-runtime characteristic of this particular CI shard on this
    large workspace — not something introduced by this diff.

I've used this session's one re-run for this workflow run already
(rerun_failed_jobs, which covered both jobs above). Given this job takes
multiple hours per attempt, I'm not triggering a third multi-hour run
speculatively; I'll pick this back up from the next CI or review event on
this subscription. This PR is still in draft, so nothing is currently
blocked on it. Every other check on the head commit (Rustfmt, Clippy,
Cargo check, and all other test shards) is green.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants