research(nightly): sim-join — vector similarity join for knowledge graph edge induction

[ruvnet]

Summary

Adds crates/ruvector-sim-join — a zero-dependency Rust PoC implementing vector similarity join (all-pairs discovery over two sets A, B where cosine_similarity(a, b) ≥ θ). Introduces ADR-273 and the 2026-07-30 nightly research document.

What was built

• ruvector-sim-join crate with three measurable variants behind a SimJoin trait:
  • BruteJoin — exact O(|A|·|B|·d) scan, recall = 1.000 (ground truth baseline)
  • LshJoin — random-hyperplane LSH bucket join (T tables × B-bit codes)
  • IvfJoin — Lloyd's k-means IVF partition join (K clusters, p probe cells)
• Deterministic benchmark binary (cargo run --release -p ruvector-sim-join --bin benchmark)
• 12 unit tests, all passing
• ADR-273 (docs/adr/ADR-273-sim-join.md) — architecture decision record
• Research doc (docs/research/nightly/2026-07-30-sim-join/README.md)
• SEO gist (docs/research/nightly/2026-07-30-sim-join/gist.md)

Key benchmark results (Linux x86_64, release build, real measured numbers)

n	d	Variant	Mean (µs)	Recall	Speedup
2000	128	BruteJoin	255,902	1.000	1.00×
2000	128	LshJoin (4b×10t)	486,740	0.887	0.53×
2000	128	IvfJoin (K=16, p=3)	169,755	0.998	1.51×
5000	128	IvfJoin	1,276,831	0.986	1.29×

Acceptance criteria: BruteJoin recall = 1.000 ✓ · LshJoin recall ≥ 0.70 ✓ · IvfJoin recall ≥ 0.70 ✓ · At least one approx variant faster than brute at n=2000 ✓ (IvfJoin 1.51×)

Key research finding

LSH join degrades below θ ≈ 0.40 because bucket sizes explode at high pair density — verification dominates and LSH becomes slower than brute force. IVF is regime-consistent and is the recommended approximate strategy for RuVector's primary use cases (knowledge graph edge induction, agent memory cross-reference, RAG deduplication).

Ecosystem connections

• ruvector-graph — can ingest join output as semantic edges
• ruvector-agent-memory — self-join for periodic memory cross-reference and deduplication
• ruvector-bounded-rag — pre-indexing corpus deduplication via self_join(chunks, 0.95)
• ruvector-mincut — join output becomes the edge set for graph partitioning

Files changed

Cargo.toml                                              # workspace member added
crates/ruvector-sim-join/Cargo.toml
crates/ruvector-sim-join/src/lib.rs                    # SimJoin trait, Pair, cosine_similarity, recall
crates/ruvector-sim-join/src/brute.rs                  # BruteJoin
crates/ruvector-sim-join/src/lsh.rs                    # LshJoin
crates/ruvector-sim-join/src/ivf.rs                    # IvfJoin
crates/ruvector-sim-join/src/dataset.rs                # ClusteredDataset + Lcg64 PRNG
crates/ruvector-sim-join/src/bin/benchmark.rs          # benchmark binary
docs/adr/ADR-273-sim-join.md
docs/research/nightly/2026-07-30-sim-join/README.md
docs/research/nightly/2026-07-30-sim-join/gist.md

---

Generated by Claude Code