Add vector-first Lance ANN path for Cypher vector rerank

[beinan]

Summary

• Add a VectorSearch.use_lance_index(True) opt-in to run a vector-first path when datasets are Lance datasets.
• When enabled, CypherQuery.execute_with_vector_rerank uses Lance ANN (nearest) to get top-k rows for the vector label, then executes Cypher on that reduced dataset.\n- Adds a Python test behind requires_lance to validate the path.

Notes

• This is an explicit opt-in and only applies when the Cypher query has no WITH/WHERE clauses; otherwise it falls back to the existing rerank behavior.
• Semantics can differ from candidate-then-rerank if you enable it on filtered queries; the guard avoids that by default.

Motivation

Using Lance datasets with ANN indices can significantly reduce latency for GraphRAG hybrid retrieval on large datasets by avoiding full-table reranking.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/lance-graph/src/lance_vector_search.rs	0.00%	12 Missing ⚠️

📢 Thoughts on this report? Let us know!

[beinan]

Manual Test Results ✅

Ran manual verification of the vector-first Lance ANN feature:

Test Setup

• Created Lance dataset with 5 documents and 3D embeddings
• Query vector: [1.0, 0.0, 0.0]

Results

Test	Description	Result
Test 1	Simple query + use_lance_index=True	✅ Returns Doc1, Doc2, Doc5 (vector-first path)
Test 2	Query with WHERE + use_lance_index=True	✅ Falls back to standard rerank, returns only tech docs
Test 3	Simple query + use_lance_index=False	✅ Same results as Test 1 (standard rerank)

Test Output

=== Test 1: Simple query with use_lance_index=True ===
Results (should be Doc1, Doc2, Doc5 - top 3 closest to [1,0,0]):
   d.id d.name      d.embedding
0     1   Doc1  [1.0, 0.0, 0.0]
1     2   Doc2  [0.9, 0.1, 0.0]
2     5   Doc5  [0.5, 0.5, 0.0]

=== Test 2: Query with WHERE (fallback to standard rerank) ===
Results (should only have tech docs: Doc1, Doc2, Doc4):
   d.id d.name      d.embedding  _distance
0     1   Doc1  [1.0, 0.0, 0.0]   0.000000
1     2   Doc2  [0.9, 0.1, 0.0]   0.141421
2     4   Doc4  [0.0, 0.0, 1.0]   1.414214

=== Test 3: Same query with use_lance_index=False ===
Results (should match Test 1):
   d.id d.name      d.embedding  _distance
0     1   Doc1  [1.0, 0.0, 0.0]   0.000000
1     2   Doc2  [0.9, 0.1, 0.0]   0.141421
2     5   Doc5  [0.5, 0.5, 0.0]   0.707107

Observations

1. Vector-first path works correctly - Uses Lance's to_table(nearest={...}) API
2. Fallback logic works - WHERE clause triggers fallback to standard rerank (note _distance column appears)
3. Results are consistent - Both paths return same ordering for simple queries

Minor Note

The vector-first path doesn't include the _distance column in output, while standard rerank does. This could be a follow-up enhancement if users need distance values with the Lance ANN path.

[ChunxuTang]

Currently, the use_lance_index=False will generate a _distance column, but the use_lance_index=True won't. I think it's better to consolidate them in a follow-up PR.