Q30 and IC6 are 10–14% slower after #55 #56

Problem

Benchmarking #55 against the published 0.2.0 wheel showed that most queries got faster or stayed level, but two got slower by more than run-to-run noise:

These are medians of medians from a focused re-run: two ABBA cycles (0.2.0, #55, #55, 0.2.0) with at least 15 rounds per query. The 0.2.0 and #55 ranges don't overlap (Q30 in memory: 40.9–42.0 vs 45.6–46.3 ms; IC6: 22.4–22.6 vs 25.5–26.9 ms). Every result assertion passed, so the answers are unchanged.

For context, the same comparison showed the IC suite about 22–25% faster overall (IC12 about 7×). The 30-query suite was level overall (0.97 in memory, 1.00 lazy).

Possible cause (unverified)

#55 brought several execution changes at once: bounded-chunk CSR expansion (#31), iterative variable-length traversal, filter-column reuse, per-snapshot plan templates and schema-aware type checks. A candidate is per-batch overhead from bounded-chunk expansion on high fan-out traversals, but no profile has confirmed it. Other changes could be responsible instead.

Suggested next steps

• Compare explain() output for Q30 and IC6 between 0.2.0 and main to see whether the physical plans differ.
• Profile both queries on main (for example with samply or cargo flamegraph on a release build).
• If expansion chunking is the cause, measure the effect of a larger chunk size on these queries, keeping the bounded-memory property fix(csr): stream high-degree expansion in bounded chunks #31 added.

How to reproduce

The 30-query suite is in graph-benchmark-ldbc and the IC suite in graph-benchmark-ldbc-complex, both in their lance_csr/ directories. Install a release wheel of main (cd python && uv run --no-sync maturin build --release), then run each engine directory's suite with the published flags, overlaying the wheel with uv run --frozen --with <wheel>. Select one query with -k query30 or -k query6, and set LANCE_CSR_MODE=memory or lazy.