IC5 is about 20% slower in memory mode since named hops moved to the CSR #92

Activity

1. prrao87

   added

   enhancementNew feature or requestNew feature or request

   on Oct 30, 2026
2. prrao87

   added a commit that references this issue on Oct 31, 2026

   perf(csr): filter in-memory relationship columns per chunk (#100)

   ...

   20c1a36
3. prrao87 commented on Oct 31, 2026

   

   prrao87

   on Oct 31, 2026

   MemberAuthor

   More actions

   Are named hops on the CSR worth it? A controlled comparison

   #87 changed several things at once, so to isolate the routing decision I built main at 20c1a36 (after #100) twice: as is, and with named relationship hops sent back to the join path (a one-line change in the hop planner; everything else, edge rows included, identical). Release wheels, CONTRIBUTING.md harness, both suites, both modes, alternated twice; Apple M5, 24 GB, macOS 26.6.2. Every result check passed.

   Suites, named hops on joins vs main:

   Query	Memory	Lazy
   IC5 (membership, about 800K edges, filtered)	-18% (88.3 to 72.5 ms)	-12% (108.5 to 95.4 ms)
   IC7 (like, one person's likes)	+26% (16.4 to 20.6 ms)	+39% (23.2 to 32.2 ms)
   IC11 (workAt, friends' jobs)	+21% (3.0 to 3.7 ms)	+7% (5.5 to 5.8 ms)
   IC1 and all others	within noise	within noise

   IC suite geomean, joins vs main: 1.022 in memory, 1.017 lazy. The custom 30 queries name no relationship and stay within noise.

   Sweep: one named hasMember hop (1.6M edges) from N start persons, CSR (main) vs joins, median ms:

   Start persons (edges)	Memory, return m.joinDate	Lazy, return m.joinDate	Lazy, filter on m.joinDate
   1 (3)	0.55 vs 1.11	0.75 vs 6.0	0.80 vs 6.4
   100 (17K)	1.0 vs 3.6	8.3 vs 15.9	7.2 vs 14.1
   1,000 (174K)	5.7 vs 7.4	67.0 vs 17.6	13.8 vs 19.1
   9,892, all (1.6M)	52.2 vs 51.7	566.8 vs 71.0	74.9 vs 83.9

   A named knows hop behaves like the memory column: the CSR wins at every size, by 1.25 to 3.4 times.

   Conclusions:

   1. Keep named hops on the CSR. From few start rows, the usual case, it's 2 to 3.6 times faster in memory and up to 8 times faster lazily, and in memory it's never meaningfully slower.
   2. Lazy edge columns have no read-once path. In lazy mode, a hop that returns a relationship column fetches it from Lance by row, one scattered batch at a time, for every expanded edge. Relationship filters switch to one read of the column (the mask) and target columns do too (Read target columns once when a CSR hop fetches a large share of a node table #86), but plain edge columns don't, hence 67 vs 18 ms at 1,000 starts and 567 vs 71 ms for all. Adding the same read-once switch for edge columns should close this.
   3. IC5's remaining gap is the grouping after the hop, not the hop. At IC5's scale the CSR hop is as fast or faster. Joins win IC5 because a hash join's output is already partitioned by forum, so WITH forum, collect(friend) skips a repartition of 800K rows that carry the forum's columns. That's what Late-materialize a grouped node's properties from Lance node tables #99 (fetching a node's properties after grouping by it) targets; I'm trying it next.
4. prrao87

   added a commit that references this issue on Oct 31, 2026

   perf(planner): look up a grouped node's columns after grouping by id (#…

   ...

   2e0e9dc
5. prrao87 commented on Oct 31, 2026

   

   prrao87

   on Oct 31, 2026

   MemberAuthor

   More actions

   Closing: IC5 is back to 0.4.0's time in memory mode.

   IC5, mean ms (GDC SF1, CONTRIBUTING harness, Apple M5)	Memory	Lazy
   0.4.0 (this issue's table)	79.8	104.0
   main before #100	97.1	116.4
   after #100	89.1	about 110
   after #101 (main at 2e0e9dc)	78.9	about 110

   What fixed it:

   • perf(csr): filter in-memory relationship columns per chunk #100: in-memory relationship filters run per chunk, in parallel, instead of building one shared mask under a lock.
   • perf(planner): look up a grouped node's columns after grouping by id #101: grouping by a node with an in-memory node index (WITH forum, collect(friend)) groups by its id and looks its properties up afterwards, once per node, instead of carrying forum.title through about 800K rows. It also took IC3 down 52% and IC7 11% in memory mode.

   Lazy mode, which this issue didn't cover, is about 6% behind 0.4.0, inside run-to-run noise. The routing comparison above showed named hops are worth keeping on the CSR. Two follow-ups stay open: #99 for doing #101's lookup on Lance node tables, and #102 for large named hops that return relationship columns from Lance (an experiment parked there, not affecting IC5).
6. prrao87

   closed this as completedon Oct 31, 2026