You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Late-materialize a grouped node's properties from Lance node tables #99
Grouping by a node (WITH forum, collect(friend) AS friends) after a hop that fans out carries every column of the node through the grouping, once per input row rather than once per node. In IC5, about 800K hasMember rows carry the forum's title into 77K groups.
#101 fixed this for in-memory node tables with late materialization. When the label has a node index (unique integer ids), the query groups by the node's id alone, and a NodeLookup operator adds the other columns after the aggregate, once per group. In memory mode that made IC3 52% faster, IC5 7% faster and the complex suite's geomean 0.930.
Lance node tables (lazy mode) still group by every column. By the time rows reach the grouping, an earlier step has already read the node's properties from the dataset, so grouping by them reads nothing more. Turning the lookup on as it stands reads them from disk a second time, and IC1 got 11% slower in lazy mode when #101 tried it.
Idea
Late materialization for Lance node tables. When the node is grouped by, the step before the grouping reads only the id, and the other columns are read from Lance after the aggregate, once per group. For a few groups that's take_rows through the node index. When the groups cover a large share of the table, read the columns once in a sequential scan instead, the way the CSR's target lookup does past 1/8 of the table (#94).
The gain depends on how far the hop fans out compared with how many nodes are left after grouping. A query that groups nearly one row per node gains nothing and pays for a second pass, so planning needs a way to tell the two apart, or a check that shows the cost stays small.
Acceptance
Lazy-mode IC5 and IC1 plans and latency, before and after, with results unchanged.
Both suites in both modes against main, run with the CONTRIBUTING.md harness and reporting every query. Memory mode should be unchanged.
changed the title [-]Fetch a node's properties after grouping by it, not before[/-][+]Late-materialize a grouped node's properties from Lance node tables[/+]on Nov 3, 2026
Activity
perf(planner): look up a grouped node's columns after grouping by id (#…
[-]Fetch a node's properties after grouping by it, not before[/-][+]Late-materialize a grouped node's properties from Lance node tables[/+]