Conversation
JingsongLi
left a comment
There was a problem hiding this comment.
Requirement fit: SUPPORTED (triage: GO)
Implementation: FINDINGS
[P2] Preserve the cheap file-index rejection before loading the deletion vector
DataEvolutionSplitRead#createReader now calls readDeletionVector at lines 253-254 before skipByFileIndex. DeletionVector.Factory#create opens and reads the DV sidecar, so every merged row-id group with a DV now pays that storage read even when the ordinary file index would have rejected the group without opening any data/DV file. This regresses the most selective scans: a table with many merged groups can add one remote sidecar read per group that used to be skipped cheaply.
Please keep a first file-index-only rejection before loading the DV (return immediately if it is already SKIP), and only load/intersect the DV when the raw index still has candidates. The new deleted-candidate behavior and its test can remain as the second-stage evaluation.
Purpose
Merged data evolution groups could use file indexes to skip an entire
group, but index evaluation did not account for deletion vectors. When
all indexed matches had been deleted, the union reader still opened
every file in the group even though no live row could match.
indexes.
file index evaluation.
rows before deciding whether to skip the whole group.
group-level pruning.
Tests