Skip to content

fix(fts): derive empty segment codec from format - #8687

Merged
Xuanwo merged 1 commit into
mainfrom
yang/fix-empty-fts-segment-codec
Aug 21, 2026
Merged

fix(fts): derive empty segment codec from format#8687
Xuanwo merged 1 commit into
mainfrom
yang/fix-empty-fts-segment-codec

Conversation

@BubbleCal

Copy link
Copy Markdown
Contributor

What is the bug?

A distributed FTS rebuild can legitimately produce an intermediate segment with zero partitions when its assigned fragments contain no indexable tokens. InvertedIndex::posting_tail_codec currently falls back to the default VarintDelta codec for such a segment, even when its declared format is V1 / Fixed32.

Merging that empty V1 segment with a populated V1 segment then fails with:

cannot merge inverted index segments with different posting tail codecs

This is the same-format empty-segment failure observed in ENT-2323. It is narrower than the cross-format behavior proposed in #8681.

What issues or incorrect behavior does the bug cause?

V1 distributed FTS rebuilds fail during merge_existing_index_segments before the replacement index can be committed. Automated maintenance can retry the same deterministic failure.

How does this PR fix the problem?

When an inverted index has no partitions, derive its posting-tail codec from the segment's declared FTS format instead of using the global codec default. Populated segments continue to use their physical partition metadata, and cross-format merge behavior is unchanged.

Tests

Adds a regression test that merges an empty V1 segment with a populated V1 segment in both input orders. It verifies that:

  • the empty segment resolves to Fixed32
  • the merged index remains V1
  • the populated document remains searchable

Local compilation and tests were intentionally skipped; CI is the validation path for this PR.

@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Aug 21, 2026
@BubbleCal
BubbleCal force-pushed the yang/fix-empty-fts-segment-codec branch from 7a39d82 to 78d8e11 Compare August 21, 2026 07:13

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The empty-segment codec now follows the segment’s declared FTS format, which fixes V1 consolidation without weakening physical-codec or cross-format validation. The regression covers both segment orders and verifies the merged V1 index remains searchable.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 21, 2026
@Xuanwo
Xuanwo merged commit 59ea16a into main Aug 21, 2026
39 of 40 checks passed
@Xuanwo
Xuanwo deleted the yang/fix-empty-fts-segment-codec branch August 21, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants