Skip to content

[python] Add search result scores and explicit relevance ordering - #10005

Open
TheR1sing3un wants to merge 1 commit into
apache:masterfrom
TheR1sing3un:codex/search-result-metadata
Open

TheR1sing3un wants to merge 1 commit into
apache:masterfrom
TheR1sing3un:codex/search-result-metadata

Conversation

@TheR1sing3un

Copy link
Copy Markdown
Member

Purpose

Multimodal search currently returns projected rows without exposing their relevance scores, and final lookup determines the output order. Add opt-in with_score(column_name="_score") and order_by_score() for data-evolution tables, including full-text/hybrid queries and local/Ray single and batch vector queries. Scores retain the existing higher-is-better convention; ordering uses descending score and ascending row ID for ties.

Queries explicitly selecting only row IDs or scores can skip final lookup when neither a post-filter nor query authorization requires it. Batch queries retain their shared lookup and attach scores separately for each query. Existing queries keep their default behavior. Document score meanings and the scope of the lookup optimization.

Related to #8484.

Tests

  • Real IVF and raw searches: scores, ordering, metadata-only lookup avoidance, shared batch lookup, duplicate projections, empty results, post-filters, authorization, deletions and pinned snapshots.
  • Native full-text and hybrid score output; local/Ray parity for single and batch vectors, including refinement.
  • 167 targeted metadata, batch lookup and multimodal tests passed on Python 3.11 / Arrow 19. Score-output tests also passed with Ray 2.44 / Arrow 18 (43 tests).
  • Repository-configured Flake8, Apache license checks and Python 3.6 grammar checks passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant