feat(table): add primary-key hybrid (vector + full-text) search

[JunRuiLee]

Purpose

Part of #567 (primary-key full-text + hybrid search on the Rust read side), the fourth and final read-side slice after #563 (reader foundation), #594 (data layer), and #599 (PK full-text read path).

This adds hybrid search on primary-key tables: fuse a primary-key vector route and a primary-key full-text route in physical-position space, mirroring Java PrimaryKeySearchRanker + HybridSearchBuilderImpl. Both routes search their own PK index, produce physical (data file, row position) hits, and are fused by a configurable ranker; the winning rows are materialized best-score-first with a __paimon_search_score column. FAST mode only on the full-text route (mirrors Java).

Brief change log

• PrimaryKeySearchPosition (table/pk_search_position.rs): a shared physical hit (partition, bucket, data file, row position) whose identity (Eq/Hash) excludes score. Built from a vector candidate via distance_to_score or from a full-text candidate with its raw score; validates a non-negative row position and a finite score. Mirrors Java PrimaryKeySearchPosition.
• PrimaryKeySearchRanker (table/pk_search_ranker.rs): weighted RRF (RRF_K = 60), weighted score (per-route normalization), and weighted MRR fusion, plus a score-descending top_k with a deterministic physical-position tie-break and duplicate-position handling — a faithful port of Java PrimaryKeySearchRanker including its Float.compare/signed-zero semantics.
• Per-route candidate producers exposed with their source splits and the real pinned snapshot id (vector search_pk_route, full-text search_route), so the hybrid builder can fuse routes over one consistent snapshot.
• Hybrid execute_read (primary-key path) (table/hybrid_search_builder.rs): require every route to be primary-key (else fail loud); pin one snapshot across routes (scan.snapshot-id + an equality backstop); enforce FAST mode on the full-text route; feed per-route limits into a builder-limit fusion; validate cross-route physical-source consistency; carry the fused raw scores into PkVectorIndexedSplits, materialize, reorder to best-score order, and strip the internal _PKEY_VECTOR_POSITION column. PK execute/execute_scored fail loud (physical positions, not global row ids); the data-evolution row-id hybrid path is unchanged.

Archive paths resolve under {table}/index/ (Java default index-file-in-data-file-dir=false), matching the PK vector and PK full-text reads; the bucket-directory layout (=true) is deferred, as it is for PK vector.

Tests

• PrimaryKeySearchRanker: weighted RRF / score / MRR fusion and top_k ordering + tie-break, ported from the Java ranker's fixtures; duplicate physical-position and non-finite-score handling.
• PrimaryKeySearchPosition: identity excludes score; from-vector vs from-full-text score handling; negative-position / non-finite-score rejection.
• Hybrid execute_read: end-to-end fusion over a self-contained PK table with a real ANN segment and a full-text archive over the same compacted data file; one-snapshot pinning; FAST-only rejection of FULL/DETAIL on the FT route; all-routes-PK dispatch; PK execute/execute_scored fail loud.
• Vector, full-text, and data-evolution regressions stay green.
• cargo test -p paimon and --features fulltext green; cargo clippy -p paimon --all-targets (+ --features fulltext) -D warnings clean; cargo fmt --all --check clean; cargo build -p paimon-datafusion (Send boundary) passes.

API and Format

No storage-format change and no change to existing public read APIs. HybridSearchBuilder gains the primary-key execute_read path. The archive-reading code is behind the existing fulltext cargo feature; the ranker, search position, and dispatch are plain code.

Documentation

No documentation changes; module- and item-level doc comments describe the ranker, the shared search position, and the hybrid read.

[JingsongLi]

• crates/paimon/src/table/hybrid_search_builder.rs:483 directly projects all user columns without reusing the retained column checks from the vector read.
• _PKEY_VECTOR_POSITION and __paimon_search_score will be appended later (crates/paimon/src/table/pk_vector_position_read.rs:224).
• If the table itself contains a column with the same name, a positional lookup will target the user column (crates/paimon/src/table/vector_search_builder.rs:1925), which may result in sorting errors, deletion of the user column, and exposure of internal position columns; when the score column has the same name, duplicate fields will be generated, and the value read may not be the fused score.
• A standalone vector read explicitly intercepts this pattern in crates/paimon/src/table/vector_search_builder.rs:582; the hybrid implementation should reuse the same checks and add regression tests for the two reserved columns.