Skip to content

feat: support TiCI vector query path#10754

Open
kolafish wants to merge 4 commits intopingcap:feature/ftsfrom
kolafish:codex/tici-vector-query-pr7
Open

feat: support TiCI vector query path#10754
kolafish wants to merge 4 commits intopingcap:feature/ftsfrom
kolafish:codex/tici-vector-query-pr7

Conversation

@kolafish
Copy link
Contributor

Summary

  • add TiCI vector query mode alongside the existing FTS TiCI path
  • wire tipb.IndexScan.tici_vector_query_info through TiCIScan, StorageTantivy, TiCIReadTaskPool, and TantivyInputStream
  • call tici-search-lib::search_vector(...) for vector scans and keep search(...) for expression/fulltext scans
  • harden vector-only rollout with query payload validation and float/vector result decoding
  • update contrib/tici and contrib/tipb to vector-capable upstream commits
  • add short design notes under docs/

Scope

This PR intentionally targets the current vector-only rollout.
Pushed-down vector + filter remains part of the long-term payload shape, but producer-side population is still deferred.

Details

  • accept either fts_query_info or tici_vector_query_info in TiFlash TiCI scan planning
  • add a dedicated vector execution path in the TiCI read task pool and input stream
  • validate top_k, column_id, dimension, metric, query_vector byte length, and finite float values before crossing the FFI boundary
  • decode FLOAT/DOUBLE result fields and Array(Float32) vector payloads from TiCI rows
  • fail fast when a vector query requests a vector column that TiCI does not materialize
  • fix Apple shared-library linking for tici_search_lib

Upstream dependencies

  • tipb: TiCIVectorQueryInfo merged upstream
  • tici: vector query support merged upstream
  • tidb planner/DDL side: PR #67102 and #67103

Validation

  • built dbms/src/Server/tiflash successfully on macOS in cmake-build-codex-release

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 19, 2026

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 19, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jayson-huang for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Mar 19, 2026
@coderabbitai
Copy link

coderabbitai bot commented Mar 19, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)
  • release-8.5
  • release-7.5
  • release-8.1

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: f8b0915e-506a-4a38-bb52-499bdcfde128

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ti-chi-bot ti-chi-bot bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Mar 19, 2026
usearch 2.18.0 (pulled in by tici submodule) compiles simsimd/c/lib.c
which triggers a segfault in clang 17.0.6 on AVX-512 ICE/SAPPHIRE/GENOA
intrinsics. Setting CC=gcc for the cargo build only affects this single
C file while keeping all SIMD targets enabled.
- Add RUNTIME_CHECK guards to getMatchExpr/getVectorQueryInfo for mode safety
- Remove overly broad null-string check that would crash on any null
  string column in vector mode, not just vector columns
- Make missing vector column check unconditional and use proper
  RUNTIME_CHECK_MSG pattern instead of RUNTIME_CHECK_MSG(false, ...)
- Add reserve() for query_vec copy in readVector()
- Remove redundant tici-vector-plan.md (tici-hybrid-vector-plan.md is
  the canonical doc)
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 20, 2026

@kolafish: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-integration-test e9ece74 link true /test pull-integration-test
pull-unit-test e9ece74 link true /test pull-unit-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant