feat: Add feature view versioning to Qdrant online store - #6753
Open
arose26 wants to merge 1 commit into
Open
Conversation
Route every Qdrant collection reference through a single helper so that,
when registry.enable_online_feature_view_versioning is set, each feature
view version gets its own collection (driver_stats_v2) instead of all
versions sharing one.
The helper is built on compute_versioned_name rather than
compute_table_id, which the other stores use. Qdrant collections have
always been named by the bare feature view name, with no {project}_
prefix, so adopting compute_table_id would rename every collection in
every existing deployment. With versioning disabled the collection name
is byte-identical to today's.
Covered by write, update, teardown and document-retrieval paths.
online_read is intentionally left out of the versioned-read allowlist:
it has a separate pre-existing defect and cannot serve a versioned read
correctly yet. See the PR description for a runnable reproduction.
Part of feast-dev#2728. Closes feast-dev#6179
Signed-off-by: arose26 <145766958+arose26@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6179. Part of #2728.
What this does
Routes every Qdrant collection reference through one helper, so that with
registry.enable_online_feature_view_versioning: trueeach feature view version getsits own collection (
driver_stats_v2) instead of all versions sharing one. Applied tothe write, create, update, teardown and document-retrieval paths.
One deliberate deviation from the FAISS/Milvus precedent
Those stores use
compute_table_id(project, table, versioning), which yields{project}_{name}[_v{N}]. This PR usescompute_versioned_nameinstead, which yields{name}[_v{N}]with no project prefix.The reason is that Qdrant collections have always been named by the bare
table.name—unlike Milvus, which already had the
{project}_prefix before versioning was added to it,so gaining the flag there was non-breaking. Switching Qdrant to
compute_table_idwouldrename every collection in every existing deployment and orphan their data. With versioning
disabled the collection name here is byte-identical to today's, which
test_unversioned_store_still_round_tripspins.Happy to switch to the project-prefixed form if you'd rather have consistency across
stores and want to handle the migration — just say so.
Why
online_readis not added to the versioned-read allowlistI did not add
QdrantOnlineStoretoOnlineStore._is_versioned_read_supported(), becauseQdrantOnlineStore.online_readcannot currently serve any read, versioned or not. It lookslike it has never been exercised — there is no unit test for it, and it fails on two
independent counts before reaching Qdrant:
EntityKeyProtoobjects intomodels.MatchAny(any=entity_keys), whichpydantic rejects (
Input should be a valid string), andcollection_name=config.online_store.collection_name, but neitherQdrantOnlineStoreConfignorVectorStoreConfigdefinescollection_name, so thatline raises
AttributeError.It also returns one entry per stored point holding a base64
str, where theOnlineStore.online_readcontract (seesqlite.py) is one entry per requested entity key,in order, holding
ValueProtovalues, with(None, None)for a miss.Reproduction, stock
main, no external service — an in-process Qdrant is enough:Fixing that properly means deciding how
entity_keyshould be stored — it is currentlywritten into the payload as raw
bytesfromserialize_entity_key, whichMatchAnycannotfilter on, and which
retrieve_online_documentsthen reads back throughstr(payload.get("entity_key")), producing abytesrepr rather than the value_build_retrieve_online_document_recordexpects. That is a storage-format call I did notwant to make unilaterally inside a versioning PR, so I have kept it out of scope. Happy to
open a separate issue, or to take it in a follow-up once you've said which encoding you want.
So this PR delivers the versioned collection namespace; versioned scalar reads stay
correctly gated behind
VersionedOnlineReadNotSupporteduntilonline_readworks.Tests
New
sdk/python/tests/unit/infra/online_store/test_qdrant_online_store.py, 6 tests, runningagainst an in-process Qdrant (
location=":memory:") — no service required, and skipped viaimportorskipwhenqdrant-clientis absent._v2when ontest_versions_write_to_separate_collections— v1 and v2 collections both created, onepoint each, no cross-contamination
test_teardown_removes_only_the_targeted_versiontest_unversioned_store_still_round_trips— the controlVerified red-before/green-after: with the change reverted, the two versioning behaviour
tests fail while the unversioned control still passes, so they are testing the change rather
than the setup.
Regression check across the 56 unit test files touching online stores: the set of failing
and erroring tests is identical with and without this change (
diffof the sortedFAILED/ERRORlines is empty). Those pre-existing failures are missing optionaldependencies in my environment.
ruff check,ruff formatandmypyare clean on bothfiles.
🤖 Written with Claude Code (Claude Opus 5), reviewed by @arose26.