feat(reindex): support tag updates - #3964
Open
zhoujh01 wants to merge 3 commits into
Open
Conversation
Add replace and append tag modes to reindex vector rebuilds, preserve omission-aware behavior, and propagate options through background tasks and namespace rebuilds. Align Python, TypeScript, Go, and CLI interfaces with tests and documentation. Co-authored-by: TRAE CLI <noreply@bytedance.com>
Use an exact path lock for existing file targets while retaining tree locks for directories and prune-orphans scopes. Add a regression test for single-file reindex. Co-authored-by: TRAE CLI <noreply@bytedance.com>
Use exact locks for existing file targets in prune-orphans mode while retaining tree scope for missing targets. Document the existing Go ReindexOptions wait semantics and add lock regression coverage. Co-authored-by: TRAE CLI <noreply@bytedance.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更概述
为 reindex 增加显式设置 tags 的能力,并让 tags 随每条重建向量记录的首次 upsert 一起写入。
tags和tag_mode。replace和append两种写入模式。tags时保留已有 tags。tags: []和tag_mode: "replace"清空 tags。wait=true和wait=false后台任务。行为说明
覆盖 tags
{ "uri": "viking://resources/docs", "mode": "vectors_only", "tags": ["team=search", "env=prod"], "tag_mode": "replace" }本次 reindex 成功重建的所有向量记录都会使用传入的 tags。
对于目录或 namespace reindex,tags 会作用于成功重建的目录 L0/L1 和叶子 L2 记录。
追加 tags
{
"uri": "viking://resources/docs",
"mode": "vectors_only",
"tags": ["region=cn"],
"tag_mode": "append"
}
已有的其他 tag key 会被保留;如果传入 tags 中包含相同 key,则使用新的值覆盖旧值。
省略与空数组
tags 继续复用已有的严格 k=v 校验和标准化逻辑,包括转小写和重复 key 处理。
tags 会在 EmbeddingMsg 入队前写入,不会在 reindex 完成后额外调用一次 set_tags。
接口示例
Python SDK
client.reindex(
"viking://resources/docs",
tags=["team=search"],
tag_mode="append",
)
TypeScript SDK
await client.reindex("viking://resources/docs", {
tags: ["team=search"],
tagMode: "append",
});
Go SDK
client.Reindex(ctx, "viking://resources/docs", &openviking.ReindexOptions{
Wait: true,
Tags: []string{"team=search"},
TagMode: "append",
})
Go 调用方传入非 nil 的 ReindexOptions 时,需要显式设置 Wait。Go 的布尔零值为 false,只有 opts=nil 时才会使用 SDK 默认的 wait=true。
CLI
ov reindex viking://resources/docs
--tag team=search
--tag env=prod
--tag-mode append
--tag 可以重复传入。
CLI 只有在至少传入一个 --tag 时才会发送 tags 字段。通过空数组清空 tags 的能力目前只在 HTTP API 和 SDK 中提供。
文件锁修复
此前 reindex 会对单文件目标使用 tree lock,导致 AGFS 把文件当作目录扫描,并返回 not a directory。
本次调整后:
兼容性
测试