feat(stores): add Valkey Search vector store backend

[daric93]

Description

Closes: #10708

Adds a new built-in Go gRPC store backend valkey-store that implements the four Stores* RPCs (Set / Get / Delete / Find) against the Valkey Search module (FT.* vector similarity), selectable via the existing per-request backend field on the /stores endpoints. It mirrors the in-memory local-store and adds persistence across restarts plus opt-in HNSW.

• No proto change, no HTTP/API change — selected with "backend": "valkey-store" (alias "valkey").
• Client: github.com/valkey-io/valkey-go v1.0.76 (official, pure Go, no CGO).
• Each vector is a Valkey HASH keyed by hex(little-endian float32); the FT index is created lazily on first Set (FLAT + COSINE by default), and cosine similarity is derived as 1 - distance to match local-store semantics exactly.
• Per-namespace, collision-resistant key prefix / index name (sanitize(name) + short sha256).
• Config via VALKEY_* env vars (addr, auth, TLS, index algo + HNSW knobs, distance metric, per-command request timeout, mandatory ClientName).

Why

The stores subsystem is designed to be pluggable, but the only shipped backend (local-store) is in-memory and loses all data on restart, and its Find is an O(N) scan. Face/voice biometric registries and the router embedding cache all consume this seam, so a durable, scalable Valkey-backed backend benefits them with zero caller changes.

Why valkey-go and not valkey-glide? The official valkey-glide Go client is built on a Rust core via FFI, which forces CGO and a per-arch native library — that breaks local-store's CGO_ENABLED=0 static build and the Linux/Darwin backend matrix. valkey-go is pure Go with a typed FT.SEARCH builder, a VectorString32 encoder, and a gomock mock package for unit tests.

Testing

• Unit tests (valkey-go mock, no container): 35 specs — RPC command-shape/wire contract, empty/len/dim rejects, omit-missing Get, tolerate-missing Delete, topK<1 reject, distance→similarity conversion (0→1, 1→0, 2→−1), lazy FT.CREATE, HNSW arg-shape, key-encoding round-trip (incl. -0.0/NaN), namespace-token collision resistance, config parsing. make test / go run ...ginkgo -r backend/go/valkey-store.
• Integration tests (Ginkgo, Label("valkey"), env-gated on VALKEY_ADDR, against valkey/valkey-bundle:9.1.0 on :6379): 14 specs mirroring the local-store suite one-for-one (set/get/delete/find, exact cosine for orthogonal/opposite unit + non-unit vectors, triangle inequality incl. random 768-d) plus a persistence/restart test and a CLIENT LIST client-name assertion. Index back-fill is absorbed with a bounded find poll (no time.Sleep). Skipped automatically when VALKEY_ADDR is unset, so unit CI needs no container.
• golangci-lint clean on the changed packages. All Valkey logic lives in backend/go/valkey-store/ (outside COVERAGE_COVERPKG), so the coverage ratchet is unaffected.

Run integration locally:

podman run -d --name valkey-store-it -p 6379:6379 valkey/valkey-bundle:9.1.0
make backends/valkey-store
VALKEY_ADDR=localhost:6379 make test-valkey-store

Scope / follow-ups

Purely additive and opt-in per request. Deferred to follow-ups: hybrid / metadata-filter search (needs new API surface beyond the four RPCs), Valkey Cluster mode, and single-instance multi-namespace consolidation.

Notes for Reviewers

cc @mudler

Signed commits

• Yes, I signed my commits.

[richiejp]

I think at least some of these should go in a model config. Then you can have multiple stores configs.

[daric93]

Done — the backend now reads its config from the store's model config instead of VALKEY_* env vars. A store is configured with a model YAML (name: = the store, backend: valkey-store, and an options: list like addr:host:6379 / index_algo:HNSW), so different stores can each point at their own Valkey server/index within one process. Dropped the env var block from .env and documented the options in stores.md.

[richiejp]

Please avoid putting env accesses throughout the code. Model config is the natural place instead of env vars.

[daric93]

Done — removed all env access from the backend. loadConfig now parses the model-config options: list threaded through LoadModel (ModelOptions.Options), and core/backend/stores.go resolves each store's ModelConfig and passes its options down. The integration test configures the store through those same options; VALKEY_ADDR remains only as the harness gate that locates the test server, not as backend config.

[daric93]

@mudler This PR is ready for a second set of eyes and merge whenever you get a chance 🙏

[richiejp]

We don't need this

[daric93]

Done — removed the comment block. The configuration is already documented in docs/content/features/stores.md.

[richiejp]

Thanks!

[localai-bot]

Heads up @daric93: this branch now conflicts with master in three files, all of the append-both-sides kind:

• .github/backend-matrix.yml
• Makefile
• backend/index.yaml

New backends landed on master since this was opened, so the resolution is a union: keep master's new entries AND the valkey-store ones (in backend/index.yaml keep both blocks; same for the matrix entry and the Makefile target). A rebase or a merge of master should take a couple of minutes.

Once that's done this is unblocked: it already has an approval from richiejp and CI was green. @mudler good to merge after the rebase.

[localai-bot]

@mudler recommendation: merge this one. Between the two competing implementations of #10708 (this and #10801), this PR is the more complete: it has richiejp's approval after three addressed review rounds, a full green CI run (including both cpu-valkey-store image builds and the darwin metal build), mock-based unit tests that don't need a container in CI, an env-gated integration suite mirroring local-store's, and it is the only one where per-store model config actually works end to end (the core/backend/stores.go threading).

Three things before merge:

1. DCO is still red: one commit (07d05fa, the .env comment removal) is missing Signed-off-by. A signoff rebase of that commit or a DCO remediation commit fixes it.
2. Merge conflicts with master in .github/backend-matrix.yml, Makefile, and backend/index.yaml: all append-both-sides, so the resolution is a simple union.
3. One checklist item from .agents/adding-backends.md worth adding (pre-merge or as an immediate follow-up): Load doesn't gate on the store.NamespacePrefix marker, so if a Valkey server happens to be reachable at the default localhost:6379, the model loader's greedy autoload probe could bind an arbitrary model name to valkey-store (the If you add opus as a backend via api, then add a model via the api that should use llama.cpp or vllm, it then tries to use opus #9287 failure mode). local-store's Load shows the pattern, and feat(stores): add valkey-store vector store backend #10801 implements exactly this gate with a test, so it's a ~5-line addition.

As a nice follow-up (no blocker), #10801's username_env/password_env credential indirection would avoid putting the Valkey password in the model YAML. Great work @daric93, especially the FT.INFO dimension recovery after restart and the documented pipelined-write deadlock finding.

[daric93]

All three items addressed:

1. DCO — rebased with --signoff on all 6 commits. The missing signoff on 07d05fa is now fixed (4db5b1da).
2. Merge conflicts — rebased onto current master; the append-both-sides conflicts in backend-matrix.yml, Makefile, and backend/index.yaml are resolved (union of both entries).
3. NamespacePrefix gate — added in b7d7287e. Load now refuses model names without store.NamespacePrefix (same pattern as local-store), with 5 unit tests covering prefixed namespaces, the prefix alone, unprefixed model names, empty model, and nil opts.

CI should be green again — all valkey-store unit tests pass locally (54/54).

[daric93]

Also added the username_env/password_env credential indirection from #10801's pattern in aadaccb8 — model configs can now reference env vars holding the credentials instead of putting secrets directly in the YAML:

options:
  - username_env:MY_VALKEY_USER
  - password_env:MY_VALKEY_PASSWORD

Direct username/password options still work and take precedence (backward compatible). Docs updated in stores.md.

[localai-bot]

Re-reviewed the current head after the rebase and the follow-up commits. The three earlier blockers are confirmed addressed: DCO is green (all commits signed off), the store.NamespacePrefix gate on Load is in with tests, and the username_env/password_env credential indirection is wired and documented. The end-to-end config threading through core/backend/stores.go (per-store model config resolving backend: + options:) is the piece that makes model-config settings actually reach the backend, and it is unique to this PR.

However, the conflict resolution in the rebase went wrong in two files, and since CI has not run on this head yet (fork awaiting approval, only the DCO check executed), nothing has caught it:

1. .github/backend-matrix.yml breaks the cloud-proxy backend. In include:, the two valkey-store entries were merged INTO the existing cloud-proxy entries instead of being added as new list items. Both cloud-proxy Linux entries now carry duplicated tag-suffix / runs-on / base-image / skip-drivers / backend keys with the valkey-store values second (see around line 5019: tag-suffix: '-cpu-cloud-proxy' followed a few lines later by tag-suffix: '-cpu-valkey-store' inside the same item). Last-key-wins turns both cloud-proxy Linux builds into valkey-store builds, so cloud-proxy Linux images silently stop being built, and yamllint's duplicate-key check should fail the Yamllint job anyway. In includeDarwin: the cloud-proxy entry also lost its build-type: "metal" and lang: "go" lines to the new valkey-store entry. Fix: make the valkey-store entries standalone list items and restore the cloud-proxy entries exactly as on master.
2. Makefile has duplicated aggregate lines. There are now two .NOTPARALLEL: lines (2 and 3) and two docker-build-backends: target lines (1411 and 1412); the PR's copies are the pre-rebase versions (missing master's bonsai, cloud-proxy, longcat-video entries) with valkey-store appended. Make accumulates prerequisites so this happens to union out, but please collapse each back to a single line that is master's current line plus the valkey-store additions.

Neither issue touches the backend code itself, which I have no further findings on: the mock-based unit suite (Ginkgo, no container in CI), the env-gated integration suite mirroring local-store plus the persistence and CLIENT LIST specs, the FT.INFO dimension recovery, and the documented pipelined-write deadlock workaround are all solid.

Three small optional pickups that #10801 has and this PR does not: a valkey-store row in docs/content/reference/compatibility-table.md, a line in backend/README.md, and a /valkey-store entry in .gitignore (the binary is built at the repo-relative backend dir, so this one mostly matters for local builds).

@mudler recommendation: this PR and #10801 are competing implementations of #10708. I recommend merging this one once the two rebase artifacts above are fixed and CI is approved to run on the head. It was opened first by the issue author, has been through three review rounds with @richiejp (his changes-requested was dismissed with thanks after the fixes), and is the only one of the two where per-store model config works end to end. #10801 should then be closed with credit: its namespace gate and env-indirect credentials arrived there first and have already been adopted here.

[daric93]

Fixed both rebase artifacts in 58d1709a:

1. backend-matrix.yml — cloud-proxy and valkey-store are now separate list items with no duplicate keys. Cloud-proxy's darwin entry has its build-type: "metal" and lang: "go" restored.
2. Makefile — collapsed to single .NOTPARALLEL: and docker-build-backends: lines that are master's current content + valkey-store.

Also picked up the three optional items: /valkey-store in .gitignore, the compatibility table row, and the backend/README.md line.

CI should be clean now.