Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,117 @@ jobs:
if: always()
run: docker compose -f /tmp/milvus-compose.yml down -v

integration-seekdb:
name: integration tests (SeekDB embedded 1.4)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6

- name: Install uv
uses: astral-sh/setup-uv@v8.2.0
with:
enable-cache: true
cache-dependency-glob: uv.lock

- name: Set up Python
run: uv python install 3.12

# pyseekdb pulls the matching pylibseekdb wheel on Linux. The version is
# locked with the rest of the development environment in uv.lock.
- name: Install dependencies (frozen)
run: uv sync --frozen --extra seekdb-embedded

- name: SeekDB repository and derived-index contracts
env:
EVEROS_TEST_SEEKDB_PATH: /tmp/everos-seekdb
run: >-
uv run --frozen pytest
tests/integration/test_seekdb_backend.py
tests/unit/test_infra/test_index_contract.py
-v

- name: SeekDB end-to-end (tiered API suites)
env:
EVEROS_TEST_SEEKDB_PATH: /tmp/everos-seekdb
run: uv run --frozen pytest tests/integration/test_tiers -v -k seekdb

integration-seekdb-remote:
name: integration tests (SeekDB remote latest)
runs-on: ubuntu-latest
services:
seekdb:
image: oceanbase/seekdb:latest
env:
ROOT_PASSWORD: everos-test
CPU_COUNT: "2"
MEMORY_LIMIT: 2G
LOG_DISK_SIZE: 2G
DATAFILE_SIZE: 2G
ports:
- 2881:2881
steps:
- uses: actions/checkout@v6

- name: Install uv
uses: astral-sh/setup-uv@v8.2.0
with:
enable-cache: true
cache-dependency-glob: uv.lock

- name: Set up Python
run: uv python install 3.12

- name: Install dependencies (frozen)
run: uv sync --frozen --extra seekdb

- name: Wait for the MySQL endpoint
run: |
uv run python - <<'PY'
import time
import pymysql

deadline = time.monotonic() + 180
while True:
try:
connection = pymysql.connect(
host="127.0.0.1",
port=2881,
user="root",
password="everos-test",
connect_timeout=3,
)
connection.close()
break
except pymysql.MySQLError:
if time.monotonic() >= deadline:
raise
time.sleep(3)
PY

- name: SeekDB remote repository and derived-index contracts
env:
EVEROS_TEST_SEEKDB_HOST: 127.0.0.1
EVEROS_TEST_SEEKDB_PORT: "2881"
EVEROS_TEST_SEEKDB_TENANT: ""
EVEROS_TEST_SEEKDB_USER: root
EVEROS_TEST_SEEKDB_PASSWORD: everos-test
EVEROS_TEST_SEEKDB_DATABASE: everos_test
run: >-
uv run --frozen pytest
tests/integration/test_seekdb_backend.py
tests/unit/test_infra/test_index_contract.py
-v

- name: SeekDB remote end-to-end (tiered API suites)
env:
EVEROS_TEST_SEEKDB_HOST: 127.0.0.1
EVEROS_TEST_SEEKDB_PORT: "2881"
EVEROS_TEST_SEEKDB_TENANT: ""
EVEROS_TEST_SEEKDB_USER: root
EVEROS_TEST_SEEKDB_PASSWORD: everos-test
EVEROS_TEST_SEEKDB_DATABASE: everos_test
run: uv run --frozen pytest tests/integration/test_tiers -v -k seekdb

package:
name: package build
runs-on: ubuntu-latest
Expand Down
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **Optional SeekDB derived-index backend.** Set `[index] backend = "seekdb"`
to store the seven rebuildable business indexes in embedded SeekDB or a
remote seekdb Server/OceanBase database. The adapter implements the same
repository and lifecycle ports as LanceDB and Milvus, including typed scalar
filters, JSON array membership, native pagination, BM25 token columns, and
cosine search across multiple vector columns. Install `everos[seekdb]` for
remote mode or `everos[seekdb-embedded]` on Linux/macOS. Markdown remains the
source of truth and the default LanceDB installation is unchanged.

### Changed

- **Derived-index routing now supports registered backends uniformly.** Stable
repository objects resolve LanceDB, Milvus, or SeekDB at call time without
leaking a concrete adapter into the port definitions.

## [1.3.1] - 2026-09-08

**One reproducible runner for four long-term-memory benchmarks, plus an
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,9 @@
EverOS is a Python library and local-first memory runtime for agents and
makers. It gives one portable memory layer across coding assistants, apps,
devices, and workflows from day one. It stores conversations, files, and agent
trajectories as readable Markdown, then syncs local SQLite and LanceDB indexes
for fast retrieval and self-evolving reuse.
trajectories as readable Markdown, then syncs SQLite and a rebuildable derived
index (LanceDB by default, with optional Milvus or SeekDB) for fast retrieval
and self-evolving reuse.

<table>
<tr>
Expand All @@ -60,7 +61,7 @@ for fast retrieval and self-evolving reuse.
</tr>
<tr>
<td><strong>Local three-part stack</strong></td>
<td>✅ Markdown + SQLite + LanceDB; no MongoDB, Elasticsearch, or Redis required</td>
<td>✅ Markdown + SQLite + LanceDB by default; optional Milvus or SeekDB backends</td>
<td>❌ Often depends on managed services, vector DBs, graph DBs, or server stacks</td>
</tr>
<tr>
Expand Down Expand Up @@ -664,7 +665,7 @@ Explore stored entities and relationships in a graph interface. Frontend demo; b
## Documentation

- [docs/everos-demo.md](docs/everos-demo.md) — Demo scope and TUI source layout
- [docs/how-memory-works.md](docs/how-memory-works.md) — Markdown, SQLite, LanceDB, and recall flow
- [docs/how-memory-works.md](docs/how-memory-works.md) — Markdown, SQLite, derived indexes, and recall flow
- [docs/use-cases.md](docs/use-cases.md) — Full use-case gallery and integration examples
- [docs/engineering.md](docs/engineering.md) — Contributor engineering reference: build, test, CI, conventions
- [docs/migration-to-1.0.0.md](docs/migration-to-1.0.0.md) — Legacy API migration notes
Expand Down
8 changes: 4 additions & 4 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@
EverOS 是面向 agents 和 makers 的 Python library 与 local-first memory
runtime。它从 day one 开始就提供一层可携带的记忆层,让记忆穿过 coding
assistants、apps、devices 和 workflows。它会把 conversations、files 和
agent trajectories 保存为可读 Markdown,并同步本地 SQLite 与 LanceDB
索引,用于快速检索和自进化复用。
agent trajectories 保存为可读 Markdown,并同步 SQLite 与可重建派生索引
(默认 LanceDB,也可选 Milvus 或 SeekDB),用于快速检索和自进化复用。

<table>
<tr>
Expand All @@ -60,7 +60,7 @@ agent trajectories 保存为可读 Markdown,并同步本地 SQLite 与 LanceDB
</tr>
<tr>
<td><strong>本地三件套</strong></td>
<td>✅ Markdown + SQLite + LanceDB;不需要 MongoDB、ElasticsearchRedis</td>
<td>✅ 默认 Markdown + SQLite + LanceDB;可选 MilvusSeekDB backend</td>
<td>❌ 常依赖 managed service、vector DB、graph DB 或 server stack</td>
</tr>
<tr>
Expand Down Expand Up @@ -663,7 +663,7 @@ Claude Code 的持久记忆插件。自动保存并回忆过去 coding sessions
## 文档

- [docs/everos-demo.md](docs/everos-demo.md) - Demo 范围与 TUI 源码布局
- [docs/how-memory-works.md](docs/how-memory-works.md) - Markdown、SQLite、LanceDB 与 recall flow
- [docs/how-memory-works.md](docs/how-memory-works.md) - Markdown、SQLite、派生索引与 recall flow
- [docs/use-cases.md](docs/use-cases.md) - 完整使用场景 gallery 和集成示例
- [docs/engineering.md](docs/engineering.md) - 贡献者工程参考:构建、测试、CI 与规范
- [docs/migration-to-1.0.0.md](docs/migration-to-1.0.0.md) - Legacy API 迁移说明
Expand Down
21 changes: 20 additions & 1 deletion config.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ max_concurrent = 5
# read_consistency_seconds = 5.0
#
# [index]
# backend = "lancedb" # or "milvus"
# backend = "lancedb" # or "milvus" / "seekdb"
#
# # Required only when index.backend = "milvus". Install with:
# # pip install "everos[milvus]"
Expand All @@ -68,3 +68,22 @@ max_concurrent = 5
# db_name = ""
# consistency_level = "Session"
# collection_prefix = "everos"
#
# # SeekDB supports an in-process engine on Linux/macOS and a remote seekdb
# # Server or OceanBase endpoint on every platform. Install with:
# # pip install "everos[seekdb]"
# # Add the embedded extra for local in-process storage:
# # pip install "everos[seekdb-embedded]"
# [seekdb]
# mode = "embedded" # "embedded" or "remote"
# path = "" # empty -> <EVEROS_ROOT>/.index/seekdb
# host = "" # required in remote mode
# port = 2881
# tenant = "" # set for OceanBase, e.g. "test"
# user = "root"
# password = "" # or export SEEKDB_PASSWORD
# database = "everos"
# table_prefix = "everos"
# vector_sync_mode = "immediate" # "immediate" or "async"
# connect_timeout_seconds = 10.0
# read_timeout_seconds = 60.0
3 changes: 2 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ layers = [

┌──────────────┐ ┌──────────────┐ ┌─────────────────┐
│ Markdown │ │ SQLite │ │ Derived index │
│ (truth) │ │ (state) │ │ LanceDB/Milvus │
│ (truth) │ │ (state) │ │ Lance/Milvus/ │
│ │ │ │ │ SeekDB │
├──────────────┤ ├──────────────┤ ├─────────────────┤
│ entries + │ │ change queue │ │ vector ANN │
│ frontmatter │ │ + state/LSN │ │ BM25 (Tantivy) │
Expand Down
37 changes: 36 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ truth regardless of backend.

| Field | Type | Default | Description |
|---|---|---|---|
| `backend` | string | `"lancedb"` | Index implementation: `lancedb` or `milvus`. |
| `backend` | string | `"lancedb"` | Index implementation: `lancedb`, `milvus`, or `seekdb`. |

### `[milvus]`

Expand All @@ -122,6 +122,41 @@ Zilliz Cloud endpoint; a Milvus Lite filesystem path is rejected.
| `consistency_level` | string | `"Session"` | Milvus consistency level used by collections. |
| `collection_prefix` | string | `"everos"` | Prefix for the seven derived-index collections. |

### `[seekdb]`

Used only when `index.backend = "seekdb"`. Install `everos[seekdb]` for a
remote seekdb Server or OceanBase endpoint. Install `everos[seekdb-embedded]`
for in-process storage on Linux or macOS; pylibseekdb does not currently ship a
Windows wheel. Embedded mode owns its database directory exclusively, so do
not run a server and a cascade CLI process against the same path concurrently.

| Field | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `"embedded"` | `embedded` for an in-process directory, or `remote` for a server endpoint. |
| `path` | string | `""` | Embedded directory; empty resolves to `<EVEROS_ROOT>/.index/seekdb`. |
| `host` | string | `""` | Required hostname in remote mode. |
| `port` | int | `2881` | MySQL-compatible server port. |
| `tenant` | string | `""` | Leave empty for seekdb Server; set the OceanBase tenant name (for example, `test`) when connecting to OceanBase. |
| `user` | string | `"root"` | User name without the tenant suffix. |
| `password` | string | `""` | Password; an empty value falls back to `SEEKDB_PASSWORD`. |
| `database` | string | `"everos"` | Database created on first use when permitted; SQL identifier, at most 64 characters. |
| `table_prefix` | string | `"everos"` | Prefix for the seven derived-index tables; SQL identifier, at most 48 characters. |
| `vector_sync_mode` | string | `"immediate"` | Vector-index synchronization: `immediate` for write-then-search consistency, or `async` for higher ingest throughput and eventual index visibility. |
| `connect_timeout_seconds` | float | `10.0` | Remote connection timeout. |
| `read_timeout_seconds` | float | `60.0` | Remote read and write timeout. |

SeekDB tables use `utf8mb4_bin`, typed scalar/JSON columns,
whitespace-tokenized full-text indexes, and cosine HNSW indexes. The default
`immediate` vector synchronization makes a successful upsert visible to the
following search, at a significant write-throughput cost. Choose `async` when
bulk-ingest throughput matters more than immediate visibility, or when a
compatible OceanBase deployment does not support seekdb's `immediate` option;
new vectors may then be absent from search results until background index
synchronization catches up.

Markdown remains the source of truth; changing the backend requires
`everos cascade rebuild`, not a data migration.

### `[llm]`

| Field | Type | Default | Required | Description |
Expand Down
Loading