Chroma support regressed, possibly due to dependency incompatibility

[srinskit]

#201 added support for Chroma, but the integration seems to have regressed over time. At first glance it appears to be a package version incompatibility. Chroma client appears to have migrated to pydantic 2.x while VDBBench installs pydantic 1.x during pip install. Some form of Python package management needs to introduced / enforced. I experienced this error on Python 3.11 across Mac OS 15.6.1 and Ubuntu 24.04.

Repro

1. Run Chroma database server: docker run --rm -p 8000:8000 chromadb/chroma
2. Install VDBBench pip install vectordb-bench[chromadb]
3. Start benchmark server: init_bench
4. In the portal http://localhost:8501/, navigate to the sidebar and then "run test", and configure test as below:
5. Step 1
   1. Select the "Chroma" checkbox
   2. Set host as localhost
   3. Set port as 8000
   4. Set password to any string (this is another issue: the password should be optional)
6. Step 2
   1. Select "Search Performance Test"
   2. Select the "Search Performance Test (50K dataset, 1536 Dim)"
7. Step 3
   1. Click "Run Your Test"

I see the following error:

Terminal

ImportError: cannot import name 'field_validator' from 'pydantic' (/opt/homebrew/lib/python3.11/site-packages/pydantic/__init__.cpython-311-darwin.so)
Traceback:

File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/frontend/components/run_test/submitTask.py", line 84, in runHandler
    benchmark_runner.run(tasks, taskLabel)
File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/interface.py", line 83, in run
    self.running_task = Assembler.assemble_all(
                        ^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/backend/assembler.py", line 62, in assemble_all
    db_instance = db.init_cls
                  ^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/backend/clients/__init__.py", line 129, in init_cls
    from .chroma.chroma import ChromaClient
File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/backend/clients/chroma/chroma.py", line 5, in <module>
    import chromadb
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/__init__.py", line 3, in <module>
    from chromadb.api.client import Client as ClientCreator
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/api/__init__.py", line 1, in <module>
    from chromadb.api.types import *  # noqa: F401, F403
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/api/types.py", line 24, in <module>
    from pydantic import BaseModel, field_validator, model_validator

Web Portal

2025-12-21 19:56:20.413 Uncaught app execution
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 121, in exec_func_with_error_handling
    result = func()
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 546, in code_to_exec
    self._session_state.on_script_will_rerun(
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/state/safe_session_state.py", line 68, in on_script_will_rerun
    self._state.on_script_will_rerun(latest_widget_states)
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/state/session_state.py", line 558, in on_script_will_rerun
    self._call_callbacks()
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/state/session_state.py", line 571, in _call_callbacks
    self._new_widget_state.call_callback(wid)
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/state/session_state.py", line 272, in call_callback
    callback(*args, **kwargs)
  File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/frontend/components/run_test/submitTask.py", line 84, in runHandler
    benchmark_runner.run(tasks, taskLabel)
  File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/interface.py", line 83, in run
    self.running_task = Assembler.assemble_all(
                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/backend/assembler.py", line 62, in assemble_all
    db_instance = db.init_cls
                  ^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/backend/clients/__init__.py", line 129, in init_cls
    from .chroma.chroma import ChromaClient
  File "/opt/homebrew/lib/python3.11/site-packages/vectordb_bench/backend/clients/chroma/chroma.py", line 5, in <module>
    import chromadb
  File "/opt/homebrew/lib/python3.11/site-packages/chromadb/__init__.py", line 3, in <module>
    from chromadb.api.client import Client as ClientCreator
  File "/opt/homebrew/lib/python3.11/site-packages/chromadb/api/__init__.py", line 1, in <module>
    from chromadb.api.types import *  # noqa: F401, F403
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/chromadb/api/types.py", line 24, in <module>
    from pydantic import BaseModel, field_validator, model_validator
ImportError: cannot import name 'field_validator' from 'pydantic' (/opt/homebrew/lib/python3.11/site-packages/pydantic/__init__.cpython-311-darwin.so)

[xiaofan-luan]

anyone who is using chroma volunteered to a fix?