Add parallel multi-tenant operations for faster bulk processing#154
Open
Add parallel multi-tenant operations for faster bulk processing#154
Conversation
Multi-tenant operations were processed sequentially, causing major bottlenecks with collections with hundreds or thousands of tenants. Changes: - tenant_manager: replace per-tenant remove/update loops with single batch API calls (1 call instead of N) - data_manager: use ThreadPoolExecutor to process multiple tenants concurrently for create/update/delete data operations - data_manager: when parallel_workers > 1, concurrent_requests per tenant is scaled down to keep total connections bounded - add --parallel_workers CLI option to create/update/delete data commands (default: MAX_WORKERS = min(32, cpu_count + 4)) - update defaults.py with parallel_workers field for data operations Test changes: - Updated test_tenant_manager.py assertions to expect batch API calls (list argument) instead of per-tenant calls; this reflects the new batch behavior which is the explicit goal of this feature - Added 11 new tests in test_data_manager.py covering parallel tenant processing, sequential fallback, error collection, and concurrent request scaling Closes #153 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![orca-security-eu[bot]](https://avatars.githubusercontent.com/in/276244?s=60&v=4){kind=small}
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
 Passed |
Infrastructure as Code |  0  0  0  0 |
View in Orca |
| Passed |
SAST | 0 0 0 0 |
View in Orca |
| Passed |
Secrets | 0 0 0 0 |
View in Orca |
| Passed |
Vulnerabilities | 0 0 0 0 |
View in Orca |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds parallel multi-tenant operations to significantly improve performance for bulk operations on collections with many tenants. The changes convert sequential tenant processing loops into parallel execution using ThreadPoolExecutor, and replace N individual tenant API calls with single batch operations.
Changes:
- Converted
delete_tenantsandupdate_tenantsto use batch API calls (single call with a list of tenants) instead of N individual calls - Added parallel tenant processing for
create_data,update_data, anddelete_datausing ThreadPoolExecutor - Added
--parallel_workersCLI option (default:min(32, cpu_count + 4)) to control parallelism, with automatic per-tenantconcurrent_requestsscaling to prevent cluster overload
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| weaviate_cli/managers/tenant_manager.py | Converted delete_tenants and update_tenants to use batch API calls with lists instead of loops |
| weaviate_cli/managers/data_manager.py | Added parallel tenant processing with ThreadPoolExecutor for create_data, update_data, and delete_data; includes concurrent_requests scaling and error aggregation |
| weaviate_cli/defaults.py | Added parallel_workers default (MAX_WORKERS) to CreateDataDefaults, UpdateDataDefaults, and DeleteDataDefaults |
| weaviate_cli/commands/create.py | Added --parallel_workers CLI option for create data command |
| weaviate_cli/commands/update.py | Added --parallel_workers CLI option for update data command |
| weaviate_cli/commands/delete.py | Added --parallel_workers CLI option for delete data command |
| test/unittests/test_managers/test_tenant_manager.py | Updated test assertions to expect batch API calls (lists) instead of individual calls |
| test/unittests/test_managers/test_data_manager.py | Added 11 new tests covering parallel processing, error collection, concurrent_requests scaling, and edge cases |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ror handling - Suppress per-tenant progress messages in parallel mode (update/delete) to avoid interleaved output; messages only print when parallel_workers <= 1 - Make sequential error handling consistent with parallel: collect all tenant errors instead of failing fast on the first one, then raise a combined exception (matches parallel path behaviour) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds four parallel-processing tests for create_data that mirror the TestUpdateDataParallel and TestDeleteDataParallel classes, covering: all-tenants-in-parallel, sequential fallback, error collection, and non-MT collection handling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
__delete_data always returns a non-negative count or raises; it never returns -1. Drop the dead sentinel branches from both the parallel and sequential paths in delete_data and rely on exceptions for error propagation instead, simplifying the control flow. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduce actual_workers = min(parallel_workers, len(tenants), concurrent_requests) and use it as both the ThreadPoolExecutor max_workers and the divisor for per-tenant concurrent_requests: - Over-throttle fix: 2 tenants + 32 workers previously divided by 32, leaving each thread with 1 request instead of half the budget. Now divides by min(32, 2, …) = 2, fully utilising the budget. - Over-budget fix: 32 workers + 4 concurrent_requests previously floored to 1/thread × 32 threads = 32 total. Capping actual_workers at concurrent_requests keeps total in-flight ≤ concurrent_requests. Update the concurrent_requests scaling test to reflect the corrected expected value (test was asserting the old over-throttled behaviour). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
delete tenantsandupdate tenantsnow issue a single batch API call instead of N individual calls, eliminating the main bottleneck for collections with many tenantscreate data,update data, anddelete datause aThreadPoolExecutorto process multiple tenants concurrently, controlled by a new--parallel_workersoption (default:min(32, cpu_count + 4))parallel_workers > 1, per-tenantconcurrent_requestsis scaled down proportionally so total in-flight connections stay bounded, preventing cluster overloadTest plan
make test)make lint)test_tenant_manager.pyassertions to expect batch API calls (list args) instead of N individual calls — reflects the explicit behavior changetest_data_manager.pycovering:parallel_workers=1concurrent_requestsreduction for parallel tenants (prevents overload)concurrent_requestsunchanged for single tenantCloses #153
🤖 Generated with Claude Code