feat(diskann): Add Windows DiskANN support with asynchronous I/O - #625
Open
iaojnh wants to merge 142 commits into
Open
feat(diskann): Add Windows DiskANN support with asynchronous I/O#625iaojnh wants to merge 142 commits into
iaojnh wants to merge 142 commits into
Conversation
… into refactor/diskann_dlopen
…windows # Conflicts: # src/db/index/column/vector_column/engine_helper.hpp # src/include/zvec/core/interface/index_param_builders.h # tests/core/interface/index_group_by_test.cc
iaojnh
marked this pull request as ready for review
August 19, 2026 07:48
# Conflicts: # tests/db/index/common/db_proto_converter_test.cc
iaojnh
marked this pull request as draft
August 20, 2026 09:02
iaojnh
marked this pull request as ready for review
August 20, 2026 12:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR enables DiskANN on 64-bit Windows using aligned, unbuffered asynchronous
reads backed by Windows I/O completion ports (IOCP).
It also fixes a Windows-specific file-handle interaction that reduced random
read throughput to about 13% of the expected level, improves hot-node cache
preloading, and exposes a byte-oriented cache budget through the existing C++,
C, Python, protobuf, and JSON parameter paths.
Root cause and fix
DiskANN issues many aligned 4 KiB random reads. The standalone IOCP reader
reached about 29K IOPS, but the same reader inside a loaded searcher reached
only about 3.8K IOPS.
An ABBA test isolated the cause:
FileReadStoragekept an ordinary bufferedfile handle open beside DiskANN's
FILE_FLAG_NO_BUFFERING | FILE_FLAG_OVERLAPPEDhandles.The fix loads the metadata required by search, closes the shared buffered
native handle, releases the storage segments, and then opens DiskANN's aligned
IOCP reader. Vector fetching and streamer providers now use that reader as
well, while providers and iterators remain valid after the source streamer is
closed.
Main changes
FILE_FLAG_NO_BUFFERING | FILE_FLAG_OVERLAPPED.completions, stale batches, and short reads.
destination buffers.
cache_node_budget_bytesas an alternative tocache_node_num, with aconservative per-node estimate and the existing 10% node cap.
windows_overlappedthrough the existing I/O backend APIs.process exit status.
Performance validation
Cohere 1M, 768 dimensions, cosine distance, FP32, 1,000 queries, Windows Server
2022, with a 128 MiB hot-node cache budget:
Both runs loaded the same 40,088 cache nodes with zero failures. The full
FP32/FP16 parameter sweep retained the expected Recall range and reached about
27.5K process read operations/s. These are process I/O counters, not
device-wide hardware IOPS.
Compatibility and tests
cache_node_budget_bytesdefaults to0; existingcache_node_numconfigurations remain supported.
0.applications using the C++ shared-library ABI must rebuild against the new
headers.
Tests cover concurrent IOCP contexts, outstanding-request destruction,
misaligned and short reads, Unicode paths, retained storage references,
provider lifetime after streamer close, cache loading, parameter
serialization/validation, and Recall worker failure propagation.
Known limitation
cache_node_budget_bytesis a nominal budget per physical DiskANN index. It isnot yet charged against the process-wide
memory_limit_mb; the default remainsdisabled (
0).