Repository navigation
Conversation
diskann_node_read(), diskann_vector_read() and IVF's rowid-map lookup reset their cached statements before each use and left them on the row they had read. fe94171 finalizes them at the next commit, but a read-only query has none, so after any DiskANN KNN query, point query or full scan, or an IVF point query or full scan, the connection kept a statement in progress and VACUUM failed with "cannot VACUUM - SQL statements in progress". Each of these lookups now resets its statement as soon as its row is copied out, on every path.
This was referenced Oct 23, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VACUUM fails after a read-only query on a DiskANN column, and after a point query or full scan on an IVF column.
diskann_node_read()anddiskann_vector_read()reset their cached statements before each use, step them to the row they read, copy the row out, and return with the statement still on that row, so it stays in progress. fe94171 (#295) finalizes the cached statements inxSync, which a read-only query never reaches: a KNN query, a point query or a full scan of a DiskANN column leaves one in progress until the connection's next commit, and VACUUM refuses to run. IVF's rowid-map lookup does the same inivf_get_vector_data()andivf_delete(), so an IVF point query or full scan leaves it in progress; an IVF KNN query does not.Each of these lookups now resets its statement as soon as its row is copied out, on every path, including the errors after the step. The IVF cell lookup in
ivf_cell_find_or_create()is left as it is: it runs inside an insert, whose commit finalizes it, and #313 rewrites that function with the same reset.Built with the IVF index, on main, VACUUM fails after each of the five reads (a DiskANN KNN query, point query and full scan; an IVF point query and full scan) and runs after rescore and flat reads; here it runs after all of them. Tests:
test_vacuum_after_diskann_readin test-cache-finalize.py, parametrized over the three DiskANN reads, andtest_ivf_vacuum_after_readin test-ivf.py over the two IVF reads. All five fail on main and pass here. The Python suite passes (242 passed, 68 skipped; with-DSQLITE_VEC_EXPERIMENTAL_IVF_ENABLE=1, 306 passed, 4 skipped), and both builds' warnings are main's.It merges cleanly with every open PR that touches the DiskANN or IVF code, #313, #344, #345 and #346 included.
Origin. Found in the DiskANN and IVF ports to photostructure/sqlite-vec, photostructure#11 (cd53413) and photostructure#12 (65817f4), which list it among the ports' differences from upstream; the change and its tests are written here for asg017's main. #19 was the same failure on a flat column, which 7fc8248 fixed by finalizing vec0's statements before each commit, and #295 asked for the DiskANN statements to be released without a disconnect. No open or closed PR or issue here makes this change.