Skip to content

feat(change-stream): SQLite-backed garbage collector - #653

Open
aldy505 wants to merge 6 commits into
mainfrom
aldy505/feat/change-stream/sqlite-garbage-collector
Open

aldy505 wants to merge 6 commits into
mainfrom
aldy505/feat/change-stream/sqlite-garbage-collector

Conversation

@aldy505

@aldy505 aldy505 commented Sep 26, 2026

Copy link
Copy Markdown
Collaborator

Supersedes #582

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 5 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7dbb16c. Configure here.

.bind(id.as_storage_path())
.bind(expires_at.map(|t| t.as_secs()))
.execute(&tx_db)
.await?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overwrites fail unique key constraint

High Severity

write always INSERTs into garbage_collector, but object_id is the primary key and the change stream uses write for overwrites. A second write of the same object hits a unique constraint, so the new expiration is dropped and the stale row remains.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7dbb16c. Configure here.

Comment thread migrations/sqlite/0001_garbage_collector.sql Outdated

// Decrement counter when done
active_tasks.fetch_sub(1, Ordering::SeqCst);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unordered tasks lose object updates

High Severity

write, update, and delete each tokio::spawn an independent task with no per-object ordering. A later update or delete can commit before the matching write, so expirations are skipped and deleted objects can be reinserted.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7dbb16c. Configure here.

Comment thread objectstore-service/src/change_stream/garbage_collector_sqlite.rs Outdated
#[cfg(feature = "storage-cogs")]
mod cost_tracker;
mod factory;
mod garbage_collector_sqlite;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Collector is never constructed

Medium Severity

SqliteGarbageCollectorStream lives in a private module and is never re-exported or built by ChangeStreamFactory. No backend can select this collector, so the new feature is unreachable.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7dbb16c. Configure here.

Comment thread objectstore-service/src/change_stream/garbage_collector_sqlite.rs Outdated
Comment thread objectstore-service/src/change_stream/garbage_collector_sqlite.rs Outdated
Comment thread objectstore-service/src/change_stream/garbage_collector_sqlite.rs Outdated
It's so hard to develop this on Windows. I need to grab my Linux laptop.
Comment thread objectstore-service/src/change_stream/garbage_collector_sqlite.rs Outdated
@codecov

codecov Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.98387% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.33%. Comparing base (bf02738) to head (b2e54d4).

Files with missing lines Patch % Lines
...vice/src/change_stream/garbage_collector_sqlite.rs 97.98% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #653      +/-   ##
==========================================
+ Coverage   91.27%   91.33%   +0.06%     
==========================================
  Files         116      117       +1     
  Lines       23496    23744     +248     
==========================================
+ Hits        21445    21686     +241     
- Misses       2051     2058       +7     
Components Coverage Δ
Rust Backend 94.85% <97.98%> (+0.04%) ⬆️
Rust Client 81.95% <ø> (ø)
Python Client 93.75% <ø> (ø)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines +329 to +338
// Write 3 records
stream.write(&id1, 1024, None);
stream.write(&id2, 2048, None);
stream.write(&id3, 4096, None);

// Update one
stream.update(&id2, Some(Timestamp::from_unix_secs(9999).unwrap()));

// Delete one
stream.delete(&id1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Concurrent write and delete calls on the same object ID can race. The delete may execute before the write, leaving a phantom record in the garbage collector database.
Severity: HIGH

Suggested Fix

To prevent this race condition, ensure that operations on the same object_id are serialized. This could be achieved by using a mechanism like a sharded actor model or a keyed mutex, where each object_id has its own lock or sequential queue. This would guarantee that operations for a specific ID are executed in the order they were submitted, while still allowing for concurrency across different IDs.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: objectstore-service/src/change_stream/garbage_collector_sqlite.rs#L329-L338

Potential issue: The `write()`, `update()`, and `delete()` methods use `tokio::spawn` to
execute database operations asynchronously without awaiting their completion. This
creates a race condition where operations on the same `object_id` can execute out of
order. For example, a `delete()` call could run before a preceding `write()` call for
the same ID. This would cause the `DELETE` to affect zero rows, and the subsequent
`INSERT` would succeed, leaving a phantom record in the garbage collector database that
should have been removed. This leads to incorrect garbage collection behavior and silent
data consistency issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant