Implement memory cache V2 in safe Rust - #7110
Conversation
Introduce a Rust memory-cache core behind the MEMORY_CACHE_V2 autogate, while retaining the existing C++ implementation as the rollback path. Keep the handwritten CXX bridge thin and isolate generated unsafe code from the core, which forbids unsafe code. Store cache entries in a LinkedHashMap for LRU ordering and maintain a separate expiration index for efficient cleanup. Use a per-key Tokio mutex to coalesce concurrent misses without a custom waiter queue. Represent registry membership with weak flight references so abandoned requests do not retain cache state, while flight destruction safely removes only its own registry entry. Make leader handoff cancellation-safe through mutex ownership. Promotion follows waiter poll order rather than request creation order, avoiding eager polling and custom scheduling machinery. Track waiter cardinality with lock-free, best-effort counters so metrics do not add contention to the cache-state lock. Expose the implementation through the existing memory-cache API and add coverage for eviction, expiration, coalescing, cancellation, abandoned reads, leader promotion, weak lifetime cleanup, and the autogated C++ integration.
|
LGTM |
| expiration: f64, | ||
| now_ms: f64, | ||
| ) -> Result<ffi::WriteTrace, memory_cache::CacheError> { | ||
| let permit = self |
There was a problem hiding this comment.
The current head fails mandatory Clippy here with significant_drop_tightening; the crate-level boxed_local expectation is also unfulfilled. Please shorten permit's lifetime and remove the obsolete expectation so the lint build passes.
| let removed = state.bindings.remove(&id); | ||
| debug_assert!(removed.is_some()); | ||
| recompute_limits(&mut state, &self.namespace); | ||
| state.resize(now_ms); |
There was a problem hiding this comment.
release() always calls resize(), which scans every entry for oversized values while holding the cache mutex, even when removing this binding leaves the effective limits unchanged. With a large shared cache and many equivalent live bindings, each isolate teardown therefore blocks all cache operations for O(entries). Could we skip resizing unless effective limits shrink, and only run the oversized-value scan when max_value_size decreases?
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7110 +/- ##
==========================================
- Coverage 67.95% 67.93% -0.02%
==========================================
Files 464 467 +3
Lines 130735 131860 +1125
Branches 21389 21452 +63
==========================================
+ Hits 88840 89579 +739
- Misses 28923 29269 +346
- Partials 12972 13012 +40 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
3 participants
Introduce a Rust memory-cache core behind the MEMORY_CACHE_V2 autogate, while retaining the existing C++ implementation as the rollback path. Keep the handwritten CXX bridge thin and isolate generated unsafe code from the core, which forbids unsafe code.
Store cache entries in a LinkedHashMap for LRU ordering and maintain a separate expiration index for efficient cleanup. Use a per-key Tokio mutex to coalesce concurrent misses without a custom waiter queue. Represent registry membership with weak flight references so abandoned requests do not retain cache state, while flight destruction safely removes only its own registry entry.
Make leader handoff cancellation-safe through mutex ownership. Promotion follows waiter poll order rather than request creation order, avoiding eager polling and custom scheduling machinery. Track waiter cardinality with lock-free, best-effort counters so metrics do not add contention to the cache-state lock.
Expose the implementation through the existing memory-cache API and add coverage for eviction, expiration, coalescing, cancellation, abandoned reads, leader promotion, weak lifetime cleanup, and the autogated C++ integration.