Add CoW-tree persistent storage with trace logging#10
Open
christianparpart wants to merge 27 commits into
Open
Add CoW-tree persistent storage with trace logging#10christianparpart wants to merge 27 commits into
christianparpart wants to merge 27 commits into
Conversation
added 27 commits
May 28, 2026 14:18
New sibling library at src/CowTree/ providing a persistent, crash- consistent key->value store. No dependency on FastCache; lives in its own ::CowTree namespace and CMake target so other projects can lift it out as-is. Highlights: - IPageStore DI seam with two implementations: InMemoryPageStore (deterministic, with FailNthWrite / FailNthSync / TornOnNthWriteMeta fault injection) and FilePageStore (pread/pwrite + fsync; Windows ReadFile/WriteFile + FlushFileBuffers; Fsync / Batched / None durability). - Dual-meta commit protocol: data pages written, fsync, then a single- page WriteMeta + fsync into the alternating slot. Recovery picks the slot with the higher txnId and valid CRC32C; a torn meta write rolls back to the previous transaction. - O(1) snapshots (captured root + txnId). - Catch2 tests: CRC vectors and parameterized bit-flip detection, meta encode/decode + per-byte CRC mutation, page-layout round-trips, B+tree round-trips and splits, random workload vs std::map oracle, crash-consistency suite that injects failures at each commit-protocol offset, FilePageStore persistence and per-durability fsync counts. Known v1 limitations are documented in src/CowTree/README.md. Top-level CMakeLists.txt moves enable_testing() ahead of all subdirectories so CowTreeTests can register with CTest, and links FastCache against the new target PUBLIC (CowTreeStorage exposes CowTree types in its public header). Signed-off-by: Christian Parpart <christian@parpart.family>
CowTreeStorage implements IStorage on top of the CowTree library. Encodes each CacheEntry into a flat byte string (flags, cas, expiry, generation, value) and routes Set/Get/Delete/CAS/etc. through CoW write transactions. Maintains an in-memory LRU mirror for byte-budget eviction (LRU order itself is not persisted). Enforces a configurable per-value byte cap (Options::maxValueBytes, default 1 MiB suited to sccache compile-cache values); the backing page size is derived from that cap. TracingStorage is a decorator over IStorage that emits one Trace log line per call describing verb, key, outcome, and latency. All boilerplate is collapsed into a single templated TraceCall helper; each override is a one-liner. CacheEngine and the protocol handlers stay unchanged - logging is pure composition. When the logger's MinLevel sits above Trace, the only overhead per call is one atomic load. Tests: - CowTreeStorage_test: Set+Get round-trip, persistence across reopen, Add KeyExists / Replace KeyNotFound, CAS mismatch + match, TTL expiry. - TracingStorage_test: HIT/MISS/STORED/NOT_STORED/DELETED/NOT_FOUND outcome strings against InMemoryLruStorage with a CapturingLogger, zero records when MinLevel > Trace, pure pass-through semantics. Signed-off-by: Christian Parpart <christian@parpart.family>
Inverse of ParseByteSize. Returns the largest unit (G/M/K) for which the byte count is an exact integer multiple; falls back to a raw byte count with a B suffix otherwise. 1024-based to match ParseByteSize. Examples: FormatByteSize(0) == "0B" FormatByteSize(1024) == "1K" FormatByteSize(64 * 1024 * 1024) == "64M" FormatByteSize(1025) == "1025B" Tests in ByteSize_test.cpp cover zero, clean multiples at each unit, and non-clean fallback. Signed-off-by: Christian Parpart <christian@parpart.family>
Three new operability knobs for the persistent backend, surfaced
identically through CLI and YAML.
- --storage=PATH : switch from in-memory to CowTreeStorage
backed by PATH. Empty (default) keeps the
existing InMemoryLruStorage behavior.
- --storage-durability : fsync | batched | none (default batched).
- --storage-max-value : per-value byte cap; same k/m/g suffix
grammar as --max-memory (no % form, which
would be meaningless for a value size).
Default 1m, sized for sccache compile-cache
objects.
YAML equivalents are storage_path, storage_durability, storage_max_value.
main.cpp gains a storage factory that picks the backend, optionally
wraps it in TracingStorage when --log-level=trace, and threads
through to CacheEngine without any change to its constructor. The
startup Info log now reads:
fastcached <ver> starting; bind=...:... max-memory=<sz> config=...
storage=<path|<in-memory>> durability=<mode> max-value=<sz>
with FormatByteSize used for the size fields.
CliParser_test adds coverage for each new flag including byte-size
suffix parsing and OutOfRange / TypeMismatch rejection paths.
Signed-off-by: Christian Parpart <christian@parpart.family>
Adds: - A mention of the optional CoW-tree persistent backend in the Status section, linking to src/CowTree/README.md. - The new flags in the --help block. - A "Run with 30% of host RAM and a persistent cache file" subsection with both POSIX shell and PowerShell invocations. - storage_path / storage_durability in the YAML example. - src/CowTree/ in the repo-layout listing. Signed-off-by: Christian Parpart <christian@parpart.family>
clang-format-20 applied to every .h/.hpp/.cpp added or modified on this
branch (mostly include grouping and brace placement).
clang-tidy errors in CowTree (Linux clang-debug -Werror):
- replace `std::size_t childIdx = static_cast<std::size_t>(-1)` sentinel
with `std::optional<std::size_t>` in PutRec / EraseRec, eliminating
modernize-use-auto + modernize-use-integer-sign-comparison.
- switch all EraseResult{...} construction to designated initializers
(modernize-use-designated-initializers).
- replace `(void) _store.Free(x)` with `std::ignore = _store.Free(x)`
so bugprone-unused-return-value is satisfied.
clang-tidy errors in FilePageStore:
- value-initialise `struct stat` (cppcoreguidelines-pro-type-member-init).
- parenthesise `(fileSize - (2 * _pageSize))`
(readability-math-missing-parentheses).
- mark `WriteAt()` const (readability-make-member-function-const).
clang-release: drop a dead `auto bufResult = _store.Read(*idResult);`
that was unused (we encode into a local vector instead).
gcc -Wmissing-field-initializers in CowTreeStorage: add a `MakeError`
helper that constructs StorageError with all three fields populated;
replace every bare `StorageError { .code = X }` with `MakeError(X)`.
Also drop the unused `ValueView` helper that gcc flagged with
-Wunused-function.
Signed-off-by: Christian Parpart <christian@parpart.family>
Rename `splitIdx` -> `splitIndex` and `childIdx` -> `childIndex` in the B+tree split / descend code. Pure renames, no behavioural change. Signed-off-by: Christian Parpart <christian@parpart.family>
After the previous CI fix run uncovered a second wave of clang-tidy
errors on Linux:
CowTree.cpp:
- Remove redundant `const_cast<IPageStore*>(&_store)` in BeginRead /
Lookup. `_store` is a non-const reference member, so `&_store` is
already `IPageStore*`; the cast was a no-op flagged by
readability-redundant-casting.
- Switch the two single-leaf `PutResult { *id, std::nullopt, false }`
constructions to designated initializers
(modernize-use-designated-initializers).
InMemoryPageStore.cpp:
- Replace `(void) EncodeMeta(...)` in the constructor bootstrap with
`std::ignore = EncodeMeta(...)` so bugprone-unused-return-value is
satisfied; add `<tuple>`.
FilePageStore.cpp:
- Drop the redundant `static_cast<std::uint64_t>` over a value that's
already u64; rename the local to `pageIndex` per the project
preference to spell out short names (the previous `idx` would have
carried the abbreviation through).
CrashConsistency_test.cpp / BasicCowTree_test.cpp:
- Remove the unused helpers `RequirePostCommit` and `S` that gcc /
clang -Werror=unused-function flagged.
.clang-tidy:
- Raise readability-function-cognitive-complexity.Threshold from 25
to 60. The B+tree Get / PutRec / EraseRec functions run 31 / 40 / 58
on the metric — their complexity is structural (recursive descent +
split / merge / child-collapse handling) rather than accidental.
Functions without an algorithmic reason to be complex still trip
the check well before 60.
Signed-off-by: Christian Parpart <christian@parpart.family>
ShardedStorage owns N inner IStorage instances and fans every call across them by `std::hash(key) % N`. Each shard carries its own `std::shared_mutex`: - Get / Snapshot / PurgeExpired take a *shared* lock — any number of threads can read the same shard simultaneously without blocking each other. - Set / Add / Replace / Append / Prepend / CompareAndSwap / IncrementOrInitialize / Delete take an *exclusive* lock and serialise only with other writers and concurrent readers on the *same* shard. - A write to shard N never blocks any operation on a different shard. This is what makes the existing single-threaded-by-contract storages (InMemoryLruStorage, CowTreeStorage) safe to share across the thread pool that lands in a separate commit. FlushWithGeneration locks every shard exclusively so the generation bump is atomic from any observer's perspective; Snapshot aggregates per-shard StorageStats so the reported totals stay global. ResizeTotal splits a new budget evenly across the shards. Tests in ShardedStorage_test.cpp: - Round-trip Set / Get / Delete via the decorator. - Deterministic hash partitioning. - An explicit "concurrent readers do not block each other" proof using a ParkableStorage stub that parks inside Get holding the shared lock; a second reader on the same shard must reach the stub before the first releases. - "Writer excludes readers on the same shard but not across shards": a writer parks inside Set; a same-shard reader stays blocked while a different-shard reader proceeds immediately. - Multi-threaded random-workload stress vs a std::map oracle. - Snapshot aggregation and FlushWithGeneration cross-shard semantics. Signed-off-by: Christian Parpart <christian@parpart.family>
New accept loop that creates its worker threads once at startup and reuses them for every connection. One accept thread pushes accepted ISockets into a bounded blocking queue (max(N*4, 64)); N workers loop popping sockets and driving the per-connection Connection::Run() coroutine via SyncRun. Bounded queue gives the accept thread back- pressure when workers fall behind. Shutdown drains and joins via a RAII pool wrapper that pushes N sentinels through the queue. Why a pool (not std::jthread-per-connection like BlockingServerLoop): sccache opens one connection per compile job, so a thousand-job build would otherwise pay a thousand pthread_create / _beginthreadex round- trips on a hot path where each cache op is sub-millisecond. Tests in PooledServerLoop_test.cpp: - Serves N pre-staged connections with a 4-worker pool (uses a ShardedStorage so concurrent workers don't race on a single unprotected InMemoryLruStorage). - Returns immediately on a pre-closed listener (drain + join with no hang). - poolSize=0 resolves to std::thread::hardware_concurrency(). Signed-off-by: Christian Parpart <christian@parpart.family>
A failed assert() or MSVC CRT debug check pops a modal dialog by default. That dialog blocks every headless ctest / CI run forever waiting for a button click. Each test main now routes CRT diagnostics to stderr, disables the abort + GPF dialogs, and sets the Windows error mode to non-interactive at the start of main(). POSIX behaviour is unchanged. Signed-off-by: Christian Parpart <christian@parpart.family>
Surface the new concurrency knobs through both CLI and YAML.
- --threading-model={threaded,reactor} (yaml: threading_model) —
default `threaded` (PooledServerLoop). `reactor` opts back into
the existing single-threaded RunReactorServer for testing or
low-resource deployments.
- --threads=<N> (yaml: threads) — worker pool size for threaded mode.
0 means `std::thread::hardware_concurrency()`. Ignored in reactor
mode.
- --storage-shards=<N> (yaml: storage_shards) — number of storage
shards used by ShardedStorage. 0 means auto. When N>1 and --storage
is set, --storage is treated as a directory holding shard-NN.cow
files; N=1 preserves the single-file behaviour.
New ThreadingModel enum in Config.hpp keeps the layer independent
from the server subsystem.
Signed-off-by: Christian Parpart <christian@parpart.family>
DaemonBody is now responsible for four storage layouts: - In-memory single shard: one InMemoryLruStorage. - In-memory N shards: N InMemoryLruStorage wrapped in ShardedStorage, each with maxMemoryBytes/N budget. - Persistent single shard: one CowTreeStorage at --storage (the file). Preserves PR #10's contract. - Persistent N shards: --storage is the directory, N CowTreeStorage instances at <dir>/shard-NN.cow, wrapped in ShardedStorage. Shard count defaults to min(16, hardware_concurrency) when the user leaves --storage-shards at 0. Server selection: when threadingModel == Threaded (default), bind a BlockingListener and drive it with RunPooledServerLoop sized by --threads. A watchdog jthread closes the listener on stop-request so the accept loop unblocks. When Reactor, fall back to RunReactorServer. ConfigReloader's Resize handler now also dispatches to ShardedStorage::ResizeTotal so SIGHUP keeps working in sharded deployments. Startup info log gains `threading=<mode> shards=<N>` so operators see what got selected. Signed-off-by: Christian Parpart <christian@parpart.family>
- Updates Status to mention the thread-pool default and the single-threaded reactor opt-in. - Adds the new flags to the --help block. - New "Concurrency: thread pool + sharded storage" subsection explaining the design and showing a sccache-shaped invocation with --storage-shards=16 --threads=16, plus the reactor opt-out. - Documents the file-vs-directory rule for --storage that depends on --storage-shards. - Adds threading_model, threads, storage_shards (and updates storage_path) in the YAML example. Signed-off-by: Christian Parpart <christian@parpart.family>
The previous Linux-clang-debug run got past the earlier files but stopped at CowTreeStorage.cpp with 8 clang-tidy errors that the next wave only saw after the prior wave was fixed. Same patterns: CowTreeStorage.cpp - Convert the LruNode push_front, GetResult returns, and IncrResult return to designated initializers (modernize-use-designated-initializers). - Replace the two `(void) EraseEntry(...)` discards in EvictToFit / Get with `std::ignore = EraseEntry(...)` so bugprone-unused-return-value is satisfied; add <tuple>. - Uppercase the integer literal suffixes `0u` -> `0U` in the saturating decrement and in the IncrResult fallback path (readability-uppercase-literal-suffix). main.cpp - Uppercase `1u`/`16u` in the ResolveThreadCount cap to satisfy readability-uppercase-literal-suffix. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
Two failures from CI on the previous fix:
Linux-clang-debug: clang-analyzer-optin.performance.Padding flagged
src/FastCache/Config/Config.hpp's `Config` struct as carrying 34
bytes of padding when 2 is optimal. The struct interleaved small
trailing fields (port, logLevel, daemon, enums) between strings and
size_ts. Reorder per the analyzer's suggested layout: size_ts first,
then strings, then the trailing scalars/enums. Update the four
designated initializers in ConfigReloader_test.cpp to match the new
declaration order so -Wreorder-init-list stays clean.
Check C++ style: clang-format-20 reported violations in six files
that were added in this branch's new commits (ShardedStorage{,_test},
PooledServerLoop{.hpp,.cpp}, CliParser.cpp, YamlReader.cpp). Run
clang-format -i over them; pure whitespace changes.
Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
After the previous Linux-clang-debug fix unblocked Config.hpp, clang-tidy moved on into src/CowTree/*_test.cpp and surfaced three more categories of violations: readability-uppercase-literal-suffix (12 sites): uppercase `u` -> `U` in Crc32c_test.cpp, InMemoryPageStore_test.cpp, FilePageStore_test.cpp. bugprone-unchecked-optional-access (10 sites): the tests do `REQUIRE(got.has_value()); REQUIRE(got->has_value()); ... **got`, but clang-tidy does not model Catch2's REQUIRE as an abort, so it still considers the second deref unchecked. Switch the deref to the optional's `.value()` (which throws on empty and is recognised as a checked access) in BasicCowTree_test.cpp, CrashConsistency_test.cpp, FilePageStore_test.cpp. No runtime change: the REQUIRE above still catches the empty case before .value() can throw. cppcoreguidelines-special-member-functions (1 site): TempFile in FilePageStore_test.cpp explicitly deleted copy ops but left move ops implicit-deleted-by-presence-of-user-defined-dtor. Add explicit `= delete` for the move ctor and assignment to satisfy the rule of five. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
Renames the user-facing flag (and YAML key threading_model → execution_model) so the knob describes what it picks, not just how threads are involved. Adds an `auto` value (now the default) that picks reactor for the in-memory cache and threaded for CoW on-disk storage — the right defaults for each backend. Tracks `executionModelExplicit` on CliResult so an explicit --execution-model=auto on the CLI overrides a non-auto value in the YAML config; without it, the existing "CLI != default → override" heuristic in Merge() silently dropped the explicit auto because Auto is now also the sentinel default. The startup banner annotates the resolved model with " (auto)" only when the user did not pick a concrete model, instead of repeating the resolved name in a parenthetical. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
- `EncodeLeafPage` / `EncodeInternalPage` reject `entries.size() > MaxEntriesPerPage` (= 65535); previously the cast to `uint16_t` silently truncated, so a page encoded with 70k tiny entries decoded back as 4464 with the rest lost. - `MaxKeyLength` lowered from 65536 to 65535 to match the on-disk `uint16_t` length field — keys of exactly 65536 bytes used to truncate to 0 on encode. - `RecoverExistingFile` tracks a visited-set over the free-list chain and decodes the `next` link via the same little-endian path as every other on-disk field. A corrupted/self-referencing freeRoot used to spin Open() forever and grow `_freeList` without bound; a big-endian host would also have byte-swapped the link relative to writers. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
Five distinct correctness issues in one file: - `Get` on expired / stale-generation entries no longer opens a write transaction. Under ShardedStorage's previous shared_lock for Get, two concurrent expired-Gets on the same shard fired two `BeginWrite()`s, violating CowTree's single-writer contract and racing the free list. Cleanup is now deferred to `PurgeExpired` (writer-locked). - `Delete` on an expired entry still erases the on-disk record before returning KeyNotFound, so dead bytes don't linger across restart. - `EvictToFit` calls `EraseEntry` BEFORE mutating `_lru`/`_index`/ `_bytesUsed`. On disk-delete failure (ENOSPC, fsync error) the failing victim is rotated out of the LRU tail and the loop bails after a full unsuccessful pass — better to violate the soft cap than to silently desync the in-memory mirror from disk and over-count evictions. - `IncrementOrInitialize` replaces `static_cast<uint64_t>(-delta)` with the `-(delta + 1) + 1` idiom that InMemoryLruStorage already uses; the original was UB when `delta == INT64_MIN`. - `PurgeExpired` is no longer a no-op: it walks the LRU mirror, erases expired / stale-generation entries on disk, and drops them from the mirror. Needed now that the read path no longer cleans up. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
The wrapped backends (InMemoryLruStorage, CowTreeStorage) mutate LRU order, stats counters, and (for CowTree) a `mutable _stats` member on `Get` and `Snapshot`. Holding only a shared_lock raced two concurrent same-shard readers on those mutations — TSan-visible data race; in production it produces torn counters and intrusive-list corruption. Switching to unique_lock here is the minimum-surface fix. Cross-shard parallelism is preserved (the whole point of sharding). The longer-term path to recover read parallelism on a single shard is to split Get into Lookup + deferred-promote, but that's a separate refactor. The "concurrent readers do not block each other" test is replaced with two tests that pin the new contract: same-shard Gets serialise, distinct- shard Gets run in parallel. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
Two related Config bugs introduced by the new flags on this branch: - `Merge()` in main.cpp used value comparison (`cliCfg.X != defaults.X`) to decide whether to override the YAML value. When the user-typed CLI value happens to equal the default (`--threads=0`, `--storage-shards=0`, `--storage-durability=batched`, `--storage-max-value=1m`), the check is `0 != 0` = false and YAML silently wins. Only `executionModel` had an explicit-tracker bool. This commit extends the pattern across every CLI scalar field; `Merge` will gate on the bools rather than value equality (consumed by the matching main.cpp change). - `ConfigReloader::ValidateImmutable` only enforced bindAddress and port. The seven new fields (storagePath, storageShards, storageDurability, storageMaxValueBytes, executionModel, workerThreads) silently swapped in via SIGHUP while the running backend kept the original settings. Now rejected with `ConfigErrorCode::ImmutableChanged`. Help text also documents the new path-aware default for `--storage-shards`. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
Two related issues with the storage factory: - The old `ResolveShardCount(0)` returned `min(16, hardware_concurrency)` unconditionally, so `--storage=/var/lib/fc/cache.cow` on any multi-core host silently `mkdir`'d the user's intended file path and wrote shard-NN.cow files inside it. The README's documented single-file invocation was effectively never single-file. The new `ResolvePhysicalShards` infers from the path: regular file or absent → 1 shard (single-file mode, matching the README); existing directory → auto fan-out. - A single-shard CowTreeStorage / InMemoryLruStorage exposed under the PooledServerLoop's thread pool had no internal locking, so workers raced on `_index`/`_lru`/`_bytesUsed`/`_nextCas`/`_stats`. The factory now always wraps in ShardedStorage (even at one shard) when the resolved execution mode is Threaded. The per-shard mutex serialises workers against the unwrapped backend. `Merge` switches to the per-flag `*Explicit` bools added in the previous commit, so `--threads=0` etc. now correctly override YAML. The startup log line and `--help` text reflect the new behaviour, and the README documents both the path-aware default and the loss of crash consistency under `--storage-durability=none`. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
Expands CowTreeStorage_test.cpp from 6 cases (~165 LOC) to 37 cases (~840 LOC, 6824 assertions). Coverage focuses on byte-for-byte roundtrip across reopen — the user's stated requirement that "we read exactly what we've written": - byte-level roundtrip: empty value, every single byte 0x00..0xFF, all- byte-values blob, 1 KiB and 64 KiB random binary, value exactly at maxValueBytes (and one above → ValueTooLarge), keys with embedded NULs and non-ASCII bytes - metadata roundtrip: flags (0, 1, 0xDEADBEEF, UINT32_MAX), expiry (TimePoint::max, far future), CAS monotonicity - B+tree shape: 1000 small entries random-order read-back across reopen, lexicographic-prefix neighbours, update-in-place, delete + reinsert - compound ops: Replace persists across reopen; Append + Prepend roundtrip and persist; Append above maxValueBytes leaves value intact; CompareAndSwap success persists, mismatch leaves entry untouched; IncrementOrInitialize create + increment across reopen, floor at 0 - delete & expiry: Delete-on-expired regression (cleans disk record), Get-on-expired regression (no tree mutation), PurgeExpired count - eviction & accounting: LRU-tail eviction under maxBytes, Resize shrinks budget immediately - persistence: 3 open/close cycles preserve every entry; mixed Set/Update/Delete script replays identically across mid-script reopen - failure modes: fresh file creation, corrupt-bytes Open, oversize value INT64_MIN decrement now under regression test (#26 in plan). CliParser_test.cpp gains regressions for `--threads=0` / `--storage-shards=0` / `--storage-durability=batched` explicit-trackers plus an all-flags-absent baseline. ConfigReloader_test.cpp gains storage_path and storage_durability immutable-rejection cases. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
`DiskStorage` was the original append-only-log persistent backend. The CoW-tree backend (`CowTreeStorage`) introduced on this branch covers the same role with a more robust format (crash-consistent copy-on-write commits, single canonical file, deterministic recovery), and `main.cpp` does not instantiate `DiskStorage` anywhere — only its test file referenced it. Keeping a second on-disk format around dilutes the codebase, adds build cost, and risks bit-rot. Drop it (and its test file) entirely. The CRC32C doc comment in `Core/Crc32c.hpp` loses its stale "used by DiskStorage" reference. Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
`LayeredStorage` composes an in-memory LRU cache (L1) in front of any
`IStorage` (L2). Reads hit L1 first; on a miss the L2 result is
mirrored into L1 with **L2's CAS preserved**, so subsequent CAS / Get
calls return the same token whether served from RAM or disk. Writes
pass through L2 first (canonical CAS) and then mirror into L1
verbatim.
The "mirror an entry with a chosen CAS" primitive is implemented as a
new public method on `InMemoryLruStorage`:
void InsertVerbatim(std::string_view key, CacheEntry entry);
It stores the entry as-is — same cas/flags/expiry/generation — bypassing
the CAS issuance the standard `Set` does. `L1` in `LayeredStorage` is
held as a concrete `InMemoryLruStorage*` (not polymorphic) so the
mirroring path can use this primitive without polluting the `IStorage`
interface. A future tiering generalisation can templatise on `L1`.
The 15 test cases cover write-through roundtrip, read-through populates
L1, CAS coherency across L1 eviction, Delete drops both tiers,
Append / Prepend / CAS / Incr canonical at L2, FlushWithGeneration on
both tiers, PurgeExpired returns the L2 count, L1 eviction never loses
data (L2 still serves), Snapshot tracks LayeredStorage stats,
LayeredStorage::Resize tunes only L1, Add/Replace consult L2 (the
canonical store), and a full sharded-and-layered end-to-end persistence
test using a real `CowTreeStorage` as L2 across two open/close cycles.
Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
When `--storage=<path>` is set, each physical shard is now composed as
`LayeredStorage(InMemoryLruStorage, CowTreeStorage)`. Reads consult the
RAM cache first; misses fall through to the CoW tree on disk and the
result is mirrored into RAM with the disk's CAS preserved. `--max-memory`
is the L1 (in-memory cache) byte budget; the on-disk file grows
independently (`CowTreeStorage::Options::maxBytes = 0`). The full
production shape:
TracingStorage(
ShardedStorage[N](
Shard 0: LayeredStorage(InMemoryLruStorage, CowTreeStorage(shard-00.cow))
Shard 1: LayeredStorage(InMemoryLruStorage, CowTreeStorage(shard-01.cow))
...))
`ShardedStorage` stays **outside** `LayeredStorage` so each shard's
mutex serialises the (L1, L2) pair as one atomic unit — a Get
populating L1 from L2 and a Set writing through both run under one
lock.
`ShardedStorage::ResizeTotal` learns about `LayeredStorage` (forwards
to `LayeredStorage::Resize`, which tunes L1's budget). The single-
disk-backend reload path uses a `LayeredStorage*` capture instead of
the raw `CowTreeStorage*` it held before.
The flaky-timing tail of `ShardedStorage_test.cpp`'s
"Concurrent same-shard Gets serialise" test (introduced in the previous
correctness pass) is simplified: assert reader2 stays parked while
reader1 holds the lock, release, then join. The original two-phase
re-park dance depended on `ParkableStorage::Release` toggle semantics
that were already racing the wake-up.
README documents the layered composition and the new meaning of
`--max-memory` (now the L1 cache budget; L2 file is unbounded).
Signed-off-by: Christian Parpart <c.parpart@lastrada.net>
christianparpart
force-pushed
the
cow-tree-persistent-storage
branch
from
1ccc0db to
d02e9ff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional persistent backend for the cache. Operators can now point fastcached at a file with
--storage=PATHand entries survive process restarts; a kill -9 mid-write leaves the file in either the previous or the new transaction, never a hybrid. The backend is a copy-on-write B+tree built as a standalone sibling library (src/CowTree/) with no dependency on FastCache, so other projects can reuse it. A separateTracingStoragedecorator gives--log-level=traceper-request visibility (verb, key, outcome, latency) without touching the cache engine or protocol handlers.The user-facing knob most likely to be tuned for a compile-cache workload is
--storage-max-value, which defaults to 1 MiB and accepts the usualk/m/gsuffixes. Oversized writes returnVALUE_TOO_LARGE(visible in the trace log).Changes
src/CowTree/— new standalone copy-on-write B+tree library. Dual-meta commit protocol (data sync, then a single-page meta write + fsync into the alternating slot, recovery picks the higher valid txnId), pluggableIPageStorewith in-memory (fault-injecting) and file-backed (pread/pwrite+ fsync; WindowsReadFile/WriteFile+FlushFileBuffers) implementations, three durability modes. O(1) snapshots. Catch2 tests: CRC vectors with parameterized bit-flip detection, meta encode/decode with per-byte CRC mutation, page-layout round-trips, B+tree splits, random workload compared againststd::map, and a crash-consistency suite injecting failures at each commit-protocol offset. Known v1 limitations documented insrc/CowTree/README.md.CowTreeStorage—IStorageadapter mapping cache operations onto CowTree transactions. Per-value byte cap is configurable; the backing page size is derived from it. Maintains an in-memory LRU mirror for byte-budget eviction.TracingStorage—IStoragedecorator emitting one Trace log line per call. All boilerplate collapses into a single templatedTraceCallhelper; each override is a one-liner.CacheEngineis unchanged.--storage/--storage-durability/--storage-max-value— CLI flags and matching YAML keys (storage_path,storage_durability,storage_max_value). Same byte-size suffix grammar as--max-memory(without the%form, which would be meaningless for a value size).FormatByteSize— inverse ofParseByteSize. Used by the startup info log somax-memoryandmax-valueprint as64M/2Gwhen the byte count is a clean integer multiple, falling back to raw bytes with aBsuffix otherwise.src/CowTree/in the repo-layout listing.