From 2789cd9b8260be8048a8c9d455c74e8034f89801 Mon Sep 17 00:00:00 2001 From: Maxwell Elliott Date: Fri, 16 Oct 2026 04:46:01 +0000 Subject: [PATCH 1/2] docs: assess a TGB-style columnar binary cache format for serve Record the investigation into porting Tango's TGB format (columnar blobs, raw fixed-stride hash column, block-aligned bisecting compare) to the serve cache. Measured on 1M-entry synthetic hash files: JSON is 212 B per node and a columnar entry would be 66 B (3.2x, not Tango's 10x, because 97% of it is two SHA-256 columns); the compare drops from 748 ms to 22 ms, and the 6.3 s per-request JSON parse disappears entirely. Deferred until serve entries or compare latency become a problem; the note lists the thresholds, the cheaper interim steps (in-memory parsed-entry LRU, direct parse, allocation-free compare, compressed shared tier) and a column layout and compare algorithm fitted to bazel-diff's hash files. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Gs1vjHt61po6uHgkYuh68N --- docs/tgb-binary-cache.md | 191 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 docs/tgb-binary-cache.md diff --git a/docs/tgb-binary-cache.md b/docs/tgb-binary-cache.md new file mode 100644 index 00000000..1bc60b30 --- /dev/null +++ b/docs/tgb-binary-cache.md @@ -0,0 +1,191 @@ +# TGB-style binary cache format for `serve`: assessment and design note + +Status: **deferred, design recorded**. Revisit when `serve` cache entries or per-request compare +time become a problem at monorepo scale (see "When to build it"). + +Uber's Tango stores its target graph in TGB (Tango Graph Binary), a columnar format whose +headline result is 10x smaller blobs and 7–14x faster compares on a 2.8M-node graph. The speedup +comes from one idea: nodes are sorted, hashes live in one raw fixed-stride column, and an unchanged +run of nodes is proven equal with a single `bytes.Equal` plus bisection on mismatch. This note +measures what that idea would buy bazel-diff, which already stores far less per node than Tango, +and records a design that fits bazel-diff's hash files so it can be built when it pays off. + +Reference: `docs/tgb/tgb.md` and `internal/tgbdiff/compare.go` in uber/tango at +`41480a27e50d3a86dc25cd4d773bbe1b60474cdb` (the commit pinned by `tools/perf/tango_compare.py`). + +## What bazel-diff stores and how it compares today + +A hash file (and a `serve` cache entry, which is the same JSON) is one object mapping each label to +`"#~"`, optionally wrapped as `{"hashes": ..., "metadata": ...}` when +it carries `moduleGraphJson`, `dependencyFingerprint` or `depEdges` (`src/model.rs`). + +`serve` has no in-memory cache of parsed entries. Every request re-reads both entries from disk, +parses each into a `serde_json::Value`, re-inserts every label into a `BTreeMap` +(three `String`s per node), and `impacted_targets` then allocates two more strings per node via +`TargetHash::identity` to compare them (`src/server.rs` `get_hashes_locked`, `src/model.rs` +`impacted_targets`). + +## Measurements + +Synthetic hash files with monorepo-shaped labels (about 90 bytes each, 5 source files, 2 rules and +1 generated file per package), 30 changed nodes, release build, this container. + +Size per entry: + +| Entries | JSON | JSON gzip -6 | Columnar estimate, two 32-byte hashes | Columnar, 16-byte hashes | +|---:|---:|---:|---:|---:| +| 100k | 21.2 MB (212 B/node) | 8.4 MB | 6.6 MB (66 B/node) | 3.4 MB | +| 1M | 212 MB | 84 MB | 66 MB | 34 MB | +| 2.8M (extrapolated) | 595 MB | 235 MB | 185 MB | 95 MB | + +Tango's 2.8M-node TGB was 52 MB with 8-byte hashes and about 86 MB at its production 20-byte +stride. bazel-diff would carry two SHA-256 digests per node, so 97% of a bazel-diff columnar entry +is the two hash columns and the size win over JSON is 3.2x, not 10x. Against a gzipped JSON entry +the win is only 1.27x. Size alone does not justify a new format. + +`depEdges` (with `--trackDeps`) is where JSON is wasteful: 77 bytes per edge, about 190 B per node +on top of the hashes. gzip already takes that to 4 B per edge; a delta-coded CSR column would be +under 1 B per edge. + +Time per compare, 1M entries (one `get-impacted-targets`, equivalent to one `serve` request on a +local cache hit): + +| Phase | Today | Notes | +|---|---:|---| +| Read both files | 0.25 s | 424 MB | +| Parse both files | 6.3 s | `serde_json::Value` then re-insert into `BTreeMap` | +| Compare, `impacted_targets` | 0.75 s | two `identity()` string allocations per node | +| Compare, same loop without allocations | 0.30 s | `&str` field comparison | +| Compare, bisecting byte compare over one aligned run | 0.022 s | what the columnar format enables | +| Peak RSS | 1.46 GB | | + +The compare-only speedup (0.75 s to 0.022 s, 34x; 14x against the allocation-free loop) matches +Tango's claim. The larger prize is the parse: with a format that can be compared without decoding, +the 6.3 s and 1.5 GB per request disappear. At 2.8M nodes today's path is roughly 18 s and 4 GB per +request pair, which is the point at which this stops being academic. + +## When to build it + +Any of these on a production `serve` deployment: + +* Local-disk entries above about 100 MB, or S3 or Nexus backfills of an entry taking longer than + the compare they unblock. +* `profile=true` reporting `cacheReadMillis` in the seconds, or request latency on a warm cache + dominated by parsing rather than `bazel query`. +* Memory limits forcing fewer concurrent requests than CPU allows. + +Below those thresholds, the cheaper steps in the next section give most of the benefit. + +## Cheaper steps first (no format change) + +In order of payoff per line of code: + +1. **Keep parsed entries in memory in `serve`.** Nearly every request shares a `from` revision + (the main branch). A small LRU of `Arc` keyed by cache key removes one of the two + parses from almost every request and both from repeated queries. Bound it by entry count or + bytes, alongside the existing disk pruning. +2. **Parse directly.** Deserialize into the final map with a `serde` visitor instead of building a + `Value` and re-inserting, and store hashes as `[u8; 32]` rather than hex `String`s. This also + halves per-node memory. +3. **Compare without allocating.** `impacted_targets` and `impacted_types` can compare the three + fields by reference instead of formatting `identity()` strings. Measured 2.5x on the compare. +4. **Compress the shared tier.** zstd or gzip the bytes written to S3 or Nexus (2.5x on hashes, + 18x on `depEdges`). The on-disk local tier can stay plain so `cache_entries` sizing and + `/metrics` keep working. Mark compressed objects by extension (`.json.zst`) so mixed fleets + keep reading. + +Steps 1 through 3 are independent of the binary format and remain useful after it lands, because +the binary reader still needs a label-level view for `explain`, distances and module changes. + +## Design: `BDH1`, a columnar hash-file container + +Modelled on TGB v3 but reduced to what bazel-diff stores. Not a graph format: it has no attributes, +tags or rule types beyond the three-valued `kind`. + +### Ordering + +Nodes are sorted by label **string** (the order `BTreeMap` already produces and the +order `generate-hashes` writes). Tango sorts by the `(package, name)` tuple to keep its package +column monotonic; bazel-diff has no package column, so string order is simpler and avoids the +mispairing bug Tango documents in §6.1. Both files in a compare must use the same order, which the +reader checks via the header. + +### Container + +``` +Header (64 B) | column blobs | column directory | footer (16 B) +``` + +Header: magic `BDH1`, format version, flags, `nodeCount`, `edgeCount`, `blockSize`, `hashBytes` +(32, or 16 if truncation is ever accepted), CRC32C. Footer: directory offset, directory CRC32C, +magic. Directory: per column `id, codec (0 raw, 1 zstd), offset, compressedSize, rawSize`. +Unknown column ids are skipped so columns can be added without a version bump. `crc32fast` is +already in `Cargo.lock`; `zstd` and `xxhash-rust` would be new crates and must be added to the +`bd_crates` universe in `MODULE.bazel` as well as `Cargo.toml`. + +### Columns + +| Id | Column | Codec | Contents | +|---:|---|---|---| +| 1 | `LABEL_DICT` | zstd | Front-coded sorted labels, restart point every 1024 entries | +| 2 | `KIND` | zstd | Per node: 0 none, 1 SourceFile, 2 GeneratedFile, 3 Rule, 4+ dict id into `KIND_DICT` | +| 3 | `KIND_DICT` | zstd | Rare kinds only | +| 4 | `HASH` | raw | Per node: `hashBytes` bytes of the transitive hash | +| 5 | `DIRECT_HASH` | raw | Per node: `hashBytes` bytes of the direct hash | +| 6 | `BLOCK_DIGEST` | raw | Per block: xxhash64 over the block's NUL-separated label bytes | +| 7 | `BLOCK_START` | zstd | Per block: uvarint delta of first node index | +| 8 | `DEG` | zstd | Per node: uvarint out-degree (only with `--trackDeps`) | +| 9 | `DEPS` | zstd | Delta-coded dep node indices: first dep zigzag-relative to the node, rest as gaps | +| 10 | `DEP_EXTERNAL` | zstd | Deps whose label is not a node, as dict ids (bazel-diff dep edges may name labels outside the hash map) | +| 11 | `META` | zstd | Length-prefixed `moduleGraphJson`, `dependencyFingerprint` | + +`HASH` and `DIRECT_HASH` are two columns, not one interleaved 64-byte stride, so the common +"did anything change" question touches only `HASH`; `DIRECT_HASH` is read for the changed nodes +when classifying Direct versus Indirect impact. + +Block boundaries are content-defined exactly as in TGB (cut after node `i` when +`xxhash64(label_i) & (blockSize-1) == 0`, forced cut at `8 * blockSize`), so an inserted label +disturbs one block. Block size 512. + +### Compare + +1. **Align.** Walk both `BLOCK_DIGEST` arrays; equal digests align a block without decoding + labels. Decode and merge-join labels only in unmatched blocks. Output: aligned runs plus inserted + and deleted node indices. Deleted nodes are never impacted (as today); inserted ones are Direct. +2. **Hash compare.** For each aligned run one slice equality over `HASH`; bisect on mismatch + (`O(k log(n/k))` for `k` changes). Nodes whose `HASH` differs are impacted; compare their + `DIRECT_HASH` bytes to classify Direct versus Indirect. Also compare `KIND` for the run, since + today a kind change with equal hashes counts as impacted (`TargetHash::identity` includes kind). +3. **Decode labels once per changed node** through the dictionary restart table, then apply the + existing `kind_rank` ordering, `targetType` filter and `//external:` exclusion. +4. **Distances and `explain`** need deps for changed nodes only: build the reverse CSR from `DEPS` + (three counting-sort passes as in TGB §6.4) and run the existing BFS on indices, or fall back to + materializing `HashFileData` for the first implementation. + +Phase 2 is the measured 0.022 s on 1M nodes; phase 1 is a few block decodes for a typical PR. + +### Integration points + +* `HashFileData::read` and `from_slice` sniff the magic and dispatch, so `get-impacted-targets`, + `explain` and the server accept either format without flags. The binary reader exposes the same + `hashes` view lazily for callers that still need a map (module changes, `explain`), and a direct + compare entry point that `impacted_targets` uses when both sides are binary. +* `serve` writes `.bdh` and falls back to `.json` on read so a rolling upgrade never + misses. The format is folded into the cache fingerprint token the same way the blob-id seeding + token was in #520, so old and new servers never serve each other's entries. +* `generate-hashes` gains `--format json|bdh` (default json, the CLI contract stays byte-identical). +* `/metrics` `cache.sizeBytes` continues to work unchanged; entries are still files. +* `tools/perf/tango_compare.py` already times a "diff only" stage, so the win is measurable + nightly without new harness work. + +### Risks + +* **Hash truncation.** Tango truncates to 20 bytes because SHA-1 is 20 bytes. Truncating SHA-256 to + 16 bytes would nearly halve the entry but changes the collision argument; keep 32 bytes unless + size is the actual problem. +* **Two codecs to keep in lockstep.** Every change to what a hash file carries lands in both the + JSON serializer and the column writer, with round-trip tests between them. +* **Coverage gate.** The repo enforces 90% line coverage; the codec needs property tests + (encode, decode, compare against `impacted_targets` on random pairs) from the first commit. +* **Dep edges that are not nodes.** bazel-diff's `depEdges` can reference labels absent from + `hashes`; TGB assumes a closed graph. Column 10 handles this but the reverse CSR must skip them. From addab77ee7cb91f73735017a7fd9ed04719eb65d Mon Sep 17 00:00:00 2001 From: Maxwell Elliott Date: Fri, 16 Oct 2026 15:23:28 +0000 Subject: [PATCH 2/2] perf(serve,model): parse hash files directly, compare without allocating, cache parsed entries, gzip the shared tier The four interim steps from docs/tgb-binary-cache.md, none of which changes the hash-file format: - model: `HashFileData` deserializes with a serde visitor straight into the final map instead of building a `serde_json::Value` and re-inserting every label. Hashes are a new `Digest` type that keeps a SHA-256 as 32 raw bytes (one table-driven hex pass; anything that is not 64 lowercase hex digits stays text and round-trips unchanged). `TargetHash::same_as` replaces the two formatted `identity()` strings per node, and `filter_and_sort_labels` ranks each label once instead of cloning it inside the sort. - serve: parsed entries are kept in an in-memory LRU bounded by `--cacheMemoryMax` (default 512MB of serialized size, 0 disables), so a request whose revisions were already parsed skips the disk read and the JSON parse. A memory hit still refreshes the on-disk entry's mtime, which `touch` now sets directly instead of rewriting the whole file. `/metrics` reports the tier under `cache.memory`. - serve: entries are published to S3 or Nexus as fast-level gzip under `.json.gz`, 2-3x smaller. A read tries that name first and falls back to the plain `.json` an older release wrote; a read failure stays a one-request miss. The local-disk tier is unchanged. Measured on a synthetic 1M-entry pair, one `get-impacted-targets` (two parses plus the compare) goes from 4.8s and 1.46GB peak RSS to 2.5s and 0.96GB; the compare alone from 0.75s to 0.30s. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Gs1vjHt61po6uHgkYuh68N --- Cargo.lock | 1 + Cargo.toml | 1 + README.md | 19 +- docs/tgb-binary-cache.md | 32 +-- src/explain.rs | 4 +- src/hash.rs | 12 +- src/main.rs | 7 + src/model.rs | 379 ++++++++++++++++++++++++++------ src/module_graph.rs | 4 +- src/server.rs | 460 ++++++++++++++++++++++++++++++++++----- tools/readme_template.md | 17 +- 11 files changed, 777 insertions(+), 159 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 1a040dc4..c8ef0652 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -147,6 +147,7 @@ dependencies = [ "buffa", "buffa-build", "clap", + "flate2", "hex", "mimalloc", "protoc-bin-vendored", diff --git a/Cargo.toml b/Cargo.toml index 30613198..1601e47f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -22,6 +22,7 @@ anyhow = "1.0" attohttpc = { version = "=0.28.5", default-features = false, features = ["basic-auth", "tls-rustls-webpki-roots"] } buffa = "0.9.1" clap = { version = "4.5", features = ["derive", "env"] } +flate2 = "1.1" hex = "0.4" mimalloc = "0.1" rayon = "1.10" diff --git a/README.md b/README.md index f7246873..772a9962 100644 --- a/README.md +++ b/README.md @@ -315,7 +315,7 @@ curl 'http://localhost:8080/metrics' "ready": true, "gitEngine": "subprocess", "trackDeps": false, - "cache": {"directory": "/var/cache/bazel-diff", "remote": "s3://my-bucket/bazel-diff/", "entries": 128, "sizeBytes": 4823913, "sizeHuman": "4.6 MB"}, + "cache": {"directory": "/var/cache/bazel-diff", "remote": "s3://my-bucket/bazel-diff/", "entries": 128, "sizeBytes": 4823913, "sizeHuman": "4.6 MB", "memory": {"entries": 2, "sizeBytes": 75374, "maxBytes": 536870912}}, "jvm": {"usedBytes": 123456789, "maxBytes": 2147483648} } ``` @@ -337,7 +337,10 @@ bazel-diff serve \ With `--s3Bucket` set the cache becomes two-tiered: reads check local disk first and fall back to the bucket (backfilling local disk on a hit), and every generated entry is published to both, so -any replica can serve a revision another replica already hashed. Credentials and region resolve +any replica can serve a revision another replica already hashed. Entries are published +gzip-compressed as `.json.gz`, 2-3x smaller than the JSON a replica keeps on local disk; a +read tries the compressed object first and falls back to the plain `.json` an older release +wrote, so a mixed fleet keeps sharing entries during an upgrade. Credentials and region resolve through the standard AWS default provider chains (environment variables, profile, IRSA web identity on EKS, IMDS), or pin the region with `--s3Region`. `--s3Endpoint` plus `--s3ForcePathStyle` point the client at an S3-compatible store (MinIO, LocalStack) for local @@ -369,8 +372,9 @@ bazel-diff serve \ --nexusUsername bazel-diff-svc ``` -Entries live at `/repository///.json` and are read -and written with plain `GET`/`PUT` against Nexus's raw content API. Include any context path Nexus +Entries live at `/repository///.json.gz` +(gzip-compressed, with a fallback read of the plain `.json` an older release wrote) and are +read and written with plain `GET`/`PUT` against Nexus's raw content API. Include any context path Nexus is served under in `--nexusUrl` (e.g. `https://example.com/nexus`). * **Repository setup.** Create a *raw (hosted)* repository and set its *Deployment policy* to @@ -429,6 +433,11 @@ Notes and operational guidance: cache is never pruned (the previous behavior). The `--cacheMax*` flags always bound the local-disk tier only; the shared S3 or Nexus tier (see above) manages its own retention via a bucket lifecycle policy or a Nexus cleanup policy. +* Parsed entries are also kept in memory, least recently used out first, within `--cacheMemoryMax` + (default `512MB`; `0` disables the tier). Nearly every request shares its `from` revision with + the previous one, and reading and parsing a large entry costs seconds, so a repeated revision + skips the disk read and the JSON parse entirely. The budget counts each entry's serialized size, + a proxy for its heap footprint, and `/metrics` reports the tier under `cache.memory`. * Query-affecting flags (`--useCquery`, `--fineGrainedHashExternalRepos`, etc.) mirror `generate-hashes`, and are folded into the cache key so a server started with different flags never serves another configuration's cached hashes. @@ -681,6 +690,8 @@ Options: --cachePruneInterval [default: 1h] + --cacheMemoryMax + [default: 512MB] --s3Bucket --s3Prefix diff --git a/docs/tgb-binary-cache.md b/docs/tgb-binary-cache.md index 1bc60b30..0259a4a5 100644 --- a/docs/tgb-binary-cache.md +++ b/docs/tgb-binary-cache.md @@ -78,21 +78,27 @@ Below those thresholds, the cheaper steps in the next section give most of the b ## Cheaper steps first (no format change) -In order of payoff per line of code: +All four landed together with this note. Measured on the same 1M-entry pair, one +`get-impacted-targets` (two parses plus the compare) went from 4.8 s and 1.46 GB peak to 2.5 s and +0.96 GB; the compare alone from 0.75 s to 0.30 s. In `serve`, a request whose revisions are already +parsed skips both parses outright. In order of payoff per line of code: 1. **Keep parsed entries in memory in `serve`.** Nearly every request shares a `from` revision - (the main branch). A small LRU of `Arc` keyed by cache key removes one of the two - parses from almost every request and both from repeated queries. Bound it by entry count or - bytes, alongside the existing disk pruning. -2. **Parse directly.** Deserialize into the final map with a `serde` visitor instead of building a - `Value` and re-inserting, and store hashes as `[u8; 32]` rather than hex `String`s. This also - halves per-node memory. -3. **Compare without allocating.** `impacted_targets` and `impacted_types` can compare the three - fields by reference instead of formatting `identity()` strings. Measured 2.5x on the compare. -4. **Compress the shared tier.** zstd or gzip the bytes written to S3 or Nexus (2.5x on hashes, - 18x on `depEdges`). The on-disk local tier can stay plain so `cache_entries` sizing and - `/metrics` keep working. Mark compressed objects by extension (`.json.zst`) so mixed fleets - keep reading. + (the main branch). An LRU of `Arc` keyed by cache key, bounded by + `--cacheMemoryMax` (default 512 MB of serialized size, `0` disables), removes one of the two + parses from almost every request and both from repeated queries. A memory hit still refreshes + the on-disk entry's modification time, so the disk pruner does not expire a hot entry. +2. **Parse directly.** `HashFileData` deserializes with a `serde` visitor straight into the final + map, no intermediate `Value`, and hashes are a `Digest` that holds a SHA-256 as 32 raw bytes + (anything that is not 64 lowercase hex digits stays text and round-trips unchanged). The hex + decode is a single table-driven pass; it was 42% of parse instructions as two passes. +3. **Compare without allocating.** `TargetHash::same_as` compares kind and both digests by value + in place of the two formatted `identity()` strings per node, and `filter_and_sort_labels` + ranks each label once instead of cloning it inside the sort. +4. **Compress the shared tier.** Entries are published to S3 or Nexus as fast-level gzip under + `.json.gz`; a read tries that name first and falls back to the plain `.json` an older + release wrote, so a mixed fleet keeps sharing during an upgrade. The local-disk tier stays plain + so `cache_entries` sizing and `/metrics` are unchanged. Steps 1 through 3 are independent of the binary format and remain useful after it lands, because the binary reader still needs a label-level view for `explain`, distances and module changes. diff --git a/src/explain.rs b/src/explain.rs index 35b25d96..a9993217 100644 --- a/src/explain.rs +++ b/src/explain.rs @@ -727,8 +727,8 @@ mod tests { fn hash(kind: &str, hash: &str, direct: &str) -> TargetHash { TargetHash { kind: kind.to_owned(), - hash: hash.to_owned(), - direct_hash: direct.to_owned(), + hash: hash.into(), + direct_hash: direct.into(), deps: Vec::new(), } } diff --git a/src/hash.rs b/src/hash.rs index 76440f1e..16461ee3 100644 --- a/src/hash.rs +++ b/src/hash.rs @@ -474,8 +474,8 @@ fn label_hash(mut instances: Vec<(String, &'static str, TargetDigest)>) -> (Stri label, TargetHash { kind: kind.to_owned(), - hash: digest_hex(overall), - direct_hash: digest_hex(direct), + hash: overall.into(), + direct_hash: direct.into(), deps, }, ) @@ -810,12 +810,6 @@ fn package_bzl_seeds( Ok(result) } -fn digest_hex(digest: DigestBytes) -> String { - let mut encoded = [0; 64]; - hex::encode_to_slice(digest, &mut encoded).expect("fixed-size hex output"); - String::from_utf8(encoded.to_vec()).expect("hex output is ASCII") -} - struct Sha256Sink<'a>(&'a mut Sha256); impl EncodeSink for Sha256Sink<'_> { @@ -1593,7 +1587,7 @@ mod tests { untracked.direct(b"direct"); untracked.transitive("//dep:a", b"dependency"); assert!(untracked.finish().deps.is_empty()); - assert_eq!(digest_hex([0; 32]).len(), 64); + assert_eq!(crate::model::Digest::from([0; 32]).to_string().len(), 64); } #[test] diff --git a/src/main.rs b/src/main.rs index 0c80a207..bbb703eb 100644 --- a/src/main.rs +++ b/src/main.rs @@ -381,6 +381,9 @@ struct ServeArgs { #[arg(long = "cachePruneInterval", default_value = "1h")] cache_prune_interval: String, + #[arg(long = "cacheMemoryMax", default_value = "512MB")] + cache_memory_max: String, + #[arg(long = "s3Bucket", conflicts_with = "nexus_url")] s3_bucket: Option, @@ -874,6 +877,7 @@ impl ServeArgs { .map(bazel_diff::server::parse_byte_size) .transpose()?, cache_prune_interval: bazel_diff::server::parse_duration(&self.cache_prune_interval)?, + cache_memory_max: bazel_diff::server::parse_byte_size(&self.cache_memory_max)?, remote_cache, s3_bucket: self.s3_bucket.clone(), s3_prefix: self.s3_prefix.clone(), @@ -1273,6 +1277,8 @@ mod tests { "10gb", "--cacheMaxEntries", "500", + "--cacheMemoryMax", + "2gb", "--s3Bucket", "bucket", "--s3Prefix", @@ -1296,6 +1302,7 @@ mod tests { assert_eq!(config.cache_max_size, Some(10 * 1024u64.pow(3))); assert_eq!(config.cache_max_entries, Some(500)); assert_eq!(config.cache_prune_interval, Duration::from_secs(3600)); + assert_eq!(config.cache_memory_max, 2 * 1024u64.pow(3)); assert_eq!( config.remote_cache.as_deref(), Some("s3://bucket/team/repo/") diff --git a/src/model.rs b/src/model.rs index 186b0782..77767172 100644 --- a/src/model.rs +++ b/src/model.rs @@ -1,16 +1,122 @@ use anyhow::{anyhow, bail, Context, Result}; +use serde::de::{self, MapAccess, Visitor}; use serde::ser::{SerializeMap, SerializeStruct}; -use serde::{Deserialize, Serialize, Serializer}; +use serde::{Deserialize, Deserializer, Serialize, Serializer}; use serde_json::{Map, Value}; +use std::borrow::Cow; use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; +use std::fmt; use std::fs; use std::path::Path; +/// A target hash as written in a hash file. +/// +/// `generate-hashes` writes lowercase hex SHA-256 digests. Those are kept as their 32 raw bytes, +/// so a million-entry file costs one allocation per label instead of three and two entries compare +/// in one fixed-width comparison. Any other string (a hand-written fixture, another producer's +/// digest) is kept verbatim. Either way the value round-trips through `Display` unchanged. +#[derive(Clone, Eq, PartialEq)] +pub enum Digest { + Sha256([u8; 32]), + Text(Box), +} + +impl Digest { + pub fn parse(value: &str) -> Self { + match decode_lowercase_hex(value) { + Some(bytes) => Self::Sha256(bytes), + None => Self::Text(value.into()), + } + } +} + +/// Decodes exactly 64 lowercase hex digits in one table-driven pass; anything else is `None`. +/// This runs twice per entry of every hash file read, so it is the parse's hot loop. +fn decode_lowercase_hex(value: &str) -> Option<[u8; 32]> { + const INVALID: u8 = 0xff; + const TABLE: [u8; 256] = { + let mut table = [INVALID; 256]; + let mut digit = 0; + while digit < 10 { + table[b'0' as usize + digit] = digit as u8; + digit += 1; + } + let mut letter = 0; + while letter < 6 { + table[b'a' as usize + letter] = 10 + letter as u8; + letter += 1; + } + table + }; + let hex = value.as_bytes(); + if hex.len() != 64 { + return None; + } + let mut bytes = [0; 32]; + for (byte, pair) in bytes.iter_mut().zip(hex.chunks_exact(2)) { + let (high, low) = (TABLE[pair[0] as usize], TABLE[pair[1] as usize]); + if high == INVALID || low == INVALID { + return None; + } + *byte = (high << 4) | low; + } + Some(bytes) +} + +impl From<[u8; 32]> for Digest { + fn from(bytes: [u8; 32]) -> Self { + Self::Sha256(bytes) + } +} + +impl From<&str> for Digest { + fn from(value: &str) -> Self { + Self::parse(value) + } +} + +impl From for Digest { + fn from(value: String) -> Self { + Self::parse(&value) + } +} + +impl fmt::Display for Digest { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Sha256(bytes) => { + let mut encoded = [0; 64]; + hex::encode_to_slice(bytes, &mut encoded).expect("fixed-size hex output"); + formatter.write_str(std::str::from_utf8(&encoded).expect("hex output is ASCII")) + } + Self::Text(text) => formatter.write_str(text), + } + } +} + +impl fmt::Debug for Digest { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Debug::fmt(&self.to_string(), formatter) + } +} + +impl PartialEq for Digest { + fn eq(&self, other: &str) -> bool { + *self == Self::parse(other) + } +} + +impl PartialEq<&str> for Digest { + fn eq(&self, other: &&str) -> bool { + *self == Self::parse(other) + } +} + #[derive(Clone, Debug, Eq, PartialEq)] pub struct TargetHash { pub kind: String, - pub hash: String, - pub direct_hash: String, + pub hash: Digest, + pub direct_hash: Digest, pub deps: Vec, } @@ -27,12 +133,18 @@ impl TargetHash { } Ok(Self { kind: kind.to_owned(), - hash: hash.to_owned(), - direct_hash: direct_hash.to_owned(), + hash: Digest::parse(hash), + direct_hash: Digest::parse(direct_hash), deps: Vec::new(), }) } + /// Whether `other` records the same state for a target: same kind and same two hashes. This + /// is the whole comparison `get-impacted-targets` makes per label, so it allocates nothing. + pub fn same_as(&self, other: &Self) -> bool { + self.hash == other.hash && self.direct_hash == other.direct_hash && self.kind == other.kind + } + pub fn json_value(&self, include_target_type: bool) -> String { if include_target_type { format!("{}#{}~{}", self.kind, self.hash, self.direct_hash) @@ -40,10 +152,6 @@ impl TargetHash { format!("{}~{}", self.hash, self.direct_hash) } } - - fn identity(&self) -> String { - format!("{}#{}~{}", self.kind, self.hash, self.direct_hash) - } } struct SerializedTargetHash<'a> { @@ -166,6 +274,114 @@ pub struct HashFileData { pub dep_edges: BTreeMap>, } +/// A JSON string borrowed from the input when it has no escapes, copied otherwise. Hash values +/// are parsed and dropped right away, so borrowing saves one allocation per entry. +struct JsonStr<'de>(Cow<'de, str>); + +impl<'de> Deserialize<'de> for JsonStr<'de> { + fn deserialize>(deserializer: D) -> Result { + struct JsonStrVisitor; + + impl<'de> Visitor<'de> for JsonStrVisitor { + type Value = JsonStr<'de>; + + fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str("a string") + } + + fn visit_borrowed_str(self, value: &'de str) -> Result { + Ok(JsonStr(Cow::Borrowed(value))) + } + + fn visit_str(self, value: &str) -> Result { + Ok(JsonStr(Cow::Owned(value.to_owned()))) + } + + fn visit_string(self, value: String) -> Result { + Ok(JsonStr(Cow::Owned(value))) + } + } + + deserializer.deserialize_str(JsonStrVisitor) + } +} + +/// The `{label: "kind#hash~direct"}` object, parsed entry by entry. +struct Hashes(BTreeMap); + +impl<'de> Deserialize<'de> for Hashes { + fn deserialize>(deserializer: D) -> Result { + struct HashesVisitor; + + impl<'de> Visitor<'de> for HashesVisitor { + type Value = Hashes; + + fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str("an object mapping labels to hash strings") + } + + fn visit_map>(self, mut map: A) -> Result { + let mut hashes = BTreeMap::new(); + while let Some((label, value)) = map.next_entry::>()? { + let hash = TargetHash::parse(&value.0).map_err(de::Error::custom)?; + hashes.insert(label, hash); + } + Ok(Hashes(hashes)) + } + } + + deserializer.deserialize_map(HashesVisitor) + } +} + +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +struct Metadata { + module_graph_json: Option, + dependency_fingerprint: Option, + #[serde(default)] + dep_edges: BTreeMap>, +} + +impl<'de> Deserialize<'de> for HashFileData { + fn deserialize>(deserializer: D) -> Result { + struct HashFileVisitor; + + impl<'de> Visitor<'de> for HashFileVisitor { + type Value = HashFileData; + + fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str("a hash file object") + } + + fn visit_map>(self, mut map: A) -> Result { + let mut data = HashFileData::default(); + // Labels always start with `//`, `@` or a repository name, so the two wrapper + // keys cannot collide with one. + while let Some(key) = map.next_key::>()? { + match &*key.0 { + "hashes" => data.hashes = map.next_value::()?.0, + "metadata" => { + let metadata = map.next_value::()?; + data.module_graph_json = metadata.module_graph_json; + data.dependency_fingerprint = metadata.dependency_fingerprint; + data.dep_edges = metadata.dep_edges; + } + label => { + let value = map.next_value::>()?; + let hash = TargetHash::parse(&value.0).map_err(de::Error::custom)?; + data.hashes.insert(label.to_owned(), hash); + } + } + } + Ok(data) + } + } + + deserializer.deserialize_map(HashFileVisitor) + } +} + impl HashFileData { pub fn serialized( &self, @@ -185,49 +401,12 @@ impl HashFileData { Self::from_slice(&bytes) } + /// Parses either hash-file shape: a bare `{label: hash}` object, or `{"hashes": {...}, + /// "metadata": {...}}`. Entries go straight from the JSON tokens into the final map, with no + /// intermediate `serde_json::Value`, which is what keeps a million-entry file to one pass and + /// one allocation per label. pub fn from_slice(bytes: &[u8]) -> Result { - let root: Value = serde_json::from_slice(bytes).context("invalid hash JSON")?; - let object = root - .as_object() - .ok_or_else(|| anyhow!("hash JSON must be an object"))?; - let (hashes_value, metadata) = match (object.get("hashes"), object.get("metadata")) { - (Some(hashes), Some(metadata)) => (hashes, metadata.as_object()), - _ => (&root, None), - }; - let raw_hashes = hashes_value - .as_object() - .ok_or_else(|| anyhow!("hashes must be a JSON object"))?; - let mut hashes = BTreeMap::new(); - for (label, value) in raw_hashes { - hashes.insert( - label.clone(), - TargetHash::parse( - value - .as_str() - .ok_or_else(|| anyhow!("hash for {label} must be a string"))?, - )?, - ); - } - let module_graph_json = metadata - .and_then(|value| value.get("moduleGraphJson")) - .and_then(Value::as_str) - .map(str::to_owned); - let dependency_fingerprint = metadata - .and_then(|value| value.get("dependencyFingerprint")) - .and_then(Value::as_str) - .map(str::to_owned); - let dep_edges = metadata - .and_then(|value| value.get("depEdges")) - .map(|value| serde_json::from_value(value.clone())) - .transpose() - .context("metadata.depEdges must map labels to label arrays")? - .unwrap_or_default(); - Ok(Self { - hashes, - module_graph_json, - dependency_fingerprint, - dep_edges, - }) + serde_json::from_slice(bytes).context("invalid hash JSON") } pub fn to_value(&self, include_target_type: bool, include_deps: bool) -> Value { @@ -326,10 +505,12 @@ pub fn impacted_targets( ) -> Result> { let impacted = to .iter() - .filter_map(|(label, hash)| { - (from.get(label).map(TargetHash::identity) != Some(hash.identity())) - .then_some(label.clone()) + .filter(|(label, hash)| { + !from + .get(*label) + .is_some_and(|previous| previous.same_as(hash)) }) + .map(|(label, _)| label.clone()) .filter(|label| !exclude_external || !label.starts_with("//external:")) .filter_map(|label| match accepts(&label, to, target_types) { Ok(true) => Some(Ok(label)), @@ -356,14 +537,18 @@ pub fn filter_and_sort_labels( Err(error) => Some(Err(error)), }) .collect::>>()?; - impacted.sort_by_key(|label| { - ( - kind_rank(to.get(label).or_else(|| from.get(label))), - label.clone(), - ) - }); - impacted.dedup(); - Ok(impacted) + // Rank once per label and sort the pairs: the rank is a function of the label, so deduping the + // pairs dedups the labels, and nothing is cloned or looked up inside the sort. + let mut ranked = impacted + .drain(..) + .map(|label| { + let rank = kind_rank(to.get(&label).or_else(|| from.get(&label))); + (rank, label) + }) + .collect::>(); + ranked.sort(); + ranked.dedup(); + Ok(ranked.into_iter().map(|(_, label)| label).collect()) } /// Classifies every label whose hash moved between `from` and `to`. Labels @@ -376,7 +561,7 @@ pub fn impacted_types( to.iter() .filter_map(|(label, final_hash)| match from.get(label) { None => Some((label.clone(), ImpactType::Direct)), - Some(starting_hash) if starting_hash.identity() != final_hash.identity() => Some(( + Some(starting_hash) if !starting_hash.same_as(final_hash) => Some(( label.clone(), if starting_hash.direct_hash != final_hash.direct_hash { ImpactType::Direct @@ -487,8 +672,8 @@ mod tests { fn target(kind: &str, hash: &str, direct: &str) -> TargetHash { TargetHash { kind: kind.to_owned(), - hash: hash.to_owned(), - direct_hash: direct.to_owned(), + hash: hash.into(), + direct_hash: direct.into(), deps: Vec::new(), } } @@ -601,6 +786,67 @@ mod tests { ); } + #[test] + fn digests_keep_sha256_as_bytes_and_everything_else_verbatim() { + let hex = "0123456789abcdef".repeat(4); + let digest = Digest::parse(&hex); + assert!(matches!(digest, Digest::Sha256(_))); + assert_eq!(digest.to_string(), hex); + assert_eq!(digest, hex.as_str()); + assert_eq!(format!("{digest:?}"), format!("{hex:?}")); + assert_eq!(Digest::from(hex.clone()), digest); + let bytes = [0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef].repeat(4); + assert_eq!(Digest::from(<[u8; 32]>::try_from(bytes).unwrap()), digest); + + // Uppercase hex, the wrong width, and fixtures stay text and round-trip unchanged. + for text in [ + hex.to_uppercase(), + hex[..63].to_owned(), + "overall".to_owned(), + ] { + let digest = Digest::parse(&text); + assert!(matches!(digest, Digest::Text(_)), "{text}"); + assert_eq!(digest.to_string(), text); + assert_eq!(digest, text.as_str()); + assert_ne!(digest, hex.as_str()); + } + } + + #[test] + fn same_as_compares_kind_and_both_hashes() { + let base = target("Rule", "a", "b"); + assert!(base.same_as(&target("Rule", "a", "b"))); + assert!(!base.same_as(&target("SourceFile", "a", "b"))); + assert!(!base.same_as(&target("Rule", "x", "b"))); + assert!(!base.same_as(&target("Rule", "a", "x"))); + } + + #[test] + fn direct_deserialization_handles_escapes_wrappers_and_real_digests() { + let hex = "0123456789abcdef".repeat(4); + // An escaped label and value take the owned path; the rest borrow from the input. + let bytes = format!(r#"{{"//a:a":"Rule#{hex}~{hex}","//b:\u0062":"Source\u0046ile#x~y"}}"#); + let parsed = HashFileData::from_slice(bytes.as_bytes()).unwrap(); + assert_eq!(parsed.hashes["//a:a"], target("Rule", &hex, &hex)); + assert!(matches!(parsed.hashes["//a:a"].hash, Digest::Sha256(_))); + assert_eq!(parsed.hashes["//b:b"], target("SourceFile", "x", "y")); + assert_eq!( + serde_json::to_value(parsed.serialized(true, false)).unwrap(), + serde_json::json!({"//a:a": format!("Rule#{hex}~{hex}"), "//b:b": "SourceFile#x~y"}) + ); + + // A wrapper without metadata, and metadata before hashes, both read. + let wrapped = HashFileData::from_slice(br#"{"hashes":{"//a:a":"a~b"}}"#).unwrap(); + assert_eq!(wrapped.hashes["//a:a"], target("", "a", "b")); + let reordered = HashFileData::from_slice( + br#"{"metadata":{"dependencyFingerprint":"fp","unknown":1},"hashes":{"//a:a":"a~b"}}"#, + ) + .unwrap(); + assert_eq!(reordered.dependency_fingerprint.as_deref(), Some("fp")); + assert_eq!(reordered.hashes.len(), 1); + assert!(reordered.dep_edges.is_empty()); + } + #[test] fn rejects_malformed_hash_files() { for invalid in [ @@ -609,6 +855,9 @@ mod tests { br#"{"//a:a": 1}"#.as_slice(), br#"{"hashes":[],"metadata":{}}"#.as_slice(), br#"{"hashes":{},"metadata":{"depEdges":[]}}"#.as_slice(), + br#"{"//a:a": "no-tilde"}"#.as_slice(), + br#"{"hashes":{"//a:a": "a~b~c"}}"#.as_slice(), + br#"{"hashes":{},"metadata":{"moduleGraphJson":1}}"#.as_slice(), ] { assert!(HashFileData::from_slice(invalid).is_err()); } diff --git a/src/module_graph.rs b/src/module_graph.rs index f93ac31d..1185b2d0 100644 --- a/src/module_graph.rs +++ b/src/module_graph.rs @@ -185,8 +185,8 @@ mod tests { fn hash(kind: &str, value: &str) -> TargetHash { TargetHash { kind: kind.to_owned(), - hash: value.to_owned(), - direct_hash: value.to_owned(), + hash: value.into(), + direct_hash: value.into(), deps: Vec::new(), } } diff --git a/src/server.rs b/src/server.rs index b7be01a3..2d4d7d82 100644 --- a/src/server.rs +++ b/src/server.rs @@ -4,6 +4,9 @@ use crate::model::{ }; use crate::module_graph::impacted_with_module_changes; use anyhow::{anyhow, bail, Context, Result}; +use flate2::read::GzDecoder; +use flate2::write::GzEncoder; +use flate2::Compression; use rustls_pki_types::pem::PemObject; use rustls_pki_types::CertificateDer; use s3::creds::Credentials; @@ -12,9 +15,10 @@ use s3::{Bucket, Region}; use serde::{Deserialize, Serialize}; use serde_json::{json, Value}; use sha2::{Digest, Sha256}; -use std::collections::{BTreeMap, BTreeSet, HashSet}; +use std::collections::{BTreeMap, BTreeSet, HashSet, VecDeque}; use std::fmt; use std::fs; +use std::io::{Read, Write}; use std::path::{Path, PathBuf}; use std::process::{Command, Stdio}; use std::sync::atomic::{AtomicBool, Ordering}; @@ -39,6 +43,9 @@ pub struct ServerConfig { pub cache_max_entries: Option, pub cache_max_size: Option, pub cache_prune_interval: Duration, + /// Byte budget for parsed entries kept in memory, measured by each entry's serialized size. + /// `0` disables the in-memory tier. + pub cache_memory_max: u64, pub remote_cache: Option, pub s3_bucket: Option, pub s3_prefix: String, @@ -59,6 +66,84 @@ struct State { started: Instant, fingerprint: String, remote: Option, + /// Parsed entries, in front of the local-disk tier. Guarded by its own lock rather than + /// `workspace_lock` so `/metrics` can read its size while a query holds the workspace. + memory: Mutex, +} + +/// The in-memory tier: entries already parsed, most recently used last. +/// +/// Nearly every request shares its `from` revision (the main branch) with the previous one, and +/// reading and parsing a large entry costs seconds, so a hit here is the difference between a +/// query bounded by the compare and one bounded by JSON parsing. Entries are sized by their +/// serialized length, the one cheap proxy for their heap footprint, and the least recently used +/// entry goes first when the budget is exceeded. +struct MemoryCache { + budget: u64, + used: u64, + entries: VecDeque, +} + +struct MemoryEntry { + key: String, + bytes: u64, + data: Arc, +} + +impl MemoryCache { + fn new(budget: u64) -> Self { + Self { + budget, + used: 0, + entries: VecDeque::new(), + } + } + + /// Returns the entry for `key`, marking it most recently used. + fn get(&mut self, key: &str) -> Option> { + let index = self.entries.iter().position(|entry| entry.key == key)?; + let entry = self.entries.remove(index)?; + let data = Arc::clone(&entry.data); + self.entries.push_back(entry); + Some(data) + } + + /// Remembers `data` for `key`, evicting least recently used entries to stay within budget. An + /// entry larger than the whole budget is not kept. + fn insert(&mut self, key: &str, bytes: u64, data: &Arc) { + if bytes > self.budget { + return; + } + if let Some(index) = self.entries.iter().position(|entry| entry.key == key) { + let previous = self.entries.remove(index).expect("index from position"); + self.used -= previous.bytes; + } + while self.used + bytes > self.budget { + let Some(evicted) = self.entries.pop_front() else { + break; + }; + self.used -= evicted.bytes; + } + self.used += bytes; + self.entries.push_back(MemoryEntry { + key: key.to_owned(), + bytes, + data: Arc::clone(data), + }); + } + + fn len(&self) -> usize { + self.entries.len() + } +} + +/// Locks the in-memory tier. A panic while holding it cannot leave the cache inconsistent in a +/// way that matters (it is only a cache), so a poisoned lock is recovered rather than propagated. +fn memory_cache(state: &State) -> std::sync::MutexGuard<'_, MemoryCache> { + state + .memory + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) } /// The shared cache tier behind local disk. Every backend degrades the same way: a failed read is @@ -84,21 +169,87 @@ impl RemoteCache { Ok(None) } + /// Reads the entry for `key`: the compressed object first, then the plain one an older + /// replica (or this one, before compression) may have written. A read failure is a miss and + /// does not fall through, so an outage costs one request per lookup, not two. fn get(&self, key: &str) -> Option> { - match self { - Self::S3(cache) => cache.get(key), - Self::Nexus(cache) => cache.get(key), + let compressed = format!("{key}{COMPRESSED_SUFFIX}"); + match self.fetch(&compressed) { + Fetched::Found(bytes) => match gunzip(&bytes) { + Ok(bytes) => Some(bytes), + Err(error) => { + eprintln!( + "[Warn] shared cache entry {compressed} is not valid gzip (treating as a miss): {error}" + ); + None + } + }, + Fetched::Missing => match self.fetch(&format!("{key}{PLAIN_SUFFIX}")) { + Fetched::Found(bytes) => Some(bytes), + Fetched::Missing | Fetched::Failed => None, + }, + Fetched::Failed => None, } } + /// Publishes `data` for `key` as a gzip-compressed object. Hash entries are repetitive label + /// text and hex, so this is a 2-3x smaller upload and backfill for every replica. fn put(&self, key: &str, data: &[u8]) { + match gzip(data) { + Ok(compressed) => self.store( + &format!("{key}{COMPRESSED_SUFFIX}"), + &compressed, + "application/gzip", + ), + Err(error) => eprintln!( + "[Warn] failed to compress shared cache entry {key} (entry not shared): {error}" + ), + } + } + + fn fetch(&self, name: &str) -> Fetched { + match self { + Self::S3(cache) => cache.get(name), + Self::Nexus(cache) => cache.get(name), + } + } + + fn store(&self, name: &str, data: &[u8], content_type: &str) { match self { - Self::S3(cache) => cache.put(key, data), - Self::Nexus(cache) => cache.put(key, data), + Self::S3(cache) => cache.put(name, data), + Self::Nexus(cache) => cache.put(name, data, content_type), } } } +/// Object name suffixes on the shared tier. New entries are written compressed; the plain suffix +/// is read for entries written before compression existed. +const COMPRESSED_SUFFIX: &str = ".json.gz"; +const PLAIN_SUFFIX: &str = ".json"; + +/// Outcome of one object read on the shared tier. A clean miss and a failure are told apart so a +/// miss on the compressed name can fall back to the plain one while a failure stays a failure. +enum Fetched { + Found(Vec), + Missing, + Failed, +} + +/// Fast-level gzip: the compressed size is within a few percent of the default level on hash +/// entries and the encode is several times quicker, which matters on a cache write that already +/// sits on the critical path of a cold query. +fn gzip(data: &[u8]) -> std::io::Result> { + let mut encoder = GzEncoder::new(Vec::with_capacity(data.len() / 2), Compression::fast()); + encoder.write_all(data)?; + encoder.finish() +} + +fn gunzip(data: &[u8]) -> std::io::Result> { + let mut decoded = Vec::with_capacity(data.len() * 3); + GzDecoder::new(data).read_to_end(&mut decoded)?; + Ok(decoded) +} + struct S3Cache { bucket: Box, prefix: String, @@ -131,8 +282,8 @@ impl S3Cache { }) } - fn object_key(&self, key: &str) -> String { - format!("{}{key}.json", self.prefix) + fn object_key(&self, name: &str) -> String { + format!("{}{name}", self.prefix) } /// Reads an entry, degrading to a miss on any failure. @@ -142,29 +293,31 @@ impl S3Cache { /// miss with `404`, `Transfer-Encoding: chunked` and no body, and every HTTP client on this /// stack fails reading that absent body -- `attohttpc: Io Error: unexpected end of file`. A GET /// miss carries a real `NoSuchKey` body, so it reads back cleanly. - fn get(&self, key: &str) -> Option> { - let failure = match self.bucket.get_object(self.object_key(key)) { - Ok(response) if response.status_code() == 200 => return Some(response.to_vec()), - Ok(response) if response.status_code() == 404 => return None, + fn get(&self, name: &str) -> Fetched { + let failure = match self.bucket.get_object(self.object_key(name)) { + Ok(response) if response.status_code() == 200 => { + return Fetched::Found(response.to_vec()) + } + Ok(response) if response.status_code() == 404 => return Fetched::Missing, Ok(response) => http_failure(&response), Err(error) => error.to_string(), }; eprintln!( "[Warn] S3 cache read of {} failed (treating as a miss): {failure}", - self.object_key(key) + self.object_key(name) ); - None + Fetched::Failed } - fn put(&self, key: &str, data: &[u8]) { - let failure = match self.bucket.put_object(self.object_key(key), data) { + fn put(&self, name: &str, data: &[u8]) { + let failure = match self.bucket.put_object(self.object_key(name), data) { Ok(response) if (200..300).contains(&response.status_code()) => return, Ok(response) => http_failure(&response), Err(error) => error.to_string(), }; eprintln!( "[Warn] S3 cache write of {} failed (entry not shared): {failure}", - self.object_key(key) + self.object_key(name) ); } } @@ -212,7 +365,7 @@ impl fmt::Debug for NexusCredentials { /// Cache entries stored as components of a Sonatype Nexus Repository *raw* hosted repository, /// addressed through Nexus's plain HTTP content API: `GET`/`PUT` on -/// `/repository///.json`. +/// `/repository///.json.gz`. struct NexusCache { session: attohttpc::Session, base: Url, @@ -249,12 +402,12 @@ impl NexusCache { }) } - fn object_url(&self, key: &str) -> Url { + fn object_url(&self, name: &str) -> Url { let mut url = self.base.clone(); url.path_segments_mut() .expect("validated by nexus_base_url") .pop_if_empty() - .push(&format!("{key}.json")); + .push(name); url } @@ -269,28 +422,30 @@ impl NexusCache { /// Reads an entry, degrading to a miss on any failure. Nexus answers a missing component with /// a plain `404`. - fn get(&self, key: &str) -> Option> { - let url = self.object_url(key); + fn get(&self, name: &str) -> Fetched { + let url = self.object_url(name); let failure = match self.authorized(self.session.get(url.as_str())).send() { Ok(response) if response.status() == attohttpc::StatusCode::OK => { match response.bytes() { - Ok(bytes) => return Some(bytes), + Ok(bytes) => return Fetched::Found(bytes), Err(error) => error.to_string(), } } - Ok(response) if response.status() == attohttpc::StatusCode::NOT_FOUND => return None, + Ok(response) if response.status() == attohttpc::StatusCode::NOT_FOUND => { + return Fetched::Missing + } Ok(response) => nexus_http_failure(response), Err(error) => error.to_string(), }; eprintln!("[Warn] Nexus cache read of {url} failed (treating as a miss): {failure}"); - None + Fetched::Failed } - fn put(&self, key: &str, data: &[u8]) { - let url = self.object_url(key); + fn put(&self, name: &str, data: &[u8], content_type: &str) { + let url = self.object_url(name); let failure = match self .authorized(self.session.put(url.as_str())) - .header(attohttpc::header::CONTENT_TYPE, "application/json") + .header(attohttpc::header::CONTENT_TYPE, content_type) .bytes(data) .send() { @@ -454,6 +609,7 @@ pub fn serve(config: ServerConfig) -> Result<()> { let fingerprint = configuration_fingerprint(&config); let remote = RemoteCache::new(&config)?; let state = Arc::new(State { + memory: Mutex::new(MemoryCache::new(config.cache_memory_max)), config, ready: AtomicBool::new(false), workspace_lock: Mutex::new(()), @@ -808,14 +964,14 @@ struct GenerationBreakdown { /// time went. A hit carries only `cache_read_millis` (reading and deserialising the entry is the /// whole cost of a hit); a miss carries only `generation`. struct Retrieval { - data: HashFileData, + data: Arc, cache_hit: bool, cache_read_millis: Option, generation: Option, } impl Retrieval { - fn hit(data: HashFileData, read_started: Instant) -> Self { + fn hit(data: Arc, read_started: Instant) -> Self { Self { data, cache_hit: true, @@ -863,9 +1019,29 @@ fn get_hashes_locked( let key = cache_key(state, &tree, modified); let path = state.config.cache_dir.join(format!("{key}.json")); let mut current_dependency_fingerprint = None; + // Bound first so the tier's lock is released before the (possibly slow) servability check. + let remembered = memory_cache(state).get(&key); + if let Some(data) = remembered { + let read_started = Instant::now(); + if cache_entry_is_servable( + state, + sha, + "memory", + &key, + &data, + &mut current_dependency_fingerprint, + )? { + // Still counts as a use of the on-disk entry, so the pruner does not expire it out + // from under an entry that is hot in memory. + touch(&path); + return Ok(Retrieval::hit(data, read_started)); + } + } if path.is_file() { let read_started = Instant::now(); - let data = HashFileData::read(&path)?; + let bytes = fs::read(&path) + .with_context(|| format!("failed to read hash file {}", path.display()))?; + let data = Arc::new(HashFileData::from_slice(&bytes)?); if cache_entry_is_servable( state, sha, @@ -875,13 +1051,14 @@ fn get_hashes_locked( &mut current_dependency_fingerprint, )? { touch(&path); + memory_cache(state).insert(&key, bytes.len() as u64, &data); return Ok(Retrieval::hit(data, read_started)); } } if let Some(remote) = &state.remote { let read_started = Instant::now(); if let Some(bytes) = remote.get(&key) { - let data = HashFileData::from_slice(&bytes)?; + let data = Arc::new(HashFileData::from_slice(&bytes)?); if cache_entry_is_servable( state, sha, @@ -891,6 +1068,7 @@ fn get_hashes_locked( &mut current_dependency_fingerprint, )? { fs::write(&path, &bytes)?; + memory_cache(state).insert(&key, bytes.len() as u64, &data); return Ok(Retrieval::hit(data, read_started)); } } @@ -940,12 +1118,13 @@ fn get_hashes_locked( let write_started = Instant::now(); let bytes = serde_json::to_vec(&data.serialized(true, state.config.track_deps))?; let temporary = state.config.cache_dir.join(format!("{key}.tmp")); - fs::write(&temporary, bytes)?; + fs::write(&temporary, &bytes)?; fs::rename(temporary, path)?; if let Some(remote) = &state.remote { - let bytes = fs::read(state.config.cache_dir.join(format!("{key}.json")))?; remote.put(&key, &bytes); } + let data = Arc::new(data); + memory_cache(state).insert(&key, bytes.len() as u64, &data); let generation = GenerationBreakdown { checkout_millis, bazel_query_millis: timings.bazel_query_millis, @@ -1315,6 +1494,10 @@ fn metrics(state: &State) -> Value { let entries = cache_entries(&state.config.cache_dir); let total = entries.iter().map(|entry| entry.size).sum::(); let (used, max) = memory_usage(); + let (memory_entries, memory_used, memory_max) = { + let memory = memory_cache(state); + (memory.len(), memory.used, memory.budget) + }; json!({ "version": env!("CARGO_PKG_VERSION"), "uptimeSeconds": state.started.elapsed().as_secs(), @@ -1328,6 +1511,11 @@ fn metrics(state: &State) -> Value { "entries": entries.len(), "sizeBytes": total, "sizeHuman": human_bytes(total), + "memory": { + "entries": memory_entries, + "sizeBytes": memory_used, + "maxBytes": memory_max, + }, }, "jvm": {"usedBytes": used, "maxBytes": max} }) @@ -1391,10 +1579,10 @@ fn prune_cache(state: &State) -> Result<()> { } fn touch(path: &Path) { - // Rewriting the same bytes is portable and gives pruning a genuine - // last-use timestamp without another platform-specific dependency. - if let Ok(bytes) = fs::read(path) { - let _ = fs::write(path, bytes); + // Refreshes the modification time pruning reads as last use. Opening for write without + // truncating leaves the bytes alone; rewriting them cost a full copy of the entry per hit. + if let Ok(file) = fs::OpenOptions::new().write(true).open(path) { + let _ = file.set_modified(SystemTime::now()); } } @@ -1495,6 +1683,7 @@ mod tests { cache_max_entries: None, cache_max_size: None, cache_prune_interval: Duration::from_secs(60), + cache_memory_max: 64 * 1024 * 1024, remote_cache: None, s3_bucket: None, s3_prefix: String::new(), @@ -1512,8 +1701,8 @@ mod tests { fn hash(kind: &str, value: &str, direct: &str) -> TargetHash { TargetHash { kind: kind.to_owned(), - hash: value.to_owned(), - direct_hash: direct.to_owned(), + hash: value.into(), + direct_hash: direct.into(), deps: Vec::new(), } } @@ -1567,6 +1756,7 @@ mod tests { config.dependency_fingerprint = true; let fingerprint = configuration_fingerprint(&config); Arc::new(State { + memory: Mutex::new(MemoryCache::new(config.cache_memory_max)), config, ready: AtomicBool::new(true), workspace_lock: Mutex::new(()), @@ -1771,12 +1961,12 @@ mod tests { fn remote_cache_operations_use_normalized_key_and_succeed() { let (remote, requests, handle) = remote_cache_with_responses( "/team/repo/", - vec![(200, b"hello".to_vec()), (200, Vec::new())], + vec![(200, gzip(b"hello").unwrap()), (200, Vec::new())], ); let RemoteCache::S3(s3) = &remote else { panic!("expected the S3 backend"); }; - assert_eq!(s3.object_key("sha.fp"), "team/repo/sha.fp.json"); + assert_eq!(s3.object_key("sha.fp.json.gz"), "team/repo/sha.fp.json.gz"); assert_eq!(remote.get("sha.fp"), Some(b"hello".to_vec())); remote.put("sha.fp", b"data"); let captured = (0..2) @@ -1785,7 +1975,52 @@ mod tests { handle.join().unwrap(); assert!(captured .iter() - .all(|request| request.url.contains("/bucket/team/repo/sha.fp.json"))); + .all(|request| request.url.contains("/bucket/team/repo/sha.fp.json.gz"))); + } + + #[test] + fn remote_cache_falls_back_to_plain_entries_and_rejects_corrupt_compressed_ones() { + // A replica that predates compression wrote `.json`: the compressed name misses + // cleanly and the plain one is read. A compressed object that does not decode is a miss, + // with no fallback (the plain name would be a stale duplicate at best). + let (remote, requests, handle) = remote_cache_with_responses( + "", + vec![ + (404, b"NoSuchKey".to_vec()), + (200, b"plain".to_vec()), + (200, b"not gzip".to_vec()), + ], + ); + assert_eq!(remote.get("legacy"), Some(b"plain".to_vec())); + assert_eq!(remote.get("corrupt"), None); + let captured = (0..3) + .map(|_| requests.recv_timeout(Duration::from_secs(5)).unwrap()) + .collect::>(); + handle.join().unwrap(); + assert!(captured[0].url.ends_with("/legacy.json.gz")); + assert!(captured[1].url.ends_with("/legacy.json")); + assert!(captured[2].url.ends_with("/corrupt.json.gz")); + assert!(requests.recv_timeout(Duration::from_millis(200)).is_err()); + } + + #[test] + fn gzip_round_trips_and_shrinks_hash_entries() { + let entry = (0..2000) + .map(|index| { + format!( + "\"//pkg/{}:target_{index}\":\"Rule#{:064x}~{:064x}\"", + index / 8, + index, + index * 7 + ) + }) + .collect::>() + .join(","); + let entry = format!("{{{entry}}}").into_bytes(); + let compressed = gzip(&entry).unwrap(); + assert!(compressed.len() * 2 < entry.len()); + assert_eq!(gunzip(&compressed).unwrap(), entry); + assert!(gunzip(b"not gzip").is_err()); } #[test] @@ -1793,6 +2028,7 @@ mod tests { let (remote, requests, handle) = remote_cache_with_responses( "", vec![ + (404, b"NoSuchKey".to_vec()), (404, b"NoSuchKey".to_vec()), (500, b"failure".to_vec()), (500, Vec::new()), @@ -1801,10 +2037,11 @@ mod tests { assert_eq!(remote.get("missing"), None); assert_eq!(remote.get("failure"), None); remote.put("failure", b"data"); - // Exactly one request per call. `rust-s3`'s `fail-on-err` feature is off precisely so a - // non-2xx stays a response: as an error it would enter the crate's `retry!`, and every - // miss would cost a second request and a one-second backoff sleep. - for _ in 0..3 { + // A clean miss probes the compressed and then the plain name; a failure is one request + // and no fallback. `rust-s3`'s `fail-on-err` feature is off precisely so a non-2xx stays + // a response: as an error it would enter the crate's `retry!`, and every miss would cost + // a second request and a one-second backoff sleep. + for _ in 0..4 { requests.recv_timeout(Duration::from_secs(5)).unwrap(); } assert!(requests.recv_timeout(Duration::from_millis(200)).is_err()); @@ -1939,8 +2176,8 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ panic!("expected the Nexus backend"); }; assert_eq!( - cache.object_url("sha.fp").as_str(), - "https://nexus.example.com/repository/cache/team/sha.fp.json" + cache.object_url("sha.fp.json.gz").as_str(), + "https://nexus.example.com/repository/cache/team/sha.fp.json.gz" ); config.nexus_repository = Some("a/b".to_owned()); assert!(RemoteCache::new(&config).is_err()); @@ -1955,7 +2192,8 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ password: "s3cret".to_owned(), }), vec![ - (200, Vec::new(), b"hello".to_vec()), + (200, Vec::new(), gzip(b"hello").unwrap()), + (404, Vec::new(), Vec::new()), (404, Vec::new(), Vec::new()), (201, Vec::new(), Vec::new()), ], @@ -1965,21 +2203,28 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ remote.put("sha.fp", b"data"); handle.join().unwrap(); let captured = requests.try_iter().collect::>(); - assert_eq!(captured.len(), 3); + assert_eq!(captured.len(), 4); let methods = captured .iter() .map(|r| r.method.clone()) .collect::>(); - assert_eq!(methods, [Method::Get, Method::Get, Method::Put]); + assert_eq!( + methods, + [Method::Get, Method::Get, Method::Get, Method::Put] + ); assert_eq!( captured[0].url, - "/nexus/repository/bazel-diff/team/repo/sha.fp.json" + "/nexus/repository/bazel-diff/team/repo/sha.fp.json.gz" ); assert_eq!( captured[1].url, + "/nexus/repository/bazel-diff/team/repo/missing.json.gz" + ); + assert_eq!( + captured[2].url, "/nexus/repository/bazel-diff/team/repo/missing.json" ); - assert_eq!(captured[2].url, captured[0].url); + assert_eq!(captured[3].url, captured[0].url); for request in &captured { // base64("svc:s3cret") assert_eq!( @@ -1988,10 +2233,10 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ ); } assert_eq!( - captured[2].content_type.as_deref(), - Some("application/json") + captured[3].content_type.as_deref(), + Some("application/gzip") ); - assert_eq!(captured[2].body, b"data"); + assert_eq!(gunzip(&captured[3].body).unwrap(), b"data"); } #[test] @@ -2151,6 +2396,7 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ config.cache_dir = cache.path().to_path_buf(); let fingerprint = configuration_fingerprint(&config); Arc::new(State { + memory: Mutex::new(MemoryCache::new(config.cache_memory_max)), config, ready: AtomicBool::new(false), workspace_lock: Mutex::new(()), @@ -2219,6 +2465,7 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ let mut bad_config = state.config.clone(); bad_config.git_path = PathBuf::from("/definitely/missing/git"); let bad = State { + memory: Mutex::new(MemoryCache::new(bad_config.cache_memory_max)), config: bad_config, ready: AtomicBool::new(true), workspace_lock: Mutex::new(()), @@ -2281,6 +2528,7 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ let mut bad_config = state.config.clone(); bad_config.git_path = PathBuf::from("/missing/git"); let bad_state = State { + memory: Mutex::new(MemoryCache::new(bad_config.cache_memory_max)), config: bad_config, ready: AtomicBool::new(true), workspace_lock: Mutex::new(()), @@ -2328,6 +2576,87 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ assert!(fs::metadata(path).unwrap().modified().unwrap() >= before); } + #[test] + fn memory_cache_evicts_least_recently_used_within_budget() { + let entry = |label: &str| { + Arc::new(HashFileData { + hashes: std::collections::BTreeMap::from([( + label.to_owned(), + hash("Rule", "a", "b"), + )]), + ..Default::default() + }) + }; + let mut cache = MemoryCache::new(100); + cache.insert("a", 40, &entry("//a")); + cache.insert("b", 40, &entry("//b")); + assert_eq!(cache.len(), 2); + // Reading `a` makes `b` the least recently used, so `c` evicts `b`. + assert!(cache.get("a").unwrap().hashes.contains_key("//a")); + cache.insert("c", 40, &entry("//c")); + assert_eq!(cache.len(), 2); + assert!(cache.get("b").is_none()); + assert!(cache.get("a").is_some()); + assert!(cache.get("c").is_some()); + assert_eq!(cache.used, 80); + // Re-inserting a key replaces it instead of double counting. + cache.insert("c", 50, &entry("//c2")); + assert_eq!(cache.len(), 2); + assert_eq!(cache.used, 90); + assert!(cache.get("c").unwrap().hashes.contains_key("//c2")); + // An entry over the whole budget is never kept; one that needs the whole budget evicts + // everything else. + cache.insert("huge", 101, &entry("//huge")); + assert!(cache.get("huge").is_none()); + cache.insert("whole", 100, &entry("//whole")); + assert_eq!(cache.len(), 1); + assert_eq!(cache.used, 100); + // A zero budget disables the tier. + let mut disabled = MemoryCache::new(0); + disabled.insert("a", 1, &entry("//a")); + assert_eq!(disabled.len(), 0); + assert!(disabled.get("a").is_none()); + } + + #[test] + fn parsed_entries_are_served_from_memory_after_the_first_read() { + let (repo, _, sha) = initialize_git_repo(); + let cache = tempfile::tempdir().unwrap(); + let state = state_for_repo(repo.path(), cache.path(), false); + let data = HashFileData { + hashes: std::collections::BTreeMap::from([( + "//app:lib".into(), + hash("Rule", "overall", "direct"), + )]), + dependency_fingerprint: state + .config + .hash_options + .bazel + .dependency_fingerprint() + .ok(), + ..Default::default() + }; + let key = cache_key_for_commit(&state, &sha); + let path = cache.path().join(format!("{key}.json")); + write_hashes(&path, &data, false); + assert_eq!(metrics(&state)["cache"]["memory"]["entries"], 0); + + let first = get_hashes_locked(&state, &sha, &BTreeSet::new()).unwrap(); + assert!(first.cache_hit); + let memory = metrics(&state)["cache"]["memory"].clone(); + assert_eq!(memory["entries"], 1); + assert_eq!(memory["sizeBytes"], fs::metadata(&path).unwrap().len()); + assert_eq!(memory["maxBytes"], state.config.cache_memory_max); + + // The on-disk entry is gone (pruned, say); the parsed one still serves, and both + // retrievals share the same allocation. + fs::remove_file(&path).unwrap(); + let second = get_hashes_locked(&state, &sha, &BTreeSet::new()).unwrap(); + assert!(second.cache_hit); + assert!(Arc::ptr_eq(&first.data, &second.data)); + assert_eq!(second.data.hashes["//app:lib"].hash, "overall"); + } + #[test] fn remote_cache_hit_is_backfilled_and_next_read_is_local() { let (repo, _, sha) = initialize_git_repo(); @@ -2347,9 +2676,11 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ ..Default::default() }; let bytes = serde_json::to_vec(&data.serialized(true, false)).unwrap(); - let (remote, requests, handle) = remote_cache_with_responses("", vec![(200, bytes)]); + let (remote, requests, handle) = + remote_cache_with_responses("", vec![(200, gzip(&bytes).unwrap())]); let fingerprint = configuration_fingerprint(&config); let state = Arc::new(State { + memory: Mutex::new(MemoryCache::new(config.cache_memory_max)), config, ready: AtomicBool::new(true), workspace_lock: Mutex::new(()), @@ -2529,9 +2860,17 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ ..Default::default() }; let bytes = serde_json::to_vec(&data.serialized(true, false)).unwrap(); - let (remote, requests, handle) = remote_cache_with_responses("", vec![(200, bytes)]); + // Written by a replica from before compression: only the plain object exists. + let (remote, requests, handle) = remote_cache_with_responses( + "", + vec![ + (404, b"NoSuchKey".to_vec()), + (200, bytes), + ], + ); let fingerprint = configuration_fingerprint(&config); let state = Arc::new(State { + memory: Mutex::new(MemoryCache::new(config.cache_memory_max)), config, ready: AtomicBool::new(true), workspace_lock: Mutex::new(()), @@ -2543,6 +2882,7 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ assert!(retrieval.cache_hit); assert_eq!(retrieval.data.hashes["//app:lib"].hash, "overall"); requests.recv_timeout(Duration::from_secs(5)).unwrap(); + requests.recv_timeout(Duration::from_secs(5)).unwrap(); handle.join().unwrap(); // With the guard on, the same fingerprint-less entry is rejected, and since bazel is @@ -2567,7 +2907,7 @@ nQBeQ4AIl47nDOrFeVnvse8R9IHJ #[test] fn miss_retrievals_serialize_the_generation_breakdown() { let retrieval = Retrieval { - data: HashFileData::default(), + data: Arc::new(HashFileData::default()), cache_hit: false, cache_read_millis: None, generation: Some(GenerationBreakdown { diff --git a/tools/readme_template.md b/tools/readme_template.md index 7144df6d..48fc18e0 100644 --- a/tools/readme_template.md +++ b/tools/readme_template.md @@ -315,7 +315,7 @@ curl 'http://localhost:8080/metrics' "ready": true, "gitEngine": "subprocess", "trackDeps": false, - "cache": {"directory": "/var/cache/bazel-diff", "remote": "s3://my-bucket/bazel-diff/", "entries": 128, "sizeBytes": 4823913, "sizeHuman": "4.6 MB"}, + "cache": {"directory": "/var/cache/bazel-diff", "remote": "s3://my-bucket/bazel-diff/", "entries": 128, "sizeBytes": 4823913, "sizeHuman": "4.6 MB", "memory": {"entries": 2, "sizeBytes": 75374, "maxBytes": 536870912}}, "jvm": {"usedBytes": 123456789, "maxBytes": 2147483648} } ``` @@ -337,7 +337,10 @@ bazel-diff serve \ With `--s3Bucket` set the cache becomes two-tiered: reads check local disk first and fall back to the bucket (backfilling local disk on a hit), and every generated entry is published to both, so -any replica can serve a revision another replica already hashed. Credentials and region resolve +any replica can serve a revision another replica already hashed. Entries are published +gzip-compressed as `.json.gz`, 2-3x smaller than the JSON a replica keeps on local disk; a +read tries the compressed object first and falls back to the plain `.json` an older release +wrote, so a mixed fleet keeps sharing entries during an upgrade. Credentials and region resolve through the standard AWS default provider chains (environment variables, profile, IRSA web identity on EKS, IMDS), or pin the region with `--s3Region`. `--s3Endpoint` plus `--s3ForcePathStyle` point the client at an S3-compatible store (MinIO, LocalStack) for local @@ -369,8 +372,9 @@ bazel-diff serve \ --nexusUsername bazel-diff-svc ``` -Entries live at `/repository///.json` and are read -and written with plain `GET`/`PUT` against Nexus's raw content API. Include any context path Nexus +Entries live at `/repository///.json.gz` +(gzip-compressed, with a fallback read of the plain `.json` an older release wrote) and are +read and written with plain `GET`/`PUT` against Nexus's raw content API. Include any context path Nexus is served under in `--nexusUrl` (e.g. `https://example.com/nexus`). * **Repository setup.** Create a *raw (hosted)* repository and set its *Deployment policy* to @@ -429,6 +433,11 @@ Notes and operational guidance: cache is never pruned (the previous behavior). The `--cacheMax*` flags always bound the local-disk tier only; the shared S3 or Nexus tier (see above) manages its own retention via a bucket lifecycle policy or a Nexus cleanup policy. +* Parsed entries are also kept in memory, least recently used out first, within `--cacheMemoryMax` + (default `512MB`; `0` disables the tier). Nearly every request shares its `from` revision with + the previous one, and reading and parsing a large entry costs seconds, so a repeated revision + skips the disk read and the JSON parse entirely. The budget counts each entry's serialized size, + a proxy for its heap footprint, and `/metrics` reports the tier under `cache.memory`. * Query-affecting flags (`--useCquery`, `--fineGrainedHashExternalRepos`, etc.) mirror `generate-hashes`, and are folded into the cache key so a server started with different flags never serves another configuration's cached hashes.