Skip to content

Add pulled Bloom summary peer lookup - #1087

Merged
bill-ph merged 8 commits into
mainfrom
codex/pushed-cache-summaries
Aug 18, 2026
Merged

Add pulled Bloom summary peer lookup#1087
bill-ph merged 8 commits into
mainfrom
codex/pushed-cache-summaries

Conversation

@bill-ph

@bill-ph bill-ph commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an opt-in pulled-summary peer lookup mode while keeping
CACHE_PEER_LOOKUP_MODE=probe as the default/current behavior.

In summary mode, each cache proxy maintains a fixed-layout counting Bloom
filter incrementally on cache insertion and eviction. It serves an immutable,
versioned snapshot through GET /cache/summary; receivers pull a deterministic
subset of peers that fits their configured memory budget. Bloom filters are
used only to eliminate definite-negative peers. Every positive or uncovered
candidate is exact-confirmed through bounded parallel /cache/has requests
before one /cache/get is sent to a confirmed holder.

This avoids fleet-wide request fanout while preserving safe origin fallback.
No cache bodies are proactively replicated, and raw URLs, ranges, object paths,
organization identifiers, and cache locators are never included in summaries.

Bounds and lifecycle

  • CACHE_MAX_ENTRIES=1000000: soft/convergent local index target, alongside
    the existing 80% disk target.
  • CACHE_SUMMARY_MEMORY_LIMIT_BYTES=536870912: reserves the local counting
    filter, overlapping snapshots, four bounded pulls, headers/bodies, and the
    retained remote filters. The default fits 415 peer filters.
  • CACHE_PEER_MAX_PROBES=5: request-wide confirmation cap, shared across
    missing blocks.
  • CACHE_MAX_PEER_PROBES_IN_FLIGHT=64: pod-wide non-blocking semaphore; excess
    confirmations skip peer work and use origin rather than queueing.
  • Snapshots are refreshed about every 20 seconds with jitter and have a
    45-second TTL. Pulls use four workers, two-second request deadlines, a
    15-second fair rotating cycle deadline, bounded headers, and cancellation on
    shutdown.
  • Block requests share the five-probe budget and attempt at most two confirmed
    peer block GETs for the entire client request.

When a new peer appears, membership selection is recomputed and a selected peer
is pulled immediately. Until a valid summary arrives, it remains uncovered and
is eligible only for bounded confirmation. A peer outside the memory-selected
subset also remains uncovered. When a peer disappears or is deselected, its
summary is removed and in-flight receipts are revalidated so stale state cannot
be reinserted. Pull failure retains the last valid record only until its
advertised TTL.

See the design and operator runbook
for sizing, false-positive math, failure behavior, and recovery.

Rollout

  1. Deploy this image to every cache-proxy pod while leaving lookup mode at the
    default probe value.
  2. Enable CACHE_PEER_LOOKUP_MODE=summary in non-production first.
  3. During rolling enablement or cold start, absent/incompatible summaries leave
    peers uncovered; confirmation remains capped and requests safely fall back
    to origin.
  4. Validate physical probes per logical lookup against the fleet-aware Bloom
    expectation (N * p, capped at five), confirmed peer GET usefulness, peer
    bytes, origin bytes/latency, summary age/coverage, pull outcomes, resident
    summary memory, and process RSS.
  5. Roll back by restoring CACHE_PEER_LOOKUP_MODE=probe; cache contents and
    layout are unchanged and must not be deleted.

Known limitation: a fully covered Bloom-negative key goes directly to origin,
so a key inserted or entering an in-flight fill after the last snapshot can be
missed until the next pull. This is a locality loss, not a correctness failure.

Validation

  • just test-cache-proxy
  • go test -race ./cmd/cache-proxy/... -count=1
  • just test-unit
  • just lint

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

Test Impact Plan

Deterministic summary of how this PR changes tests, CI runners, and coverage-risk signals.

Summary

Area Added Changed Deleted
Test files 2 1 0
E2E/journey files 0 0 0
Workflow files 0 0 0

Signals

  • Test cases: +48 / -0
  • Assertions: +155 / -0
  • Skips or known failures added: 0
  • Workflow continue-on-error added: 0
  • Workflow path filters added: 0
  • Test commands removed from justfile: 0
  • E2E/journey retry lines added: 0

Coverage risk: neutral or increased

No coverage-reduction warnings detected.

Comment thread cmd/cache-proxy/main.go Fixed
Comment thread cmd/cache-proxy/main.go Fixed
Comment thread cmd/cache-proxy/main.go Fixed
@bill-ph bill-ph changed the title Add pushed cache summary peer lookup Add pulled Bloom summary peer lookup Aug 18, 2026
@bill-ph
bill-ph marked this pull request as ready for review August 18, 2026 17:23
@bill-ph
bill-ph merged commit 951097e into main Aug 18, 2026
32 checks passed
@bill-ph
bill-ph deleted the codex/pushed-cache-summaries branch August 18, 2026 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants