Skip to content

chore(benchmark): add Redis write benchmark and record the 0.18 to 0.19 measurements - #135

Merged
lan17 merged 1 commit into
mainfrom
claude/write-benchmark
Aug 10, 2026
Merged

chore(benchmark): add Redis write benchmark and record the 0.18 to 0.19 measurements#135
lan17 merged 1 commit into
mainfrom
claude/write-benchmark

Conversation

@lan17

@lan17 lan17 commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Summary

Adds pnpm benchmark:redis-write, a maintainer benchmark for the Redis write path, and records the measured server-side cost of the 0.18.0 → 0.19.0 write-protocol change (#131). The 0.19.0 release notes carried an expectation extrapolated from #123's read measurements ("expect ~95% server-side command-time reduction at 1 MiB writes"); this PR replaces that expectation with numbers.

Measured: 0.18.0 (Lua write scripts) vs 0.19.0 (native SET + payload-free stamp)

Three full runs per version, sequential awaited writes via the node-redis adapter against a dedicated local Redis 6.2 container. Server-side cost is INFO commandstats µsec summed over the commands the client dispatches per logical write (0.18.0 tracked: EVALSHA; 0.19.0 tracked: SET + EVALSHA; the EVALSHA entry envelopes script-internal calls), divided by writes. Ranges are min–max across the three runs.

Path Payload 0.18.0 server µs/write 0.19.0 server µs/write Server-side reduction
tracked 100 B 6.8–8.8 6.7–8.2 parity (within noise)
tracked 10 KiB 12.7–15.7 10.5–15.7 parity (within noise)
tracked 100 KiB 24.0–25.7 7.3–10.3 60–69%
tracked 1 MiB 218–241 11.4–18.5 92–95%
untracked 100 B 6.0–7.8 0.5–0.7 ~90%
untracked 10 KiB 12.2–13.4 0.8–1.6 87–94%
untracked 100 KiB 20.5–22.6 0.8–1.5 93–96%
untracked 1 MiB 210–237 2.4–3.0 ~99%

Client-side latency (localhost, RTT-dominated): 1 MiB tracked p50 improved from 772–861 µs to 566–703 µs (~25%); small payloads are roughly flat with overlapping ranges.

Readings:

  • The release-notes expectation is confirmed where it was made: ~95% server-side reduction at 1 MiB tracked writes (observed 92–95%).
  • The new protocol's server cost is nearly payload-size-independent (tracked ~7–18 µs at every size), which is the direct signature of the payload no longer crossing the Redis↔Lua boundary; the old protocol's cost grows roughly linearly with payload.
  • Small tracked payloads (≤10 KiB) are parity within this machine's noise floor: per-run deltas swung from −23% to +33% across runs, i.e. a few µs either way. The protocol's costs there are offset by its savings; neither wins measurably.
  • Untracked writes win at every size, ~90%+ — one native SET versus a payload-carrying script.

Methodology caveats

Single machine (Apple Silicon laptop, Docker Desktop VM) with unrelated containers running; sequential awaited writes (per-op cost, not throughput under pipelined concurrency); Redis 6.2 only; three runs. Treat the small-payload cells as "no measurable difference" rather than precise deltas — the µs-scale numbers there sit at the environment's noise floor. The 100 KiB and 1 MiB signals are stable across runs and large relative to noise.

What ships in this PR

  • scripts/benchmark-redis-write.mjs + pnpm benchmark:redis-write — the same methodology, pointed at the local build, for future regression checks (maintainer tool, not part of the published package, no timing thresholds asserted).
  • A README section documenting how to run it.

…19 measurements

Adds a maintainer benchmark for the write path (server-side cost per
write from INFO commandstats plus client latency percentiles, tracked
and untracked, 100 B to 1 MiB) mirroring the methodology used to
measure the 0.18.0 to 0.19.0 protocol change, and documents it in the
README benchmark section.
@lan17
lan17 merged commit 8bf3a5f into main Aug 10, 2026
5 checks passed
@lan17
lan17 deleted the claude/write-benchmark branch August 10, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant