Add deterministic trace replay benchmark#18
Open
tillkruss wants to merge 5 commits into
Open
Conversation
Add replay.js, a k6 script that fires a fixed corpus of captured requests against a stateless replay endpoint instead of a live WordPress site. Each run executes an identical, reproducible workload (default 100 traces x 100 repeats = 10,000 requests, closed-loop) using per-vu-iterations with a segment-safe trace mapping, so it scales across load generators via k6 execution segments. Add stubs/replay-server.php, a dummy endpoint that fakes the work and returns the metric shape replay.js parses, as a starting point for the real payload executor.
Add TODO.md with a high-level overview of the trace-capture/replay benchmark, component status, remaining work, and a separate PhpBench track for running the Redis fake workload as an isolated micro-benchmark. Make replay.js load profile configurable via SCENARIO (fixed / ramping / constant / shared), with tunable stages, duration and concurrency. The deterministic, segment-safe trace mapping now works uniformly across all executors.
Switch replay.js to the shared Metrics class from lib/metrics.js (same as wp.js and the woo-* scripts) instead of bespoke replay_* metrics, and add an errors Rate to match. Metrics are now parsed from the Object Cache Pro footnote comment in the response body, so the real SUT needs no special handling beyond enabling analytics.footnote. Make stubs/replay-server.php print a fake OCP footnote comment in the exact format the real mu-plugin emits (see k6-metrics.php), with a configurable ?client= so the using-* metrics populate.
Add CAPTURE.md: handoff instructions for building the WordPress request-capture mu-plugin (Redis commands, SQL queries, and per-op timeline), including capture sources, binary-safe encoding rules, race-free file writing, and verification. Add corpus/00000.json and corpus/manifest.json as the canonical, hand-authored examples of the capture format: an ordered php/redis/sql timeline with verbatim (base64) Redis values, pipeline round-trips, full SQL text, and a summary block aligned with the existing metric vocabulary.
Add replay/ executor that replays captured corpus traces against a real Redis:
- config.php: env-driven config (.env supported), client/connection/SQL settings
- redis.php: phpredis/relay wrapper issuing commands via rawCommand, preserving
single/pipeline/multi round-trips
- replayer.php: walks the timeline (php usleep, redis commands with {b64} value
decoding, SQL sleep-or-execute), returns measured metrics; reusable by PhpBench
- index.php: stateless ?id=N HTTP endpoint, preloaded corpus + persistent
connections per worker, emits a measured OCP-style footnote for replay.js
SQL defaults to reproducing timing as a sleep (no DB needed); REPLAY_SQL_MODE=
execute runs queries against MySQL. Credentials come from environment variables
(.env.example added, .env gitignored). Fix the example trace's zero-TTL SETEX to
a plain SET. README documents both the real executor and the no-Redis dummy.
Verified end-to-end against a live Redis via phpredis: writes land, hit/miss
detection works (cold=misses, pre-warmed=hits), pipeline/multi replay, and the
footnote parses via lib/metrics.js.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add replay.js, a k6 script that fires a fixed corpus of captured requests
against a stateless replay endpoint instead of a live WordPress site. Each
run executes an identical, reproducible workload (default 100 traces x 100
repeats = 10,000 requests, closed-loop) using per-vu-iterations with a
segment-safe trace mapping, so it scales across load generators via k6
execution segments.
Add stubs/replay-server.php, a dummy endpoint that fakes the work and returns
the metric shape replay.js parses, as a starting point for the real payload
executor.