Skip to content

Add repeatable FYRE dataplane scaling benchmarks - #18

Merged
lucarlig merged 1 commit into
user/luca/distributed-load-workersfrom
user/luca/fyre-scaling-benchmarks
Sep 16, 2026
Merged

lucarlig merged 1 commit into
user/luca/distributed-load-workersfrom
user/luca/fyre-scaling-benchmarks

Conversation

@lucarlig

Copy link
Copy Markdown
Contributor

The existing load workflow cannot compare one larger Rust dataplane VM with multiple smaller replicas under the same total CPU and memory, and its local setup cannot demonstrate that the load generator and Fast Time backend have headroom. This adds a repeatable FYRE campaign that provisions isolated infrastructure, finds zero-error capacity, produces raw and Slack-ready reports, and destroys only resources owned by the run.

Depends on #17. This branch is based on its current head (e37240f) so the distributed Locust and load CLI changes are preserved.

Behavior

  • Adds load fyre run|status|destroy and l f r|s|d, with -f/--file and -i/--run-id.
  • Packages pinned Terraform, Compose, workload, telemetry, recovery, and reporting assets under benchmarks/fyre/.
  • Compares baseline, matched vertical/horizontal 2x and 3x allocations, and the vertical 4x extension.
  • Uses MCP 2026-07-28, FastHttpUser, six zero-delay Fast Time tools, balanced direct replica traffic, and exact post-ramp measurement windows.
  • Stops on the first request or worker error, refines capacity to 12.5%, confirms it three times, and caps the matrix at six hours and 32,000 users per scenario.
  • Grows saturated Locust or Fast Time helpers, archives invalidated attempts, reruns the matrix with one validated helper size, and marks headroom failures inconclusive.
  • Captures host, process, container/OOM, CPU/steal, memory/swap, PSI, socket/network, worker, and per-replica telemetry.
  • Downloads artifacts before cleanup, retries destroy, protects manually created VMs through run-owned Terraform state, and uses an eight-hour FYRE expiry backstop.
  • Produces JSON, CSV, and a Slack-ready PNG with RPS, latency, speedup, efficiency, horizontal advantage, RPS/vCPU, variability, and imbalance.

Validation

  • cargo test --all-targets — 426 passed
  • cargo clippy --all-targets -- -D warnings
  • cargo check --all-targets
  • FYRE campaign unit tests — 9 passed, covering fail-fast load boundaries, plateau detection, helper saturation, exact measurement-window filtering, network/OOM pressure, worker exits, replica statistics, and aggregate latency
  • Terraform fmt, provider initialization, and validate with pinned hashicorp-forge/fyre 0.0.3
  • Both Compose files validated with docker compose config
  • Pinned Locust image imported the six-tool FastHttpUser workload successfully
  • Packaged asset inventory checked; Python caches are excluded
  • Provision failure, interruption, and cleanup-retry paths exercised with a fake Terraform provider runner
  • Slack PNG generated and visually checked

The live FYRE campaign remains pending because FYRE credentials are not present in the CLI process environment. No credential value is stored in command arguments, Terraform variables, manifests, reports, or repository state.

Signed-off-by: lucarlig <luca.carlig@ibm.com>
@lucarlig
lucarlig merged commit 32c9cb5 into user/luca/distributed-load-workers Sep 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant