[psanders]

Two changes for calibration runs, which will be read in the workflow run instead of on the website.

Results in the GitHub run summary

• New command perfkit summary --report <report.json> prints the report as GitHub Markdown. It covers the run id, the target version and commit, the overall verdict and each benchmark's verdict, with the reason when a benchmark is inconclusive, aborted or not run.
  • Capacity benchmarks get the capacity limit (or a lower bound), where the ramp stopped, and a step table: offered rate, placed rate, p99, failed %, and ok or the stop reason.
  • Soak benchmarks get the load, the rules against their limits (memory growth MB/h per process, failed calls %, restarts), and p99 for the first and the last hour.
  • Exit codes: 0 printed, 1 usage error, 2 invalid, unreadable or wrong-schema report.
• run.yaml: finish appends perfkit summary to $GITHUB_STEP_SUMMARY when report.json exists. Otherwise it writes a note that names the artifact holding the raw results. A failed wait-* job writes the same kind of note.
• The code is in the new src/summary/ folder. src/render is unchanged; the summary does its own Markdown and number formatting.

Keep raw results when the agent fails

In Routr's run 41898433488 the agent failed near the end (Maximum call stack size exceeded, exit 3). The soak had no done marker, so collect never downloaded its raw results. The upload found no raw/ directory, and the cleanup job then deleted the droplets: 6.5 h of raw data was lost.

• collect now downloads the raw directory of every benchmark it has not downloaded yet, as it is, before it exits on an agent failure. This works with --wait (nothing is deleted) and without it (before teardown). It does the same before it tears down a cancelled run. Each one prints [<id>] downloaded the partial raw results to <dir>.
  • The partial directory gets no done marker, so it never counts as finished and the report's verdicts don't change.
  • A download that fails is skipped, and the exit code stays 3.
• New helper src/orchestrate/downloadPartialRaw.ts. It lists the load generator's raw/ directory and keeps only names that are valid benchmark ids.
• run.yaml: each wait-* job already uploaded perfkit-raw-<id> with if: always(), inside the job and before cleanup. cleanup now lists needs: [start, wait-1, wait-2, finish] explicitly, so it can only start after every upload.

Compatibility

• One new CLI command. The flags, exit codes, workflow inputs, secrets and outputs are unchanged.
• No schema changes.

Tests

• npm test: 478 passing. This adds tests for every summary function (including the validation failure), for perfkit summary in runCli, for downloadPartialRaw, and three createCollect tests for the agent-failed path:
  • with --wait: the soak's partial raw results are downloaded, nothing is deleted, exit 3;
  • without --wait: the raw results are on disk when teardown runs;
  • a failed download still exits 3.
• The two createCollect download tests fail without the fix.
• npm run typecheck and npm run lint are clean.

🤖 Generated with Claude Code