These scripts run the TestServer_Console benchmarks on a Hugging Face Jobs machine and bring the results back. The published figures in the repository README come from the publish profile on a cpu-performance host.
launch.sh <ref> <flavor> [runs|publish]starts a detached job on<flavor>(for examplecpu-performance) in themcr.microsoft.com/dotnet/sdk:10.0image, with a two-hour timeout.<ref>is a branch, tag or commit. It needs thehfCLI, logged in.run.sh <ref> [runs|publish]is what the job runs: it clones the repository, checks out<ref>, builds in Release and runs the harness. The core count comes from the cgroup CPU quota when there is one, elsenproc.fetch.py <job id> <out dir> [--log <file>]reads the job's log (hf jobs logs, or a saved log with--log) and writesjob.log,job-id.txt, every framed JSON file, and each framed run's console output as<mode>-<run>.txtwith the colour codes removed.
With a number, or nothing (3), run.sh runs every mode once at three runs per point (--scale 3 <cores> 4.0, --sync 3, --async 3 1, --async 3 <cores>, --kestrel 3, --kestrel 3 async, --compare 3) and then that many --sweep 2 runs, each printed between == BEGIN sweep-<k>.json and == END sweep-<k>.json.
With publish it also builds benchmarks/Baseline, sets JSONRPC_BENCH_RESULTS=/results so every mode writes a JSON file there, and records the commit in each file. It then runs:
- the scaling gate once (its exit code is printed as
Scale gate exit code: Nand recorded inscale-1.json), - two rounds of
--sync 3,--async 3 1,--async 3 <cores>,--kestrel 3,--kestrel 3 asyncand--compare 3, interleaved, so drift over the job spreads across the modes, - the 1.x-compatible string loop (menu entry
t) three times, alternating with two runs ofbenchmarks/Baseline(1.2.3 from NuGet), - five
--sweep 2runs.
Each run's console output is framed by == BEGIN <mode> <run> and == END <mode> <run>, and at the end every file under /results is printed between == BEGIN <name>.json and == END <name>.json. The whole profile has to finish inside the two-hour timeout that launch.sh sets.
To republish, run benchmarks/hf/launch.sh <ref> cpu-performance publish, then python benchmarks/hf/fetch.py <job id> <dir> to extract each run's JSON and text. python benchmarks/charts/ingest.py <dir> --source <job id> --conditions "..." folds the JSON into benchmarks.json and the sweep files, and render.py regenerates the charts, the explorer and the README tables between <!-- benchmarks:... --> markers. render.py --check fails when any of them is stale.
The same files come from a local run: set JSONRPC_BENCH_RESULTS=<dir> (and optionally JSONRPC_BENCH_RUN=<n> and JSONRPC_BENCH_COMMIT=<sha>) before running a mode.