Skip to content

Commit b525eaa

Browse files
committed
AUS-1012: Publish the benchmarks measured on the cpu-performance job
1 parent 6c3c272 commit b525eaa

27 files changed

Lines changed: 2612 additions & 1248 deletions

‎.github/workflows/build_pull_request.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ jobs:
4646

4747
# The measurement: ProcessAsync must scale from 1 to 4 workers. A process-wide serialization point holds the ratio
4848
# near 1.3 on any core count. Diagnostic on the shared runner (its core count and isolation are not promised, so
49-
# this job is not required and continues on error); the release gate is `--scale 3 16 4.0` on the reference machine.
49+
# this job is not required and continues on error); the release gate is `--scale 3 32 4.0` in the cpu-performance publish job.
5050
# The per-serializer diagnostics after the gate (one run per cell, never part of the exit code) go to their own summary section.
5151
scaling:
5252
runs-on: ubuntu-latest

‎AustinHarris.JsonRpc.AspNetCore/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -161,9 +161,9 @@ invoked.
161161
Replies already finished are flushed before the connection waits on a slow method. When the connection closes,
162162
the handler waits for the running method to finish and discards its response.
163163
- **Cost:** every document then goes through `ProcessAsync`. With methods that complete inline, the TCP row measured
164-
15.0 M to 15.4 M against 14.3 M to 16.5 M for the synchronous mode on the same day, inside its spread. A method that
164+
17 M to 17.1 M against 18.5 M for the synchronous mode in the same benchmark job, about 8 % lower by the medians. A method that
165165
suspends pays for its own async state, the library's completion state and a continuation per request; the main README's
166-
Async table measured 559 B per request for the yielding None row in process at one worker, including the service's own
166+
Async table measured 548 B per request for the yielding None row in process at one worker, including the service's own
167167
allocations. The main README's Kestrel table has both rows, measured with `TestServer_Console --kestrel 3 async`.
168168

169169
A method receives the token by declaring a `[JsonRpcCancellation] CancellationToken` parameter; see

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ behaviour: a breaking change to either means a new major version.
5050
- Notifications never get a wire response, whatever their outcome.
5151
- Dates and non-finite numbers are written the same way by every serializer (fraction only when non-zero, `Z`/offset/nothing by `Kind`; `NaN` and the infinities as quoted strings).
5252
- Clarified that the WebAssembly sample is not a Native AOT or full-trimming compatibility guarantee.
53+
- The published benchmarks are measured on a Hugging Face Jobs `cpu-performance` host (AMD EPYC 7R13, 32 cores) and regenerated from the job's data (`benchmarks/hf/`, `ingest.py`, `render.py`); the desktop figures are gone from the README except the browser table.
5354

5455
### Deprecated
5556

‎Json-Rpc/README.md‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -135,25 +135,26 @@ The synchronous entry points do not await those methods.
135135

136136
## Performance
137137

138-
These results compare 1.2.3 and 2.0 with the same five requests on the same machine in one session.
139-
The 2.0 runs used the built-in serializer on an AMD Ryzen 7 7800X3D with .NET 10, Release and Server GC.
138+
These results compare 1.2.3 and 2.0 with the same five requests on the same machine in one benchmark job.
139+
The 2.0 runs used the built-in serializer on a Hugging Face Jobs `cpu-performance` host (AMD EPYC 7R13, 32 cores) with .NET 10, Release and Server GC.
140+
Each range is the low and high over the job's runs; the ratios are of the medians.
140141

141142
![JSON-RPC.Net 1.2.3 and 2.0 throughput through the string and byte entry points](https://raw.githubusercontent.com/Astn/JSON-RPC.NET/master/benchmarks/charts/headline-1x-vs-2.svg)
142143

143144
<!-- benchmarks:headline -->
144145
| Path | RPC/s | Against 1.2.3 |
145146
| --- | ---: | ---: |
146-
| 1.2.3, `Task<string> Process(string)`, thread pool, best batch size | 3.08 M | |
147-
| 2.0, the same string API and the same loop | 13.3 M | 4.3× |
148-
| 2.0, `Process(bytes)`, 16 dedicated threads | 31.7 M | 10.3× |
149-
| 2.0, `ProcessAsync(bytes)`, 16 awaited workers | 32.1 M | 10.4× |
147+
| 1.2.3, `Task<string> Process(string)`, thread pool, best batch size | 1.29 M to 2.41 M | |
148+
| 2.0, the same string API and the same loop | 9.83 M to 11.1 M | 6.0× |
149+
| 2.0, `Process(bytes)`, 32 dedicated threads | 44.2 M to 45.7 M | 24.3× |
150+
| 2.0, `ProcessAsync(bytes)`, 32 awaited workers | 36.3 M to 40.5 M | 20.7× |
150151

151152
<!-- /benchmarks:headline -->
152153

153154
These measurements cover the library without a transport.
154155
The asynchronous byte row uses methods that complete inline.
155156
For Kestrel TCP with 256 requests in flight per connection and `EnableAsyncMethods = false`,
156-
the measured range is 14.3 M to 16.5 M RPC/s on loopback, with clients and server on the same machine.
157+
the measured figure is 18.5 M RPC/s on loopback, with clients and server on the same machine.
157158

158159
See the [benchmark tables](https://astn.github.io/JSON-RPC.NET/#benchmarks) for conditions
159160
and the [benchmark explorer](https://astn.github.io/JSON-RPC.NET/benchmarks/charts/explorer.html) for the data.

0 commit comments

Comments
 (0)