Skip to content

[trillium.rs] update dependencies - #1471

Open
jbr wants to merge 2 commits into
MDA2AV:mainfrom
jbr:update-trillium-patches
Open

[trillium.rs] update dependencies#1471
jbr wants to merge 2 commits into
MDA2AV:mainfrom
jbr:update-trillium-patches

Conversation

@jbr

@jbr jbr commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Description

Updates some trillium dependencies that may have performance impacts (alloc counts).


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp --compare genhttp-kestrel

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk) are built as self-contained Docker images on first run.

@jbr

jbr commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark-multiple --save

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 2 | Test: all tests

trillium

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
baseline 4096 596,428 6042.1% 208MiB -0.3% +4.0%
pipelined 4096 2,942,210 6337.3% 225MiB -0.9% +21.0%
limited-conn 4096 511,540 2397.7% 219MiB +2.7% +5.3%
json-comp 4096 112,571 6247.4% 762MiB -3.2% -1.7%
json-comp 16384 110,848 6011.9% 774MiB -1.8% +2.9%
json-tls 4096 613,708 6468.7% 418MiB +2.7% -2.3%
8gbit 512 48,662 0.9732 ⚠️ 50.22us 149.0us 4362.0us 242.5% 228MiB -0.9% -5.0%
static-tls 1024 541,037 6401.1% 514MiB -2.4% -2.8%
async-db 1024 214,163 3256.4% 241MiB +1.5% +1.3%
fortunes 1024 207,201 5009.9% 319MiB +4.2% +0.3%
baseline-h2 256 1,113,605 5457.2% 950MiB -2.0% +8.4%
baseline-h2 1024 1,242,176 5325.8% 3.8GiB +9.7% +35.7%
static-h2 256 20,151 4108.2% 1.8GiB -7.0% -5.3%
static-h2 1024 18,419 3762.7% 2.5GiB ~0% -16.7%
baseline-h2c 256 1,281,402 5258.6% 897MiB +10.3% +6.2%
baseline-h2c 1024 1,151,879 5493.4% 3.1GiB -9.7% +34.8%
baseline-h2c 4096 1,237,468 5530.5% 9.6GiB +10.4% +47.7%
json-h2c 1024 955,862 5717.6% 1.9GiB -2.5% +26.7%
json-h2c 4096 928,509 5668.6% 4.9GiB +2.6% -3.9%
baseline-h3 64 425,126 2539.1% 301MiB +1.3% +3.8%
static-h3 64 219,349 2439.4% 507MiB +3.3% -17.7%
gateway-64 512 81,132 5405.6% 3.0GiB -1.2% +3.4%
gateway-64 1024 75,235 5125.8% 5.8GiB -2.6% +9.4%
gateway-h3 64 82,606 4092.8% 896MiB -1.0% -1.9%
gateway-h3 256 60,594 2788.9% 1.9GiB -1.8% -20.8%
unary-grpc 256 690,051 5421.3% 467MiB -7.2% +6.9%
unary-grpc 1024 709,412 5595.0% 1.4GiB +1.4% +7.7%
unary-grpc-tls 256 760,673 5288.4% 440MiB +4.9% +3.8%
unary-grpc-tls 1024 650,980 5682.8% 1.4GiB +1.0% ~0%
echo-ws 512 440,297 3279.7% 306MiB -0.3% -7.0%
echo-ws 4096 355,252 3987.8% 602MiB +0.4% +5.4%
echo-ws 16384 342,662 3660.7% 1.8GiB -2.1% +152.8%
echo-ws-pipeline 512 9,127,299 3753.9% 292MiB ~0% -16.6%
echo-ws-pipeline 4096 7,857,900 4070.6% 659MiB -0.6% -4.2%
echo-ws-pipeline 16384 6,786,700 3387.4% 1.3GiB -9.2% +35.4%
echo-ws-limited 512 270,939 2913.4% 471MiB +0.4% -1.1%
echo-ws-limited 4096 272,378 2821.0% 461MiB ~0% -14.9%
latency-1m 1024 632,675 0.6327 ⚠️ 96.22us 7055360.0us 7182336.0us 6343.3% 159MiB +7.2% +15.2%
latency-10k 1024 9,912 0.9912 27.16us 107.0us 612.0us 25.0% 159MiB -0.2% +4.6%
latency-500k-8cpu 1024 281,645 0.5633 ⚠️ 28.05us 7960576.0us 8169472.0us 772.3% 73MiB -3.4% -6.4%
async 32000 407,249 3325.1% 1.5GiB -3.0% +7.1%

trillium-tuned

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
baseline 4096 574,772 6310.2% 417MiB -11.1% +2.0%
pipelined 4096 2,797,935 6396.3% 365MiB -2.3% +18.9%
limited-conn 4096 514,442 5959.0% 495MiB -9.4% -1.2%
json-comp 4096 110,562 6253.1% 1.0GiB +0.3% +2.1%
json-comp 16384 99,874 6433.0% 1.4GiB +0.6% -6.7%
json-tls 4096 526,770 6249.0% 503MiB -3.0% +4.1%
8gbit 512 49,155 0.9831 54.28us 130.0us 1354.0us 247.1% 270MiB -0.4% -2.5%
static-tls 1024 490,320 6380.1% 912MiB -13.6% -19.0%
async-db 1024 229,592 3557.5% 256MiB +1.8% -7.9%
fortunes 1024 159,782 4420.5% 375MiB +2.3% -0.5%
baseline-h2 256 1,244,433 5454.5% 770MiB -8.8% -1.8%
baseline-h2 1024 1,254,584 5748.0% 2.6GiB +2.0% +13.0%
static-h2 256 190,910 6199.4% 11.1GiB +0.8% +6.7%
static-h2 1024 179,689 6148.9% 15.4GiB +0.5% -4.3%
baseline-h2c 256 1,381,238 5407.8% 836MiB +0.2% +13.1%
baseline-h2c 1024 1,414,848 5669.5% 2.9GiB +14.3% +16.0%
baseline-h2c 4096 1,343,323 5256.3% 10.1GiB +0.9% +11.0%
json-h2c 1024 1,179,716 5540.8% 1.3GiB +0.1% -7.1%
json-h2c 4096 1,059,700 5858.0% 4.6GiB +0.9% +4.5%
baseline-h3 64 427,713 2596.2% 416MiB -2.6% +8.1%
static-h3 64 226,765 2574.6% 701MiB -2.9% -1.1%
gateway-64 512 76,041 5713.4% 2.8GiB +3.8% -6.7%
gateway-64 1024 68,759 5308.3% 5.5GiB -7.2% -1.8%
gateway-h3 64 82,375 4075.8% 917MiB -0.2% -1.6%
gateway-h3 256 60,216 2921.6% 2.4GiB -7.5% +118.2%
unary-grpc 256 664,634 5385.9% 695MiB -6.8% -8.4%
unary-grpc 1024 652,387 5703.7% 2.2GiB -10.9% ~0%
unary-grpc-tls 256 784,140 5448.0% 935MiB -2.7% +34.9%
unary-grpc-tls 1024 796,764 5787.1% 2.6GiB +1.5% +36.8%
echo-ws 512 1,692,428 6529.4% 351MiB +2.2% +2.3%
echo-ws 4096 425,623 6516.5% 868MiB +0.2% -8.8%
echo-ws 16384 406,003 6201.0% 3.0GiB -0.2% ~0%
echo-ws-pipeline 512 15,414,595 6683.9% 325MiB +1.4% -4.4%
echo-ws-pipeline 4096 10,954,723 6591.2% 908MiB -1.1% -1.7%
echo-ws-pipeline 16384 6,918,067 6209.8% 2.9GiB -0.1% -6.5%
echo-ws-limited 512 1,072,355 5302.1% 482MiB +3.6% -11.7%
echo-ws-limited 4096 533,162 5562.6% 1.0GiB +2.2% -9.1%
latency-1m 1024 639,132 0.6391 ⚠️ 93.01us 9687040.0us 11448320.0us 6185.3% 193MiB +11.0% -1.5%
latency-10k 1024 9,983 0.9983 33.40us 118.0us 163.0us 32.3% 192MiB ~0% -4.5%
latency-500k-8cpu 1024 276,261 0.5525 ⚠️ 28.86us 9367552.0us 9629696.0us 764.3% 111MiB -2.6% +26.1%
async 32000 594,863 6064.1% 1.4GiB -4.9% -12.5%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant