Skip to content

render benchmarks on vitest - #45

Merged
jbolda merged 1 commit into
mainfrom
render-benchmarks
Oct 25, 2026
Merged

render benchmarks on vitest#45
jbolda merged 1 commit into
mainfrom
render-benchmarks

Conversation

@jbolda

@jbolda jbolda commented Oct 23, 2026

Copy link
Copy Markdown
Member

What does this PR do?

We were starting to see places which would bottleneck so it was worth getting a few rough checks in place so we at least know we are in the ballpark.

Type of change

  • Bug fix
  • Feature
  • Refactor (no behavior change)
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • All tests pass (pnpm test)
  • Files are formatted (pnpm format)
  • I have added/updated tests for my changes (if applicable)
  • I have added a changeset

AI-generated code disclosure

  • This PR includes AI-generated code

I used it to help spike out and play around with edge cases, look and feel, etc. I refined and edited it from there.

Perf Results

Results on an m1 air 
 ✓  bench  bench/render.bench.ts > initial render 15906ms
   name                    hz      min      max     mean      p75      p99     p995     p999      rme  samples
   maple (100)       3,022.54   0.2450   5.3240   0.8859   0.4981   5.0549   5.1358   5.2981  ±17.77%      342   fastest
   bush (100)        2,877.69   0.2583   6.5268   0.8970   0.4990   5.3024   5.6115   6.2499  ±17.79%      335
   evergreen (100)   2,139.82   0.3626   5.3887   0.9946   0.5855   5.1734   5.2561   5.3532  ±16.64%      302
   grass (100)       3,074.33   0.2302  29.1015   1.0193   0.4903   5.4465   6.9776  22.8067  ±25.27%      296
   sequoia (100)     1,290.59   0.6233   5.4755   1.2339   0.8797   5.2956   5.4060   5.4660  ±14.04%      244
   bamboo (100)      1,252.01   0.6420   5.7294   1.2385   0.8939   5.1461   5.2473   5.6176  ±13.53%      244
   grass (500)         920.23   0.8837   7.2930   1.6088   1.1255   6.4978   6.6694   7.1683  ±14.76%      187
   maple (500)         802.51   1.0007   6.5820   1.7446   1.2844   6.5170   6.5704   6.5797  ±13.83%      172
   bush (500)          711.99   1.1184   6.6588   1.8262   1.5803   6.0789   6.5549   6.6380  ±11.85%      166
   grass (1000)        494.68   1.6483   7.7744   2.4147   2.0833   7.0405   7.3307   7.6857  ±11.15%      125
   maple (1000)        449.20   1.8613   7.2810   2.6213   2.2462   7.1545   7.2105   7.2669  ±10.69%      115
   bush (1000)         388.81   2.1342   8.2688   2.9413   2.7117   8.1681   8.2221   8.2595   ±9.77%      102
   grass (2000)        322.96   2.5015   8.9270   3.5536   3.2986   8.8999   8.9135   8.9243  ±10.67%       86
   maple (2000)        279.59   2.9676   9.4627   4.0072   3.7432   9.1853   9.3240   9.4350  ±10.16%       76
   bush (2000)         227.28   3.7202   9.9249   4.8012   4.4783   9.6223   9.7736   9.8946   ±9.29%       63
   evergreen (500)     199.08   4.4810  10.3549   5.3623   4.8436   9.9533  10.1541  10.3147   ±8.65%       56
   sequoia (500)      70.6822  12.2680  23.4280  15.1154  15.2713  23.3485  23.3883  23.4201  ±14.05%       20
   bamboo (500)       68.7517  13.0976  24.4598  15.3900  13.5593  24.3598  24.4098  24.4498  ±13.40%       20
   evergreen (1000)   53.9601  17.0238  31.8780  19.2836  17.9543  31.8229  31.8505  31.8725  ±13.47%       16
   sequoia (1000)     19.2264  49.8677  76.6130  52.8602  50.3888  74.2760  75.4445  76.3793  ±11.30%       10
   bamboo (1000)      17.2719  53.5120  83.2485  59.4387  54.9284  82.8335  83.0410  83.2070  ±13.72%       10
   evergreen (2000)   13.2559  69.4968   101.59  76.8917  73.6226   101.32   101.46   101.57  ±11.45%       10
   sequoia (2000)      4.8666   201.27   235.24   205.90   202.75   232.55   233.89   234.97   ±3.60%       10
   bamboo (2000)       4.5735   214.78   246.20   218.99   216.40   243.72   244.96   245.95   ±3.14%       10   slowest
 ✓  bench  bench/render.bench.ts > single value update 37007ms
   name                    hz      min      max     mean      p75      p99     p995     p999      rme  samples
   grass (100)       4,840.91   0.1588   3.0955   0.4074   0.2232   2.3245   2.4623   2.8585  ±10.25%      740   fastest
   maple (100)       4,435.71   0.1751   3.5413   0.4197   0.2412   2.2107   2.3007   2.7506   ±9.84%      719
   bush (100)        4,142.32   0.1903   3.3660   0.4393   0.2578   2.2504   2.3728   2.7902   ±9.83%      686
   evergreen (100)   2,869.11   0.2905   3.0537   0.4971   0.3315   2.2458   2.3817   2.7932   ±8.08%      604
   grass (500)       1,510.92   0.6142   1.8876   0.6901   0.6471   1.6583   1.6748   1.7978   ±2.86%      435
   sequoia (100)     1,553.33   0.5503   2.2153   0.7105   0.6171   2.0701   2.1585   2.2106   ±3.96%      423
   bamboo (100)      1,459.97   0.5667   2.3178   0.7605   1.0276   2.0370   2.1552   2.2870   ±4.00%      395
   maple (500)       1,269.75   0.7210   2.9001   0.8304   0.7596   1.7567   1.7671   2.4912   ±3.39%      362
   bush (500)        1,099.05   0.8358   2.7956   0.9439   0.8892   1.9096   1.9607   2.5421   ±2.88%      319
   grass (1000)        708.25   1.2448   2.4776   1.4438   1.4692   2.3876   2.4141   2.4647   ±2.32%      208
   maple (1000)        598.94   1.4527   4.3019   1.7926   1.6875   4.1804   4.2306   4.2876   ±5.50%      168
   bush (1000)         517.49   1.7086   4.5684   2.0707   1.8333   4.4538   4.4947   4.5537   ±5.65%      145
   grass (2000)        500.01   1.7818   4.9412   2.0839   1.9952   4.8560   4.8963   4.9323   ±4.61%      144
   maple (2000)        422.18   2.1817   4.5835   2.4259   2.3061   4.3649   4.5581   4.5784   ±3.37%      124
   bush (2000)         304.05   2.9518   5.7878   3.4226   3.3053   5.7216   5.7547   5.7812   ±5.16%       88
   evergreen (500)     213.24   4.1838   9.6613   5.0624   4.4401   9.5444   9.6028   9.6496   ±9.02%       60
   sequoia (500)      70.7556  11.9330  23.4388  15.3312  22.3781  23.3448  23.3918  23.4294  ±15.16%       20
   bamboo (500)       68.5211  12.7423  24.0340  15.6625  15.7431  24.0211  24.0275  24.0327  ±14.57%       20
   evergreen (1000)   52.2001  16.5952  32.0158  20.5644  20.7997  31.9249  31.9704  32.0067  ±16.73%       16
   sequoia (1000)     19.9125  49.5385  51.2825  50.2272  50.7365  51.2711  51.2768  51.2813   ±0.93%       10
   bamboo (1000)      14.9268  53.0400  82.0603  69.6996  80.3149  81.9684  82.0144  82.0511  ±14.28%       10
   evergreen (2000)   12.7170  68.3919  99.5127  80.9838  98.0207  99.4240  99.4684  99.5038  ±13.29%       10
   sequoia (2000)      4.7545   201.33   238.03   210.87   217.11   236.38   237.21   237.86   ±3.96%       10
   bamboo (2000)       4.3892   216.54   251.55   228.30   232.37   250.03   250.79   251.39   ±3.46%       10   slowest
 ✓  bench  bench/render.bench.ts > batch 100 value updates 34430ms
   name                    hz      min      max     mean      p75      p99     p995     p999      rme  samples
   grass (100)       2,429.77   0.2973   5.3805   0.8806   0.5518   4.9194   5.0144   5.3071  ±15.74%      341   fastest
   maple (100)       2,352.99   0.3039   6.6590   0.9101   0.5948   4.8986   4.9342   6.1010  ±15.78%      330
   bush (100)        2,254.40   0.3200   5.3682   0.9181   0.5941   4.8525   5.0978   5.3076  ±15.50%      328
   maple (500)       1,114.21   0.8399   2.3383   0.9377   0.8757   2.2593   2.3024   2.3328   ±3.36%      320
   evergreen (100)   1,752.53   0.4240  13.3050   1.0122   0.6833   4.7775   6.2969  11.5978  ±15.66%      297
   bush (500)        1,003.87   0.9145   2.4971   1.0533   0.9529   2.3218   2.3527   2.4617   ±3.80%      285
   sequoia (100)     1,168.00   0.6880   4.6188   1.1083   1.1175   4.5551   4.5859   4.6113   ±9.34%      271
   bamboo (100)      1,133.07   0.7053   4.9050   1.1233   1.1697   4.2496   4.3847   4.7706   ±8.87%      268
   grass (500)         930.83   1.0027   4.0843   1.1296   1.0364   2.4351   2.5334   3.6852   ±3.98%      266
   grass (1000)        624.16   1.4207   4.2579   1.6609   1.6539   3.7863   4.2452   4.2553   ±3.73%      181
   maple (1000)        521.54   1.7104   3.3138   1.9670   1.9965   3.2342   3.2691   3.3049   ±2.92%      153
   grass (2000)        494.02   1.8281   5.7993   2.0757   2.0190   3.8714   4.5417   5.5478   ±3.60%      145
   bush (1000)         488.73   1.7931   5.3438   2.1450   2.0598   3.4163   4.0212   5.0793   ±4.38%      140
   maple (2000)        396.88   2.2536   6.4623   2.6484   2.6902   6.2923   6.3755   6.4449   ±5.60%      114
   bush (2000)         295.43   3.0378   5.7675   3.4877   3.4326   5.7249   5.7462   5.7633   ±4.41%       87
   evergreen (500)     205.34   4.2530  10.7225   5.1991   4.6857  10.2770  10.4997  10.6779   ±8.32%       58
   sequoia (500)      73.5231  12.1922  25.0563  14.3458  13.1130  24.8939  24.9751  25.0400  ±13.23%       21
   bamboo (500)       69.2801  12.8535  25.1245  15.4154  13.3324  25.0783  25.1014  25.1199  ±14.62%       20
   evergreen (1000)   56.8297  16.8524  31.5589  17.9732  17.1588  29.3046  30.4317  31.3334  ±10.02%       17
   bamboo (1000)      17.8126  53.4470  80.3975  56.9394  54.3857  78.2555  79.3265  80.1833  ±10.41%       10
   sequoia (1000)     17.1035  50.6884  79.7612  60.5991  72.8530  79.7000  79.7306  79.7551  ±15.28%       10
   evergreen (2000)   11.6195  73.3220   108.54  88.2266   100.43   107.85   108.20   108.47  ±11.87%       10
   sequoia (2000)      4.4131   204.43   287.90   228.76   237.43   283.69   285.80   287.48   ±7.75%       10
   bamboo (2000)       4.2714   218.34   253.20   234.95   248.87   253.02   253.11   253.18   ±4.48%       10   slowest
 ✓  bench  bench/render.bench.ts > add one component 34732ms
   name                    hz      min      max     mean      p75      p99     p995     p999      rme  samples
   maple (100)       4,010.77   0.1799   7.2239   0.8024   0.4416   5.4747   6.1940   6.9253  ±18.98%      374   fastest
   grass (100)       4,444.96   0.1559   7.4477   0.8107   0.4363   6.2221   6.4878   7.1391  ±19.65%      371
   bush (100)        3,756.59   0.1920   7.6555   0.8311   0.4430   6.3293   6.4851   7.2478  ±19.21%      361
   maple (500)       1,227.18   0.7424   3.2970   0.8892   0.7906   2.4886   2.5711   3.0886   ±4.95%      338
   evergreen (100)   2,585.25   0.2967   8.6157   1.0241   0.5549   6.4553   6.6229   8.0327  ±19.35%      297
   bush (500)        1,029.48   0.8637   3.0365   1.0723   0.9355   2.6474   2.6810   2.9399   ±5.28%      280
   grass (500)         900.38   1.0078   3.0566   1.1815   1.0886   2.7075   2.7918   2.9934   ±4.30%      254
   bamboo (100)      1,386.86   0.5795   7.0001   1.2800   0.7039   6.5753   6.6093   6.9091  ±16.36%      238
   sequoia (100)     1,413.14   0.5630   8.7203   1.3104   0.8692   7.3543   7.4874   8.4399  ±17.48%      229
   maple (1000)        586.11   1.4860   3.3521   1.7678   1.7176   3.3111   3.3348   3.3486   ±3.35%      170
   bush (1000)         502.03   1.7317   5.2188   2.1208   1.9751   5.0779   5.1394   5.2029   ±5.35%      142
   grass (2000)        486.55   1.7894   4.4793   2.1319   2.0336   4.3351   4.4563   4.4747   ±4.05%      141
   maple (2000)        414.51   2.2084   4.8883   2.4763   2.3625   4.4064   4.6216   4.8350   ±3.60%      122
   grass (1000)        288.54   3.1755   6.2011   3.5433   3.5512   6.1548   6.1780   6.1965   ±3.88%       85
   bush (2000)         295.36   2.9920   8.1794   3.6029   3.4455   7.9991   8.0892   8.1613   ±6.97%       84
   evergreen (500)     205.12   4.2745   9.8165   5.2101   4.6443   9.7811   9.7988   9.8130   ±8.24%       58
   sequoia (500)      75.6292  12.1512  23.7133  13.9077  12.7443  23.7034  23.7084  23.7123  ±12.68%       22
   bamboo (500)       68.0041  13.0208  25.9668  15.8007  13.7054  25.9037  25.9352  25.9605  ±15.69%       19
   evergreen (1000)   50.6415  16.8810  32.9560  21.4517  28.4533  32.8555  32.9058  32.9460  ±19.30%       14
   bamboo (1000)      18.3372  53.9508  55.2493  54.5372  54.8806  55.2369  55.2431  55.2480   ±0.57%       10
   sequoia (1000)     14.6881  50.3573  80.8083  71.1514  80.0098  80.7962  80.8022  80.8070  ±14.31%       10
   evergreen (2000)   12.4954  69.8645   100.84  82.4068  99.7875   100.79   100.81   100.84  ±13.26%       10
   sequoia (2000)      4.5427   204.05   307.23   223.47   228.28   301.23   304.23   306.63  ±10.31%       10
   bamboo (2000)       4.4581   218.43   255.58   224.88   220.43   254.12   254.85   255.43   ±3.99%       10   slowest
 ✓  bench  bench/render.bench.ts > add 100-component subtree 31761ms
   name                    hz      min      max     mean      p75      p99     p995     p999      rme  samples
   maple (500)         884.55   0.9664   3.0886   1.2678   1.1270   3.0552   3.0648   3.0835   ±5.64%      237   fastest
   bush (500)          792.24   1.0843   4.6235   1.3907   1.2689   3.3057   3.3275   4.3438   ±5.54%      217
   maple (100)       1,488.64   0.4759  10.1623   1.7145   0.8548   9.7796   9.8957  10.1090  ±24.86%      175
   bush (100)        1,481.88   0.4878  10.0890   1.7262   0.8450   9.8364   9.9257  10.0563  ±25.06%      174
   grass (100)       1,284.48   0.5465  10.1925   1.8248   1.0124   9.9551  10.0346  10.1609  ±23.84%      165
   grass (500)         578.81   1.5347   3.8818   1.8254   1.7558   3.6761   3.7545   3.8563   ±4.44%      165
   evergreen (100)   1,259.49   0.5923  10.9835   1.8905   0.9451  10.2619  10.7724  10.9412  ±24.38%      161
   bamboo (100)        887.02   0.8771  10.1423   2.0095   1.2206   9.9053  10.0380  10.1214  ±20.37%      151
   sequoia (100)       891.43   0.8598   9.8265   2.0422   1.4506   9.3264   9.4632   9.7539  ±20.55%      147
   maple (1000)        501.21   1.7003   4.1993   2.0917   2.2955   3.9491   4.0221   4.1639   ±4.29%      144
   grass (2000)        452.13   1.9257   5.0187   2.3279   2.1749   4.9108   5.0036   5.0157   ±5.18%      129
   bush (1000)         423.34   1.9659   5.8905   2.5158   3.1084   4.3233   4.9605   5.7045   ±5.29%      120
   maple (2000)        392.27   2.3281   5.5202   2.6233   2.4967   4.8891   5.1602   5.4482   ±3.91%      116
   grass (1000)        299.71   3.0029   6.5090   3.4192   3.5904   5.5487   6.0289   6.4130   ±3.94%       88
   bush (2000)         273.75   3.1127  22.0301   4.1694   3.7500  13.3543  17.6922  21.1625  ±14.61%       72
   evergreen (500)     188.46   4.4878  10.0912   5.8282   8.1513  10.0537  10.0725  10.0875   ±9.87%       52
   bamboo (500)       69.1511  13.2088  26.1357  15.2907  13.6427  26.0713  26.1035  26.1292  ±13.98%       20
   sequoia (500)      70.1809  12.3772  25.3443  15.5041  15.6539  25.1948  25.2695  25.3293  ±16.10%       20
   evergreen (1000)   52.0812  17.1300  33.6423  20.5322  17.8066  33.5571  33.5997  33.6338  ±17.64%       15
   sequoia (1000)     19.4747  50.6992  52.7933  51.3568  51.8008  52.7251  52.7592  52.7865   ±0.95%       10
   bamboo (1000)      17.6918  54.2414  80.0750  57.2733  55.1425  77.9096  78.9923  79.8585  ±10.03%       10
   evergreen (2000)   12.8137  70.1793   101.62  80.1427  93.8452   101.61   101.62   101.62  ±13.09%       10
   sequoia (2000)      4.7409   204.96   234.06   211.48   206.38   234.04   234.05   234.06   ±4.01%       10
   bamboo (2000)       4.2454   218.29   252.16   236.44   247.33   252.13   252.14   252.16   ±4.57%       10   slowest

@socket-security

socket-security Bot commented Oct 23, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security		 Vulnerability Quality Maintenance License
Updated@​bomb.sh/​tty@​0.7.0 ⏵ 0.10.084 +71009993 +3100

View full report

@pkg-pr-new

pkg-pr-new Bot commented Oct 23, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/bombshell-dev/ui/@clack/ui@45

commit: 04fc887

cowboyd
cowboyd approved these changes Oct 25, 2026
View reviewed changes

@cowboyd cowboyd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we integrate this with codspeed so that we can get these clocked on each PR?

@jbolda

jbolda commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

There is a plugin. I can go ahead and wire that up in a follow up PR!

@jbolda
jbolda merged commit ae4e0e7 into main Oct 25, 2026
7 checks passed
@jbolda
jbolda deleted the render-benchmarks branch October 25, 2026 05:57
@cowboyd cowboyd mentioned this pull request Oct 27, 2026
Merged
5 of 12 tasks
@jbolda jbolda mentioned this pull request Oct 28, 2026
Merged
4 of 12 tasks
natemoo-re pushed a commit that referenced this pull request Oct 29, 2026
@jbolda
benchmarks in codspeed (#47)
f3ec856 
## What does this PR do?

Follow up on #45. This uses codspeed to execute the benchmarks in CI. It
gives us deeper information including memory timing. I set it up similar
to the TTY repo.

## Type of change

<!-- Check one. -->

- [ ] Bug fix
- [ ] Feature
- [ ] Refactor (no behavior change)
- [ ] Documentation
- [ ] Performance improvement
- [ ] Tests
- [x] Chore (dependencies, CI, tooling)

## Checklist

- [x] All tests pass (`pnpm test`)
- [x] Files are formatted (`pnpm format`)
- [x] I have added/updated tests for my changes (if applicable)
- [ ] I have added a changeset

## AI-generated code disclosure

<!-- If any part of this PR was generated by AI tools (Copilot, Claude,
GPT, Cursor, etc.), check the box. This is fine — we just need to know
so reviewers can pay extra attention to edge cases. -->

- [ ] This PR includes AI-generated code
cowboyd added a commit that referenced this pull request Nov 6, 2026
@cowboyd
perf: traverse layout with an iterator stack (#46)
24b5a76 
## What does this PR do?

Deep trees make layout's recursive generator delegation increasingly
expensive due to the way JavaScript handles generator delegation. This
change traverses child iterators with an explicit stack, so that
externally the API remains exactly the same, but internally it is much
more performant.

The performance issue is demonstrated by the rendering benchmarks added
in #45, so no separate issue is needed. Tests cover a 20,000-level tree,
adjacent text grouping, stable IDs, subtree layout, and early-return
closing behavior.

## Benchmark comparison

Ran the existing rendering benchmark suite before and after: 120 cases
per version, sequentially on the same machine with identical
dependencies and settings. Values below are end-to-end mean rendering
times in milliseconds.

Initial render, 2,000 components:

| Shape | Recursive (ms) | Iterator stack (ms) | Speedup |
| --- | ---: | ---: | ---: |
| Grass | 3.06 | 2.79 | 1.1× |
| Maple | 3.32 | 2.88 | 1.2× |
| Bush | 4.32 | 2.87 | 1.5× |
| Evergreen | 68.92 | 3.22 | 21× |
| Sequoia | 185.15 | 2.98 | 62× |
| Bamboo | 189.57 | 3.01 | 63× |

Mutation workloads, 2,000-component bamboo trees:

| Workload | Recursive (ms) | Iterator stack (ms) | Speedup |
| --- | ---: | ---: | ---: |
| Single value update | 205.97 | 1.91 | 108× |
| Batch 100 updates | 202.68 | 2.01 | 101× |
| Add one component | 206.72 | 2.07 | 100× |
| Add 100-component subtree | 206.69 | 2.25 | 92× |

## Type of change

- [ ] Bug fix
- [ ] Feature
- [ ] Refactor (no behavior change)
- [ ] Documentation
- [x] Performance improvement
- [ ] Tests
- [ ] Chore (dependencies, CI, tooling)

## Checklist

- [x] All tests pass (`pnpm test`)
- [x] Files are formatted (`pnpm format`)
- [x] I have added/updated tests for my changes (if applicable)
- [ ] I have added a changeset

## AI-generated code disclosure

- [x] This PR includes AI-generated code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cowboyd cowboyd cowboyd approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jbolda @cowboyd