Repository navigation
test(bench): re-time ir-seconds.json from CI and cut plaintext-set-100kb to 400 ops (LAB-12407) - #306
Conversation
…0kb to 400 ops Shard balance read seconds that #303 scaled from local runs; in CI the first shard ran 2.36x its prediction, 27m34s of the 30-minute timeout. Every row now holds CI's base.json seconds / repeats. The 100 KB set's single run (566 s) set the floor of every shard's base and head suites; at 400 ops it reads the same Ir/op within 0.01%.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 4 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Instruction counts: PASS (no runtime change)Head |
…r runner Fourteen shard runs measured each workload on runners up to 1.9x apart in speed. Each row is now its cost on a runner in the middle of that range, so any runner seen so far lands within 1.4x of its shard's prediction. The 100 KB set's row is its first CI figure at 400 ops.
…asures this PR" This reverts commit eeecac2.
Code Review Completed! 🔥The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Providing Context (Files & MCPs)Add these hints in your PR description (or a comment) to unlock deeper checks:
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
|
![kodus-27b[bot]](https://avatars.githubusercontent.com/u/223823690?s=60&v=4){kind=small}
The first measured instruction-count run after #303 (run 41364076750, on #304) put shard 1's gate step at 27m34s of the job's 30-minute timeout, 2.36x the load
bench/ir-seconds.jsonpredicted for it. #303 refreshed that file from local runs scaled to CI, and the scaling did not hold:plaintext-set-100kbalone read 566 s per run against 191 predicted.With this PR's file, run 41370359224 measured all four shards (36 workloads ok):
The shards now sit within 15% of each other; before, they spread 3x (312 to 929). All four runners in that run were on the slow side of the range below, which is why every ratio is above 1.
This PR:
bench/ir-seconds.jsonfrom CI. The figures come from thebase.jsonartifacts (seconds / repeats) of four measured runs since test(bench): open the bench:ir window after V8's tier-up (LAB-12188) #303 merged, 14 shard runs in all, including this PR's first. A shard's runner sets the speed of everything on it: across those runners speed varied 1.9x, and every workload on one runner moved together (median residual 0.6%). So each row is fitted as cost × one speed factor per runner, and the factors are centred so the fastest and slowest runner seen miss their prediction by the same ratio, about 1.4x. Taking one run's figures would carry its runners' speeds into the balance: in run 41364076750, shard 1's runner ran 1.8x slower than shard 3's.plaintext-set-100kbfrom 1,000 to 400 ops. Each shard runs its base suite, then its head suite, and each suite waits for its slowest single run. At up to 566 s per run, this one workload put a floor of about 19 minutes under any shard that held it, whatever the shard count. At 400 ops its run is about a third shorter. In CI it reads 18,091,693 Ir/op against 18,090,276 and 18,089,422 at 1,000 (+0.01%), with an A/A spread of 0.0003%.CONTRIBUTING.md: refresh a row from CI when a workload's n or warm-up changes, scale it by how its shard's other workloads compare with their rows, and keep any one workload's run under about 7 minutes in CI.eeecac2added one HTML comment line topackages/cachekit/README.mdso the gate would measure this PR rather than pass it as same-build;d07adffreverts it, so the squashed diff does not touch the README.Closes LAB-12407