Repository navigation
fix(analyze): judge soak memory growth on a Theil–Sen slope after a settle period - #31
Conversation
A soak's memoryGrowthMBPerHour is now the least-squares slope over the
second half of the window after the warm-up, for processes and Kubernetes
pods alike, and the live page uses the same window. A heap that climbs while
it settles after start-up and then levels off no longer fails the limit; a
real leak still shows in the second half.
Each memoryGrowthMBPerHour rule in report.json gets an optional window
({ from, to }) and the soak's checks card says which window the slope covers.
|
Recomputed on the raw samples of Routr run 42699126885 (10 min warm-up, 2 h measured). This branch does not make that run pass:
Why A separate issue: the analyzer also counts samples taken after k8s A maintainer should decide whether the second-half rule stays, or whether a fixed settle period after the warm-up (or a longer warm-up) fits better. |
…ettle period Replace the second-half rule: memory growth is now the Theil-Sen slope (median of pairwise slopes, an evenly spaced subset of 600 samples on long runs) from the end of the warm-up plus a settle period to the end of the load. Samples taken after meta.endedAt, while the load winds down, no longer count. The settle period is a new optional soak field, settle (default 30m, or half of duration when that is shorter); the report records settleS. Processes, Kubernetes pods and the live page all use the same window.
psanders
changed the title
Why
In Routr run 42699126885, RoutrOne's RSS climbed during start-up and levelled off around minute 25. The least-squares slope over the whole window after the warm-up gave +13.7 MB/h and failed the 5 MB/h limit, although the process was not leaking. The analyzer also counted samples taken after the load ended, while RSS fell as the load wound down.
The first commit on this branch judged growth on the second half of the window. On the real data that made RoutrOne worse (+23.8 MB/h), because a dip at minutes 60–80 sat at the start of the second half. Decision (Pedro, 2026-11-10): use a settle period and a robust slope instead.
What
settle. It defaults to30m, or to half ofdurationwhen that is shorter, so 20-minute smoke soaks still work. An explicitsettlemust be shorter thanduration. Only memory growth skips it; the other rules count from the end of the warm-up.theilSenSlope): the median of the slopes between every pair of samples. Past 600 samples it uses an evenly spaced subset of 600, about 180,000 pairs.memoryGrowthWindow): from the end of the warm-up plussettle, to the end of the load (meta.endedAt). Samples taken after the load ends are left out.memoryGrowthMBPerHourrule gets an optionalwindow: { from, to }, and a measured soak gets an optionalsettleS. Older reports stay valid.schemas/*.jsonare regenerated.docs/perfkit-yaml.md(settle),docs/report-v1.md.Run 42699126885, recomputed
Warm-up 10 min, settle 30 min. The window runs from 23:32 or 23:33 to 01:02 or 01:03 UTC.
K8s
edgeportclimbs steadily, from 2,761 to 2,809 MB over 2 h, so it still fails.Tests
test/fixtures/soak/routr-42699126885-one-rss.json) passes, with |slope| < 1 MB/h. A least-squares fit from the warm-up on fails it.endedAt(rising 500 MB/h) are left out.settlemoves the window. The default is half of a short duration. Asettlethat is not shorter thandurationis rejected.theilSenSlope(median, a dip, subsampling, edge cases),memoryGrowthWindow,settleSeconds, the live builder, and the page note.npm test(835 passing),npm run typecheck,npm run lint(ESLint and Prettier).🤖 Generated with Claude Code
https://claude.ai/code/session_01WSt25UoKGj2mgDNQ4dHVrB