Skip to content

perf: avoid reordering the range cache on every hit - #893

Open
jeet-dhandha wants to merge 1 commit into
npm:mainfrom
jeet-dhandha:perf/lrucache-two-generation
Open

perf: avoid reordering the range cache on every hit#893
jeet-dhandha wants to merge 1 commit into
npm:mainfrom
jeet-dhandha:perf/lrucache-two-generation

Conversation

@jeet-dhandha

Copy link
Copy Markdown

Range.parseRange is memoized in internal/lrucache.js. On every cache hit, the current implementation does map.delete(key) + map.set(key, value) to move the entry to the end of the insertion order. That reorder is pure overhead on a hot path.

This replaces the single map with two generations and drops the reorder. A hit in the old generation is promoted back into the live one; when the live generation fills, it becomes the old one and the previous old generation is dropped.

parseRange is deterministic, and the memo key already covers the range string plus both flags that affect parsing, so the eviction policy cannot change any result — only whether a given call re-parses. I verified that directly: 43,200 cases across satisfies / validRange / maxSatisfying / minSatisfying / minVersion / intersects / subset, each under {}, {loose}, {includePrerelease}, and both, produce byte-identical output before and after.

Benchmarks on node v24.8.0, one variant per process, best-of-5 within a process, min across 5 rounds. Corpus is every distinct range and version in the eslint / express / lodash / react / typescript / webpack packuments (1,965 ranges, 7,947 versions):

shape speedup
benchmarks/bench-satisfies.js as written 1.72x
400 ranges x 300 versions 2.05x
5 ranges x 7,900 versions 3.52x
1,960-range dependency tree 1.09x
hot set of 900 ranges + cold stream 1.10x
maxSatisfying over 7,900 versions 1.01x

No shape I tried is slower than stock. The retained ceiling goes from 1,000 entries (~0.78 MB) to 2,000 (~1.88 MB), measured by flooding 80,000 distinct ranges and sampling retained heap.

Two things worth flagging for review:

  • test/internal/lrucache.js asserted that the oldest key is gone immediately after the first over-cap insert, which is specific to strict LRU. I replaced it with tests for the properties the cache actually needs: value round-trip, set(key, undefined) as a no-op, delete clearing both generations, eviction under flood, and the 2 * max bound holding under promotion.
  • The class is no longer an LRU. I left the name and filename alone to keep the diff small, but happy to rename if you'd prefer.

For context on why the second generation is there rather than just deleting the reorder: dropping the reorder alone turns the cache into FIFO, which evicts a hot set that strict LRU would pin. On a hot set of 900 ranges just under the 1,000 cap, that regresses to 0.40x. The second generation is what avoids that.

@jeet-dhandha
jeet-dhandha requested a review from a team as a code owner July 31, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant