Skip to content

Fix expired hourly query retention#2060

Open
zs311521 wants to merge 1 commit into
TechnitiumSoftware:masterfrom
zs311521:reduce-expired-hourly-query-retention
Open

Fix expired hourly query retention#2060
zs311521 wants to merge 1 commit into
TechnitiumSoftware:masterfrom
zs311521:reduce-expired-hourly-query-retention

Conversation

@zs311521

@zs311521 zs311521 commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • fix a reproduced retention defect in expired hourly query-prefetch state
  • release _hourStat._queries when minute detail ages out of the last-hour window
  • preserve scalar totals, dashboard maps, on-disk minute data, and repeated cleanup

Related to #2030. This fixes the isolated defect described below; it does not claim that every source of RSS growth in #2030 is resolved.

Root cause

Each locked minute counter is merged into HourlyStats._hourStat, including its high-cardinality _queries map. After an hour leaves the last-hour window, UnloadMinuteStats() releases the 60 minute counters but leaves that merged query map in the hourly cache for up to 24 hours.

Cache prefetch builds its last-hour view from _lastHourStatCountersCopy. The dashboard paths retain their scalar totals and separate domain/client/protocol maps. I found no live reader that needs expired _hourStat._queries after minute detail is unloaded; maintainer confirmation of that boundary is welcome.

The maintenance threshold excludes the current and previous hour. Hourly files are written from the minute counters before this older cache entry is unloaded, so the change does not rewrite or truncate the persisted hourly file.

Why this shape

The patch clears only _queries, a ConcurrentDictionary; all reporting counters and maps remain intact. It deliberately keeps repeated UnloadMinuteStats() cleanup: if an unusually late maintenance overlap adds a query after one pass, the next pass removes it instead of returning early.

The more invasive alternative—never merging queries into _hourStat—would lower peak duplication further but has a wider behavior surface. Clear-on-unload is the conservative fix.

Verification

  • synthetic 5,000-unique-query lifecycle regression:
    • stock v15.4: 5,000 expired query keys remain
    • patched: 0 expired query keys remain
    • scalar query totals and reporting-domain cardinality are unchanged
    • a key introduced after the first unload is removed by the next pass
  • current-hour/sentinel entries are outside the unload threshold
  • exact v15.4 TechnitiumLibrary dependency (dns-server-v15.4.0)
  • .NET SDK 10.0.302 Release build: 0 warnings, 0 errors

Production corroboration

On a busy v15.4 forwarder using concurrent DNS-over-QUIC upstreams, RSS rose from 758.4 MiB to 6,777.9 MiB over 15 hours (about 401 MiB/hour by endpoint slope). An equivalently configured mostly-idle standby stayed near baseline, and a low-volume standalone node remained near 740 MiB after about nine days.

As a short mitigation experiment, enabling in-memory stats and restarting held the busy node to 872.4 MiB at 1 hour 49 minutes. Depending on the comparison window, observed growth was roughly 69–100 MiB/hour rather than ~401 MiB/hour. This is strong evidence that the disk-backed stats path is a major contributor, but the window is short, the restart is a confounder, and residual growth may have other managed or native causes. That is why this PR remains scoped to the reproduced defect and does not close #2030.

@zs311521 zs311521 mentioned this pull request Jul 22, 2026
@zs311521 zs311521 changed the title Reduce expired hourly query retention Fix expired hourly query retention Jul 22, 2026
@ShreyasZare

Copy link
Copy Markdown
Member

Thanks for the PR. Will evaluate this in detail soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory Leak

2 participants