Skip to content

Fix HttpServiceEndpointResolver leak: share a single resolver instead of one per HTTP handler - #7671

Open
epricer-polly wants to merge 2 commits into
dotnet:mainfrom
epricer-polly:fix/service-discovery-resolver-leak
Open

Fix HttpServiceEndpointResolver leak: share a single resolver instead of one per HTTP handler#7671
epricer-polly wants to merge 2 commits into
dotnet:mainfrom
epricer-polly:fix/service-discovery-resolver-leak

Conversation

@epricer-polly

@epricer-polly epricer-polly commented Aug 1, 2026

Copy link
Copy Markdown

Fixes #7670.

Summary

AddServiceDiscovery creates a new HttpServiceEndpointResolver for every HTTP message handler that is built. Handlers are rebuilt on the normal HttpClientFactory handler-lifetime rotation, so a fresh resolver is allocated periodically for the life of the process. Both call sites are affected:

  • the AddHttpMessageHandler delegate in AddServiceDiscovery(IHttpClientBuilder), and
  • CreateHandler in ServiceDiscoveryHttpMessageHandlerFactory.

HttpServiceEndpointResolver is IAsyncDisposable and owns endpoint-refresh timers plus configuration change-token subscriptions, but ResolvingHttpDelegatingHandler wraps it and never disposes it. Each orphaned resolver stays rooted (by the runtime timer queue and the configuration root) and never becomes eligible for collection; its live timers keep firing, which shows up as a slow, uptime-correlated CPU climb. My two-day production heap diff for a minimal service (Azure Container Apps) showed the instance count grow from 0 to 199.

Fix

Register HttpServiceEndpointResolver as a singleton. Notably this is how it's done with the ServiceEndpointResolver registration in AddServiceDiscoveryCore, and resolve it at both sites. It has no per-client state and caches watchers per service name internally, so a single shared instance is correct and is disposed once with the container.

Tests

Adds HttpServiceEndpointResolverSharingTests:

  • AddServiceDiscoveryCore_RegistersResolverAsSingleton: the resolver is registered and resolves to a single shared instance.
  • AddServiceDiscovery_HttpClient_HandlerUsesSharedResolverSingleton: the resolver captured by the built ResolvingHttpDelegatingHandler is the same container-owned singleton.

Both tests fail against the current code and pass with this change.

@epricer-polly

Copy link
Copy Markdown
Author

@dotnet-policy-service agree company="Polly Insurance, LLC"

AddServiceDiscovery creates a new HttpServiceEndpointResolver for every
HTTP message handler that is built. Handlers are rebuilt on the normal
HttpClientFactory handler-lifetime rotation, so a fresh resolver is
allocated periodically for the lifetime of the process. Both call sites
are affected: the AddHttpMessageHandler delegate in
AddServiceDiscovery(IHttpClientBuilder) and CreateHandler in
ServiceDiscoveryHttpMessageHandlerFactory.

HttpServiceEndpointResolver is IAsyncDisposable and owns endpoint-refresh
timers plus configuration change-token subscriptions. ResolvingHttp-
DelegatingHandler wraps it but never disposes it, so each orphaned
resolver stays rooted (by the runtime timer queue and the configuration
root) and never becomes eligible for collection. Its live timers keep
doing work, which shows up as a slow, uptime-correlated CPU climb. A
two-day production heap diff showed the count grow from 0 to 199.

Register HttpServiceEndpointResolver as a singleton, mirroring the
existing ServiceEndpointResolver registration, and resolve it at both
sites. It has no per-client state and caches watchers per service name
internally, so a single shared instance is correct and is disposed once
with the container.
@epricer-polly
epricer-polly force-pushed the fix/service-discovery-resolver-leak branch from 2142499 to 5db5c8d Compare August 1, 2026 02:46
@epricer-polly
epricer-polly marked this pull request as ready for review August 1, 2026 03:37
Copilot AI review requested due to automatic review settings August 1, 2026 03:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an uptime-correlated leak/CPU climb in Microsoft.Extensions.ServiceDiscovery by ensuring HttpServiceEndpointResolver is shared (singleton) instead of being reallocated on every HttpClientFactory handler rotation. This aligns the HTTP resolver’s lifetime with the container so its timers and configuration subscriptions are disposed once with the service provider.

Changes:

  • Register HttpServiceEndpointResolver as a singleton in AddServiceDiscoveryCore.
  • Update both HTTP handler construction paths to resolve and reuse the singleton resolver.
  • Add tests validating singleton registration and that ResolvingHttpDelegatingHandler uses the shared resolver instance.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
test/Libraries/Microsoft.Extensions.ServiceDiscovery.Tests/HttpServiceEndpointResolverSharingTests.cs Adds tests asserting the resolver is registered as a singleton and reused by the HTTP handler.
src/Libraries/Microsoft.Extensions.ServiceDiscovery/ServiceDiscoveryServiceCollectionExtensions.cs Registers HttpServiceEndpointResolver as a singleton to prevent per-rotation allocations/leaks.
src/Libraries/Microsoft.Extensions.ServiceDiscovery/ServiceDiscoveryHttpClientBuilderExtensions.cs Updates AddServiceDiscovery(IHttpClientBuilder) to use the singleton resolver instead of instantiating a new one per handler build.
src/Libraries/Microsoft.Extensions.ServiceDiscovery/Http/ServiceDiscoveryHttpMessageHandlerFactory.cs Updates factory to inject and reuse the singleton resolver when creating handlers.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HttpServiceEndpointResolver leaked per HTTP handler build in Microsoft.Extensions.ServiceDiscovery

2 participants