Add HTTPXWrapper behind use_httpx flag#23822
Draft
mwdd146980 wants to merge 8 commits into
Draft
Conversation
Adds an httpx-backed HTTP client wrapper that implements the HTTPClientProtocol so a check can opt in via 'use_httpx: true' in instance config. RequestsWrapper remains the default. Opts kong and kuma into the new wrapper as a proof of concept. Extends mock_http_response so the existing test fixtures intercept both requests and httpx code paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Codecov Report❌ Patch coverage is Additional details and impacted files🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fix docstring drift on mock_http_response (patches Client.request, not send). Add options['proxies'] = None for shape-parity with RequestsWrapper. Document the silent kwarg-drop policy in _build_request_kwargs. Add tests for password-only auth branch, iter_content branches, elapsed RuntimeError fallback, and AgentCheck.http httpx dispatch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Map HTTPConnectionError alongside requests ConnectionError in OM v2 scraper so ignore_connection_errors keeps working for use_httpx opt-ins - Bridge reason vs reason_phrase: prefer httpx's reason_phrase, fall back to the protocol-standard reason exposed by MockHTTPResponse - Narrow HTTPXWrapper.__del__ to AttributeError, matching RequestsWrapper - Drop unimplemented STANDARD_FIELDS keys (use_legacy_auth_encoding, tls_ignore_warning) so the config surface only lists what is honored - Strengthen response-surface tests with concrete assertions - Pin the single-shared-client invariant in test_lifecycle.py - Move use_httpx into kuma's dd_environment yield so e2e exercises the wrapper the same way kong does Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cut stdlib-passthrough tests, parametrize duplicates, remove private-attr asserts, fold test_tls.py into test_config.py, move parse_basic_auth to common.py. 87 to 52 tests, same wrapper coverage.
Contributor
Validation Report
Run Passed validations (20)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a minimum-viable
HTTPXWrapperthat implements the existingHTTPClientProtocolso checks can opt into anhttpx-backed HTTP client viause_httpx: truein instance config.RequestsWrapperremains the default for every check that does not opt in.Wires the selector into
AgentCheck.http. Adds a parallel test suite for the wrapper. Opts kong and kuma in as the first integrations to use the new wrapper. Extendsmock_http_responseto also patchhttpx.Client.requestso existing test fixtures intercept both code paths transparently.Motivation
This PR is part of a larger migration of
datadog_checks_base's HTTP layer fromrequeststohttpx. It ships only the minimum needed to prove the wrapper-swap mechanism works end-to-end through a real integration. It does NOT aim forRequestsWrapperfeature parity. The full surface (auth tokens, proxies, UDS, Kerberos / NTLM / AWS / Digest auth, connection-pool tuning, HTTP/2, multipart uploads) is deferred to follow-up PRs.Approach
MVP feature surface in
datadog_checks_base/datadog_checks/base/utils/http_httpx.py:get,post,put,delete,head,patch,options_methodHTTPXResponseAdapter:.content,.text,.headers,.status_code,.json(),.raise_for_status(),.close(),.iter_lines(),.iter_content(),.encoding,.url,.cookies,.elapsedheaders,params,json,data,timeout,verify,cert,allow_redirectsusername/passwordhttpx.*intohttp_exceptions.*so existing widened catches keep workinghttpx.Clientper check instance (matchesRequestsWrapper's session sharing)ImportErrorifhttpxis not installed (no soft-import branching)AgentCheck.httpselects betweenHTTPXWrapperandRequestsWrapperonis_affirmative(instance.get('use_httpx', False)). Default behavior is unchanged for every integration that does not opt in.Verification
datadog_checks_base/tests/base/utils/http_httpx/(51 tests, all green) covering methods, config, basic auth, TLS, exception mapping, response surface, and lifecycle.kongandkumatest suites both pass withuse_httpx: trueset in their instance fixtures, exercising the OM v2 scraper path end-to-end throughHTTPXWrapper.KumaCheck(... use_httpx=True).httpreturns anHTTPXWrapperinstance.Review checklist (to be filled by reviewers)
qa/requiredif this PR needs QA validation, orqa/skip-qaif it does not. Exactly one of the two is required.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged