Fix #1632: fall back to certifi when the OS trust store is empty (macOS) - #1933
Closed
vinayK34 wants to merge 2 commits into
Closed
Fix #1632: fall back to certifi when the OS trust store is empty (macOS)#1933vinayK34 wants to merge 2 commits into
vinayK34 wants to merge 2 commits into
Conversation
HTTPie always uses its own `SSLContext` (needed for `--ssl`/`--ciphers`), so Requests intentionally skips loading any CA bundle for `--verify=yes` and we load the defaults ourselves via `ensure_default_certs_loaded()`. That relied solely on `SSLContext.load_default_certs()`, which uses OpenSSL's default verify paths. On the python.org macOS builds those paths are empty, so the context ended up with an *empty* trust store and every HTTPS request failed with CERTIFICATE_VERIFY_FAILED β even though `requests` in the same environment worked fine, because it defaults to the certifi bundle. Now, if the OS trust store yields no certificates, we fall back to `requests.adapters.DEFAULT_CA_BUNDLE_PATH` (certifi), matching what Requests itself trusts by default.
`test_ensure_default_certs_loaded_falls_back_to_certifi` fails against the previous implementation (empty trust store) and passes with the fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1632
Problem
On the python.org macOS builds, every HTTPS request fails out of the box:
β¦while
requestsin the same virtualenv works fine, andhttps --verify "$(python -m certifi)" ...also works. So it's not a missingcertifiβ HTTPie just isn't finding a trust store.Root cause
HTTPie always installs its own
SSLContext(HTTPieHTTPSAdapter._create_ssl_context, needed to support--ssl/--ciphers). Requests deliberately loads no CA bundle whenverify is Trueand a custom context is in play β seeHTTPAdapter.cert_verify:So loading the defaults is entirely on us, via
compat.ensure_default_certs_loaded()(added for #1583). That function relied solely onSSLContext.load_default_certs(), which uses OpenSSL's default verify paths. On the python.org macOS builds those paths point at a location OpenSSL can't use, soload_default_certs()is a no-op and the context is left with an empty trust store β hence "unable to get local issuer certificate" for every host.requestsdoesn't hit this because its own default iscertifi(DEFAULT_CA_BUNDLE_PATH), not the OpenSSL default paths.Fix
If the OS trust store yields no certificates, fall back to
requests.adapters.DEFAULT_CA_BUNDLE_PATH(the certifi bundle) β i.e. trust exactly what Requests itself trusts by default:Deliberately conservative:
certifiis already a hard dependency ofrequests, andDEFAULT_CA_BUNDLE_PATHis what Requests uses itself.--verify=noand--verify=<path>paths are untouched.Verification
Reproduced and verified in a sandbox with
httpie 3.2.4+requests 2.32.3, forcing OpenSSL's default verify paths to an empty location to emulate the macOS build (SSL_CERT_FILE/SSL_CERT_DIRβ empty dir), withcertifiinstalled:requests.get('https://example.com')python -m httpie GET https://example.comHTTP/1.1 200 OKAdapter-level check after the fix, with the OS store empty:
HTTPieHTTPSAdapter._create_ssl_context(verify=True)β 121 CA certs loaded,verify_mode=CERT_REQUIRED;verify=FalseβCERT_NONE(unchanged).--verify=noand a normal environment with a working OS store both still return200 OK.tests/test_compat.pyadds 4 tests, incl. the regression case and edge cases (already-loaded certs are not reloaded; a missing certifi bundle doesn't raise).test_ensure_default_certs_loaded_falls_back_to_certififails against the previous implementation (asserts on an emptyget_ca_certs()) and passes with this change β confirmed by reverting the function and re-running:1 failed, 3 passedβ4 passed.flake8clean on both changed files.