Better exception when missing optional dependencies (like Brotli)

[MicaelJarniac]

I've recently spent an unreasonable amount of time hunting a bug in my code, that ended up being that for whatever reason an API was returning br compressed data, and I didn't have the brotli optional installed.

The error I was getting was extremely unhelpful.

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "httpx",
# ]
# ///


import httpx


async def test_brotli() -> None:
    async with httpx.AsyncClient(base_url="https://httpbin.io") as client:
        resp = await client.get(
            "/brotli",
        )
        resp.raise_for_status()
        print("status:", resp.status_code)
        print("content-type:", resp.headers.get("content-type"))
        print("content-encoding:", resp.headers.get("content-encoding"))
        print("first 16 bytes:", resp.content[:16])
        print(resp.json())


if __name__ == "__main__":
    import asyncio

    asyncio.run(test_brotli())

❯ uv run --script ./httpxme.py
status: 200
content-type: application/json; charset=utf-8
content-encoding: br
first 16 bytes: b'\x1b_\x01\x00deN_\xa3\x9a\xca!\xb9\x9f\xe3\x7f'
Traceback (most recent call last):
  File "/Users/micael/projects/playground/./httpxme.py", line 28, in <module>
    asyncio.run(test_brotli())
    ~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/micael/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/Users/micael/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/micael/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/base_events.py", line 719, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/micael/projects/playground/./httpxme.py", line 22, in test_brotli
    print(resp.json())
          ~~~~~~~~~^^
  File "/Users/micael/.cache/uv/environments-v2/httpxme-b8dc774a8de1adab/lib/python3.13/site-packages/httpx/_models.py", line 832, in json
    return jsonlib.loads(self.content, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/micael/.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/json/__init__.py", line 341, in loads
    s = s.decode(detect_encoding(s), 'surrogatepass')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 8: invalid start byte

What would help is if the error was like MissingDependency: 'br' encoded content received, but 'brotli' dependency not installed or something along those lines.

[amasen02]

Why this happens: Silent Fallback to IdentityDecoder

This occurs because of how httpx resolves Content-Encoding when optional compression packages are missing:

1. Why the server sent br without brotli installed

By default, httpx builds its Accept-Encoding request header dynamically based on available decoders:

• If brotli is installed: Accept-Encoding: gzip, deflate, br, zstd
• If brotli is not installed: Accept-Encoding: gzip, deflate

However, some endpoints (such as httpbin.org/brotli, or misconfigured origin servers/reverse proxies) unconditionally respond with Content-Encoding: br regardless of what the client advertised in Accept-Encoding.

2. The Silent Fallback

When httpx.Response reads Content-Encoding: br:
In httpx/_decoders.py, SUPPORTED_DECODERS only maps encodings whose dependencies are present in the current Python environment:

SUPPORTED_DECODERS = {
    "gzip": GZipDecoder,
    "deflate": DeflateDecoder,
}
if brotli is not None or brotlicffi is not None:
    SUPPORTED_DECODERS["br"] = BrotliDecoder
if zstandard is not None:
    SUPPORTED_DECODERS["zstd"] = ZStandardDecoder

When a server sends Content-Encoding: br but brotli is missing:

1. httpx does not find "br" in SUPPORTED_DECODERS.
2. Instead of raising an exception, it silently defaults to raw passthrough (IdentityDecoder).
3. resp.content contains the raw binary compressed Brotli stream (as seen in your snippet: b'\x1b_\x01\x00deN_\xa3\x9a\xca...').
4. When resp.json() or resp.text attempts to decode those raw compressed bytes as UTF-8, Python fails with UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3, masking the real issue.

---

Immediate Workarounds

1. Install the optional extra:

   pip install "httpx[brotli]"
   # or with uv
   uv add "httpx[brotli]"

2. If dealing with an origin that ignores Accept-Encoding:
   You can assert or check the encoding header before parsing:

   if resp.headers.get("content-encoding") == "br" and "br" not in httpx._decoders.SUPPORTED_DECODERS:
       raise RuntimeError("Server forced Brotli compression but 'brotli' is not installed.")

---

Proposed Upstream Improvement for httpx

Rather than silently falling back to IdentityDecoder for known compression formats, httpx should distinguish between:

• Unknown custom encodings (which might be user-handled custom streams), and
• Standard IANA content encodings (br, zstd) whose corresponding optional packages are simply missing.

In httpx/_decoders.py:

KNOWN_OPTIONAL_DECODERS = {
    "br": ("brotli", "pip install httpx[brotli]"),
    "zstd": ("zstandard", "pip install httpx[zstd]"),
}

def get_decoder(encoding: str) -> ContentDecoder:
    if encoding in SUPPORTED_DECODERS:
        return SUPPORTED_DECODERS[encoding]()
    if encoding in KNOWN_OPTIONAL_DECODERS:
        pkg, hint = KNOWN_OPTIONAL_DECODERS[encoding]
        raise httpx.DecodingError(
            f"Received response compressed with '{encoding}', but optional dependency '{pkg}' "
            f"is not installed. Install with '{hint}'."
        )
    return IdentityDecoder()

This would transform cryptic UnicodeDecodeError / JSON decode crashes into an actionable httpx.DecodingError.

[VimalN2005]

Hi! The root cause here is that in Response._get_content_decoder(), if a response specifies Content-Encoding: br or zstd and the optional package is not installed, SUPPORTED_DECODERS raises a KeyError which is silently caught, causing httpx to fall back to IdentityDecoder. This returns raw compressed binary bytes and causes confusing JSONDecodeError / UnicodeDecodeError downstream.

I have prepared a solution where known optional encodings (br, zstd) raise an explicit DecodingError guiding the user to install the required extra (pip install httpx[brotli] or pip install httpx[zstd]), along with regression tests.

Since pull requests are currently restricted to collaborators, please let me know if you would like a PR for this!