Fix/bound compound count#46953
Open
j7nw4r wants to merge 8 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens the internal PyAMQP decoders in azure-servicebus and azure-eventhub by introducing a maximum allowed element count for large AMQP compound types (list32/map32/array32) and adding unit tests to validate the new bounds.
Changes:
- Added
_MAX_COMPOUND_COUNT = 65536and enforced it in_decode_list_large,_decode_map_large, and_decode_array_largefor both Service Bus and Event Hubs. - Added new unit tests covering oversized COUNT rejection and boundary acceptance in both packages.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| sdk/servicebus/azure-servicebus/azure/servicebus/_pyamqp/_decode.py | Adds _MAX_COMPOUND_COUNT and enforces it for list32/map32/array32 decoding. |
| sdk/servicebus/azure-servicebus/tests/unittests/test_decode_bounds.py | New tests validating oversized COUNT rejection and boundary behavior for decoders. |
| sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_decode.py | Same compound COUNT cap and validations as Service Bus. |
| sdk/eventhub/azure-eventhub/tests/pyamqp_tests/unittest/test_decode_bounds.py | Same decoder bounds tests as Service Bus, for Event Hubs. |
Comments suppressed due to low confidence (2)
sdk/servicebus/azure-servicebus/azure/servicebus/_pyamqp/_decode.py:277
_decode_map_largeclaims the pre-halving check "catches hostile odd values", but the code doesn’t validate that the on-wire count is even. For an odd COUNT the currentint(raw_count / 2)will silently floor and leave trailing bytes in the buffer, which can corrupt subsequent decoding. Consider rejecting odd counts (e.g.,raw_count % 2 != 0) and using integer division (raw_count // 2) to avoid float truncation.
if raw_count > _MAX_COMPOUND_COUNT:
raise ValueError(
f"AMQP map element count {raw_count} exceeds maximum {_MAX_COMPOUND_COUNT}"
)
count = int(raw_count / 2)
sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_decode.py:277
_decode_map_largeclaims the pre-halving check "catches hostile odd values", but the code doesn’t validate that the on-wire count is even. For an odd COUNT the currentint(raw_count / 2)will silently floor and leave trailing bytes in the buffer, which can corrupt subsequent decoding. Consider rejecting odd counts (e.g.,raw_count % 2 != 0) and using integer division (raw_count // 2) to avoid float truncation.
if raw_count > _MAX_COMPOUND_COUNT:
raise ValueError(
f"AMQP map element count {raw_count} exceeds maximum {_MAX_COMPOUND_COUNT}"
)
count = int(raw_count / 2)
271e834 to
c4f0192
Compare
Member
Author
|
The two failing CI checks on this PR are unrelated to the diff and tracked in separate fixes:
Once #46987 and #46988 land, I'll rebase this branch onto |
j7nw4r
added a commit
that referenced
this pull request
May 20, 2026
…#46987) The test previously measured wall-clock time around real time.sleep(0.8) calls and asserted the duration was strictly less than 1.6 seconds. On slow CI agents (notably macOS hosts) the elapsed time can exceed the budget by a few ms (e.g., 1.6399s observed in PR #46953), causing intermittent failures. Mock time.sleep in azure.servicebus._base_handler and assert the value passed to sleep instead. This is deterministic and ~80x faster. Co-authored-by: Johnathan Walker <johwalker@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
decode_frame() decoded the list32 COUNT field as a signed int and skipped the _MAX_COMPOUND_COUNT cap that already protects the per-type _decode_list_large / _decode_map_large / _decode_array_large sites. A malicious peer could send a frame advertising a multi-billion field count, driving `fields = [None] * count` into a multi-gigabyte allocation before the body is even consumed. Switch to c_unsigned_long (the AMQP 1.0 wire definition for list32 COUNT) and apply the existing _MAX_COMPOUND_COUNT cap on the same path. Add negative-path coverage to both vendored copies.
_decode_map_small and _decode_map_large both used `int(raw_count / 2)`, which silently floors odd counts and leaves a trailing key with no value. The half-decoded pair leaks bytes into the next decoder and corrupts subsequent values on the wire. Reject odd raw counts explicitly and switch to integer division. The cap in _decode_map_large already protects against resource exhaustion; this patch is purely a correctness fix for the small variant and a robustness fix for the large variant. Negative-path tests added to both vendored copies.
Resolves 16 mypy errors that surface in CI (e.g., PR Azure#46953) without changing runtime behavior: - _common.py: widen message_id/content_type/correlation_id setters to Optional[str] so EventData.__init__ can clear them, narrow Optional accesses on annotations/application_properties, and silence one residual update() arg-type with a typed ignore. - _transport/_base.py, aio/_transport/_base_async.py: keep_alive_interval is Optional[int] (None means 'no keep-alive') and create_source.offset is Optional[Union[int, str, datetime.datetime]] (matches event_position). - _transport/_pyamqp_transport.py, _uamqp_transport.py, and async equivalents: align keep_alive_interval signatures with the new abstract type to satisfy Liskov. - _utils.event_position_selector: accept Optional value (already handled at runtime via the else branch). No source files imported by user-facing API surfaces change behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
c4f0192 to
466e8de
Compare
EldertGrootenboer
approved these changes
May 20, 2026
j7nw4r
pushed a commit
that referenced
this pull request
May 21, 2026
Resolves 16 mypy errors that surface in CI (e.g., PR #46953) without changing runtime behavior: - _common.py: widen message_id/content_type/correlation_id setters to Optional[str] so EventData.__init__ can clear them, narrow Optional accesses on annotations/application_properties, and silence one residual update() arg-type with a typed ignore. - _transport/_base.py, aio/_transport/_base_async.py: keep_alive_interval is Optional[int] (None means 'no keep-alive') and create_source.offset is Optional[Union[int, str, datetime.datetime]] (matches event_position). - _transport/_pyamqp_transport.py, _uamqp_transport.py, and async equivalents: align keep_alive_interval signatures with the new abstract type to satisfy Liskov. - _utils.event_position_selector: accept Optional value (already handled at runtime via the else branch). No source files imported by user-facing API surfaces change behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validate the COUNT field of compound AMQP types (array / list / map) during decode in pyamqp's
_decode.py. Without these checks a malicious peer can send anArray32/List32/Map32whose header advertises a multi-billion element count paired with a zero-width or 1-byte element constructor, driving the decoder into an unbounded loop and/or excessive list allocation before the body is even consumed.Both vendored copies of pyamqp are updated so eventhub and servicebus receive the fix in lockstep:
sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/_decode.pysdk/servicebus/azure-servicebus/azure/servicebus/_pyamqp/_decode.pyMatching test modules are added under each SDK's tests tree.
Behavior
Two checks are applied before any per-element work:
Array32ofnull/true/false). This is intentional.The cap is enforced at every large-variant compound decode site and at the outer
decode_frame()list32 path, so a hostile peer cannot bypass it via the control-frame field list.Commits
sdk/eventhub/azure-eventhub/tests/pyamqp_tests/unittest/test_decode_bounds.pysdk/servicebus/azure-servicebus/tests/unittests/test_decode_bounds.pydecode_frame()—decode_frame()decoded the list32 COUNT as a signed int and skipped the_MAX_COMPOUND_COUNTcap, leaving the outer frame field-list path open to the same allocation attack. Switches toc_unsigned_long(matching the AMQP 1.0 wire spec for list32 COUNT) and applies the cap. Addresses Copilot review feedback._decode_map_small/_decode_map_largeusedint(raw_count / 2), which silently floored odd counts and left a trailing key with no value, leaking bytes into the next decoder. Now rejects odd counts explicitly and uses integer division. Addresses Copilot review feedback.Test plan
pytest sdk/eventhub/azure-eventhub/tests/pyamqp_tests/unittest/test_decode_bounds.pypytest sdk/servicebus/azure-servicebus/tests/unittests/test_decode_bounds.pyWhy duplicate the fix in two trees
_pyamqpis currently vendored separately into eventhub and servicebus rather than shared. Touching both copies in one PR keeps the two decoders from drifting on a security-relevant check. Happy to split into two PRs if Event Hubs and Service Bus owners prefer.Risk
Low. Strictly additional rejection of malformed frames; well-formed traffic decodes identically.