PYTHON-5781 Coverage increase for `network_layer.py` by aclark4life · Pull Request #2774 · mongodb/mongo-python-driver

aclark4life · 2026-04-23T21:59:00Z

Changes in this PR

Adds unit tests for pymongo/network_layer.py. The async and sync receive paths are structured differently and share no common implementation, so coverage is split into two single-API files, neither of which goes through synchro:

test/asynchronous/test_async_network_layer.py (async-only; registered in async_only_test() in tools/synchro.py) — PyMongoProtocol, _async_socket_receive.
test/test_network_layer.py (sync-only; hand-maintained, not generated) — receive_message, receive_data.

Why two un-synchro'd files (cf. #2771)? Per the rule established in #2771, the only tests that should not go through synchro are those that don't change between sync/async or that test modules shared between them. Here the two APIs have no shared implementation to mirror: the async path reads via PyMongoProtocol (a BufferedProtocol) / _async_socket_receive, while the sync path reads via the plain receive_message / receive_data functions. Each has no counterpart in the other API, so synchro has nothing to generate — each set of tests belongs to its own API, matching the existing test_async_loop_safety.py convention on the async side.

Tests cover:

Async — PyMongoProtocol

process_header() — OP_MSG, OP_REPLY, OP_COMPRESSED, and the three ProtocolError paths (compressed length ≤ 25, uncompressed length ≤ 16, length > max message size).
process_compression_header() — snappy and zlib compressor IDs.
buffer_updated() — completes pending future on full message.
close() / connection_lost() — abort transport, exception propagation to pending futures, idempotency on repeated connection_lost.

Async — _async_socket_receive()

Multi-chunk accumulation (covers the read loop).
Raises OSError("connection closed") when the socket returns 0 bytes.

Sync — receive_message()

request_id mismatch, length ≤ 16, and length > max-message-size ProtocolError paths.
OP_COMPRESSED decompression path and normal-opcode unpack.
unknown-opcode ProtocolError.

Sync — receive_data()

Multi-chunk accumulation (covers the read loop).
Raises OSError("connection closed") when recv_into returns 0 bytes.

Earlier revisions also tested NetworkingInterfaceBase and PyMongoProtocol's timeout getter/setter; those only asserted NotImplementedError on abstract stubs and trivial attribute round-trips — implementation, not behavior — so they were dropped.

Test Plan

Added unit tests using MagicMock transports/sockets/connections and the AsyncUnitTest event-loop harness for the asyncio-dependent protocol code. No live socket or MongoDB server required.

uv run --extra test python -m pytest test/test_network_layer.py test/asynchronous/test_async_network_layer.py -v

Checklist

Checklist for Author

Did you update the changelog (if necessary)?
Is there test coverage?
Is any followup work tracked in a JIRA ticket? If so, add link(s).

Checklist for Reviewer

Does the title of the PR reference a JIRA Ticket?
Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
Is all relevant documentation (README or docstring) updated?

Copilot

Pull request overview

Adds a dedicated unit test suite to increase coverage of pymongo/network_layer.py, focusing on logic that can be validated without live sockets or a running MongoDB server.

Changes:

Introduces test/test_network_layer.py with unit tests for PyMongoProtocol state transitions and message parsing helpers.
Adds delegation/behavior tests for NetworkingInterfaceBase, NetworkingInterface, and AsyncNetworkingInterface.
Adds tests for small helper functions (sendall, _async_socket_receive) using mocks.

codecov-commenter · 2026-04-23T22:34:57Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

caseyclements

LGTM

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

NoahStapp · 2026-05-06T13:24:57Z

+
+
+class TestSendall(AsyncUnitTest):
+    def test_delegates_to_sock_sendall(self):


What's the purpose of this test? Testing that our wrapper calls what it wraps doesn't seem to test useful behavior.

NoahStapp · 2026-05-06T13:25:44Z

+
+
+class TestNetworkingInterfaceBase(AsyncUnitTest):
+    def setUp(self):


Suggested change

def setUp(self):

def asyncSetUp(self):

NoahStapp · 2026-05-06T13:26:33Z

+        mock_socket.sendall.assert_called_once_with(b"hello")
+
+
+class TestNetworkingInterfaceBase(AsyncUnitTest):


This class doesn't need to be synchro'd since it's testing code common to both APIs.

NoahStapp · 2026-05-06T13:28:42Z

+            _ = self.base.sock
+
+
+class TestNetworkingInterface(AsyncUnitTest):


Are we gaining useful coverage by testing that all of NetworkingInterface's methods delegate to the underlying socket instead of testing the actual result of these calls?

NoahStapp · 2026-05-06T13:34:08Z

+        self.assertIs(self.network_interface.sock, self.mock_socket)
+
+
+if not _IS_SYNC:


All of this code will be duplicated but unused in the synchronized version of this file. A better pattern is to put all of the async-specific tests that don't produce a meaningful synchronous version into a test/asynchronous/test_async_network_layer.py file that is not synchro'd.

This file would then become tests common to both APIs (like the NetworkingInterfaceBase tests) as well as the synchronous tests. Then there would be no code duplication and more isolated test modules.

NoahStapp · 2026-05-06T13:37:13Z

+            self.assertIsNone(protocol.gettimeout)
+
+        async def test_normal_op_msg(self):
+            header = _make_header(32, 1, 99, 2013)


Using keyword arguments for these _make_header calls would be more readable (less magic numbers).

NoahStapp · 2026-05-06T13:39:55Z

+            self.assertEqual(op_code, 1)
+            self.assertFalse(expecting_compression)
+
+        async def test_compression_header_returns_op_code_and_compressor_id(self):


Should this be named something like test_compression_header_snappy_compressor_id for consistency with the following test?

NoahStapp · 2026-05-06T13:41:24Z

+            with self.assertRaises(OSError) as ctx:
+                await future
+            self.assertIn("connection reset", str(ctx.exception))


Suggested change

with self.assertRaises(OSError) as ctx:

await future

self.assertIn("connection reset", str(ctx.exception))

with self.assertRaisesRegex(OSError, "connection reset"):

await future

NoahStapp · 2026-05-06T13:44:08Z

+            self.assertIn("connection reset", str(ctx.exception))
+
+    class TestAsyncSocketReceive(AsyncUnitTest):
+        async def test_reads_full_data_in_one_call(self):


This seems to be primarily testing the behavior of loop.sock_recv_into(), which is a Python built-in method. Are we gaining useful coverage with this test?

NoahStapp · 2026-05-06T13:44:49Z

+                result = await _async_socket_receive(mock_socket, length, loop)
+            self.assertEqual(bytes(result), data)
+
+        async def test_reads_data_in_multiple_chunks(self):


Same question here.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Add test/test_network_layer.py with 56 unit tests covering: - PyMongoProtocol: process_header (including ProtocolError paths), process_compression_header, get_buffer (all state branches), buffer_updated (state machine), close/connection_lost lifecycle - NetworkingInterfaceBase: abstract method NotImplementedError raises - NetworkingInterface: socket delegation methods - AsyncNetworkingInterface: transport/protocol delegation - sendall: trivial delegation - _async_socket_receive: success and connection-closed paths

- Fix mypy type error in test_network_layer.py - Use 'from test import unittest', remove self-evident docstrings from helpers - Fix inaccurate comment: second field in compression header is uncompressed_size

- Add test/asynchronous/test_network_layer.py as the async source of truth with AsyncUnitTest and _IS_SYNC = False - Regenerate test/test_network_layer.py via synchro from the async source - Register test_network_layer.py in synchro converted_tests (alpha order) - Add "AsyncMock": "MagicMock" replacement after AsyncMockPool to avoid substring collision in translate_docstrings

Applying feedback from PYTHON-5784 to all open codecov PRs

…_data coverage Remove the NetworkingInterfaceBase NotImplementedError tests and the PyMongoProtocol timeout getter/setter tests (implementation, not behavior), along with the now-empty synchro'd test_network_layer.py pair and its synchro.py registration (including the unused AsyncMock->MagicMock rule). Add a hand-maintained sync-only test/test_network_layer.py covering receive_message and receive_data, mirroring the async-only protocol tests. Neither file goes through synchro: the async (PyMongoProtocol) and sync (receive_message) paths share no implementation to mirror.

On PyPy, receive_data takes the wait_for_read() path before calling recv_into(). wait_for_read() checks sock.fileno() == -1 as an early exit; without a real return value, sock.pending() > 0 raises TypeError on PyPy (MagicMock comparison against int is not supported). Set fileno() = -1 on the mock so wait_for_read returns immediately.

aclark4life requested a review from a team as a code owner April 23, 2026 21:59

aclark4life requested review from caseyclements and Copilot April 23, 2026 21:59

Copilot started reviewing on behalf of aclark4life April 23, 2026 21:59 View session

Copilot AI reviewed Apr 23, 2026

View reviewed changes

Comment thread test/test_network_layer.py Outdated

aclark4life changed the title ~~PYTHON-5781 Increase coverage for network_layer.py~~ PYTHON-5781 Increase code coverage for network_layer.py Apr 24, 2026

caseyclements previously approved these changes Apr 29, 2026

View reviewed changes

aclark4life mentioned this pull request Apr 29, 2026

PYTHON-5784 Coverage increase for periodic_executor.py #2771

Merged

6 tasks

aclark4life dismissed caseyclements’s stale review via 67e9992 May 1, 2026 21:53

aclark4life requested a review from Copilot May 1, 2026 21:54

Copilot started reviewing on behalf of aclark4life May 1, 2026 21:55 View session

Copilot AI reviewed May 1, 2026

View reviewed changes

Comment thread tools/synchro.py Outdated

aclark4life force-pushed the PYTHON-5781 branch 7 times, most recently from c6991e0 to 72fd1a4 Compare May 6, 2026 00:42

NoahStapp requested changes May 6, 2026

View reviewed changes

aclark4life marked this pull request as draft May 6, 2026 18:18

aclark4life force-pushed the PYTHON-5781 branch from c24a85d to cdc07e3 Compare May 6, 2026 18:26

aclark4life requested a review from Copilot May 6, 2026 19:48

Copilot started reviewing on behalf of aclark4life May 6, 2026 19:49 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

Comment thread test/test_network_layer.py Outdated

aclark4life force-pushed the PYTHON-5781 branch from 39a360c to 151e39f Compare May 6, 2026 21:41

aclark4life requested a review from Copilot May 7, 2026 02:01

aclark4life changed the title ~~PYTHON-5781 Increase code coverage for network_layer.py~~ PYTHON-5781 Improve code coverage for network_layer.py May 7, 2026

aclark4life changed the title ~~PYTHON-5781 Improve code coverage for network_layer.py~~ PYTHON-5781 Improve coverage for network_layer.py May 7, 2026

aclark4life changed the title ~~PYTHON-5781 Improve coverage for network_layer.py~~ PYTHON-5781 Improve coverage for network_layer.py May 7, 2026

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread test/test_network_layer.py Outdated

aclark4life changed the title ~~PYTHON-5781 Improve coverage for network_layer.py~~ PYTHON-5781 Coverage improvements for network_layer.py May 7, 2026

aclark4life force-pushed the PYTHON-5781 branch from 54630a6 to a696390 Compare May 7, 2026 02:19

aclark4life changed the title ~~PYTHON-5781 Coverage improvements for network_layer.py~~ PYTHON-5781 Coverage improvement for network_layer.py May 7, 2026

aclark4life requested a review from Copilot May 7, 2026 02:27

Copilot started reviewing on behalf of aclark4life May 7, 2026 02:28 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread test/test_network_layer.py Outdated

Comment thread test/asynchronous/test_network_layer.py Outdated

aclark4life changed the title ~~PYTHON-5781 Coverage improvement for network_layer.py~~ PYTHON-5781 Coverage increase for network_layer.py May 7, 2026

aclark4life force-pushed the PYTHON-5781 branch from 8af10d4 to 33c8039 Compare May 7, 2026 02:44

aclark4life requested a review from Copilot May 7, 2026 03:01

Copilot started reviewing on behalf of aclark4life May 7, 2026 03:01 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread test/test_network_layer.py Outdated

Comment thread test/asynchronous/test_network_layer.py Outdated

aclark4life requested a review from Copilot May 7, 2026 03:07

Copilot started reviewing on behalf of aclark4life May 7, 2026 03:08 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

aclark4life force-pushed the PYTHON-5781 branch from 33c8039 to cea9d15 Compare May 29, 2026 23:45

aclark4life added 7 commits June 4, 2026 11:35

PYTHON-5781 Nit fixes

cb77bb6

- Fix mypy type error in test_network_layer.py - Use 'from test import unittest', remove self-evident docstrings from helpers - Fix inaccurate comment: second field in compression header is uncompressed_size

Address Noah review by proxy

23f10b7

Applying feedback from PYTHON-5784 to all open codecov PRs

Rename vars

143cee8

Update OP_COMPRESSED comment

8fc0673

Noah + Copilot review

aa853e7

aclark4life force-pushed the PYTHON-5781 branch from cea9d15 to aa853e7 Compare June 4, 2026 15:35

aclark4life marked this pull request as ready for review June 4, 2026 16:26

aclark4life requested a review from blink1073 June 4, 2026 16:27



		class TestSendall(AsyncUnitTest):
		def test_delegates_to_sock_sendall(self):



		class TestNetworkingInterfaceBase(AsyncUnitTest):
		def setUp(self):

		mock_socket.sendall.assert_called_once_with(b"hello")


		class TestNetworkingInterfaceBase(AsyncUnitTest):

		_ = self.base.sock


		class TestNetworkingInterface(AsyncUnitTest):

		self.assertIs(self.network_interface.sock, self.mock_socket)


		if not _IS_SYNC:

Conversation

aclark4life commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes in this PR

Test Plan

Checklist

Checklist for Author

Checklist for Reviewer

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

codecov-commenter commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

caseyclements left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

aclark4life commented Apr 23, 2026 •

edited

Loading

codecov-commenter commented Apr 23, 2026 •

edited

Loading