Fix rest_tornado dropping events to concurrent websocket clients#69698
Open
ggiesen wants to merge 3 commits into
Open
Fix rest_tornado dropping events to concurrent websocket clients#69698ggiesen wants to merge 3 commits into
ggiesen wants to merge 3 commits into
Conversation
EventListener._handle_event_socket_recv iterated over the futures list while removing delivered futures from that same list, which skips every other future. With multiple websocket clients registered under the same (tag, matcher) key, each event only reached some of the waiting clients. Iterate over a snapshot of the list so every matching future is resolved. Fixes saltstack#35798
…delivery The direct test subscribes multiple clients through get_event(request) with the exact defaults the websocket handlers in saltnado_websockets.py pass (no tag, no matcher, so tag="" with prefix_matcher) and asserts a single event resolves every waiting future; it fails without the snapshot fix in _handle_event_socket_recv. The inverse test guards against overcorrection: an already-done future must keep its original result and stay in the tag_map, and a future waiting on a different exact tag (the SaltAPIHandler.get_minion_returns shape) must stay pending; it passes with and without the fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
EventListener._handle_event_socket_recv in salt/netapi/rest_tornado/saltnado.py iterated over the futures list while calling .remove() on that same list, so with multiple websocket clients waiting under one (tag, matcher) key every other future was skipped and each event reached only some clients. The fix iterates over a snapshot (list(futures)) so every matching future gets resolved. Verified via the shadowed interpreter that the added unit test fails on the unpatched code ([True, False, True, False] of four waiters) and passes with the fix (all four delivered); note pytest itself hangs at collection in this sandbox even for pre-existing tests, so verification was done by executing the test logic directly through the shadowed salt-venv python.
What issues does this PR fix or reference?
Fixes #35798
Previous Behavior
See #35798.
New Behavior
Fix rest_tornado dropping events to concurrent websocket clients. Validated by a unit test proven to fail on unpatched 3006.x and pass with the fix (confirmed by adversarial review).
Merge requirements satisfied?
Commits signed with GPG?
No