Skip to content

fix(core-amqp): avoid uAMQP pollable registry deadlock - #7373

Open
Johnathan W (j7nw4r) wants to merge 4 commits into
mainfrom
fix/uamqp-pollable-deadlock
Open

fix(core-amqp): avoid uAMQP pollable registry deadlock#7373
Johnathan W (j7nw4r) wants to merge 4 commits into
mainfrom
fix/uamqp-pollable-deadlock

Conversation

@j7nw4r

Copy link
Copy Markdown
Member

Summary

GlobalStateHolder::RemovePollable can deadlock when CBS refresh or sender cleanup waits for a poll in progress.

Motivation

RemovePollable held m_pollablesMutex while waiting for an in-flight Poll completion. This blocks unrelated registry operations and can deadlock uAMQP connection or link cleanup paths. Event Hubs CreateBatch cleanup and CBS management refresh share affected lock-order risks.

Changes

  • Makes RemovePollable release the registry mutex while waiting for in-flight polling and preserve completion barriers for earlier snapshots.
  • Moves sender and receiver open async registry activation outside connection locks.
  • Audits CBS management refresh and Event Hubs CreateBatch failed-open cleanup lock scopes.
  • Adds repeated concurrent core registry tests and a changelog entry for issue uAMQP: GlobalStateHolder::RemovePollable can deadlock via CBS refresh and sender cleanup #7370.
  • Leaves Rust AMQP unchanged.

Test plan

Affected uAMQP and Event Hubs build passed. Core AMQP suite passed 210 of 210. Playback runner passed 224 tests with 14 live-only skips. Concurrency tests passed 100 repetitions on the first validation pass, 50 on the second, and 20 in the root rerun. clang-format, CSpell, git diff check, commit boundary check, and Rust diff audit passed. Direct CBS and CreateBatch integration tests were not added because unchanged source aborts in an existing TransportImpl handle assertion before their assertions; core concurrency tests and static lock-scope audit cover these paths. The exact live Event Hubs runner was not run because EVENTHUBS_HOST is absent.

Closes #7370

Add a registry progress test for concurrent pollable operations and a completion barrier characterization test.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
8 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes uAMQP pollable-registry deadlocks while preserving removal completion barriers.

Changes:

  • Replaces busy-waiting with generation-based condition-variable synchronization.
  • Moves sender/receiver registry activation outside connection locks.
  • Adds concurrent registry tests and release notes.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
sdk/core/azure-core-amqp/src/common/global_state.cpp Implements non-blocking registry synchronization.
sdk/core/azure-core-amqp/inc/azure/core/amqp/internal/common/global_state.hpp Adds generation and condition-variable state.
sdk/core/azure-core-amqp/src/impl/uamqp/amqp/message_sender.cpp Adjusts sender lock scope.
sdk/core/azure-core-amqp/src/impl/uamqp/amqp/message_receiver.cpp Adjusts receiver lock scope and guidance.
sdk/core/azure-core-amqp/test/ut/global_state_tests.cpp Tests concurrent registry operations and barriers.
sdk/core/azure-core-amqp/test/ut/CMakeLists.txt Registers the new tests.
sdk/core/azure-core-amqp/CHANGELOG.md Documents the deadlock fix.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
8 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

The old polling loop hit continue inside a do-while when the registry
was empty, which skipped the 100 ms sleep. The thread spun on one core
after a process closed its last connection. The condition variable wait
fixes that, and the changelog entry for #7370 now says so.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

uAMQP: GlobalStateHolder::RemovePollable can deadlock via CBS refresh and sender cleanup

2 participants