fix(connectors): hand off SDK worker during blocking send callbacks - #3797
fix(connectors): hand off SDK worker during blocking send callbacks#3797mlevkov wants to merge 1 commit into
Conversation
|
Thanks for the PR. It is labeled Slash commands (own line, regular comment) move it around the queue:
See CONTRIBUTING.md for details. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3797 +/- ##
=============================================
- Coverage 82.99% 68.19% -14.80%
Complexity 1339 1339
=============================================
Files 1218 1218
Lines 165627 139885 -25742
Branches 133940 108198 -25742
=============================================
- Hits 137459 95398 -42061
- Misses 24509 40788 +16279
- Partials 3659 3699 +40
🚀 New features to boost your workflow:
|
|
|
|
/request-review @hubcio |
2772550 to
cc9107f
Compare
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either Thank you for your contribution! |
The send callback runs synchronously inside the SDK's polling task, on a tokio runtime shared by every connector instance loaded from the same plugin library. A callback that blocks for backpressure pins one worker for the duration, and with enough saturated instances iggy_source_close for a sibling waits behind them, since the close blocks on the sibling's polling task getting scheduled to observe its shutdown signal. Wrap the callback in tokio::task::block_in_place so the worker is handed off before the callback runs. The SDK runtime is multi-threaded, which block_in_place requires. No FFI or ABI change; plugins pick this up when rebuilt against the updated SDK. Fixes apache#3796. Co-authored-by: Claude <noreply@anthropic.com>
cc9107f to
50ef34d
Compare
Summary
Closes #3796. The send callback runs synchronously inside the SDK's polling task (
handle_messagesincore/connectors/sdk/src/source.rs), on the tokio runtime shared by every connector instance loaded from the same plugin library (static RUNTIME: OnceLock<Runtime>inlib.rs). A callback that blocks pins one worker for the duration. With #3795's bounded forwarding channel, a callback legitimately blocks for backpressure while the channel is full, so enough saturated instances of one library can occupy all workers, andiggy_source_closefor a sibling instance then waits behind them: the close blocks on the sibling's polling task getting scheduled to observe its shutdown signal.Change
One call site: wrap the callback invocation in
tokio::task::block_in_place, so the worker is handed off before the callback runs and the runtime keeps scheduling sibling tasks regardless of how long the callback blocks.Notes from the analysis in #3796:
block_in_placeconsults the calling thread's tokio context, which is only set on the plugin runtime's own worker threads.Runtime::new(), i.e. multi-threaded, whichblock_in_placerequires.This composes with #3795 (bounded channel + shutdown signaling) but does not depend on it: any long-blocking callback benefits.
Test plan
cargo clippy -p iggy_connector_sdk --all-targets --all-features -- -D warningscleancargo test -p iggy_connector_sdk --all-featurespassescargo build -p iggy_connector_stdout_sink -p iggy_connector_random_source -p iggy-connectors(macro consumers + runtime rebuild cleanly)