Skip to content

net: keep mempool accept off tokio workers - #316

Merged
reardencode merged 4 commits into
masterfrom
net/reactor-safe-work
Sep 3, 2026
Merged

net: keep mempool accept off tokio workers#316
reardencode merged 4 commits into
masterfrom
net/reactor-safe-work

Conversation

@rearden-grok

@rearden-grok rearden-grok Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Why

Hours after tip, PID 1454645 froze with all 8 tokio-rt-workers in futex_wait. tip-accept was idle. Last tip: perf showed accept_lock_us=1.78s / accept_utxo_us=1.39s then the 5s timer died.

P2P NetworkMessage::Tx called MempoolHub::accept_tx on the session task. inner.write() covered store UTXO lookups. Other workers blocked on that lock (contains_wtxid, more accepts). Nobody parked the I/O driver.

Same class as the earlier connect-on-tokio freeze, different function.

What

  • No new dedicated thread. Tokio blocking pool (already used by Electrum/RPC) runs accept. Role threads (tip-accept, IBD pipeline, rbtc-sh-wb, rbtc-scripts-*) stay.
  • assert_not_reactor: panics if accept_tx / sync run_on_tip_accept wait run on a tokio worker without BlockingRegion. Tokio names both workers and spawn_blocking threads tokio-rt-worker; the region is the allow flag.
  • accept_tx_async for P2P Tx and Esplora POST /tx / POST /txs/package. Electrum/RPC blocking closures enter the region.
  • Prepare under graph read, not write. contains_wtxid proceeds during slow UTXO. Orphans park under a short write. Relay maps update after commit drops the graph lock.
  • Node runtime: max_blocking_threads = nCPU.max(4) instead of tokio’s 512.

Not in this PR (follow-up): ArcSwap inv set / compact list_live off reactor; making JoinOnDrop 'static so Drop can detach on cancel (still waits for soundness of the borrowed job).

Tests

cargo test -p rbitcoin-net --lib tx_relay::tests
cargo test -p rbitcoin-net --lib tip_accept::
cargo test -p rbitcoin-node --lib blocking_pool

rbitcoin-grok added 3 commits September 3, 2026 09:02
P2P Tx and Esplora POST /tx ran MempoolHub::accept_tx on tokio-rt-worker,
taking inner RwLock across store UTXO lookups. All workers parked on that
lock; the I/O driver never epoll_waited (mainnet freeze after tip).

assert_not_reactor panics on a worker unless BlockingRegion is entered
(tokio names spawn_blocking threads tokio-rt-worker too). accept_tx_async
and Electrum/RPC/Esplora blocking closures enter the region.
inner.write() around Query get_coin serialized every P2P accept and
blocked contains_wtxid on tokio workers. prepare_admit is now &self;
orphans park under a short write. Relay map updates run after commit
drops the graph lock.
Runtime::new() allowed 512 spawn_blocking threads. Cap at nCPU (min 4).
Sync run_on_tip_accept panics on a tokio worker unless BlockingRegion is
entered (RPC/generate). concurrency.md names reactor vs role vs pool.
@rearden-grok rearden-grok Bot added the core-functional Run Core functional nightly.sh on this PR (inventory + release-pin warn) label Sep 3, 2026
fmt wants pub use reactor after rbitcoin_mempool. The async connect test
called ensure_genesis (sync tip-accept wait) on a tokio worker.
@reardencode
reardencode merged commit d826f56 into master Sep 3, 2026
13 checks passed
@rearden-grok
rearden-grok Bot deleted the net/reactor-safe-work branch September 4, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core-functional Run Core functional nightly.sh on this PR (inventory + release-pin warn)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant