Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .claude/skills/thread-safety-itc/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ Throttled events (e.g. `positionChanged`) use `PositionChangedDispatcher`, which
- Clear callbacks in the **most-derived HostObject first** (each layer clears its own events; base destructors run afterward and clear parent events)
- Do **not** inherit `EventCaller` on multiple node bases (ambiguous API); add member `EventCaller` fields per event instead

**Time-delayed dispatch outside the node's lifetime** (e.g. firing `ended` at a context time the render path will not reach for this node): use `BaseAudioContext::deferEmptyEventDispatch(event, callbackId, dueTime)`. The context keeps a preallocated pending list drained each quantum in `processAudioEvents()`; the context clock is the timer (suspend pauses it, offline fires at render speed) and the context holds no node reference. Never use a detached timer thread for this — spawning a thread from an audio-event body runs on the audio thread (allocation + syscalls), and every capture choice has been a real bug in `AudioScheduledSourceNode::stop()`: raw `this` was a use-after-free, `weak_from_this().lock()` ran the node destructor on the timer thread (bypassing the graph's disposal path), and `std::move`-ing the `EventCaller` member raced the JS/GC-thread `assignCallbackId` paths (the registry `shared_ptr` inside `EventCaller` is only safe for concurrent access while it is never mutated — which is why `EventCaller` is non-movable).

Low-level registry API (used internally by `EventCaller`):

```cpp
Expand Down Expand Up @@ -166,7 +168,7 @@ On Android, `AudioPlayer::onErrorAfterClose` also takes `driverMutex_` because O

**Live `AudioContext` render quiescence:** `currentRenders_` on `AudioContext` is incremented at the start of each platform I/O callback (`IOSAudioPlayer::deliverOutputBuffers` / `AudioPlayer::onAudioReady`) via a reference passed in `initialize()`, and decremented when the callback returns (RAII scope). `suspend()` and `close()` call `waitForRenderQuiescence()` (under `driverMutex_`) before `processAudioEvents()` / `cleanup()`. Platform drivers share the `CommonPlayer` abstract base (`common/cpp/audioapi/core/CommonPlayer.h`).

**Graph Channel A producer self-drain:** `Graph::setProducerSelfDrain(true)` makes the JS/main producer drain Channel A after each enqueue. Enable only when there is no audio/render consumer (realtime: construction + after `suspend`/`close` quiescence; offline: before `startRendering` and after a scheduled suspend). Before disabling for `start`/`resume`/`renderAudio`, call `processEvents()` once (still as sole consumer) so the bounded channel is empty, then disable, then start the audio/render consumer; re-enable if start/resume fails. After enabling, call `processEvents()` once to flush backlog (avoids `WAIT_ON_FULL` deadlock if the channel was already full).
**Graph producer self-drain:** `Graph::enableProducerSelfDrain()` makes producer threads drain the event channels themselves after each enqueue; `disableProducerSelfDrain()` hands consumption back to the audio/render thread. Both flush the channels internally (no separate `processEvents()` call needed) and serialize with in-flight drains via `selfDrainMutex_`, because two producers can drain concurrently: the JS thread (mutations) and the GC finalizer thread (`removeNode`, which self-drains after its Channel B orphan send — otherwise a finalizer burst with no consumer fills the bounded channel and blocks forever, e.g. at process exit). Enable only when there is no audio/render consumer (realtime: construction + after `suspend`/`close` quiescence; offline: before `startRendering` and after a scheduled suspend); disable *before* starting the audio/render consumer; re-enable if start/resume fails.

**Context lifecycle promises:** HostObjects wrap JSI `Promise`s via `ContextPromiseResolver`
(`jsi/ContextPromiseResolver.hpp`); tasks are queued as `ContextPromiseTask`
Expand All @@ -177,8 +179,15 @@ ops (`resume` / `suspend` / `close` / offline start) are **control messages** on
SPSC single-producer). A dedicated `TaskOffloader` worker thread drains the queue under
`driverMutex_`, runs `collectDisposedNodes()` (host-graph ghost cleanup — never on the audio
thread), then executes the lifecycle body and settles the promise on the CallInvoker. The
`TaskOffloader` destructor joins the worker and drains any queued control messages. When the driver
is stopped, `scheduleAudioEvent` still drains already-queued SPSC events then runs
`TaskOffloader` destructor joins the worker and drains any queued control messages — which is why
`~AudioContext` / `~OfflineAudioContext` MUST call `joinPendingPromiseWorker()` as their first
statement (and never while holding `driverMutex_`): default member destruction order would join the
worker last, letting drained tasks run against an already-destroyed player/graph/disposer. The
offline render thread is likewise owned (`renderThread_` + `stopRendering_` flag, joined in the
destructor, detach only when the last context reference drops on the render thread itself); the
render lambda must release its `resumePromise` right after resolving, since resolver callbacks
capture the context and would otherwise make the render thread its own context's last owner. When
the driver is stopped, `scheduleAudioEvent` still drains already-queued SPSC events then runs
the new event synchronously under `driverMutex_` (FIFO with prior messages). Control-message
bodies must not re-lock `driverMutex_` or `waitForRenderQuiescence()` while `currentRenders_ > 0`
(audio-callback self-deadlock). Apply the visible `state` attribute and settle the promise
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,13 @@ AudioContext::AudioContext(
const std::shared_ptr<IAudioEventHandlerRegistry> &audioEventHandlerRegistry)
: BaseAudioContext(sampleRate, audioEventHandlerRegistry), isInitialized_(false) {
// Context starts SUSPENDED with no audio-thread consumer. Let the producer
// drain Channel A itself until start()/resume() hands draining to the
// drain the channels itself until start()/resume() hands draining to the
// audio callback (same pattern as OfflineAudioContext before rendering).
getGraph()->setProducerSelfDrain(true);
getGraph()->enableProducerSelfDrain();
}

AudioContext::~AudioContext() {
joinPendingPromiseWorker();
if (getState() != ContextState::CLOSED) {
std::scoped_lock lock(driverMutex_);
close(nullptr);
Expand Down Expand Up @@ -65,11 +66,7 @@ bool AudioContext::tryStartDriver() {
return false;
}

// Flush while we are still the sole consumer, then hand the channel to the
// audio callback. flushing first avoids blocking forever if the bounded
// channel was full when self-drain is turned off.
getGraph()->processEvents();
getGraph()->setProducerSelfDrain(false);
getGraph()->disableProducerSelfDrain();

if (audioPlayer_->start()) {
isInitialized_.store(true, std::memory_order_release);
Expand All @@ -81,7 +78,7 @@ bool AudioContext::tryStartDriver() {
return true;
}

getGraph()->setProducerSelfDrain(true);
getGraph()->enableProducerSelfDrain();
return false;
}

Expand All @@ -96,8 +93,7 @@ void AudioContext::close(const std::shared_ptr<ContextPromiseResolver<void>> &pr

// No audio-thread consumer after stop; allow producer self-drain for any
// remaining graph mutations (and flush events already queued).
getGraph()->setProducerSelfDrain(true);
getGraph()->processEvents();
getGraph()->enableProducerSelfDrain();
processAudioEvents();
audioPlayer_->cleanup();

Expand All @@ -117,12 +113,11 @@ bool AudioContext::resume(const std::shared_ptr<ContextPromiseResolver<void>> &p

bool result = false;
if (isInitialized_.load(std::memory_order_acquire)) {
getGraph()->processEvents();
getGraph()->setProducerSelfDrain(false);
getGraph()->disableProducerSelfDrain();
if (audioPlayer_->resume()) {
result = true;
} else {
getGraph()->setProducerSelfDrain(true);
getGraph()->enableProducerSelfDrain();
}
} else {
result = tryStartDriver();
Expand Down Expand Up @@ -150,8 +145,7 @@ bool AudioContext::suspend(const std::shared_ptr<ContextPromiseResolver<void>> &

// Audio callback is no longer the consumer; enable self-drain so graph
// mutations while suspended cannot fill the bounded channel and block.
getGraph()->setProducerSelfDrain(true);
getGraph()->processEvents();
getGraph()->enableProducerSelfDrain();
processAudioEvents();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,39 @@ BaseAudioContext::BaseAudioContext(
gcAudioEventScheduler_(GC_AUDIO_SCHEDULER_CAPACITY),
disposer_(
std::make_unique<utils::DisposerImpl<DISPOSER_PAYLOAD_SIZE>>(AUDIO_SCHEDULER_CAPACITY)),
graph_(std::make_shared<utils::graph::Graph>(AUDIO_SCHEDULER_CAPACITY, disposer_.get())) {}
graph_(std::make_shared<utils::graph::Graph>(AUDIO_SCHEDULER_CAPACITY, disposer_.get())) {
deferredEmptyEvents_.reserve(DEFERRED_EMPTY_EVENTS_CAPACITY);
}

void BaseAudioContext::deferEmptyEventDispatch(
AudioEvent event,
uint64_t callbackId,
double dueTime) {
if (callbackId == 0) {
return;
}

deferredEmptyEvents_.push_back({.dueTime = dueTime, .event = event, .callbackId = callbackId});
}

void BaseAudioContext::dispatchDueDeferredEvents() {
if (deferredEmptyEvents_.empty()) {
return;
}

const auto now = getCurrentTime();
std::erase_if(deferredEmptyEvents_, [&](const DeferredEmptyEvent &deferred) {
if (deferred.dueTime > now) {
return false;
}

if (audioEventHandlerRegistry_ != nullptr) {
audioEventHandlerRegistry_->dispatchEventFromAudioThread(
deferred.event, deferred.callbackId, AudioEventPayload{EmptyPayload{}});
}
return true;
});
}

void BaseAudioContext::initialize(const AudioDestinationNode *destination) {
destination_ = destination;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include <audioapi/core/utils/Constants.h>
#include <audioapi/core/utils/Disposer.hpp>
#include <audioapi/core/utils/graph/Graph.h>
#include <audioapi/events/AudioEvent.h>
#include <audioapi/utils/AudioBuffer.hpp>
#include <audioapi/utils/CrossThreadEventScheduler.hpp>
#include <audioapi/utils/TaskOffloader.hpp>
Expand All @@ -15,6 +16,7 @@
#include <cassert>
#include <complex>
#include <cstddef>
#include <cstdint>
#include <memory>
#include <mutex>
#include <utility>
Expand Down Expand Up @@ -67,8 +69,15 @@ class BaseAudioContext : public std::enable_shared_from_this<BaseAudioContext> {
// finalization that clears the callback id).
audioEventScheduler_.processAllEvents(*this);
gcAudioEventScheduler_.processAllEvents(*this);
dispatchDueDeferredEvents();
}

/// @brief Schedules a one-shot, payload-less event dispatch for when
/// `currentTime` reaches `dueTime`.
/// @note Runs on the render-serialized domain (audio thread, or the
/// synchronous `scheduleAudioEvent` path) — same as audio event bodies.
void deferEmptyEventDispatch(AudioEvent event, uint64_t callbackId, double dueTime);

template <typename F>
bool scheduleAudioEvent(F &&event) noexcept { // NOLINT(cppcoreguidelines-missing-std-forward)
std::scoped_lock lock(driverMutex_);
Expand Down Expand Up @@ -123,6 +132,16 @@ class BaseAudioContext : public std::enable_shared_from_this<BaseAudioContext> {
mutable std::mutex driverMutex_;
std::atomic<ContextState> state_;

/// @brief Joins the pending-promises worker after draining any queued
/// lifecycle tasks. Idempotent.
///
/// Derived-class destructors MUST call this as their first teardown step:
/// the drained task bodies lock `driverMutex_` and touch the player, graph,
/// and disposer, all of which start being destroyed once the destructor bodies return.
void joinPendingPromiseWorker() {
pendingPromisesOffloader_->shutdown();
}

/// Debug-only: `driverMutex_` must already be held by the calling thread.
void assertDriverMutexHeld() const {
#ifndef NDEBUG
Expand Down Expand Up @@ -164,6 +183,22 @@ class BaseAudioContext : public std::enable_shared_from_this<BaseAudioContext> {
std::unique_ptr<utils::DisposerImpl<DISPOSER_PAYLOAD_SIZE>> disposer_;
std::shared_ptr<utils::graph::Graph> graph_;

struct DeferredEmptyEvent {
double dueTime;
AudioEvent event;
uint64_t callbackId;
};

/// Reserved up front so `deferEmptyEventDispatch` stays allocation-free on
/// the audio thread until this many dispatches are pending at once.
static constexpr size_t DEFERRED_EMPTY_EVENTS_CAPACITY = 16;

/// Render-serialized only (audio thread or the synchronous
/// `scheduleAudioEvent` path) — no lock, so no other thread may touch it.
std::vector<DeferredEmptyEvent> deferredEmptyEvents_;

void dispatchDueDeferredEvents();

[[nodiscard]] virtual bool isDriverRunning() const = 0;
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,26 @@ OfflineAudioContext::OfflineAudioContext(
// graph-event channel, so a large graph would otherwise fill it and block
// the producer forever. Let the producing thread drain it itself for now;
// renderAudio() hands draining back to the render thread.
getGraph()->setProducerSelfDrain(true);
getGraph()->enableProducerSelfDrain();
}

OfflineAudioContext::~OfflineAudioContext() {
// Join the promise worker first: a queued resume() task could spawn a fresh
// render thread after the join below. Both must be gone before base-class
// members (graph, disposer, driverMutex_) are destroyed.
joinPendingPromiseWorker();
stopRendering_.store(true, std::memory_order_release);
if (renderThread_.joinable()) {
if (renderThread_.get_id() == std::this_thread::get_id()) {
// The render thread can drop the context's last reference itself (a
// drained event lambda holding the final shared_ptr). Self-join would
// throw; the thread is already past its render loop, so let it finish
// unobserved.
renderThread_.detach();
} else {
renderThread_.join();
}
}
}

void OfflineAudioContext::resume(const std::shared_ptr<ContextPromiseResolver<void>> &promise) {
Expand Down Expand Up @@ -81,15 +100,22 @@ bool OfflineAudioContext::suspend(

void OfflineAudioContext::renderAudio(
const std::shared_ptr<ContextPromiseResolver<void>> &resumePromise) {
// Flush while we are still the sole consumer, then hand the channel to the
// render thread.
getGraph()->processEvents();
getGraph()->setProducerSelfDrain(false);
getGraph()->disableProducerSelfDrain();

std::thread([this, resumePromise = resumePromise]() mutable {
// first wait for the previous render thread to finish
// can be in the middle because of suspend and then resume call again
if (renderThread_.joinable()) {
renderThread_.join();
}

renderThread_ = std::thread([this, resumePromise = resumePromise]() mutable {
ContextPromiseResolver<void>::resolve(resumePromise);
// The resolver's callbacks (transitively) own this context — both
// `startRendering` and the HostObject resume path capture it. Release the
// reference so the render thread never keeps its own context alive:
resumePromise.reset();

while (currentSampleFrame_ < length_) {
while (!stopRendering_.load(std::memory_order_acquire) && currentSampleFrame_ < length_) {
Locker locker(driverMutex_);
int framesToProcess =
std::min(static_cast<int>(length_ - currentSampleFrame_), RENDER_QUANTUM_SIZE);
Expand All @@ -111,16 +137,19 @@ void OfflineAudioContext::renderAudio(
// The render thread is about to exit; with no consumer again, let the
// producer self-drain any graph mutations made from the suspend
// callback until resume() restarts rendering.
getGraph()->setProducerSelfDrain(true);
getGraph()->processEvents();
getGraph()->enableProducerSelfDrain();
locker.unlock();
ContextPromiseResolver<void>::resolve(promise);
return;
}
}

if (stopRendering_.load(std::memory_order_acquire)) {
return;
}
OfflineAudioContextResultPromise::resolve(resultPromise_, resultBuffer_);
}).detach();
resultPromise_.reset();
});
}

void OfflineAudioContext::startRendering(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
#include <audioapi/utils/AudioBuffer.hpp>
#include <audioapi/utils/Macros.h>

#include <atomic>
#include <memory>
#include <thread>
#include <unordered_map>

namespace audioapi {
Expand All @@ -17,7 +19,7 @@ class OfflineAudioContext : public BaseAudioContext {
size_t length,
float sampleRate,
const std::shared_ptr<IAudioEventHandlerRegistry> &audioEventHandlerRegistry);
~OfflineAudioContext() override = default;
~OfflineAudioContext() override;
DELETE_COPY_AND_MOVE(OfflineAudioContext);

void resume(const std::shared_ptr<ContextPromiseResolver<void>> &promise);
Expand All @@ -35,6 +37,11 @@ class OfflineAudioContext : public BaseAudioContext {
std::shared_ptr<DSPAudioBuffer> audioBuffer_;
std::shared_ptr<AudioBuffer> resultBuffer_;

/// Render worker. Owned: it captures `this`, so the destructor must be able
/// to stop and join it before members are torn down.
std::thread renderThread_;
std::atomic<bool> stopRendering_{false};

void renderAudio(const std::shared_ptr<ContextPromiseResolver<void>> &resumePromise);

bool isDriverRunning() const override;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,8 @@ bool AudioFileSourceNode::initDecoder(
std::move(frameSender),
frameReceiver_);

seekDecoderThread_ = std::thread(std::move(*seekDecoderDaemon_));

if (!decoderState_->isReady.load(std::memory_order_acquire)) {
return false;
}
Expand Down Expand Up @@ -393,11 +395,6 @@ void AudioFileSourceNode::start(double when) {
endOfStreamStopPending_ = false;
endOfStreamDrainPending_ = false;
positionChanged_.requestFlush();

if (seekDecoderDaemon_) {
seekDecoderThread_ = std::thread(std::move(*seekDecoderDaemon_));
seekDecoderDaemon_.reset();
}
}

void AudioFileSourceNode::bindMediaElementSource(uint64_t bindingId) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
#include <audioapi/core/BaseAudioContext.h>
#include <audioapi/core/sources/AudioScheduledSourceNode.h>
#include <audioapi/dsp/AudioUtils.h>
#include <audioapi/events/AudioEvent.h>
#include <audioapi/events/IAudioEventHandlerRegistry.h>
#include <audioapi/utils/AudioArray.hpp>
#if !RN_AUDIO_API_TEST
#include <audioapi/core/AudioContext.h>
#endif // RN_AUDIO_API_TEST

#include <algorithm>
#include <chrono>
#include <limits>
#include <memory>
#include <thread>

namespace audioapi {

Expand Down Expand Up @@ -46,13 +45,9 @@ void AudioScheduledSourceNode::stop(double when) {
if (stopFrame <= startFrame) {
playbackState_ = PlaybackState::FINISHED;
AudioNode::disable();
// Fire-and-forget: defer event dispatch off the caller thread.
static constexpr double kMillisecondsPerSecond = 1000.0;
std::thread([this]() {
std::this_thread::sleep_for(
std::chrono::milliseconds(static_cast<int>(stopTime_ * kMillisecondsPerSecond)));
onEndedEvent_.dispatchEmpty();
}).detach();
// The node never plays, so the render path will not fire onended. Defer a
// dispatch until the context clock reaches the requested stop time
context->deferEmptyEventDispatch(AudioEvent::ENDED, onEndedEvent_.getCallbackId(), when);
return;
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ Graph::Res Graph::removeNode(HNode *node) {
// could possibly reference this node lives at index < barrier.
auto barrier = eventSender_.sendCursor();
gcEventSender_.send(OrphanEnvelope{.barrier = barrier, .action = std::move(event)});
drainProducedEventsIfSelfDraining();
return NoneType{};
});
}
Expand Down
Loading