diff --git a/.claude/skills/audio-nodes/SKILL.md b/.claude/skills/audio-nodes/SKILL.md index 31f0545b4..af9fbd587 100644 --- a/.claude/skills/audio-nodes/SKILL.md +++ b/.claude/skills/audio-nodes/SKILL.md @@ -405,6 +405,7 @@ Tail-bearing nodes (Delay/Convolver/Biquad) need no special handling: while conn - `AudioNode` — standard effect or analysis node - `AudioScheduledSourceNode` — source with start/stop scheduling - `AudioBufferBaseSourceNode` — source that plays back an AudioBuffer with pitch control + With `pitchCorrection` the WSOLA stretcher holds ~30 ms of audio, so the node must outlive its last sample. It does that itself (`stretchTailFramesLeft_`): once the processor reports the source ran dry it keeps the node playing and feeds the stretcher the zero-filled input until `WsolaTimeStretcher::latencyTailFrames` are consumed. Never pad the buffer with silence to get the same effect — that forces a private copy per node and breaks buffer sharing with the JS-facing `AudioBuffer`. 2. **Header file** (`core//MyNode.h`) - Annotate every method with `/// @note JS Thread only` or `/// @note Audio Thread only` diff --git a/.claude/skills/build-compilation-dependencies/SKILL.md b/.claude/skills/build-compilation-dependencies/SKILL.md index 67427ff37..cd02e28f5 100644 --- a/.claude/skills/build-compilation-dependencies/SKILL.md +++ b/.claude/skills/build-compilation-dependencies/SKILL.md @@ -321,7 +321,7 @@ CI runs a parallel `cpp-coverage` job via `.github/workflows/cpp-coverage-job.ym - Completely standalone — no Gradle, no Xcode, no prebuilt Android libraries needed - Sources resolved from `node_modules` (symlinked to `packages/` in yarn workspaces) - HostObjects, worklets nodes, AudioContext, and FfmpegDecoder are excluded from the test build -- HostObject **headers** still compile under `RN_AUDIO_API_TEST`; only their `.cpp` bodies are missing. When a test first links core code that constructs one (e.g. `AudioRecorderCallback` → `AudioBufferHostObject`), the linker fails on the constructor. Fix it with a stub translation unit under `test/src/` that defines just the constructors (see `test/src/AudioBufferHostObjectStub.cpp`), never with an `#if RN_AUDIO_API_TEST` test double inside the production header. The static lib also compiles `jsi/jsi.cpp` so `jsi::HostObject`'s virtuals resolve +- The static lib compiles `jsi/jsi.cpp`, so real JSI types (`jsi::HostObject`, `jsi::MutableBuffer`) work in tests. When core code a test links constructs a HostObject (e.g. `AudioRecorderCallback` → `AudioBufferHostObject`), add that HostObject's `.cpp` to the `list(APPEND RNAUDIOAPI_SRC …)` exceptions in `test/CMakeLists.txt` (as for `JsEnumParser.cpp` and `AudioBufferHostObject.cpp`) instead of writing a stub translation unit or an `#if RN_AUDIO_API_TEST` test double - Compile definitions: `RN_AUDIO_API_ENABLE_WORKLETS=0`, `RN_AUDIO_API_TEST=1` (FFmpeg is off because the define is simply absent) - Google Test auto-fetched via `FetchContent` if not installed locally - New test files in `test/src/**/*.cpp` are picked up automatically by glob — no CMakeLists edit needed diff --git a/.claude/skills/host-objects/SKILL.md b/.claude/skills/host-objects/SKILL.md index 8a6d4b6ae..ef48dc534 100644 --- a/.claude/skills/host-objects/SKILL.md +++ b/.claude/skills/host-objects/SKILL.md @@ -340,8 +340,7 @@ return result; ```cpp JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, getChannelData) { - auto channel = static_cast(args[0].getNumber()); - auto audioArrayBuffer = audioBuffer_->getSharedChannel(channel); + auto audioArrayBuffer = sharedBuffer_->exposeChannel(static_cast(args[0].getNumber())); auto arrayBuffer = jsi::ArrayBuffer(runtime, audioArrayBuffer); auto float32ArrayCtor = runtime.global() @@ -355,6 +354,14 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, getChannelData) { } ``` +The view aliases native memory for as long as JS keeps it, and the `shared_ptr` inside the `jsi::ArrayBuffer` keeps that memory alive on its own. When the native side later needs the view to stop aliasing (Web Audio's "acquire the content" on `AudioBufferSourceNode.start()`), it must retain the returned storage and replace it afterwards, so the old view keeps the old storage and stops reaching the buffer. + +That bookkeeping — copy-on-write against playing nodes, tracking exposed views, the spec's `copyFromChannel` / `copyToChannel` range rules — lives in the engine class `core/sources/SharedAudioBuffer`, not in `AudioBufferHostObject`. An AudioBuffer involves no inter-thread events (every member runs on the JS thread), so nothing forces its logic into the host layer; the host object only unpacks JSI arguments and wraps the returned storage. `SharedAudioBuffer` is unit-tested in `test/src/core/sources/SharedAudioBufferTest.cpp`, which the HostObject layer cannot be (see `build-compilation-dependencies`). Its per-channel flag vectors are sized in the constructor: sizing them lazily on first share crashed the view cut-off the moment the order of the two steps flipped. + +`AudioBufferSourceNodeHostObject` holds the assigned `SharedAudioBuffer` and hands samples to the audio thread in exactly one place, `acquireBufferContent`, which runs on `start()` and on a `setBuffer()` made after start. A `setBuffer()` before start only records the buffer and publishes its channel count; the node reads nothing until acquire, so every JS write made in between is played and nothing has to track content versions. `SharedAudioBuffer::acquireContent` both cuts off the live views and returns the storage-sharing buffer the node plays. + +`AudioBufferQueueSourceNodeHostObject::enqueueBuffer` reuses the same mechanism: a buffer of the queue's width is shared through `SharedAudioBuffer::acquireContent` instead of copied, only a buffer of another width is mixed into a private copy, and the TS wrapper calls `invalidateChannelDataCache` after every enqueue. The stretcher is configured inline on the JS thread by the host base `initStretch`, once per node and with no audio event: the audio thread touches it only once the node has content, and the first content is always scheduled behind that call. Audio events are `FatFunction<72, …>`; a handle, a node pointer, two `shared_ptr`s and one scalar already fill the slot, so derive what you can on the audio thread (the queue event reads the width off its content) instead of capturing it. + ### External memory pressure Call `setExternalMemoryPressure` whenever returning a HostObject or typed array that wraps a large native buffer. This lets the JS GC schedule collection correctly: diff --git a/packages/audiodocs/docs/fundamentals/best-practices.mdx b/packages/audiodocs/docs/fundamentals/best-practices.mdx index a9ac181a5..5156040a8 100644 --- a/packages/audiodocs/docs/fundamentals/best-practices.mdx +++ b/packages/audiodocs/docs/fundamentals/best-practices.mdx @@ -53,6 +53,8 @@ user experience, and maintainability. Here are some key best practices to consid - **Scheduled source nodes are single-use**: [`AudioBufferSourceNode`](../sources/audio-buffer-source-node.mdx), [`OscillatorNode`](../sources/oscillator-node.mdx), and other [`AudioScheduledSourceNode`](../sources/audio-scheduled-source-node.mdx) subclasses can be [`start()`](../sources/audio-scheduled-source-node.mdx#start)ed only once. Create a new node to replay a sound, but reuse the same [`AudioBuffer`](../sources/audio-buffer.mdx) — nodes are inexpensive to create. +- **Seeking**: since a started `AudioBufferSourceNode` can't be repositioned, seeking means stopping/disconnecting it and creating a fresh node with the same `AudioBuffer` at a new `start()` offset. Reassigning the same buffer this way is cheap. The underlying PCM data is copied once per `AudioBuffer`, not once per node, so recreating the source node on every seek doesn't re-copy it. + - **Use [`AudioBufferQueueSourceNode`](../sources/audio-buffer-queue-source-node.mdx) for chunked playback**: When audio arrives in segments (streaming TTS, progressive download), enqueue buffers into a queue source node rather than recreating the entire graph per chunk. ## [**AudioParam**](../core/audio-param.mdx) changes @@ -87,4 +89,5 @@ Prefer logging plain values instead: ```tsx console.log({ channelCount: node.channelCount, numberOfInputs: node.numberOfInputs }); ``` + ::: diff --git a/packages/audiodocs/docs/sources/audio-buffer-queue-source-node.mdx b/packages/audiodocs/docs/sources/audio-buffer-queue-source-node.mdx index e6049f2aa..c849ff64e 100644 --- a/packages/audiodocs/docs/sources/audio-buffer-queue-source-node.mdx +++ b/packages/audiodocs/docs/sources/audio-buffer-queue-source-node.mdx @@ -111,6 +111,10 @@ Adds another buffer to queue. Returns `bufferId` that can be used to identify th #### Returns `string`. +:::tip +The first enqueued buffer sets the number of channels the queue plays at. A buffer with that same number of channels is played directly from its own memory, so enqueuing it costs no copy. A buffer with a different number of channels is first mixed into a copy of the queue's width, which is slower and allocates memory for every such buffer. For best performance, enqueue buffers that all have the same number of channels. +::: + ### `dequeueBuffer` Removes a buffer from the queue. Note that [`onbufferended`](audio-buffer-queue-source-node#onbufferended) event will not be fired for the removed buffers. diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.cpp b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.cpp index 4f72bd149..b0f55e1e7 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.cpp @@ -53,12 +53,12 @@ JSI_HOST_FUNCTION_IMPL(ConvolverNodeHostObject, setBuffer) { irBytes_ = bufferHostObject->getSizeInBytes(); thisValue.asObject(runtime).setExternalMemoryPressure(runtime, getMemoryPressure()); - setBuffer(bufferHostObject->audioBuffer_); + setBuffer(bufferHostObject->sharedBuffer_->getSamples()); return jsi::Value::undefined(); } -void ConvolverNodeHostObject::setBuffer(const std::shared_ptr &buffer) { +void ConvolverNodeHostObject::setBuffer(const std::shared_ptr &buffer) { if (buffer == nullptr) { return; } diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.h b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.h index e363f85ec..7301b64bc 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/effects/ConvolverNodeHostObject.h @@ -33,6 +33,6 @@ class ConvolverNodeHostObject : public AudioNodeHostObject { bool normalize_; size_t irBytes_ = 0; - void setBuffer(const std::shared_ptr &buffer); + void setBuffer(const std::shared_ptr &buffer); }; } // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.cpp b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.cpp index 590a858ac..e8b9214fb 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.cpp @@ -81,18 +81,19 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferBaseSourceNodeHostObject, getOutputLatency) { } void AudioBufferBaseSourceNodeHostObject::initStretch(int channelCount, float sampleRate) { - auto handle = node_->handle; + if (stretchHasBeenInit_) { + return; + } + stretchHasBeenInit_ = true; + inputLatency_ = WsolaTimeStretcher::INPUT_LATENCY_MS / 1000.0; outputLatency_ = WsolaTimeStretcher::OUTPUT_LATENCY_MS / 1000.0; auto playbackRateBuffer = std::make_shared( WsolaTimeStretcher::MAX_PLAYBACK_RATE * RENDER_QUANTUM_SIZE, channelCount, sampleRate); - auto event = [handle, node = bufferBaseSourceNode_, playbackRateBuffer, channelCount, sampleRate]( - BaseAudioContext &) { - node->initStretch(static_cast(channelCount), sampleRate, playbackRateBuffer); - }; - bufferBaseSourceNode_->scheduleAudioEvent(std::move(event)); + bufferBaseSourceNode_->initStretch( + static_cast(channelCount), sampleRate, playbackRateBuffer); } } // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.h b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.h index 886d5c7ca..78c653ce9 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferBaseSourceNodeHostObject.h @@ -40,7 +40,9 @@ class AudioBufferBaseSourceNodeHostObject : public AudioScheduledSourceNodeHostO double inputLatency_ = 0; double outputLatency_ = 0; bool pitchCorrection_; + bool stretchHasBeenInit_ = false; + /// Configures the node's stretcher for content of this shape, inline and only once void initStretch(int channelCount, float sampleRate); }; diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.cpp b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.cpp index a2276c050..dc8f10a5b 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.cpp @@ -2,15 +2,13 @@ #include -#include #include #include -#include namespace audioapi { AudioBufferHostObject::AudioBufferHostObject(const std::shared_ptr &audioBuffer) - : audioBuffer_(audioBuffer) { + : sharedBuffer_(std::make_shared(audioBuffer)) { addGetters( JSI_EXPORT_PROPERTY_GETTER(AudioBufferHostObject, sampleRate), JSI_EXPORT_PROPERTY_GETTER(AudioBufferHostObject, length), @@ -23,28 +21,24 @@ AudioBufferHostObject::AudioBufferHostObject(const std::shared_ptr JSI_EXPORT_FUNCTION(AudioBufferHostObject, copyToChannel)); } -AudioBufferHostObject::AudioBufferHostObject(AudioBufferHostObject &&other) noexcept - : HostObject(std::move(other)), audioBuffer_(std::move(other.audioBuffer_)) {} - JSI_PROPERTY_GETTER_IMPL(AudioBufferHostObject, sampleRate) { - return {audioBuffer_->getSampleRate()}; + return {sharedBuffer_->getSamples()->getSampleRate()}; } JSI_PROPERTY_GETTER_IMPL(AudioBufferHostObject, length) { - return {static_cast(audioBuffer_->getSize())}; + return {static_cast(sharedBuffer_->getSamples()->getSize())}; } JSI_PROPERTY_GETTER_IMPL(AudioBufferHostObject, duration) { - return {audioBuffer_->getDuration()}; + return {sharedBuffer_->getSamples()->getDuration()}; } JSI_PROPERTY_GETTER_IMPL(AudioBufferHostObject, numberOfChannels) { - return {static_cast(audioBuffer_->getNumberOfChannels())}; + return {static_cast(sharedBuffer_->getSamples()->getNumberOfChannels())}; } JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, getChannelData) { - auto channel = static_cast(args[0].getNumber()); - auto audioArrayBuffer = audioBuffer_->getSharedChannel(channel); + auto audioArrayBuffer = sharedBuffer_->exposeChannel(static_cast(args[0].getNumber())); auto arrayBuffer = jsi::ArrayBuffer(runtime, audioArrayBuffer); auto float32ArrayCtor = runtime.global().getPropertyAsFunction(runtime, "Float32Array"); @@ -58,19 +52,11 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, getChannelData) { JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, copyFromChannel) { auto arrayBuffer = args[0].getObject(runtime).getPropertyAsObject(runtime, "buffer").getArrayBuffer(runtime); - auto *destination = reinterpret_cast(arrayBuffer.data(runtime)); - auto destinationLength = arrayBuffer.size(runtime) / sizeof(float); - auto channelNumber = static_cast(args[1].getNumber()); - auto rawStart = args[2].getNumber(); - auto channelSize = audioBuffer_->getSize(); - - // Per spec, an out-of-range (or negative) startInChannel copies nothing, and a - // copy that would run past the end of the channel is truncated rather than throwing. - if (rawStart >= 0 && static_cast(rawStart) < channelSize) { - auto startInChannel = static_cast(rawStart); - auto framesToCopy = std::min(destinationLength, channelSize - startInChannel); - audioBuffer_->getChannel(channelNumber)->copyTo(destination, startInChannel, 0, framesToCopy); - } + sharedBuffer_->copyFromChannel( + reinterpret_cast(arrayBuffer.data(runtime)), + arrayBuffer.size(runtime) / sizeof(float), + static_cast(args[1].getNumber()), + args[2].getNumber()); return jsi::Value::undefined(); } @@ -78,19 +64,11 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, copyFromChannel) { JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, copyToChannel) { auto arrayBuffer = args[0].getObject(runtime).getPropertyAsObject(runtime, "buffer").getArrayBuffer(runtime); - auto *source = reinterpret_cast(arrayBuffer.data(runtime)); - auto sourceLength = arrayBuffer.size(runtime) / sizeof(float); - auto channelNumber = static_cast(args[1].getNumber()); - auto rawStart = args[2].getNumber(); - auto channelSize = audioBuffer_->getSize(); - - // Per spec, an out-of-range (or negative) startInChannel copies nothing, and a - // copy that would run past the end of the channel is truncated rather than throwing. - if (rawStart >= 0 && static_cast(rawStart) < channelSize) { - auto startInChannel = static_cast(rawStart); - auto framesToCopy = std::min(sourceLength, channelSize - startInChannel); - audioBuffer_->getChannel(channelNumber)->copy(source, 0, startInChannel, framesToCopy); - } + sharedBuffer_->copyToChannel( + reinterpret_cast(arrayBuffer.data(runtime)), + arrayBuffer.size(runtime) / sizeof(float), + static_cast(args[1].getNumber()), + args[2].getNumber()); return jsi::Value::undefined(); } diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.h b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.h index 04a463fb0..bcb231fd8 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.h @@ -1,37 +1,30 @@ #pragma once +#include #include #include #include #include #include -#include namespace audioapi { using namespace facebook; class AudioBufferHostObject : public HostObject { public: - std::shared_ptr audioBuffer_; + std::shared_ptr sharedBuffer_; explicit AudioBufferHostObject(const std::shared_ptr &audioBuffer); AudioBufferHostObject(const AudioBufferHostObject &) = delete; AudioBufferHostObject &operator=(const AudioBufferHostObject &) = delete; - AudioBufferHostObject(AudioBufferHostObject &&other) noexcept; - AudioBufferHostObject &operator=(AudioBufferHostObject &&other) noexcept { - if (this != &other) { - HostObject::operator=(std::move(other)); - audioBuffer_ = std::move(other.audioBuffer_); - } - return *this; - } + AudioBufferHostObject(AudioBufferHostObject &&other) noexcept = default; + AudioBufferHostObject &operator=(AudioBufferHostObject &&other) noexcept = default; ~AudioBufferHostObject() override = default; [[nodiscard]] size_t getSizeInBytes() const { - // *2 because every time buffer is passed we create a copy of it. - return audioBuffer_->getSize() * audioBuffer_->getNumberOfChannels() * sizeof(float) * 2; + return sharedBuffer_->getSizeInBytes(); } JSI_PROPERTY_GETTER_DECL(sampleRate); diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.cpp b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.cpp index e4ffc912f..e3d31a96b 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.cpp @@ -77,47 +77,44 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferQueueSourceNodeHostObject, resume) { JSI_HOST_FUNCTION_IMPL(AudioBufferQueueSourceNodeHostObject, enqueueBuffer) { auto audioBufferHostObject = args[0].getObject(runtime).asHostObject(runtime); - // TODO: add optimized memory management for buffer changes, e.g. - // when the same buffer is reused across threads and - // buffer modification is not allowed on JS thread + const auto &sharedBuffer = audioBufferHostObject->sharedBuffer_; + const auto &samples = sharedBuffer->getSamples(); + const auto bufferChannelNumber = samples->getNumberOfChannels(); - auto swapBuffer = false; // whether to swap internal node buffer with the new buffer + // The first buffer fixes the width the queue plays at. if (outputChannelNumber_ == 0) { - outputChannelNumber_ = audioBufferHostObject->audioBuffer_->getNumberOfChannels(); - swapBuffer = true; + outputChannelNumber_ = bufferChannelNumber; } - // first buffer defines the output channel number, the rest are mixed to it - auto copiedBuffer = std::make_shared( - audioBufferHostObject->audioBuffer_->getSize(), - outputChannelNumber_, - audioBufferHostObject->audioBuffer_->getSampleRate()); + std::shared_ptr content; - copiedBuffer->sum(*audioBufferHostObject->audioBuffer_); + if (bufferChannelNumber == outputChannelNumber_) { + content = sharedBuffer->acquireContent(); + } else { + // A buffer of another width is mixed into a private copy of the queue's width. + content = std::make_shared( + samples->getSize(), static_cast(outputChannelNumber_), samples->getSampleRate()); + content->sum(*samples); + } std::shared_ptr tailBuffer = nullptr; if (pitchCorrection_ && !stretchHasBeenInit_) { - initStretch( - static_cast(copiedBuffer->getNumberOfChannels()), copiedBuffer->getSampleRate()); + initStretch(static_cast(outputChannelNumber_), samples->getSampleRate()); auto extraTailFrames = - static_cast((inputLatency_ + outputLatency_) * copiedBuffer->getSampleRate()); + static_cast((inputLatency_ + outputLatency_) * samples->getSampleRate()); tailBuffer = std::make_shared( - extraTailFrames, copiedBuffer->getNumberOfChannels(), copiedBuffer->getSampleRate()); + extraTailFrames, static_cast(outputChannelNumber_), samples->getSampleRate()); tailBuffer->zero(); - stretchHasBeenInit_ = true; } - auto event = [node = bufferQueueSourceNode_, - copiedBuffer, + auto event = [handle = node_->handle, + node = bufferQueueSourceNode_, + content, bufferId = bufferId_, - tailBuffer, - swapBuffer, - outputChannelNumber = outputChannelNumber_](BaseAudioContext &) { - if (swapBuffer) { - node->resizeOutputBuffer(static_cast(outputChannelNumber)); - } - node->enqueueBuffer(copiedBuffer, bufferId, tailBuffer); + tailBuffer](BaseAudioContext &) { + node->resizeOutputBuffer(content->getNumberOfChannels()); + node->enqueueBuffer(content, bufferId, tailBuffer); }; bufferQueueSourceNode_->scheduleAudioEvent(std::move(event)); diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.h b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.h index f13e7c966..b1afbd969 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferQueueSourceNodeHostObject.h @@ -42,7 +42,6 @@ class AudioBufferQueueSourceNodeHostObject : public AudioBufferBaseSourceNodeHos AudioBufferQueueSourceNode *const bufferQueueSourceNode_; size_t bufferId_ = 0; - bool stretchHasBeenInit_ = false; /// Width of the first enqueued buffer; later buffers are mixed to it. 0 until then. size_t outputChannelNumber_ = 0; }; diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.cpp b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.cpp index 141388eb8..6e4e75cd7 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.cpp @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -24,10 +25,6 @@ AudioBufferSourceNodeHostObject::AudioBufferSourceNodeHostObject( loopSkip_(options.loopSkip), loopStart_(options.loopStart), loopEnd_(options.loopEnd) { - if (options.buffer != nullptr) { - setBuffer(options.buffer); - } - addGetters( JSI_EXPORT_PROPERTY_GETTER(AudioBufferSourceNodeHostObject, loop), JSI_EXPORT_PROPERTY_GETTER(AudioBufferSourceNodeHostObject, loopSkip), @@ -126,6 +123,9 @@ JSI_PROPERTY_SETTER_IMPL(AudioBufferSourceNodeHostObject, onloopended) { } JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, start) { + hasBeenStarted_ = true; + acquireBufferContent(); + auto handle = node_->handle; auto event = [handle, node = audioBufferSourceNode_, @@ -139,6 +139,16 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, start) { return jsi::Value::undefined(); } +void AudioBufferSourceNodeHostObject::acquireBufferContent() { + auto buffers = + prepareNodeBuffers(assignedBuffer_ == nullptr ? nullptr : assignedBuffer_->acquireContent()); + auto event = + [handle = node_->handle, node = audioBufferSourceNode_, buffers](BaseAudioContext &) { + node->setBuffer(buffers.nodeBuffer, buffers.audioBuffer); + }; + audioBufferSourceNode_->scheduleAudioEvent(std::move(event)); +} + JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, setBuffer) { if (args[0].isNull()) { setBuffer(nullptr); @@ -147,62 +157,57 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, setBuffer) { thisValue.asObject(runtime).setExternalMemoryPressure( runtime, getMemoryPressure() + bufferHostObject->getSizeInBytes()); - setBuffer(bufferHostObject->audioBuffer_); + setBuffer(bufferHostObject->sharedBuffer_); } return jsi::Value::undefined(); } -void AudioBufferSourceNodeHostObject::setBuffer(const std::shared_ptr &buffer) { - // TODO: add optimized memory management for buffer changes, e.g. - // when the same buffer is reused across threads and - // buffer modification is not allowed on JS thread - auto handle = node_->handle; +AudioBufferSourceNodeHostObject::NodeBuffers AudioBufferSourceNodeHostObject::prepareNodeBuffers( + const std::shared_ptr &acquiredContent) { + NodeBuffers buffers; - std::shared_ptr copiedBuffer; - std::shared_ptr audioBuffer; - const size_t newOutputChannelNumber = buffer == nullptr - ? AudioBufferSourceOptions::kDefaultOutputChannelNumber - : buffer->getNumberOfChannels(); - - if (buffer == nullptr) { - copiedBuffer = nullptr; - audioBuffer = std::make_shared( + if (acquiredContent == nullptr) { + buffers.nodeBuffer = nullptr; + buffers.audioBuffer = std::make_shared( RENDER_QUANTUM_SIZE, AudioBufferSourceOptions::kDefaultOutputChannelNumber, audioBufferSourceNode_->getContextSampleRate()); - } else { - if (pitchCorrection_) { - initStretch(static_cast(buffer->getNumberOfChannels()), buffer->getSampleRate()); - auto extraTailFrames = - static_cast((inputLatency_ + outputLatency_) * buffer->getSampleRate()); - size_t totalSize = buffer->getSize() + extraTailFrames; - copiedBuffer = std::make_shared( - totalSize, buffer->getNumberOfChannels(), buffer->getSampleRate()); - copiedBuffer->copy(*buffer, 0, 0, buffer->getSize()); - copiedBuffer->zero(buffer->getSize(), extraTailFrames); - } else { - copiedBuffer = std::make_shared(*buffer); - } - - audioBuffer = std::make_shared( - RENDER_QUANTUM_SIZE, - copiedBuffer->getNumberOfChannels(), - audioBufferSourceNode_->getContextSampleRate()); + return buffers; + } + + if (pitchCorrection_) { + initStretch( + static_cast(acquiredContent->getNumberOfChannels()), acquiredContent->getSampleRate()); } + buffers.nodeBuffer = acquiredContent; + + buffers.audioBuffer = std::make_shared( + RENDER_QUANTUM_SIZE, + buffers.nodeBuffer->getNumberOfChannels(), + audioBufferSourceNode_->getContextSampleRate()); + return buffers; +} + +void AudioBufferSourceNodeHostObject::setBuffer(const std::shared_ptr &buffer) { + assignedBuffer_ = buffer; + // Publish the new output width on the host thread before renegotiation so // MAX / CLAMPED_MAX downstream nodes see it immediately + const size_t newOutputChannelNumber = buffer == nullptr + ? AudioBufferSourceOptions::kDefaultOutputChannelNumber + : buffer->getSamples()->getNumberOfChannels(); if (newOutputChannelNumber != audioBufferSourceNode_->getOutputChannelNumber()) { audioBufferSourceNode_->setOutputChannelNumber(newOutputChannelNumber); renegotiate(); } - auto event = - [handle, node = audioBufferSourceNode_, copiedBuffer, audioBuffer](BaseAudioContext &) { - node->setBuffer(copiedBuffer, audioBuffer); - }; - audioBufferSourceNode_->scheduleAudioEvent(std::move(event)); + // Per Web Audio, assigning a buffer to an already-started source acquires its + // content right away, because start() had nothing to acquire back then. + if (hasBeenStarted_) { + acquireBufferContent(); + } } } // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.h b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.h index b82b9709f..1f986ff4a 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/sources/AudioBufferSourceNodeHostObject.h @@ -11,7 +11,7 @@ using namespace facebook; struct AudioBufferSourceOptions; class BaseAudioContext; -class AudioBufferHostObject; +class SharedAudioBuffer; class AudioBufferSourceNode; class AudioBufferSourceNodeHostObject : public AudioBufferBaseSourceNodeHostObject { @@ -33,7 +33,7 @@ class AudioBufferSourceNodeHostObject : public AudioBufferBaseSourceNodeHostObje JSI_PROPERTY_SETTER_DECL(loopEnd); JSI_PROPERTY_SETTER_DECL(onloopended); - JSI_HOST_FUNCTION_DECL(start); + JSI_HOST_FUNCTION_DECL(start) override; JSI_HOST_FUNCTION_DECL(setBuffer); [[nodiscard]] size_t getMemoryPressure() const override { @@ -51,7 +51,29 @@ class AudioBufferSourceNodeHostObject : public AudioBufferBaseSourceNodeHostObje double loopStart_; double loopEnd_; - void setBuffer(const std::shared_ptr &buffer); + /// The buffer assigned last, held here until "acquire the content" hands its samples to + /// the audio thread. Until then the node has not seen it, so JS can keep writing into it + /// and every write is honoured. Null when cleared. + std::shared_ptr assignedBuffer_; + bool hasBeenStarted_ = false; + + /// The samples the node will read (storage shared with the assigned buffer) plus its + /// render-quantum scratch buffer. + struct NodeBuffers { + std::shared_ptr nodeBuffer; + std::shared_ptr audioBuffer; + }; + + NodeBuffers prepareNodeBuffers(const std::shared_ptr &acquiredContent); + + /// Records the assignment and publishes the buffer's channel count to the graph. The + /// samples themselves reach the node only through `acquireBufferContent`, which runs + /// right away when the node has already started. + void setBuffer(const std::shared_ptr &buffer); + + /// Web Audio's "acquire the content" step, the single point where the node receives + /// samples: runs on start(), and on setBuffer() once already started. + void acquireBufferContent(); }; } // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/utils/NodeOptionsParser.h b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/utils/NodeOptionsParser.h index 84273fb88..75985cd21 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/utils/NodeOptionsParser.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/HostObjects/utils/NodeOptionsParser.h @@ -164,8 +164,9 @@ inline ConvolverOptions parseConvolverOptions( auto bufferValue = optionsObject.getProperty(runtime, "buffer"); if (bufferValue.isObject() && bufferValue.getObject(runtime).isHostObject(runtime)) { - options.buffer = - bufferValue.getObject(runtime).asHostObject(runtime)->audioBuffer_; + options.buffer = bufferValue.getObject(runtime) + .asHostObject(runtime) + ->sharedBuffer_->getSamples(); } } return options; @@ -284,15 +285,6 @@ inline AudioBufferSourceOptions parseAudioBufferSourceOptions( const jsi::Object &optionsObject) { AudioBufferSourceOptions options(parseBaseAudioBufferSourceOptions(runtime, optionsObject)); - if (optionsObject.hasProperty(runtime, "buffer")) { - auto bufferValue = optionsObject.getProperty(runtime, "buffer"); - if (bufferValue.isObject() && - bufferValue.getObject(runtime).isHostObject(runtime)) { - options.buffer = - bufferValue.getObject(runtime).asHostObject(runtime)->audioBuffer_; - } - } - auto loopValue = optionsObject.getProperty(runtime, "loop"); if (loopValue.isBool()) { options.loop = loopValue.getBool(); diff --git a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.cpp b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.cpp index d2724b7ff..e92110289 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.cpp @@ -49,6 +49,7 @@ void AudioBufferBaseSourceNode::initStretch( const std::shared_ptr &playbackRateBuffer) { wsolaStretcher_.configure(channelCount, sampleRate); playbackRateBuffer_ = playbackRateBuffer; + stretchTailFramesLeft_ = WsolaTimeStretcher::latencyTailFrames(sampleRate); } std::shared_ptr AudioBufferBaseSourceNode::getDetuneParam() const { @@ -141,8 +142,19 @@ void AudioBufferBaseSourceNode::processWithPitchCorrection( detune / static_cast(SEMITONES_PER_OCTAVE)); const float bufferPlaybackRate = rate >= 0.0f ? rate : -rate; + const bool stopTimeReached = isStopScheduled(); runBufferProcessor(playbackRateBuffer_, startOffset, offsetLength, bufferPlaybackRate, false); + // The source ran dry (as opposed to stop() time arriving), but the stretcher still + // holds audio. The processor has already zero-filled the rest of the input, so keep + // feeding that silence until the tail has drained, then let the node finish. + const bool sourceRanDry = !stopTimeReached && isStopScheduled(); + if (sourceRanDry && stretchTailFramesLeft_ > 0) { + playbackState_ = PlaybackState::PLAYING; + stretchTailFramesLeft_ -= + std::min(stretchTailFramesLeft_, static_cast(framesNeededToStretch)); + } + wsolaStretcher_.process( *playbackRateBuffer_, static_cast(framesNeededToStretch), diff --git a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.h b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.h index d4f4d6bf5..d3d3e7157 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferBaseSourceNode.h @@ -27,7 +27,7 @@ class AudioBufferBaseSourceNode : public AudioScheduledSourceNode { const std::shared_ptr &context, const BaseAudioBufferSourceOptions &options); - /// @note Audio Thread only + /// @note JS Thread only, and only while the node has no content yet void initStretch( size_t channelCount, float sampleRate, @@ -70,6 +70,10 @@ class AudioBufferBaseSourceNode : public AudioScheduledSourceNode { const bool pitchCorrection_; WsolaTimeStretcher wsolaStretcher_; std::shared_ptr playbackRateBuffer_; + /// Silence still owed to the stretcher after the source runs dry, so the audio it + /// buffers is played out instead of being cut off with the last real sample. This + /// replaces padding the buffer itself, which would force a private copy per node. + size_t stretchTailFramesLeft_ = 0; const std::shared_ptr detuneParam_; const std::shared_ptr playbackRateParam_; diff --git a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.cpp b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.cpp index d06fda2c3..0484ccc6f 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.cpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.cpp @@ -25,7 +25,7 @@ AudioBufferSourceNode::AudioBufferSourceNode( loopStart_(options.loopStart), loopEnd_(options.loopEnd), onLoopEndedEvent_(context->getAudioEventHandlerRegistry(), context->getAudioEventProducer()) { - auto onLoopEnded = [this]() { + auto onLoopEnded = [this] { sendOnLoopEndedEvent(); }; @@ -73,18 +73,8 @@ void AudioBufferSourceNode::setBuffer( context->getDisposer()->dispose(std::move(audioBuffer_)); } - if (buffer == nullptr) { - loopEnd_ = 0; - - buffer_ = nullptr; - processor_->setBuffer(nullptr); - audioBuffer_ = audioBuffer; - return; - } - buffer_ = buffer; audioBuffer_ = audioBuffer; - loopEnd_ = buffer_->getDuration(); processor_->setBuffer(buffer_); } @@ -99,13 +89,16 @@ void AudioBufferSourceNode::start(double when, double offset, double duration) { return; } - offset = std::min(offset, static_cast(buffer_->getSize()) / buffer_->getSampleRate()); + // Past the (effective) loop end, or past the buffer when not looping, playback starts + // at that boundary instead. + const auto sampleRate = buffer_->getSampleRate(); + offset = std::min(offset, getVirtualEndFrame(sampleRate) / sampleRate); if (loop_) { offset = std::min(offset, loopEnd_); } - vReadIndex_ = buffer_->getSampleRate() * offset; + vReadIndex_ = sampleRate * offset; } void AudioBufferSourceNode::disable() { diff --git a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.h b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.h index 37ae6c803..4541bd9cc 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/AudioBufferSourceNode.h @@ -29,6 +29,9 @@ class AudioBufferSourceNode : public AudioBufferBaseSourceNode { /// @note Audio Thread only void setLoopEnd(double loopEnd); + [[nodiscard]] double getLoopEnd() const { + return loopEnd_; + } /// @note Audio Thread only void setBuffer( diff --git a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/SharedAudioBuffer.cpp b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/SharedAudioBuffer.cpp new file mode 100644 index 000000000..b8665db0f --- /dev/null +++ b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/SharedAudioBuffer.cpp @@ -0,0 +1,75 @@ +#include + +#include +#include +#include +#include + +namespace audioapi { + +SharedAudioBuffer::SharedAudioBuffer(std::shared_ptr samples) + : samples_(std::move(samples)), + channelSharedWithNode_(samples_->getNumberOfChannels(), false), + channelExposed_(samples_->getNumberOfChannels(), false) {} + +std::shared_ptr SharedAudioBuffer::exposeChannel(size_t channel) { + makeChannelWritable(channel); + channelExposed_[channel] = true; + return samples_->getSharedChannel(channel); +} + +void SharedAudioBuffer::copyFromChannel( + float *destination, + size_t destinationLength, + size_t channel, + double startInChannel) const { + auto channelSize = samples_->getSize(); + if (startInChannel < 0 || startInChannel >= static_cast(channelSize)) { + return; + } + + auto start = static_cast(startInChannel); + auto framesToCopy = std::min(destinationLength, channelSize - start); + samples_->getChannel(channel)->copyTo(destination, start, 0, framesToCopy); +} + +void SharedAudioBuffer::copyToChannel( + const float *source, + size_t sourceLength, + size_t channel, + double startInChannel) { + auto channelSize = samples_->getSize(); + if (startInChannel < 0 || startInChannel >= static_cast(channelSize)) { + return; + } + + makeChannelWritable(channel); + auto start = static_cast(startInChannel); + auto framesToCopy = std::min(sourceLength, channelSize - start); + samples_->getChannel(channel)->copy(source, 0, start, framesToCopy); +} + +std::shared_ptr SharedAudioBuffer::acquireContent() { + for (size_t channel = 0; channel < channelExposed_.size(); ++channel) { + if (channelExposed_[channel]) { + replaceChannelStorage(channel); + channelExposed_[channel] = false; + } + } + + channelSharedWithNode_.assign(samples_->getNumberOfChannels(), true); + return samples_->shareChannels(); +} + +void SharedAudioBuffer::makeChannelWritable(size_t channel) { + if (channelSharedWithNode_[channel]) { + replaceChannelStorage(channel); + } +} + +void SharedAudioBuffer::replaceChannelStorage(size_t channel) { + samples_->detachSharedChannel(channel); + channelSharedWithNode_[channel] = false; +} + +} // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/audioapi/core/sources/SharedAudioBuffer.h b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/SharedAudioBuffer.h new file mode 100644 index 000000000..d2622f6fb --- /dev/null +++ b/packages/react-native-audio-api/common/cpp/audioapi/core/sources/SharedAudioBuffer.h @@ -0,0 +1,71 @@ +#pragma once + +#include +#include + +#include +#include +#include + +namespace audioapi { + +/// @brief Samples of an AudioBuffer, shared between the control thread and the source +/// nodes playing them. Nodes read the channel storage directly instead of deep-copying it, +/// so a js trying to write invokes copy-on-write and leaves the old one to the nodes. +/// All members must be called on the JS thread. +class SharedAudioBuffer { + public: + explicit SharedAudioBuffer(std::shared_ptr samples); + + [[nodiscard]] std::shared_ptr getSamples() const { + return samples_; + } + + [[nodiscard]] size_t getSizeInBytes() const { + return samples_->getSize() * samples_->getNumberOfChannels() * sizeof(float); + } + + /// @brief Storage of @p channel for a JS-writable view (`getChannelData`). Writes through + /// it reach this buffer until the next `acquireContent` cuts the view off. + [[nodiscard]] std::shared_ptr exposeChannel(size_t channel); + + /// @brief An out-of-range (or negative) @p startInChannel copies nothing, and a copy that + /// would run past the end of the channel is truncated rather than failing. @p startInChannel + /// is the unconverted JS number, so a negative value is rejected instead of wrapping around. + /// https://webaudio.github.io/web-audio-api/#dom-audiobuffer-copyfromchannel + void copyFromChannel( + float *destination, + size_t destinationLength, + size_t channel, + double startInChannel) const; + + /// @brief Same range rules as `copyFromChannel`. + void + copyToChannel(const float *source, size_t sourceLength, size_t channel, double startInChannel); + + /// @brief Web Audio's "acquire the content" step, run when a source node schedules + /// playback of this buffer. Cuts off every view handed out by `exposeChannel`, then + /// returns a buffer sharing the current storage for the node to play, so JS writes made + /// before this call are played and later ones are not. + /// https://webaudio.github.io/web-audio-api/#acquire-the-content + [[nodiscard]] std::shared_ptr acquireContent(); + + private: + std::shared_ptr samples_; + + /// One flag per channel: true while a buffer returned by `acquireContent` may still be + /// reading that channel's current storage. + std::vector channelSharedWithNode_; + /// One flag per channel: true once `exposeChannel` handed out that channel's current + /// storage, cleared when `acquireContent` cuts the view off. + std::vector channelExposed_; + + /// Moves @p channel to fresh storage if a node may be reading its current one; afterwards + /// the channel is safe to write or expose. + void makeChannelWritable(size_t channel); + + /// Gives the channel fresh storage holding a copy of its samples, shared with nothing. + void replaceChannelStorage(size_t channel); +}; + +} // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/audioapi/dsp/WsolaTimeStretcher.h b/packages/react-native-audio-api/common/cpp/audioapi/dsp/WsolaTimeStretcher.h index 9af54135b..8272f9d0b 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/dsp/WsolaTimeStretcher.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/dsp/WsolaTimeStretcher.h @@ -41,10 +41,18 @@ class WsolaTimeStretcher { // some arbitrary value has to be set here to limit the size of the buffer for wsola algorithm static constexpr float MAX_PLAYBACK_RATE = 4; - /// Rough latency estimates for buffer tail padding (seconds). + /// Rough latency estimates (milliseconds) reported to JS and used to size the + /// silent tail that drains the stretcher once its source runs out. static constexpr float INPUT_LATENCY_MS = 20.0f; static constexpr float OUTPUT_LATENCY_MS = 10.0f; + /// @brief How many frames of silence a source has to keep feeding after its last real + /// sample so the audio still buffered in the stretcher reaches the output. + [[nodiscard]] static size_t latencyTailFrames(float sampleRate) { + constexpr float kMsPerSecond = 1000.0f; + return static_cast((INPUT_LATENCY_MS + OUTPUT_LATENCY_MS) / kMsPerSecond * sampleRate); + } + private: static constexpr float OLA_WINDOW_MS = 20.0f; static constexpr float SEARCH_INTERVAL_MS = 30.0f; diff --git a/packages/react-native-audio-api/common/cpp/audioapi/events/AudioEventPayload.h b/packages/react-native-audio-api/common/cpp/audioapi/events/AudioEventPayload.h index f2d9a4059..27778b8f0 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/events/AudioEventPayload.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/events/AudioEventPayload.h @@ -80,7 +80,7 @@ struct AudioReadyPayload { facebook::jsi::Object toJsiObject(facebook::jsi::Runtime &rt) const { facebook::jsi::Object obj(rt); - const bool hasBuffer = buffer != nullptr && buffer->audioBuffer_ != nullptr; + const bool hasBuffer = buffer != nullptr && buffer->sharedBuffer_ != nullptr; if (hasBuffer) { obj.setProperty(rt, "buffer", facebook::jsi::Object::createFromHostObject(rt, buffer)); diff --git a/packages/react-native-audio-api/common/cpp/audioapi/types/NodeOptions.h b/packages/react-native-audio-api/common/cpp/audioapi/types/NodeOptions.h index 290c1e12e..d8cc8500e 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/types/NodeOptions.h +++ b/packages/react-native-audio-api/common/cpp/audioapi/types/NodeOptions.h @@ -121,7 +121,7 @@ struct PannerOptions : public AudioNodeOptions { struct ConvolverOptions : AudioNodeOptions { bool disableNormalization = false; - std::shared_ptr buffer = nullptr; + std::shared_ptr buffer = nullptr; ConvolverOptions() { requiresTailProcessing = true; @@ -191,7 +191,6 @@ struct AudioBufferSourceOptions : BaseAudioBufferSourceOptions { /// Spec: with no buffer assigned the node emits one channel of silence. static constexpr size_t kDefaultOutputChannelNumber = 1; - std::shared_ptr buffer = nullptr; float loopStart = 0.0f; float loopEnd = 0.0f; bool loop = false; diff --git a/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioArrayBuffer.hpp b/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioArrayBuffer.hpp index 4094d7ae1..535c19cb8 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioArrayBuffer.hpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioArrayBuffer.hpp @@ -2,21 +2,13 @@ #include -#if !RN_AUDIO_API_TEST #include -using JsiBuffer = facebook::jsi::MutableBuffer; -#else -// Dummy class to inherit from nothing if testing -struct JsiBuffer { - virtual size_t size() const = 0; - virtual uint8_t *data() = 0; -}; -#endif namespace audioapi { template -class AlignedAudioArrayBuffer : public JsiBuffer, public AlignedAudioArray { +class AlignedAudioArrayBuffer : public facebook::jsi::MutableBuffer, + public AlignedAudioArray { public: explicit AlignedAudioArrayBuffer(size_t size) : AlignedAudioArray(size) {}; AlignedAudioArrayBuffer(const float *data, size_t size) diff --git a/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioBuffer.hpp b/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioBuffer.hpp index 8f32141fa..6ac1f1550 100644 --- a/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioBuffer.hpp +++ b/packages/react-native-audio-api/common/cpp/audioapi/utils/AudioBuffer.hpp @@ -141,6 +141,26 @@ class AlignedAudioBuffer { return channels_[index]; } + /// @brief Creates a buffer that shares this buffer's channel storage instead of copying + /// it. Both buffers read the same samples until one of them calls detachSharedChannel() + /// on a channel, which is how a source node can play a JS-visible buffer without a deep + /// copy while the JS side keeps copy-on-write semantics. + [[nodiscard]] std::shared_ptr shareChannels() const { + auto shared = std::make_shared(); + shared->numberOfChannels_ = numberOfChannels_; + shared->sampleRate_ = sampleRate_; + shared->size_ = size_; + shared->channels_ = channels_; + return shared; + } + + /// @brief Gives channel @p index fresh storage holding a copy of its current samples. + /// Every handle previously obtained through getSharedChannel() keeps the old storage + /// alive but no longer points to this buffer, so writes through it can't reach here anymore. + void detachSharedChannel(size_t index) { + channels_[index] = std::make_shared>(*channels_[index]); + } + AlignedAudioArray &operator[](size_t index) { return *channels_[index]; } @@ -345,13 +365,18 @@ class AlignedAudioBuffer { {2, {ChannelLeft, ChannelRight}}, {4, {ChannelLeft, ChannelRight, ChannelSurroundLeft, ChannelSurroundRight}}, {5, {ChannelLeft, ChannelRight, ChannelCenter, ChannelSurroundLeft, ChannelSurroundRight}}, - {6, - {ChannelLeft, - ChannelRight, - ChannelCenter, - ChannelLFE, - ChannelSurroundLeft, - ChannelSurroundRight}}}; + { + 6, + { + ChannelLeft, + ChannelRight, + ChannelCenter, + ChannelLFE, + ChannelSurroundLeft, + ChannelSurroundRight, + }, + }, + }; template void discreteSum( @@ -365,6 +390,8 @@ class AlignedAudioBuffer { } } + // NOLINTBEGIN(cppcoreguidelines-avoid-magic-numbers, readability-magic-numbers) + template void sumByUpMixing( const AlignedAudioBuffer &source, @@ -464,3 +491,5 @@ using AudioBuffer = AlignedAudioBuffer; using DSPAudioBuffer = AlignedAudioBuffer; } // namespace audioapi + +// NOLINTEND(cppcoreguidelines-avoid-magic-numbers, readability-magic-numbers) diff --git a/packages/react-native-audio-api/common/cpp/clangd/CMakeLists.txt b/packages/react-native-audio-api/common/cpp/clangd/CMakeLists.txt index 89c173456..6b9131291 100644 --- a/packages/react-native-audio-api/common/cpp/clangd/CMakeLists.txt +++ b/packages/react-native-audio-api/common/cpp/clangd/CMakeLists.txt @@ -5,8 +5,6 @@ set(CMAKE_EXPORT_COMPILE_COMMANDS ON) set(CMAKE_CXX_STANDARD 20) set(COMMON_CPP_DIR ${CMAKE_SOURCE_DIR}/..) -# =1 swaps real types for dummy stand-ins too (e.g. JsiBuffer), so production -# code needs =0; only test/*.cpp needs =1 — split via separate targets below. set(RN_AUDIO_API_TEST 0) set(REACT_NATIVE_DIR ../../../../../node_modules/react-native) diff --git a/packages/react-native-audio-api/common/cpp/test/CMakeLists.txt b/packages/react-native-audio-api/common/cpp/test/CMakeLists.txt index f9a76bd33..2ae3081ee 100644 --- a/packages/react-native-audio-api/common/cpp/test/CMakeLists.txt +++ b/packages/react-native-audio-api/common/cpp/test/CMakeLists.txt @@ -45,6 +45,7 @@ list(FILTER RNAUDIOAPI_SRC EXCLUDE REGEX ".*/audioapi/HostObjects/.*\\.cpp$") list(APPEND RNAUDIOAPI_SRC "${REACT_NATIVE_AUDIO_API_DIR}/common/cpp/audioapi/HostObjects/utils/JsEnumParser.cpp" + "${REACT_NATIVE_AUDIO_API_DIR}/common/cpp/audioapi/HostObjects/sources/AudioBufferHostObject.cpp" ) # AudioThreadGuard provides global operator new/delete overrides. Compile it into diff --git a/packages/react-native-audio-api/common/cpp/test/src/AudioBufferHostObjectStub.cpp b/packages/react-native-audio-api/common/cpp/test/src/AudioBufferHostObjectStub.cpp deleted file mode 100644 index 619b6db9b..000000000 --- a/packages/react-native-audio-api/common/cpp/test/src/AudioBufferHostObjectStub.cpp +++ /dev/null @@ -1,22 +0,0 @@ -#include - -#include -#include - -/// The test build compiles no HostObject sources, yet core code it does compile -/// (AudioRecorderCallback, ContextPromiseResolver) constructs an AudioBufferHostObject -/// when a recording or decode finishes. These definitions satisfy the linker in place of -/// AudioBufferHostObject.cpp, which cannot be compiled here: under RN_AUDIO_API_TEST an -/// AudioArrayBuffer is not a jsi::MutableBuffer (see AudioArrayBuffer.hpp), so its JSI -/// accessor bodies do not build. The stub registers no accessors, and no test ever -/// reaches the object through a JS runtime. - -namespace audioapi { - -AudioBufferHostObject::AudioBufferHostObject(const std::shared_ptr &audioBuffer) - : audioBuffer_(audioBuffer) {} - -AudioBufferHostObject::AudioBufferHostObject(AudioBufferHostObject &&other) noexcept - : HostObject(std::move(other)), audioBuffer_(std::move(other.audioBuffer_)) {} - -} // namespace audioapi diff --git a/packages/react-native-audio-api/common/cpp/test/src/core/sources/AudioBufferSourceNodeTest.cpp b/packages/react-native-audio-api/common/cpp/test/src/core/sources/AudioBufferSourceNodeTest.cpp index 2767ad5b1..424d2f745 100644 --- a/packages/react-native-audio-api/common/cpp/test/src/core/sources/AudioBufferSourceNodeTest.cpp +++ b/packages/react-native-audio-api/common/cpp/test/src/core/sources/AudioBufferSourceNodeTest.cpp @@ -95,23 +95,19 @@ class AudioBufferSourceNodeTest : public ::testing::Test { } std::unique_ptr makeNode(bool pitchCorrection) { - BaseAudioBufferSourceOptions baseOptions; - baseOptions.pitchCorrection = pitchCorrection; - baseOptions.playbackRate = 1.0f; - baseOptions.detune = 0.0f; - AudioBufferSourceOptions options(baseOptions); - - auto node = std::make_unique(context, options); + auto node = makeNodeWithOptions(pitchCorrection); - // Enough source for NUM_QUANTA at 2x plus WSOLA tail padding (mirrors HostObject setBuffer). - const size_t extraTailFrames = static_cast( - (WsolaTimeStretcher::INPUT_LATENCY_MS + WsolaTimeStretcher::OUTPUT_LATENCY_MS) / 1000.0f * - SAMPLE_RATE); - const size_t bufferFrames = static_cast(NUM_QUANTA * QUANTUM * 4) + extraTailFrames; - auto buffer = makeSilentBuffer(bufferFrames); - auto outputBuffer = - std::make_shared(QUANTUM, 1, static_cast(SAMPLE_RATE)); + // Enough source for NUM_QUANTA at 2x. + return startWithBuffer( + std::move(node), makeSilentBuffer(NUM_QUANTA * QUANTUM * 4), pitchCorrection); + } + /// Mirrors the HostObject: a pitch-corrected node gets its stretcher initialised + /// before the buffer lands, and the buffer is handed over as-is (never padded). + std::unique_ptr startWithBuffer( + std::unique_ptr node, + const std::shared_ptr &buffer, + bool pitchCorrection) { if (pitchCorrection) { auto playbackRateBuffer = std::make_shared( static_cast(WsolaTimeStretcher::MAX_PLAYBACK_RATE * QUANTUM), @@ -120,11 +116,20 @@ class AudioBufferSourceNodeTest : public ::testing::Test { node->initStretch(1, static_cast(SAMPLE_RATE), playbackRateBuffer); } - node->setBuffer(buffer, outputBuffer); + node->setBuffer(buffer, makeOutputBuffer()); node->start(0.0); return node; } + std::unique_ptr makeNodeWithOptions(bool pitchCorrection) { + BaseAudioBufferSourceOptions baseOptions; + baseOptions.pitchCorrection = pitchCorrection; + baseOptions.playbackRate = 1.0f; + baseOptions.detune = 0.0f; + return std::make_unique( + context, AudioBufferSourceOptions(baseOptions)); + } + std::unique_ptr makeNodeWithoutBuffer() { return std::make_unique( context, AudioBufferSourceOptions(BaseAudioBufferSourceOptions())); @@ -269,6 +274,73 @@ TEST_F(AudioBufferSourceNodeTest, BufferSwapKeepsHostPublishedOutputChannelNumbe EXPECT_EQ(node->getOutputChannelNumber(), 1u); } +/// The stretcher holds ~INPUT_LATENCY + OUTPUT_LATENCY of audio. The buffer used to be +/// copied with that much silence appended so the node outlived its last sample; now +/// the node plays the buffer as handed over and feeds the stretcher silence itself +/// once the source runs dry, ending only after the tail has drained. +TEST_F(AudioBufferSourceNodeTest, PitchCorrectedSourceDrainsStretcherTailAfterBufferEnds) { + static constexpr uint64_t ENDED_CALLBACK_ID = 54; + // At the modulated 2x rate the whole buffer is consumed in two quanta. + static constexpr size_t BUFFER_FRAMES = 2 * QUANTUM * 2; + static constexpr size_t QUANTA_TO_CONSUME_BUFFER = 2; + const size_t tailFrames = WsolaTimeStretcher::latencyTailFrames(SAMPLE_RATE); + const size_t quantaToDrainTail = tailFrames / (2 * QUANTUM) + 1; + + auto *node = addNode(startWithBuffer( + makeNodeWithOptions(/*pitchCorrection=*/true), + makeSilentBuffer(BUFFER_FRAMES), + /*pitchCorrection=*/true)); + node->assignOnEndedCallbackId(ENDED_CALLBACK_ID); + EXPECT_CALL(*eventRegistry, unregisterHandler(AudioEvent::ENDED, ENDED_CALLBACK_ID)) + .Times(testing::AnyNumber()); + EXPECT_CALL( + *eventRegistry, + dispatchEventFromAudioThread(testing::_, AudioEvent::ENDED, ENDED_CALLBACK_ID, testing::_)) + .WillOnce(testing::Return(true)); + + for (size_t q = 0; q < QUANTA_TO_CONSUME_BUFFER; ++q) { + renderQuantumWithModulation(*node, MODULATION); + } + EXPECT_TRUE(node->isPlaying()) << "the source ran dry but the stretcher tail is not drained"; + + const double bufferDuration = static_cast(BUFFER_FRAMES) / SAMPLE_RATE; + size_t quantaUntilEnded = 0; + while (!node->isFinished() && quantaUntilEnded <= quantaToDrainTail) { + renderQuantumWithModulation(*node, MODULATION); + ++quantaUntilEnded; + if (node->isPlaying()) { + EXPECT_LE(node->getCurrentPosition(), bufferDuration) + << "the reported position must not run past the real buffer into the tail"; + } + } + + EXPECT_TRUE(node->isFinished()) << "the node never ended after the tail drained"; + EXPECT_GE(quantaUntilEnded, quantaToDrainTail - 1); +} + +/// Without the stretcher in the loop (rate 1 skips it) there is nothing to drain, so a +/// pitch-corrected node ends as soon as its buffer does. +TEST_F(AudioBufferSourceNodeTest, PitchCorrectedSourceAtUnityRateEndsWithItsBuffer) { + static constexpr uint64_t ENDED_CALLBACK_ID = 55; + static constexpr size_t BUFFER_FRAMES = 2 * QUANTUM; + + auto *node = addNode(startWithBuffer( + makeNodeWithOptions(/*pitchCorrection=*/true), + makeSilentBuffer(BUFFER_FRAMES), + /*pitchCorrection=*/true)); + node->assignOnEndedCallbackId(ENDED_CALLBACK_ID); + EXPECT_CALL(*eventRegistry, unregisterHandler(AudioEvent::ENDED, ENDED_CALLBACK_ID)) + .Times(testing::AnyNumber()); + EXPECT_CALL( + *eventRegistry, + dispatchEventFromAudioThread(testing::_, AudioEvent::ENDED, ENDED_CALLBACK_ID, testing::_)) + .WillOnce(testing::Return(true)); + + processGraph(); + processGraph(); + EXPECT_TRUE(node->isFinished()); +} + /// A source that is disconnected before its stop frame is never pulled again, so its /// render step (the only place it used to advance to FINISHED) never runs. It must still /// end once the clock passes the stop frame, fire `ended` exactly once and then be diff --git a/packages/react-native-audio-api/common/cpp/test/src/core/sources/SharedAudioBufferTest.cpp b/packages/react-native-audio-api/common/cpp/test/src/core/sources/SharedAudioBufferTest.cpp new file mode 100644 index 000000000..05bc4fbbb --- /dev/null +++ b/packages/react-native-audio-api/common/cpp/test/src/core/sources/SharedAudioBufferTest.cpp @@ -0,0 +1,129 @@ +#include +#include +#include + +#include +#include + +using namespace audioapi; + +// NOLINTBEGIN +static constexpr size_t BUF_SIZE = 8; +static constexpr float SR = 44100.0f; + +class SharedAudioBufferTest : public ::testing::Test { + protected: + std::shared_ptr samples = std::make_shared(BUF_SIZE, 2, SR); + SharedAudioBuffer buffer{samples}; + + void SetUp() override { + fill(*samples, 0, 0.5f); + fill(*samples, 1, 0.25f); + } + + static void fill(AudioBuffer &target, size_t channel, float value) { + for (size_t i = 0; i < target.getSize(); ++i) { + (*target.getChannel(channel))[i] = value; + } + } + + void writeToChannel(size_t channel, float value, double startInChannel = 0) { + std::vector source(BUF_SIZE, value); + buffer.copyToChannel(source.data(), source.size(), channel, startInChannel); + } + + static void expectChannel(const AudioBuffer &target, size_t channel, float value) { + for (size_t i = 0; i < target.getSize(); ++i) { + EXPECT_FLOAT_EQ((*target.getChannel(channel))[i], value) + << "channel " << channel << " frame " << i; + } + } +}; + +TEST_F(SharedAudioBufferTest, AcquireContentSharesStorageInsteadOfCopying) { + auto content = buffer.acquireContent(); + + EXPECT_EQ(content->getSharedChannel(0), samples->getSharedChannel(0)); + EXPECT_EQ(content->getSharedChannel(1), samples->getSharedChannel(1)); +} + +TEST_F(SharedAudioBufferTest, CopyToChannelAfterAcquireDoesNotReachThePlayedContent) { + auto content = buffer.acquireContent(); + + writeToChannel(0, 1.0f); + + expectChannel(*content, 0, 0.5f); + expectChannel(*buffer.getSamples(), 0, 1.0f); + EXPECT_EQ(content->getSharedChannel(1), buffer.getSamples()->getSharedChannel(1)) + << "Only the written channel is copied."; +} + +TEST_F(SharedAudioBufferTest, CopyToChannelWithoutAcquireWritesInPlace) { + auto storageBefore = samples->getSharedChannel(0); + + writeToChannel(0, 1.0f); + + EXPECT_EQ(samples->getSharedChannel(0), storageBefore); + expectChannel(*samples, 0, 1.0f); +} + +TEST_F(SharedAudioBufferTest, WritesThroughAnExposedViewBeforeAcquireArePlayed) { + auto view = buffer.exposeChannel(0); + (*view)[2] = 1.0f; + + auto content = buffer.acquireContent(); + + EXPECT_FLOAT_EQ((*content->getChannel(0))[2], 1.0f); +} + +TEST_F(SharedAudioBufferTest, AcquireContentCutsOffExposedViews) { + auto view = buffer.exposeChannel(0); + auto content = buffer.acquireContent(); + + (*view)[2] = 1.0f; + + expectChannel(*content, 0, 0.5f); + expectChannel(*buffer.getSamples(), 0, 0.5f); + EXPECT_NE(buffer.exposeChannel(0), view) << "The next view must point to fresh storage."; +} + +TEST_F(SharedAudioBufferTest, ExposingAChannelAfterAcquireDoesNotHandOutPlayedStorage) { + auto content = buffer.acquireContent(); + + auto view = buffer.exposeChannel(1); + (*view)[0] = 1.0f; + + expectChannel(*content, 1, 0.25f); +} + +TEST_F(SharedAudioBufferTest, ExposingTheSameChannelTwiceReusesItsStorage) { + auto first = buffer.exposeChannel(0); + + EXPECT_EQ(buffer.exposeChannel(0), first); +} + +TEST_F(SharedAudioBufferTest, OutOfRangeStartCopiesNothing) { + writeToChannel(0, 1.0f, -1); + writeToChannel(0, 1.0f, BUF_SIZE); + + expectChannel(*samples, 0, 0.5f); + + std::vector destination(BUF_SIZE, -1.0f); + buffer.copyFromChannel(destination.data(), destination.size(), 0, -1); + buffer.copyFromChannel(destination.data(), destination.size(), 0, BUF_SIZE); + + EXPECT_EQ(destination, std::vector(BUF_SIZE, -1.0f)); +} + +TEST_F(SharedAudioBufferTest, CopiesPastTheEndOfTheChannelAreTruncated) { + writeToChannel(0, 1.0f, BUF_SIZE - 2); + + std::vector destination(BUF_SIZE, -1.0f); + buffer.copyFromChannel(destination.data(), destination.size(), 0, BUF_SIZE - 3); + + EXPECT_FLOAT_EQ(destination[0], 0.5f); + EXPECT_FLOAT_EQ(destination[1], 1.0f); + EXPECT_FLOAT_EQ(destination[2], 1.0f); + EXPECT_FLOAT_EQ(destination[3], -1.0f) << "Frames past the channel end stay untouched."; +} +// NOLINTEND diff --git a/packages/react-native-audio-api/common/cpp/test/src/utils/AudioBufferTest.cpp b/packages/react-native-audio-api/common/cpp/test/src/utils/AudioBufferTest.cpp index 6f458af68..bba6bd061 100644 --- a/packages/react-native-audio-api/common/cpp/test/src/utils/AudioBufferTest.cpp +++ b/packages/react-native-audio-api/common/cpp/test/src/utils/AudioBufferTest.cpp @@ -634,4 +634,49 @@ TEST_F(AudioBufferTest, DeinterleaveZeroFramesIsNoop) { expectChannel(buf, 0, 42.0f); } +TEST_F(AudioBufferTest, DetachSharedChannelCutsOffEscapedHandleAndKeepsSamples) { + AudioBuffer buf(BUF_SIZE, 2, SR); + fillChannel(buf, 0, 0.5f); + fillChannel(buf, 1, 0.25f); + auto escapedHandle = buf.getSharedChannel(0); + auto untouchedHandle = buf.getSharedChannel(1); + + buf.detachSharedChannel(0); + + EXPECT_NE(buf.getSharedChannel(0), escapedHandle) << "Detached channel must get fresh storage."; + EXPECT_EQ(buf.getSharedChannel(1), untouchedHandle) + << "Other channels keep their storage; only the escaped one is replaced."; + expectChannel(buf, 0, 0.5f); + + (*escapedHandle)[3] = 1.0f; + + expectChannel(buf, 0, 0.5f); + EXPECT_FLOAT_EQ((*escapedHandle)[3], 1.0f) + << "The old handle stays alive and writable, it just no longer reaches the buffer."; +} + +// A source node plays a JS-visible buffer through shareChannels(). The JS side then relies +// on detachSharedChannel() as copy-on-write: its writes must never reach the node's view. +TEST_F(AudioBufferTest, ShareChannelsAliasesStorageUntilOneSideDetaches) { + AudioBuffer owner(BUF_SIZE, 2, SR); + fillChannel(owner, 0, 0.5f); + fillChannel(owner, 1, 0.25f); + + auto shared = owner.shareChannels(); + + ASSERT_EQ(shared->getNumberOfChannels(), 2u); + ASSERT_EQ(shared->getSize(), BUF_SIZE); + ASSERT_FLOAT_EQ(shared->getSampleRate(), SR); + EXPECT_EQ(shared->getSharedChannel(0), owner.getSharedChannel(0)) + << "shareChannels() must alias, not copy."; + + owner.detachSharedChannel(0); + fillChannel(owner, 0, 1.0f); + + expectChannel(*shared, 0, 0.5f); + expectChannel(owner, 0, 1.0f); + EXPECT_EQ(shared->getSharedChannel(1), owner.getSharedChannel(1)) + << "Detaching one channel must not touch the others."; +} + // NOLINTEND diff --git a/packages/react-native-audio-api/src/core/AudioBuffer.ts b/packages/react-native-audio-api/src/core/AudioBuffer.ts index b73fcf441..385d61ee2 100644 --- a/packages/react-native-audio-api/src/core/AudioBuffer.ts +++ b/packages/react-native-audio-api/src/core/AudioBuffer.ts @@ -56,6 +56,18 @@ export default class AudioBuffer implements AudioBufferLike { return data; } + /** + * Forgets the cached channel views once the native side has cut them off from + * the buffer (a source node acquired this buffer's content), so the next + * `getChannelData` hands out a fresh view the way a browser does after it + * detaches the old ones. + * + * @internal + */ + public invalidateChannelDataCache(): void { + this.channelDataCache.length = 0; + } + public copyFromChannel( destination: Float32Array, channelNumber: number, diff --git a/packages/react-native-audio-api/src/core/AudioBufferQueueSourceNode.ts b/packages/react-native-audio-api/src/core/AudioBufferQueueSourceNode.ts index c42bdf433..280dce06c 100644 --- a/packages/react-native-audio-api/src/core/AudioBufferQueueSourceNode.ts +++ b/packages/react-native-audio-api/src/core/AudioBufferQueueSourceNode.ts @@ -24,9 +24,13 @@ export default class AudioBufferQueueSourceNode extends AudioBufferBaseSourceNod } public enqueueBuffer(buffer: AudioBuffer): string { - return (this.node as IAudioBufferQueueSourceNode).enqueueBuffer( + const bufferId = (this.node as IAudioBufferQueueSourceNode).enqueueBuffer( buffer.buffer ); + // The node plays the buffer's own storage from now on, so native has cut off the + // views this buffer handed out; the wrapper must stop returning those dead views. + buffer.invalidateChannelDataCache(); + return bufferId; } public dequeueBuffer(bufferId: string): void { diff --git a/packages/react-native-audio-api/src/core/AudioBufferSourceNode.ts b/packages/react-native-audio-api/src/core/AudioBufferSourceNode.ts index ef6f85bca..b56024a5f 100644 --- a/packages/react-native-audio-api/src/core/AudioBufferSourceNode.ts +++ b/packages/react-native-audio-api/src/core/AudioBufferSourceNode.ts @@ -55,6 +55,12 @@ export default class AudioBufferSourceNode extends AudioBufferBaseSourceNode { (this.node as IAudioBufferSourceNode).setBuffer(buffer.buffer); this._buffer = buffer; this.bufferHasBeenSet = true; + + if (this.hasBeenStarted) { + // Assigning a buffer to an already-started source acquires its content right + // away, so native has just cut off the views this buffer handed out. + buffer.invalidateChannelDataCache(); + } } public get loopSkip(): boolean { @@ -114,6 +120,9 @@ export default class AudioBufferSourceNode extends AudioBufferBaseSourceNode { this.hasBeenStarted = true; (this.node as IAudioBufferSourceNode).start(when, offset, duration); + // Native cut off every view handed out by getChannelData() while acquiring the + // buffer's content, so the wrapper must stop returning those dead views. + this._buffer?.invalidateChannelDataCache(); this.context.markRunningOnSourceStart(); } diff --git a/packages/react-native-audio-api/wpt_tests/wpt/wpt-results.mjs b/packages/react-native-audio-api/wpt_tests/wpt/wpt-results.mjs index babd011c7..0061f1050 100644 --- a/packages/react-native-audio-api/wpt_tests/wpt/wpt-results.mjs +++ b/packages/react-native-audio-api/wpt_tests/wpt/wpt-results.mjs @@ -53,9 +53,6 @@ const COVERAGE_NOTES = { 'the-analysernode-interface': 'One failure: minDecibels/maxDecibels are stored as float, so reading back a ' + 'high-precision double value loses a few digits (cosmetic round-trip mismatch).', - 'the-audiobuffer-interface': - 'Remaining failures: acquire-the-content expects AudioBufferSourceNode.start() to ' + - 'snapshot the buffer contents (we still share the underlying memory).', 'the-biquadfilternode-interface': 'The filter runs in double precision, so the static frequency response matches the ' + 'reference. The remaining failures are the parameter-automation tests, which expect ' +