Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .claude/skills/audio-nodes/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -377,8 +377,10 @@ All graph mutations are queued via `AudioGraphManager` using its own SPSC channe
### Settable channel attributes (channelCount / channelCountMode / channelInterpretation)
These are mutable after construction. `AudioNode` (core) exposes virtual `setChannelCount` / `setChannelCountMode` / `setChannelInterpretation`. `channelCount` and `channelCountMode` are read only on the host thread during negotiation, so the JSI setter updates the core field directly then calls `HostNode::renegotiate()` → `Graph::renegotiateNodeChannels()` → `HostGraph::renegotiateNodeChannels()` (reuses `collectNegotiations` + an `AGEvent` buffer swap, self-drain aware when there is no audio/render consumer — offline construction/suspend and realtime suspended/stopped windows). When `AudioBufferSourceNode` `setBuffer` changes channel width, update `outputChannelNumber_` together with the buffer then `renegotiate()` so MAX/CLAMPED_MAX downstream nodes update; the audio event still installs the prebuilt buffer (no audio-thread alloc). `channelInterpretation` is read on the audio thread in `processInputs` (`getInputBuffer()->sum(*input, channelInterpretation_)`), so it MUST be applied via `scheduleAudioEvent`, not mutated directly.

Negotiation decides whether to queue a buffer swap by comparing against `HostVertex::pendingChannelCount` (the width of the last swap queued, seeded in `HostGraph::addNode`), never against the live `getNegotiatedBuffer()`. On a running context every event queued in one JS task lands in the same render quantum, so the live buffer has not seen earlier swaps from the batch yet: a node going 2 → 1 → 2 would see the live 2, skip the second swap, and keep the queued 1 (issue #1332, e.g. `source.connect(gain); source.buffer = stereo;`). The width a node presents to its consumers is derived on demand as `getOutputChannelNumber().value_or(pendingChannelCount)`; it is not cached per pass, so a negotiation pass only walks upstream (`outputs`) from the changed nodes — inputs outside that walk are already current. Inputs inside it are not: in a triangle (`A → C`, `A → B → C`) the walk can reach C before B, so `collectNegotiations` first collects the walked nodes in post-order (`collectAffectedPostOrder`) and computes them in reverse, which is topological — every node after each input the same pass renegotiates (`GraphTest.RenegotiateNode_TriangleReadsSiblingAfterItNarrows`). Never compute a node while walking. A source (`getNumberOfInputs() == 0`, `ownsOutputBuffer`) is never given a buffer by the graph: it swaps `audioBuffer_` itself, in the same audio event as the content that defines its width, so the two can never disagree on the audio thread. A pass never reaches a source: `connect()` rejects it as a destination (`HostGraph::addEdge` asserts this — it has no guard of its own, so C++ tests calling it raw must respect it too) and `renegotiateNodeChannels(source)` starts the pass at the source's outputs. The same rule applies to any new host-side "is a swap needed?" check.

### Output channel number vs `channelCount` (sources)
`channelCount` is the spec's input-mixing attribute and never decides how many channels a node emits. `AudioNode` keeps a separate atomic `outputChannelNumber_` (`getOutputChannelNumber()`), initialised from `AudioNodeOptions::outputChannelNumber` and falling back to `channelCount` for nodes whose output follows their negotiated input layout. `HostGraph` negotiation reads the output channel number for source inputs (`getUpstreamChannelCount` returns it when `numberOfInputs_ == 0`), so a spec-default `channelCount = 2` oscillator still negotiates a mono downstream buffer. Rules:
`channelCount` is the spec's input-mixing attribute and never decides how many channels a node emits. `AudioNode` keeps a separate atomic `outputChannelNumber_` (`getOutputChannelNumber()`), initialised from `AudioNodeOptions::outputChannelNumber` and falling back to `channelCount` for nodes whose output follows their negotiated input layout. `HostGraph` negotiation reads each input's `getOutputChannelNumber()` and falls back to its pending negotiated width only when that is `nullopt` (`pendingOutputChannelNumber` in `HostGraph.cpp`), so a spec-default `channelCount = 2` oscillator still negotiates a mono downstream buffer. Rules:
- A node that always emits one channel (Oscillator, ConstantSource) derives its options from `MonoSourceNodeOptions`; do not lower `channelCount` or override `setChannelCount` to fake it.
- A source that learns its width later (AudioBufferSource, AudioFileSource, AudioBufferQueueSource, RecorderAdapter) writes `outputChannelNumber_` in the same step it swaps `audioBuffer_`, never `channelCount_`.
- When the host object schedules that swap (AudioBufferSourceNode `setBuffer`), it must first call `setOutputChannelNumber()` on the host thread and `renegotiate()`, otherwise downstream MAX / CLAMPED_MAX nodes negotiate against the stale width until the audio event lands (this broke the StereoPanner WPT `stereopannernode-panning` test when it lived on `updateChannelCount`). Never route this through `updateChannelCount`: that changes the JS-visible `channelCount` attribute.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ class AudioNode : public utils::graph::GraphObject, public std::enable_shared_fr
/// `setChannelCount` from the JS thread is race-free with audio processing.
[[nodiscard]] size_t getChannelCount() const;

/// @brief Number of inputs fixed at construction; 0 for source nodes.
[[nodiscard]] int getNumberOfInputs() const {
return numberOfInputs_;
}

void setOutputChannelNumber(size_t outputChannelNumber) {
outputChannelNumber_.store(static_cast<int>(outputChannelNumber), std::memory_order_release);
}
Expand Down Expand Up @@ -127,9 +132,8 @@ class AudioNode : public utils::graph::GraphObject, public std::enable_shared_fr
setOutputBuffer(buffer);
}

/// @brief Number of channels this node emits toward AudioDestinationNode
/// when that does not follow its `computedNumberOfChannels`. Default:
/// nullopt, the output is as wide as the inputs are mixed to. Sources
/// @brief Number of channels this node emits toward AudioDestinationNode.
/// Default: nullopt, the output is as wide as the inputs are mixed to. Sources
/// present `outputChannelNumber_`; StereoPanner always outputs stereo.
[[nodiscard]] virtual std::optional<size_t> getOutputChannelNumber() const {
return std::nullopt;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ void AudioBufferSourceNode::start(double when, double offset, double duration) {
offset = std::min(offset, loopEnd_);
}

vReadIndex_ = static_cast<double>(buffer_->getSampleRate() * offset);
vReadIndex_ = buffer_->getSampleRate() * offset;
}

void AudioBufferSourceNode::disable() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,13 @@
#include <audioapi/core/utils/graph/HostGraph.h>
#include <audioapi/utils/AudioBuffer.hpp>
#include <algorithm>
#include <cassert>

#include <cassert>
#include <cstddef>
#include <iterator>
#include <memory>
#include <optional>
#include <ranges>
#include <span>
#include <utility>
#include <vector>
Expand Down Expand Up @@ -47,37 +48,51 @@ inline audioapi::AudioNode *audioNodeOf(const HostGraph::HostVertex *node) {
return node->handle->audioNode->asAudioNode();
}

/// @brief Computes the spec's `computedNumberOfChannels` for `dest`: the width
/// its inputs (`dest->inputs`) are mixed to, which is also the width of its
/// @brief Width `node` will emit toward AudioDestinationNode once every
/// queued swap has landed: a fixed override (StereoPanner's stereo output, a
/// source's declared width) or else the width of its pending negotiated
/// buffer. 0 for payloads without an AudioNode (e.g. BridgeNode).
size_t pendingOutputChannelNumber(const HostGraph::HostVertex *node) {
auto *audio = audioNodeOf(node);
if (audio == nullptr) {
return 0;
}
return audio->getOutputChannelNumber().value_or(node->pendingChannelCount);
}

/// @brief True when `node` is a source: it has no inputs to mix, so it owns
/// its output buffer and swaps it together with the content that defines its
/// width (f.e. AudioBufferSourceNode's buffer number of channels).
bool ownsOutputBuffer(const HostGraph::HostVertex *node) {
auto *audio = audioNodeOf(node);
return audio != nullptr && audio->getNumberOfInputs() == 0;
}

/// @brief Computes the spec's `computedNumberOfChannels` for `node`: the width
/// its inputs (`node->inputs`) are mixed to, which is also the width of its
/// negotiated buffer. Follows the Web Audio rules for `channelCountMode`:
/// - EXPLICIT -> `channelCount` attribute (inputs ignored)
/// - MAX -> max over inputs' output channel numbers
/// - CLAMPED_MAX -> min(channelCount attribute, max over inputs')
///
/// When there are no inputs the node keeps its own `channelCount` attribute
/// (matches the shape the buffer already had at construction time).
///
/// `term` identifies the current negotiation pass.
/// @pre Every input is resolved for `term`
/// (`resolveOutputChannelNumberForNode`); its pending output channel number is
/// read from `channelLayout`, never from the live node.
size_t computeNumberOfChannels(const HostGraph::HostVertex *dest, size_t term) {
auto *destAudio = audioNodeOf(dest);
if (destAudio == nullptr) {
size_t computeNumberOfChannels(const HostGraph::HostVertex *node) {
auto *audio = audioNodeOf(node);
if (audio == nullptr) {
return 0;
}

const auto channelCount = destAudio->getChannelCount();
const auto mode = destAudio->getChannelCountMode();
const auto channelCount = audio->getChannelCount();
const auto mode = audio->getChannelCountMode();

if (mode == audioapi::ChannelCountMode::EXPLICIT || dest->inputs.empty()) {
if (mode == audioapi::ChannelCountMode::EXPLICIT || node->inputs.empty()) {
return channelCount;
}

size_t maxInputChannels = 0;
for (const HostGraph::HostVertex *input : dest->inputs) {
assert(input->channelLayout.isResolvedFor(term));
maxInputChannels = std::max(input->channelLayout.outputChannelNumber, maxInputChannels);
for (const HostGraph::HostVertex *input : node->inputs) {
maxInputChannels = std::max(pendingOutputChannelNumber(input), maxInputChannels);
}

if (maxInputChannels == 0) {
Expand All @@ -90,31 +105,27 @@ size_t computeNumberOfChannels(const HostGraph::HostVertex *dest, size_t term) {
return maxInputChannels;
}

/// @brief If `dest` needs a newly negotiated channel layout, allocates a
/// replacement `DSPAudioBuffer` on the host thread and returns it.
/// Returns `nullptr` when no swap is needed (no audio payload, negotiation
/// converged, or context gone).
/// @brief If `node` needs a newly negotiated channel layout, allocates a
/// replacement `DSPAudioBuffer` on the host thread, records its width as
/// pending, and returns it. Returns `nullptr` when no swap is needed (no audio
/// payload, or the pending buffer already has `computedNumberOfChannels`).
std::shared_ptr<audioapi::DSPAudioBuffer> buildNegotiatedBufferIfNeeded(
const HostGraph::HostVertex *dest,
HostGraph::HostVertex *node,
size_t computedNumberOfChannels) {
auto *destAudio = audioNodeOf(dest);
if (destAudio == nullptr) {
return nullptr;
}

if (computedNumberOfChannels == 0) {
auto *audio = audioNodeOf(node);
if (audio == nullptr) {
return nullptr;
}

const auto current = destAudio->getNegotiatedBuffer();
if (current != nullptr && current->getNumberOfChannels() == computedNumberOfChannels) {
if (computedNumberOfChannels == 0 || node->pendingChannelCount == computedNumberOfChannels) {
return nullptr;
}
node->pendingChannelCount = computedNumberOfChannels;

return std::make_shared<audioapi::DSPAudioBuffer>(
audioapi::RENDER_QUANTUM_SIZE,
static_cast<int>(computedNumberOfChannels),
destAudio->getContextSampleRate());
audio->getContextSampleRate());
}

struct ChannelNegotiation {
Expand All @@ -124,59 +135,26 @@ struct ChannelNegotiation {

using NegotiationBatch = std::vector<ChannelNegotiation>;

/// @brief Recursively resolves output channel numbers for `node` and every
/// downstream ancestor (`node->inputs`) in negotiation pass `term`.
void resolveOutputChannelNumberForNode(HostGraph::HostVertex *node, size_t term) {
if (node == nullptr || node->channelLayout.isResolvedFor(term)) {
return;
}

for (HostGraph::HostVertex *input : node->inputs) {
resolveOutputChannelNumberForNode(input, term);
}

auto *audio = audioNodeOf(node);
if (audio == nullptr) {
// Stamp visited for non-AudioNode payloads (e.g. BridgeNode) so shared DAG
// paths cannot re-enter this node in the same negotiation pass.
node->channelLayout.setResolved(term, 0);
return;
}

const size_t computedNumberOfChannels = computeNumberOfChannels(node, term);
node->channelLayout.setResolved(
term, audio->getOutputChannelNumber().value_or(computedNumberOfChannels));
}

/// @brief Starting at `dest` (the connect `to` node), negotiates channel
/// layouts for `dest` and every node further upstream (toward
/// AudioDestinationNode via `dest->outputs`).
void collectChannelNegotiations(HostGraph::HostVertex *dest, size_t term, NegotiationBatch &out) {
if (dest == nullptr || dest->channelLayout.isResolvedFor(term)) {
/// @brief Appends `node` (one whose inputs or channel attributes changed, f.e.
/// the connect `to` node) and every node further upstream (toward
/// AudioDestinationNode via `node->outputs`) to `postOrder`, each one after
/// every node it feeds.
void collectAffectedPostOrder(
HostGraph::HostVertex *node,
size_t term,
std::vector<HostGraph::HostVertex *> &postOrder) {
if (node == nullptr || !node->channelLayoutTraversal.visit(term)) {
return;
}

for (HostGraph::HostVertex *input : dest->inputs) {
resolveOutputChannelNumberForNode(input, term);
}

if (auto *destAudio = audioNodeOf(dest)) {
const size_t computedNumberOfChannels = computeNumberOfChannels(dest, term);
dest->channelLayout.setResolved(
term, destAudio->getOutputChannelNumber().value_or(computedNumberOfChannels));

if (auto negotiatedBuffer = buildNegotiatedBufferIfNeeded(dest, computedNumberOfChannels)) {
out.push_back({.node = dest, .buffer = std::move(negotiatedBuffer)});
}
} else {
dest->channelLayout.setResolved(term, 0);
}

for (HostGraph::HostVertex *upstream : dest->outputs) {
collectChannelNegotiations(upstream, term, out);
for (HostGraph::HostVertex *upstream : node->outputs) {
collectAffectedPostOrder(upstream, term, postOrder);
}
postOrder.push_back(node);
}

// clang-tidy does not search deep enough to find the usage of this fun
// NOLINTNEXTLINE
void applyChannelNegotiations(
const NegotiationBatch &negotiations,
Disposer<audioapi::DISPOSER_PAYLOAD_SIZE> &disposer) {
Expand All @@ -193,24 +171,36 @@ void applyChannelNegotiations(
}
}

/// @brief Builds a fresh negotiation batch by cascading upstream from each
/// destination in `destinations`. All destinations share the same `term`, so a
/// node reachable from several of them is negotiated at most once per pass.
/// The batch is heap-allocated so ownership can be moved into the AGEvent and
/// later disposed off the audio thread.
/// @brief Builds a fresh negotiation batch by cascading upstream (toward
/// AudioDestinationNode) from each node in `changedNodes`: the nodes whose
/// inputs or channel attributes just changed. They all share the same `term`,
/// so a node reachable from several of them is negotiated once per pass.
std::unique_ptr<NegotiationBatch> collectNegotiations(
size_t term,
std::span<HostGraph::HostVertex *const> destinations) {
std::span<HostGraph::HostVertex *const> changedNodes) {
std::vector<HostGraph::HostVertex *> postOrder;
for (HostGraph::HostVertex *changedNode : changedNodes) {
collectAffectedPostOrder(changedNode, term, postOrder);
}

// Reversed post-order is topological: a node reads its inputs' pending
// widths only after every input this pass renegotiates has been updated,
// whichever order the cascade reached them in (f.e. a triangle).
auto negotiations = std::make_unique<NegotiationBatch>();
for (HostGraph::HostVertex *dest : destinations) {
collectChannelNegotiations(dest, term, *negotiations);
for (HostGraph::HostVertex *node : std::views::reverse(postOrder)) {
if (auto negotiatedBuffer =
buildNegotiatedBufferIfNeeded(node, computeNumberOfChannels(node))) {
negotiations->push_back({.node = node, .buffer = std::move(negotiatedBuffer)});
}
}
return negotiations;
}

/// @brief Single-destination convenience overload.
std::unique_ptr<NegotiationBatch> collectNegotiations(size_t term, HostGraph::HostVertex *dest) {
return collectNegotiations(term, std::span<HostGraph::HostVertex *const>(&dest, 1));
/// @brief Single-node convenience overload.
std::unique_ptr<NegotiationBatch> collectNegotiations(
size_t term,
HostGraph::HostVertex *changedNode) {
return collectNegotiations(term, std::span<HostGraph::HostVertex *const>(&changedNode, 1));
}

} // namespace
Expand Down Expand Up @@ -283,6 +273,13 @@ auto HostGraph::addNode(std::shared_ptr<NodeHandle> handle) -> std::pair<HostVer
std::scoped_lock lock(nodesMutex_);
HostVertex *newNode = new HostVertex();
newNode->handle = handle;
// AudioGraph learns about the node only from the event below, so the audio
// thread cannot be swapping this buffer yet.
if (auto *audio = audioNodeOf(newNode)) {
if (auto negotiatedBuffer = audio->getNegotiatedBuffer()) {
newNode->pendingChannelCount = negotiatedBuffer->getNumberOfChannels();
}
}
nodes.push_back(newNode);

auto event = [h = std::move(handle)](auto &graph, auto &) mutable {
Expand Down Expand Up @@ -325,6 +322,11 @@ auto HostGraph::addEdge(HostVertex *from, HostVertex *to) -> Result<AGEvent, Gra
return Err(GraphError::CYCLE_DETECTED);
}

// `connect()` already rejects a source as destination; negotiation below
// would otherwise build a buffer narrower than the content the source
// writes into it.
assert(!ownsOutputBuffer(to));

from->outputs.push_back(to);
to->inputs.push_back(from);
edgeCount_++;
Expand Down Expand Up @@ -451,8 +453,11 @@ auto HostGraph::renegotiateNodeChannels(HostVertex *node) -> Result<AGEvent, Gra
return Err(GraphError::NODE_NOT_FOUND);
}

// Recompute channel layouts for this node and everything downstream.
auto negotiations = collectNegotiations(++channelLayoutTerm_, node);
// Recompute channel layouts for this node and everything upstream. A source
// owns its output buffer, so only its consumers have anything to renegotiate.
auto negotiations = ownsOutputBuffer(node)
? collectNegotiations(++channelLayoutTerm_, node->outputs)
: collectNegotiations(++channelLayoutTerm_, node);

return Ok([negotiations = std::move(negotiations)](AudioGraph &graph, auto &disposer) mutable {
applyChannelNegotiations(*negotiations, disposer);
Expand Down
Loading
Loading