Skip to content

libobs: Fix use-after-free of canvas view in audio thread#736

Open
summeroff wants to merge 1 commit into
streamlabsfrom
libobs-fix-audio-view-uaf
Open

libobs: Fix use-after-free of canvas view in audio thread#736
summeroff wants to merge 1 commit into
streamlabsfrom
libobs-fix-audio-view-uaf

Conversation

@summeroff
Copy link
Copy Markdown

Problem

Sentry crash in the audio thread: EXCEPTION_ACCESS_VIOLATION_READ / 0x2c200000007 at obs-audio.c:519, pthread_mutex_lock(&view->channels_mutex). The faulting value is a freed-and-reused heap pointer — an in-array mix had a non-NULL view pointing at a freed obs_view.

Root cause

audio_callback() walks obs->video.mixes under mixes_mutex and locks each mix->view->channels_mutex (obs-audio.c:513-538). mixes_mutex guards the mixes array, not the lifetime of the obs_view objects the mixes point at. Freeing a view takes no lock, so the design is safe only if every freeing path nulls/erases the referencing mix pointer (under mixes_mutex) before the view is freed.

  • Standalone views honor this via obs_view_remove() (obs-view.c:222), which nulls every matching mix->view under mixes_mutex; the audio thread then skips them via if (!view) continue; (obs-audio.c:516).
  • Canvas-embedded views (obs_view lives inside obs_canvas) had no equivalent. obs_canvas_destroy() freed canvas->view relying only on obs_canvas_clear_mix() having erased canvas->mix — which removes only that one mix, not any other in-array mix referencing the view.

Triggered in the wild by a GPU device-removed storm (DXGI 887A0006) driving repeated partial video resets (SetVideoContextobs_deactivate_video_infostop_video() + obs_free_video(false)). stop_video() joins only the graphics thread; the audio thread keeps running across the reset.

Fix

Call obs_view_remove(&canvas->view) in obs_canvas_destroy() immediately before obs_view_free(), so the canvas path honors the same contract as the standalone path. Placed in destroy only — not in obs_canvas_clear_mix(), which is also used by the reset path where the canvas survives and a new mix is created pointing at the same live view.

/* Null mixes referencing this view before freeing it (obs-audio.c:519). */
obs_view_remove(&canvas->view);
obs_view_free(&canvas->view);

Lock ordering: obs_view_remove() takes only mixes_mutex, respecting the existing encoders_mutex -> mixes_mutex order. Shutdown ordering is safe: obs_free_data() (canvas destroy, obs.c:1612) runs before obs_free_video(true) destroys mixes_mutex (obs.c:1654).

Verification

  • Build: cmake --build build_x64 --target libobs --config RelWithDebInfo → exit 0, obs.dll links clean.
  • clang-format: clean under locally-installed v19. ⚠️ CI pins v16 — re-run before merge to be certain (trivial change, very unlikely to differ).
  • Runtime: the GPU device-removed race can't be reproduced on demand; verified by build + static reasoning (lock/shutdown ordering, contract parity with the standalone-view path).

Follow-ups (out of scope, not fixed here)

  • obs_free_video(false) assumes index 0 is always the main mix (obs.c:920-921), but obs_canvas_clear_mix's da_erase can shift a secondary mix into index 0.
  • During a partial reset the graphics thread (reaper of null-view mixes) is stopped, so null-view mixes aren't reaped until video re-init.

🤖 Generated with Claude Code

audio_callback() walks obs->video.mixes under mixes_mutex and locks each
mix->view->channels_mutex. mixes_mutex guards the mixes array, not the
lifetime of the obs_view objects the mixes point to, so the audio thread
can dereference a view that was already freed and crash in
pthread_mutex_lock (obs-audio.c:519).

Standalone views are protected by obs_view_remove(), which nulls every
matching mix->view under mixes_mutex before the view is freed; the audio
thread then skips them via its "if (!view) continue" guard. Canvas-owned
views had no equivalent: obs_canvas_destroy() freed canvas->view while a
mix could still reference it, relying only on obs_canvas_clear_mix()
having erased canvas->mix.

Call obs_view_remove() in obs_canvas_destroy() before obs_view_free() so
the canvas path honors the same contract. Triggered in the wild by a GPU
device-removed storm driving repeated partial video resets while the
audio thread kept running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a use-after-free risk where the audio thread can lock view->channels_mutex through an obs_core_video_mix that still references a canvas-embedded obs_view after the canvas is destroyed.

Changes:

  • Ensure canvas destruction nulls all mix->view pointers referencing the canvas view by calling obs_view_remove(&canvas->view) before obs_view_free(&canvas->view).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread libobs/obs-canvas.c
pthread_mutex_destroy(&canvas->sources_mutex);
obs_context_data_free(&canvas->context);

/* Null mixes referencing this view before freeing it (obs-audio.c:519). */
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants