[guhyun9454]

Summary

SortformerDiarizationOnline.diarize() is async def, but it ran the mel feature extraction and forward_streaming_step synchronously, so every chunk blocked the asyncio event loop for the whole forward pass. This is the same failure mode #220 described and fixed for the ASR path (asyncio.to_thread(process_iter)); the Sortformer path was left on the loop.

This PR keeps the buffer slicing on the loop (so insert_audio_chunk and diarize still never touch buffer_audio concurrently) and moves the per-chunk inference into _diarize_chunk, run via asyncio.to_thread.

User impact

• While Sortformer runs, the loop stays free for WebSocket I/O, keepalive pings, the ASR task and other sessions.
• Observed in practice (live mic, --backend mlx-whisper --diarization, Sortformer on an Apple M1 Pro under memory pressure): the web UI repeatedly showed Connected. Using MediaRecorder (WebM).. The client auto-reconnected, each reconnect started a new session, and speaker labels were reset. Per-chunk inference that takes ~0.6 s on an idle machine grows to several seconds when the host is swapping, which is enough to miss the server's keepalive ping.
• No API or config change. Segment output is unchanged (same segments before and after, see below).
• Concurrency note: diar_model is shared across sessions, so with several sessions the forward passes can now overlap in worker threads, as the ASR backends already do with to_thread. Streaming state is per session; the model runs under torch.inference_mode().

Validation

Regression test: tests/test_sortformer_event_loop.py. It uses a fake model whose forward pass blocks for 0.5 s and asserts that a heartbeat coroutine keeps ticking during diarize(). It also checks the returned segments, _chunk_index and buffer consumption.

• Before the fix: fails, assert 0.521 < 0.25 (the loop was blocked for the whole forward pass).
• After the fix: passes.

Real model: nvidia/diar_streaming_sortformer_4spk-v2 on CPU, Apple M1 Pro, Python 3.13.12, torch 2.14.1, NeMo 3.0.0. I fed the two tests/fixtures/sortformer_2spk clips at real-time pace and measured event-loop stalls with a 10 ms heartbeat.

== before (upstream/main 363e4f6)
audio=5.6s chunks=5 segments=6 wall=5.00s
event-loop stall: max=662 ms  p99=644 ms  p50=1.1 ms  ticks=172
== after (this PR)
audio=5.6s chunks=5 segments=6 wall=5.00s
event-loop stall: max=6 ms  p99=3 ms  p50=1.1 ms  ticks=461

Suite:

• uv run ruff check .: all checks passed
• uv lock --check: OK
• uv run pytest -q tests/ --ignore=tests/test_pipeline.py --ignore=tests/test_asr_coalescing_pipeline.py: 314 passed, 16 skipped

Checklist

• I searched for an existing issue or discussion and linked it when relevant. (Fix Error CLOSE 1011 (internal error) keepalive ping timeout [24 bytes] #220: same root cause on the ASR path)
• I added or updated tests for behavior changes.
• I updated documentation for user-facing changes. (No user-facing interface change)
• I ran ruff check ..
• I ran the relevant pytest suite.
• I did not commit credentials, model weights, generated caches, or private data.