Skip to content

voice: arecord ALSA-level overruns (up to 56.8s) during live capture — likely a major no-transcript contributor #283

Description

@adrianwedd

What

bin/px-wake-listen's arecord subprocess is logging genuine ALSA-level
buffer overruns (arecord: overrun!!! (at least N ms long)) — not the
benign, already-understood "backlog dropped while idle" case CLAUDE.md
documents as expected during STT/LLM/speech (that one is counted separately
as dropped_chunks and explicitly harmless). This is arecord's own kernel
buffer overflowing, meaning audio was physically lost before
ArecordStream's reader thread ever had a chance to drain it.

Scale

Over logs/px-wake-listen.log's retained history (2026-08-15 through today):

  • 520 overrun events, mean 5.67s, median 3.5s, max 56.8s
  • 191 of 520 (37%) exceed 5s; 89 (17%) exceed 10s
  • Today alone: 58 events, with individual overruns up to 34.8s
  • Not clustered in one bad day — present every day in the window (18–142/day)

A live example from a real conversation (2026-08-23T12:35–12:39, trace_id
f89d14b4ae5c4cab9304ed077e815326): 8 overruns in one ~4-minute exchange
(2.4s–5.5s each), overlapping directly with sensevoice hallucinating 1–2
word transcripts ('sky', 'she did') and a phantom-transcript retry, and
the conversation ultimately ending on "silence — ending" rather than a real
reply.

Why this matters for wake/STT reliability

This is a strong candidate for a meaningful share of the reported 19/50
"no usable transcript" bucket from the recent production sample — an
overrun during the capture window doesn't just degrade audio quality, it
deletes seconds of it outright, which is exactly the shape of failure that
produces 1-2-word hallucinations and empty/phantom transcripts rather than a
garbled-but-parseable utterance.

Relationship to other open issues

CLAUDE.md documents the ArecordStream reader-thread/deque design
(src/pxh/mic_stream.py) as specifically built to prevent arecord from
"block[ing] and overrun[ning] its own ALSA buffer" — and states the
regression test (bin/px-mic-check) scores 18/18 chirps clean. That test
requires the mic free and px-wake-listen stopped, so it cannot exercise
the condition these overruns actually occur under: real production load,
with other processes (STT/LLM inference, other daemons, concurrent dev/agent
activity) competing for the same 4 CPU cores.

That's the same shape of finding as #270 (resident voice_turn sometimes
burning its full 45s budget, uncorrelated with ask_brain's own lock
queueing, but correlated with concurrent host activity) and the same day
also shows a px-alive restart genuinely timing out
(sudo -n systemctl start px-alive timed out after 15s, 2026-08-23T12:39:38,
same conversation as the overrun example above; 10 such timeouts total
2026-08-21 through 2026-08-23). Three independent subsystems — the resident
brain session, the mic capture pipe, and systemctl itself — all showing
budget/deadline blowouts on days with heavy concurrent activity is starting
to look like one shared root cause (CPU/IO contention on this Pi) wearing
three different faces, rather than three unrelated bugs.

Suggested next step

Not proposing a fix yet — root cause (genuine CPU starvation vs. a specific
reader-thread bug under load vs. something else) isn't established. #282
adds loadavg/PSI snapshots to brain.py's telemetry for the #270 side of this;
the same instrumentation applied to mic_stream.py's overrun handler (log
loadavg/PSI at the moment an overrun is caught) would let this be checked
against the same signal, and either confirm or rule out the shared-root-cause
hypothesis directly.

Where

  • src/pxh/mic_stream.pyArecordStream, overrun detection/logging
  • bin/px-wake-listen_restart_alive, the 15s sudo -n systemctl start
    timeout

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingreliabilityReliability and error handlingvoiceWake word, STT, espeak, voice loop

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions