Skip to content

Refine back-channel classification when strict text lookup is empty #65

Description

@julien731

Background

While testing PR #59 / issue #51 against the Nose Tea Pre-kickoff Meeting (~80 min, 6 speakers), interaction analysis produced 190 interruption events. Spot-checking revealed that 28 of the 190 (~15%) are short (≤ 1.5s) and have ack-y context strings ("Yeah.", "Okay.", "Yes.", "Yeah, yeah, yeah."), suggesting they should have been classified as back-channel overlap rather than interruption.

Examples (real meeting 955eeb48-…):

t (s) overlap dur a → b context
136.16 0.71 Pat → Carlos "Yeah."
192.46 0.47 Pat → Julien "Okay."
332.89 1.15 Julien → Got "Yes."
1481.01 1.01 Carlos → Julien "Yeah."
1708.22 1.13 Pat → Carlos "Yeah, yeah, yeah."

Root cause

_detect_overlaps in backend/services/interaction_analyzer.py classifies an overlap as a back-channel only when _is_backchannel(duration_b, text_b_strict) returns True. The strict text lookup (introduced in af44ac7 to avoid mis-attributing nearby segments) returns \"\" when there is no transcript segment from speaker B that strictly contains the midpoint of B's PyAnnote turn.

When the strict lookup is empty, _is_backchannel only catches the case where B's turn is ≤ 0.5s AND has no text. For 0.5s < duration_b ≤ 1.5s, the lexical check on the empty string returns False and the event falls through to INTERRUPTION.

The displayed context field of those events comes from a lenient lookup (nearest-edge segment), which can pull a Carlos "Yeah." from minutes away — making the JSON output misleading: the displayed text is not actually B's utterance at that timestamp.

Why this happens in practice

PyAnnote sometimes emits a sub-1.5s diarization turn for a speaker for whom WhisperX did not produce a corresponding speaker-tagged transcript segment (typical for very short vocalizations during another speaker's run). The current rule is conservative: "no strict text → assume substantive interruption." In this dataset that produces ~15% false-positive interruptions for what subjectively look like back-channels.

Proposal (to discuss)

Extend the back-channel heuristic so that, when the strict text lookup is empty:

  • Treat turns with duration_b ≤ BACKCHANNEL_MAX_DURATION (1.5s) as back-channel iff the lenient lookup at the midpoint resolves a segment that is (a) within ±X seconds of start_b/end_b AND (b) lexically a back-channel. Otherwise default to interruption.

Tune X empirically against the existing fixture corpus + this real meeting. Add a test fixture covering the "PyAnnote turn with no strict speaker-B segment" path.

Independently, consider revising the event's context field so we don't emit lenient text that misleads the reader — either label it (context_source: 'lenient' | 'strict') or omit it when no strict text was found.

Out of scope

  • Any UI rendering changes (Phase 4).
  • The dominance / pause / per-segment annotation paths — those tested clean against AC.

Verification

Re-run audio analysis on 955eeb48-b55e-4e8f-9d1a-89fcb99ca609 (or the fixture corpus) and confirm:

  • Total interruption count drops by ≈ the number of obvious back-channel false-positives
  • Newly-reclassified events appear under overlap
  • AC1/AC3 tests still pass

Priority

Optional / nice-to-have. The current behaviour is conservative-by-design; this is a refinement, not a defect.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or enhancement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions