Skip to content

Polish spoken translation UX: quick start/stop and natural speech pauses #69

Description

@janitor-manager

Build 55 feedback: local spoken translation works well in use, but needs a smoother way to start/stop and more natural phrase boundaries. Brief pauses currently make it feel too eager to speak before the user has finished.

Follow-up to #57. Preserve the working first version while polishing these interactions.

Scope

  • Add a convenient keyboard shortcut and/or deliberate gesture to toggle Voice independently of agent listening and privacy mute. Choose a simple, conflict-aware binding; debounce/hold gestures so normal movement does not toggle translation accidentally. Keep the toolbar control available.
  • Improve utterance segmentation so short thinking pauses and breaths do not prematurely split speech or trigger translated playback. Inspect the current fixed ASR windows and downstream scheduling before choosing endpoint detection or buffering; the reported symptom does not establish the cause.
  • Balance waiting for a completed thought against translation delay. Keep a bounded maximum utterance/wait and queue; avoid an ever-growing backlog during continuous speech.
  • Give concise feedback for on/off and waiting/speaking so the user can tell whether the control worked. Keep the popup simple.

Acceptance

  • Synthetic speech covers brief pauses within a sentence, a clear end of speech, slow/fast speakers, breaths, and continuous speech. Compare premature segment/playback starts and end-of-speech-to-translated-audio delay with build 55.
  • Shortcut/gesture start and stop work while another app is focused. Stop cancels pending and playing translation; a later Start accepts fresh speech only.
  • Preserve independent captions and agent listening, agent-answer priority, global privacy mute, and the existing original-plus-translated audio route. A Voice action must never unmute AI Camera.
  • Confirm the experience through AI Camera Microphone in an independent calling/recording client, followed by a brief listening review. Synthetic tests remain the primary development check.

No shortcut, gesture, or endpointing model is selected yet. This issue tracks UX polish; the separate spoken-translation agent tool and remote provider remain in #57.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions