Skip to content

feat(chat): pace native-agent streaming with an adaptive per-session smoother - #237

Open
Ukaykhingmarma28 wants to merge 1 commit into
pacifio:0.3.1from
Ukaykhingmarma28:ukaykhing25/stream-smoother
Open

feat(chat): pace native-agent streaming with an adaptive per-session smoother#237
Ukaykhingmarma28 wants to merge 1 commit into
pacifio:0.3.1from
Ukaykhingmarma28:ukaykhing25/stream-smoother

Conversation

@Ukaykhingmarma28

Copy link
Copy Markdown
Collaborator

What

Native-agent (Atlas Agent) responses stream into the chat in visible lumps. A live wire probe against ai.tryatlas.cc confirmed the cause: Claude-via-Vertex emits text in ~400–650 ms bursts (occasional 1–1.5 s stalls), and every hop of our pipeline — gateway, engine SSE decoder, native-agent pump, Tauri emit, RAF batch — faithfully forwards per-delta, so the bursts reach the UI intact. (Full code-read + measurements in research/stream-smoothing.md, repo-local.)

This adds a presentation-only StreamSmoother between the atlas:agents listener and the RAF batcher in App.tsx:

  • Per-session char queues drained on a 33 ms tick with an elapsed-time budget, so WebKit-throttled timers (unfocused window) degrade to chunkier-but-current, never a growing backlog.
  • Adaptive ratchet-up rate targeting ~450 ms lag (derived from the measured burst gap), with a floor rate so an already-smooth stream passes through at chunk speed.
  • Wire order preserved: every non-text delta becomes a queue barrier behind the text ahead of it; turn_finished barriers too, so the tail keeps typing before the idle flip.
  • Stop feels like stop: cancelled turns, turn_failed, and agent_disconnected flush synchronously. Permission deltas bypass entirely (modal latency).
  • Scope-gated to native-agent sessions; ACP sessions and unknown sessions pass through untouched. The persisted transcript never sees the pacing.

Tests

12 unit tests (stream-smoother.test.ts, fake timers): burst spreading, elapsed-time drain under throttling, floor-rate passthrough, barrier ordering, tail-after-finish, instant cancel/fail flush, per-session independence, surrogate-pair safety.

bun run typecheck clean; the 3 failing suites on this branch (atlas-comms CI-coverage/opt-level guards + a comms midnight test) pre-date this change and are untouched by it.

🤖 Generated with Claude Code

…smoother

Claude-via-Vertex emits text in ~400-650 ms bursts (measured on the live
gateway; research/stream-smoothing.md) and every hop from engine to store
forwards per-delta, so the lumps reached the UI intact. Add a
presentation-only StreamSmoother between the atlas:agents listener and
the RAF batcher: per-session char queues drained on a 33 ms tick with an
elapsed-time budget (throttled timers degrade to chunky, never laggy), a
ratchet-up rate targeting ~450 ms lag, and a floor so already-smooth
streams pass at chunk speed.

Wire order is preserved by turning every non-text delta into a queue
barrier; turn_finished barriers too, so the tail keeps typing before the
idle flip. Cancel/failure/disconnect flush synchronously, and permission
deltas bypass entirely. ACP sessions and unknown sessions pass through
untouched. The persisted transcript never sees the pacing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant