feat(chat): pace native-agent streaming with an adaptive per-session smoother - #237
Open
Ukaykhingmarma28 wants to merge 1 commit into
Open
feat(chat): pace native-agent streaming with an adaptive per-session smoother#237Ukaykhingmarma28 wants to merge 1 commit into
Ukaykhingmarma28 wants to merge 1 commit into
Conversation
…smoother Claude-via-Vertex emits text in ~400-650 ms bursts (measured on the live gateway; research/stream-smoothing.md) and every hop from engine to store forwards per-delta, so the lumps reached the UI intact. Add a presentation-only StreamSmoother between the atlas:agents listener and the RAF batcher: per-session char queues drained on a 33 ms tick with an elapsed-time budget (throttled timers degrade to chunky, never laggy), a ratchet-up rate targeting ~450 ms lag, and a floor so already-smooth streams pass at chunk speed. Wire order is preserved by turning every non-text delta into a queue barrier; turn_finished barriers too, so the tail keeps typing before the idle flip. Cancel/failure/disconnect flush synchronously, and permission deltas bypass entirely. ACP sessions and unknown sessions pass through untouched. The persisted transcript never sees the pacing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Native-agent (Atlas Agent) responses stream into the chat in visible lumps. A live wire probe against
ai.tryatlas.ccconfirmed the cause: Claude-via-Vertex emits text in ~400–650 ms bursts (occasional 1–1.5 s stalls), and every hop of our pipeline — gateway, engine SSE decoder, native-agent pump, Tauri emit, RAF batch — faithfully forwards per-delta, so the bursts reach the UI intact. (Full code-read + measurements inresearch/stream-smoothing.md, repo-local.)This adds a presentation-only
StreamSmootherbetween theatlas:agentslistener and the RAF batcher inApp.tsx:turn_finishedbarriers too, so the tail keeps typing before the idle flip.turn_failed, andagent_disconnectedflush synchronously. Permission deltas bypass entirely (modal latency).Tests
12 unit tests (
stream-smoother.test.ts, fake timers): burst spreading, elapsed-time drain under throttling, floor-rate passthrough, barrier ordering, tail-after-finish, instant cancel/fail flush, per-session independence, surrogate-pair safety.bun run typecheckclean; the 3 failing suites on this branch (atlas-commsCI-coverage/opt-level guards + a comms midnight test) pre-date this change and are untouched by it.🤖 Generated with Claude Code