[PhantomDave]

Logs often took seconds to show up. Rendering a backlog was never the slow part (500 lines render in ~100ms); the time went to output bursts and to buffered child output. Four fixes, one commit each:

1. perf(core): the server stayed blocked for the whole burst. Each line was its own autocommit INSERT (prepared again every time), and pumpStream read already-buffered pipe chunks as microtasks, so a burst ran in a single event-loop turn. Lines are now buffered and written once per turn in one transaction with a cached statement. The stored rows (ids and session_id included) are still what the broadcaster publishes, and close() flushes pending lines before the DB closes. pumpStream yields with setImmediate after each chunk. The logs_fts trigger stays: with batching alone the burst is well under target, so grep keeps working as it does now.
2. perf(ui): the viewer froze during a live burst. SSE entries are applied once per animation frame. mergeLogs appends directly when ids follow what's held and only dedups and sorts when they overlap (history fetch vs SSE replay). The viewer keeps the newest 5000 lines and says older output is in History. Rows are memoized. Auto-scroll now follows the array instead of its length, because the length stops changing once the cap is reached.
3. perf(core): history queries scanned the timestamp index. Adds idx_logs_process (process_id, id) and orders history by id (insertion order), so the query reads only that process's rows. idx_logs_command (command_id) already ends in the rowid, so it needed no change. Building the index on an existing 1M-row DB takes ~0.3s, once, at startup.
4. fix(core): Python output arrived in 8KB blocks. PYTHONUNBUFFERED=1 is now the lowest env layer, under process.env, so every other layer can override it (an empty value turns it off). Documented in docs/CONFIG.md.

Before / after

Compiled sidecar + built UI, headless Chromium, Linux.

Benchmark	Before	After
100k-line burst: worst /api/health latency	8946 ms	67–76 ms (3 runs)
100k-line burst: time spent in >200ms stalls	8946 ms	0 ms
Viewer open during a 20k-line burst: main-thread long tasks	8638 ms	324–593 ms (3 runs)
Viewer open during a 20k-line burst: last line visible after opening (burst starts ~4s in)	13.4 s	4.6 s
History for one process, 1M-row table (oldest pid / fresh pid with no rows)	188 ms / 188 ms	0.3 ms / 0.0 ms
Python printing a line every 200ms: first line in the viewer	not within 30 s (only at exit)	0.5 s, then one line every 200ms (last of 20 at 4.3 s)

The 20k-burst "before" row uses the old UI against a core that already had fix 1, so it isolates the UI change. Against the original core it was 9.0 s of long tasks and 13.5 s to settle.

Not done here

• listLogRuns is still ~900ms unfiltered on a 1M-row table (History page open). Grouping by process_id alone over the new index takes it to ~125ms, but a pid reused by a different command (common on macOS, where pids wrap at 99999) would then merge two runs. A covering index takes it to ~180ms but adds a five-column index to every insert. A rollup table, as the existing ponytail: note suggests, is the clean fix, but it belongs in a separate PR.
• PTY for every runtime. Running commands under a PTY would make every runtime line-buffered, not just Python, but it merges stdout and stderr into one stream (and loses the stderr gutter). Possible follow-up.
• The duplicate history fetch (/api/logs + SSE replay) stays. With the index both are sub-millisecond, and the fetch still gives an instant first paint.

Checks

bun test (only the two known localhost→::1 healthcheck failures), bun run lint, bun run typecheck, bun run format:check, and the run-desktop sidecar smoke before every commit.

🤖 Generated with Claude Code