Repository navigation
bgtail: a widening can repeat a line the caller already saw when coverage has a gap #28
Description
What
bgtail's widening is documented as returning "the lines above your current coverage rather than the whole window again". The stronger claim it shipped with — "never the ones already in your context" — is false whenever coverage has a gap: a widening can hand back a line the caller has already seen.
Why
The tail bookmark tracks coverage as a single scalar, served — the length of the contiguous trailing run the caller has been shown. Two paths shrink it and forget a larger earlier coverage:
- a delta read that shows fewer lines than were appended (
fresh.length > lines) setsserved = lines, which leaves the fresh lines above the requested window as an unseen gap between the old run and the new one; - a reset read (
raw: true, a changedbyteswindow, a shrunken or replaced log) with a narrowerlinessetsserved = min(lines, total).
A later widening slices from that shrunken served, so it returns lines from the forgotten run.
Reproduction (deterministic, 13-line log)
- read
lines: 5→ shows lines 6–10,served = 5 - append lines 11–13
- read
lines: 2→freshis 3 > 2, so it shows 12–13 and setsserved = 2; line 11 is never shown - read
lines: 4→ returns line-10 (already shown) plus line-11
Impact
Duplicate context, not lost output: served never overstates coverage, so no line can be hidden. The repeat is bounded by the ~8 KB condenser cap. What was actually wrong was the claim, which the tool description, README and skill all repeated — now softened to "coverage is a trailing run, so a line seen before an intervening gap can repeat".
Options
- Accept and document (already done): the wording describes trailing coverage honestly. Cheapest, and the repeat is at most one capped window of already-seen text.
- Track coverage properly — a low-water mark or an interval set instead of one scalar — and slice the widening from the true unseen region. Correct, but the bookmark is in-memory per-job state, and this adds structure for a small win.
- Raise
servedto a high-water mark on the delta path (served = max(served, shown)): removes the delta-shrink case, not the reset-read case, and sends more on the next widening — a partial fix that trades one repeat for another.
Found by an adversarial review of #23. The test added there pins the no-loss half only (line-11 must come back), deliberately not the repeat, so any of these can land without breaking it.
Activity
docs(bgtail): describe widening as trailing coverage, and pin the shr…
lloydsk commentedon Sep 26, 2026
Second symptom, same bookmark: the condenser cap can hide the newest lines
The invariant stated above — "
servednever overstates coverage, so no line can be hidden" — does not hold when the condenser'sTOTAL_CAPtruncates. Hit against a real job today.Reproduction (job log 979 lines at read time,
h8b-trace-log-1790395867-41381):bgtail <id> lines: 400— first read. Returns ~183 lines, starting at the head of the requested window and ending mid-log; the note saysoutput capped at 8000 chars — 400 raw lines total. The log's last ~217 lines — run summary,1 tests failed, the failing test's name — are not returned.bgtail <id> lines: 60→(no new lines since last read — log at 979 lines). The lines the cap dropped in step 1 are now unreachable: they sit inside the range the bookmark recorded as served.bgtail <id> lines: 50 raw: true→ the true tail (summary, failing test). So the cap, not the read window, is what dropped them.Mechanism (
extension/index.ts):condenseLogLineswalkscollapsedoldest-first andbreaks ontotal > TOTAL_CAP(lines 3494-3497), returningout.join("\n")— the oldest lines of the selection. A caller asking for the newest lines gets the oldest that fit.served = window.length(full-tail, line 3837),served: widenedTo(widening, line 3872),served = fresh.length > lines ? lines : Math.min(prev.served + fresh.length, total)(delta, lines 3934-3937). Nothing reconcilesservedwith the capped render.So a capped read returns the wrong end and marks the unreturned lines as seen: lost output, where the original symptom is duplicate context. Both come from the same place — coverage is a scalar not derived from what the caller was actually shown.
Fix sketch, both symptoms in one change:
condenseLogLinesspends its budget newest-first and reports how many raw lines it actually rendered (whole emissions only, so folded runs stay exact).servedmust be the contiguous trailing run of the rendered text. If the interval form is taken, overflow must forget oldest coverage rather than merge across a gap, so the failure direction stays "re-sent" rather than "lost".Not fixed here — reported from a measurement session (found while profiling agent sessions for a benchmark, not by code review); the log above is a real job's, not a synthetic one.
docs(bench): the trace-root cell, and what the evidence path actually…