fix(bgtail): a wider lines window returns the earlier lines, not "no new lines"

README.md

@@ -80,7 +80,7 @@ The host also decides several details, all handled internally:
 | ------ | --------- |
 | `bgrun` | Launch a command detached in the background. Optional `name` gives the job a short human-readable label. Returns `started: <job-id>` immediately. Wakes the session automatically on completion. |
 | `bgstatus` | Show job status. With an id: any job's state + exit code — a `bg_N` id (the host's own background job on omp) is answered with its state, where its output went and how to cancel it, rather than "not found". Without: this session's running jobs (finished jobs hidden by default — pass `includeDone: true` or set `showCompletedJobs`), then the host's background jobs under their own heading. Other sessions' *running* jobs are listed only when `adoptForeignJobs` is enabled; finished foreign logs from the shared dir can also appear when finished jobs are included. |
-| `bgtail` | Read the newest lines of a job's log (default 40; it reads the log's **last 2 MB** — widen with `bytes`, max 64 MiB + 4 KiB of wrapper overhead), **condensed for context**: ANSI escapes stripped, repeated lines collapsed (a run of 3+ folds into one line carrying its `[xN]` count; a pair is kept as two lines — a fold needs its count to stay legible), long lines and total size capped. First read = full last-N tail; repeat reads return **only lines appended since your last read** (delta tailing) — polling a running job never re-pays for lines already seen. Pass `raw: true` for the unprocessed last-N window (still advances the bookmark). On omp, a native `bg_N` job's spilled output is read the same way, stamped as the host's file. |
+| `bgtail` | Read the newest lines of a job's log (default 40; it reads the log's **last 2 MB** — widen with `bytes`, max 64 MiB + 4 KiB of wrapper overhead), **condensed for context**: ANSI escapes stripped, repeated lines collapsed (a run of 3+ folds into one line carrying its `[xN]` count; a pair is kept as two lines — a fold needs its count to stay legible), long lines and total size capped. First read = full last-N tail; repeat reads return **only lines appended since your last read** (delta tailing) — polling a running job never re-pays for lines already seen. A repeat read asking for **more** lines than you have already been shown is a widening, not a poll: it returns the lines above your current coverage rather than the whole window again — coverage is a trailing run, so a line seen before an intervening gap can repeat. Pass `raw: true` for the unprocessed last-N window (still advances the bookmark). On omp, a native `bg_N` job's spilled output is read the same way, stamped as the host's file. |
 | `bggrep` | Regex search over the **last 2 MB** of a job's log (`bytes` widens the window, max 64 MiB + 4 KiB of wrapper overhead; on omp, a native `bg_N` job's spilled output is searchable the same way): line-numbered matches, optional `context` lines, each line pre-truncated to 10 000 chars before matching, results capped (~50 matches, ~8KB) and condensed. Resolves the job id to the configured jobs dir itself — no log path to reconstruct. `ctx_execute_file` can read the same file (it takes an absolute path; only your Read-deny rules apply), but it needs that path. Matching runs under a wall-clock budget ([Bounded matching](#bounded-matching)). With no `pattern`, a generic failure-signature default is used (override it — convenience, not guarantee). |
 | `bgclean` | Remove old job logs. **Default scope: this session's jobs only** — other sessions' logs are untouched — and it also drops stale per-project digest markers (`.bgrun-used-*`, `.digest-nudge-*`) in the session's jobs dir (markers are not session data). Pass `all: true` to sweep every shared jobs dir — under the project-local default that is the project's dir plus the machine-global one, while an explicit absolute `jobsDir` is swept alone — and do the same marker sweep across them. Retention: `cleanupDays` config (7 days); `days` must be a positive number (`days: 0` is rejected rather than purging everything). Never removes a running job's log. |
 | `bgkill` | Stop a running job. Optional `force: true` sends `SIGKILL` instead of `SIGTERM`. Signals the job's **whole process group** (the child is spawned detached, so the command and anything it started go together; where a platform cannot signal groups, it falls back to the single pid and says so). Refuses an id that already finished, one that is not a bgrun job (`bg_N` belongs to the host — `hub cancel`), another session's job unless `includeForeign: true`, and a pid that cannot be the job's — gone, unusable, or recycled onto an unrelated process. It reports the signal it sent, not a guess at the outcome: the authoritative result is the wake, or the stale-check for a foreign job. |
@@ -204,8 +204,12 @@ ever enter the conversation:
 - **Quick peek:** `bgtail <id>` — condensed newest lines (ANSI stripped, repeats
   collapsed only where the count can be shown, ~2KB/line and ~8KB caps). The first read is the last-40-lines tail; each later
   read returns only what was appended since, so repeated polling is nearly
-  free. The wake message itself already carries the exit code and the log's
-  last line, so many turns need no follow-up read at all.
+  free. Asking for **more lines than you have been shown** widens the view
+  instead of polling: you get the lines above your coverage, so widening costs
+  the difference rather than the whole window again (a line seen before an
+  intervening gap can repeat). The wake message itself
+  already carries the exit code and the log's last line, so many turns need no
+  follow-up read at all.
 - **Pattern search:** `bggrep <id> [pattern] [context]` — line-numbered matches,
   capped and condensed (~50 matches, ~2KB/line, ~8KB); takes the job id, so
   there is no log path to reconstruct. Searches the **last 2 MB** by default —

extension/index.test.ts

@@ -3379,6 +3379,167 @@ test("bgtail: delta tailing — first read full tail, then only new lines, then
   });
 });
 
+test("bgtail: a wider `lines` hands back the earlier lines not yet shown, never the seen ones", async () => {
+  await withJobsDir(async (dir, h) => {
+    const { wakes, tools, ctx } = h;
+    const bgrun = tools.get("bgrun")!;
+    const bgtail = tools.get("bgtail")!;
+    const res = await bgrun.execute(
+      "w1",
+      { command: "seq 1 12 | sed 's/^/line-/'", name: "widening" },
+      undefined,
+      undefined,
+      ctx,
+    );
+    const id = ((res.content[0].text as string).match(/^started: ([^\n]+)/) ||
+      [])[1];
+    assert.ok(id, "got a job id");
+    await waitForWakes(wakes, 1);
+
+    // First read: the last 3 lines.
+    const t1 = await bgtail.execute(
+      "w2",
+      { id, lines: 3 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    const text1 = t1.content[0].text as string;
+    assert.match(text1, /line-12/);
+    assert.doesNotMatch(text1, /line-9/);
+
+    // Widening to 6: the three earlier lines it has not seen — and NOT a re-send
+    // of the three it already has, which is what delta tailing exists to avoid.
+    const t2 = await bgtail.execute(
+      "w3",
+      { id, lines: 6 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    const text2 = t2.content[0].text as string;
+    assert.match(
+      text2,
+      /no new lines since last read — showing the 3 earlier lines/,
+    );
+    assert.match(text2, /line-9/);
+    assert.match(text2, /line-7/);
+    assert.doesNotMatch(text2, /line-12/, "already-seen lines are not re-sent");
+    assert.doesNotMatch(text2, /line-6/, "the window stops at the width asked for");
+    assert.equal(t2.details.widened, true);
+    assert.equal(t2.details.linesShown, 3);
+
+    // Narrowing is still the cheap poll, not a widening.
+    const t3 = await bgtail.execute(
+      "w4",
+      { id, lines: 2 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    assert.match(t3.content[0].text as string, /^\(no new lines since last read/);
+    assert.equal(t3.details.linesShown, 0);
+
+    // Wide enough to cover the log: the remaining earlier lines, once.
+    const t4 = await bgtail.execute(
+      "w5",
+      { id, lines: 99 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    const text4 = t4.content[0].text as string;
+    assert.match(text4, /line-1\b/);
+    assert.match(text4, /line-6/);
+    assert.doesNotMatch(text4, /line-9/, "line-9 came back last read");
+
+    // Everything is now covered, so a still-wider request has nothing older to
+    // give and must fall back to the cheap poll instead of inventing output.
+    const t5 = await bgtail.execute(
+      "w6",
+      { id, lines: 99 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    assert.match(t5.content[0].text as string, /^\(no new lines since last read/);
+
+    // Appended lines win over a widening request: the delta is what a caller
+    // most needs, and `served` advances with it.
+    appendFileSync(join(dir, `${id}.log`), "line-13\n");
+    const t6 = await bgtail.execute(
+      "w7",
+      { id, lines: 40 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    assert.match(t6.content[0].text as string, /\+1 new line since last read/);
+  });
+});
+
+test("bgtail: a widening after a shrinking read still returns the missed lines", async () => {
+  await withJobsDir(async (dir, h) => {
+    const { wakes, tools, ctx } = h;
+    const bgrun = tools.get("bgrun")!;
+    const bgtail = tools.get("bgtail")!;
+    const res = await bgrun.execute(
+      "s1",
+      { command: "seq 1 10 | sed 's/^/line-/'", name: "shrinking" },
+      undefined,
+      undefined,
+      ctx,
+    );
+    const id = ((res.content[0].text as string).match(/^started: ([^\n]+)/) ||
+      [])[1];
+    assert.ok(id, "got a job id");
+    await waitForWakes(wakes, 1);
+    const logPath = join(dir, `${id}.log`);
+
+    // Coverage is the last 5 lines (6-10 of 10).
+    const first = await bgtail.execute(
+      "s2",
+      { id, lines: 5 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    assert.match(first.content[0].text as string, /line-10/);
+
+    // Three lines arrive, then a NARROWER read shows only the last two: line-11 has
+    // never been seen, and `served` shrinks to 2 — forgetting the earlier run. That
+    // shrink is the case the widening invariant has to survive.
+    appendFileSync(logPath, "line-11\nline-12\nline-13\n");
+    const narrow = await bgtail.execute(
+      "s3",
+      { id, lines: 2 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    const narrowText = narrow.content[0].text as string;
+    assert.match(narrowText, /line-13/);
+    assert.doesNotMatch(narrowText, /line-11/, "line-11 has never been shown");
+
+    // So widening to 4 must hand back line-11 — the line nothing has shown yet.
+    // It may ALSO repeat a line seen before the gap (coverage is a single trailing
+    // run, so a repeat is the documented behaviour), which is why only the no-loss
+    // half is asserted here: a future dedup fix should keep this test passing.
+    const widened = await bgtail.execute(
+      "s4",
+      { id, lines: 4 },
+      undefined,
+      undefined,
+      ctx,
+    );
+    assert.match(
+      widened.content[0].text as string,
+      /line-11/,
+      "a widening must not lose the line the narrowing read skipped",
+    );
+  });
+});
+
 test("bgtail: raw:true keeps the verbatim window but still advances the bookmark", async () => {
   await withJobsDir(async (dir, h) => {
     const { wakes, tools, ctx } = h;

extension/index.ts

@@ -1919,6 +1919,13 @@ export default function (pi: ExtensionAPI) {
     bytes: number;
     first: string;
     window: number;
+    /**
+     * Trailing content lines the caller has been shown. A later read asking for
+     * MORE lines than this is a widening request — the caller wants the older
+     * tail, not appended output — and is served as the slice between the two
+     * windows instead of the misleading "(no new lines since last read)".
+     */
+    served: number;
     /**
      * The resolved path this bookmark was taken against. An id can outlive its
      * file: a `bg_N` id reused by the host after a session transition, or a
@@ -3759,10 +3766,15 @@ export default function (pi: ExtensionAPI) {
     let window: string[];
     let header: string | undefined;
     let newLines: number | undefined;
+    // Trailing content lines the caller will have been shown by this read.
+    // Bookmarked so a later read can tell "asked for a wider window" (wants
+    // older lines it has not seen) apart from "polling" (wants appended lines).
+    let served: number;
     if (raw || prev === undefined || shrank || replaced || windowChanged || pathChanged) {
       // Full tail: first read, raw mode, a shrunken/replaced log, or a changed
       // search window (all resets).
       window = rawLines.slice(-lines);
+      served = window.length;
       if (!raw) {
         header = pathChanged
           ? "log path changed since last read (the id now resolves elsewhere) — showing full tail"
@@ -3778,12 +3790,62 @@ export default function (pi: ExtensionAPI) {
       const fresh = rawLines.slice(prev.lines);
       newLines = fresh.length;
       if (fresh.length === 0) {
+        // No appended lines — but a `lines` larger than what the caller has
+        // already been shown is not a poll, it is a request for the OLDER tail
+        // (`tail -40`, then `tail -80`). Answering that with "no new lines"
+        // replies to the wrong question and reads as "nothing more to see", so
+        // hand back the slice between the two windows instead. When everything
+        // is already covered (served === total) there is nothing older to give,
+        // and the cheap poll response stands.
+        const widenedTo = Math.min(lines, total);
+        if (lines > prev.served && total > prev.served) {
+          const older = rawLines.slice(
+            Math.max(0, total - lines),
+            total - prev.served,
+          );
+          rememberTail(id, {
+            lines: total,
+            bytes: size,
+            first,
+            window: readWindow,
+            path: logPath,
+            served: widenedTo,
+          });
+          const condensed = condenseLogLines(older, { raw });
+          const olderNotes =
+            condensed.truncated.length > 0
+              ? `\n\n(${condensed.truncated.join("; ")})`
+              : "";
+          return {
+            content: [
+              {
+                type: "text",
+                text:
+                  `no new lines since last read — showing the ${older.length} earlier line${older.length === 1 ? "" : "s"} of the widened window ` +
+                  `(now the last ${widenedTo} of ${total})\n${condensed.text}${olderNotes}${capNote}${windowNote}${lineNote}`,
+              },
+            ],
+            details: {
+              id,
+              linesShown: older.length,
+              logPath,
+              notFound: false,
+              condensed: true,
+              newLines: 0,
+              totalLines: total,
+              widened: true,
+              windowBytes: readWindow,
+              ...truncDetails,
+            },
+          };
+        }
         rememberTail(id, {
           lines: total,
           bytes: size,
           first,
           window: readWindow,
           path: logPath,
+          served: prev.served,
         });
         return {
           content: [
@@ -3806,6 +3868,13 @@ export default function (pi: ExtensionAPI) {
         };
       }
       window = fresh.length > lines ? fresh.slice(-lines) : fresh;
+      // Coverage is contiguous from the end: every fresh line when they all fit
+      // the request, otherwise just the requested window (the unseen gap above
+      // it is why `served` falls back to `lines`).
+      served =
+        fresh.length > lines
+          ? lines
+          : Math.min(prev.served + fresh.length, total);
       header =
         `+${fresh.length} new line${fresh.length === 1 ? "" : "s"} since last read — ` +
         `log at ${total} lines${fresh.length > lines ? ` (showing last ${lines})` : ""}`;
@@ -3816,6 +3885,7 @@ export default function (pi: ExtensionAPI) {
       first,
       window: readWindow,
       path: logPath,
+      served,
     });
     const shown = window;
     const { text, truncated } = condenseLogLines(shown, { raw });
@@ -3850,7 +3920,7 @@ export default function (pi: ExtensionAPI) {
     name: "bgtail",
     label: "Tail Background Log",
     description:
-      "Read the newest lines of a background job's log, condensed for context: ANSI escapes stripped, repeated lines collapsed, long lines truncated, output capped (~8KB). Strips the exit-marker line. The first read returns the last N lines (default 40); REPEAT reads return only lines appended since your last read (delta tailing) — polling a running job never re-pays for the same lines. raw: true returns the unprocessed last-N window. A shrunken, replaced, or differently-windowed log resets to a full tail. Reads only the log's last 2 MiB by default (`bytes` widens it, max 64 MiB) and says so when the log is bigger. For pattern search use bggrep; for whole-log analysis, read the log path directly (it is printed by bgstatus and in these notes).",
+      "Read the newest lines of a background job's log, condensed for context: ANSI escapes stripped, repeated lines collapsed, long lines truncated, output capped (~8KB). Strips the exit-marker line. The first read returns the last N lines (default 40); REPEAT reads return only lines appended since your last read (delta tailing) — polling a running job never re-pays for the same lines. A larger `lines` than you have already been shown is a widening request: it returns the lines above your current coverage rather than the whole window again (coverage is a trailing run, so a line seen before an intervening gap can repeat). raw: true returns the unprocessed last-N window. A shrunken, replaced, or differently-windowed log resets to a full tail. Reads only the log's last 2 MiB by default (`bytes` widens it, max 64 MiB) and says so when the log is bigger. For pattern search use bggrep; for whole-log analysis, read the log path directly (it is printed by bgstatus and in these notes).",
     promptSnippet: "Read the last N lines of a bgrun job's log",
     parameters: Type.Object({
       id: Type.String({