Repository navigation
fix(bgrun): discourage polling a job you just started - #27
Merged
Merged
Conversation
Every measured session that used bgrun opened the same way: `bgrun`, then `bgstatus`, then `bgtail` — before the job could possibly have finished. In one run those two calls were 2,231 of 8,049 context chars (28%), spent learning what the wake delivered for free moments later. Nothing told the agent not to. BGRUN_GUIDELINES mentioned the wake only in passing, the `started:` result said merely "You'll be woken automatically when it finishes", and the run-bg skill actively licensed the polls in two places: - step 2: "`exit: 0` → success. `bgtail` to confirm." — a read even when the wake already carried the counters; - step 3: "If you need to check before the wake (non-blocking): call `bgstatus`…" — framed as ordinary practice. Now: - BGRUN_GUIDELINES gains: "Do not poll a job you just started — no bgstatus/bgtail round after bgrun. The wake is the signal: it carries the exit code, duration, log line count, the log's last line and the project's digest. Read the log after the wake only for detail the wake does not carry…" - The `started:` result ends "…no need to poll; the wake carries the exit code, counters and digest." - Skill step 2 makes the wake itself the result and drops "bgtail to confirm"; step 3 is reframed — checking before the wake is rarely worth it, so reach for `bgstatus` when the job is NOT one you just started (another session's, one from before a restart, or one whose wake never came). Verified: `tsc --noEmit` clean; 224 tests pass (no test asserted the old wording); and a live smoke where an agent asked to start a job makes exactly one call — bgrun — and answers "I'll be woken when it finishes — no polling needed", where every earlier arm polled twice.
Contributor
CI report
Ref: |
…xit code Review of the first commit found the change contradicted itself: the wake still ended with an unconditional "Review the result now: call `bgtail` …", emitted on exit 0 too — the last instruction the agent reads, telling it to make exactly the read this branch discourages. The smokes could not catch it: they are one-shot sessions that exit before the wake arrives, so they only measure the pre-wake decision. The verification was blind by construction. - The wake now ends with one of two lines: on success "The exit code, stats and last output above are the result — report it, and continue the task that depended on this…"; on failure "Analyze the failure: `bgtail` for a peek at the end of the log, or `bggrep` to search it — then continue…". - The ❌ test asserts the failure wording AND that the success wording is absent, so the two paths cannot silently converge again. - `digestBlockOf` no longer delimits the digest block by that instruction's wording — which is what made four digest tests fail on the first attempt; it now takes the block from the `digest (…)` line to the wake's last line. - README's sample wake (an exit-1 job) shows the failure variant. - Accuracy fix from the same review: the guideline and the `started:` line claimed the wake carries the line count, the last line and the digest unconditionally, but all three are conditional in the wake construction (line count when the log is present and within the read window, last line when the log has content, digest only when a project configures one). Both now carry qualifiers. Verified: `tsc --noEmit` clean, 224 tests pass, and no remaining reference to the old instruction anywhere in the tree.
lloydsk
added a commit
that referenced
this pull request
Sep 26, 2026
…grun does not help Three additions, prompted by the obvious hazard: a fixture we can tune is a fixture we can tune in the tool's favour. - **How these numbers are produced** — the protocol as one table (arms, prompt template, model, live sessions, n=3 reported as median and range, metric definitions, and the command that re-derives every table from the transcripts), plus the `vanilla-hinted` control arm: the agent hand-rolling redirect-and-grep, which is the strongest baseline in the set. - **Pre-registered expectations** — nine cells with the predicted winner written down before the runs, including the ones where bgrun is expected to lose (R1 green, R2 tail-shaped failure, R5 fail-fast) and the one where our own story may be wrong (D1: if the advantage is reliability-driven it should grow with a weaker model). C1-C4 are dialogues — overlap, three jobs at once, a mid-run restart, asking where a run is — because no generated fixture can express them. - **The fixture's design and why each property is under test** — the shared chain that makes duration scheduler-independent, the 65% failure position, the realistic failure block, and the knobs for the negative cells. Records that the earlier long-suite run used the verbose 8-lines-per-test density while the realistic default is 2, and that the difference is an axis rather than an oversight. - **Where bgrun helps, and where it does not** — the strongest claim the data supports is the removal of the worst case, not a better best case (vanilla's `tail -100` run beat every bgrun run on cost). Names the papercuts the measuring found: the 28% pre-wake poll (#27 fixed), window-relative bggrep line numbers (#25 open), and re-paid log reads before delta `bgtail` (#4).
lloydsk
added a commit
that referenced
this pull request
Sep 28, 2026
…grun does not help Three additions, prompted by the obvious hazard: a fixture we can tune is a fixture we can tune in the tool's favour. - **How these numbers are produced** — the protocol as one table (arms, prompt template, model, live sessions, n=3 reported as median and range, metric definitions, and the command that re-derives every table from the transcripts), plus the `vanilla-hinted` control arm: the agent hand-rolling redirect-and-grep, which is the strongest baseline in the set. - **Pre-registered expectations** — nine cells with the predicted winner written down before the runs, including the ones where bgrun is expected to lose (R1 green, R2 tail-shaped failure, R5 fail-fast) and the one where our own story may be wrong (D1: if the advantage is reliability-driven it should grow with a weaker model). C1-C4 are dialogues — overlap, three jobs at once, a mid-run restart, asking where a run is — because no generated fixture can express them. - **The fixture's design and why each property is under test** — the shared chain that makes duration scheduler-independent, the 65% failure position, the realistic failure block, and the knobs for the negative cells. Records that the earlier long-suite run used the verbose 8-lines-per-test density while the realistic default is 2, and that the difference is an axis rather than an oversight. - **Where bgrun helps, and where it does not** — the strongest claim the data supports is the removal of the worst case, not a better best case (vanilla's `tail -100` run beat every bgrun run on cost). Names the papercuts the measuring found: the 28% pre-wake poll (#27 fixed), window-relative bggrep line numbers (#25 open), and re-paid log reads before delta `bgtail` (#4).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every measured session that used bgrun opened the same way —
bgrun, thenbgstatus, thenbgtail— before the job could possibly have finished. In one run those two calls were 2,231 of 8,049 context chars (28%), spent learning what the wake delivered for free moments later.Nothing told the agent not to: the guideline mentioned the wake only in passing, the
started:result said merely "You'll be woken automatically when it finishes", and the run-bg skill licensed the polls in two places:exit: 0→ success.bgtailto confirm." — a read even when the wake already carried the counters;bgstatus…" — framed as ordinary practice.Change
BGRUN_GUIDELINESgains: "Do not poll a job you just started — no bgstatus/bgtail round after bgrun: the wake is the signal and carries the outcome (exit code, stats, the log's last line, and the digest when the project configures one). Read the log only when the wake is not enough — typically the detail behind a failure."started:result ends "…no need to poll; the wake carries the exit code, the stats and the log's last line."bgstatuswhen the job is not one you just started (another session's, one from before a restart, or one whose wake never came).The
bgtail/bgstatusdescriptions are deliberately untouched: they describe what those tools do, not when to reach for them.Follow-up from review (second commit)
The first commit left the contradiction in place: the wake still ended with an unconditional
…emitted on
exit: 0as well. That is the last instruction the agent reads on every completion, and it asks for exactly the read this PR discourages. The smokes could not catch it — one-shot sessions exit before the wake arrives, so they only measure the pre-wake decision.Fixed:
bgtail…bggrep…"), never both;❌test pins the failure wording and the absence of the success wording;digestBlockOfno longer delimits the digest block by that instruction's wording (it broke four digest tests), and the README sample (an exit-1 job) shows the failure variant;started:line no longer claim the wake carries the line count, last line and digest unconditionally — each is conditional in the wake construction.Verification
tsc --noEmitclean; 224 tests pass, including the new failure/success wake assertions.bgrun) and answer "I'll be woken when it finishes", where every earlier arm polled twice.