agent: bound the agent that works forever and never reports - #47
Open
whilo wants to merge 1 commit into
Open
Conversation
MEASURED on dev.simm.is, 2026-08-06 06:00–08:11. Vár ran turns 0→15, every
one a `clojure_eval`, `content-len 0` on all of them — she never said a word
to the room — then the cycle restarted at turn 0 and did it again. From
where the humans sat, nothing happened for two hours. The read loop she was
driving also produced 56,802 datahike fulltext warnings, ~2,400 a second at
peak.
Nothing objected, and both existing bounds were behaving correctly:
- `detect-doom-loop` fingerprints `[:tool-use/name :tool-use/input]`, so it
catches an agent REPEATING itself. She varied her code every turn (input
tokens climbed 40,275 → 42,265). Not repetition — non-termination.
- the dollar budget bounds the cycle, but at ~40k input tokens a turn it
buys a great many turns, and it ends the turn silently, which is why the
room saw nothing rather than an error.
`detect-silent-tool-run` adds the missing bound: consecutive tool-only
assistant turns since the agent last actually spoke. Past twelve it gets a
system note — nobody can see tool calls, report what you have — and KEEPS
GOING. That is the difference from the doom-loop path, which stops: an agent
working silently is not necessarily stuck, it may be mid-way through
something real. It just has to tell the room. Replying is what resets the
streak, so the nudge is said once per streak by construction.
Ordering matters and cost me a rewrite: I first checked silence before
repetition, which meant an agent doing both got nudged to CONTINUE instead
of being stopped. Repetition is the stronger signal and is checked first.
The streak walks back to the last turn that SPOKE, then counts tool turns in
between — rather than stopping at the first non-tool turn. Otherwise a model
emitting a bare newline resets the bound: that turn says nothing, so it must
not count as a report, and it has no tools, so it must not count as work.
My own test caught that; the first implementation had the weaker rule.
Threshold is twelve, not three. A long tool run is ordinary work — reading a
dozen pages before answering is what these agents are for, and cutting that
short would be worse than the bug.
`no_reply_loop_test` reproduces the production shape (sixteen varying calls,
no prose) and asserts `detect-doom-loop` returns nil on it — the gap is
documented as a test, not just a comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The incident
dev.simm.is, 2026-08-06 06:00–08:11. Vár ran turns 0→15, every one a
clojure_eval,content-len 0on all of them — she never said a word to the room — then the cycle restarted at turn 0 and did it again. From where the humans sat, nothing happened for two hours.The read loop she was driving also produced 56,802
datahike/secondary-index-restore-failedwarnings, ~2,400 per second at peak. (That flood is a separate datahike issue — a missing negative cache on secondary-index restore — but this is what drove it.)Why nothing objected
Both existing bounds were behaving correctly:
detect-doom-loopfingerprints[:tool-use/name :tool-use/input], so it catches an agent repeating itself. She varied her code every turn (input tokens climbed 40,275 → 42,265). Not repetition — non-termination. It logged zero times.The fix
detect-silent-tool-run: consecutive tool-only assistant turns since the agent last actually spoke. Past twelve, a system note — nobody can see tool calls; report what you have — and it keeps going.That last part is the difference from the doom-loop path, which stops. An agent working silently is not necessarily stuck; it may be mid-way through something real. It just has to tell the room. Replying resets the streak, so the nudge is said once per streak by construction.
Threshold is twelve, not three: a long tool run is ordinary work. Reading a dozen pages before answering is what these agents are for, and cutting that short would be worse than the bug.
Two things worth reviewing
Ordering. I first checked silence before repetition — which meant an agent doing both got nudged to
:continueinstead of being stopped. Repetition is the stronger signal and is now checked first.Streak semantics. It walks back to the last turn that spoke, then counts tool turns in between, rather than stopping at the first non-tool turn. Otherwise a model emitting a bare newline resets the bound: that turn says nothing, so it must not count as a report, and it has no tools, so it must not count as work. My own test caught this; the first implementation had the weaker rule.
Tests
no_reply_loop_testreproduces the production shape — sixteen varying calls, no prose — and assertsdetect-doom-loopreturnsnilon it, so the gap is documented as an executable fact rather than a comment. Also covers: speaking resets the streak, whitespace is not speaking, and empty turns don't extend it.clojure -M:test— 415 tests, 1643 assertions, 0 failures.