Finish runs instead of crashing on recoverable checks, and write down how to fail well

[egeozin]

What and why

Over the last 14 days, 129 of 661 production runs failed (about 20%). Many causes are already fixed (#273, #275, #278, #279, #282, #285, #287, #297). Several of the rest were checks that crashed a run, often one that had already paid for its model calls, over something the run could have finished from. This PR fixes four of them and writes the rule down so new code fails well.

1. "Already covered" no longer crashes on an unreadable extra page. On Sheepdogs, content.generate fe7a1329 refreshing /how-the-sheep-get-home:

• read that page in full, tried the canvas homepage (unreadable), and judged the brief already covered;
• the validator required every compared page to be readable, so the paid run crashed: "Unavailable coverage is not evidence that the brief is already covered."

Now:

• Unreadable pages don't count, for or against. "Already covered" needs the site's own coverage read, which for an update means its destination.
• When that's missing, the judgment ends as insufficient_evidence ("Coverage could not be established"). The run finishes with that outcome instead of crashing.
• Draft rules are unchanged.

2. Revision notes never fail a paid revision. article_review.validate_changes required exactly one "What changed" and one "Feedback not followed" section of 4–2,000 characters. A revision that followed all feedback and omitted the second section crashed (content.generate on 10-02: "Explain 'Feedback not followed' once…").

• These notes are for the founder, so the check is removed.
• change_summary already returns nothing when "What changed" is missing.
• Notes must still be bounded and separate from the copy.

3. A task that pauses without a question asks a plain one. Two project.task turns failed on 10-02 with "Codex requested input without a question". The task bridge now asks "How should I continue?", and the founder can answer or stop.

• This is in sandbox/task_app_server.py, which is baked into the sandbox templates. It takes effect with the next rebuild of tin-lite-codex-isolated.

4. A page refresh never fails over how its review tables are written. content.refresh 0de1f797, the first run after #297, failed after its paid run with "Show the current and proposed title in the review table."

• The validator required each old and new value to appear verbatim in the prose table, and Tin's earlier-results table to be copied unchanged.
• Both checks are removed. Every exact change is in the replacements block, which the founder also sees and which is still fully checked: the old text must match the live page, the new text must be plain and within its limit, and so on.

5. AGENTS.md: "Failing well".

• Refuse only to protect security, money or outside effects.
• A check on model output never fails a run that already paid for it.
• Accept partial provider data on reads.
• Limits are runaway guards.
• A failure names its cause.
• Run a workflow change once for real when a provider, sandbox or outside site is involved.

No migration and no Temporal change.

Failures in the last 14 days, by real cause

From run errors and the bridge's own messages in the journal:

Count	Cause	Status
9	"Codex procedure turn failed" (a model turn errored; last 09-30)	#297 now names in-stream OpenAI failures; the rest remain for the hardening pass
6	Code map / deep dive: "memory index changed outside" their section	separate PR (Tin keeps only the run's own section)
5	Codex observed token limit	#297 (v5 runaway limits)
4	"Codex usage limit reached" (OpenAI rate limit inside a 200 stream)	#297 names it
3	"Codex changed an invalid number of repository files"	#282 (context in the prompt)
2	"no pull-request metadata" on a no_change outcome	#282's bridge change, live after the template rebuild
2	Task paused without a question	this PR (3)
1+1	Content judgment and revision notes	this PR (1, 2)
1	Page refresh review table (after #297's deploy)	this PR (4)

Verification

• New tests, each failing on main:
  • test_an_unreadable_extra_page_does_not_sink_an_already_covered_judgment, the Sheepdogs case;
  • test_covered_without_reading_the_destination_finishes_as_insufficient_evidence;
  • test_copy_and_notes_are_different_contracts, rewritten: missing, prose-only or long sections are accepted, and unbounded or non-notes content is still refused;
  • test_a_task_that_pauses_without_a_question_asks_a_plain_one;
  • test_the_review_tables_never_fail_a_paid_refresh, replacing the test that expected those failures.
• Full suite: 5,221 passed, 21 skipped against a disposable Postgres. The 2 macOS-only localtime tests fail as on main. ruff, format, lint-imports and dump_catalog.py --check are clean.

🤖 Generated with Claude Code