Skip to content

fix: show founders what they can fix and keep Tin's internals for operators - #403

Merged
demegire merged 1 commit into
mainfrom
fix/failure-classification
Oct 12, 2026
Merged

demegire merged 1 commit into
mainfrom
fix/failure-classification

Conversation

@demegire

Copy link
Copy Markdown
Contributor

What and why

We kept fixing the same bug one workflow at a time (#212, #270, #298, #387, #388, #393, #394), and it went both ways. Some failures hid the cause a founder could act on: X voice capture said "could not confirm completion" when X needed reconnecting, and a billing refusal became "the model result is unconfirmed". Others showed Tin's internals verbatim: KeyError: 'rows', database constraint names, sandbox SDK text, pydantic dumps. Nothing in the code separated "safe to show" from "the founder can act on it".

This PR adds that distinction once, in failures.py. Every failure is one of:

Kind What the founder sees
needs you Tin's words, plus a link to Integrations or Billing
temporary Tin's words or "try again in a few minutes"
Tin's own sentence kept as written
internal "Tin hit a problem on its side. Nothing you set up caused it. Try again; if it happens again, tell us." The raw text goes to the operator-only error_detail.

1. Classification for every run

  • An activity interceptor attaches the kind to each failed activity. The exception's type, message and retryability are unchanged, so retry policies and non_retryable_error_types behave as before.
  • Migration 063_run_failure_kind.sql adds error_kind, error_action and error_detail to workflow_runs. error_detail is never returned by the API or MCP.
  • project_failure keeps the first message of a run that already failed, so a later generic projection no longer overwrites a specific one.
  • The dashboard run detail links a needs-you failure to Integrations or Billing. MCP get_run and list_project_runs return error_kind and error_fix.
  • Silent auto-retry still decides on the raw text, exactly as before; the kind does not widen it.

2. Billing wording

  • billing_refusal replaces per-site handling. Only project_limit and run_limit read as spending limits. insufficient_funds and schedule_not_funded get their own wording, and usage_pending reads as temporary. Tin's pricing and ledger states read as internal, and API diagnostics hide them.
  • Applied to workflow.code model steps and paid reads, studio, content delivery, X delivery, keyword plan, organic audit, the Ads assessment, monitor and launch, payment recovery and the AI answers ledger.
  • While an earlier paid step is still settling, a schedule skips that occurrence instead of pausing.

3. Failure activities that dropped the cause

  • The 18 run-ID-only failure activities now receive the classified cause through failure_input, behind a failure-reason-v1 patch. Older histories still pass a plain run ID and keep their old wording.
  • Each activity keeps its own fixed sentence only when the cause is internal. Specific messages saved on receipts still take priority.
  • Covered: X style, revise, draft and publish; style capture; growth plan and onboarding; content plan; organic system and audit; keyword plan; Ads assessment, monitor and launch; payment recovery; awesome submit; connection collection; code workflows.

4. HTTP and MCP responses

  • Every detail=str(exc) now goes through public_error_text. Tin's own sentences pass, including typed ApplicationErrors and anything raised outside the standard library and installed packages. Library, parser, driver and provider text does not.
  • MCP masks the text of internal RuntimeErrors (provider, sandbox, Temporal) but keeps input errors, so an agent can still fix its call.
  • run_service stops rewrapping internal errors as input errors. GitHub install and account-choice errors keep their type, so the dashboard's install prompt fires again.

AGENTS.md "Failing well" now states the rule.

Verification

  • Full suite with a disposable local Postgres 16: 5868 passed, 66 skipped. The skips are Temporal CLI and opt-in live cases. CI uses Postgres 17.
  • After rebasing onto Plan and monitor Google Ads in the account's own currency #402: the Ads, failures and Temporal replay tests pass (196 passed, 7 skipped).
  • New tests/test_failures.py covers the classifier, billing wording, the interceptor, the workflow side, receipts, and HTTP/MCP masking. Each changed failure activity has tests for an old plain-string history, a needs-you cause and an internal cause, and each billing fix checks that a retry makes no second paid call.
  • Ruff, format and import-linter are clean.

Notes

🤖 Generated with Claude Code

…rators

Classify every run failure as needs you, temporary, Tin's own sentence or internal
(failures.py). An activity interceptor attaches the kind; workflows pass it to their
failure activities behind failure-reason-v1; runs store error_kind, error_action and an
operator-only error_detail (migration 063). Billing refusals get per-code wording, and
HTTP and MCP bodies stop relaying library, driver and provider text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@demegire
demegire requested a review from a team as a code owner October 12, 2026 15:58
@demegire
demegire merged commit 83ddb68 into main Oct 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant