Skip to content

observability: every voice turn logs backend "unknown" — paid-vs-free token attribution is blind again #306

Description

@adrianwedd

Split out of #304, where it was parked as a secondary finding.

What

state/token_usage.json on the live robot, 2026-08-26:

"by_backend": {
  "ollama-m5":    {"call_count": 4862, "last_ts": "2026-08-26T08:06:25Z"},
  "claude":       {"call_count": 2659, "last_ts": "2026-08-20T03:54:44Z"},
  "ollama-local": {"call_count": 39,   "last_ts": "2026-08-19T13:13:20Z"},
  "unknown":      {"call_count": 1078, "last_ts": "2026-08-26T02:35:12Z"}
}

The claude bucket stops on 2026-08-20. Everything Claude has served since
lands in unknown — 1,078 calls and counting.

Mechanism

token_log.log_usage(input_text, output_text, backend="unknown") defaults the
label, and src/pxh/voice_loop.py:1292 is the one production caller:

log_usage(prompt, stdout)

No backend=. It is the only log_usage() call site in src/ or bin/, so
every voice turn — now served by the resident spark-brain session — is
recorded unattributed.

Why it matters

CLAUDE.md states the reason by_backend exists: "The top-level totals mix free
Ollama with paid Claude and cannot answer 'what am I spending'."
With the
largest active consumer landing in unknown, that question is unanswerable
again — which is the exact regression the split was added to prevent.

The information is already available at the call site: call_llm() sets
result["backend"] to the tier that actually served, and ask_brain() knows
its session. Nothing needs to be inferred.

Wanted

  • Pass the serving backend at voice_loop.py:1292 (the value call_llm()
    already returns, not the configured primary).
  • Consider making backend a required argument of log_usage() so a future
    call site cannot silently re-open this — the same fail-closed direction as
    session_for_kind raising on an unclassified kind.
  • Leave existing unknown history in place; do not retro-attribute it.

Acceptance

  • A voice turn served by spark-brain increments by_backend["claude"].
  • A turn served by M5 increments by_backend["ollama-m5"].
  • No production call site can reach log_usage() without a backend label.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions