Skip to content

fix(dash): a keep-alive credit can be booked on a thread that never sent a ping, and the read-time gate accepts any thread's ping in the session #864

Open

Description

@OsherElhadad

Summary

The keep-alive saving credits a returning request when the keeper "decided" to ping its thread. It does not check that a ping row exists on that thread. The read-time gate that was added to catch this (kaSavedGate) checks the whole session, not the thread, so a ping on a sibling subagent satisfies it. On one hosted deployment, 91 of 279 credited rows have no ping on their own thread in the idle span. They carry $47.97 of the $249.51 credit.

What we found

Data (class 2, operational): union of two read-only snapshots of one hosted internal build (core ~v0.6.x), metadata columns only, 2026-10-28 to 2026-11-02. A "credited row" is a real request with keepalive_saved_usd > 0. "Own ping" is a ping row (keepalive = 1) with the same session and thread, between the thread's previous real request and the credited row.

Credited rows (279, $249.51) Rows Credit
a ping on the row's own thread 188 $201.54
no own ping, but a ping on another thread of the session 75 $37.03
no ping anywhere in the session in the span 16 $10.95
  • All 91 rows without an own ping have keepalive_pings = 1. So the write path took them as "a ping happened".
  • 89 of the 91 are in one session (about 13 concurrent subagent threads), all after 2026-11-01. 85 of the 91 are subagent threads; 90 follow a tool_use stop.
  • The gap from the previous request on the thread is 266 to 749 s (median about 380 s). 77 of 91 are over 330 s.
  • Typically the returned request read all of its previous prefix (median read / previous prefix = 1.0), so the entry was alive past 5 minutes with no ping of ours on it. The credit takes the whole cache_read because KeepAliveRefreshed is 0 (no ping read anything), so keepaliveSavedUSD falls back to n = e.CacheRead.
  • Of the 91, the read-time gate (kaSavedGate) still passes 62 ($30.78) because some other thread's ping row read the cache inside the window, and rejects 29 ($17.20).
  • 37 of the 91 sit on threads that had never been pinged before.

This is not the same count as the thread-key question: the thread id stored on the row is the one passed to arriveAt (proxy/proxy.go: threadID = th.ID, then e.ThreadID = c.thread, proxy/dashcapture.go), so the stored label and the lookup key agree at request time. I did not test relabelling directly; the 91 are the rows where the stored thread id has no ping, so a relabel after the fact is not excluded.

Why it happens (source at b4dfbfe; the cause of the missing ping rows is not proven)

  1. Keeper.Sweep increments e.pings under its lock before the ping is sent (keepalive/keeper.go, e.pings++ before building the pingJob). Arrive returns that count and it becomes KeepAlivePings (dash/event.go, field doc: "dispatch intent ... not the same as how many were sent").
  2. fire can return with no request and no row: k.spare refusing the tenant's headroom (k.skipped.Add(1); return), a body that cannot be built, or a transport error. giveBackFailedPing undoes the count only for a transport error. The tenant limiter keeps a quarter of its budget back (reserveFrac, keepalive/ping.go), so a session with a dozen threads due at the same second is the likely case. I cannot see the skipped counter in the snapshot, so this is a hypothesis.
  3. keepaliveSavedUSD (dash/event.go) gates on KeepAlivePings > 0, which step 2 leaves true. The comment there records the same leak at a smaller size on an older corpus (7 of 105).
  4. The read-time repair kaSavedGate (dash/keepalive.go) joins ping rows on tenant_id and session_id only (kp.session_id = <alias>session_id), not on thread_id. Dashboard queries: use the thread id where they ask about one cache entry #427 lists the other dashboard queries that need thread_id; this one is not in its table.

Related, not the same: #853 (the credit counts a shared 20k head that siblings keep warm) is about how much of a real rescue is ours. This is about rows with no rescue by us at all. #427 is the general "key by thread" change; this is one more query for it.

Suggested fix

  1. In kaSavedGate, add AND kp.thread_id = <alias>thread_id (rows from before feat(keepalive,dash): key cache attribution and keep-alive per thread #426 have thread_id = '' and keep today's result).
  2. Count the intent only when it is sent: move KeepAlivePings to the number of ping rows actually booked, or have fire hand back the count (as giveBackFailedPing does) on a limiter refusal.
  3. Do not fall back to the whole cache_read when KeepAliveRefreshed == 0; credit 0 or the head only.
  4. Test: a session with two threads where only one is pinged; the other thread's return must not be credited.

Who it affects

Any deployment with parallel subagents and the keep-alive credit shown on the dashboard. It overstates the credit and the net keep-alive figure. It does not change what is billed. On this deployment about 19% of the credit by value ($47.97 of $249.51), concentrated in one session. One developer's session holds most of it, so do not read the percentage as a rate.

Limits

  • One deployment, 5 days, one session carries 89 of the 91 rows.
  • Whether the missing pings were refused by the limiter is not shown by the data.
  • The entry survived without our ping in these rows; why (sibling prefix, longer provider lifetime, a ping I cannot see) is unknown.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions