[Feature]: LLM Context Inspector should retain failed physical request attempts and retry lineage #179

Product or interface

Source build or repository tooling

Use case and problem

The LLM Context Inspector currently loses some of the most useful evidence for debugging and evaluation: failed physical provider requests that precede either recovery or terminal failure.

In:
packages/local-runtime-v2/src/service/llm-context-inspector/capture-recorder.ts
the current behavior is explicitly documented as:

/** Captures one successful logical Call while discarding every failed attempt. */

The persistence condition also requires the final logical outcome to be successful:

if (event.scope !== CAPTURED_SCOPE || event.final.outcome !== 'success') return false;

The persisted call record contains attemptCount, so the Inspector can tell that a successful logical call required multiple physical requests, but it cannot show what happened during those attempts.

For example:

logical call A
  request 1 -> timeout
  request 2 -> rate limited
  request 3 -> success

is effectively retained as:

logical call A
  attemptCount: 3
  final request -> success

The first two requests disappear from Inspector evidence.

A logical call that exhausts retries and fails is even more problematic because the Inspector's success-only persistence rule means there may be no equivalent captured logical-call record at all.

Failed attempts are often the most valuable evidence when investigating:

• provider instability
• rate limiting
• timeouts
• retry effectiveness
• malformed or incomplete responses
• transient protocol failures
• latency introduced by recovery
• differences between providers or models
• eventual-success cases that conceal substantial degradation
• terminal failures after retry exhaustion

The retry subsystem already distinguishes logical calls from physical provider requests and tracks physical request ordinals and outcomes. The Inspector should preserve that trajectory rather than reducing it to a successful final result plus an aggregate count.

Desired behavior

Retain evidence for every observed physical provider request associated with a captured logical LLM call, including failed attempts.

A physical-attempt record could contain information along the lines of:

{
  callId,
  requestId,
  requestAttempt,
  startedAtMs,
  endedAtMs,
  outcome,
  errorKind?,
  requestState,
  responseState,
  usage?
}

The exact event schema is flexible, but the resulting Inspector evidence should support both:

logical call A
  attempt 1 -> timeout
  attempt 2 -> rate_limited
  attempt 3 -> success
logical call A -> success

and:

logical call B
  attempt 1 -> server_error
  attempt 2 -> server_error
  attempt 3 -> timeout
logical call B -> error

Desired properties:

• Each physical attempt is retained with stable identity and ordering.
• Attempts remain associated with the logical callId.
• Recovered calls retain the failures that occurred before the successful attempt.
• Terminally failed logical calls remain inspectable.
• Normalized failure classification is retained where available.
• Request/response capture continues to respect the Inspector's existing size bounds, privacy controls, and capture enablement.
• Missing or unavailable raw payloads do not prevent retaining lifecycle metadata for the attempt.
• Existing logical-call summaries can remain available as aggregate views.
• Capture remains fail-open and never changes agent behavior, retry behavior, or provider request settlement.
• Existing Inspector history remains readable or has an explicit version/migration strategy if the persisted event schema changes.

The purpose is to preserve the native retry trajectory, not merely increase the amount of raw error data stored.

Platform

Multiple platforms

Alternatives and additional context

Keeping only attemptCount is insufficient for reliability or behavioral analysis because it says that retries occurred without retaining their causes, timing, ordering, or outcomes.

Generic runtime logs can sometimes reveal failures, but they do not provide the same durable association with Inspector request/response evidence and logical-call history.

Likewise, persisting only the final failure reason would still lose the sequence of physical attempts that led to the terminal outcome.

Related: #162 proposes exposing stable physical-request identity, logical-call correlation, attempt ordinal, and settlement outcome through the generic host observer. This issue is specifically scoped to retaining that physical-attempt and retry evidence in the LLM Context Inspector.