Define the telemetry snapshot contract #224

Activity

1. lbacik

   added

   wayfinder:grillingDecision requiring a live human discussionDecision requiring a live human discussion

   on Oct 14, 2026
2. lbacik

   added a parent issue on Oct 14, 2026

   • Agent fleet dashboard: map #222
3. lbacik

   marked this as blocked on Oct 14, 2026

   • Research: which limit and context-window data the model provider and SDK expose #223
4. lbacik

   marked this as blocking 3 issues on Oct 14, 2026

   • Decide the agentctl telemetry command: socket operation, snapshot publication, failure isolation #225
   • Prototype the dashboard UI: fleet overview and instance detail #227
   • Define the dashboard project: repo, layout, stack and delivery slices #228
5. lbacik

   unmarked this as blocking on Oct 14, 2026

   • Prototype the dashboard UI: fleet overview and instance detail #227
6. lbacik

   marked this as blocked by 2 issues on Oct 14, 2026

   • Prototype the dashboard UI: fleet overview and instance detail #227
   • Run the live checks from the provider-limits research #229
7. lbacik commented on Oct 14, 2026

   

   lbacik

   on Oct 14, 2026

   OwnerAuthor

   More actions

   Requirement from the resolved ticket "Decide instance discovery and the dashboard's Docker access" (#226): the snapshot must identify its own target repository (TARGET_REPO), because the dashboard performs no separate status check before reading and displays the repository the snapshot reports. Instance identity on the dashboard side is the container name, so the snapshot does not need its own instance id for v1.
8. lbacik

   self-assigned this

   on Oct 14, 2026
9. lbacik commented on Oct 14, 2026

   

   lbacik

   on Oct 14, 2026

   OwnerAuthor

   More actions

   Resolution

   Decided in a grilling session; the operator asked to take the recommended answer on every question. Facts were checked against the code at 8b69ac3 (file references below).

   Terminology

   • Telemetry snapshot and Figure basis added to CONTEXT.md on branch docs/fleet-dashboard-glossary (commit 17fec34).
   • The per-figure marker the prototype called "provenance" is named basis in the contract: "provenance" is already taken by provenance verification of the pinned runtime.

   Shape (schema version 1)

   {
     "schema_version": 1,
     "generated_at": "…Z",            // when this reply was serialised
     "sequence": 412,                 // +1 per published telemetry update; resets with the process
     "identity": {
       "repository": "owner/name",    // TARGET_REPO (requirement from #226)
       "agent_version": "0.4.4",
       "sdk_version": "0.2.163",      // from provenance evidence
       "cli_version": "2.1.286",
       "process_started_at": "…Z"     // new: recorded at startup
     },
     "control": { … },                // the `agentctl status` payload, same builder, plus additions below
     "current_attempt": { … } | null, // null = no attempt is active (not applicable, not "unavailable")
     "last_finished": { … } | null,   // null = no finished attempt known
     "recent_events": [ … ],
     "truncated": false
   }

   Figure object. Every numeric figure in current_attempt / last_finished is {"value": <number|null>, "basis": "measured|estimated|configured|sdk_assumed|unavailable"}; unavailable always has value: null and may carry a short reason. Plain fields (ids, timestamps, enums, booleans) are bare values.

   control

   • Reuses build_status (control.py:2053) verbatim, so status and telemetry cannot drift: agent_version, repository, intake, recovering, active_attempt, pending_command, pending_next_issue, stop_plan, recovery (incl. free-text next_action, shown ready to copy), handoff (phase, deadlines), consecutive_errors (count / max), commands.
   • Additions: active_attempt.issue_title (new checkpoint field written at claim; today the title lives only on the claim object, lifecycle.py:2530).
   • Requirement for Decide the agentctl telemetry command: socket operation, snapshot publication, failure isolation #225: building the telemetry reply performs no network I/O. Today recovery.publication is filled by live GitHub lookups (lifecycle.py:2260-2295); in the snapshot it carries the last known values and comment_confirmed: null / pull_request: null mean "not known yet".

   current_attempt

   Present from claim until the attempt is finalised; after the model execution ends (pushing, publishing, handoff publication) the figures stay frozen at their final values and observed_at stops advancing.

   Field	Content	Basis	Source
   attempt_id, issue_number, observed_at	checkpoint started_at, issue, time of the last telemetry publication	—	checkpoint, executor
   budget.used	budget_tokens	measured if estimated part is 0, else estimated	TokenLedger
   budget.measured, budget.estimated	split of used (drives the hatched bar)	measured / estimated	TokenLedger
   budget.hard_ceiling	MAX_BUDGET_TOKENS	configured	config
   budget.soft_threshold	{value, rule: percentage|reserve, percentage, reserve_turns, crossed, crossed_at_tokens, crossed_at}; value is the effective threshold after the handoff reserve	configured under percentage; under reserve inherits main context's basis	executor _check_limits (model_execution.py:1925)
   budget.hard_ceiling_reached	bool	—	executor
   tokens_by_category	input_tokens, cache_read_input_tokens, cache_creation_input_tokens, output_tokens (measured part only; the estimated remainder is budget.estimated)	measured	measured_by_category
   turns	main-thread response count, with max (MAX_TURNS, configured)	measured	response_count("main")
   continuation	{index, max}; index computed once at claim and stored in the checkpoint (today a tracker lookup, lifecycle.py:2306)	configured for max	checkpoint, config
   cache.main_hit_rate	with counted responses; unavailable below PROMPT_CACHE_MIN_MEASURED_RESPONSES	measured	main_cache_stats
   context.main, context.main_peak	latest and peak main-thread context	per ledger reading	context_tokens("main"), peak_main_context_tokens
   context.window	the window the CLI actually uses	sdk_assumed unless CLAUDE_CODE_MAX_CONTEXT_TOKENS is set, then configured	get_context_usage().rawMaxTokens, called on the attempt's event loop at session start and after each compaction (cost verified in #229)
   context.auto_compact_threshold	CLI threshold	same as window	get_context_usage()
   context.compactions	{count, last: {at, trigger, pre_tokens, post_tokens}}	measured	compact_boundary / compact_metadata
   context.subagents	list {id, type, context, running}; running ones first, at most 20 entries	estimated	thread_contexts() + subagent start/finish tracking (new)
   provider_rate_limit	always {value: null, basis: "unavailable", reason: "not exposed through the gateway"} in v1	unavailable	#229 check 3
   elapsed_seconds, timeout_seconds		measured / configured	executor, MODEL_TIMEOUT

   Cut from v1 (prototype decision): USD cost (total_cost_usd is not sent).

   Between attempts: last_finished

   {attempt_id, issue_number, issue_title, outcome, finished_at, budget_used, turns, pull_request: {number, url} | null}.

   • Held in memory by the lifecycle, updated at finalisation, and seeded at startup from disk (latest completed_attempts.json entry + token_budget from that attempt's attempt.json), so it survives restarts.
   • Agent-side change: the published PR number/URL is written into the attempt's attempt.json at publication (today it is not persisted anywhere).
   • Shown alongside current_attempt too (the drawer always has a "last finished" line).

   recent_events

   • New in-memory ring buffer of the last 50 events, fed by JsonEventLogger.emit after redaction, so every entry has already passed through Redactor.
   • Entry: {timestamp, level, event, issue_number, phase, text}; text is detail truncated to 300 characters.
   • Excluded high-volume events already represented as figures or too noisy: limits_checked and tool_call.
   • Newest last. The buffer is process-local; it starts empty after a restart.

   Versioning and compatibility

   • schema_version is an integer major. Additive changes (new fields, new enum values for basis, phase, events) do not bump it; removing, renaming or changing the meaning of a field does.
   • Dashboard rules: ignore unknown fields; an unknown basis value is rendered as estimated; a missing field means unavailable; a null section (current_attempt, last_finished) means not applicable; a schema_version higher than supported shows the instance as "unsupported schema vN" using only the stable fields.
   • Stable across all versions: schema_version, generated_at, identity.repository, identity.agent_version.
   • An agent without the telemetry operation (version skew) is detected by agentctl telemetry failing; its error shape is Decide the agentctl telemetry command: socket operation, snapshot publication, failure isolation #225's.

   Size bound

   • Serialised snapshot capped at 256 KiB (a quarter of the 1 MiB reply limit, control_server.py:56). Over the cap the agent trims recent_events (oldest first), then context.subagents, and sets truncated: true. Typical size is a few tens of KiB.

   Freshness

   • generated_at (serve time) and sequence + current_attempt.observed_at (publication time) let the dashboard tell "agent answered" from "telemetry moving". A restart is visible as a new process_started_at (sequence resets).

   Agent-side changes surfaced (for the later split)

   1. telemetry socket operation and agentctl telemetry --json returning this shape (mechanics: Decide the agentctl telemetry command: socket operation, snapshot publication, failure isolation #225).
   2. Process start time recorded; SDK/CLI versions from provenance evidence passed to the lifecycle.
   3. Checkpoint gains issue_title and the automatic-continuation index at claim.
   4. Executor publishes an immutable telemetry view (ledger figures, thresholds incl. crossed_at timestamp, compactions, subagent running state, window via get_context_usage()).
   5. In-memory last_finished, seeded from disk; PR number/URL persisted in attempt.json.
   6. Redacted ring buffer of the last 50 events in JsonEventLogger.
   7. Snapshot size cap with trimming.

   Records

   • No ADR: the schema is versioned and additive, so it is cheap to evolve.
   • No new tickets or fog; requirements for "Decide the agentctl telemetry command" (Decide the agentctl telemetry command: socket operation, snapshot publication, failure isolation #225) are posted there.
10. lbacik

    closed this as completedon Oct 14, 2026