Skip to content

Report the last finished attempt in telemetry #243

Closed

Description

@lbacik

Parent

Agent-side change from the map Agent fleet dashboard: map; the consolidated list is in the resolution of Define the dashboard project. The snapshot shape is decided in Define the telemetry snapshot contract and the command mechanics in Decide the agentctl telemetry command. Terms: Telemetry snapshot and Figure basis in CONTEXT.md; ADR 0006.

What to build

Between attempts and alongside a running one, telemetry reports the last finished attempt: issue, title, outcome, tokens, turns, finish time and pull request. The value survives agent restarts.

Acceptance criteria

  • The published pull request's number and URL are written into the attempt's archive record at publication.
  • The telemetry module gains finish_attempt(summary), called at finalisation, which updates last_finished in memory.
  • At startup last_finished is seeded from disk: the latest completed attempt, with its token budget and pull request from that attempt's archive record. With no completed attempt it is null.
  • Shape: {attempt_id, issue_number, issue_title, outcome, finished_at, budget_used, turns, pull_request: {number, url} | null}. budget_used and turns are figure objects (measured). Archives written before this change give pull_request: null and an unknown title.
  • Tests: finishing an attempt updates last_finished; a restart seeds it from disk; with no history it is null; it is reported alongside an active current_attempt.

Blocked by

Activity

  1. added
    ready-for-agentFully specified implementation issue ready for an unattended agent
    on Oct 14, 2026
  2. self-assigned this
    on Oct 14, 2026
  3. lbacik commented on Oct 14, 2026

    @lbacik
    OwnerAuthor

    Agent Attempt Result: handoff

    Branch: agent/issue-243
    PR: none
    Handoff note commit: 12ee76b11939d0949eb71e0416b68376bb91ed17

    Evidence

    • Check command: UV_PROJECT_ENVIRONMENT=/home/agent/venv-py311 uv run --python 3.11 --extra dev ruff check && UV_PROJECT_ENVIRONMENT=/home/agent/venv-py311 uv run --python 3.11 --extra dev pytest && UV_PROJECT_ENVIRONMENT=/home/agent/venv-py313 uv run --python 3.13 --extra dev pytest → exit code None
    • Reviews performed: 1
    • Review repair cycles: 0/2
    • Review findings: not collected

    Details

    Handoff note: issue #243

    • issue: 243
    • started_at: 2026-10-14T13:48:04.076479Z
    • reason: token_soft_threshold
    • last_work_commit: 24d85f3

    Summary

    Implemented the last-finished attempt telemetry end to end on branch
    agent/issue-243 (work uncommitted; sealing will commit it).

    • simple_coding_agent/telemetry.py: finish_attempt(summary=None) normalizes
      and stores last_finished (attempt_id, issue_number, issue_title,
      outcome, finished_at, budget_used/turns as measured figures,
      pull_request: {number, url} | null); legacy summaries seed title
      "unknown" and pull_request: null; malformed summaries log once and keep
      the previous view. snapshot() now serves last_finished. Added
      load_last_finished(data_dir) built from the latest completed attempt
      archive record.
    • simple_coding_agent/observability.py: added read_latest_completed_attempt
      scanning data_dir/logs/<issue>/<started>/attempt.json (skips
      unreadable/incomplete records; latest by (completed_at, started_at)).
    • simple_coding_agent/lifecycle.py: new data_dir constructor param with
      startup seeding (_seed_telemetry_last_finished, never raises);
      _telemetry_finish(summary) passes summaries through;
      _archive_pull_request writes the PR into the archive at publication
      (hooked into _publish_bounded, _publish_terminal, recovery retry);
      _commit_finalization writes issue number/title into the outcome record,
      backfills the PR safety net, and finishes telemetry with the full summary;
      _write_outcome now returns completed_at.
    • tests/fakes.py: FakePublisher/PublishedAttempt accept pull_request.
    • simple_coding_agent/__main__.py: passes data_dir to AgentLifecycle.
    • New tests/test_telemetry_last_finished.py (15 tests): reporter shape,
      legacy/unknown handling, unavailable figures, malformed-summary behavior,
      finalisation wiring + archive PR, restart seeding, null with no history,
      latest-wins seeding, alongside-active reporting.
    • code-review skill invoked this run: Standards (0 hard violations,
      4 smells, worst = duplicated PR normalizer across telemetry/lifecycle) and
      Spec (0 missing, 0 creep, 3 wrong-looking, worst = shared
      _build_failed_logged flag) reports aggregated in the run transcript.

    Remaining work

    1. Re-run tests/test_telemetry_last_finished.py (was 14/15 green; fixed the
      malformed-summary log-count expectation from 2 to 1 to match the
      log-once-per-episode pattern; re-run was blocked by the budget hook).
    2. Decide on review findings: extract one shared PR normalizer (strongest
      Standards finding); consider separate warning flags for snapshot vs finish
      failures, timestamp parsing instead of string ordering in the archive scan,
      and narrowing the except TypeError: finish() shim.
    3. Run ruff check, full pytest on both gate Pythons (3.12 and 3.13).
    4. Small docs touch: docs/agents/telemetry-snapshot.md last_finished
      bullet (measured basis, legacy "unknown" title) and
      examples/telemetry/snapshot-idle-with-last-finished.json (budget_used
      basis estimated -> measured to match the issue spec).
    5. Commit the work on agent/issue-243.

    Evidence

    • tests/test_telemetry_last_finished.py (new, untracked): 14/15 passing;
      the single failure was the test's own log-count expectation, corrected to 1
      (mirrors test_malformed_seed_keeps_the_previous_view_and_logs_once).
    • Touched files (uncommitted): simple_coding_agent/__main__.py,
      simple_coding_agent/lifecycle.py, simple_coding_agent/observability.py,
      simple_coding_agent/telemetry.py, tests/fakes.py, plus the new test file.
    • Gate not yet run: ruff + full pytest suites still pending.
    • Code-review gate satisfied: code-review invoked with both axes reported.

    Automatic continuation 0/2 used; the agent will continue automatically.

  4. added
    round-finishedAgent round finished; requires human evaluation to continue
    and removed
    ready-for-agentFully specified implementation issue ready for an unattended agent
    on Oct 14, 2026
  5. assigned and unassigned on Oct 14, 2026
  6. removed
    round-finishedAgent round finished; requires human evaluation to continue
    on Oct 14, 2026
  7. lbacik commented on Oct 14, 2026

    @lbacik
    OwnerAuthor

    Agent Attempt Result: complete

    Branch: agent/issue-243
    PR: #253
    Merge: auto-merge enabled (merge)

    Evidence

    • Check command: UV_PROJECT_ENVIRONMENT=/home/agent/venv-py311 uv run --python 3.11 --extra dev ruff check && UV_PROJECT_ENVIRONMENT=/home/agent/venv-py311 uv run --python 3.11 --extra dev pytest && UV_PROJECT_ENVIRONMENT=/home/agent/venv-py313 uv run --python 3.13 --extra dev pytest → exit code 0
    • Reviews performed: 1
    • Review repair cycles: 0/2
    • Review findings: not collected

    Details

    Claude SDK completed the implementation workflow.

  8. removed their assignment
    on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions