Repository navigation
Feat/structured agent event - #6
Closed
AuceptinFang wants to merge 29 commits into
Closed
AuceptinFang wants to merge 29 commits into
AuceptinFang wants to merge 29 commits into
Conversation
Collectors now report input, cache read, cache write, and output as disjoint counters with reasoning as a breakdown of output. Derive a missing total from those four counters instead of also adding reasoning, and return output and cache counters from the token_usage summary projection so charts can split cached input from uncached input. Rewrite stored rows for both SQLite and Postgres: remove OpenAI-style cached input from input_tokens and recompute total_tokens, which older collectors set to input_tokens alone and so dropped Anthropic cache reads and writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Viewing a large session loaded every accumulated request snapshot at once
(66 MB for a 1,590-event session). Add three read paths so clients can load
only what they show:
- GET /v1/agent-usage/events/{event_pk} returns one event verbatim.
- GET /v1/agent-usage/sessions/{session_pk}/outline returns the timeline
without text or metadata bodies, plus text length, a short preview, and
attachment counts (about 1 MB for the same session).
- GET /v1/agent-usage/sessions/{session_pk}/trajectory merges the timeline
so each request and response contributes only its new content: new
request paragraphs, response text, and tool calls paired with their
first-seen results. Side requests such as classifiers form their own
streams.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Store disjoint input, cache read, cache write, and output token counters alongside the session outline and trajectory endpoints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add GET /v1/agent-usage/sessions, returning one page of owner-scoped sessions ordered by their latest event, with turn, event, and token counts plus the device host and platform. Sessions can be filtered by an activity window and Agent name, with Claude aliases matched together. Implemented for both the SQLite and PostgreSQL backends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep sessions with at least one event from the given LLM provider or model, so the session index can answer which sessions used a provider or model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude Code and Codex ask a model to name each session and return
{"title": ...}. Recognize those calls at ingestion by their official naming
instruction plus a response made only of agreeing title objects, store the
result in session_titles, and expose title/purpose on session info and the
session list. Codex naming threads are marked purpose=title_generation.
A one-time backfill covers events stored before capture existed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex compacts its context by sending a normal request that ends with a fixed "CONTEXT CHECKPOINT COMPACTION" prompt, and replays the answer at the start of the compacted history behind a fixed "Another language model started to solve this problem" sentence. The trajectory showed the prompt as user input, the summary as an assistant reply, and the summary again as user input. Recognize both fixed texts in the legacy and the snapshot projection. The prompt and the summary from its response form one compaction item, and the replayed handoff is hidden. When the summary request was not observed, the handoff alone records the compaction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…entity and order" This reverts commit 9904c12.
Codex names a task from a separate thread whose requests carry no link to the task (verified on 169 captured naming requests across Codex 0.154-0.159). The naming request quotes the task's user message, and Codex starts the naming turn right after the task turn. A task session is matched when its turn ID (UUIDv7) starts within 60 s of the naming turn on the same device and its latest user message opens with the quoted prompt. Only a single candidate is matched; a later second candidate withdraws the match. The title is then recorded for the task session, and the naming thread keeps a link to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…models Query endpoints accept repeated agent_name, session_id, llm_provider, and llm_model keys; values within one filter match any, filters still combine. Single-value calls are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SQLite summary loaded every matching event row, including text and metadata that hold whole request bodies, parsed the metadata JSON, and grouped in memory. A week of usage meant reading over 2 GB to add up token counts. It now groups in one query that mirrors the PostgreSQL aggregate, reading only the token, grouping, session, and turn columns, with the time bounds always bound so the owner-time index limits the scan. Summaries now cover every event in range, as on PostgreSQL, instead of only the newest 100,000. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…exes Token columns sit after the event text in each row, so reading them walks the text's overflow pages. Two indexes carry every column the usage summary and the session list read, and the session list counts an indexed column, so both queries no longer touch event rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # crates/abyss-backend/src/storage/sqlite/repository.rs
Event text and metadata hold whole request bodies, so ten events came to about 8 MB while an event list shows a few hundred bytes of each. With fields=list the events endpoint returns what a list shows, and neither storage backend reads text, its digest, or metadata, nor loads attachments. Without the parameter the response is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stream was named by every system message in a request. Claude Code puts a billing header with a per-request hash and previous-request id at the top of its system prompt, a context budget at its end, and system messages inside the conversation that grow with it, so nearly every request opened a new stream and replayed the whole conversation. A 480-turn session came to 1,255 streams and 349,000 items, 227 MB, with one user message 2,167 times. Streams are now named by the system prompt the conversation opens with, without the billing header and context budget lines, and those lines are also left out when telling system prompts apart. The same session has 6 streams and 6,837 items, 10 MB, and keeps every distinct item it had. The archived bodies keep both lines, so billing can still be counted from them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atches A search for a common word took 10 to 50 seconds on a real database while the full-text match itself took milliseconds. Every step after the match paid for every hit: fragments were built for all hits of the page's sessions before keeping three per session, every hit was loaded to dedupe sessions in Rust, three queries re-ran the match, reading session keys through the FTS table loaded each matching document, and ordering by observation time read through event text that sits before it in the row. One set of expressions now ranks hits by rowid and bm25 alone, reads keys from the FTS content table, and takes observation times from the owner-time covering index when hits are many or from event rows when few, applying the time window after that join. Window functions give each session's match count and strongest event, so sessions are ordered and paged in SQL with the total in the same query, and fragments are built only for the matches returned. On the user's 41,000 events, "entry" goes from 13.8 to 0.24 seconds and "the" from 36.9 to 0.79, with identical responses across a 21-query benchmark, which is added under scripts/bench. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… bodies Titles and trajectories read the collector's role-aware `metadata.conversation`; the provider-body decoders are removed, so the backend never parses raw request or response bodies. Turns are numbered by the backend from the traffic's turn identity (`codex_turn_id`, `claude_turn_id`, `turn_id`, then provider ids); the sender's `turn_index` is ignored. A Claude Code event without a turn id joins the turn of a same-session `cc_prompt_id`, else the latest turn started at or before it. Claude `thread` continue requests join the stream of the response they continue. Session titles come from collector fields only: a `session_title` names its own session and outranks the first-user-input default, which never replaces it; `purpose` requests are not user input; Codex `thread_source == "thread_title"` sessions are marked `title_generation`. The Codex link heuristic, text-based naming recognition and their backfills are removed. Stored titles and links are kept; no migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI installs the latest stable toolchain, now 1.99.0. Its `redundant_field_names` fires on code generated by diesel_derives 2.3.9 for `QueryableByName`; diesel 2.3.14 / diesel_derives 2.3.10 no longer trigger it (lockfile only). Empty-collection test asserts now use `assert_eq!` for the new `assert_is_empty` lint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
adapt to the changes of abyss-rs