Repository navigation
feat!: derive session streams and turns from conversation entries - #12
Closed
AuceptinFang wants to merge 14 commits into
Closed
AuceptinFang wants to merge 14 commits into
AuceptinFang wants to merge 14 commits into
Conversation
- Conversation reads `entries` (role, text, call_id, name, item_id) in place of the retired `messages`; default titles take the first user entry. - The trajectory shows every entry each event carries, in order: messages with their reported role, tool calls and results from their entries with their call id. Requests no longer hide history their stream has seen, and tool results are neither de-duplicated nor attached to their call. - Trajectory tests become two case tables plus a few focused tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New `llm_usage_event_stream_facts` table, one row per new event, written in the ingest transaction; the event table keeps its columns. - A request keeps where its new part starts, whether it opens a turn, the last identity of its last run and the roles of the model-side entries after it, and, without model output, the call ids of its tool results. A response keeps the identities it emitted and its output skeleton. - Facts come only from a completely decoded conversation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add GET /v1/agent-usage/sessions/{session_pk}/streams. A query groups the
session's events into exchanges, finds each request's base (pointer, last
output, results only) or why it has none, and cuts segments at roots,
turns, and branches. It returns structure only: ids, bases, rules, new
part offsets, and segments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The session page reads one layer per request, by session_pk:
- GET /sessions/{session_pk}/header: agents, time span, event, exchange,
turn and segment counts, token totals, providers and models.
- GET /sessions/{session_pk}/streams: segments only, each identified by its
first exchange (head), with its kind, parent head, the exchange it forks
from, time span and exchange count.
- GET /sessions/{session_pk}/segments/{head}: one page of a segment's
exchanges from head on (limit, after), each with its base, rule or reason,
new part, turn start, times, model, tokens, and a preview of its request
and response.
- GET /events/{event_pk}/conversation: an event's entries through the same
window as a preview (entries runs, text_preview).
Previews give the asked runs of entries as reported, cut texts carrying
their full UTF-16 length, and count what was left out.
streams.rs splits into facts, base, segmentation and test fixtures; the
code moves without change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ents Events carry their content only in conversation entries: - Ingest takes no text; responses drop text and text_sha256; /outline drops text_chars and preview. The text columns stay in the schema and are no longer written or read. - Search indexes every entry's text in order and the names of called tools; tool_content stays empty. - Trajectory drops its text fallback; a compaction summary is the response's assistant entries. Each request's stream facts keep a label: the last user entry with text in its new part, as its index, its first 50 UTF-16 code units, and its whole length, in three columns of the stream facts migration. /streams gives each segment the label of its first request, and the session header's title is the label of the session's first request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /sessions/{session_pk}/segments/{head} returns the exchanges from head
to the segment's end at once, as the session page design asks; limit,
after, and next go away.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…SQLite
Search follows the confirmed search semantics: every event carrying the
query matches, requests and responses alike, inherited context included,
and each hit says whether it falls in what the event added.
- Pieces are the text and the name of every conversation entry. Each
distinct piece is stored once per owner (search_texts) and every piece
of an event refers to it (search_refs); a contentless trigram index over
the folded texts narrows candidates that the backend then verifies.
usage_events_fts goes away. The tables join the stream facts migration;
ingest writes the index in the event's transaction and startup
backfills events stored before it.
- Matching is a case-insensitive literal substring under a simple
per-character fold, shared through search::matching.
- Session search adds new_match_count and each match's new; matches and
fragments prefer hits in what an event added.
- GET /sessions/{session_pk}/search gives the hits within one session with
their UTF-16 ranges and the head of each hit's segment.
- A request's new part starts at its stored new_from; events without
stream facts count from 0. PostgreSQL keeps its Elasticsearch search
and answers in-session search with 503.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Turns no longer come from a collector-declared turn_id:
- agent_turns, the events' turn_pk and turn_index, and ingest's turn
placement go away, as do the turn_id and naming_thread ingest fields
and the timeline, outline, and trajectory session endpoints built on
them. The schema changes are folded into the initial schema.
- Turn counts in the session list, the summary, and the session header
count the requests whose stream facts open a turn.
- A session title is the label of its earliest request, read from that
request's stream facts; the list, the header, and search return the
same EntryLabel. The first-user-input title and agent-reported titles
go away, and the facts migration rebuilds the title anchors.
- Session structures are cached by a per-session structure revision that
ingest advances. GET /events/{event_pk}/location, a single event, and
/events?locate=true give an event's current segment, its one-based
position, its kind, and its exchange.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /summary takes an optional tz, an IANA zone name, that decides which local day the day dimension groups by; without it days stay UTC, and an unknown zone is rejected. from and to remain instants. PostgreSQL groups AT TIME ZONE; SQLite, without a zone database, groups by the zone's UTC offsets over the queried range, computed with chrono-tz. Ported from feat/local-day-usage d1f2d43 onto the current summary query. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ingest stores the subagent_id, parent_subagent_id, and root_turn_id an event declares in its stream facts. A sub-agent's root segment attaches to a segment of its parent stream: the first one holding its declared root turn, else the earliest one whose time range holds its start. Attachment only decides where a segment shows; bases and segments are unchanged. /streams gives each segment subagent_id, attached_to, and attached_by, and /segments gives each exchange its declared ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stream facts keep the turn_id ingest field (the facts migration's
root_turn_id column becomes turn_id). Each exchange's turn is derived with
the session structure:
- In a session where any exchange declares a turn, a declared exchange
belongs to that turn. An undeclared one inherits its base's turn; with
no base it joins the turn that last began, and before any turn it
belongs to none.
- In a session without declarations, an exchange that starts a turn opens
one named by its exchange id; the rest follow as above.
- Turns are numbered from one by first appearance.
A Turn segment is cut where an exchange's turn differs from its base's, so
a segment holds one turn. Segments and exchanges carry turn_id and turn,
and an event's location gives its turn in place of segment_index. The
session list, the summary, and the header count distinct declared turns,
or turn-starting requests where nothing is declared; sub-agent attachment
compares turn_id.
GET /v1/agent-usage/sessions/{session_pk}/turns gives one node per turn:
its executors grouped by sub-agent, the main line first, each split into
the root-to-leaf path of its segments with the most exchanges and the
other segments; and whether the main line begins without a base.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the token comparison queries (#10) and the 1.0.1 and 1.1.0 releases. The response-id fallback turns of #8 are not taken: turns are derived from the stream facts and the declared turn_id, so agent_turns, ingest's turn placement, and scripts/tests/turns_api.py stay retired. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clippy 1.99 adds assert_is_empty, and CI installs the latest stable, so the nine assert!(x.is_empty()) checks failed both Clippy runs under -D warnings. They now compare against an empty value, which also shows the value when a check fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trailing is a reserved word in PostgreSQL, so the event stream facts migration failed with a syntax error and the backend could not start on Postgres. Diesel already quotes the column in its queries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reads the new
AgentEventcontract (one conversation entry per protocol block, declaredsubagent_id/parent_subagent_id/turn_id,omitted,headers) and serves the session page from structure derived at query time instead of turn rows written at ingest.llm_usage_event_stream_factsrow: where the request's new part begins, whether it opens a turn, its last model-output identity, the identities a response emits, its label, and the ids the client declared. Nothing reads event bodies again after ingest.turn_idwhen the session declares any; otherwise a request opens a turn when its new part has user text and no tool result, and the rest follow their base. Session structures are cached per session revision.GET .../sessions/{pk}/header,/streams,/turns,/segments/{head},/search,GET .../events/{pk}/conversation,/events/{pk}/location.main: keeps the token comparison queries (feat: add token comparison queries #10) and the 1.0.1 / 1.1.0 releases. The response-id fallback turns of fix: fall back to response IDs for turns #8 are not taken: turns come from stream facts and the declaredturn_id, soagent_turnsstays retired.Breaking changes
agent_turns, events'turn_pk/turn_index, ingest-time turn placement, thenaming_threadingest field, andGET .../sessions/{pk},/outline,/trajectory,/timeline.metadata.conversation.entries. Events from collectors before this contract still ingest (all new fields are optional), but carry no entries, so their sessions show usage without conversation or structure.