Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.0.19] - 2026-05-25

### Fixed

- **`claude_code` adapter — assistant turns were being silently dropped from `agent*_traj.json`.** The session-JSONL parser (`parse_session_jsonl` in `src/cooperbench/agents/claude_code/parsers.py`) treated `message.role` as authoritative and rejected any event whose role wasn't in `{user, assistant, system}`. Recent claude-code session writers emit assistant turns with `message.role: None` — the role lives only in the top-level `event.type` — so every LLM turn (text, thinking, tool_use) got filtered out, leaving traj files that contained only `user` tool_result entries. Affected every `claude_code` run since the session-format shift (including all 0.0.17 / 0.0.18 trajectories on disk). Now falls back to `event.type` when `message.role` is missing; on a representative session (`anyhow_task/390/f1_f4/agent2_session.jsonl`) this recovers all 86 assistant events, taking the parsed trajectory from 43 messages (all user) to 129 (43 user + 86 assistant). The underlying `*_session.jsonl` and `*_stream.jsonl` files were always complete — only the derived `*_traj.json` was wrong, so historical runs can be re-parsed by calling `parse_session_jsonl` on the on-disk session file.

## [0.0.18] - 2026-05-25

### Removed
Expand Down
2 changes: 1 addition & 1 deletion src/cooperbench/__about__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""Version information for CooperBench."""

__version__ = "0.0.18"
__version__ = "0.0.19"
8 changes: 7 additions & 1 deletion src/cooperbench/agents/claude_code/parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,12 @@ def parse_session_jsonl(text: str) -> list[dict[str, str]]:

Returns a list of ``{"role": ..., "content": ...}`` dicts, sorted by
``timestamp``. ``content`` is always a string.

Role resolution: prefer ``message.role`` when present, otherwise fall
back to ``event.type``. Recent claude-code session writers emit
assistant turns with ``message.role: None`` (the role is only in the
top-level ``type`` field), so a strict role-validation check would
silently drop every LLM turn.
"""
events: list[dict[str, Any]] = list(_iter_json_lines(text))
events.sort(key=lambda e: e.get("timestamp") or "")
Expand All @@ -145,7 +151,7 @@ def parse_session_jsonl(text: str) -> list[dict[str, str]]:
message = event.get("message")
if not isinstance(message, dict):
continue
role = message.get("role")
role = message.get("role") or event.get("type")
if role not in {"user", "assistant", "system"}:
continue
content_text = _content_blocks_to_text(message.get("content"))
Expand Down
Loading