Skip to content

fix(claude): tolerate duplicate snake/camel keys in session JSONL - #47

Merged
tuchg merged 1 commit into
mainfrom
fix/claude-jsonl-duplicate-keys
Aug 2, 2026
Merged

tuchg merged 1 commit into
mainfrom
fix/claude-jsonl-duplicate-keys

Conversation

@tuchg

@tuchg tuchg commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Fixes #45
Fixes #46

Problem

Claude Code v2.x (v2.1.206, v2.1.208, …) writes both session_id and sessionId (and other snake/camel pairs) on the same JSONL line. serde's generated struct visitors reject the second occurrence as a duplicate field, so every assistant/user line fails to parse — Lucarne sees no Claude activity at all (probe, history, and watch all fail with duplicate field session_id).

Fix

Wrap all claude JSONL parsing (probe, history body, watch meta, plus nested content/usage/progress parses) in a DupTolerant serde Deserializer:

  • Buffers each object as borrowed (key, raw value) pairs — zero-copy, no allocation for canonical keys
  • Folds camelCase spellings onto their snake_case field via a generic camel→snake canonicalization (works for every current alias and future ones, including fields with no declared alias)
  • Drops repeats, last occurrence wins (same semantics as serde_json::Value)
  • Replays the deduplicated object; nested structs (e.g. progress payloads) are covered recursively

All logic stays inside the claude provider — no provider ids, cursor formats, or parse rules moved into shared layers (per AGENTS.md boundaries).

Tests

  • parse_claude_json_str_tolerates_duplicate_keys_last_wins — dual session_id/sessionId + message_id/messageId lines, asserts last-wins
  • probe_session_meta_accepts_dual_key_lines — probe path on the exact bug shape
  • reader_accepts_dual_key_lines_and_nested_progress — watch/history path with dual keys + nested fullOutput/full_output progress payload
  • All existing 219 agent-sessions tests pass; clippy clean for the touched file

Tradeoff

claude_parse bench: 37ms → 60ms per full session (~1.6x). Tolerance requires a second pass over the line — streaming serde deserializers can't rewind to dedup in one pass. Watch deltas are single lines (µs-scale), so the practical impact is negligible.

Claude Code v2.x writes both session_id and sessionId (and other
snake/camel pairs) on the same JSONL line; serde's generated struct
visitors reject the second occurrence, so every assistant line failed
to parse and Claude activity was invisible to Lucarne.

Wrap claude JSONL parsing (probe, history, watch) in a DupTolerant
deserializer that buffers each object as borrowed (key, raw value)
pairs, folds camelCase spellings onto their snake_case field, drops
repeats (last occurrence wins, like serde_json::Value), and replays
the deduplicated object. Zero-copy borrows into the input line are
preserved; nested payloads (e.g. progress) are covered recursively.

Benchmark: claude_parse 37ms -> 60ms per full session (~1.6x); the
tolerance requires a second pass over the line, inherent to
streaming serde deserialization.
@tuchg
tuchg merged commit 0200a63 into main Aug 2, 2026
6 checks passed
@tuchg
tuchg deleted the fix/claude-jsonl-duplicate-keys branch August 2, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant