feat: add ccs prune to losslessly shrink bloated conversations#14
Merged
Conversation
Conversation JSONL files grow large over time. `ccs prune` rewrites them removing only data that duplicates content kept elsewhere, so pruned conversations still resume with full dialogue: - toolUseResult fields (a copy of the tool_result already in message.content) - file-history-snapshot lines (rewind/checkpoint backups; pruning loses rewind history, not the conversation) User and assistant messages are never modified. Each file is rewritten to <path>.pruned and atomically renamed only if its conversation line count is unchanged, so a failed/partial prune never clobbers the original. Dry run by default - `ccs prune` only previews savings; pass --apply to actually rewrite. Dry-run on real files reclaims ~25% (e.g. a 75MB conversation -> 53MB) with no dialogue loss. CLI: `ccs prune [--apply] [--min-size=N] [--no-tool-results] [--no-snapshots] [-y]`. Tests cover the stream transform, per-category options, atomic file replace + integrity check, and dry-run leaving files untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A
ccs prunesubcommand that shrinks large conversation files by removing only data that duplicates content kept elsewhere, so pruned conversations still resume with their full dialogue.Investigating real files showed the referenced pruning blog post targets a message shape that no longer exists (no
agent_progress/bash_progresslines), and that the genuinely-duplicated data is different:toolUseResult- byte-for-byte a copy of thetool_resultalready insidemessage.content(verified: 189,154 vs 189,156 bytes on the biggest tool line). The model resumes frommessage.content, so dropping this loses nothing it reads.file-history-snapshotlines - rewind/checkpoint backups, redundant across snapshots and with the files on disk. Pruning loses rewind history, not the conversation.Across the 8 largest files these are ~25% of bytes; everything else (assistant/user messages, attachments) is real content and left untouched.
Safety
ccs pruneonly previews;--applyis required to rewrite anything.<path>.prunedand atomically renamed only if the conversation line count is unchanged - a failed or partial prune never clobbers the original.--applyconfirms before writing (-yto skip).CLI
Read-only default run on real data:
Tests
TestPruneStreamRemovesDuplicatesKeepsDialogue- snapshot dropped,toolUseResultstripped,message.content/dialogue preserved, conv-line count invariant, output stays valid JSON.TestPruneOptsRespected---no-snapshots/--no-tool-resultshonoured.TestPruneFileReplacesAndShrinks- atomic replace, file shrinks,.prunedtemp cleaned up, integrity preserved.TestPruneFileDryRunLeavesFileUnchanged- dry run reports savings but doesn't touch the file.go test -cover= 63.9% (uncovered part is therunPruneCLI driver; the data-safety core is covered).Note
CLI half only. The TUI per-conversation prune action (a key + confirmation, like Ctrl+D delete) is a planned follow-up. Resume-safety of dropping
toolUseResultis provable for the model's view but not for Claude Code's own UI/rewind, so try--applyon a low-risk conversation first.