feat(caveman-compress): deterministic phrase-lookup pre-pass (216 entries)

[drewg2009]

Summary

• Adds a fixed, deterministic phrase-to-word lookup table (scripts/phrase_map.py, 216 entries) that runs before the Claude compression call in caveman-compress, collapsing multi-word phrases like "due to the fact that" → "because" and "please make sure that" → "confirm" for free — no model call needed for the part a dictionary already knows.
• Skips fenced code blocks and inline code spans so code samples/comments are never touched.
• Word-boundary-anchored and runs to a fixed point (re-applies until a pass makes no further change), so a replacement that creates a new matchable phrase gets caught too.
• Only targets phrases that collapse to fewer tokens, not single-word synonyms ("extensive" → "big" costs the same either way, so it's not in the table).

Why

compress.py currently only compresses via an LLM call — every phrase gets rewritten by Claude even when the correct rewrite is a fixed, unambiguous fact of English ("due to the fact that" always means "because"). Pre-collapsing those phrases with a lookup table shrinks the prompt Claude receives, for zero extra tokens or latency.

Sourcing and filtering

Entries were pulled from plain-language style guides across four domains — general plain-language (plainlanguage.gov, Federal Plain Language Guidelines), legal/government, academic/scientific writing, and technical documentation (Microsoft's and Google's own developer-docs style guides) — then hand-filtered against one hard rule: a phrase only qualifies if swapping it in verbatim, at any position in any sentence, produces grammatically valid text.

A meaningful chunk of raw source-list entries failed this and were deliberately excluded (see the module docstring for the full list and reasoning):

• "perform an analysis of X" → "analyze of X" — dangling preposition, broken.
• "have a discussion about X" → "discuss about X" — discuss doesn't take about.
• "table this" → "postpone" — means the opposite in British English (real regional ambiguity).
• "as prescribed by" → "under" — breaks on the common "as prescribed by the doctor" (person) sense vs. the statute sense.
• "will allow you to" → "lets you" — collapses future tense to present, which could misdescribe an unshipped feature as already live.
• "it was demonstrated that" → "evidently" — weakens a scientific claim's epistemic strength, not just wordiness.

Two bugs found and fixed while vetting the expanded set

1. No word-boundary anchoring. A short entry like "point in time" could match mid-word inside unrelated text — e.g. the literal substring "point in time" occurs inside "checkpoint in time-series data", which an unanchored regex would mangle. Added \b anchors on both ends.
2. Double-preposition bug. "close proximity to" → "near" broke on the common "in close proximity to X" phrasing, producing "in near X". Fixed by adding the longer "in close proximity to" variant, which wins via longest-first matching.

Honest measurement (no fabricated numbers — see tests/test_phrase_map.py::PhraseMapTokenReductionTests)

Measured with tiktoken (o200k_base, same tokenizer evals/measure.py uses):

Input	Tokens before → after	Cut
This repo's own tests/caveman-compress/*.original.md fixtures (terse engineering notes)	6198 → 6197	~0.02%
README-style instructional prose ("please make sure that...", "this allows you to...")	78 → 58	~26%
Deliberately wordy/corporate-style prose	60 → 41	~32%

This repo's own fixtures barely move — that style of prose doesn't use the phrases in the table. It helps most on verbose writing (meeting notes, policy docs, over-explained setup instructions, corporate email pasted into a memory file) and does close to nothing on prose that's already terse. Documented this conditional impact directly in SKILL.md and README.md rather than overstating it.

Changes

• skills/caveman-compress/scripts/phrase_map.py — new module, apply_phrase_map(), 216-entry PHRASE_MAP, word-boundary-anchored, fence-aware, case-preserving, runs to a fixed point.
• skills/caveman-compress/scripts/compress.py — calls apply_phrase_map() on the body before building the Claude prompt.
• skills/caveman-compress/SKILL.md / README.md — sections explaining the pre-pass, sourcing, the two bug fixes, and the measured (conditional) impact.
• tests/test_phrase_map.py — 17 tests: fence/code-span safety, case preservation, longest-phrase-wins ordering, word-boundary mid-word false-match prevention, the double-preposition regression, recursive-pass convergence (proven with a synthetic map), and tiktoken-based reduction tests (skip cleanly if tiktoken isn't installed, matching the existing optional-dependency pattern in evals/).

Test plan

• python3 -m unittest tests.test_phrase_map -v — 15/17 pass without tiktoken installed (2 skip)
• uv run --with tiktoken python3 -m unittest tests.test_phrase_map -v — 17/17 pass including token-count assertions
• python3 -m unittest discover -s tests -p "test_*.py" — full suite, 73 tests, no regressions
• Manually verified fence/inline-code exclusion, case preservation, and the double-preposition fix
• Scanned PHRASE_MAP for duplicate keys (none — 216 unique)