Skip to content

feat(renderer): chunk oversized topic rule files into index + per-section files#94

Merged
gbrbks merged 3 commits into
mainfrom
feature/chunked-rule-files
Jun 10, 2026
Merged

feat(renderer): chunk oversized topic rule files into index + per-section files#94
gbrbks merged 3 commits into
mainfrom
feature/chunked-rule-files

Conversation

@csacsi

@csacsi csacsi commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Topic rule files over 8 KB no longer render as monoliths. They split into an index + per-section chunk files, so agents route with ~1 KB instead of swallowing 20–50 KB.

How rules look now

.claude/rules/
  patterns.md              ← 1.7 KB routing INDEX (same path AGENTS.md links to)
  patterns/
    communication-patterns.md   ← 2–6 KB chunk per H2 section
    key-decisions.md
    trade-offs-accepted.md
    ...
  pitfalls.md              ← small topics stay single-file
  enforcement/             ← unchanged (already chunked: index + by-topic/)

The index is a routing table the agent reads first:

Section File ~Tokens Contains
Communication Patterns patterns/communication-patterns.md ~1643 8 entries: Singleton service access, Firebase RTDB → Rx wrapper, …
Key Decisions patterns/key-decisions.md ~1493 6 entries: Firebase RTDB as the only backend, …
  • Chunking splits at H2 sections; bodies wrapped in a single H2 (dev-rules, infrastructure) fall back to H3-category splitting
  • Frontmatter (paths: globs) is preserved on every chunk
  • cleanup_stale_rule_files() removes the retired enforcement.md monolith, orphaned chunks after section renames, and chunk dirs for topics that de-chunk — wired into renderer.main() and finalize.py
  • AGENTS.md explains the chunked-index routing to agents; ARCHITECTURE.md documents the mechanism
  • .archie-bench/ gitignored (local benchmark artifacts)

Real-world validation (fresh deep scan on Gasztroterkepek.iOS)

Surface Before After
Default-loaded .claude/rules/*.md ~225 KB monoliths ~22 KB indexes, largest file 6 KB
patterns.md 49 KB 1.7 KB index → 8 chunks
architecture.md 27 KB 0.7 KB index → 3 chunks
AGENTS.md 21 KB 9.1 KB
Stale enforcement.md (70 KB pre-2.5 leftover) present auto-removed

0 validation failures; full content remains available on demand.

Tests

  • 8 new tests in tests/test_renderer_chunking.py: threshold behavior, fence-aware H2 split, H3 fallback, slug collisions, index summaries, stale cleanup (incl. foreign-file safety)
  • Full suite: 847 passed, 1 skipped; verify_sync.py green; npm assets synced

🤖 Generated with Claude Code

…tion files

Topic files over 8 KB now render as a small routing index at the
existing .claude/rules/<topic>.md path (section, file, ~tokens,
contents table) with per-H2-section chunk files under
.claude/rules/<topic>/. Bodies wrapped in a single H2 fall back to
H3-category splitting (dev-rules, infrastructure). Real-world effect
(Gasztroterkepek.iOS blueprint): default-loaded rule surface drops
from ~225 KB of monoliths to ~15 KB of indexes; full content stays
available on demand.

Also adds cleanup_stale_rule_files(): removes the retired
enforcement.md monolith, stale chunks after section renames, and
chunk dirs for topics that de-chunk — wired into renderer.main()
and finalize.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
archie Ready Ready Preview, Comment Jun 10, 2026 12:53pm
archie-viewer Ready Ready Preview, Comment Jun 10, 2026 12:53pm

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants