Turn the books you keep downloading into compounding knowledge — not one-off summaries. A 5-layer book-distillation skill for AI agents (Codex / Hermes / Claude Code / generic). MIT licensed.
You download faster than you read. Even when you read, the value evaporates within weeks. You re-encounter the same concept in three different books and never notice the connection. Your knowledge base accumulates fragments, not insight.
This skill teaches your AI agent a disciplined pipeline so that every book ingested compounds with every prior book — and after N books, you have a structured, callable knowledge asset, not a folder of summaries.
Books / PDFs / EPUBs ─┐
│ L0 → raw text + chapter outline (scratch)
│ L1 → RQ-driven focus gate (scratch)
│ L2 → 9-section reading note (your vault)
│ L3 → cross-book concept page (your vault)
│ — Convergence | Divergence | Synthesis
│ L3.5→ topic-level synthesis (your vault)
│ — Authority Matrix | New-book protocol
└ L4 → a callable skill (skills dir + vault pointer)
Each layer has explicit quality gates and the agent refuses to ship outputs that fail them (empty Synthesis, abstract praise, name-entity pollution, missing citations).
Five things most reading-note systems miss — and how this skill enforces them:
| Discipline | How it's enforced |
|---|---|
| 求同存异 (Convergence + Divergence) | Multi-source concept pages must contain a Divergence table; flattening conflicts is forbidden. |
| Differential Value | Every reading note must compare to already-read books in the cluster. Abstract praise ("a classic", "comprehensive") triggers lint failure. |
| RQ-driven focus | An L1 gate filters chapters by your research questions before deep reading. Books not aligned with your active questions go to 99_Archive/. |
| Compounding new books | A 5-tag merge protocol (SUPPORTS / REFINES / CONTRADICTS / NEW sub-aspect / NEW concept) makes the Nth book strengthen prior work rather than reset it. |
| Citations are mandatory | Every framework in a reading note must be anchored to its source chapter, or the lint rejects. |
# 1. Clone
git clone https://github.com/noahnan-max/reading-pipeline
cd reading-pipeline
# 2. Configure (once per machine)
cp config.example.yaml config.yaml
$EDITOR config.yaml
# Required fields:
# vault_path — your knowledge vault root (Obsidian / Logseq / Markdown)
# scratch_path — temp workspace (any folder outside the vault)
# research_questions — your 3-7 active research questions (defines L1 focus)
# canonical_domains — your domain taxonomy (defines frontmatter `domain:`)
# 3. Wire to your AI runtime — see INSTALL.md for details
# A. Codex: ln -sf "$(pwd)" ~/.codex/skills/reading-pipeline
# B. Hermes: ln -sf "$(pwd)" ~/.hermes/skills/reading-pipeline
# C. Claude Code: ln -sf "$(pwd)" ~/.claude/plugins/reading-pipeline
# D. Generic: point your agent's skill loader at this directoryIn any wired runtime, ask the agent:
> "Distill this book into the knowledge base: <a PDF or EPUB file>"
> "Add this book to my <topic> cluster"
> "Build a cross-book synthesis for these N books"
> "Turn this cluster into a callable skill (book2skill)"
> "Run the L1 RQ-match for <book>"
# Chinese works too:
> "拆解这本书"
> "求同存异综合一下这几本书"
> "把这个 cluster 做成 skill"
The agent reads SKILL.md, consults references/, and walks the pipeline. Each step's output goes to your vault per config.yaml.paths.
reading-pipeline/
├── README.md ← you are here
├── LICENSE ← MIT
├── CHANGELOG.md ← release history
├── CONTRIBUTING.md ← how to extend
├── INSTALL.md ← runtime-specific installation
│
├── SKILL.md ← skill entry, AI agents read this first
├── _meta.json ← optional Hermes metadata
├── config.example.yaml ← config template (copy to config.yaml)
│
├── references/ ← agent's working manuals, loaded on demand
│ ├── 00-pipeline-overview.md ← 5-layer flow, request triage, summary template
│ ├── 10-l0-extraction.md ← L0 raw extract spec
│ ├── 20-l1-question-match.md ← L1 RQ gate algorithm
│ ├── 30-l2-single-book.md ← L2 reading-note (9 mandatory sections)
│ ├── 40-l3-concept-aggregation.md ← L3 cross-book concepts
│ ├── 50-l3-5-cross-book-synthesis.md ← L3.5 topic-level synthesis
│ ├── 60-l4-skill-distillation.md ← L4 skill body + pointer card
│ ├── 70-new-book-merger.md ← incremental-update protocol
│ ├── 80-quality-gates.md ← per-layer lint + quarterly re-distillation
│ ├── 90-prompt-templates.md ← drop-in LLM prompts per layer
│ ├── 95-case-studies.md ← 3 worked clusters as gold standard
│ ├── self-test.md ← verify skill behavior
│ ├── known-issues.md ← common pitfalls (e.g. entity pollution) + fixes
│ ├── skill-bridges.md ← cooperation with docling / book-collector / etc.
│ └── source-templates-index.md ← canonical paths to templates and schemas
│
├── templates/ ← Markdown templates (copy into your vault)
│ ├── L2-single-book-reading-note.md
│ ├── L3-wiki-concept-multi-source.md
│ ├── L3.5-cross-book-synthesis.md
│ └── L4-skill-pointer.md
│
├── schema/ ← reference specs
│ ├── reading-pipeline-schema.md ← authoritative 5-layer spec (the constitution)
│ └── knowledge-classification-schema.md
│
├── examples/ ← worked examples (use as patterns)
│ └── minimal/ ← single-book minimal demo
│
└── scripts/ ← (future) lint.py / promote.py / concept-grep.sh
The skill is portable across major agent runtimes — the frontmatter is plain YAML (name: + description:) and the references are plain Markdown.
| Runtime | Status | Wiring |
|---|---|---|
| OpenAI Codex | ✅ Primary | symlink to ~/.codex/skills/reading-pipeline |
| Hermes | ✅ | symlink to ~/.hermes/skills/reading-pipeline (bypasses sync — see INSTALL.md) |
| Claude Code | ✅ | symlink to ~/.claude/plugins/reading-pipeline |
| Any agent reading SKILL.md frontmatter | ✅ | point its skill loader at this directory |
The skill writes plain Markdown with YAML frontmatter — Obsidian-friendly but not Obsidian-dependent. Use it with Logseq, Roam, or any plain-Markdown PKM tool by adjusting config.yaml.paths.
- Not a book downloader → delegate to a legal-sources skill or supply files yourself
- Not a PDF extractor → delegates to
docling/pandoc/pdftotext - Not a casual summarizer → use ChatGPT for that
- Not an auto-publisher → the vault is internal; publication is downstream
See examples/ for a minimal single-book run. A larger demonstration: three completed clusters covering strategy thinking (Porter ×2 + Rumelt + Meadows), structured thinking (Minto + 周国元 McKinsey Structured Strategic Thinking), and quant investing (Grinold/Kahn + López de Prado), with the full L2 → L3 → L3.5 lineage are referenced as patterns in references/95-case-studies.md.
Built on top of several intellectual ancestors:
- Karpathy's LLM Wiki /
Raw → Wiki → Schema— the conceptual ancestor of the layered pipeline - GAP workflow (Grasp / Arrange / Present) — the underlying methodology stage model
- Minto's Pyramid Principle — the structure for reading notes
- López de Prado's Advances in Financial Machine Learning — inspired the "leakage / overfitting prevention" discipline applied to knowledge work
PRs welcome. See CONTRIBUTING.md. Key extension points:
- New
references/<N>-<topic>.mdfor additional layer specs (e.g. L5 = teaching-back / public output) - New
templates/for non-Obsidian vault formats (Logseq, Roam, plain folders) - New
scripts/(lint / promote / citation-resolver are currently LLM-judgment only) - New
examples/of fully worked clusters in different domains
MIT — use it, fork it, adapt it. Attribution appreciated but not required.
Built originally by Noah (南道山 ｜AI 工作流实验室) for his own knowledge base, then generalized for open release. Issues, ideas, and PRs welcome.