Compact technical books, docs, and references into agent-optimized markdown. Template-driven AI compaction with incremental updates. Works with Claude, GPT, and Codex.
Template-driven compaction that produces consistent, structured output every time
Every feature exists to make agent context production repeatable and maintainable
Four built-in templates (rules, principles, patterns, raw) enforce consistent output structure. Custom templates supported via markdown files.
Read from PDFs, markdown files, Notion pages, web URLs, EPUBs, and GitHub repos. All normalized into uniform chunks.
SHA-256 tracks source content, template, and agent identity. Changed sources are re-compacted automatically. Unchanged sources are skipped.
Claude CLI, Codex CLI, Anthropic API, and OpenAI API. Same interface, swap with --agent. Claude CLI is zero-config default.
Auto-generated index.md files at every level. Agents know what to load and when — no token waste from loading everything.
distill update skips clean sources. Improve a template? All sources using it are automatically re-compacted.
Split books by chapter with split_by: chapter. Each chapter becomes a separate output file with its own entry in the index.
Configure priority order for subdirectories. Org-specific principles override framework guidelines, which override reference books.
distill publish pushes to a context repo. distill install clones it into ~/.claude/docs/. Full producer → consumer pipeline.
Written in Go with Cobra. Cross-compiles to macOS and Linux. Install via go install, Homebrew, or GitHub Release.
Three steps from source material to agent context
Point distill at a PDF, markdown file, Notion page, or URL. Choose a compaction template (rules, principles, patterns, or raw).
Run distill <name>. The AI reads the source, applies the template, and writes agent-optimized markdown to your output directory.
Output plugs directly into ~/.claude/docs/, Cursor rules, or any agent that reads markdown. Hierarchical indexes guide selective loading.
See distill in action — CLI commands, configuration, and output format
# Add a source and compact it
$ distill add pdf ~/Books/tao-of-react.pdf \
--name tao-of-react --template rules --output-dir tao
✓ Added source "tao-of-react" (pdf)
$ distill tao-of-react
→ source: tao-of-react (pdf)
→ template: rules
→ agent: claude-cli (sonnet)
→ Compacting (1 chunk)...
✓ Compaction complete.
→ tokens: 2847 (budget: 4000)
→ output: ./output/tao/tao-of-react-minified.md
# Re-compact everything after improving a template
$ distill update --force
→ Re-compacting 4 sources...
✓ 4 sources updated.
# List tracked sources
$ distill list
tao-of-react pdf rules tao/ 2847 tok ✓ current
ddia pdf principles ddia/ - ✗ not yet
# Install a shared context repo for agents
$ distill install https://github.com/myteam/context.git
✓ Context repo installed at ~/.claude/docs/distill turns any technical knowledge into agent-ready context
Compact Tao of React, Tao of Node, or any framework guide into numbered rules your agents follow when writing code.
Distill DDIA, Clean Architecture, or domain-driven design books into chapter-based principles for data and system design decisions.
Turn your team's coding conventions doc into agent-readable rules. Agents produce code that matches your patterns.
Publish compacted docs to a shared git repo. Every teammate's agent loads the same rules — consistent AI behavior across the team.
When source material updates or you improve a template, distill update re-compacts automatically. Context stays current.
Extract named patterns (Problem/Solution/Rationale) from pattern catalogs. Agents apply the right pattern for the right situation.
Install distill and compact your first source in under a minute
go install github.com/smeltery/distill@latest# Initialize config
distill config init
# Add a source
distill add pdf ~/Books/tao-of-react.pdf \
--name tao-of-react --template rules
# Compact it
distill tao-of-react
# Check agents
distill agentsTip: The default agent is claude-cli — no API key needed if you have Claude Code installed.
A deep-dive into the four technical contributions that make repeatable knowledge compaction possible
Technical Paper
Nicholas Adamou — smeltery
AI agents are constrained by their context windows. Technical books contain critical guidance but at 200-600 pages they are too verbose to load. This paper presents distill and the four design decisions that make knowledge compaction repeatable, consistent, and cheap to maintain.
Read the PaperFour built-in output formats enforce consistent structure across heterogeneous source material. Custom templates supported.
Reads from PDFs, markdown, Notion, web URLs, EPUBs, and GitHub repos — all normalized into uniform text chunks.
SHA-256 tracks source, template, and agent identity. Changed inputs trigger re-compaction; unchanged sources are skipped.
CLI-based (Claude, Codex) and API-based (Anthropic, OpenAI) providers with the same registry pattern as prr.
Install distill and compact your first book in under a minute