I spent a few hours building a personal knowledge base where Claude acts as the librarian. The idea came from Andrej Karpathy's LLM Wiki pattern: instead of retrieving from a database, the LLM incrementally builds and maintains a structured wiki from raw sources. Every time I add something to the inbox, Claude reads it, extracts what matters, and updates the wiki. Over time, the wiki gets smarter.
Here's how it's set up, what went wrong during the build, and what I'm actually using it for.
The Core Idea
Most second brain setups treat the AI as a search tool. You dump notes in, you ask questions, it retrieves. The Karpathy approach is different. The LLM is the wiki maintainer. It reads raw sources, decides what's important, creates structured pages for concepts and entities, cross-links everything, and keeps the index updated. The knowledge compounds session over session because the wiki itself is the persistent layer, not a vector database.
The practical version of this: I have a folder called raw/inbox/. Articles I clip, voice notes I transcribe, PDFs I convert, things I'm thinking about — they all land there. When I open Claude Code pointed at the vault and run /second-brain-ingest, Claude reads everything in the inbox, builds or updates wiki pages, and moves the processed files to raw/archive/. The wiki grows. The inbox empties.
Where It Lives
First decision: where to put the vault.
My initial instinct was iCloud, inside the Obsidian directory at ~/Library/Mobile Documents/.... Tried it. The shell returned "Operation not permitted" immediately. iCloud's mobile documents directory is sandboxed — shell tools can't write there.
Moved it to ~/Documents/brain/. No iCloud sync, which means no access from my phone. I thought about this for about thirty seconds and realized I don't use Obsidian on my phone anyway. The vault lives on my laptop and gets git-committed. That's fine.
The Folder Structure
~/Documents/brain/
├── raw/
│ ├── inbox/ ← everything lands here
│ └── archive/ ← processed files move here
├── wiki/
│ ├── sources/ ← one summary page per ingested source
│ ├── entities/ ← people, orgs, tools
│ ├── concepts/ ← ideas, frameworks, mental models
│ └── synthesis/ ← cross-topic analysis
├── wiki/index.md ← master catalog of every wiki page
├── wiki/log.md ← append-only operation record
└── CLAUDE.md ← operating rules
I went back and forth on whether to add domain subfolders inside raw/inbox/ — an articles/ folder, a videos/ folder, a thoughts/ folder. Decided against it. Everything lands in inbox, flat. Claude sorts it during ingest. Adding folder structure to raw/ would just create friction for zero benefit.
Same decision on domain-specific tags. The initial design had a rigid taxonomy of seven domains. Dropped it. Replaced with a list of suggested tags in CLAUDE.md that Claude can use or adapt. Tags should emerge from the content, not get imposed before anything is in there.
The CLAUDE.md
Every wiki page Claude creates follows the same structure: YAML frontmatter with tags, sources, created date, and updated date. Internal cross-references use [[wikilink]] syntax. Source pages are factual summaries. Interpretation goes in concept and synthesis pages.
CLAUDE.md defines all of this. It also tells Claude the commands, the index format, the log format, and the rules for what not to do (never modify raw files, never invent facts, always append to the log, always update the index).
The commands:
/second-brain-ingest— process everything in raw/inbox//second-brain-query [question]— search the wiki and synthesize an answer/second-brain-lint— health check: find orphans, contradictions, gaps
What Went Wrong
Two things.
markitdown installation. The vault needs a way to convert PDFs, audio files, and other formats into markdown before they land in the inbox. markitdown is a Microsoft open-source tool that handles this. I tried pip install markitdown. macOS returned "externally managed environment" and refused. Tried pip3. Same error. The fix was brew install pipx && pipx install markitdown. Works fine.
Slash commands not loading. The /second-brain-ingest skill came from a GitHub package installed via npx skills add. The install worked, but the commands didn't appear in Claude Code. The problem: the skill installed to ~/.agents/skills/, which is the wrong location for Claude Code. Claude Code reads from ~/.claude/skills/. Fixed with:
cp -r ~/.agents/skills/second-brain ~/.claude/skills/second-brain
After that, the commands loaded correctly.
What It's Actually For
I didn't have a clear answer to this at the start. Two use cases clarified through the build.
Content fuel. I produce a lot of content across personal brand and client work. The hard part isn't writing — it's having enough material, specific angles, and remembered things that I've read or thought about. A wiki that accumulates everything I consume and organizes it into concepts and entities gives me a searchable, cross-linked library to draw from. Instead of trying to remember an article I read three months ago, I run /second-brain-query and get a synthesized answer with citations to wiki pages.
Agent memory. The second brain is also a persistent knowledge layer that any AI session can access. Claude doesn't have memory across conversations by default. But if everything I research, every decision I make, every thing I learn gets processed into the wiki, then any future Claude session can read it and have the accumulated context. The vault is memory that outlasts the context window.
Where This Is Going
I've been studying how Internet Vin, a builder who's been deep in this setup for over a year, uses his vault. The commands he's built give a clear picture of what this system looks like when it matures.
/trace — tracks how an idea has evolved across the vault over time. You ask it to trace a concept, and it reads every file where that idea appears, then builds a chronological history of how your thinking on it has developed. He ran it on his relationship with Obsidian itself and got a 13-month timeline: where he first mentioned it, his early skepticism, the pivot in how he used backlinks, where it started producing real output. For someone who writes a lot and thinks in public, this is a way to see intellectual development you can't see from inside it.
/emerge — surfaces ideas the vault implies but never states. Not ideas you've had, but conclusions scattered across multiple notes that haven't been named yet. Unnamed patterns. Unarticulated directions. He describes this as the command that produces the biggest breakthroughs — not because it invents anything, but because it names something you've been circling around for months without realizing it.
/ghost — answers a question the way you would. It builds a voice profile from the vault, responds in that voice, then evaluates how faithful the response is to your actual patterns. For a personal brand, this is interesting: after enough writing is in the vault, the system can approximate how you think about any given topic.
/challenge — pressure tests your current beliefs using the vault's own history. It finds where your stated positions contradict each other, where your thinking has shifted without you acknowledging it, where you've written one thing but done another. The point isn't to win an argument. It's to keep the beliefs honest.
/drift — compares stated intentions against actual behavior over a 30 to 60 day window. You say you care about something. Your notes over the past two months tell a different story. This surfaces the gap.
/ideas — a full vault scan with cross-domain pattern detection, producing a report on tools to build, things to write, conversations to have, and systems to implement, all grounded in what you've actually been writing about. Not generic suggestions. Things that come directly from the patterns in your notes.
None of these exist in my vault yet. They're custom commands Vin built over a year of daily use. The vault I have now is three days old and has about a dozen pages.
But the architecture is the same. The commands are built on top of a vault of interlinked markdown files. If I keep the capture habit going — clip articles, transcribe voice notes, drop in things I'm thinking about — the vault will eventually have enough material that these commands become worth building.
Month one is about building the habit. The commands come later.