Karpathy Named It. I Built One on My Notes.

2025-02-06 · Source: Decoding AI Magazine · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Robotics & Autonomous Systems · Depth: Advanced, long

Summary

An author developed an LLM Knowledge Base for personal deep research, integrating private data from Obsidian, Readwise, and NotebookLM. This system, named after Andrej Karpathy's concept, utilizes three Claude Code skills: /research_create, /research_search, and /research_distill. It operates without a vector database or Retrieval-Augmented Generation (RAG), instead using the filesystem for state and Markdown, YAML, and JSON for data exchange. The /research_create skill builds a memory/ folder with an index.yaml through multi-round query expansion and subagents that search specific CLIs, performing deduplication and re-ranking. The /research_search skill enables querying this memory/ folder with progressive disclosure across three detail layers. Finally, /research_distill extracts only utilized sources into a research.md file for grounded content generation. This architecture prioritizes context isolation and offers advantages over RAG for personal-scale research, including full lineage, portability, and reduced costs.

Key takeaway

For AI Engineers building agentic research systems, if you are integrating diverse personal data sources like Obsidian or Readwise, consider implementing a CLI-driven, filesystem-based LLM knowledge base. This approach offers superior context isolation, full lineage, and lower operational costs compared to traditional RAG pipelines for personal-scale data. You should explore this architecture to create highly grounded and portable research agents.

Key insights

A private LLM knowledge base can outperform RAG for personal research using CLIs and filesystem-based memory.

Principles

• Prioritize context isolation in agent design.
• CLIs improve token economics and composability.
• Filesystem and Markdown are LLM-native data stores.

Method

Orchestrate three Claude Code skills (create, search, distill) with CLI adapters to build and query a memory/ folder, using multi-round query expansion and subagents for data processing.

In practice

• Integrate personal data from Obsidian, Readwise, NotebookLM.
• Manipulate index.yaml with jq for structured queries.
• Consider this RAG alternative for hundreds of sources.

Topics

• LLM Knowledge Base
• Agentic AI
• Claude Code Skills
• Personal Knowledge Management
• RAG Alternatives
• Context Isolation

Best for: AI Engineer, Machine Learning Engineer, AI Architect

Related on AIssential

• Your Second Brain Is a Graveyard. Make It Agent Memory. — An LLM wiki acts as a dynamic agent memory, transforming static notes into a queryable, evolving knowledge base.
• One Flexible Tool Beats a Hundred Dedicated Ones — Flexible CLI tools outperform dedicated MCP tools for LLM agents due to improved compositionality and reduced context bloat.
• Build enterprise search for agents with Amazon Bedrock Managed Knowledge Base — Amazon Bedrock Managed Knowledge Base streamlines agentic RAG by fully managing infrastructure, data ingestion, and complex retrieval.
• What is OpenRAG? Unlocking the Future of RAG in Generative AI — RAG remains critical for cost, accuracy, and domain-specific knowledge, even with large LLM context windows.
• Context-as-a-Service: Surfacing Cross-File Dependency Chains for LLM-Generated Developer Documentation — CaaS enhances LLM documentation agents by surfacing non-obvious cross-file dependencies, improving accuracy and efficiency.
• How This Creator Turned 7,000 Obsidian Notes Into A Lucrative Research Engine — Connecting a large personal knowledge base to an AI agent swarm can transform digital clutter into a lucrative, autonomous research engine.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Decoding AI Magazine.