Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Hørup Eskildsen of Turbopuffer

2026-03-10 · Source: Latent Space: The AI Engineer Podcast · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Software Development & Engineering · Depth: Advanced, extended

Summary

Simon Hørup Eskildsen founded Turbopuffer after realizing the prohibitive cost of vector search for article recommendations at Readwise, aiming to build a cost-effective "search engine for unstructured data" that connects AI models to full-fidelity knowledge. The company's architecture leverages recent cloud primitives like S3 strong consistency, NVMe SSDs, and object storage compare-and-swap, enabling a simple, serverless design without traditional consensus layers. Turbopuffer successfully reduced costs by 95% for early customers like Cursor and met stringent latency demands for Notion by prioritizing architectural conviction, even buying dark fiber. In the age of AI, Turbopuffer addresses the "build vs. buy" dilemma by offering rapid deployment and adapting to agentic workloads that drive highly concurrent, parallel queries, leading to a 5x reduction in query pricing. The future roadmap includes expanding state-of-the-art full-text search features, scaling to common crawl-level datasets with advanced ANN algorithms, and eventually supporting a broader range of query plans based on customer needs.

Key takeaway

Turbopuffer delivers a highly scalable and cost-efficient search engine for unstructured data, achieving 95% cost reductions for customers like Cursor by leveraging S3 strong consistency and NVMe SSDs without a traditional consensus layer. Its ANNv3 searches 100 billion vectors with P50 40ms, and its full-text engine outperforms Lucene on web-scale, LLM-generated queries. This enables AI/ML professionals to deploy agentic applications requiring massive, concurrent retrieval against vast datasets at significantly lower operational overhead.

Topics

• Vector Search
• Cloud Database Architecture
• Full-Text Search
• AI Agent Workloads
• Cost Optimization

Best for: CTO, VP of Engineering/Data, Entrepreneur, Software Engineer, Data Engineer, MLOps Engineer

Related on AIssential

• Databricks built a RAG agent it says can handle every kind of enterprise search — Multi-task reinforcement learning enables RAG agents to generalize across diverse enterprise search behaviors, improving performance and efficiency.
• TurboQuant: Redefining AI efficiency with extreme compression — TurboQuant offers extreme compression for LLMs and vector search with zero accuracy loss and significant speedups.
• The Secret Behind Claude Code’s Retrieval: Why Live Search Fits Better than RAG — Claude Code's retrieval uses live search with `ripgrep` and LSP, not persistent RAG, to explore dynamic codebases like an engineer.
• Agents need vector search more than RAG ever did — AI agents significantly increase retrieval demands, making purpose-built vector search infrastructure more critical, not less.
• Proxy-Pointer RAG: Structure Meets Scale at 100% Accuracy with Smarter Retrieval — Embedding document structure into RAG vector indexes dramatically improves retrieval accuracy for complex, structured documents.
• not much happened today — AI agent tooling, infrastructure, and robotics are rapidly advancing, while platform stability and search access pose new challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Latent Space: The AI Engineer Podcast.