How HubSpot Scaled Semantic Search to 20 Billion Vectors

2026-07-07 · Source: InfoQ · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Software Development & Engineering · Depth: Advanced, short

Summary

HubSpot has detailed the scaling of its internal semantic search platform, VaaS (Vector as a Service), to manage over 20 billion vectors across more than 38 teams. This system, which supports agents, RAG, and contact deduplication, utilizes Qdrant running on-premises for features like named vectors, hybrid search, and cost controls. VaaS itself provides access control, embeddings generation, data versioning, and feedback collection. The platform now spans over 200 indexes, 140-plus clusters, five regions, and two environments, handling write traffic peaking at 100,000 requests per second. Initially built with Helm, the team transitioned to an internal Kubernetes Operator framework with "Translators" to automate cluster creation, decommissioning, shard movement, and replication recovery. This shift reduced operational load and cluster spin-up times from hours to minutes, highlighting that operational automation is crucial for managing vector search at scale.

Key takeaway

For AI Architects designing large-scale retrieval systems, recognize that the vector database is only one component. You should prioritize building robust operational automation, like HubSpot's Kubernetes Operator framework, to manage cluster lifecycle, scaling, and recovery. This approach reduces operational overhead and ensures system stability as vector counts and demand rise, preventing performance degradation and costly manual interventions.

Key insights

Scaling vector search demands robust operational automation beyond just the database.

Principles

• Manual operations don't survive growth.
• Recall and reliability are paramount at scale.
• Operational automation reduces system load.

Method

HubSpot implemented an internal Kubernetes Operator framework with "Translators" to reconcile desired and actual system states every 60 seconds, automating cluster lifecycle management.

In practice

• Implement Kubernetes Operators for automation.
• Balance shard placement to avoid memory skew.
• Monitor recall and latency for quality.

Topics

• Semantic Search
• Vector Databases
• Qdrant
• Kubernetes Operators
• Platform Engineering
• AI Retrieval Systems
• Infrastructure Automation

Best for: CTO, VP of Engineering/Data, MLOps Engineer, AI Architect, Director of AI/ML

Related on AIssential

• How Airtable Built the Search Layer Behind Their AI Features — Data access patterns, not algorithms, dictate optimal system architecture for vector search.
• Operational AI Is Hard — Until You Understand This One Shift — Operationalizing AI requires a structured approach to deploy, scale, and manage models in production environments.
• Zilliz Launches Vector Lakebase, Extending the World’s Most Adopted Vector Database into a Unified Data Platform for AI — Unifying vector search, discovery, and analytics on a single data foundation streamlines AI data operations.
• Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Hørup Eskildsen of Turbopuffer — Simon Hørup Eskildsen founded Turbopuffer after realizing the prohibitive cost of vector search for article recommendations at Readwise, aiming to build a cost-effective "search engine for unstructure…
• Building Real-Time Product Search on Databricks — Modern product search requires an integrated platform for real-time ingestion, retrieval, and refinement, balancing speed with relevance and business outcomes.
• Optimizing ML Compute & Orchestration with H2O MLOps | Part 17 — Kubernetes orchestrates AI workloads, enabling resource management, cost optimization, and infrastructure autoscaling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by InfoQ.