Build your own Hybrid Search

2025-11-26 · Source: Abhishek Thakur · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Intermediate, long

Summary

This content details the process of upgrading a BM25-based search engine to a scalable hybrid search system using Vespa, capable of handling over 1.3 million documents. The upgrade involves integrating semantic search capabilities with traditional keyword search. Key components introduced include Hierarchical Navigable Small World (HNSW) for high-speed embedding indexing and global phase ranking to combine keyword (BM25) and AI (embedding) scores. The schema is modified to include a `text_embedding` field of type `tensor<float>(x[384])`, utilizing a Sentence Transformer model (all-MiniLM-L6-v2) for embedding generation. New rank profiles, "semantic" and "fusion," are configured, with "fusion" employing reciprocal rank fusion (RRF) to combine BM25 and semantic similarity scores for improved relevance, especially for complex queries. Performance demonstrations show hybrid search yielding significantly more relevant results compared to BM25 alone, with search times in single-digit milliseconds.

Key takeaway

For AI Engineers building scalable search systems, integrating hybrid search with Vespa offers a significant leap in relevance. You should consider adopting HNSW for fast embedding indexing and reciprocal rank fusion in your ranking profiles to effectively combine lexical and semantic signals, especially when dealing with large document corpora and complex user queries. This approach ensures more accurate results without sacrificing performance.

Key insights

Hybrid search combining BM25 and semantic embeddings dramatically improves relevance for complex queries over large datasets.

Principles

• HNSW accelerates embedding search for millions of documents.
• Global phase ranking allows combining keyword and AI scores.
• Reciprocal Rank Fusion (RRF) merges ranks, not raw scores.

Method

Modify schema to add `text_embedding` field (e.g., `tensor<float>(x[384])` with HNSW and angular distance). Configure "fusion" rank profile using RRF to combine BM25 and semantic closeness scores in a global phase.

In practice

• Use Sentence Transformers for generating document embeddings.
• Set `rerank_count` to limit expensive global phase operations.
• Employ angular distance for normalized vector similarity.

Topics

• Hybrid Search
• Semantic Search
• Vespa
• HNSW
• Reciprocal Rank Fusion

Best for: Machine Learning Engineer, AI Engineer, Software Engineer

Related on AIssential

• What is reciprocal rank fusion in hybrid search? — Reciprocal Rank Fusion effectively combines diverse ranker outputs into a robust global ranking without training.
• Vespa AI and Surpassing the Limits of Vector Search — Vector similarity alone is insufficient for robust search relevance; tensor-based retrieval offers superior flexibility and efficiency.
• Building a Hybrid Semantic Search Engine from Scratch: A Deep Dive into TF-IDF, GloVe, and Dual GPU… — Hybrid search combining TF-IDF and GloVe embeddings significantly improves intent-based query relevance over keyword-only methods.
• Article: Why Vector Search Alone Isn't Enough: Hybrid Retrieval for RAG — Hybrid retrieval combining vector search and BM25 via Reciprocal Rank Fusion is essential for robust RAG systems handling diverse query types.
• Modernizing the Facebook Groups Search to Unlock the Power of Community Knowledge — Hybrid retrieval, multi-objective ranking, and AI-driven evaluation enhance community content discovery and validation.
• Not your usual search system — feat. BM25, FAISS and XGBoost — Production-grade search systems combine lexical and semantic retrieval with learning-to-rank for optimal performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Abhishek Thakur.