Understand HNSW: Why Your Vector Search Returns Garbage (Build your own minimalist HNSW from…

2026-07-11 · Source: Towards AI - Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Advanced, long

Summary

The article explains HNSW (Hierarchical Navigable Small World) graphs, a common Approximate Nearest Neighbor (ANN) algorithm used in vector databases for scalable vector search. It highlights that poor search results in RAG systems are often attributed to embedding models but are more frequently caused by misconfigured HNSW parameters. The author builds a minimalist HNSW index from scratch using Python and NumPy to demonstrate how M (node connections), ef_construction (build-time search width), and ef_search (query-time search width) impact recall. Experiments with a synthetic dataset of 2050 vectors (five clusters, fifty outliers, 64 dimensions) show that a "weak" index (M=4, ef_construction=10) caps outlier recall at 0.21, while a "strong" index (M=32, ef_construction=200) achieves 0.655. The article emphasizes that build-time parameters (M, ef_construction) set the recall ceiling, while ef_search determines how close to that ceiling the system gets, often with diminishing returns on latency. It also provides direct mappings to parameters in production vector databases like Pinecone, Qdrant, and FAISS.

Key takeaway

For MLOps Engineers optimizing RAG systems, if you encounter consistently poor or inconsistent search results, especially for uncommon queries, prioritize HNSW index configuration over embedding model changes. Your M and ef_construction settings critically define the graph's recall ceiling; ef_search only determines how close you get. Measure recall against latency to avoid over-tuning ef_search for negligible gains, saving real money and user-facing delay.

Key insights

Misconfigured HNSW parameters, not embedding models, are often the root cause of poor vector search recall in RAG systems.

Principles

• Build quality sets the recall ceiling.
• Outlier queries are sensitive to graph quality.
• Greedy search can miss true neighbors.

Method

The article describes building a minimalist HNSW index in Python/NumPy, involving a _random_level function for layering, a two-phase insertion process with greedy descent and neighbor selection/pruning, and a layered descent search.

In practice

• Tune M and ef_construction for graph quality.
• Adjust ef_search for query-time recall/latency.
• Diagnose recall issues with brute force comparison.

Topics

• HNSW
• Vector Search
• Approximate Nearest Neighbor
• RAG Systems
• Index Tuning
• Recall Optimization
• Graph Databases

Best for: AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Issue #117 - Scaling Vector Search: HNSW and Approximate Search — ANN search, particularly HNSW, enables scalable, high-performance vector retrieval by trading exactness for speed.
• Agent Memory with Vector Stores: HNSW, Forgetting, and Budgets — HNSW significantly improves vector store search latency for agent memory with minimal practical recall loss.
• HNSW with Accuracy Guarantees Using Graph Spanners -- A Technical Report — The "Certify-then-Rectify" framework combines HNSW speed with exact search guarantees using statistical certification and graph spanner theory.
• How Vector Databases Use Specialized Indexes (HNSW, IVF, PQ, and Annoy) — Vector databases use specialized Approximate Nearest Neighbor (ANN) indexes to achieve millisecond similarity search across millions of high-dimensional vectors.
• Vector Data Types & Native Indexing: The ANN Revolution We Didn’t Know We Needed — Native vector types and indexing in SQL engines enable efficient ANN searches, fundamentally changing data management for AI workloads.
• OpenSearch Optimizations for Production RAG — Production RAG retrieval quality hinges on balancing recall and latency through precise OpenSearch vector search tuning.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.