HNSW offers 98% recall at 5ms latency while IVF-Flat reduces memory by 40%. Compare performance metrics for 1B+ embeddings in this deep dive. Read now.
What the Benchmark Is Measuring
At billion-scale embedding search, the useful comparison is not “which index is better” in the abstract. It is how each index trades recall, latency, and memory under the same corpus size, dimensionality, and query mix. HNSW and IVF-Flat sit on opposite sides of that tradeoff: one is built to keep neighbors close in a multi-layer graph so queries stay fast at high recall; the other partitions the space so most of the vector store can stay cold until a shortlist of clusters is probed.
For 1B+ embeddings, those choices stop being theoretical. Graph edges, inverted lists, and resident working sets dominate cost. A fair benchmark therefore holds the dataset fixed, varies the knobs that operators actually tune (efSearch, nprobe, candidate list size, and how much of the index is kept in RAM), and reports recall and end-to-end latency together—not either metric alone.
HNSW: High Recall With Tight Latency Budgets
HNSW builds a hierarchical navigable small-world graph. Queries start in a sparse upper layer and descend into denser layers, greedily walking toward the query vector. That structure is why HNSW often delivers strong recall at low single-digit millisecond latency when the graph and vectors fit in memory and efSearch is set high enough for the target recall.
In the framing of this deep dive, HNSW’s headline profile is about 98% recall at roughly 5ms latency. That combination is attractive for interactive retrieval: RAG, semantic search, and real-time ranking where users notice lag more than they notice a slightly larger index. The cost shows up in memory and build time. Each node stores neighbor lists; higher connectivity and higher construction quality improve search but inflate RAM and index-build duration. At a billion vectors, plan for graph overhead as a first-class capacity item, not a footnote.
IVF-Flat: Memory Efficiency at Scale
IVF-Flat first learns a coarse quantizer (typically k-means centroids), assigns each vector to a list, then stores full-precision vectors inside those lists. At query time you probe a subset of lists and brute-force only the candidates in those lists. Flat storage inside lists means you avoid product quantization error inside the inverted file—you trade pure scan cost of the shortlist for fidelity of the residual comparison.
Relative to a dense HNSW layout, IVF-Flat is often chosen when the goal is footprint. The comparison here highlights roughly a 40% memory reduction versus the HNSW path under comparable workload assumptions. That matters when the index must stay on a single machine, when multi-replica RAM is the dominant bill, or when you can accept more nprobe work (and slightly higher or more variable latency) to keep the working set smaller. Latency becomes a function of how many lists you open and how large those lists are; under-probing cuts latency and recall together, so tuning is mandatory rather than optional.
How to Choose for Production Workloads
Use the metrics the title forces into view: recall near 98%, latency near 5ms, memory pressure, and corpus size at 1B+. Prefer HNSW when the product SLA is interactive and you can pay for graph RAM. Prefer IVF-Flat when memory headroom is the binding constraint and you can spend probe budget to reclaim recall. Many systems run both: HNSW for hot or high-SLA collections, IVF-style indexes for colder or larger shards.
- Benchmark with your real embedding dimension, distance metric, and query distribution—not a generic leaderboard dump.
- Sweep recall vs latency with the operator knobs you will ship (efSearch / nprobe), then pick the knee that still meets the SLA.
- Measure peak RSS and index build wall time at a representative shard size before extrapolating to a full billion-vector fleet.
- Re-test after rebalancing or daily insert rates; both indexes degrade if parameters stay frozen while the data distribution shifts.
The practical outcome of an HNSW vs IVF-Flat deep dive is a capacity plan: where you spend memory for graph connectivity, where you spend CPU for inverted-list probes, and which index keeps recall high enough that downstream ranking does not have to compensate for missing neighbors.