Billion-point vector search requires HNSW tuning and PQ compression. Master Milvus, Pinecone, and Weaviate scaling with this dev reference. Full breakdown.
What Breaks at Billion Scale
At modest sizes, a vector database can keep full-precision embeddings in memory and serve nearest-neighbor queries with default index settings. At billion-point scale those assumptions fail. RAM pressure forces compression or disk-backed segments; index build time becomes a first-class ops concern; and recall–latency tradeoffs that were invisible on a laptop show up as production incidents. The practical fix is not a single “faster engine,” but deliberate choices on the approximate nearest neighbor (ANN) index, how vectors are encoded, and how the system shards and queries across machines.
HNSW (Hierarchical Navigable Small World) graphs and product quantization (PQ) are the usual levers. HNSW delivers strong recall and low query latency when the graph fits hot storage, but its edge lists grow with data size and connectivity parameters. PQ reduces each vector to compact codes so you can fit more points per node and scan candidates cheaply. Scaling to billions almost always means tuning both together rather than treating either as a default you never touch.
HNSW Tuning That Actually Matters
HNSW quality and cost are driven by a small set of knobs. Build-time parameters control graph connectivity and how thoroughly each new point is linked; higher connectivity and deeper candidate search at insert improve recall and make the graph more robust, at the cost of longer builds and more memory. Query-time search depth (how many candidates you expand) trades CPU and latency for recall: raise it when quality slips, lower it when p99 latency or cost is the bottleneck. At billion scale, treat build and query settings as independent: over-connect the graph once, then dial query effort per traffic class or SLA.
Operational guidance: measure recall on a fixed, representative query set when you change any of these settings; never retune from latency alone. Prefer rebuilding (or rebuilding a shard) when you materially change graph parameters rather than hoping online inserts will “heal” a sparse graph. Keep segment or collection sizes large enough that each HNSW component amortizes overhead, but small enough that rebuilds and rebalancing stay manageable.
PQ Compression Without Blinding Recall
Product quantization splits high-dimensional vectors into subspaces and replaces each subspace with a short code from a learned codebook. The result is a much smaller memory footprint and faster distance approximations, which is what makes billion-point in-memory or hybrid layouts feasible. The tradeoff is approximation error: aggressive compression can flatten distances so that true nearest neighbors fall out of the candidate set before you re-rank.
- Train codebooks on data that matches production distribution (same embedding model, same preprocessing).
- Use PQ (or a hybrid with residual/refined codes) for coarse candidate generation, then re-rank top candidates with full-precision or higher-precision vectors when quality matters.
- Validate end-to-end recall at the compression level you plan to ship; if recall drops, increase code budget or candidate list size before blaming the graph alone.
Plan storage for both the compressed index used in search and whatever full or semi-full vectors you keep for re-ranking and updates. That dual footprint is normal at this scale and should be sized into capacity planning up front.
Milvus, Pinecone, and Weaviate: Shared Scaling Patterns
Despite different product shapes, Milvus, Pinecone, and Weaviate are scaled with the same mental model: partition the corpus so each unit owns a manageable slice, run HNSW (or an equivalent ANN structure) plus compression inside those units, and fan out queries with a merge step. Focus configuration on shard or collection sizing, index type and PQ/quantization options, replica count for read throughput and availability, and resource isolation so build jobs do not starve online search. Prefer fewer, well-tuned indexes over many ad hoc collections that fragment memory and complicate recall testing.
Use this as a working checklist: (1) fix embedding dim and distance metric before index experiments; (2) establish a recall baseline on uncompressed or lightly compressed HNSW; (3) introduce PQ and re-rank until memory and latency fit budget without breaking that baseline; (4) shard for build time and blast radius, then add replicas for QPS; (5) automate rebuilds and config promotion so parameter changes are deliberate. That sequence turns “billion-point vector search” from a vendor slogan into an operable system across Milvus, Pinecone, and Weaviate alike.