OpenAI embeds at 1536 or 3072 dims, with prices from $0.02 per 1M tokens. Use this 2026 semantic search reference for setup and tuning. Read now.
What dimensions and cost tell you up front
OpenAI embedding models expose two practical knobs you care about immediately: vector size and price. Common options sit at 1536 or 3072 dimensions, with pricing from $0.02 per 1M tokens. Higher dimensions give the model more room to separate near-duplicates and fine-grained meaning; they also increase storage, index size, and the cost of every similarity comparison. Lower dimensions are cheaper to hold and search, and often good enough when your corpus is narrow or your queries are short and concrete.
Treat dimension choice as a capacity decision, not a quality score. Start with the smaller size if you are still validating retrieval quality. Move up only when you see systematic confusion between similar topics that a better index or better chunking cannot fix.
Set up the retrieval path cleanly
Semantic search works only as well as the text you embed. Split documents into chunks that answer one idea each—roughly a short section or a few paragraphs—so a query can land on a self-contained unit of meaning. Keep metadata (title, section, source id, updated date) next to each vector so you can filter before or after similarity search. Embed queries with the same model family and the same preprocessing you used for documents; mixing models or silently changing normalization breaks ranking.
Store vectors in a system that supports approximate nearest-neighbor search at your scale, and always keep a path back to the original text. The index should return top-k candidates; your application then re-ranks, filters by metadata, and presents snippets. Log the query, the top hits, and whether the user accepted a result so you can tune later with real failure cases.
- Normalize whitespace and casing consistently before embed and at query time.
- Embed the full chunk text, not titles alone, unless titles are the only content users will match on.
- Cap k, then apply hard filters (tenant, language, product area) so you do not rank irrelevant neighbors.
- Cache embeddings for unchanged documents; re-embed only when content or the model choice changes.
Tune for precision, not just recall
Default top-k search often surfaces related but wrong neighbors. Improve precision with hybrid retrieval: combine dense vectors with keyword or BM25 scores so exact identifiers, error codes, and product names still win. Use a shortlist from the vector store, then re-score with a cross-encoder or a simple rule layer if latency allows. Threshold on similarity only after you plot scores on real queries; absolute cosine cutoffs transfer poorly across models and corpora.
When results are noisy, fix the corpus before the model. Merge tiny chunks, split long ones that mix topics, and remove boilerplate that collapses many pages into the same neighborhood. For multi-tenant apps, isolate indexes or always filter by tenant so similarity never crosses boundaries.
Operate it like a product feature
Budget from token volume: every reindex multiplies cost by document length and dimension-related storage. Prefer incremental updates. Monitor empty result rates, click-through on top hit, and “no useful answer” feedback. Revisit dimension and model size when latency, disk, or quality plateaus—not on a fixed calendar. This 2026 cheat sheet is a setup and tuning reference: pick dimensions for capacity, price embeddings from the published per-token rate, wire a consistent embed–index–filter–re-rank loop, and iterate on chunking and hybrid scoring until your real queries stop failing for predictable reasons.