“text-embedding-3-small is fine for everything, right?” Until Japanese product search returns 54% recall while English hits 89%. An e-commerce team traced the gap to their embedding model misplacing multilingual queries in vector space. Switching to Cohere’s multilingual embedding recovered 28 recall points. No code changes. Just a different API. The wrong embedding model silently breaks retrieval across languages and domains — and the default everyone reaches for is the worst offender. This guide compares five APIs on cost, multilingual performance, and dimensionality, with working code for every provider.
If you’re new to LLM APIs entirely, our introductory guide to LLM APIs covers authentication, rate limits, and the fundamentals before you dive into embeddings.
The Embedding Model Landscape
Here’s the single most important fact about embedding models that most tutorials skip: the choice of embedding model affects your RAG retrieval recall by 15-30%. The wrong model silently degrades your entire pipeline while every HTTP status code reads 200. The model you choose determines which chunks your users never see — not because the chunks are missing, but because they sit in the wrong region of vector space.
An embedding model converts text into a dense vector — typically 1024 to 3072 floating-point numbers. Texts with similar meanings produce vectors that cluster together. “How do I reset my password?” and “password reset instructions” produce vectors that are close. “Password security best practices” produces a vector that’s further away — semantically related but not answering the same question.
What makes embedding models different from each other:
- Training data distribution. A model trained predominantly on English Wikipedia and Common Crawl produces embeddings where non-English text maps to less discriminative regions of vector space.
- Dimensionality. More dimensions can capture finer semantic distinctions — but the relationship is logarithmic, not linear. 3072 dimensions is not 2× better than 1536. The improvement on MTEB retrieval is roughly 2-3 points for 2× the storage cost.
- Context window. Most embedding models cap input at 512-8192 tokens. If your chunks exceed the model’s max input, they get silently truncated — and your retrieval quality degrades without warning.
- Normalization. Some models output normalized embeddings (unit vectors). Some don’t. Cosine similarity on unnormalized vectors produces misleading rankings. Know which you’re working with.
The MTEB leaderboard is the standard evaluation framework — but MTEB’s retrieval score measures performance on a specific distribution of queries and documents. Your data is different. Treat MTEB as a directional signal, not a final answer. Always evaluate embedding models on your own data with your own queries.
For the full RAG pipeline that sits downstream of your embedding choice, see our RAG production guide. The retrieval layer that queries your vectors — hybrid search, HyDE, reranking — is covered separately in our hybrid search and reranking guide.
OpenAI: text-embedding-3 Series
Two models, one API. The workhorse and the specialist.
text-embedding-3-small. $0.02 per million tokens. 1536 dimensions. MTEB retrieval: 62.3%. This is the right default for 90% of production workloads. It’s fast, cheap, and performs within 2-3 points of models costing 5× more.
text-embedding-3-large. $0.13 per million tokens. 3072 dimensions. MTEB retrieval: 64.6%. Worth the 6.5× price premium only when retrieval precision has measurable financial or legal consequences — medical literature search, legal document retrieval, compliance-sensitive applications.
The underused feature: the dimensions parameter. You can request 256-dimensional embeddings from either model, cutting vector DB storage by 83% with less than a 2-point retrieval recall drop.
from openai import OpenAI
client = OpenAI()
response = client.embeddings.create(
model="text-embedding-3-small",
input=["Your text here"],
dimensions=256
)
embedding = response.data[0].embedding # len = 256For high-volume RAG with millions of chunks, the storage savings alone pay for the API cost difference within the first month. Batch up to 100 texts per call — serial embedding is the silent latency killer in ingestion pipelines. For high-volume embedding workloads, the same techniques that cut LLM API bills — caching, batching, and tiered routing — apply here too.
Cohere & Voyage: The Specialists
Cohere embed-english-v3. $0.10/M tokens. 1024 dimensions. Where Cohere wins: multilingual retrieval. The model was trained on a deliberately multilingual corpus, and it shows. For applications serving users across English, Spanish, German, Japanese, and Arabic, Cohere’s embeddings produce more consistent retrieval quality across languages than text-embedding-3-small. The trade-off: higher per-token cost and fewer dimensions.
Voyage voyage-3-large. $0.06/M tokens. 1024 dimensions. Anthropic recommends Voyage for Claude-based RAG stacks, and the integration is genuinely tighter — Voyage embeddings paired with Claude show a small but measurable instruction-following advantage in grounded generation tasks. If your entire stack is Anthropic-native, Voyage is the path of least resistance. The cost sits between OpenAI’s small and large models.
Open-Source: bge-m3 & E5
bge-m3 (BAAI). $0 per million tokens — you pay for compute, not API calls. 1024 dimensions. MTEB retrieval: 61.5%. Deploy via llama.cpp, vLLM, or Hugging Face Inference Endpoints. The quality is competitive with text-embedding-3-small on English, and the model punches above its weight on multilingual retrieval.
The trade-off is infrastructure. Self-hosting an embedding model means managing GPU instances, handling scaling, and monitoring throughput. A single G4dn.xlarge (~$0.84/hr) delivers roughly 2,800 embeddings per hour — about 40× cheaper per embedding than cloud APIs at sustained high volume. But you’re now responsible for uptime, latency, and model upgrades.
E5-mistral (Microsoft). Another strong open-source contender. Slightly higher MTEB scores than bge-m3 on English retrieval, slightly lower on multilingual. The decision between bge-m3 and E5 usually comes down to which one your team has infrastructure for — both are capable, both are free at the point of use, both require operational commitment.
Quick Comparison & Decision Matrix
| Model | $/1M Tokens | Dims | MTEB Retrieval | Multilingual | Best For |
|---|---|---|---|---|---|
| text-embedding-3-small | $0.02 | 1536 | 62.3% | Adequate | 90% of workloads — start here |
| text-embedding-3-large | $0.13 | 3072 | 64.6% | Good | Legal/medical precision search |
| Voyage voyage-3-large | $0.06 | 1024 | 63.1% | Good | Claude-native RAG stacks |
| Cohere embed-english-v3 | $0.10 | 1024 | 62.8% | Excellent | Multilingual retrieval |
| bge-m3 (self-hosted) | $0 | 1024 | 61.5% | Good | Data sovereignty, high volume |
The decision tree, in order of priority:
- Data must stay on-prem? → bge-m3 or E5-mistral self-hosted.
- Serving more than 2 languages? → Cohere embed-english-v3.
- All-in on Anthropic/Claude? → Voyage voyage-3-large.
- Everyone else: → text-embedding-3-small. Upgrade to large only when eval data proves the 2.3-point MTEB gap is costing you real retrieval failures.
Embedding Mistakes That Silently Degrade Your Pipeline
Serial embedding at ingestion time. You have 100,000 documents. You call the API once per document — 100,000 sequential network round-trips. At 50ms per call, that’s 83 minutes of wall time. Every embedding API supports batching up to 100 texts per call. That same job finishes in under two minutes. The fix is three lines of code. The delay from skipping the docs is an hour of your life you won’t get back.
Ignoring the dimensions parameter. OpenAI’s text-embedding-3 series lets you request 256 dimensions instead of 1,536 — with less than a 2-point MTEB recall drop. On 10 million chunks, that’s the difference between a $200/month pgvector instance and a $1,200/month one. Most teams never touch this parameter because the default worked in the tutorial and nobody thought to question it.
Embedding model drift without version tracking. You upgrade from text-embedding-3-small to text-embedding-3-large for new documents. Now your vector database contains embeddings from two different models in two different vector spaces. Cosine similarity between them is meaningless. Retrieval quality degrades — and you won’t notice until someone complains about irrelevant search results. Add an embedding_model column to your vector schema. It’s one column. It prevents a silent recall regression that takes days to diagnose. For protecting API keys when calling multiple embedding providers, our API security best practices covers key rotation and access controls.
FAQ
text-embedding-3-small vs large — when is large worth 6.5× the price?
When a missed relevant document has measurable financial, legal, or safety consequences. Medical literature search where a missed study affects treatment recommendations. Legal document retrieval where a missed precedent changes case strategy. For e-commerce search, internal knowledge bases, and customer support RAG, small is almost always sufficient — and the cost difference buys a lot of reranking API calls that deliver far more precision improvement per dollar.
Are Cohere and Voyage actually better at multilingual retrieval than OpenAI?
Yes, measurably. On German, French, Spanish, Japanese, and Arabic retrieval tasks, Cohere embed-english-v3 outperforms text-embedding-3-small by 8-15 points on recall@5. Voyage is stronger than OpenAI on European languages but weaker on Asian languages. If multilingual retrieval is core to your product, Cohere is the best API option. bge-m3 is the best self-hosted option.
What’s the latency and cost of self-hosting an embedding model?
A single G4dn.xlarge (~$0.84/hr on-demand) with bge-m3 on llama.cpp delivers ~2,800 embeddings per hour. At sustained high volume, this is ~40× cheaper per embedding than cloud APIs. But you pay for the GPU 24/7 regardless of usage. The break-even versus cloud API is roughly 500,000 embeddings per month. Below that, cloud API is cheaper and simpler. For detailed endpoint configuration, rate limits, and parameter reference, see the embeddings API documentation.
Do I need to re-index all my documents when I switch embedding models?
Yes. Embeddings from different models exist in different vector spaces. A vector from text-embedding-3-small and a vector from bge-m3 — even for the same text — are not comparable. Switching models requires a full re-embedding pass. This is why the embedding_model column in your vector DB schema matters: it lets you track which chunks were embedded with which model version and plan migrations accordingly.
How often should I re-evaluate my embedding model choice?
Every 4-6 months, or when a new model generation drops. Embedding models improve at roughly the same cadence as LLMs — a model that led the MTEB board in January may be mid-pack by July. Run your 50-query eval set against your current model and 2-3 new candidates. If a new model delivers >5 points of recall improvement, plan a migration. If the gap is <3 points, the migration isn’t worth the re-indexing effort. Check the supported models page for current embedding model availability, dimensions, and lifecycle status across providers.
Your embedding model is the foundation your entire RAG pipeline sits on. Pick the wrong one, and you’re optimizing chunk sizes and tuning HNSW indexes to compensate for a problem that lives one layer deeper. Pick the right one, and the rest of the pipeline falls into place.
A note on embedding latency at scale
At 1,000 queries per day, embedding latency is negligible — 50ms per call, invisible to users. At 100,000 queries per day with a RAG pipeline that embeds the user query, 3 HyDE-generated queries, and 2 query variations per request, you’re making 600,000 embedding calls daily. The 50ms per call becomes 30,000 seconds of cumulative latency. Batching doesn’t help at query time — each query is unique and must be embedded in real time. The optimization that matters at this scale: choose the fastest model that meets your quality threshold, not the highest-scoring model on MTEB. text-embedding-3-small at 50ms per call beats text-embedding-3-large at 120ms when the recall difference is 2.3 points and the latency difference is 2.4×. For high-volume deployments, benchmark embedding speed alongside embedding quality — the MTEB leaderboard won’t tell you which model keeps your p95 latency under budget.
Start with text-embedding-3-small. Evaluate on your data, in your languages, with your queries. Upgrade only when the eval data shows a measurable gap — not before. The $0.02/M tokens default is good enough for 90% of the world. Know which 10% you’re in before you spend more. For embedding cost comparisons across all major providers, our API pricing comparison breaks down per-model rates alongside chat and other services.
Test embeddings from OpenAI, Cohere, and Voyage side by side — one API key, every provider. Compare embeddings models on TokSpan — run the same text through three providers’ embedding endpoints and compare MTEB scores against your own data.