OpenAI’s text-embedding-3-large moved MTEB from 61.0% to 64.6%. And the argument that number started is reshaping frontier LLM embedding models. Research on arXiv splits the field cleanly: LLMs used as embedders beat dedicated embedding models on reasoning-heavy retrieval, dedicated models win on classification. And the two tie in aggregate on clustering and semantic textual similarity. The LLM route delivers comparable quality at substantially higher cost and lower throughput. And it inherits quirks like recency bias plus a September 2021 knowledge cutoff. For most small operators, the dedicated embedder still wins.
The LLM embedder earns its bill only when your retrieval genuinely requires cross-document reasoning.
I ship retrieval pipelines for small businesses, and embeddings are the layer everyone treats as settled plumbing.
They aren’t. OpenAI’s own docs define embeddings as numerical representations that measure relatedness between pieces of text, powering search, clustering, recommendations, anomaly detection, and classification. When embedder behavior shifts, all five of those systems shift with it. No exception gets thrown.
Retrieval just gets subtly wrong, which is the hardest kind of wrong to debug.
What the 64.6% Actually Tells You
OpenAI shipped text-embedding-3-small and text-embedding-3-large on January 25, 2024, calling them its “newest and most performant embedding models” in the embeddings guide.
The large variant outputs 3072 dimensions by default, the small 1536. And a dimensions parameter lets you shorten vectors “without losing its concept-representing properties.” Max input sits at 8192 tokens. And the guide cites roughly 9,615 pages per dollar as the cost-efficiency figure. Pinecone’s model docs position it the same way, as the “most powerful OpenAI embedding model, with a larger embedding size,” aimed at vector search applications “requiring higher degrees of accuracy.”
Now look at the benchmark split instead of the headline. On MTEB, the jump from ada-002 to text-embedding-3-large was 61.0% to 64.6%. On MIRACL, a multilingual retrieval benchmark, it was 31.4% to 54.9%, per OpenAI’s release notes. The multilingual gain dwarfs the aggregate one. If you run English-only product search or internal doc retrieval, the aggregate number is not your number. And treating it as such is how teams end up disappointed after a migration that tested fine on paper.
The Embedder’s Dilemma: Reasoning Wins, the Meter Runs
A paper titled “The Embedder’s Dilemma: LLMs Are Better. But at What Cost?” (arXiv 2608.12875) is the sharpest articulation of the emerging problem.
Its finding: LLMs used as embedders outperform dedicated embedding models on reasoning-heavy retrieval, dedicated models lead on classification. And the paradigms tie on clustering, semantic textual similarity, and pair classification. The paper’s own summary is blunt: “In aggregate, the paradigms tie.”
The split follows the shape of the work. LLM-based embedding pipelines excel when cross-document reasoning is required. Traditional embedding pipelines excel wherever geometric similarity or labeled-reference matching suffices. That distinction is the whole decision for a small team. Because comparable LLM quality arrives with substantially higher cost and lower throughput. The paper quantifies the meter: “Reasoning tokens account for 28 to 81% of LLM inference cost,” and lower reasoning budgets preserved or improved retrieval quality for most models in their ablation. Paying frontier prices to embed a support-ticket queue is a category error.
The frontier is also invading multimodal territory. A 2026 comparison paper (arXiv 2608.11343) reports that GPT-4.1 and Claude Sonnet 4.6 perform “on par with Gemini Embedding 2” on hard-negative text-to-image retrieval. General-purpose LLMs matching a specialized embedding model on a deliberately hard benchmark is the trend in one data point. Specialized embedders are not dead. But their moat is narrowing to cost and speed, exactly the two things a one-person agency cares most about.
The Quirks You Inherit: Recency Bias and a 2021 Wall
Frontier embedders inherit their base models’ limitations, and the docs admit it. OpenAI’s guide states plainly that text-embedding-3-small and text-embedding-3-large “lack knowledge of events that occurred after September 2021.” Your vector database happily embeds documents about things the model never saw in training. Relatedness gets measured against stale world knowledge, and nothing in the pipeline warns you.
The deeper issue is architectural. Research cited in the dilemma paper, drawing on Causal2Vec work, indicates that decoder-only LLMs used as embedding models are prone to recency bias: the embedding of an output token such as EOS depends disproportionately on the most recent tokens in the input. That degrades the representations the model builds and has motivated architectural adjustments to mitigate the bias. In practice, long documents get summarized by their endings more than their middles.
Here is why a small business owner should care at all. These quirks propagate silently into everything downstream of the vector: search results, recommended products, clustered tickets, flagged anomalies. Swap an embedding model and your RAG answers drift a week later, with no stack trace pointing at the cause. The fix is not a better LLM. It is knowing that the embedder is a load-bearing component with its own failure modes.
What Small Operators Should Actually Do
– Match the embedder to the task. Classification, deduplication, and similarity matching belong on a dedicated embedder. Save LLM embedders for retrieval that truly needs cross-document reasoning.
– Go multilingual deliberately. An AWS Machine Learning blog built Indian-language RAG on Cohere embed-multilingual-v3, which produces 1,024-dimension vectors across over 100 languages via Amazon Bedrock. That is a production-grade config, not a benchmark fantasy.
– Shrink vectors where storage matters. The dimensions parameter exists so 3072 dimensions do not have to be your storage bill.
– Benchmark on your own corpus, not MTEB. The aggregate scores tie. Your task does not.
– Plan for throughput. LLM embedder quality comes with lower throughput and frontier inference costs, and the 28 to 81% reasoning-token overhead is real money at volume.
Your retrieval stack is only as good as its embedder, and the embedder market just got genuinely task-dependent.
Treating embeddings as a solved commodity was safe in 2024.
It is not anymore, and the teams that notice first are the ones whose search stops quietly rotting.
If you want a second pair of eyes on your retrieval pipeline, from embedder choice to evaluation, that is exactly the work I do at Mediascout: book a call and we will pressure-test what your vectors are actually measuring.
