Best embedding model for RAG: how to choose one

There is no single best embedding model for RAG. The authors of MTEB, the standard embedding benchmark, found that no model dominates across all tasks, and your documents are not in their datasets. Shortlist two or three models that fit your constraints (whether data may leave your servers, your languages, your document length, your storage), then keep the one that finds the right passage most often on a test set of your own questions.

This is a Labs note built from model cards and official docs. We quote no benchmark of our own.

Open model or API?

An API model needs no hardware, but every chunk and every question goes to the provider, you pay per token, and the provider decides when a model is retired. An open model runs on your own servers at a fixed hardware cost, and you decide when to upgrade. Read the licence: EmbeddingGemma asks you to accept Google's Gemma terms before download.

If your contracts allow documents to go to a cloud provider, an API model is the quickest way to start testing. If they must stay on your premises, BGE-M3 and Qwen3-Embedding-0.6B are sensible first candidates: both have permissive licences, long inputs and support for over 100 languages.

Either way, the choice is sticky. Vectors from two different models cannot be compared, so switching embedding model means re-embedding every chunk you hold.

What to read on a model card

Figures below are from each provider's docs or model card, checked on 10 November 2026. Our post on semantic search for company documents lists more open models.

Max input must cover your largest chunk. Text past the limit is cut off, so a 512 token model silently loses the end of a long clause.

Dimensions set your storage. In pgvector each vector takes 4 bytes per dimension plus 8, so a million chunks at 1,024 dimensions is about 4.1 GB of raw vectors and at 3,072 about 12.3 GB, before any index. The pgvector README also caps HNSW indexes at 2,000 dimensions for the standard vector type and 4,000 for halfvec, so a 3,072 dimension model needs shortening or half precision. Our pgvector comparison covers the rest.

Shortening works on models trained for it. OpenAI's dimensions parameter, Gemini's output size, Qwen3's custom dimensions and EmbeddingGemma's smaller sizes all keep the vector useful at a fraction of the size. OpenAI tells you to normalise vectors you shorten by hand, and Google says gemini-embedding-001 needs manual normalisation below 3,072.

Prompts are easy to miss. multilingual-e5-large-instruct needs an instruction line on every query, and its card says leaving it out hurts results. Qwen3 reports 1% to 5% gains from query instructions. EmbeddingGemma and gemini-embedding-2 expect fixed prefixes for queries and documents. BGE-M3 needs none.

What about Hindi, Tamil and other Indian languages?

"Over 100 languages" on a card describes the training data, not results on your files. Read the detail. IBM's Granite multilingual R2 model lists Hindi among 52 languages with enhanced support, and Tamil is not on that list. Krutrim's Vyakyarth reports results for nine Indian languages including Tamil, Telugu and Bengali, but reads at most 128 tokens and ships under the Krutrim Community License.

The MMTEB paper added a regional MTEB(Indic) benchmark, a better first filter than the overall average. Then test three cases yourself: questions in native script, the same language typed in Latin script, and an English question against an Indian language document.

How to test on your own documents

MTEB is a guide, not an answer. Its leaderboard averages tasks such as classification and clustering that a RAG system never runs, so look at the retrieval scores and at your languages, then test.

1. Write 50 to 100 real questions, each with the passage that answers it; here is how to build that set.
2. Shortlist two or three models that pass your constraints.
3. Embed the same chunks with each model, with the prefixes their cards require.
4. Record how often the right passage is in the top 5 and in the top 20, per language. A large gap between the two suggests a reranker will help more than a new model.
5. Read the failures by hand; a pattern such as missed part numbers points to keyword search, not a new model.
6. Weigh speed, storage and cost, and keep the cheapest model that is not clearly worse.

Keep the chunking fixed while you compare, or you will not know which change made the difference.

Choosing with us

Our RAG chatbot builds use hosted APIs or open weight models on your own hardware, and every change to chunking, search or prompts is scored against a question set written with your team.

If your documents mix English with Tamil or Hindi and you want a shortlist tested on them, send us a sample. Book a consultation.