Hybrid search: BM25 plus vectors, and how to merge them

Hybrid search runs a BM25 keyword search and a vector search on the same query and merges the two ranked lists, usually with reciprocal rank fusion (RRF). It fixes the main weakness of vector search, which is exact strings: part numbers, names, error codes and section numbers. Elasticsearch, OpenSearch, Azure AI Search, Weaviate, Qdrant, Milvus and MongoDB Atlas all support it natively, and PostgreSQL can do it with pgvector and its full-text search.

This Labs note is for teams whose document chatbot finds the right topic but the wrong part. For the basics of meaning-based search, see our glossary entry on semantic search.

Why does vector search miss part numbers and codes?

An embedding model turns text into a vector that captures meaning. Two passages about replacing a pump seal land close together even when worded differently, which is the point. But "TS-998" and "TS-999" can look almost identical to the model, and a supplier name or an internal project code may be a string it has never seen. Anthropic's 2024 write-up on contextual retrieval gives exactly this case: a query for "Error code TS-999" where an embedding model can miss the exact match and BM25 finds it.

Keyword search fails the other way. A user who types "machine keeps stopping" will not match a manual that says "intermittent shutdown". You want both.

What BM25 adds

BM25 scores a passage by how often the query words appear in it, gives more weight to rare words, and adjusts for passage length. A part number that appears in three passages out of a million is a very rare term, so the passage containing it rises to the top. BM25 is the default similarity in Elasticsearch, with its two tuning settings at k1 = 1.2 and b = 0.75.

One trap: the analyser decides what counts as a word. Elasticsearch's standard tokenizer splits "Brown-Foxes" into "Brown" and "Foxes", so "TS-999" becomes "TS" and "999" and loses some of its precision. Index codes and part numbers in an extra field that keeps them whole.

How reciprocal rank fusion merges the two lists

BM25 scores have no upper limit, while cosine similarity sits between fixed bounds, so adding raw scores rewards whichever method happens to produce bigger numbers. RRF ignores the scores and uses positions. Each passage gets 1/(k + rank) from each list it appears in, and the totals are sorted.

The method comes from a 2009 SIGIR paper by Cormack, Clarke and Büttcher, which fixed k at 60 in a pilot and found that value near optimal, though not critical. A worked example with k = 60:

Passage A, found by both methods, wins clearly. That is the behaviour you want: agreement between the two lists counts for more than a single first place.

The alternative is to normalise scores and blend them with a weight. Weaviate's relative score fusion, the default since v1.24, does this with an alpha setting where 1 is pure vector and 0 pure keyword. Qdrant offers distribution-based score fusion alongside RRF. Blending gives finer control but needs tuning per collection; RRF has a single constant and works from ranks alone.

Then rerank the shortlist

Fusion decides what makes the shortlist; a reranker decides the order. A cross-encoder such as bge-reranker-v2-m3 reads the question and each passage together and outputs a relevance score, which usually ranks better than comparing two separate vectors and is much slower. So run it only on the top 20 to 50 fused results. In Anthropic's tests, contextual embeddings plus BM25 plus a reranker cut the share of queries missing the right chunk in the top 20 from 5.7% to 1.9%, a 67% reduction against their starting point.

Which databases support hybrid search natively?

Most teams can do this inside a vector database or search engine they already run.

Note the different defaults: Qdrant's k of 2 weights first places far more heavily than 60 does. And PostgreSQL's built-in ranking functions, ts_rank and ts_rank_cd, are not BM25, so test them on your own codes and names.

Where to go from here

Two rules apply whichever system you pick. Apply the same permission filter to both the keyword and the vector query, or one leg will leak what the other hides. And measure: keep a set of real questions, including ones with codes and names, and check that the right passage reaches the top 5 before and after each change.

Our guide to semantic search over company documents covers the full pipeline, from OCR to permissions. Our RAG chatbot builds use vector and keyword search together with a reranker by default.

If your assistant answers well about policies but stumbles on part numbers, client names or clause references, tell us what it gets wrong and we will look at the search underneath. Book a consultation.