CyberFreezeDevSearch that understands synonyms
A search for "phone cover" finds items listed as "mobile case".
Read more about Search that understands synonyms
A product catalogue search where "phone cover" finds items listed as "mobile case".
- Home
- AI glossary
- Embeddings
What are embeddings?
Lists of numbers that capture meaning, so software can search and group things by meaning.
The plain definition
An embedding is a vector of hundreds or thousands of numbers. Similar meanings land close together, which is the basis of semantic search and RAG.
Read more about The plain definition
An embedding is a vector: an array of numbers, often a few hundred to a few thousand long. For example, the open model all-MiniLM-L6-v2 outputs 384 numbers per sentence, and OpenAI's text-embedding-3-small outputs 1,536 by default (OpenAI docs).
The numbers on their own mean nothing to a person. What matters is distance: "invoice overdue" and "payment not received" land near each other even though they share no words, while "invoice template" lands further away. That is the basis of semantic search and RAG.
How it works
-
01 Choose an embedding model
Pick a small model trained so related texts get nearby vectors.
Read more about Choose an embedding model
A neural network trained so that related texts get nearby vectors. It is a separate, much smaller model than a chat LLM.
-
02 Embed your content
Run each chunk through the model once and store the vector.
Read more about Embed your content
Each chunk of text (or each image) goes through the model once and you store the vector, usually in a vector database.
-
03 Embed the query
At search time, the question goes through the same model.
Read more about Embed the query
At search time the user's question goes through the same model.
-
04 Compare
Similarity scores rank the stored vectors and the closest ones come back.
Read more about Compare
Cosine similarity or dot product scores how close the query is to each stored vector, and the closest ones are returned.
Where it matters in business
Deduplication
Finds near-duplicate leads, tickets or supplier names spelled differently.
Read more about Deduplication
Finding near-duplicate leads, tickets or supplier names that differ in spelling.
Classification and routing
Routes incoming email by comparing it to labelled examples, without an LLM call each time.
Read more about Classification and routing
Comparing an incoming email to labelled examples to send it to the right team, cheaply and without calling an LLM every time.
A practical example
A store routes each new support email using the tags of the ten most similar past emails, with no model retraining.
Read more about A practical example
An e-commerce store has several thousand past support emails, each tagged by issue. Each is embedded once. When a new email arrives, the system finds the ten most similar past emails and uses their tags to route it, and shows the agent how those earlier cases were resolved. No model is retrained; adding new tagged emails improves routing straight away.
Common mistakes
Mixing models
Vectors from different embedding models do not compare. Change model, re-embed everything.
Read more about Mixing models
Vectors from two different embedding models are not comparable. If you change model, re-embed everything.
Ignoring language
English-only models handle Tamil or Hindi poorly. Test a multilingual model on your data.
Read more about Ignoring language
An English-only model handles Tamil or Hindi text poorly. For multilingual content, pick a multilingual embedding model and test it on your own data.
Trusting similarity for exact facts
Invoice numbers, GSTINs and SKUs need exact lookup alongside vector search.
Read more about Trusting similarity for exact facts
Embeddings are fuzzy. Invoice numbers, GSTINs and SKUs need exact keyword or database lookup alongside vector search.
Questions
Are embeddings the same as an LLM?
No. An embedding model outputs a vector, not text. It is usually far smaller and cheaper to run than a chat model, and many run comfortably on a CPU.
Can embeddings be reversed into the original text?
Not directly, but research has shown that text can be partly reconstructed from some embeddings. Treat stored vectors with the same care as the documents they came from.
Last updated