CyberFreezeDev
Loading
Search your own documents first, then the model answers from what was found, with sources.
Retrieval finds the relevant passages in your content; generation hands them to a model to answer and cite, with no retraining needed.
Retrieval-augmented generation (RAG) combines two steps. Retrieval: find the passages in your own content that relate to a question. Generation: hand those passages to a large language model with an instruction like "answer using only the text below and cite it".
The term comes from a 2020 research paper by Lewis and colleagues. Today it is the standard way to build a chatbot or assistant over company policies, manuals, contracts or a product catalogue, without retraining any model.
Documents are converted to text and split into chunks of a few hundred words.
Documents (PDF, DOCX, HTML, spreadsheets, scanned files after OCR) are converted to text and split into chunks of a few hundred words.
Each chunk becomes an embedding in a vector database, with source, date and access group.
Each chunk is turned into an embedding and stored in a vector database, usually with metadata such as source file, date and access group.
The question is embedded the same way and the closest chunks are fetched.
A user question is embedded the same way, and the closest chunks are fetched. Many systems also run keyword search and merge the two result lists.
The model answers from the top chunks and points to the ones it used.
The top chunks go into the prompt with the question. The model writes the answer and points to the chunks it used.
Staff ask a policy question and get the answer with a link to the clause.
Staff ask "what is our refund policy for B2B orders?" and get the answer with a link to the exact clause.
An assistant answers from current product documents and hands over when it finds nothing.
A website or WhatsApp assistant that answers from current product documents, and hands over to a person when it finds nothing.
Edit a document and the answers change at the next re-index, with no retraining.
Updating a document updates the answers on the next re-index. With fine-tuning you would have to train again.
A clinic's assistant answers aftercare questions by quoting its own PDFs with file names, and follows edits after the nightly re-index.
A clinic keeps treatment guides, price lists and aftercare sheets as PDFs. A RAG assistant indexes them. When the front desk asks "what aftercare do we give after a root canal?", the system retrieves the two relevant sections and the model writes a short answer quoting them, with file names. When the clinic edits a PDF, the index is rebuilt overnight and answers follow the new version.
Chunks that split tables or clauses hide the full fact. Chunk along headings.
Splitting tables or clauses in half means the model never sees the full fact. Chunk along headings and keep tables whole.
Filter by user permissions at retrieval, or anyone can pull HR files.
If HR files share an index with sales notes, anyone can retrieve them. Filter by user permissions at retrieval time.
Without real questions and correct answers, you cannot tell if a change helped.
Without a fixed list of real questions and correct answers you cannot tell whether a change helped. See model evaluation.
It reduces it a lot when retrieval finds the right passages, but it does not remove it. Ask the model to cite sources, tell it to say "not found" when the context is empty, and test with questions that have no answer in your documents.
Use RAG when the model needs facts from your documents. Use fine-tuning when it needs a consistent style, format or narrow skill. Many production systems use RAG alone.
Yes. The embedding model, vector database and LLM can all run on your own hardware. See private AI deployment.
Last updated