Your AI doesn't know
your documents. RAG fixes that.
A plain-language walkthrough of how RAG lets a language model search your own data before it answers — with every stage shown as a diagram, not a wall of text.
A model can't answer questions about data it never saw
Ask a plain LLM about your college's attendance policy, and it guesses — because your policy PDF was never part of its training. RAG gives it a way to look that up first.
Every RAG system is built from two moving parts
One phase happens once, ahead of time. The other happens every time someone asks a question.
Indexing
Done in advance. Your documents are loaded, cut into chunks, converted to embeddings, and stored in a vector database — so they're ready to be searched later.
Retrieval + Generation
Done at question time. The user's question is embedded, matched against stored chunks, and the best matches are handed to the LLM as context for its answer.
Getting a document ready to be searched
You never paste a 100-page PDF into a prompt. Instead, it's broken down step by step.
Why documents get cut into pieces
Retrieval works at the chunk level — so a giant document becomes several small, independently searchable pieces.
Turning meaning into numbers
An embedding model converts a sentence into a list of numbers that captures what it means. Sentences with similar meaning end up with similar numbers.
→ similar vectors
Where all those embeddings live
Every chunk sits in the database next to its embedding, ready to be compared against future questions.
| Chunk | Embedding (shortened) |
|---|---|
| Leave policy | [0.21, -0.43, 0.87, …] |
| Salary policy | [0.72, 0.05, -0.18, …] |
| Insurance policy | [0.11, 0.60, 0.02, …] |
| Work-from-home policy | [0.55, -0.22, 0.39, …] |
Common choices: Pinecone, Qdrant, Weaviate, Chroma, Milvus, or pgvector on Postgres.
Finding the right chunk for a question
The question gets embedded too, then compared against everything in the database. Closest matches win.
| Result | Similarity |
|---|---|
| Leave Policy | 95% |
| Employee Handbook | 81% |
| Salary Policy | 22% |
Handing the answer key to the LLM
The top chunks get pasted into the prompt as context, right alongside the user's original question.
CONTEXT: "Employees receive 24 paid leaves per year. Leave must be requested through the HR portal."
USER: How many paid leaves do employees get?
The complete pipeline, end to end
Indexing happens once, offline. Retrieval and generation happen live, every time someone asks something.
RAG doesn't retrain the model — it feeds it
Update your documents anytime; the underlying LLM never needs retraining.