RAG

RAG (Retrieval-Augmented Generation) answers questions by first retrieving relevant chunks from your documents, then generating an answer conditioned on them.

The pipeline:

1. Chunk documents into overlapping pieces.
2. Embed each chunk into a vector.
3. Store vectors in a vector database.
4. Retrieve the top-k chunks for a question.
5. Generate an answer with the LLM, citing the chunks.

Why it exists: models cannot know your private data, and training/fine-tuning on it is expensive and slow. RAG gives up-to-date, private, citable grounding with a cheap, local stack.

Common failure modes: bad chunking, weak embeddings, or a model that ignores the context. Start with a minimal stack and evaluate on real questions.

See the guide: Build a RAG pipeline with local models.