Glossary · simply explained

RAG (retrieval-augmented generation)

RAG connects LLMs with your own knowledge: answers from your documents, with sources — without retraining the model.

Retrieval-augmented generation (RAG) is the architecture pattern that lets LLMs speak reliably about your own company knowledge: before answering, the system searches matching passages from your own sources — documents, wikis, tickets, databases — and hands them to the model as context. The model phrases, the retrieval supplies the facts.

RAG thus solves the three core problems of pure LLM use: outdated training knowledge, missing internal knowledge and hallucinations — and makes answers verifiable via citations, without sensitive data flowing into model training.

How a RAG pipeline works

Preparation: documents are split into sections (chunking), translated into vectors by an embedding model and indexed in a vector database. Query: the user question is embedded likewise, the semantically most similar passages are found — often combined with classic keyword search (hybrid search) and re-ranking of hits.

Generation: the best passages go into the prompt together with the question; the LLM answers exclusively on this basis and cites the sources. Quality stands and falls with retrieval — chunking strategy, index freshness and clean source curation decide more about success than model choice.

What matters in enterprise use

Pass through permissions: the system may only find what the asking user would be allowed to see.
Mandatory citations: every answer with evidence — otherwise the hallucination question stays open.
Index maintenance: outdated and contradictory documents spoil any answer quality.
Mind the injection risk: indexed documents too can contain manipulative instructions.

Frequently asked questions about RAG (retrieval-augmented generation)

What is the difference between RAG and fine-tuning?

RAG hands the model knowledge at runtime — current, traceable, instantly changeable. Fine-tuning changes the model itself and suits style, format and domain language, not facts: it is more expensive, goes stale and cannot cite knowledge. In practice both complement each other.

Does RAG prevent hallucinations completely?

No, but it reduces them considerably: the model can still summarise context wrongly or fill gaps creatively — especially when retrieval finds nothing suitable. Antidotes: strict answer-only-from-context prompts, I-don’t-know paths and mandatory citations.

What role does the vector database play?

It stores the embeddings and finds semantically similar passages in milliseconds — even across millions of documents. Options range from specialised systems via extensions of classic databases (pgvector, say) to managed services; for many mid-market setups the latter fully suffices.

How does a RAG system stay privacy-compliant?

The knowledge base stays within your own sphere of control — decisive are permission inheritance from source systems, processing location of embedding and language model (EU hosting or own infrastructure) and DPAs for all involved services. That is exactly why RAG is often the most GDPR-friendly path to enterprise AI.

When is RAG the wrong tool?

When no documented knowledge base exists, the question demands calculation instead of lookup (for that: tool use/agents), or it is about consistent style instead of facts (for that: fine-tuning). RAG shines at question answering over curated, access-protected document sets.