RAG

Problem

An LLM knows what it learned during training. It doesn't automatically know your company's current internal documentation. You ask: “What's our PROD deployment procedure?” The answer exists in your private wiki.

Solution — Retrieval-Augmented Generation

Retrieve relevant information first and provide it to the model as context. Typically, documents are split into passages and indexed for search. For each question, the application retrieves relevant passages the user may access and sends them with the question to the model. An Azure implementation can use Azure AI Search to retrieve the passages and an Azure OpenAI model to generate the answer from that context.

Question
   ↓
Search company knowledge
   ↓
Relevant documents
   ↓
LLM
   ↓
Answer

Core idea: RAG lets a model reason over external knowledge supplied at request time.