What Is RAG? Follow One Question from Source to Answer
Retrieval-augmented generation, or RAG, is a way to give a language model relevant source material when it answers a question. The application finds evidence, puts selected passages into the model's input, and asks it to answer from that evidence. Adding a document to a RAG system does not, by itself, retrain the model.
Consider an employee asking, “How many days do I have to submit a travel receipt?” The answer may change when the expense policy changes. RAG can bring the current policy into the response, but only if the application retrieves the right version and the model uses its conditions correctly.
What happens before the question arrives?
The ingestion path reads the policy, preserves its source identity, and prepares searchable records. A long policy might be split into passages. Each passage should retain its document, section, version, and access scope. An embedding can help match a paraphrased question; a lexical index can help match an exact policy number.
Indexing is preparation, not proof of answer quality. A scanned table may have been extracted incorrectly. A passage may be searchable while the user lacks permission to read it. Both problems must be resolved before supplying the text as evidence.
What happens when someone asks a question?
The application first resolves who is asking and which sources they may access. It retrieves candidate passages, optionally changes their order, and selects enough context to answer the question. The model then receives the question and selected evidence.
For the receipt example, retrieving “submit within 30 days” is insufficient if the next sentence says “international trips require submission within 14 days.” The selected context must preserve the relevant condition. A citation should point to the passage that supports the answer, not merely the policy homepage.
Why can a RAG answer still be wrong?
There are several different failures to diagnose:
| Observed result | Boundary to inspect |
|---|---|
| The current policy is absent | Ingestion and indexing |
| Another department's policy appears | Access scope and filtering |
| The correct passage ranks too low | Candidate selection and ranking |
| The passage is present but its exception is ignored | Context assembly and generation |
| No policy answers the question | Missing evidence, not necessarily a retrieval bug |
A plausible response does not tell you which boundary worked. Inspect the evidence that reached the model. A model may answer from prior knowledge even when your retrieval path returned nothing useful.
Where should you start?
Build a small question set with exact answers, conditional answers, a revised policy, and questions the documents cannot answer. Record the acceptable passage for each answerable question. Compare source support and final answers separately before adding more retrieval components.
When one retrieval path cannot answer all question types, the query-routing guide explains how to introduce a deliberate branch.
Compare semantic chunking, embedding compatibility, and the vector workload cost model when designing the retrieval stages.
The RAG chatbot implementation guide develops that pipeline. For a managed option, review Supermemory's search API, then start with a small document set in Supermemory. The first useful result is one authorized question answered with inspectable evidence.