Answers from your own documents, with the sources to check them.
Retrieval-augmented generation (RAG) connects a language model to your organisation’s material at the moment a question is asked: the system finds the most relevant passages, and the model answers from those passages and cites them.
Most of the work — and most of the quality — sits in the pipeline around the model: which sources are included, how they are parsed and indexed, who may see what, and how retrieval is tested.
01The problem
The answer exists somewhere. Finding the current one takes too long.
Knowledge in most organisations is spread across shared drives, wikis, ticket histories, manuals and inboxes. People learn to ask whoever has been there longest, and outdated copies keep circulating.
A knowledge system shortens that path — provided it shows where each answer came from, respects who may read what, and says plainly when the sources do not contain an answer.
Potential use case Examples to test against your own sources and questions.
Policy and procedure questions
Staff asking what the current procedure says, with the answer linked to the clause and revision it came from.
Technical documentation
Engineers and support teams searching manuals, specifications and release notes in plain language.
Support history
Finding how similar issues were resolved before, across tickets and notes the person asking is allowed to read.
Project records
Locating decisions, approvals and correspondence across a project archive without knowing which folder to open.
02How it works
Two pipelines around one index.
RAG is not “documents in a vector database”. Ingestion decides what can be found; the query path decides what each person is allowed to receive and how the answer is grounded.
Ingestion — runs when sources change
- InputSource inventoryOwners, formats, access rules
- ToolParse & OCRLayout, tables, scanned pages
- ToolChunk & enrichStructure-aware, with metadata
- StoreIndexKeyword + vector, with permissions
Query — runs for every question
- InputQuestionFrom a signed-in user
- CheckAccess filterOnly passages they may read
- ToolHybrid retrievalExact terms and meaning
- ToolRerankWhere it measurably helps
- ModelGrounded answerFrom retrieved text only
- OutputAnswer + citations↳ or: “not enough evidence”
- Input
- Tool
- Check
- Store
- Model
Inventory and permissions
List each source, its owner, how often it changes and how access is controlled. Permissions travel into the index so retrieval can be filtered per user — the model is never relied on to withhold something it has been shown.Parsing and OCR
Digital text is extracted with its structure; scanned pages go through OCR; tables stay tables. Parsing errors here are the most common cause of poor answers later.Chunking and metadata
Documents are split along their own headings and sections rather than at arbitrary lengths, and every chunk carries its source, section, revision, date and access attributes.Indexing
Chunks are indexed for keyword search — exact terms, part numbers, clause references — and for vector search by meaning, often in the same engine.Retrieval and reranking
Hybrid retrieval combines both signals. A reranking model re-orders the top results only when tests show the gain in relevance justifies its added latency and cost.Grounded generation
The model is instructed to answer only from the supplied passages, cite them, and say when they do not contain the answer.Freshness and versions
Changed or withdrawn documents are re-indexed or removed on a schedule, and superseded revisions are excluded or clearly marked as such.
03RAG or fine-tuning?
For private, changing knowledge, retrieval usually wins.
The two solve different problems and can be combined: a tuned model can still retrieve. The decision process for tuning is on Model Adaptation & Fine-tuning.
| Aspect | RAG | Fine-tuning |
|---|---|---|
| Best at | Answering from specific, changing or private material | Consistent behaviour, format or labelling |
| When knowledge changes | Re-index the source; the next question sees it | Retrain and re-evaluate the model |
| Traceability | Each answer can cite the passages it used | No built-in link from an output to its source |
| Access control | Enforced per user at retrieval time | Whatever was learned in training is available to every user |
| Typical failure | The right passage is not retrieved | Outdated or invented facts stated confidently |
04Limits and security
Where RAG falls short, and how the data stays protected.
Limitations
- If retrieval misses the passage that holds the answer, the answer will be incomplete or wrong — however capable the model.
- A model can still misstate or over-generalise a passage it was given. Citations make that checkable, not impossible.
- Questions that need calculation across many records — “total spend by supplier last year” — belong in a database query or report, not a text search.
- Contradictory or outdated sources produce contradictory answers until the sources themselves are cleaned up.
Security implications
- Permission filtering happens during retrieval, before generation.
- Documents can contain hidden instructions, so retrieved text is treated as untrusted input rather than as commands.
- Indexes and embeddings are derived copies of your data and inherit its protection, retention and deletion rules.
- Logged prompts, passages and answers may hold confidential material; logs are access-controlled and kept only as long as agreed.
When it is not the right tool
- If people mainly need to find a document rather than an answer, a well-configured enterprise search may serve them better and cost less.
- If the sources are thin, unowned or badly out of date, the first project is the content, not the AI.
05How quality is evaluated
Retrieval and answers are tested separately.
A good answer from bad retrieval is luck; a bad answer from good retrieval is a generation problem. Measuring both shows which part to fix. Definitions follow public guidance such as Microsoft’s RAG evaluators (external site).
- Retrieval
- On a labelled set of real questions, do the passages containing the answer come back, and near the top?
- Groundedness
- Is every claim in the answer supported by the passages it cites?
- Answer quality
- Relevance and completeness against reference answers reviewed by people who know the subject.
- Citations
- Do the citations point to the passages actually used?
- Abstention
- On questions the sources cannot answer, does the system say so instead of guessing?
- Access
- Test users with different permissions receive only what they are entitled to see.
06Related
Related services.
Next step
Start with the questions people already ask.
Bring a sample of real questions and the sources that should answer them. That is enough to judge whether retrieval will work, and what preparation the documents need first.