Retrieval over your own documents
The answer is already in a document nobody is going to open
A site supervisor rings the office to ask whether a particular membrane is approved for a particular substrate. The answer is on page 34 of a manufacturer PDF that was saved to SharePoint in 2019. Somebody spends twenty minutes finding it, or guesses.
That is the problem this solves. Not chat for its own sake — a question, an answer, and a link to the page it came from.
What it is, precisely
The technique is called retrieval-augmented generation. In plain terms: when someone asks a question, the system searches your documents first, pulls back the handful of passages most likely to contain the answer, and hands those to a language model along with the question. The model answers using that material and cites where each part came from.
The model is not taught your business. It is handed the relevant pages at the moment of asking. That distinction matters for everything downstream: how you update it, whether it can be wrong, and whether you can check its work.
The practical test: if the answer cannot come with a link to the paragraph it came from, this is the wrong tool and you should be suspicious of whoever is selling it to you.
Retrieval, fine-tuning, or just a long prompt
These three get conflated constantly, including by vendors. They solve different problems and the right answer is often a combination.
| Feature | Retrieval (RAG) | Fine-tuning | Long context | |
|---|---|---|---|---|
| Where the knowledge lives | In documents, read at question time | Baked into model weights | Pasted into the prompt each time | |
| Updating it | Re-index the changed file, minutes | Retrain, days and real money | Change the paste, instant | |
| Can it cite a source | Yes, that is the point | No | Yes, but only what fitted | |
| Scale of corpus | Millions of pages | Whatever you trained on | Hard ceiling per request | |
| Respects who may see what | Yes, filter at retrieval | No, it is all in the weights | Only if you pre-filter | |
| Good for | Facts that change and must be attributable | Tone, format, specialist reasoning | Small, stable reference sets |
Most of the work we are asked for is retrieval, because most of the time the problem is “we cannot find our own information” rather than “the model does not know our field”.
What people actually build
A support desk that stops asking the same person
Product manuals, past ticket resolutions and internal notes, searchable in one place. The measurable outcome is usually not deflection rate, it is that the one engineer who knows everything stops being interrupted eleven times a day.
Contract and policy lookup
“What is our notice period with this supplier?” answered against the executed agreements rather than someone’s memory of the negotiation. Citations matter enormously here because the answer may end up in a dispute.
Onboarding that does not consume a senior person
New staff ask the system the questions they are embarrassed to ask a human twice. Unglamorous, and consistently the one people report as most useful six months in.
Tender and bid response
Pulling accurate past answers, capability statements and referee details out of previous submissions. Saves days per bid, and the citation trail means you can verify nothing was invented before it goes out.
Before you commission anything
Read the page about where this goes wrong. It is the most useful thing on this site and it will tell you whether your document set is in a fit state for it yet. A fair number of the enquiries we get end with us saying “fix your file structure first”, which is free advice and usually correct.
Where RAG goes wrong