Retrieval-augmented generation

Also known as: RAG

Illustration of practical fractional leadership playbooks and guides
Definition

Retrieval-augmented generation (RAG) is a way of building AI tools in which a large language model answers questions using information retrieved from a business's own documents or systems, rather than relying only on what it learned in training. It makes answers more accurate, current and specific to the organisation.

What it means

A RAG system has two parts. A retrieval step searches a document store, database or knowledge base for the passages most relevant to a question. A generation step passes those passages to a language model, which writes an answer based on them and can cite where each point came from.

Because the model reads your information at the moment of answering, you do not need to train a new model. Updating the documents updates the answers.

How it works in a UK business

RAG is the most common way UK SMEs and scale-ups build custom AI: internal assistants that answer from policies or contracts, customer service tools that draw on product data, and sales tools that search past proposals. Most of the effort goes into organising and securing the source documents, setting permissions so people only see what they are allowed to, and testing answers on real questions.

Where personal data is involved, data protection rules still apply, so a data protection impact assessment is usually needed before launch.

Frequently asked questions

Is RAG cheaper than training a custom AI model?

Usually, yes. RAG uses an existing commercial model and connects it to your information, so there is no model training cost. The main costs are preparing and structuring the documents, integration, access controls, testing and ongoing usage fees. Training or fine-tuning a bespoke model is generally reserved for specialist prediction tasks.

Does RAG stop AI making things up?

It reduces the risk considerably, because answers are grounded in retrieved documents and can show their sources. It does not remove the risk completely. Answers still need testing before launch, and the system should say when it cannot find relevant information rather than guess.

Last updated: 13 November 2026

Need senior expertise on this?

Our bench of 70+ senior directors supports UK SMEs and scale-ups part-time — from £1,795 per month, starting within one week, with no long-term tie-ins.