Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (RAG) is a method in which a language model first retrieves relevant documents from an external source, such as a web index, and then writes its answer from those documents.

Timothé Merle

TL;DR

Key Definition

Retrieval-augmented generation (RAG) pairs a search engine with a language model. The system finds relevant passages and slips them into the model context, and the model then generates an answer grounded in them. The term comes from a paper by Lewis and co-authors (Facebook AI Research, 2020). This mechanism is what allows ChatGPT Search, Perplexity or Google AI Overviews to cite web pages.

Importance

Why Does It Matter?

  • An AI's training knowledge stops at a fixed date. RAG explains how it can still cite a page published last week.
  • A page the system doesn't retrieve will never be cited. Being crawlable and ranking well in the index it uses is still the starting condition for AI visibility.
  • The model reads passages, not whole pages. Content where each section clearly answers a specific question starts with an edge.
  • The citations AI engines display are the visible part of the retrieval step. A brand can measure them and act on them.
  • Once you understand RAG, you stop betting on tricks that do nothing and put the effort into crawler access, relevance and passage clarity.

How Does It Work?

A RAG system runs three steps in a row, retrieval, context augmentation and generation. AI search engines apply this pattern to the entire web, often firing several searches for a single question.

  1. 1

    Retrieval

    The question is converted into one or more queries sent to an index. That index can be a web search engine, a vector database built with embeddings, or both at once. According to Google, AI Overviews and AI Mode may use a "query fan-out" technique that fires off several related searches on subtopics of the question. The top results are then cut into passages and ranked by relevance.

  2. 2

    Context Augmentation

    The selected passages go into the model prompt, along with their URL and title. From each page, the model sees only an excerpt, capped by the size of its context window. A passage that leans on the previous paragraph, or on a table rendered as an image, loses its meaning once isolated. It's then less likely to be used.

  3. 3

    Generation and Citation

    The model writes its answer from the passages provided, then ties some sentences to their source. In the OpenAI API, these links take the form of url_citation annotations holding the URL, the title and the position of the cited text. WebGPT (OpenAI, 2021) already showed a model that could browse the web and gather references to back its answers. Citation flows directly from retrieval.

  4. 4

    Limits of RAG

    RAG cuts errors down but doesn't wipe them out. The system can land on an outdated page, misread a passage or pin a sentence on a source that doesn't support it. Answer quality depends as much on the retrieved pages as on the model itself.

Metrics

How Do You Measure It?

  • Dig through your server logs to spot retrieval crawlers such as OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User and Claude-User, along with the pages they request.
  • Check in Google Search Console that your strategic pages are indexed. AI Overviews pull their sources from the Google index.
  • Run a prompt set across several AI engines and record the cited URLs. You will see which pages make it through the retrieval step.
  • Put the cited passage side by side with your page to see which paragraph was extracted and whether it was reproduced faithfully.
  • Hikoo's Agent site scores how readable your pages are for AI, and a free audit is available at /tools/free-audit.

Pitfalls

What Are the Common Mistakes?

  • Assuming an AI knows your recent content from training alone. Without retrieval, it cannot see your new pages.
  • Blocking AI search crawlers in robots.txt or at the firewall. The retrieval step never happens, and no citation is possible.
  • Writing sections that refer back to context ("as explained above") and make no sense once extracted.
  • Hiding important information in images, JavaScript-loaded tabs or heavy PDFs, which are harder to extract as text.
  • Mixing up RAG and model training. Blocking a training crawler such as GPTBot does not remove your pages from ChatGPT search.

What Should You Do First?

Follow these steps to get started.

  • Allow, in robots.txt and at the firewall, the AI search crawlers you want citations from.
  • Check that your important pages are indexed in Google and Bing, which feed part of the retrieval systems.
  • Write each section so it answers a question on its own, naming the subject in the first sentence.
  • Put important facts in HTML text, not in images or downloadable files.
  • Date your figures and keep them current. An outdated passage eventually gives way to a fresher source.
  • Record the cited URLs on a prompt set every month to see which pages get retrieved.

Examples

One Question, Several Searches

Illustrative example. A user asks "which fleet management software for 300 vehicles". The engine runs searches on features, pricing and reviews, retrieves about ten passages and writes an answer that cites three pages. A vendor that never shows up in those searches has no chance of appearing, however good its product.

Recent Page Cited

Typical scenario. A company publishes a page about a new regulation. A few days after indexing, Perplexity cites it on related questions, even though the underlying model never saw the page in training. The retrieval step is what put it within reach.

Frequently Asked Questions

  • What is RAG in artificial intelligence?

    RAG, or retrieval-augmented generation, is a technique in which a language model fetches documents from an external source before answering. The passages it finds join the model context, and the model uses them to write a grounded answer. Lewis and co-authors formalized the method in 2020.

  • How is RAG related to AI search engine citations?

    Citations are the visible side of RAG. The engine retrieves web pages, generates its answer from their passages, then shows links to the pages it used. Without retrieval, a model answers from memory and usually cannot cite any specific page. Before you can be cited, you have to be retrieved.

  • Does ChatGPT use RAG?

    Yes, when web search is on. ChatGPT then sends queries, retrieves pages through crawlers such as OAI-SearchBot and ChatGPT-User, and writes an answer with clickable citations. Without web search, it answers from its training knowledge, which stops at a given date and holds no verified links.

  • How do I optimize content for RAG?

    Start by opening your pages to AI search crawlers and getting them indexed. Then write self-contained sections, each built around one question, with the subject named, precise and dated facts, and HTML text rather than images. A passage that still makes sense once extracted gets used and cited more often.

  • What is the difference between RAG and fine-tuning?

    Fine-tuning changes the model weights by retraining it on new data. RAG leaves the model untouched and hands it documents at question time. With RAG, you can use fresh information and cite where it came from. Fine-tuning is more about adjusting the model's style or specialty.

Sources

  1. 1arXiv Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)
  2. 2arXiv WebGPT, Browser-assisted question-answering with human feedback (Nakano et al., 2021)
  3. 3OpenAI Platform Docs Web search (tools guide)
  4. 4Google Search Central AI features and your website

About the author

Timothé MerleTimothé MerleCo-founder of Hikoo, AEO and GEO expert
View LinkedIn profile

Related Content

Test your site's AI visibility in 30 seconds.

GEO score, technical health, and an overview of your visibility on ChatGPT and Gemini.

Run my free auditStart for free