How Do ChatGPT, Perplexity, Gemini and Claude Choose Which Sources to Cite?
By The GEO Pilot · Published
Short answer
AI assistants that cite sources generally retrieve web information, use it to ground an answer, and link pages they judge relevant. No engine publishes its full ranking formula. OpenAI, Perplexity, Google and Anthropic each document their crawlers, and those controls shape visibility. Publishers should be crawlable, indexed, and the clearest answer.
What is retrieval-augmented generation (RAG) in plain language?
Retrieval-augmented generation (RAG) is a way of answering with information fetched for the question, instead of relying only on what a model learned in training. The system retrieves potentially relevant material, uses it while writing the response, and may cite the pages it retrieved.
RAG is a useful mental model for AI citations, but it is not a universal pipeline. Each product retrieves, ranks, and cites differently, and those details are not fully disclosed. A citation is also just a link attached to an answer. It does not prove the page was the only basis for what the model said.
At GEO Pilot, we use RAG as a teaching model, not a rulebook. It explains why retrievable, clearly written pages get a chance to be cited. It does not tell you how any single engine scores them. For wider context, see our guide to generative engine optimization and our GEO vs. SEO explainer.
How does ChatGPT choose sources?
OpenAI says ChatGPT Search may use partner search providers and typically rewrites a user's prompt into one or more targeted queries. OpenAI does not publish a complete source-ranking formula.
According to OpenAI's ChatGPT Search help page:
- ChatGPT Search may partner with other search providers and typically rewrites a user's query into targeted queries sent to those providers.
- Answers that use web search may include citations. Users can open cited sources or view a Sources panel when available.
- Websites should allow OAI-SearchBot to crawl them to be eligible for inclusion in ChatGPT Search.
- The host or CDN must also allow traffic from OpenAI's published searchbot IP addresses.
The last two points are easy to miss. You can allow a bot in robots.txt and still block it at the firewall or CDN. If you are researching how ChatGPT chooses sources, check both layers. Our guide on how to get cited by ChatGPT covers the practical steps.
How do Perplexity citations work?
Perplexity citations are links to web pages used or surfaced in a Perplexity answer. According to Perplexity's crawler documentation, PerplexityBot is designed to surface and link websites in Perplexity search results.
Perplexity makes two further points:
- PerplexityBot is not used to crawl content for AI foundation models.
- Publishers should allow PerplexityBot in
robots.txtand permit requests from Perplexity's published IP ranges.
That separation matters for publishers weighing access decisions. Perplexity describes this crawler as serving search discovery, not model training. The documentation does not specify a fixed formula or ranking weights for which pages become citations.
What do we know about Gemini sources?
Google documents that its Google-Extended control governs Gemini training and grounding, but not Google Search. That distinction decides how you should think about Gemini sources and AI Overviews.
According to Google's common crawlers documentation:
- Google-Extended controls whether your content is used to train Gemini models.
- It also controls use of your content for grounding in Gemini Apps and in Grounding with Google Search on Vertex AI.
- It does not affect inclusion or ranking in Google Search.
Because Google-Extended does not touch Search inclusion, AI Overviews follow normal Search indexing. Whether a page is indexed in Google Search is the question that matters for AI Overviews. Your Google-Extended setting does not change it.
This page describes crawler controls, not a citation formula. Google does not publish a ranking formula for Gemini citations here, so we won't invent one.
What do we know about Claude web search?
Anthropic documents three separate crawlers, and blocking two of them may reduce your visibility in Claude answers. Each bot has a different job, so you can make a different decision for each.
According to Anthropic's support documentation:
- ClaudeBot collects training data.
- Claude-User fetches pages when a user asks Claude to.
- Claude-SearchBot indexes content to improve search results.
Anthropic says blocking Claude-User or Claude-SearchBot may reduce your visibility in Claude answers. Blocking ClaudeBot is a separate training-data decision.
Names alone can mislead. Here, Anthropic's own descriptions split training, user-requested fetching, and search indexing into three bots. We think that is the right standard for the whole industry. Much of what circulates about AI citations is observed behavior presented as product rules. Observation is valuable, but it should be labeled as observation.
How do the four engines compare on documentation?
OpenAI, Perplexity, Google and Anthropic each document crawler controls, but none publishes a complete citation formula. The table shows what each vendor says and what remains unpublished.
| Engine | Crawlers named | What the vendor documents | What is not published |
|---|---|---|---|
| ChatGPT (OpenAI) | OAI-SearchBot | May use partner search providers; rewrites prompts into targeted queries; answers may include citations and a Sources panel; allow OAI-SearchBot and its published IPs for eligibility | A complete source-ranking formula |
| Perplexity | PerplexityBot | Crawler designed to surface and link sites in search results; not used to crawl for AI foundation models; allow it in robots.txt and permit its published IP ranges | A fixed formula or ranking weights for citations |
| Gemini (Google) | Google-Extended | Controls use of content for Gemini model training and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI; does not affect inclusion or ranking in Google Search; AI Overviews follow normal Search indexing | A citation formula for Gemini answers |
| Claude (Anthropic) | ClaudeBot, Claude-User, Claude-SearchBot | ClaudeBot collects training data; Claude-User fetches pages when a user asks Claude; Claude-SearchBot indexes content to improve search results; blocking the last two may reduce visibility in Claude answers | A complete source-selection formula |
How differently do the engines cite sources in practice?
Very differently. In October 2026, GEO Pilot's visibility tracking sent the same 10 buyer questions about GEO and content tools to ChatGPT, Perplexity, Gemini, Claude and Grok, and recorded every source each engine retrieved or cited for its answer, using each engine's API with web search turned on. This is observed behavior from one small run, not vendor documentation, and the tracker stores at most 40 sources per answer.
| Engine | Average sources retrieved or cited per answer |
|---|---|
| ChatGPT | 34.8 |
| Grok | 31.5 |
| Claude | 21.8 |
| Perplexity | 11.0 |
| Gemini | 4.4 |
Two patterns stood out. ChatGPT surfaced reddit.com pages 81 times, in 8 of its 10 answers, more than any other domain in the whole run, while the other four engines surfaced no Reddit pages at all. Gemini surfaced the fewest sources, and its links pointed to Google redirect URLs rather than showing the publisher's domain directly. The practical lesson: an engine that pulls in 35 sources per answer and one that pulls in 4 create very different levels of competition, so measure each engine separately.
Does allowing a crawler guarantee AI citations?
No. Allowing a crawler removes one barrier to access. The documentation from OpenAI and Perplexity describes access and discovery, not a guaranteed citation outcome.
A crawl does not promise indexing. Indexing does not promise selection for an answer. Selection does not promise a visible link. Each step is a separate gate.
At GEO Pilot, we treat crawler access as a prerequisite, not a citation strategy. A page still has to be useful, readable, and relevant to the question it is meant to answer.
What is the Three-Gate Citation Checklist for earning AI citations?
The Three-Gate Citation Checklist is our framework for publishers: Access, then Eligibility, then Answer clarity. It turns "be crawlable, be indexed, be the clearest answer" into steps, and each gate has to be open before the next one matters.
Gate 1: Access (be crawlable)
- Allow the crawlers each vendor documents: OAI-SearchBot for ChatGPT Search and PerplexityBot for Perplexity.
- For Claude, decide on each bot separately. Anthropic says blocking Claude-User or Claude-SearchBot may reduce visibility in Claude answers. ClaudeBot is a training-data decision.
- Check
robots.txtand your host or CDN rules. OpenAI and Perplexity both say published IP ranges must be permitted. - Treat Google-Extended as a Gemini training and grounding decision. Per Google, it does not affect Search inclusion.
Gate 2: Eligibility (be indexed and interpretable)
- Make sure important pages are indexed in Google Search. AI Overviews follow normal Search indexing.
- Make pages easy to discover, with clean internal links and a clear URL structure.
- Make them technically readable. Structured data such as JSON-LD schema and an llms.txt file help machines interpret what a page is about. We treat these as clarity aids, not as documented ranking signals.
Gate 3: Answer clarity (be the clearest answer)
- Put a direct, self-contained answer under a question-shaped heading.
- Support it with specific, sourced facts and clear context.
- Never present an assumed ranking factor as a confirmed rule.
To keep yourself honest, add one rule we call the Claim Ledger. Before publishing any statement about how an engine behaves, tag it as documented (a vendor says so, with a link), observed (you saw it, and you say so), or unverified (leave it out). This article follows that rule. Every engine claim above is attributed to a vendor or marked as unpublished.
Gate 3 is where most of your controllable advantage sits, because you can't change a vendor's retrieval system. We recommend measuring visibility rather than assuming it.
GEO Pilot is built around that workflow. It turns source material into 12 content formats, scores each piece from 0 to 100 across nine signals, generates JSON-LD schema and llms.txt support, and tracks AI visibility across ChatGPT, Perplexity, Gemini, Claude, and Grok. These are product capabilities, not guarantees of placement or citation. You can see how GEO scores work, and our GEO glossary defines the terms used here.
Key takeaways
- RAG means an AI system retrieves material for the question, uses it to write the answer, and may cite what it retrieved. Details differ by product.
- OpenAI says ChatGPT Search may use partner search providers and rewrite prompts into targeted queries. Allow OAI-SearchBot and its published IPs for eligibility.
- Perplexity says PerplexityBot surfaces and links sites in search results and is not used to crawl for AI foundation models.
- Google says Google-Extended controls Gemini training and grounding but does not affect Search inclusion or ranking. AI Overviews follow normal Search indexing.
- Anthropic documents three bots: ClaudeBot (training), Claude-User (user-requested fetches), and Claude-SearchBot (search indexing). Blocking the last two may reduce visibility in Claude answers.
- Crawler access is necessary but never a guarantee of citation.
- Work through three gates in order: Access, Eligibility, Answer clarity.
- Separate what vendors document from what you observe, and measure your visibility instead of assuming it.
Frequently asked questions
How does ChatGPT choose which sources to cite?
OpenAI says ChatGPT Search may use partner search providers and rewrite a prompt into targeted queries. Answers that use web search may include citations. OpenAI does not publish a complete ranking formula, and it says sites should allow OAI-SearchBot to be eligible for inclusion.
Is PerplexityBot used to train AI models?
No, according to Perplexity. It describes PerplexityBot as a crawler designed to surface and link websites in Perplexity search results, and states it is not used to crawl content for AI foundation models. Perplexity recommends allowing it in robots.txt.
Does blocking Google-Extended remove my site from Gemini or AI Overviews?
Google says Google-Extended controls use of your content for Gemini model training and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI. It does not affect inclusion or ranking in Google Search. AI Overviews follow normal Search indexing, so Google-Extended is not the control for them.
What is the difference between ClaudeBot, Claude-User and Claude-SearchBot?
Anthropic says ClaudeBot collects training data, Claude-User fetches pages when a user asks Claude, and Claude-SearchBot indexes content to improve search results. Anthropic says blocking Claude-User or Claude-SearchBot may reduce your visibility in Claude answers.
What should I optimize first to earn AI citations?
Start with access, then eligibility, then clarity. Make sure intended crawlers can fetch your key pages, make sure the pages are indexed and easy to parse, and answer a specific question directly. These are practical publishing principles, not a guarantee of citation.
Make your content the source AI cites
Turn any draft, link or recording into GEO-optimized formats, scored and ready to publish. Start with $5 of free credit.
Start free