How ChatGPT, Perplexity, Copilot and Gemini Each Find Your Store

ChatGPT, Perplexity, Gemini, Copilot and Claude each read the web through their own search crawler or through a search index. A page that crawler cannot reach has nothing to offer that assistant. The five differ in which crawler you allow and whether the platform reports AI impressions or citations to you. None of them publishes how it chooses a source.

One strategy covers all five, with a few platform checks on top. The table below gives those checks, and every cell comes from the platform's own documentation, read on November 9, 2026, or notes where that documentation is silent.

How do the five assistants differ?

What does each platform say about its crawlers?

OpenAI says OAI-SearchBot surfaces websites in ChatGPT search, and sites that opt out are not shown in search answers. The details, including the firewall step, are in how to get your website to show up in ChatGPT.

Perplexity says PerplexityBot is designed to surface and link websites in its search results and is not used to crawl content for AI foundation models. Perplexity-User fetches a page when a user asks, and since a user triggered the fetch, Perplexity says it generally ignores robots.txt. Perplexity publishes both IP lists, at perplexity.ai/perplexitybot.json and perplexity.ai/perplexity-user.json. Our guide to how Perplexity picks products covers the shopping side.

Google describes grounding as providing content from the Google Search index to the model at prompt time. It also says Google-Extended governs whether your content may be used for training future Gemini models and for grounding in Gemini Apps. Google adds that Google-Extended does not affect inclusion in Google Search and is not a ranking signal there. So a site that blocks Google-Extended to stay out of training also keeps its content out of grounded Gemini answers.

Microsoft's AI Performance report in Bing Webmaster Tools shows how often a site is cited across Microsoft Copilot, AI summaries in Bing and select partner integrations. It counts total citations, the average number of your pages cited per day, citations per page and a sample of grounding queries. Our guide to how Copilot and Bing find a site covers the rest.

Anthropic runs three bots. ClaudeBot collects content that may be used for training, Claude-User retrieves a page when a person asks Claude a question, and Claude-SearchBot navigates the web to improve search results. Anthropic says disabling Claude-SearchBot prevents its system from indexing your content for search. Disabling Claude-User prevents Claude from retrieving your pages for a user, and Anthropic says both may reduce your visibility. Anthropic also says its bots honor robots.txt.

What should I fix first?

Start with access, since every platform documents it and none documents its ranking. Work through these steps in order.

1. Open yoursite.com/robots.txt and check the groups for OAI-SearchBot, PerplexityBot, Claude-SearchBot and Claude-User. If a crawler has no group of its own, RFC 9309 says it must obey the group marked with an asterisk, so read that group too.
2. Decide on the training crawlers, GPTBot and ClaudeBot, as a separate question. Blocking them does not remove you from search.
3. Treat Google-Extended as a training and Gemini grounding decision together, not as training alone.
4. In your CDN or firewall, allow the IP ranges OpenAI and Perplexity publish, since a robots.txt allow does not open a firewall.
5. In Google Search Console, go to Settings > Search generative AI and confirm the site is included. Included is the default.
6. Verify the site in Bing Webmaster Tools and in Google Search Console, so you can read the two reports that exist.

Anthropic's bots honor robots.txt, so a store whose asterisk group closes a policy page also closes it to Claude-User when a shopper asks Claude to read it. Give Claude-User its own group if you want it to reach those pages. The full list of names is in our guide to AI crawler user agents.

What did one Disallow group do to a vintage furniture dealer in four different assistants?

A vintage furniture dealer on Shopify lists 300 one of a kind pieces. A previous developer blocked "all AI bots" in robots.txt by listing GPTBot, ClaudeBot, Google-Extended and PerplexityBot under one Disallow: / group.

That single group did four different things. It kept the store out of training for OpenAI and Anthropic, which the owner wanted. It also blocked PerplexityBot, which surfaces sites in Perplexity search, and it blocked Google-Extended, which kept the catalog out of grounded Gemini answers. The owner wanted neither.

The owner splits the group. GPTBot and ClaudeBot stay blocked. PerplexityBot, OAI-SearchBot, Claude-SearchBot and Claude-User each get a group that allows the public pages and closes the cart and checkout paths. Google-Extended is left open so Gemini can ground answers on the catalog.

How does each assistant choose what to cite?

Google says its generative AI features are rooted in its core Search ranking and quality systems. Its guide also describes query fan out, a set of related queries the model issues to find more information. OpenAI, Perplexity, Anthropic and Microsoft describe what their answers include rather than how sources are picked. None of the five lists its citation factors.

The free scan checks the robots.txt rule for each page it reads against eight AI crawler names from OpenAI, Anthropic and Perplexity, and reports Google-Extended on its own. You will see which of these decisions your file has already made. For Grok on X, see how Grok and X search find your store.

Questions owners ask

Do I need a different strategy for each AI assistant?

You need one strategy and a few platform checks. The shared work is crawlable pages that state the same facts everywhere. The differences are which crawler to allow for each assistant and which report to read, and the table on this page lists both.

How do I get my business cited by Gemini?

Google says Gemini Apps ground answers with content from the Google Search index at prompt time. So the path is the same as for Google Search: an indexed page with clear, specific content. Do not block Google-Extended if you want your content used for grounding in Gemini Apps.

Does blocking GPTBot or ClaudeBot remove me from AI search?

OpenAI and Anthropic both treat their training crawlers separately from search. Blocking GPTBot or ClaudeBot keeps your future content out of training, while OAI-SearchBot, Claude-SearchBot and Claude-User decide whether you can appear in their answers.

Which AI assistants report citations to site owners?

Two platforms report on AI answers today. Google Search Console has a generative AI performance report with impressions in AI Overviews and AI Mode, and Bing Webmaster Tools has an AI Performance report with citations in Copilot. For ChatGPT, count visits tagged utm_source=chatgpt.com.

Does Perplexity-User follow robots.txt?

Perplexity says Perplexity-User generally ignores robots.txt, since a user requested the fetch. PerplexityBot, the crawler that surfaces sites in Perplexity search, is the one your robots.txt rules govern. Perplexity publishes the IP ranges for both.

Sources

• Overview of OpenAI crawlers, OpenAI, read November 9, 2026.
• Perplexity crawlers, Perplexity developer documentation, read November 9, 2026.
• Google's common crawlers, Google, read November 9, 2026.
• Optimizing your website for generative AI features on Google Search, Google Search Central, read November 9, 2026.
• Search generative AI control and Generative AI performance report, Search Console Help, read November 9, 2026.
• Introducing AI Performance in Bing Webmaster Tools, Bing Webmaster blog, February 2026, read November 9, 2026.
• Does Anthropic crawl data from the web?, Claude Help Center, read November 9, 2026.
• RFC 9309, Robots Exclusion Protocol, IETF, September 2022, read November 9, 2026.