Which AI crawlers can access your site?

Your robots.txt might be blocking ChatGPT, Claude or Perplexity without you knowing.
Website

Enter your site URL to check AI crawler access.

Free, no account or credit card required

Sample reportExampleexample.com

7allowed

3blocked

ClaudeBot, Google-Extended, CCBot cannot read example.com.

AI bot access to example.com
BotAccess
GPTBotOpenAIAllowed
OAI-SearchBotOpenAIAllowed
ChatGPT-UserOpenAIAllowed
ClaudeBotAnthropicBlocked
Claude-UserAnthropicAllowed
PerplexityBotPerplexityAllowed
Google-ExtendedGoogleBlocked
Applebot-ExtendedAppleAllowed
CCBotCommon CrawlBlocked
BingbotMicrosoftAllowed
example.com/robots.txt
User-agent: *Allow: / User-agent: ClaudeBotDisallow: / User-agent: Google-ExtendedDisallow: / User-agent: CCBotDisallow: / Sitemap: https://example.com/sitemap.xml

In red, the rules that block an AI bot.

Does AI cite your site?

Allowing crawlers is only the first step. Run a free audit to see whether ChatGPT and Gemini actually recommend your site.

Run the free audit

Why does it matter?

AI assistants like ChatGPT, Claude and Perplexity use crawlers to read your site. If they are blocked in your robots.txt, your site stays invisible in their answers.

This test reads your robots.txt and shows, crawler by crawler, which ones are allowed or blocked.

Frequently asked questions about AI crawlers

Another question? Write to us

  • An AI crawler is a bot that AI assistants use to read web pages, such as OpenAI GPTBot, Anthropic ClaudeBot, PerplexityBot and Google-Extended. If a crawler can access your site, the AI behind it can read and cite your content.

  • Paste your domain into this checker. It reads your robots.txt and shows, crawler by crawler, which AI bots are allowed or blocked, including GPTBot, ClaudeBot, PerplexityBot and Google-Extended.

  • If you want to appear in ChatGPT, Claude, Perplexity and Google AI Overviews, allow their crawlers. Blocking them keeps your content out of AI answers. Block only content you do not want used by AI.

  • Blocking GPTBot stops OpenAI from crawling your site, which lowers your chances of being cited in ChatGPT. To stay visible, allow GPTBot, OAI-SearchBot and ChatGPT-User.

  • Add a User-agent rule for each AI crawler (or User-agent: *) without a Disallow on your key pages, for example User-agent: GPTBot then Allow: /. Re-run this checker to confirm.

Your customers ask AI. Be the answer.

Connect your visibility tools. Hikoo does the work, and you stay in control of what goes out.