GPTBot & AI Crawler Checker
Is ChatGPT blocked on your site? Test GPTBot and every major AI crawler — and see exactly why each one is allowed or blocked.
What each AI robot does
Different AI companies send different crawlers, and they are not all the same. Some decide whether an assistant can mention your store. Others only gather data to train a model. Blocking the wrong one can quietly remove you from AI answers. Here is each major crawler in one line:
- GPTBot OpenAI Trains ChatGPT’s models. Controlled by robots.txt.
- OAI-SearchBot OpenAI Powers ChatGPT Search — affects whether ChatGPT mentions you.
- ChatGPT-User OpenAI Fetches your page live when a user asks ChatGPT about you.
- ClaudeBot Anthropic Trains Claude’s models. Controlled by robots.txt.
- Claude Search Anthropic Helps Claude find and cite your store in answers.
- PerplexityBot Perplexity Lets Perplexity read and recommend your pages.
- Googlebot Google Google search — also feeds AI Overviews and Gemini answers.
- Google-Extended Google Opts your content in or out of Gemini model training.
- Bingbot Microsoft Bing search — powers Copilot answers too.
The robots.txt vs firewall trap
Here is a mistake that catches many stores. Your robots.txt clearly says a crawler is welcome — but your CDN, WAF or firewall still blocks it with an error like 403 or 429. The bot never gets through, and you never notice, because the robots.txt file looks perfect.
That is why this checker does more than read robots.txt. For each AI search crawler, it sends a live request pretending to be that bot, so it can catch a block hiding at the edge. When a bot is allowed in robots.txt but refused live, the tool flags it as an edge block and shows the HTTP status.
How to unblock a bot on Shopify
Start with your robots.txt. Shopify lets you edit crawler rules with a robots.txt.liquid
template, or an app can manage them for you. If robots.txt already allows the bot but it is still
blocked, the block lives at your CDN or firewall — add the bot’s user agent to your allow list
there. The simplest path is a per-bot toggle, so you never touch code.
One toggle per bot.
No robots.txt editing.
Kedra AI Index lets you allow or block GPTBot, ClaudeBot, PerplexityBot and Google-Extended with a single switch each — and shows you which AI crawlers actually visit your store in real time.
AI crawlers, answered
What is GPTBot?
GPTBot is the crawler OpenAI uses to collect data for training its models. If you block GPTBot, OpenAI will not train on your content. Note that GPTBot is not the crawler that decides whether ChatGPT mentions your store in an answer — that job belongs to OAI-SearchBot and ChatGPT-User, which this tool checks separately.
How do I know if GPTBot is blocked on my site?
Enter your URL above. This checker reads your robots.txt and reports whether GPTBot is allowed or blocked, and why. For the AI search crawlers, it also makes a live request to catch cases where robots.txt allows a bot but your CDN or firewall quietly refuses it. GPTBot itself is controlled through robots.txt.
Should I block GPTBot?
It depends on your goal. Blocking GPTBot stops OpenAI from using your content to train models, which some brands want. But blocking it does not remove you from ChatGPT answers, and it does not protect your images — that is a separate content-protection question. If your goal is to be recommended by AI, focus on the search crawlers instead.
What is the "robots.txt says yes but firewall says no" trap?
Your robots.txt can clearly allow a crawler while your CDN, WAF or firewall still blocks it with an error like 403 or 429. The bot never gets in, and you never see it, because robots.txt looks correct. This checker makes a live request as each search crawler, so it catches that hidden edge block — not just what robots.txt claims.
How do I unblock an AI crawler on Shopify?
First, check your robots.txt. On Shopify you can adjust crawler rules with a robots.txt.liquid template or an app. If robots.txt already allows the bot but it is still blocked, the block is at your CDN or firewall — allow the bot’s user agent there. Kedra AI Index gives you a simple per-bot toggle so you do not have to edit code.
Which AI crawlers matter most for my store?
The ones that affect whether AI can mention you: OAI-SearchBot and ChatGPT-User (OpenAI), Claude Search (Anthropic), PerplexityBot, Googlebot and Bingbot. These are marked "Search" in the table. Training-only bots like GPTBot, ClaudeBot and Google-Extended feed model training but do not decide whether AI recommends your store today.