Should You Block GPTBot? Usually No, and Here's Why

Blocking GPTBot is usually the wrong default for a brand, because GPTBot only controls whether OpenAI may use your pages for model training. ChatGPT search runs on a different bot, OAI-SearchBot, which you can keep open. In AEOeye's crawler access study (October 2026), 4 of 193 SaaS sites blocked GPTBot, against 20 of 38 news publishers.

OpenAI facts below come from its crawler documentation, checked on 3 October 2026.

Should you block GPTBot?

If your pages exist to get you found, leave GPTBot open. If your pages are the product, block it on purpose and keep OAI-SearchBot open. The usual mistake is not the block itself but a block made for the wrong reason: believing GPTBot controls ChatGPT.

• Brands, SaaS, stores, local and service businesses: allow GPTBot. A block gives up a possible route into ChatGPT's baseline knowledge of your category and protects almost nothing you sell.
• Publishers, paywalled research, licensing-first sites: block GPTBot as a deliberate position, and stay in search.
• Everyone: never block OAI-SearchBot or ChatGPT-User unless you want out of ChatGPT answers.

We hold this position with a caveat. Nobody outside OpenAI can show that allowing GPTBot makes ChatGPT name you, and we do not claim it. The case rests on an asymmetry: a block has visible downsides, and its upside for a brand is small.

What does GPTBot actually do?

GPTBot is OpenAI's training crawler. OpenAI describes it as crawling "content that may be used in training our generative AI foundation models", and says disallowing it indicates your content should not be used that way. It does not decide whether ChatGPT cites you.

OpenAI runs separate agents for separate jobs:

Agent	Job	What a Disallow does
GPTBot	Crawls content that may train foundation models	Signals your content should not be used for training
OAI-SearchBot	Surfaces sites in ChatGPT's search features	Removes you from ChatGPT search answers
ChatGPT-User	Visits a page when a user's action in ChatGPT triggers it	Per OpenAI, robots.txt rules "may not apply"

For a fuller walkthrough of training, search and live fetching, see does ChatGPT use my website and our GPTBot glossary entry.

Which OpenAI bot controls ChatGPT search visibility?

OAI-SearchBot does. OpenAI says sites opted out of it "will not be shown in ChatGPT search answers, though can still appear as navigational links", and it recommends allowing it. ChatGPT-User, by contrast, is "not used to determine whether content may appear in Search".

The settings are independent. OpenAI's own example: allow OAI-SearchBot to appear in search results while disallowing GPTBot to opt out of training. After a robots.txt change, OpenAI says search systems can take about 24 hours to adjust.

How does a GPTBot block turn into a ChatGPT search block?

A GPTBot block silently becomes a search block when the rule is written carelessly or applied by a tool you forgot about. In our data, 16 of the 35 sites that block GPTBot also block OAI-SearchBot, so nearly half took themselves out of ChatGPT search answers.

Check for these three causes:

• One group, two bots. Listing User-agent: GPTBot and User-agent: OAI-SearchBot above one Disallow: / blocks both. Per RFC 9309, a crawler obeys the group that matches its token.
• A wildcard. Three of the 35 GPTBot blockers never name GPTBot at all; a blanket User-agent: * rule catches it.
• A CDN toggle. Cloudflare's managed robots.txt prepends Disallow rules for known AI crawlers, GPTBot included. Cloudflare also says it changed its defaults in 2025 so that, for new domains, AI training crawlers are blocked unless the owner chooses otherwise.

Is blocking GPTBot worth it for a brand?

For most brands, no. A block is a request, not enforcement, and its main promise, keeping your pages out of training, matters little if your business depends on being recommended. Meanwhile a mistake in the same file can cost you ChatGPT search.

Three facts shape the trade:

1. robots.txt is advisory. RFC 9309 says its rules are "not a form of access authorization", and Cloudflare's docs call compliance voluntary.
2. A block is not a recall. OpenAI's page says what a disallow signals; it does not promise to remove content already collected.
3. The grievance is real, but it is a publisher's. Cloudflare's data shows crawl-to-referral ratios in July 2025 of about 1,091 pages crawled per referred visit for OpenAI, 38,065 for Anthropic and 5.4 for Google. That imbalance is a fair complaint for a site paid by clicks. For a brand, a recommendation matters more than a click.

The direction of travel makes it worse. Cloudflare reports AI training at 52% of crawler requests in June 2026, up from 22% in spring 2025. Training is now the biggest share of crawler traffic, which explains why publishers fight it. Their reason is revenue from content, and most brands do not share it.

When is blocking GPTBot the right call?

Block GPTBot when uncompensated training use erodes what you sell. This is a narrow set, and the publisher pattern in our data fits it.

• News and original-journalism archives with licensing value.
• Paywalled, premium or proprietary research that customers pay to access.
• Sites in licensing talks, where an open robots.txt weakens your position.
• Sites that need a documented opt-out for legal reasons. Cloudflare frames its content-signal lines as express reservations of rights under Article 4 of the EU copyright directive 2019/790, so ask counsel what your reservation should say; robots.txt alone is not a legal instrument.

An academic study found 60.0% of reputable news sites disallow at least one AI crawler, against 9.1% of misinformation sites. Even among publishers, 9 of the 20 that block GPTBot still allow OAI-SearchBot.

What do other sites do?

Brands almost never block GPTBot; publishers often do. AEOeye's study fetched robots.txt from 384 well-known sites on 3 October 2026. "Blocks" means the bot's effective rules disallow the homepage /.

Group	Block GPTBot	Block OAI-SearchBot	Block GPTBot, allow OAI-SearchBot
SaaS (n=193)	4 (2.1%)	1 (0.5%)	3
E-commerce (n=90)	5 (5.6%)	1 (1.1%)	4
AI-visibility vendors (n=43)	0	0	0
News publishers (n=38)	20 (52.6%)	11 (28.9%)	9
UGC and review (n=20)	6 (30%)	3 (15%)	3
All (n=384)	35 (9.1%)	16 (4.2%)	19 (4.9%)

Two details stand out. Every site that blocks OAI-SearchBot also blocks GPTBot, so the "split" setup is a deliberate minority of 19. And 90 sites name GPTBot in robots.txt, yet 58 of those allow it, so naming a bot is not the same as blocking it. aeoeye.com names GPTBot in an explicit allow group.

Limits: one fetch per site on one day, main host only, and a convenience sample that skews toward established brands. A block is a request, and CDN-level blocking is invisible to robots.txt. The bot-by-bot picture, with a per-site CSV you can recheck, is in our AI crawler directory.

How do you decide, and what goes in robots.txt?

Decide by asking whether your content is a lead magnet or a product, then write the file so the intent is readable. Do this in order:

1. Read the live file at yourdomain.com/robots.txt. A CDN or plugin may have added rules your repository does not show.
2. Find the three tokens: GPTBot, OAI-SearchBot, ChatGPT-User. Decide each one separately.
3. Write explicit groups, so a later wildcard cannot override your intent.
4. Wait about 24 hours, then re-check.

For a brand, allow everything and protect only private paths. For a publisher opting out of training while staying in search:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

Our AI robots.txt generator drafts this for other bots too, and how to block AI crawlers covers enforcement beyond robots.txt.

Access is only the first gate. A crawler reaching you proves nothing about whether ChatGPT recommends you. To see that, run our free audit, which asks ChatGPT three unbranded buyer questions live and shows the real answers.