Paste it and see which AI crawlers — GPTBot, ClaudeBot, PerplexityBot and every other one — may read your site.
Why this matters
Search engines like GPTBot, ClaudeBot and PerplexityBot do not use
Googlebot. They are separate user-agent tokens, and each one is allowed
or blocked on its own.
GPTBot stops training crawls but says nothing about
ChatGPT search — that one is OAI-SearchBot.Google-Extended does not affect Google Search at all; it only
governs Gemini and Vertex training.ChatGPT-User fetches pages a person asked for, so it usually
ignores robots.txt anyway.Paste your robots.txt above to see where you stand today.