ChatGPT-UserOpenAI says ChatGPT-User fetches pages when a person uses ChatGPT, a Custom GPT, or a GPT Action. It is not an automatic crawl, and OpenAI says robots.txt rules may not apply. It is not the token for ChatGPT search opt-out; that token is OAI-SearchBot.
user_fetchmay_not_apply. The vendor says robots.txt may not apply, because a person triggered the fetch.Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot OpenAI publishes this as the full user-agent string.curl -s https://whosthatbot.com/v1/crawlers/chatgpt-user
tools/call with name check_robots is the same check at POST /mcp. The pasted file disallows ChatGPT-User on /docs and allows *. The call does not fetch a host.
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: ChatGPT-User\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["chatgpt-user"]}}}'
That call returns fetched_target false. This row is disallow via group ChatGPT-User, rule Disallow: /.