# Claude-User

Anthropic says Claude-User fetches sites when a person asks Claude something. Disabling the token stops Anthropic from retrieving that content for the user query, which can reduce visibility in user-directed web search. This article treats the token as a robots.txt control. OpenAI's own ChatGPT-User page says robots.txt may not apply to that different token.

- Operator: Anthropic
- Kind: user_fetch
- robots.txt: disallow_is_opt_out. The vendor says disallowing this token opts that use out.
- HTTP user-agent: The cited help article names the robots.txt token and does not publish a full HTTP user-agent string.
- IP list: https://claude.com/crawling/bots.json
- Source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (retrieved 2026-10-17)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/claude-user`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows Claude-User on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Claude-User\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["claude-user"]}}}'
```

That call returns fetched_target false. This row is disallow via group Claude-User, rule Disallow: /.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.