# CloudFlare-AlwaysOnline

Cloudflare says it crawls a site when Always Online is enabled. The published user agent contains CloudFlare-AlwaysOnline/1.0. The page does not say whether robots.txt controls this token. This check matches the whole token, so a group named Cloudflare does not match, and it still reports the RFC 9309 decision.

- Operator: Cloudflare
- Kind: owner_requested
- robots.txt: not_stated. The cited page does not say whether robots.txt controls this token.
- HTTP user-agent: Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online) The page prints this string as the user agent. It does not print a separate robots.txt User-agent line. A group named Cloudflare does not match this token. A group named CloudFlare-Prefetch does not match this token.
- IP list: none on the cited page
- Source: https://developers.cloudflare.com/fundamentals/reference/cloudflare-site-crawling/ (retrieved 2026-10-18)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/cloudflare-alwaysonline`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows CloudFlare-AlwaysOnline on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: CloudFlare-AlwaysOnline\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["cloudflare-alwaysonline"]}}}'
```

That call returns fetched_target false. This row is disallow via group CloudFlare-AlwaysOnline, rule Disallow: /.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.