`Applebot`

Apple says Applebot crawls the web for search in Spotlight, Siri, and Safari. Enabling the token in robots.txt allows site content to appear in those results. The same page says crawled data may also train Apple foundation models, and that opting out of that training is the Applebot-Extended token, not this one. A nosnippet tag, not this token, is how the page says to opt out of broad world-knowledge answers. Apple says this crawler respects robots.txt on general search crawls aimed at Applebot, does not follow crawl-delay, and follows the Googlebot group when the file does not mention Applebot but does mention Googlebot. The page does not say that a Disallow removes the site from search. The iTMS user agent on the same page is not this token. Applebot-Extended is a different token and is not this card.

robots.txt token: Applebot
Operator: Apple
Kind: search
robots.txt: respects_no_crawl_delay. The vendor says this crawler respects robots.txt on general search crawls aimed at this token, and does not follow crawl-delay.
When this token is absent: If the file does not mention Applebot and does mention Googlebot, the cited page says this crawler follows the Googlebot group. The checker reports follows_when_unmentioned and does not apply it to the RFC decision.
HTTP user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) Apple publishes this as a desktop example of the search user agent. The same section publishes a mobile example that also contains Applebot/0.1, and says the advertised browser version changes while the general format stays. Match the token, not one full string. The iTMS user agent on the same page is not this token.
IP list: http://search.developer.apple.com/applebot.json The Identifying Applebot section links this file of CIDR prefixes and says traffic is also identified by reverse DNS in the applebot.apple.com domain. On 2026-10-18 this URL returned HTTP 301 to https://search.developer.apple.com/applebot.json, which returned HTTP 200 and JSON. Botcard does not copy the address list.
Source: About Applebot. Retrieved 2026-10-18, page published 2026-09-04.

JSON

curl -s https://whosthatbot.com/v1/crawlers/applebot

Check this token

tools/call with name check_robots is the same check at POST /mcp. The pasted file disallows Applebot on /docs and allows *. The call does not fetch a host.

curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Applebot\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["applebot"]}}}'

That call returns fetched_target false. This row is disallow via group Applebot, rule Disallow: /. follows_when_unmentioned is Googlebot and is not the decision.