Googlebot-ImageGoogle says preferences for Googlebot-Image affect Google Images, Discover, Video mode, and Search features that show images, logos, and favicons. The page also lists Googlebot as a robots.txt token and says matching one of a crawler's tokens is enough. This check matches the whole token, so a group named Googlebot does not match Googlebot-Image, and still reports the RFC 9309 decision. Common crawlers obey robots.txt on automatic crawls.
searchobeys. The vendor says this crawler obeys robots.txt on automatic crawls.Googlebot. Google says one matching token is enough.Googlebot-Image/1.0 The page prints this string as the HTTP user agent. It does not print a longer browser string. The robots.txt tokens on this row are Googlebot-Image and Googlebot. A group named Googlebot does not match this token.curl -s https://whosthatbot.com/v1/crawlers/googlebot-image
tools/call with name check_robots is the same check at POST /mcp. The pasted file disallows Googlebot-Image on /docs and allows *. The call does not fetch a host.
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Googlebot-Image\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["googlebot-image"]}}}'
That call returns fetched_target false. This row is disallow via group Googlebot-Image, rule Disallow: /. also_matches_tokens lists Googlebot and is not merged into this decision.