`APIs-Google`

Google's special-case crawlers page says these crawlers are used where there is an agreement between the site and the product. The robots.txt token is APIs-Google. The page says the global user agent (*) is ignored. Crawling preferences for this token affect delivery of push notification messages by Google APIs. The list is not exhaustive. This check matches the whole token and still reports the RFC 9309 decision.

robots.txt token: APIs-Google
Operator: Google
Kind: owner_requested
robots.txt: ignores_star. The vendor says this robot does not take the general User-agent: * rules in robots.txt into account.
HTTP user-agent: APIs-Google (+https://developers.google.com/webmasters/APIs-Google.html) The page prints this string as the HTTP user agent. The robots.txt token is the same string, APIs-Google.
IP list: https://developers.google.com/static/crawling/ipranges/special-crawlers.json The page says special-case crawlers operate from a different range than the common crawlers, published in this file. The URL returned HTTP 200 on 2026-10-18, creationTime 2026-10-16. Botcard does not copy the prefixes. The page says the reverse DNS mask matches rate-limited-proxy-***-***-***-***.google.com. A user-agent string can be spoofed.
Source: List of Google's special-case crawlers. Retrieved 2026-10-18, page says updated 2026-09-17. CC BY 4.0 (Google Developers page, except as that page notes).

JSON

curl -s https://whosthatbot.com/v1/crawlers/apis-google

Check this token

tools/call with name check_robots is the same check at POST /mcp. The pasted file disallows APIs-Google on /docs and allows *. The call does not fetch a host.

curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: APIs-Google\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["apis-google"]}}}'

That call returns fetched_target false. This row is disallow via group APIs-Google, rule Disallow: /.