Google-CloudVertexBotGoogle says Google-CloudVertexBot performs crawls a site owner requested in order to build Vertex AI Agents. It does not affect Google Search. On this crawler, a robots.txt rule matches if it names Google-CloudVertexBot or Googlebot; Google says matching one of a crawler's tokens is enough.
owner_requestedobeys. The vendor says this crawler obeys robots.txt on automatic crawls.Googlebot. Google says one matching token is enough.curl -s https://whosthatbot.com/v1/crawlers/google-cloudvertexbot
tools/call with name check_robots is the same check at POST /mcp. The pasted file disallows Google-CloudVertexBot on /docs and allows *. The call does not fetch a host.
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Google-CloudVertexBot\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["google-cloudvertexbot"]}}}'
That call returns fetched_target false. This row is disallow via group Google-CloudVertexBot, rule Disallow: /. also_matches_tokens lists Googlebot and is not merged into this decision.