# Google-Agent

Google says Google-Agent is used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request. The user-triggered fetchers section says these fetchers generally ignore robots.txt rules because a user requested the fetch. The page publishes mobile and desktop HTTP user-agent strings and does not print a separate robots.txt User-agent line. That section says Google is also experimenting with Web Bot Auth, using the https://agent.bot.goog identity. This check does not verify that signature.

- Operator: Google
- Kind: user_fetch
- robots.txt: generally_ignores. The vendor says this fetcher generally ignores robots.txt, because a person requested the fetch.
- HTTP user-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) The page also publishes this desktop agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.36. It does not print a separate robots.txt User-agent line. Chrome/W.X.Y.Z is the page's placeholder.
- IP list: https://developers.google.com/static/crawling/ipranges/user-triggered-agents.json
- Source: https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers (retrieved 2026-10-18)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/google-agent`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows Google-Agent on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Google-Agent\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["google-agent"]}}}'
```

That call returns fetched_target false. This row is disallow via group Google-Agent, rule Disallow: /.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.