# Google-Extended

Google says publishers use the Google-Extended robots.txt token to manage whether content Google has crawled may train future Gemini models (Gemini Apps and Vertex AI API for Gemini) and whether it may ground answers in Gemini Apps and in Grounding with Google Search on Vertex AI. Google says this token does not change inclusion or ranking in Google Search.

- Operator: Google
- Kind: control_token
- robots.txt: control_token. This is a robots.txt control. It is not a separate HTTP crawler.
- HTTP user-agent: Google says Google-Extended has no separate HTTP user-agent. Crawling uses existing Google user-agent strings. The token is a robots.txt control.
- IP list: none on the cited page
- Source: https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers (retrieved 2026-10-17)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/google-extended`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows Google-Extended on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Google-Extended\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["google-extended"]}}}'
```

That call returns fetched_target false. This row is disallow via group Google-Extended, rule Disallow: /.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.