# Googlebot-News

Google says Googlebot News has no separate HTTP user-agent. Crawling uses various Googlebot strings. Preferences for Googlebot-News affect Google News, including news.google.com and the Google News app. The page also lists Googlebot as a robots.txt token and says matching one of a crawler's tokens is enough. This check matches the whole token, so a group named Googlebot does not match Googlebot-News, and still reports the RFC 9309 decision. Common crawlers obey robots.txt on automatic crawls.

- Operator: Google
- Kind: search
- robots.txt: obeys. The vendor says this crawler obeys robots.txt on automatic crawls.
- HTTP user-agent: The page says Googlebot-news does not have a separate HTTP request user agent string. Crawling is done with various Googlebot user agent strings. The robots.txt token spelling is Googlebot-News. A group named Googlebot does not match this token.
- Also matches: Googlebot
- IP list: https://developers.google.com/static/crawling/ipranges/common-crawlers.json
- Source: https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers (retrieved 2026-10-18)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/googlebot-news`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows Googlebot-News on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Googlebot-News\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["googlebot-news"]}}}'
```

That call returns fetched_target false. This row is disallow via group Googlebot-News, rule Disallow: /. also_matches_tokens lists Googlebot and is not merged into this decision.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.