# Baiduspider

Baidu says Baiduspider is the search-engine program that visits pages and builds them into the Baidu index. PC search, mobile search, and other search share this token. The FAQ says Baiduspider works on the robots.txt protocol, and that disallowing it keeps the site's pages out of Baidu search results and out of other search results Baidu provides. An updated robots.txt usually takes 48 hours to stop new indexing. Pages already indexed can take several months to leave the results. The same FAQ says a block of this token blocks all crawling from Baidu. This check matches the whole token and does not treat Baiduspider as a prefix. Baiduspider-image, Baiduspider-news, and Baiduspider-favo are different tokens. Baiduspider-video does not support the rules currently. Baiduspider-cpro and Baiduspider-ads do not follow robots.txt records. The FAQ says a meta robots tag can ask Baidu to index a page without showing a cached link. This check does not read meta tags.

- Operator: Baidu
- Kind: search
- robots.txt: obeys. The vendor says this crawler obeys robots.txt on automatic crawls.
- HTTP user-agent: Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html） The Chinese FAQ at https://help.baidu.com/question?prod_id=99&class=476&id=2996 publishes this as the PC search string and closes it with a fullwidth parenthesis. The mobile search string on that page is Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html). The page says a PC fetch is identified by the keyword Baiduspider/2.0, and a mobile fetch by the keywords android and mobile, with Baiduspider/2.0 confirming the crawler. The English FAQ does not publish these strings. Mobile search and other search use this same token.
- IP list: none on the cited page
- Source: https://help.baidu.com/question?prod_id=99&class=0&id=3001 (retrieved 2026-10-18)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/baiduspider`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows Baiduspider on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Baiduspider\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["baiduspider"]}}}'
```

That call returns fetched_target false. This row is disallow via group Baiduspider, rule Disallow: /.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.