`AI2Bot`

Ai2 says the AI2 Bot explores certain domains to find web content used to train open language models. The page publishes one HTTP user-agent and says that string can be used to filter or reject the crawler's traffic. It does not say whether robots.txt controls this token.

robots.txt token: AI2Bot
Operator: Ai2
Kind: training
robots.txt: not_stated. The cited page does not say whether robots.txt controls this token.
HTTP user-agent: Mozilla/5.0 (compatible) AI2Bot (+https://www.allenai.org/crawler) The page says the crawler uses this User-Agent header when looking for documents.
IP list: None on the cited page. This page does not publish an IP list.
Source: Crawling notice. Retrieved 2026-10-18.

JSON

curl -s https://whosthatbot.com/v1/crawlers/ai2bot

Check this token

tools/call with name check_robots is the same check at POST /mcp. The pasted file disallows AI2Bot on /docs and allows *. The call does not fetch a host.

curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: AI2Bot\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["ai2bot"]}}}'

That call returns fetched_target false. This row is disallow via group AI2Bot, rule Disallow: /.