Paste the file and a path. Botcard compares them to the 68 tokens in dataset 2026-10-18.9. It does not fetch a host.
Each row is an allow or disallow under RFC 9309: the matching group and the rule line. The vendor stance is the same sentence as on the token's card. OpenAI says robots.txt may not apply to ChatGPT-User. Perplexity says Perplexity-User generally ignores robots.txt. Perplexity recommends allowing PerplexityBot so the site appears in search. That recommendation does not say a Disallow removes the site. Google-Extended is not Googlebot. A card that lists another token, such as Google-CloudVertexBot and Googlebot, is not merged into this decision. Apple says Applebot does not follow crawl-delay, and follows a Googlebot group only when the file does not mention Applebot. That fallback is listed and not applied. Common Crawl says publishing User-agent: CCBot and Disallow: / prevents the crawl. The page also says some crawlers falsely use that name. A separate post names an Opt-Out Ledger of legal requests. That ledger is listed and not applied. Amazon says Amzn-User may not follow all robots.txt directives, and that Amzn-SearchBot follows other search bots when the file does not name it. That fallback is not applied. Meta says Meta-ExternalFetcher may bypass robots.txt. DuckDuckGo says a DuckAssistBot disallow takes effect after 72 hours.
A blank Disallow does not block the site. Crawl-delay is reported and not applied. /robots.txt is always allowed. A final $ is the end of the path plus its query.
curl -s https://whosthatbot.com/v1/robots-check -H 'content-type: application/json' -d '{"robots_txt":"User-agent: GPTBot\nDisallow: /\n","path":"/docs"}'tools/call with name check_robots is the same check at POST /mcp. It refuses a URL.