# Check a robots.txt

Paste a robots.txt and a path. Botcard does not fetch the host.

POST https://whosthatbot.com/v1/robots-check

JSON body: robots_txt (string), path (string, default /), optional ids (card ids).
The response is an RFC 9309 allow or disallow for each token, the matching group, the winning rule, and the vendor stance already printed on that token's card.
A URL, a url field, or any other instruction to fetch is rejected.

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. list_crawlers and get_crawler return the cards. None of the tools fetch a host.

A blank Disallow is skipped. Crawl-delay is noted and not applied. /robots.txt is implicitly allowed.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.