# Meta-ExternalAgent

Meta says Meta-ExternalAgent crawls for uses such as training foundation AI models or improving products by indexing content directly. The page says to block a crawler by adding a Disallow for it, and that a robots.txt change can take up to 24 hours because crawlers may cache the file. The robots.txt example on the page uses User-agent: meta-externalagent. Meta-ExternalFetcher is a different token and may bypass robots.txt. Meta-WebIndexer is a different token.

- Operator: Meta
- Kind: training
- robots.txt: disallow_is_opt_out. The vendor says disallowing this token opts that use out.
- HTTP user-agent: meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers) Meta says the log string will be similar to this one or to meta-externalagent/1.1. On 2026-10-18 the page was text/markdown, and the parenthetical path has no host. The robots.txt example on that page uses User-agent: meta-externalagent.
- IP list: none on the cited page
- Source: https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers (retrieved 2026-10-18)

## JSON

`curl -s https://whosthatbot.com/v1/crawlers/meta-externalagent`

## Check this token

tools/call with name check_robots is the same check at https://whosthatbot.com/mcp. Send `Content-Type: application/json` and `Accept: application/json, text/event-stream`. The pasted file disallows Meta-ExternalAgent on `/docs` and allows `*`. The call does not fetch a host.

```
curl -s https://whosthatbot.com/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_robots","arguments":{"robots_txt":"User-agent: Meta-ExternalAgent\nDisallow: /\n\nUser-agent: *\nAllow: /\n","path":"/docs","ids":["meta-externalagent"]}}}'
```

That call returns fetched_target false. This row is disallow via group Meta-ExternalAgent, rule Disallow: /.

Each card paraphrases the linked vendor page and was checked on source.retrieved. This is not legal advice. robots.txt is a request from the site owner. Botcard does not fetch third-party sites and does not suggest ways around a disallow rule.