Applebot: One of Two Crawlers That Document JavaScript Rendering

User-agent: Applebot
Disallow: /

User-agent: Applebot
Allow: /

Applebot is the crawler behind Siri and Spotlight Suggestions, and it is one of only two major crawlers whose operator has put rendering behaviour in writing.

Apple’s documentation states that “Applebot may render the content of your website within a browser. If javascript, CSS, and other resources are blocked via robots.txt, it may not be able to render the content properly.”

That second sentence is the useful one, and it catches people out.

Can it read JavaScript

Yes, subject to a hedge and a trap.

The hedge is Apple’s own word: may render, not does render. Read literally, that is a documented capability rather than a promise about every fetch. It is still far more than OpenAI, Anthropic or Perplexity have said, all of whom say nothing at all.

The trap is the second sentence. Applebot can only render what it is allowed to fetch. If your robots.txt blocks /js/, /assets/ or /static/, a rendering crawler loses the ability to render, and you have accidentally converted a capable crawler into a blind one. This is a common pattern in older robots.txt files written when blocking asset directories seemed tidy.

Check your own file for directives that disallow script or stylesheet paths. If a rendering crawler cannot fetch the resources, the rendering capability is worthless.

For how Applebot compares with the rest, see our crawler directory and which AI crawlers render JavaScript.

Applebot versus Applebot-Extended

These are two different decisions and conflating them is the most common Apple crawler mistake.

Token	What it does	Blocking it means
Applebot	Crawls pages for Siri and Spotlight Suggestions	You disappear from Apple’s search surfaces
Applebot-Extended	Controls training use of already-crawled content	Your content is not used to train Apple’s generative models

Apple states that Applebot-Extended does not crawl webpages. It is a permission signal attached to what Applebot already collected, not a second crawler.

So if your objection is to model training, block Applebot-Extended and leave Applebot alone. Blocking Applebot to prevent training removes you from Siri while achieving nothing you could not have achieved with the narrower control.

How to control it

Block Apple’s crawler entirely:

User-agent: Applebot
Disallow: /

Allow crawling but opt out of generative training:

User-agent: Applebot
Allow: /

User-agent: Applebot-Extended
Disallow: /

That second block is what most people actually want when they say they want to block Apple’s AI.

Verifying it

Apple publishes IP ranges for Applebot and supports the same forward-confirmed reverse DNS check used for other major crawlers: resolve the requesting IP to a hostname, then resolve that hostname back and confirm it matches. Do not trust the user agent string alone, which anything can send.