Free Tool

llms.txt + robots.txt generator for AI crawlers

Tell ChatGPT, Claude, Perplexity, and Google AI Overviews how to read and quote your site. Paste the output at /llms.txt and /robots.txt.

Your site

Site nameSite URLOne-line description (used in llms.txt)Key pages (one URL or path per line)

AI crawlers — allow each one?

OpenAI GPTBot (ChatGPT)AllowedAnthropic ClaudeBot (Claude)AllowedPerplexityBotAllowedGoogle-Extended (Gemini, AI Overviews)AllowedCCBot (Common Crawl, training)AllowedBingbot (Bing search + Copilot)AllowedApplebot-Extended (Apple Intelligence)Allowed

llms.txt

Save as /llms.txt at your site root.

robots.txt (AI bots only)

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: CCBot
Allow: /

User-agent: Bingbot
Allow: /

User-agent: Applebot-Extended
Allow: /

Append these blocks to your existing /robots.txt. Keep your existing rules for Googlebot etc.

Why this matters

AI search is rewriting how customers find you.

ChatGPT and Perplexity already pull from your pages — but only if you let them, and only if your content is structured for extraction. Want a full audit of how AI-quotable your site is?

Run the free AI Quotability audit