AI readiness guides by platform
WordPress: robots.txt for AI crawlers and llms.txt
Allow AI search crawlers and choose about training bots on WordPress: the virtual robots.txt, editing it with Yoast or a real file, the 'Discourage search engines' trap, and llms.txt.
Shopify: robots.txt for AI crawlers and llms.txt
Control AI crawlers on a Shopify store with the robots.txt.liquid theme template, keep Shopify's default rules, and check the store's built-in /llms.txt and /agents.md.
Wix: robots.txt for AI crawlers and llms.txt
Use the Wix Robots.txt Editor (SEO & GEO > Tools and settings) to choose which AI crawlers may read your site, and review the llms.txt file Wix generates automatically.
Squarespace: the AI crawlers setting and llms.txt
Squarespace doesn't let you edit robots.txt. What its 'Block known artificial intelligence crawlers' checkbox does (and doesn't) block, and how to add llms.txt on version 7.1.
Webflow: robots.txt for AI crawlers and llms.txt
Set AI crawler rules in Webflow (Site settings > SEO > Indexing), upload llms.txt (UTF-8, under 100 KB) and use the Content-Signal header. Publish, then check the live domain.
Cloudflare: AI crawler settings and robots.txt
How Cloudflare's AI Crawler and Agent access policies, Bot Preference Sync, managed robots.txt and the Block AI bots rule affect who can read your site, and how to check the result.
Open the free ai readiness checker
30-day free trial, no card required. Then from $14 a month. Cancel anytime.
Start free for 30 days