AI Insights

It’s Content Independence Day: Cloudflare, along with a majority of the world's leading publishers and AI companies, is changing the default to block AI crawlers unless they pay creators for content.

• Pay Per Crawl
• AI Bots
• AI

Cloudflare Radar now shows how often a given AI model sends traffic to a site relative to how often it crawls that site. This helps site owners make decisions about which AI bots to allow or block.

• Pay Per Crawl
• Radar
• Internet Traffic
• AI
• Bots

Cloudflare is making it easier for publishers and content creators of all sizes to prevent their content from being scraped for AI training by managing robots.txt on their behalf.

• Pay Per Crawl
• AI
• Bots
• Impact