Adding Content-Signal to a docs site
Read OriginalThis article discusses Cloudflare's Content-Signal policy, an extension to robots.txt introduced in September 2025, which allows sites to declare permissions for AI search, training, and input. The author, who manages docs.expo.dev, explains the three flags (search, ai-train, ai-input) and emphasizes that omitting them neither grants nor restricts permission. They argue that documentation sites should adopt this policy because docs are often the product, and it's a low-effort way to ensure AI agents can properly index and use content. The article also notes that this policy complements other methods like JSON-LD and llms.txt for AI readiness.
Comments
No comments yet
Be the first to share your thoughts!
Browser Extension
Get instant access to AllDevBlogs from your browser