Domain prefix in robots.txt file


Hi all, I have some sites on the CloudFlare free platform. I don’t use www on these sites just I’ve read that if sites are on CloudFlare then www should be used. Is this true?
Should I also be using www in my robots.txt file? Many Thanks.


Some hosting services require you to use ‘www’ because of a CNAME issue. If non-www works for you, keep using it. Just make sure you have a redirect in place to send www visitors to your non-www site.

Robots files shouldn’t have the domain part of your URL anyhere. Just the path or file itself.

closed #3

This topic was automatically closed after 14 days. New replies are no longer allowed.