How to non index domain & its pages

Hello there, I don’t know how to NON INDEX my domain and its pages. Can I do this and how on CloudFare?

Thank you so much.

Do you want to set a x-robots-tag: noindex on all your HTTP responses?

You can do that easily with Transform Rules

traffic must be proxied through Cloudflare for this to work

make sure you’re in RESPONSE headers, not REQUEST headers


Hi user4358, thank you so much for your reply. I don’t have a specific method in mind, I am not a coder :sweat_smile: … whatever works and it is easy to set up. Is your setup within Cloudflare? If so, sorry can you direct me where to find it?

I was reading that robots-tag bots don’t 100% prevent the site to be indexed though, is it true? I am using them in WIX because it was super easy to just toggle them :smiley:

Thank you so much!

Yes it’s done through Cloudflare. Rules → Transform Rules → Create transform rule → modify response header

The X-robots-tag is actually the best way to block indexing/following. Keep in mind that it does not block crawling. You can block crawling via robots.txt, however, this does not block indexing, and if you block crawling via robots.txt, they won’t be able to see the x-robots-tag and hence may index anyway. Hence I rarely use robots.txt anymore.

1 Like

You are awesome! Thank you so much‼️

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.