Cloudflare will add X-Robots-Tag to some pages


Recently I find Cloudflare will add X-Robots-Tag(noindex, follow) to some of our pages. For example:

The response header in Chrome is below:


To make sure the flag is set by Cloudflare, I do the following check:

  1. Check all .htaccess files and confirm there is no setting of X-Robots-Tag.
  2. Use script to setDNS to our server’s IP directly, to bypass Cloudflare, as below:
setDns ##.##.##.##

Then the response header will NOT contain X-Robots-Tag any more, the result is at : Chr...irginia USA - EC2 - WebPageTest Details

So how to prevent Cloudflare from adding X-Robots-Tag?

I check some related posts:

but still cannot solve the issue.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.