Cloudflare will add X-Robots-Tag to some pages

Hi,

Recently I find Cloudflare will add X-Robots-Tag(noindex, follow) to some of our pages. For example:

The response header in Chrome is below:

image

To make sure the flag is set by Cloudflare, I do the following check:

  1. Check all .htaccess files and confirm there is no setting of X-Robots-Tag.
  2. Use www.webpagetest.org script to setDNS to our server’s IP directly, to bypass Cloudflare, as below:
setDns www.datanumen.com ##.##.##.##
navigate https://www.datanumen.com/fr/blogs/chatgpt-join-drawing-exam/

Then the response header will NOT contain X-Robots-Tag any more, the result is at datanumen.com : Chr...irginia USA - EC2 - WebPageTest Details

So how to prevent Cloudflare from adding X-Robots-Tag?

I check some related posts:

but still cannot solve the issue.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.