I have a website on cloudflare pro plan.
I set a x-robots-tag for pdf files in my htaccess file and with a cloudflare transform rules.
When i dns-only my website, everthings is fine and pdf are not indexed. When i do an url inspect on google search console, pdf files are not indexed.
When i proxified with cloudflare, pdf files are marqued as indexed in google inspector.
I purge the pdf file and it’s the same issue.
Could you help me?
That header generally works on Cloudflare. Can you post a sample URL?
The header is sent.
$ curl -I https://www.calendriergratuit.fr/calendrier-pdf/annuel/calendrier-2036-P.pdf | grep robots x-robots-tag: noindex
yes but i’m not sure google can see this header.
When i test it on google inspector,
‘HTTP/1.1 200 OK
x-robots-tag: noindex is not appearing
If i disable cloudflare with dns-only, it’s ok…
I am not sure what “Google Inspector” is, but that would be a question for Google I am afraid. The header itself is sent, respectively forwarded, by Cloudflare.
There’s also no cache status header, so it seems as if Google did not show all headers.
There is a discrepancy with the
curl -I https://www.calendriergratuit.fr/calendrier-pdf/annuel/calendrier-2036-P.pdf HTTP/2 200 date: Thu, 27 Jul 2023 10:32:10 GMT content-type: application/pdf content-length: 390309 # <-- Not the same as shown by Google. # and the rest of the headers
I add this to my htaccess and it’s working now for pdf
pdf files are now marked as noindex in google search console.
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.