I’m trying to use Rules → Transform Rules to modify the HTTP header to add the header: X-Robots-Tag with a value of “noindex”.
I’ve tried the following expression:
(http.request.full_uri contains “.pdf”)
The response headers do not have “X-Robots-Tag” added. Could someone shed light on what is happening and how to troubleshoot this?
It does not allow you to change the response headers, only the request headers
Thanks, that would explain it. Would the alternative to be using firewall rules instead to block search engine access?
Assuming you want to block official search engines (and others won’t care about a header) a firewall rule might just work.
You could also try a Worker of course. There you could set a response header or you set it straight at your origin.
I’ll go with the firewall solution for now. Thank you. I’ll be setting the response header at the origin as well.
You can also do this in your robots.txt:
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.