I have some questions:
-
The firewall rule for blocking bots is blocking the bots that don’t have X-Middleton in the user agent. Do you think that this is because in the origin server is setup X-Middleton headers?
-
I didn’t check my backlinks for a while. I discover that they were added a lot of backlinks that link to the internal search of my site with pharma keywords. This links are followed by the bots Google Bing, etc.
I had setup a firewall rule, to block the referral traffic from those domains. However as I have allow good bots in another firewall rule I think that they are not going to be blocked those request. Any suggestion?