Hello, I’ve been having bot problems since Dec 29th, and since that time I’ve been using the firewall rules to block a lot of them. Most of these bad bots are on infested servers and burst crawl my site constantly.
I’ve used country block, blocking certain pages, country captcha, and some user agents.
It’s been really effective, however, I now have problems with Microsoft Azure infested servers. The most effective way is using ASN numbers. I can block ASN: 8075 but the problem with that is search.msn.com bingbot.htm gets blocked too as it also uses ASN: 8075. Is there any way to whitelist bingbot but keep ASN:8075 in my Challenge firewall rules?
These bots keep changing IP’s and the only effective way is to block with ASN.