User Agent Blocking not working


#1

I’m trying to block a few different bots but it’s not working. I’ve added them to the User Agent Blocking section of the Firewall, they’re turned on, set to block but they’re not being blocked. In checking my server logs I can see they’re coming from cloudflare IP’s so obviously they’re passing through cloudflare, I even went into my server and blocked access from any IP not belonging to cloudflare. I’ve tried entering the user agent multiple ways to no avail. For instance with SemrushBot I’ve tried using
Mozilla/5.0+(compatible;+SemrushBot/2~bl;++http://www.semrush.com/bot.html)
SemrushBot/2
SemrushBot
but it still gets through. Any ideas?


#2

Well I finally got this figured out for some reason how it showed up in my logs was not how to block it, I had to replace some of the +'s with spaces. For SemrushBot in my logs it looked like

Mozilla/5.0+(compatible;+SemrushBot/2~bl;++http://www.semrush.com/bot.html)

but to get it to work i had to change it to

Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)

It’s working perfectly now.


#3

This topic was automatically closed after 14 days. New replies are no longer allowed.