Firewall rule not working (block BLEXBot)

Hi,

I have a firewall running to block certain bots. This is working fine for most of them but not for the BLEXBot bot.

This is the expression:
(http.user_agent contains “BLEXBot”) or (http.user_agent contains “PetalBot”) or (http.user_agent contains “SemrushBot”)

I still see this in access log:

5.9.70.221 - - [26/Sep/2022:07:55:03 +0000] “GET /shop/xxxxxxx/ HTTP/1.1” 200 32069 “-” “Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)”

Any idea how to block this bot?

Regards,
Jaap

Greetings,

Thank you for asking.

May I ask if you’ve tried with a Firewall Rule where it has (http.user_agent contains "BLEXBot") or (http.user_agent contains "webmeup") just inc case? :thinking:

I am not sure why it doesn’t work :thinking:

I usually see it’s coming from the Hetzner ASN and since I am blocking the ASN at the IP Access Rules, I can see the the request being blocked as Access Rules:ASN service and for that particular user-agenta.

However, I am not quite sure why it doesn’t catch it with a firewall rule.

Let me check this somehow on other zones I’ve manage. Will write back.

UPDATE: It’s on the list of “good” and “known” bots, that’s why it doesn’t catch it at all.

In case if you’re a Hetzner customer, allow your origin server IPv4 and IPv6 in the WAF → Tools → IP Access Rules. However, also make sure block the Hetzner ASN (AS24940) and you’d levarage a lot of crawlers and traffic from their network :wink:

Maybe, there is a trick with combination, like if cf.client.bot “true” and if user-agent contains blexbox then action “block”, or something similar - haven’t tested, just thinking about it now :thinking:

@fritex thanks for your answer!
I now blocked the ASN number.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.