Good Bot VS Bad Bot (firewall rules)

So I set up 2 different rules. One for Known Bots which as I’m understanding it allows in all good bots and then under it I made another rule for bad bots Assuming that first it will look to see if its a good bot then look to see if it’s a bad one. (is that correct thinking?)
But it is allowing Semrush Which is on the bad bot rule.
Am I misunderstanding?

There is no “bad bots” flag on Cloudflare. Can you post a screenshot of your rules list?

I found it here How to Use Cloudflare Firewall Rules to Protect Your Website
Scroll down to Example 3 — Block bad bots (use Expression Editor)

That doesnt really block bad bots but simply requests with certain user agents. Yandex, for example, is a standard search engine crawler.

Ok so maybe I’m calling them the wrong thing? (sorry newbie to this type of thing) So block yandex yes and it seems to be doing that and allowing google but why is semrush not blocked if its within that block statement?

You are referring to http.user_agent contains "SemrushBot"?

Do you have requests showing up with “SemrushBot” in your logs?

Yes, in the firewall log (I assume that’s what its called where it shows you what’s blocked and what’s allowed) there is Action Allow from Semrush

with this ip which if you do an ip look up also shows its semrush. …

20 Jan, 2020 15:50:33
United States
Firewall rules
Mozilla/5.0 (compatible; SemrushBot/6~bl; +

Can you post a screenshot of that particular firewall event?

The info I posted above is the event just copy/pasted as text not screen shot

I asked for a screenshot, just as here as well

I’m not sure what the difference in a screen shot is than the actual info I’m sorry please explain

This topic was automatically closed after 30 days. New replies are no longer allowed.