Hi. For usage in WAF custom rules, I am trying to distinguish the difference between “Known Bots” vs “Verified Bots Category”. I can already see that Verified bots is a maintained “good bots” list, but it’s unclear what the “known bots” represents. Is it simply a list of all known bots, good and bad, which also includes the “verified bots”? I think there should be some basic information about this.
The reason I am asking, is because I would like to make a custom firewall rule to “challenge” all requests from “known bots” unless they are “verified bots”. Surely this makes some sense? The only way I find to achieve that, was to check for known bots, and then exclude all verified bots categories.
The “Known Bots” would represent all of these bots listed on that page, - so there are no difference between the two.
Except an unfortunate (and possibly confusing) difference in their name.
The category would allow you for granularty, such as e.g. if you select:
IF Known Bots equals 
AND
Verified Bot Category is not in [Security]
Action: Block, Challenge, [...]
→ Will take action against any of these Known Bots / Verified Bots, that aren’t listed in the category Security.
Alternatively, something like:
IF Known Bots equals
AND
Verified Bot Category is not in [Page Preview]
Action: Block, Challenge, [...]
→ May take action against any of these Known Bots / Verified Bots, that aren’t listed in the category Page Preview.
That wouldn’t allow search engines such as e.g. Google to look at your site, however, it may eventually allow bots that are generating page previews, such as e.g.:
But block/challenge all other kind of bots.
The above link also present the following phrase:
So wouldn’t suggest spending more time on trying to distinguish the two.
2 Likes
Ahh ok, “kinda” makes sense, although by wording one might think that “known bots” would include all known bots, even potentially malignant ones. This changes my perspective on potential WAF custom rules, although in some ways it simplifies everything.
I guess in many cases, it might make sense to include [Known Bots 
Off] in WAF rules, to simply let verified bots pass through.
Thanks for your input!
I’m not quite sure this is accurate. Today, I noticed that a rule with “ Known Bots” was blocking a bot that is not listed in Cloudflare’s Verified Bots. I disabled the rule, and the bot came through immediately. This leads me to believe that “Known Bots” may include other known bots, not just “good bots” verified by Cloudflare. The bot in mention is GKD_IPN v1.2 by 2Checkout.com, which handles payment notifications. Either Cloudflare’s “verified bots” list is not updated, or “known bots” includes other well known bots, which I suspect is the case.
This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.