Known Bots

Under the Firewall Rules, there is a rule that is called “Known Bots”. Is there a way to find out which bots those are? The reason I ask, my site has been getting hit with spam bots and it’s pushing my CPU resource usage thru the roof and it’s causing my Host to limit me. I’ve gone in and blocked a few of the spam bots individually, but no change. However, when I use the “known bots” option, my resource usage goes down significantly.

The issue is, I think it also blocks GoogleBot because when I try to index a url using Webmaster Tools, it says the URL can’t be indexed.

I’m hoping to find a list of what “known bots” includes so that I can determine which one I need to block and leave googlebot working.

You should make sure your robots.txt is setup correctly so bots that do respect it, will stop indexing your site and a list of Cloudflare’s known bots can be found here.

This topic was automatically closed after 30 days. New replies are no longer allowed.