Cloudflare Bot Detection Not Working

I have firewall rules in place to block all bot traffic, but there’s a ton of it coming through all of a sudden this morning. I can’t find any way to get an actual IP of these requests to see where it’s coming from. The only thing Cloudflare seems to offer in this regard is its Analytics which are pretty useless in this sense. Not sure what else to do.

What firewall rule(s) do you have to block all bot traffic? Other than “known bots”, there really isn’t a No-Bots rule.

That’s what I have in place.

(cf.client.bot)

The requests are obviously crawls and I feel like they’re getting through that filter and are known bots. I have other sites not shielded by Cloudflare getting hammered with the same stuff from Microsoft this morning. I need some way to tell, is there any way to see actual traffic logs?

It’s bots on the list? Have you verified they’re not fakes?

Those would be on your server. Enterprise plans have log access; other plans would have to get outside help, like a subscription to Logflare.

Lol, Cloudflare doesn’t push the origin IP in the header anymore so there’s no way for me to tell what it is. It’s just Cloudflare Cloudflare Cloudflare. I know they have a paid feature but it’s prohibitively expesnisive.

And it’s not like I just want everything for free, but Cloudflare doesn’t offer anything useful to me. There’s 1727 domains in my account. It’s just not reasonable for me to pay $20 / month for each domain

It does. They still use HTTP_X_FORWARDED_FOR and HTTP_CF_CONNECTING_IP, as shown in:

<?php print_r($_SERVER); ?>
2 Likes

According to the docs, you have to turn on True IP and it’s only available to enterprise users:

https://support.cloudflare.com/hc/en-us/articles/206776727-Understanding-the-True-Client-IP-Header#:~:text=Overview,only%20available%20to%20Enterprise%20users.

I guess I’ll have to look through the headers and see instead of taking the docs word for it.

From the same documentation page you linked:

True-Client-IP supplements the current CF-Connecting-IP and X-Forwarded-For headers. We recommend relying on the CF-Connecting-IP (or True-Client-IP) instead of X-Forwarded-For headers if you need the actual client (visitor) IP address.

3 Likes

Doesn’t that say you’re not getting the actual IP? Why would they recommend using something else if it’s the same thing?

This feature adds a third header, True-Client-IP , for compatibility with another vendor.

So you’re fine using the other two in your quest to block bots.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.