Getting crawled hard by Cloudflare IP addresses

We just recently got our sites on Cloudflare WordPress APO and caching, but almost immediately we are getting a lot of crawlers from hosts with IP address of 172.70.xx.xx.

These hosts all have their user-agent set to either google crawler or amazonbot.

But if I do an IP lookup, these IP belongs to cloudflare. What is going on? Should I block them?

Welcome to the Cloudflare Community. :logodrop:

This sounds like you are not restoring vistor IPs st your webserver.

When you use Cloudflare, your vistors connect to the Cloudflare proxy which then connects to your webserver. If you don’t configure your server to use the vistor IP, you will log the traffic as being from the Cloudflare proxy IP. You dont want to block those as it would affect your human vistors as well as any automated traffic.

Start by configuring your webserver to record the actual visitor IPs.

Once you are receiving the actual visitor IPs you can add rules to Cloudflare to block or challenge them. You may also want to look at Cloudflare’s solutions to protect your domain from bad bots.

2 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.