I have wordpress site and i am using Cloudflare free plan at the moment.
I have noticed some very high traffic peaks which overload my system services (mysql, nameserver or spamd, most of the time it’s the mysql) with OS Out of Memory event and it goes down.
I have investigated the logs and it seems that there is a Python/aiohttp script coming from different IPs which is flooding my wordpress with requests for existing and not existing pages.
Does CF detect and stop such traffic?
Is it possible that some traffic to my site does not go trough CF?
Can you advice how can i protect from such scripts in general?
Without knowing many of the variables which make up the issue you’re seeing, I’d implement more stringent firewall rules to mitigate this bad traffic on the CF free plan. On paid, I’d look at a WAF to stop this traffic.
This is a high level overview - there’s so much that goes into why you’re getting that traffic, but this is what comes to mind if one of my applications had an influx of bad traffic.
I think that of the IPs might be even CF IPs.
Is it possible that they can query the pages to index them?
Actually my bad, even though i have apache cloudflare module installed, i think it still logs CF IPs.
Can you show us an example of the traffic you’re seeing, and what pages it’s hitting?
It’s a python/aiphttp script which is scraping all my pages and other not existing urls in my site.
Basically it sends a few hundred requests in a very short period of time, which overload my services.
It runs from many different IPs, i checked some of them and they are from some UK hosting company.
I would need some throttling or page limiting feature to detect and block such IPs.
no need to pay for anything, just setup apache to rate limit and limit connections. This should solve the issue. Also caching with cloudflare really helps take loads from traffic, so consider caching your pages.
also, if the bot has a certain user agent you can setup a firewall rule or page rule to deny traffic from the user agent (such as, python-requests/2.2.0)
This topic was automatically closed after 30 days. New replies are no longer allowed.