So I’ve had a site thats been under a pretty heavy load lately. I have a dedicated server with a VERY respectable stats, with only 2 sites on it. Both of these sites are protected via Cloudflare.
For the last few weeks, my dedicated server, which usually runs at about a .9 1 minute average, has been running at anywhere between 30-70 constantly, for a couple of weeks. Yesterday, I finally had some time to dig into it. I found out that my tcp stack was overloaded, and that none of my outgoing traffic (recaptcha, emails, etc) was working.
I’ll leave the multitude of steps I went thru off of this explanation, as its pretty lengthy. One of the things I finally did (after I I implemented iptables tules to verify my traffic was ONLY coming thru cloudflare) was to install mod_cloudflare to get the original IPs from the incoming traffic. I hadnt bothered doing this at first, because I believed that if I was being hit from a specific ip or ip range, Cloudflare’s DDOS would have kicked in.
Yeah, I was wrong.
So, I found out that more than 95% of my connections was coming from ips ranging from 66.249.64.xx to 66.249.79.xx. And each one of those was hitting a dynamic page causing processing of both php and mysql climb thru the roof.
I went into cloudflare and edited the firewall rules of my free account, and blocked 184.108.40.206/16 (yeah, I know I could have been more exact, but at this point it was 4am and I wanted to nail this thing).
Boom. My traffic dropped and my box load came back down to less than 1.
Ok. So thats where I am right now. The traffic is blocked and my sites are fine.
BUT, I am now blocking Googlebot, which means my sites can no longer be indexed by google.
So, my questions:
Was this actually Google? Has someone figured out how to weaponize the Google bot (like I said, this has been happening for several weeks at a minimum, constant)? Or can people spam and make it look like google?
Someone said if I upgrade to PRO that the WAF has something that would automatically block this. Is that true?
Someone ELSE said that this was definitely a DDOS and Cloudflare’s mitigation should have stopped it. This makes me wonder, though, if they have special logic that allows “known bots” unmetered access?
Before someone asks, yeah, I turned on the “Im under attack” logic but that didnt seem to mitigate the traffic at all. It even went up higher at the same time I activated it (coincidence, Im sure).
So… What do I need to do here? I can upgrade to Pro of that will help, but I need some direction. My sites survive because of Google traffic and I cant leave that range blocked forever.
Any assistance would be appreciated.