For the last few weeks my origin server has been under a sustained 1GB per second DDoS attack. Some of you might have read about it here. ( I know Cloudflare can’t help me if my origin is exposed, bear with me )
After working with my hosting provider to mitigate some of the traffic, changing IP addresses, verifying firewall rules and finally moving to an entirely new data center, the attacks have continued with a very short turnaround. After an IP address change, the DDoS attack would resume in less than 2 hours.
My main question is how are they doing this? How are they able to bypass Cloudflare and discover my origin IP so effortlessly? I’ve read the Cloudflare documentation on best practice, and I think I’m doing everything right. I have to be missing something painfully obvious, and at this point, I’m out of ideas about what I can do here. It seems no matter the mitigation I try, the attackers seem to have unlimited resources.
Below is a summary of what I’m working with, the origin server is running Ubuntu Linux 20.4 LTS:
- Origin server configured to only accept TCP traffic on port 443 from the Cloudflare IP address list.
- SSH port configured using key based authentication, and only accepts connections from a specific IP address.
- Server IP isn’t listed on Shodan.
- Server IP isn’t listed on Censys.
- Cloudflare proxying A and AAAA records.
- No mail servers.
- No rDNS record set.
- Hosting provider firewall enabled, also whitelisting Cloudflare IP addresses.
- No rate limiting for Cloudflare IP addresses.
- Some TXT records are present for Google Verification & KeyBase verification.
During the attack, my origin NIC is flooded with UDP packets, it looks to me like just junk packets to try and drain system resources. Errors seen on the frontend are a combination of 520, 521, 522, 524 and 526, with POST requests being the slowest.
I should mention I am a Cloudflare Free plan member.
Can anyone provide some insights into what could be happening here? I know there are a multitude of ways to discover origin IP addresses, but this is getting out of hand for me and I don’t know what I’m missing. Please feel free to ask for any configuration clarifications.