Automatic DDoS protection & Heroku

Hello everyone,

I have the following scenario:

  1. There’s an app running on Heroku
  2. There’s a standard Cloudflare setup, where the Firewall → HTTP DDoS attack protection is set to “High” sensitivity & block action for all.

When I run my DDoS tests, nothing from this automatic DDoS protection gets triggered.

Reading from here - https://support.cloudflare.com/hc/en-us/articles/200172676-Understanding-Cloudflare-DDoS-protection - there are some specific details:

Cloudflare determines the error rate based on all HTTP errors in the 52X range (Internal Server Error) and in the 53X range, except for error 530.

When the Heroku apps get overwhelmed with requests, it starts timing out, returning 503 errors.

Is that the reason why the automatic DDoS protection is not being triggered? It expects 52x / 53x errors, but Heroku is returning 503?

If that’s the case, can this be configured?

Cloudflare has to develop a generic definition of a DDoS attack detection that fits millions of websites across the web. While it sounds simple, detecting DDoS attacks is a big challenge before mitigating them (two separated layers).

In many cases, customers’ testing does not reassemble what Cloudflare sees in real case scenarios. Another option could be that the attack isn’t big enough to trigger an alert.

Either way, my advice is to have your monitoring system on top of Cloudflare that double-checks your website’s availability. That way, if an attack went unnoticed, the monitoring system would catch your site being slow or down.

Can you share how you did the testing? It should give us some sense as to why you were not alerted.

1 Like

Hello & thanks for the reply!

Let me correct myself:

It’s more of a case of HTTP flooding & not a large-scale DDoS attack.

The way we test it is with a suite of locust tests and different loads.

For example, the application goes down when we hit it with 200 RPS and above. We catch that with additional monitoring and if that happens, someone manually activates a set of page rules, that mitigate the spike.

What I’d like to do is to make this more automatic & that’s why I was wondering about the automatic prevention.

I was imagining I can define some kind of “rate limit” per “source IP” and say what should happen - either challenge or straight-up block the request.

https://support.cloudflare.com/hc/en-us/articles/360053216191-Understanding-Cloudflare-DDoS-alerts

  • HTTP attacks for a duration over 2 minutes that generate more than 2,000 requests per second

When attacks are smaller, the mitigation might kick in without sending you any notification. IMHO this is confusing. I already pointed out to CF that it can be problematic because rarely any service can withstand that load on production. Even if the mitigation kicks in, it might not mitigate 100% of the attack.

I suggest setting up an external monitoring tool; many services can do this for you cheap (or even free).

This exists, and it’s called rate limit, Rate Limiting | Advanced Network Rate Limiting | Cloudflare note that it has a pay-as-you-go model that can get expensive quickly.

You can change the DDoS Protection sensibility. However, it might not be enough to fit your needs. Your best bets are the following:

  1. Setting up external monitoring tools that trigger the pre-defined rules to kick in (needs custom development).
  2. Buy a managed Cloudflare service; this is typically the best option for critical applications that require constant monitoring and tunning of rules; however, it can be pricey and is typically tied to a 1-year contract.

Again, thanks for the quick reply!

Thanks for giving this link. Somehow I missed it, even though I went thru the entire documentation :grinning:

2000 rps is way above our current setup for the application (although, improvements are being pushed constantly), but it’s good to know.

We’ll further investigate what are our options.

One that I can think of is to use Cloudflare’s API, hook to the external monitoring & activate the said page rules automatically, when there’s an increased traffic alert.

Thanks!

1 Like

We do this, and it works flawlessly!
The docs might be confusing at times, but they are getting better. If you have questions while implementing that, feel free to create a new topic, and somebody will help.

1 Like

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.