I’m trying to build a serverless app using cloudflare workers. My whole site runs through the workers. While testing the worker I found out that cf doesn’t seems to be stopping http floods. Is there any way to prevent that. I thought about rate limiting but it’s not economical to run rate limiting sitewide
You can use the CF rate-limiting feature.
But there’s a cost, 5$ per 1M requests.
Did you read the question I mentioned rate limiting in the question. I would have to pay way more if you enable it or not. Consider a high traffic website, you still have to pay more because only a small fraction of users DDOS the server .So you have to pay for millions of good requests. And if I didn’t enable it, I still have to pay more if someone DDOS the server. So the cost of running an app on worker will be expensive either way.
I read it and it’s the only known and supported way to do that.
There’s been some hacks through the last couple of years, where you can use a global to do the counting but that doesn’t prevent distributed floods (multiple locations).
The only other option is to have a hosted server that does the counting and saves it to a stateful server like Reddis which can handle the load. Then the worker would load a block-list from the reddis cluster which can be cached in KV as to not affect loading times.
To apply the block-list, you’d use the CF API and add them to the CF firewall (or they won’t be blocked before the worker loads) - which has a limited set of rules - so again, cost.
Thanks for the info anyway
I’m hoping it will be implemented sometime soon, it’s a common issue.
For a company built on DDOS protection it does seem quite bizarre that there is no DDOS protection on Workers.
A customer of mine has been a victim of a distributed cost attack as well. Our average and expected traffic is <100K reqs/day, but there are days some bad actor is driving our graph to ~200M/day. I have reached out to an engineer on Discord community and they suspended the billing for us but how long it will stay like that is surely a question of fact.
DDoS protection is enabled by default for Workers as well. Can you please either DM me the details or open a support ticket and send the ticket number so we can look into this?
That’s good to know. But the replies under this topic shows that this has occurred before. If worker have DDOS protection, Does it need any minimum required plan to prevent http flooding. Like do I need to enable something like ‘Super Bot Fight Mode’ or simply a paid worker subscription under a free domain is enough?
You need to adjust your mindset a smidge.
What’s an HTTP Flood if you have 1 user connect 1,000,000 times or a million users connect one time.
Your servers getting overwhelmed is just that. You will need to run a limit on your end and tell the worker to screw off with sending traffic my way if I am full.
So example, set 200 connections at a time, or what you feel keeps your server below a CPU%; and when you reach those connections at a single time, slow down or keep up. Think of a DDOS as excited users, don’t displease them! Most sites will absorb the attack free just do your best to mitigate.
If you absolutely need them out of your site, you need to make a safelist/blacklist and wrap around offending IPs then using the API cancel them all out.
That’s what Cloudflare WAF and CF Firewall are for just you need to provide hints via rules as to what is legit or not legit traffic to your application. Layer 7 application level attacks can’t just rely on CF automated migitation as Cloudflare would not 100% know what is good or bad traffic unless you provide those hints.
With CF Business or Enterprise plans you can also work with Cloudflare to develop custom WAF rules for your specific application and with Enterprise you have very powerful tool in Enterprise Bot Management
For me my main use of CF Workers is for full page HTML caching via CF CDN Cache so for me it’s partly used as a form of HTTP flood migitation as CF Workers at $5 per 10 million requests is still cheaper than trying to scale my origin servers to handle request spikes itself
worker isn’t sending traffic anywhere. I’m talking about the billing occurred due to worker requests
Blockquote Layer 7 application level attacks can’t just rely on CF automated migitation as Cloudflare would not 100% know what is good or bad traffic unless you provide those hints.
I think cf should block or capcha challenge those requests when a single IP starts sending thousands of request per second
Yeah that does seem simple but it probably isn’t. To be able to rate limit requests you need for a way to count all the requests and store them across all CF data centers and that adds to operating cost. Hence why CF Rate limiting is a paid service.
But you can still do rate limiting at your origin server end which will return a 503 error on CF CDN edge server end to visitors. That’s what I do for rate limiting and micro full page HTML cache on origin servers and also optimize my origin Nginx servers, PHP-FPM and MariaDB to handle such spikes. My WordPress origin servers can handle 10,000 concurrent user requests per second without Cloudflare on a 1-2Gb VPS server and on dedicated servers I’ve tested my Nginx optimised servers to 80000 to 350000 concurrent connections and then I add Cloudflare with CF Worker full page HTML caching on top with CF WAF & CF Firewall rules
One additional way is to install and configure fail2ban rules to talk with Cloudflare via Firewall API so bad requests determined by your origin fail2ban rules pass the bad IP to be blocked at CF Firewall level. This can be combined with rate limiting at origin server level so rate limit requests get logged and then fail2ban picks up the logged rate limit IP for passing to CF Firewall API
Blockquote Hence why CF Rate limiting is a paid service.
I don’t have anything against your argument. But when you run a simple fully serverless application on workers, Rate limiting costs more than running the application. Lets just say that am doing something simple with worker and it was invoked 10m times. I would be paying 10x more for Rate limiting than for the worker requests
There is no minimum required plan. Unmetered DDoS protection is provided to all plans including the Free plan.
Hence on my edited reply above I mentioned doing rate limiting on your origin server too to save on costs. But yeah different story if your app is serverless without an origin
Is that just for volumetric network attacks or attacks you deem as such. For level 7 application level attacks such as HTTP floods alot aren’t picked up out of the box. Especially if the HTTP floods are coming from 1000s of IP addresses. I am blocking HTTP floods targeting specific urls of my forums with custom CF Firewall rules which can peak above 650,000 requests. Cloudflare does not automatically consider them attacks so I had to tell it via Firewall rules
Yes, Being serverless without origin is the problem. Else I could put rate limiting on server and add the IP dynamically to cloudflare through api to block the request at cf.