The way you handle bandwidth over-usage

Hello i won’t lie. I am mad.

Let me start by saying that the way you handle “Article 10” enforcement is the worst possible way.

This is what happened to me:
I added storage servers to my website under several sub-domains, and i made the mistake to assume huge compressed files (.rar .zip .7z) would not be cached, i honestly thought they weren’t cached by default and you needed a “cache everything” rule for them, but i learned that’s not the case at all.
As a result i went from consuming 80 GB of daily bandwidth to 10 TB.

This went on for 15 days. Until yesterday when my site went offline. And i received an email about violating article 10.

It took me a while to figure what did i do wrong, i had never looked at the bandwidth stats but when i realized what had happened i immediately disabled cloudflare for all subdomains (turn off orange cloud) and bought business plan ($200), After several hours and i mean like 8 hours, the site comes back up. Trying to get an update of the situation via support ticket does not produce any luck. One would think that on a $200 a month plan they would reply faster but no.

Look, i’m an idiot but even i know you are terrible at this. Let me give you some ideas on how you should have handled this:

  1. Repeat a thousand times “NEVER TAKE A CLIENT SITE OFFLINE”.
  2. You contact me on day 1 so i can correct the setup (not day 15 to disconnect me). Even a kid can code something like: IF (BANDWIDTH*24hs > 1TB) {SEND EMAIL ALERT} (that’s pseudo-code don’t copy/paste)
  3. See number 1. If i’m using so much bandwidth why not go to DNS ONLY mode bypassing all caches? Why stop resolving DNS? Seriously.

I honestly cannot believe a huge company like Cloudflare is so bad at this. Seriously Cloudflare? I hope the person in charge reads this and if he gives a hug reflects on why he does what he does.

Thank you.

Which traffic are you talking about here? Your host’s?! In thast case I am not sure what you are referring to as you said yourself you did not expect them to be cached (which they are not by default) and hence hit your server.

Can you elaborate on what the actual issue is?

you are right! It’s not even cached by cloudflare so now i’m even more puzzled. Why am i being treated like a criminal for 60 GB a day? Support does only vague answers, not concrete numbers.

I guess the real issue is my first post except they are mad at 60 GB a day not 10 TB like i thought. Ridiculous!

Treated by whom? Can you explain what exactly happened and who sent what message?

I think i understand now, it was supposed to bypass cache but the server wasn’t setup to handle https traffic so it wen’t offline.
I need at least some guideline on how much bandwidth is allowed on each plan:

FREE: 1GB a day
PRO: 10GB a day
BUSINESS: 100GB a day
etc

What did support say? Usually there are no traffic limits, however if there is a potential abuse of service (which the message seems to hint at) there can be still a suspension.

Is the network at risk for 80 GB a day? Are they trolling me?

Shane - Cloudflare Support
Join the Cloudflare Community

support.cloudflare.com/hc/requests/1596389

Shane (Cloudflare)

Nov 5, 5:02 PM PST

Hi ***,

Caching non-HTML content on Cloudflare is covered by Section 10: Limitation on Non-HTML Caching of our Terms of Service:

https://www.cloudflare.com/terms/

These terms are in place to protect our network and other users of our network - so the important thing to note here is we will act upon these when a specific website’s traffic is large enough to start impacting the service we offer to everyone. What this means is we’re not automatically stopping service for anyone caching a high proportion of non-HTML content, but if your traffic grew to a size that was impacting us and our customers, we’d look at it in more detail and at that point decide whether you are violating Section 10 of our terms.

These Terms of service apply to all self-service plans and are negotiable based on your requirement as part of our Enterprise plan, so if that is something you’re interested in specifically I can put you in touch with our Enterprise team.

See below if you’re interested in our Stream product.

Stream is a video-on-demand platform for developers and content teams building video applications. With Stream, it’s easy and fast to upload a video and get an embed code. In the background, Cloudflare encodes it, stores it, delivers it, and takes care of optimizing the right format and bitrate for every device and network connection.

Please visit our Getting Started guide to start uploading videos.

We will mark this as solved for the time being. Please let us know if you have any further questions or issues by replying to this e-mail or ticket to have it automatically reopened.

Kindly,

Shane
Join the Cloudflare Community

They seem to refer to

2.8 Limitation on Non-HTML Caching

The Service is offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as a part of a Paid Service purchased by you, you agree to use the Service solely for the purpose of serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. Use of the Service for the storage or caching of video (unless purchased separately as a Paid Service) or a disproportionate percentage of pictures, audio files, or other non-HTML content, is prohibited.

Thats a new to me as well. While 80 gigabytes is not little I was still not aware of such a limit.

Anyhow, I am afraid only support can help you in this case.

Now i’m scared to do anything. I’m scared to re-activate the sub-domains even though that traffic is not cached at all. Maybe they don’t want DNS traffic either?

I did have a “cache everything” rule on the wp-content folder (All JPG/PNG files) which i suspect was the cause of most of the 80 GB a day traffic. So now i removed that rule. But i’m even considering converting that same rule to a “bypass all caches”. Since 80 GB a day seem o be a “massive network disruption traffic” i’ll let my server handle it.

I wish support talked straight to me, or at all.

I don’t think the problem was the caching… I think they want you to take all the non html bandwidth out of cf at all, you can put the files in subdomain and just make that subdomain a grey cloud so the files will not use cf bandwidth at all.

I am totally not sure if they are also talking about images or just the .rar and .zip files

but you need ofcourse to talk to someone official

I think you misinterpret what the orange cloud does. Cloudflare largely doesn’t care about what files you set to be cached, but it does care about how much bandwidth you send through them. If a DNS zone is set to :orange:, all traffic is being proxied. This means that, while you only did ~60 gigs of cached files, that amount of bandwidth being proxied raises some alarms and disables your domain to make sure they don’t waste thousands of dollars on bandwidth.

If this is resolved and you can re-activate your subdomains, you will need to make sure they’re grey-clouded :grey: so that it’s only DNS.

I see, ok so sub-domains must be gray even though they are not cached at all they still consume traffic. I didn’t know this. I only wanted DDOS protection.

I will work on reducing these 80 Gb’s of cached traffic too.

pay attention that if the subdomain files sits on your website server it will reveal you website ip and it will not be totally protected against ddos

I know but what can i do? They will want enterprise plan to handle that traffic, and i can’t afford it.

Did they ask for that? I’d first clarify it. However I can easily understand why 12 terabyte of traffic in one day(!), which it seems to be, could justify an enterprise plan.

These Terms of service apply to all self-service plans and are negotiable based on your requirement as part of our Enterprise plan, so if that is something you’re interested in specifically I can put you in touch with our Enterprise team.

Yes they suggested it. I cannot get in touch with support for 12 hs now. I think they went on vacation all at the same time. If this is business plan i’m not impressed. I can say it doesn’t worth it, support doesn’t exist.

You are on a business plan?

Yes sir.

That should definitely have a better response time than 12 hours.

Maybe @cloonan?

1 Like

Mention @ryan and @cscharff as well, they have different schedules.