Substantial Increase in Data Served

Hi,

Hoping someone with a better brain than me can help me understand this, and apologies I did search but couldn’t find anything matching this use case, feel free to redirect me if I’ve missed something somewhere :slight_smile:

For the last pretty much forever my site has been tracking at comfortably less than 1TB of traffic per 30 days and ~20GB per day give or take. In the last week or so data transfer has gone up to ~430GB per day and rising.

My site(s) still appear to be available and coping well. But the telemetry I have on the backend is not showing any particular increase which would indicate a ~1700% increase in data transfer. All the telemetry I do have is very highly anonymised but I’d hope to be able to spot massive trends like this.

Under my domain I have the following proxied through Cloudflare:

  • www = WordPress site which I admit I don’t write as much as I’d like to. Hosted on a fairly modest VM
  • Sub-site = Hugo site that offers easily digestible files for firewalls, web servers and analytics platforms. These are zip files and size ranges from ~3Kb to ~5Mb. Hosted on a Storage Bucket.
  • Cloudflare Worker = Serving mta-sts records only, usually ~100 hits a day

Other than the above proxied through Cloudflare I have the usual CNAME, TXT and MX records. I will admit I have a few stale proxied records which I have since deleted but that doesn’t appear to have influenced the figures at all.

(Apologies if the screenshots are difficult to read, as a new Community user I can only post a single image.)

Since the vast majority of this data is cached correctly I wouldn’t expect to see much in the way of additional bandwidth charges from the hosting and so far that is absolutely the case. If it’s a DDoS then I’m not seeing this either in the logs I can see, I have WordPress monitoring my site themselves and they’ve not emailed saying they’ve noticed any issues.

My first thought was that the Sub-site suddenly got really popular really quickly but unfortunately I can’t see the other data backing this up. It would be nice if it has suddenly become super popular though I shan’t lie.

I will admit my concern is that although Cloudflare are undeniably awesome, I’m on the Free tier and I’d imagine someone being not overly happy with what could be at current trajectory 17TB of free bandwidth a month…

Happy to provide actual URLs for my sites if it helps, but don’t want to break any TOS for the community support. Also happy to post the actual numerical usage data from the graphs above if it’s useful.

Questions:

  • Is there any other places I should be looking for where this bandwidth is being generated?
  • Is there any other analytics I can leverage here?
  • Would upgrading to the Pro tier help with either analytics or making sure Cloudflare don’t kick me off?

Hi there,

From what I was able to see, most traffic is hitting cache, and therefore it might not be noticeable at your origin:


Over the last 24h cloudflare served 855.97GB and your origin only served 412.43MB.
99% of this traffic consists of .zip files.

In the regular analytics, this subdomain has 30x the requests that www has, but it seems pretty consistent, and apart from a weird spike on Jun 09 from 10:30 to 13:00, I don’t see any issue with it.

I do notice that there are some IP addresses doing thousands of requests daily, I don’t know if this is expected, but I assume they have some type of automated system directly linking to your assets.

Would upgrading to the Pro tier help with either analytics or making sure Cloudflare don’t kick me off?

If I’m analyzing the correct zone, and I think I am as per your description, you already upgraded to the pro plan.

Is there any other places I should be looking for where this bandwidth is being generated?
Is there any other analytics I can leverage here?

You can check the cache analytics under Caching > Overview > Data transfer.

As for advices, I see you haven’t updated the managed rules yet, so I would start there. Managed rules can be updated under Security > WAF > Managed rules by clicking Review configuration.

You also are completely allowing bots, again, don’t know if this is on purpose or not, but in case it’s not, and you rather block them, please navigate to Security > Bots, press Configure Super Bot Fight Mode on top, enable JS detections and set Definitely automated to Block or Challenge.

Take care.

1 Like

Hi there,

Thanks for the response :slight_smile: You have the right sites in you analysis, thank you for checking my logic, hugely appreciated. Yes the Pro plan gives me a heck of a lot more visibility in what is happening and is awesome.

1 Like