Cloudflare's non cached static files download speed is painfully slow compared to non-proxied via CF connection

Performance
#1

Hello everyone!

So, I was serving some static files on my server and I noticed that the speed is painfully slow compared to a direct IP connection, so much that it isn’t even good to use Cloudflare when serving those kinds of files.

Here’s a difference: The top one was downloaded via a website proxied via Cloudflare, while the second one I downloaded via direct IP. Both are hosted in the same server, using the same nginx server. Both downloads were started at the same time.

firefox_3bymt1Ghl3

Here’s a GIF showing the difference in speed:
lXHHPzgkOM

The speed difference is gigantic! The one proxied via Cloudflare never reaches more than 1 MB/s while the direct download one goes way beyond that (almost 10MB/s!)

So… why this is happening?

#2

(almost 4MB/s!)

But that aside can you please post the headers of the files?
Please do like this:

  1. open Chrome in an anonym Tab
  2. open Dev-Console (F12) and click on “Network”
  3. download (in this tab) the file and inspect the header and post it here so we can see some technicall details

Can you please also share the Domain and the directlinks to the files with and without CloudFlare?

#3

The peak speed I had while downloading that 512MB file was 10MB/s, that’s why I mentioned it in the post itself

Response headers:

    HTTP/2 200 OK
    date: Tue, 05 Jan 2021 22:31:52 GMT
    content-type: application/octet-stream
    content-length: 536870912
    last-modified: Tue, 05 Jan 2021 21:52:55 GMT
    etag: "5ff4dfb7-20000000"
    accept-ranges: bytes
    cf-cache-status: DYNAMIC
    cf-request-id: 077646a32c0000f66b842b3000000001
    expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
    report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=jZ7aepAly8bTlx%2FUZpNNW0Y5wgjHgL0KD9OvmhYBspgr4PfKlo568w2NgTPHYYisCkaRH13Roj8aTu9K9AhKJPrrAhFYcIPYkDsLYS4S12WXA9vwzoEr"}],"group":"cf-nel","max_age":604800}
    nel: {"report_to":"cf-nel","max_age":604800}
    server: cloudflare
    cf-ray: 60d0a6e51ac2f66b-GRU
    X-Firefox-Spdy: h2

Now in a Firefox Anonymous Tab:

HTTP/2 200 OK
date: Tue, 05 Jan 2021 22:33:56 GMT
content-type: application/octet-stream
content-length: 536870912
set-cookie: __cfduid=d1f4902237a42abea96a4ed725df4edea1609886036; expires=Thu, 04-Feb-21 22:33:56 GMT; path=/; domain=.loritta.gg; HttpOnly; SameSite=Lax; Secure
last-modified: Tue, 05 Jan 2021 21:52:55 GMT
etag: "5ff4dfb7-20000000"
accept-ranges: bytes
cf-cache-status: DYNAMIC
cf-request-id: 0776488acf0000f76c3d2e5000000001
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=JZXJVwSUldm3o6hTz04aMZxw8V4pSwwMMvD%2FFQbHGKn57V2dxc1rVe5dRHx53WhGg%2BSM%2FK%2BdH0fPvRHhEa9dh6OmrhCFIJdKIYv%2FIIgf5AJIDtc1AwwJ"}],"group":"cf-nel","max_age":604800}
nel: {"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 60d0a9f14b90f76c-GRU
X-Firefox-Spdy: h2
#4

what about for a <512MB file as CF CDN cache max file size is 512MB for free, pro and biz plans https://support.cloudflare.com/hc/en-us/articles/200172516

The maximum file size Cloudflare’s CDN caches is 512MB for Free, Pro, and Business customers and 5GB for Enterprise customers. Enterprise customers can contact their Cloudflare Account Team to request caching of larger files.

and only some file extensions are cached by default unless you specifically configure via page rules to cache non-supported file extensions https://support.cloudflare.com/hc/en-us/articles/200172516#h_a01982d4-d5b6-4744-bb9b-a71da62c160a

#5

what about for a <512MB file as CF CDN cache max file size is 512MB for free, pro and biz plans https://support.cloudflare.com/hc/en-us/articles/200172516

I’ve already tested with smaller files (less than 40MBs) because my plan was to host a few mp4 files that I could access them via Discord & stuff but the result was the same: MP4 files were almost unplayable because the file was constantly loading.

Yes, for this use case I could just not use Cloudflare, but the reason I wanted to use Cloudflare is because:

  1. I already manage DNS in Cloudflare
  2. I don’t want to expose my origin IP

and only some file extensions are cached by default unless you specifically configure via page rules to cache non-supported file extensions

That’s a good tip and something that I didn’t know :), but in this case the problem is that, even if the file isn’t being cached, Cloudflare download speed is painfully slow compared to a direct IP connection. I don’t even care that the file is not cached, I just wanted it to download at a similar speed (heck, even if it was just 0.75x compared to the direct IP connection I would be happy!) as my direct IP connection, because, at this point, the speed is literally unusable (90% less than the speed I get when connecting directly to the http server via the IP!) :frowning:

#6

GRU Brazil data center, could be related to network peering related. I’d contact CF tech support so they can investigate on their end.

I’m definitely not seeing the issue as my downloads behind CF are pushing up to 200-256MB/s when CF cached.

#7

I’m definitely not seeing the issue as my downloads behind CF are pushing up to 200-256MB/s when CF cached.

But remember that the issue isn’t when CF cached the file (when it is cached it is blazing fast!), the issue is when the file isn’t cached yet, which takes a looooooong time just to download something from the webserver. (This issue doesn’t happen if the connection isn’t being proxied via CF or if I’m connecting directly to the IP of the webserver)

I know that proxing via CF will incurr some speed loss if the file isn’t cached, but the point is that via CF the speed is extremely slower than just connecting directly to the webserver if the file isn’t cached yet.

What I did as a workaround was adding .mp4 to always be cached via a page rule, however the issue still stands that even if it wasn’t cached, the download speed of a file that isn’t cached yet via CF is ~300KB/s while without CF it is more than 4MB/s. This could affect stuff that you don’t really need to be cached via CF but it would be nice to have a good download speed (example: You uploaded a .zip to your website and people can download it… it would be slow)

Another thing that I forgot to mention before anyone asks: The webserver is hosted at a SoYouStart server in Canada, so it isn’t “faster” just because the webserver is near me (I live in Brazil).

#8

Hi,

It is possible that this is either related to connection issues or that your zone was throttled. As @eva2000 pointed out, we’d need to further investigate this and I’d recommend you to contact Cloudflare’s support team.

1 Like
#9

I would contact Cloudflare’s support but sadly the domain I’m using for that test is not on a Pro plan (Free plan woo)

I do have a domain in a Pro plan (open source project yay) but I don’t think that it would be a good idea to contact the support team about a issue in another domain.

And I did test downloading that “foo” file via a URL in my Pro plan domain, it works fine there so I guess it would be a limitation of the Free plan. :man_shrugging:

But anyway, thanks for your attention :3

As a test, here’s the same file, on the same webserver, but using two different domains:

Pro Plan: https://testignore.loritta.website/foo
Free Plan: https://testignore.lori.fun/foo

So this begs the question: Is this intentional? I tried searching on Google to see if there is any documentation about this but doesn’t seem to have.

(And yes, I did check if the file was in CF’s cache or not, in both cases the “cf-cache-status” was “DYNAMIC”)

#10

If you’re transferring 512MB files through Cloudflare I’d point you to our Terms of Use. If you :grey: and the performance is better but I don’t think it’s a question of free vs pay-go plan

#11

The 512MB file was just a example (if you see the file contents, it is just a dummy file), I’m not serving files of that size via Cloudflare and I’m not caching files of that size with Cloudflare :stuck_out_tongue:

The same issue also happens with smaller files too. (~20MBs zip files as an example or even smaller files)

Also TIL that you are not allowed to cache files that aren’t related to HTML… Good to know that but I guess the website should make it more clear that there are restrictions to what files you can cache via page rules & etc.

The point is that the transfer of non cached files seems to be faster if you have a pro plan vs free plan, I’ve tried if three of my domains in CF and two of them had the painfully slow speed, while the one with the pro plan (via the open source program) had a higher than normal speed. None of them were cached via CF.

Of course, at the end of the day it doesn’t even matter because I’m not serving big files in my website (aside from the example I gave a few posts above, when I found out about this issue) but I’m still curious to figure out why this happens.

1 Like
#12

I’ve got this part memorized: ToS 2.8. Any time someone mentions performance related to massive amounts of large files, alarm bells go off.

#13

Yeah, in my case I was hosting a ~80MB mp4 file just for fun (it wouldn’t have a lot of connections because it was something more for personal use, the reason I was using Cloudflare for that is because I didn’t want to leak my origin IP) and that’s how I found out that the speed was slow.

2 Likes
#14

As I said, I’d recommend contacting our support in this case. That way be can investigate this issue and identify possible causes. You can contact our support regardless of the plan type.

Best,
Tom

1 Like
#15

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.