CDN rate limit for new assets

caching

#1

Can anyone confirm if there is a limit on the number of assets that can be loaded concurrently by the CDN?

We see the following behaviour which can be consistently reproduced:

2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET / HTTP/1.1" 200 10460 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/logo_horizontal_black.png HTTP/1.1" 200 6529 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/font-awesome_4-7-0_thumbs-up_256_0_2ea3f2_none.png HTTP/1.1" 200 2744 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/font-awesome_4-7-0_group_256_0_2ea3f2_none.png HTTP/1.1" 200 4270 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/payment_methods.png HTTP/1.1" 200 9419 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/logo-hp-8.png HTTP/1.1" 200 4474 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/logo-intel-8.png HTTP/1.1" 200 5167 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/typicons_2-0-7_stopwatch_256_0_2ea3f2_none.png HTTP/1.1" 200 5838 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/logo-plesk-6.png HTTP/1.1" 200 4024 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/uploads/logo-dell-8.png HTTP/1.1" 200 4263 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-includes/css/dashicons.min.css HTTP/1.1" 200 28645 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:10 -0500] "GET /wp-content/cache/autoptimize/css/autoptimize_a9c300cd5f209edcd402ba626c406323.css HTTP/1.1" 200 69285 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:11 -0500] "GET /wp-content/themes/Divi/js/custom.min.js HTTP/1.1" 200 60713 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:12 -0500] "GET /wp-includes/js/jquery/jquery-migrate.min.js HTTP/1.1" 200 4014 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:12 -0500] "GET /wp-content/themes/Divi/core/admin/js/common.js HTTP/1.1" 200 573 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:25 -0500] "GET /wp-includes/js/comment-reply.min.js HTTP/1.1" 200 589 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:26 -0500] "GET /wp-content/uploads/Flag-US.png HTTP/1.1" 200 120 "https://domain.com/wp-content/cache/autoptimize/css/autoptimize_a9c300cd5f209edcd402ba626c406323.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:25 -0500] "GET /wp-includes/js/jquery/jquery.js HTTP/1.1" 200 33766 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:26 -0500] "GET /wp-content/themes/Divi/core/admin/fonts/modules.ttf HTTP/1.1" 200 37445 "https://domain.com/wp-content/cache/autoptimize/css/autoptimize_a9c300cd5f209edcd402ba626c406323.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"
2001:19f0:6801:46:5400:ff:fe06:f84b - - [24/Sep/2018:12:57:26 -0500] "GET /wp-content/uploads/Cityscape_Butler_20170314.jpg HTTP/1.1" 200 205852 "https://domain.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3542.0 Safari/537.36"

What this shows is that 15 resources are requested/loaded within around 1s and then the remaining 5 resources are loaded ~15s later. This causes the page itself to have a >15s load time. The results are the same for requests from different geographical regions.

This behaviour only occurs the first time that the content is loaded by CloudFlare’s CDN - once it’s cached the pages are incredibly fast to load.

By contrast if we use a different/third-party CDN all assets are requested concurrently and are loaded in <2s (the first time for uncached assets). This behaviour has also been replicated across multiple CDN providers.

Is this normal behaviour for CloudFlare’s CDN when requesting/loading new assets?

For reference we are using CloudFlare free.

Thanks,
Chris


#2

I have a page that loads about thirty thumbnail in parallel. And they weren’t cached. Total page load time was 1.56 seconds.


#3

There’s no rate limiting, it comes down to how your html assets are ordered/loaded according to your site’s style and configuration. You’d want to look at how your web pages html assets are configured to be loaded style wise. Best tool for that is using webpagetest.org. I wrote a guide for my users which maybe useful to you as well https://community.centminmod.com/threads/how-to-use-webpagetest-org-for-page-load-speed-testing.13859/

Though a lot of the deferred assets from your log show as javascript so make sure if you have Cloudflare rocket loader enabled

Although from my experience if you have rocket loader enabled and configured properly - page load speed can be better for user perceived page render time as you’re more likely to have optimised your pages critical css render path above fold when Cloudflare rocket loader is enabled. Full page load time my seem higher but if you optimise the visual render time and critical css render path, your visitors will feel and experience faster page loads for assets that actually matter on their browser above the fold viewport. You can verify this by testing your site with webpagetest.org and focus on SpeedIndex times and start render and document content loaded times and if you enable webpagetest.org lighthouse report also see your first meaningful paint and perceived visual speedindex and input latency times.

Rocket Loader prioritises your website’s content (text, images, fonts etc) by deferring the loading of all of your JavaScript until after rendering. On pages with JavaScript, this results in a much faster loading experience for your users and improves the following performance metrics:

  • Time to First Paint (TTFP)
  • Time to First Contentful Paint (TTFCP)
  • Time to First Meaningful Paint (TTFMP)
  • Document Load

To learn more about the latest version of Rocket Loader visit our blog:

https://blog.cloudflare.com/we-have-lift-off-rocket-loader-ga-is-mobile/

Addition optional configuration:


#4

Thanks for your replies.

The results are the same both with and without Rocket Loader enabled - there is always a ~15s delay when loading the assets for the first time if they are not already cached by the CDN. Once cached they are very fast to load.

Some of the assets that are loaded after the delay (eg jquery.js) are loaded at the top of the page header so I don’t see any logic in why they would be loaded so late - eg jQuery is the first JS file to be loaded on the page but yet it’s the 17th asset to be loaded according to the log in the original post (15s after the original page request).

When we use a third-party CDN all the assets are loaded immediately with no delay. From WebPagetest.org the Start Render is 2.000s and Speed Index is 3124.

To me it seems that CloudFlare’s CDN doesn’t request all assets in a timely manner which creates an unacceptable loading delay (15s page load is unviable hence why we need to use an alternative CDN).


#5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.