Origin shield


#1

The size of the Cloudflare network is quite impressive, and in theory this gives Cloudflare customers lightning fast speed.

However, a problem with an ever-growing network is the increased likelihood of a cache MISS for small to modest websites. After all, the more data centers there are, the greater the likelihood that a particular requested file is not cached in the visitor’s region.

An origin shield will help with that. For the bigger websites, an origin shield will also prevent a spike of traffic at the origin server after a cache purge.

(Since an origin shield requires more storage on Cloudflare’s end, I’m totally onboard with making this a feature for paying customers only.)


Testing Argo Performance
#2

isn’t that partially controlled by Edge Cache expire TTL value ?

This is the refetch time for when Cloudflare checks the origin server for a new resource.

so a larger Edge Cache expire TTL value will allow smaller traffic sites to hold the cached data on Cloudflare’s Edges for longer


#3

Not to my understanding. To my knowledge, the Edge Cache is the maximum time that a resource is kept in the data center’s cache. If that resource is not particular popular enough (meaning, requested more than a certain treshold), the data center removes it from the cache.

This makes of course sense to keep the storage of the data center from exploding. But with more data centers, a certain file (like style.css) is requested less often at each data center (because traffic is spread out amongst the POPs).

(This is at least my understanding of it.)


#4

Thanks for raising this, we are definitely working on ways of solving this problem as our Network grows.

For assets that are requested worldwide, setting longer edge cache TTLs using page rules is a reasonable measure (remembering of course you can always purge by URL if you need to) to ensure items stay in the cache for longer once they are populated. Remember of course as we operate in a pull mode we’ll only request assets that are being requested by visitors in that location, so unless your traffic is truly very global, you won’t necessarily see requests from every location of ours.


#5

Maybe Cloudflare can add a pre-warm edge cache feature which is available as a page rule to set a time interval to do a pull from the origin from their edges?

So you could pre-warm cache pull from origin every hour :slight_smile:


URL Pre-loading for maximum HIT response - goal
#6

It’s a good idea in principle and fairly common for CDNs to do this, but something to consider is that this could end up populating the cache in locations where it might not ever be requested, which is placing unnecessary load on your origin and also caching something that won’t ever be used on Cloudflare. There are other ways we’re looking to help people who share this concern so stay tuned!


#7

indeed you might need to inspect what’s the most frequently accessed requests per edge cache location within say last 24-72hrs time frame and only pre-warm those requests ?

if pre-warming was tied into page rules, you could specify specific assets on the page to pre-warm i.e. commonly requested css and js and common framework assets i.e. bootstrap, jquery etc

being able to optionally pre-warm the cache also allows folks to properly benchmark and compare their sites i.e. page load speed performance for cache hit vs cache miss situations


#8

@JuM we have a product in development (testing in prod and being monitored at the moment) that will solve this. It’s logically similar to tiered caching but takes advantage of the view of the internet that Cloudflare has.

If you are interested in beta testing it and are an Enterprise customer then get in touch with your account manager who should be able to get you on to the beta. I’m only an engineer so cannot help you get onto it, you need your account manager to do so.

The beta is predominantly open to Enterprise customers because:

  1. typically the larger customers who are putting significant traffic through the network are the Enterprise customers and those are the ones for whom this makes the most sense and we are already talking to
  2. the nature of a beta is that we would like interaction/communication about the benefits that the customer sees too and larger customers typically have better insight into their own systems and will be better placed to see the benefits from their side (rather than solely relying on our measurements)

When launched it will be available to non-Enterprise customers, so if you are not an Enterprise customer you only need wait a little longer and this will be announced and made available.


#9

Thanks for the update and insight into what’s coming (also thanks @simon). I’m not an Enterprise customer, so I look forward to the blog post.

For now I’ll hold off my enthusiasm until the pricing and Cloudflare’s view of the internet is clear. :slight_smile:


#10

This will be a very useful feature! Can’t wait till its released.


#11

Hi All,

Can you create Cache headers in .htaccess file or are they set on the Server?

I am not a developer and just learning about Cache, headers etc

Also can you set different Cache headers for different URLs or pages?

Explanation of my questions: We have some website where product images or content may change regularly, so want to make sure that if something changes, lets say a product image or promotional banner, that the latest gets served, rather than a cached version, we plan to move these types of websites to Pro so can have Browser Cache Expiration at 1 hour, or better use headers.

CF support have been great giving me info on Cache headers but now I need to learn how to set them up etc

Thank you
Stuart


#12

If you have mod_expires installed in apache you can configure this in your .htaccess, here’s an example guide:

https://www.fastcomet.com/tutorials/website-optimization/mod-expies

Note that you don’t need to do configure cache headers on your origin if you don’t want to or are not able to. You can set both the Browser Cache TTL and the Edge Cache TTL using Page Rules:


#13

Hi,

You can set

Cache-Control:public, max-age=1, s-maxage=3600

You can set custom value of edge cache TTL for any file using s-maxage
For browser cache, keep max-age value greater than 0 in public mode.

Hope this helps.


#14

Hi, is there any new info or timeframe to when this new product will be released? (or is this ‘AGRO’? )
Thanks!


#15

This is a feature called Argo Tiered Cache (not sure if it is included in non-ENT plans by default) which I believe addresses the original concern. It allows POPs in a region to check with a tiered cache to see if it has the content (if the first POP doesn’t already have it). If it’s on the tiered cache server it would request from origin and then make it available to the POP which requested it (and store a copy). Subsequent requests from other POPs in the region to their regional tier would retrieve from it.