How many calls do get through to origin when cache is deemed stale

When a resource is deemed stale by Cloudflare, and a call happens, Cloudflare fetches content to from the origin, and serves it to the caller. The response is tagged with the EXPIRED header. If someone else makes a call when Cloudflare is fetching the content from origin, Cloudflare serves the old, and stale ressource from its cache, and tags this with “UPDATING”. This is perfectly fine, and the way i want it to work. You only get on call, the first, that hangs a bit, but every other that calls get a stale response and thus not hitting my origin.

What happens under high load? Can it potentially be many that hit my origin? Imagine that 100 hits the resource right after it has become stale at the exact same time (with som miliseconds apart). Does all 100, or a subset of this, calls my origin, or can Cloudflare figure out to only let one of the call through?

Well, Cloudflare will serve a cached resource as long as the resource is cached on the Cloudflare edge, and for that to happen, it must have completely loaded from origin->Cloudflare. Any requests to the resource after it is cached, will be served from cache. In the relatively extreme scenario where you have multiple requests “at the same time” + “before a resource is cached”, I would assume Cloudflare will have to fetch it from origin. I doubt there is any magic on Cloudflare that would hold one request, waiting for the request to cache for another request, made a few milliseconds earlier.

If you do in fact get several requests at the same time, surely it can’t affect more than an extremely tiny percentage of your requests anyway. How many requests can you get within a timeframe while a request is being cached, compared to the amount of requests you get for the cached request (which might be cached for hours, days or even months).

Keep in mind, Cloudflare has 100’s of edge servers. For example, if you have visitors from three different locations, you will get three hits to origin before the resource is cached on all three servers.

2 Likes

I have tested this, and with a cold cache, and many hits in a small amount of time, all most all requests get through to origin. If i have a max-age=60 and i let the cache be for several hours, the requests is actually acting as i thought: only one request gets through, the rest gets the stale ressource (even though it should have been removed hours ago - and i get it that Cloudflare will hold it longer than 60 seconds, but several hours later they can still deliver a stale response. Maybe because it has been hit so many times that Cloudflare sees it as “hot”? I dont know).

Thanks for your response.

With what max-age / edge cache TTL? Depending on the edge cache TTL, you should definitely see a large cache hit. If it’s set as low as 60, you may get a very low cache-hit, but depends on many factors.

I must admit I am not 100% sure what you are saying. If you set max-age to 60 seconds, indeed the resource will be re-requested from origin again after 60 seconds, in which case it will again cache for 60 seconds. That could mean for example, if you have 3600 visitors spread across 1 hour, 60 requests will go to origin, while 3540 visitors will get the cached result, as it gets re-cached every 60 seconds. However, by example, it could also mean that if you have 59 visitors spread across 1 hour, all requests might go to origin if there is 60 seconds between each request.

I am not quite sure what you mean by “stale” response. There is an “expired” response, but that means the request will fetch at origin. If you are experiencing requests being cached past their Edge cache TTL time (normally from max-age), I would be happy to check an example link …