My website refreshes itself every day at 10am EST and I have my header directive on my webpages set to:
Cache-Control: public, max-age=86400
Pragma: cache
Expires: [Tomorrow Date] 14:00:00 GMT. (which is 10am EST)
In my research, the max-age directive will overrule the Expires directive. Meaning even if the expires data has passed, the max-age will still be valid based upon when the page was accessed. So if the webpage wasn’t accessed until 6pm, the page technically would remain in cache for another 24 hours, well beyond the desired 10am time expired time.
Since I can’t control when a page is accessed and it’s all random, my thought was to set the max-age to a smaller value, like 8 hours. This should work most of the time, but obviously it depends on when the page was accessed by the user and subsequently cached.
Is there a more optimal setting, perhaps using must-revalidate?
Typically, Cache-Control will indeed override the Expires header, when both are present.
must-.revalidate is indicates that the caches must re-validate the cache once that the data is stale (e.g. after it’s expiration).
Within your max-age=86400, it won’t become stale for the first 86400 seconds (24 hours), and as such, that wouldn’t change a thing in your use case.
These parts altogether indicate that you want a dynamic cache time, due to the random access times.
First step I would take, would be to look at your origin server / backend, and program it in there, to return dynamically calculated Expires/Cache-Control headers, depending on the actual access time.
In other words, if we’re currently at 13:00 GMT, you’ll calculate it accordingly, and return max-age=3600 in Cache-Control, and today’s GMT timestamp for 14:00 GMT in Expires.
Or if we’re currently at 09:30 GMT, you’ll calculate it accordingly, and you’ll return max-age=16200 in Cache-Control, and today’s GMT timestamp for 14:00 GMT in Expires.
Thanks for the comments and genius idea about also making the max-age time dynamic! I didn’t even think of that and would be easy to code!
I was going to do a purge cache, but purging the entire host seems to be reserved for Enterprise clients. Since I have 7000+ cached pages, I didn’t want to try do that one url at a time.
Notice that this approach will only work with must-revalidate, otherwise that request AND its headers will be cached by Cloudflare Edge, so subsequent requests for the same URL will get that Cache-Control with the max-age time defined by a first visitor (16200 in the example above), and not whatever dynamically programmed Cache-Control the origin might return if visited.
Also, setting only max-age may be complicated, as it sets the same amount of time for both Browser and Edge. What you can do is set max-age and s-maxage, the former for the browser, the latter for proxies such as Cloudflare.
Something you could do is a mixed approach, purging with a script as suggested by @eva2000 but only, say, the top n pages (in terms of visitors, or perhaps in how much time sensitive these pages are), then use Cache-Control from origin with must-revalidate and both max-age (set, say, to a day) and s-maxage set to an hour or less, depending on how tolerant you want to be with the difference between the refresh at the origin and that at the edge.
Controlling cache with Cache-Control header requires that the Cache setting for the zone or application is set to “Respect existing headers”. Otherwise you can set different values using Browser Cache TTL and Edge Cache TTL in a Cache Rule.
Or you can script to purge cache by cache age or last modified age. I wrote a cache purge script for this specific case
./cf-purge.sh
Usage:
where XX = purge based on cache age
where XY = purge based on last modified header age
./cf-purge.sh purge-url XX
./cf-purge.sh purge-url XX "https://yourdomain.com https://yourdomain.com/news"
./cf-purge.sh purge-url-modage XY
./cf-purge.sh purge-url-modage XY "https://yourdomain.com https://yourdomain.com/news"
./cf-purge.sh purge-all
./cf-purge.sh check XX
./cf-purge.sh check XX "https://yourdomain.com https://yourdomain.com/news"
./cf-purge.sh check-modage XX
./cf-purge.sh check-modage XX "https://yourdomain.com https://yourdomain.com/news"