Possible caching bug in cloudflare

Hi,

I am little confused. Please Explain. If I clear the browser cache completely And access a page

% curl -I https://www.example.com/

HTTP/2 200
date: Fri, 15 Nov 2019 14:30:16 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=d602eca7becb4864082f6c2ea38f2ae9e1573828216; expires=Sat, 14-Nov-20 14:30:16 GMT; path=/; domain=.example.com; HttpOnly; Secure
cache-control: private, max-age=14400
expires: Thu, 14 Nov 2019 21:02:49 GMT
vary: Accept-Encoding
cf-cache-status: HIT
strict-transport-security: max-age=31536000; includeSubDomains; preload
alt-svc: h3-23=":443"; ma=86400
expect-ct: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct
server: cloudflare
cf-ray: 5361ec10f92edd26-SIN

I accessed page at date: Fri, 15 Nov 2019 14:30:16 GMT. Expires header is saying it is stale resource. showing as expires: Thu, 14 Nov 2019 21:02:49 GMT. While cache-control saying it should be valid for next 4 hour. Expires should show expires: Fri, 15 Nov 2019 18:30:16 GMT

There is a simple test case required to generate this issue. As per my understating Expire is always = access time + max-age (if browser cache is cleaned). Generated a ticket but got a very generic answer regarding how to set up cache headers.

Using this page rule-

Browser Cache TTL: 4 hours, Cache Level: Cache Everything, Edge Cache TTL: a month, Bypass Cache on Cookie: wp-.*|wordpress.*, Cache Deception Armor: On

For Example in apache when you set up mod expire like this

<IfModule mod_expires.c> 
ExpiresActive on 
ExpiresDefault "access plus 4 hours" 
</IfModule>

and access a resource like
curl -I https://www.example.com/resource.txt
u will get these thing in result

date: current date and time (Lets say for example time Fri, 15 Nov 2019 14:30:16 GMT )
cache-control: max-age=14400
expires: current date and time+14400 seconds (time Fri, 15 Nov 2019 18:30:16 GMT )

Regards,

I have another example to put my point.

<IfModule mod_expires.c>
	ExpiresActive on
	ExpiresByType image/jpg "access plus 24 hours"
</IfModule>

Now If i have an image says https://www.example.com/magic.jpg. I cleared browser cache. and use curl.

curl -I https://www.example.com/magic.jpg
date: current date and time (for example Fri, 15 Nov 2019 14:30:16 GMT)
cache-control: max-age=86400
expires: current date and time+maxage (Fri, 16 Nov 2019 14:30:16 GMT)

and wait for 30 minutes and clear browser cache. and run again.

curl -I https://www.example.com/magic.jpg
date: current date and time (for example Fri, 15 Nov 2019 15:00:16 GMT)
cache-control: max-age=86400
expires: current date and time+maxage (Fri, 16 Nov 2019 14:30:16 GMT)

it will not show 24 hour cache for that image in browser. only 23 hour 30 minutes (remember u have already clear browser cache it should save new resource for 24 hours not 23 hour 30 minutes).

and after 1 day it will still show stale expire header.

For all these thing to show cdn edge must have cached the resource.

Note the cache control is set to private so I have no idea why we’re even showing a Cloudflare cache status of hit, but if you want us to cache things on our edge, sending the private header tells us to do the opposite.

what do you have for CF browser cache expiration time ? which ever has longer cache TTL between CF setting and your origin’s asset’s browser cache expire TTL will take priority. So if you set 4hr expires on original asset. You’d have to set CF browser cache expiration to <4hrs or set to respect existing headers.

What does the Browser Cache Expiration do?

The Browser Cache Expiration specifies how long cached files will remain in your visitor’s browser cache. This expiration time is what Cloudflare will set unless longer time periods are specified at your web server.

A longer expiration time ensures faster load times for repeat visitors. However, a longer expiration time also means slower update times if those files are modified.

To learn more about Browser Cache Expiration see here.

Note: You can also increase the cache expiration by specifying a longer cache time on the origin server, or you can set a different Cache Expiration for a specific path or resource using the Page Rules app. Cloudflare will use whichever value is longer between your origin server cache headers and your Cloudflare setting

Additional Resources

Understanding Browser Cache Expiration

@cscharff
I have removed private from header.

The issue which I am facing (basically most of the self serve people at cloudflare). cloudflare consider expires as edge cache directive while it is browser cache directive. I have another example.

I cleared Browser cache and Edge Cache both for this experiment.

% curl -I https://www.example.com/

HTTP/2 200
date: Fri, 15 Nov 2019 23:27:53 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=dc3820da4893f6d9452bd8a027de93f791573860472; expires=Sat, 14-Nov-20 23:27:52 GMT; path=/; domain=.example.com; HttpOnly; Secure
cache-control: max-age=14400
expires: Sat, 16 Nov 2019 03:27:52 GMT
vary: Accept-Encoding
cf-cache-status: EXPIRED
strict-transport-security: max-age=31536000; includeSubDomains; preload
alt-svc: h3-23=":443"; ma=86400
expect-ct: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct
server: cloudflare
cf-ray: 5364ff923f97dcfe-SIN

As it is first time cf-cache-status: EXPIRED showing the right header and expired showing 4 hour expire time along with cache-control.

I run it second time (without any browse/edge cache clear)

% curl -I https://www.example.com/

HTTP/2 200
date: Fri, 15 Nov 2019 23:28:13 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=d53d640825c9439ff3187fb6482ea0d021573860493; expires=Sat, 14-Nov-20 23:28:13 GMT; path=/; domain=.example.com; HttpOnly; Secure
cache-control: max-age=14400
expires: Sat, 16 Nov 2019 03:27:52 GMT
vary: Accept-Encoding
cf-cache-status: HIT
strict-transport-security: max-age=31536000; includeSubDomains; preload
alt-svc: h3-23=":443"; ma=86400
expect-ct: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct
server: cloudflare
cf-ray: 536500152e48dd1e-SIN

Now cf-cache-status: HIT and the page is cache in edge. Now Date I access date: Fri, 15 Nov 2019 23:28:13 GMT and Expire expires: Sat, 16 Nov 2019 03:27:52 GMT has some second difference not exactly 4 hour (but I have not clear the cache yet so it is correct to show expire little cache coz It might be serving from browser cache).

Now I clear browser cache and wait for few more minutes.

% curl -I https://www.example.com/

HTTP/2 200
date: Fri, 15 Nov 2019 23:37:21 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=d7d53fdfbceb4fc07d15e5ab21ec69e9e1573861041; expires=Sat, 14-Nov-20 23:37:21 GMT; path=/; domain=.example.com; HttpOnly; Secure
cache-control: max-age=14400
expires: Sat, 16 Nov 2019 03:27:52 GMT
vary: Accept-Encoding
cf-cache-status: HIT
strict-transport-security: max-age=31536000; includeSubDomains; preload
alt-svc: h3-23=":443"; ma=86400
expect-ct: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct
server: cloudflare
cf-ray: 53650d73ee036fe8-SIN

Now as I have cleared the browser cache There need to be a fresh copy which should be downloaded from cloudflare edge cdn or my own server (in cloudflare edge do not have the copy). and as Cache-Control has 14400 it should be remain saved in my browser for 14400 second that is 4 hours.

Now compare date/time I accessed and Expire in last curl. Do you find it 4 hour difference?

do you have cache everything page rule on your www.example.com/* route along with edge cache ttl value set ? because Cloudflare by default does not cache dynamically generated HTML pages i.e. index pages unless you set ‘cache everything’ page rule as outlined at https://support.cloudflare.com/hc/en-us/articles/202775670-How-Do-I-Tell-Cloudflare-What-to-Cache-. This would explain your experience in terms of a cf-cache-status = HIT.

If a Page Rule has the setting " Cache Level > Cache Everything ", then resources that match the page rule will be cached. Note that this is the only way to tell us to cache resources beyond what we consider static, including HTML.

Cache Everything - Treats all content as static and caches all file types beyond the Cloudflare default cached content. Respects cache headers from the origin web server unless Edge Cache TTL is also set in the Page Rule.

Try disable the cache everything page rule if it’s enabled.

@eva200

that’s what I am saying Expires: is caching expire headers. It is a browser side header not cdn edge side header. Browser should control this header.

It should always take action based on if there is no browser cache than access time(current time)+max-age, irrespective of it get served from edge or direct server.