Solved: Origin cache-control and expires headers have no effect (Cloudflare is incompatible with Imunify360, solution: disable WebShield)

I have total complete lost my mind over how to get Cloudflare to cache a simple css file. I have been working with htaccess to set expiration headers, but Cloudflare does not seem to respect them.

I have a site where the origin returns the following headers for a CSS file:

Cache-Control: public, max-age=2592000,s-maxage=14400
Expires: Sun, 26 May 2019 07:29:49 GMT
Content-Type: text/css
Content-Length: 3195
Date: Fri, 26 Apr 2019 07:29:49 GMT
Server: LiteSpeed
Access-Control-Allow-Origin: *
Alt-Svc: quic=":443"; ma=2592000; v="35,39,43,44"
Connection: Keep-Alive

With Caching Level set to standard and Browser Cache Expiration set to respect existing headers I would expect the following:
The file would be edge cached for 14400 seconds, and browser cached for 1 month.

However the Cloudflare response I get is:

Date: Fri, 26 Apr 2019 07:30:25 GMT
Content-Type: text/css
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-cache
Expires: Fri, 26 Apr 2019 07:30:23 GMT
Vary: Accept-Encoding
Access-Control-Allow-Origin: *
X-Turbo-Charged-By: LiteSpeed
CF-Cache-Status: MISS
Expect-CT: max-age=604800, report-uri="https://report-uri.Cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: Cloudflare
CF-RAY: 4cd6d9e7fd17bdac-AMS

If I turn on a page rule with Origin Cache Control: On nothing changes from the above. However if I make the page rule explicitly with an Edge Cache TTL (any value) I get the following response:

Date: Fri, 26 Apr 2019 07:39:31 GMT
Content-Type: text/css
Content-Length: 3183
Connection: keep-alive
Access-Control-Allow-Origin: *
Cache-Control: no-cache
Cf-Bgj: minify
Cf-Polished: origSize=3195
Expires: Fri, 26 Apr 2019 07:39:26 GMT
X-Turbo-Charged-By: LiteSpeed
CF-Cache-Status: HIT
Accept-Ranges: bytes
Expect-CT: max-age=604800, report-uri="https://report-uri.Cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: Cloudflare
CF-RAY: 4cd6e7426c38c835-AMS

Again by adding a Browser Cache TTL (any value) I can get a cache-control header (the rest is the same as above even the expires header that is in the past)

Date: Fri, 26 Apr 2019 07:41:14 GMT
Cache-Control: private, max-age=345600
Expires: Fri, 26 Apr 2019 07:39:26 GMT
CF-Cache-Status: HIT

This is on the free plan, but for simple css files that should not make a difference, right?

In your first two header samples, the Date and Expires match. This looks like CF is doing it’s job.

When you have CF-Cache-Status: MISS it makes sense that CF will do what it can to serve a fresh and expiring resource.

The header responses can be dependent on the request headers. For example you’re forcing your browser to request a fresh copy.
Can you post the url(s) for us to test against? Or at least post the request+response pairs?

The first header response is the origin response.

I am unsure what you mean with Date and Expire headers match? In the origin response the expire is a month from date, and in the Cloudflare response the expire is 2 seconds ago.

The file I am testing with is https://content.zeal.global/zeal.css (please do not pay attention to the content, that is a test.)
(online test: https://apitester.com/shared/checks/6bead0fc948e48b8a56b0383b68a1b53)

The origin response can be found through http://origin.zeal.global/zeal.css
(online test: https://apitester.com/shared/checks/90be18fd4f034c229a028fd24132ab7c)

I have done tests with different browsers and ARC as well as a few different online test servers. All with the same responses.

The current setup is: No pages rules enabled, and all caches have been flushed 30 minutes ago.

The only thing I can think of off hand is that CF is seeing the Pragma request header and so bypassing the cache and changing the expire time.

Trying with firefox where I can disable the function : “Disable cache”, I get the same response with no Pragma: no-cache request header. I also double checked with ARC that something like that was sent. Nothing. On next requests firefox will automatically add a new Cache-control request header with max-age: 0 because the server responded with the “no-cache” header on the first request.

Based on the idea that the client might be sending some random stuff to invalidate the cache I just did raw request testing with fiddler. There is no headers that should invalidate the cache. No matter what I do with standard settings nothing is cached by Cloudflare edge servers, unless I make a page rule to specifically cache everything.

Any idea what “Server”:“imunify360-webshield/1.6” is or why it might be returned in a request from the origin? I could be doing it wrong but when testing with a Cloudflare internal tool I am seeing this from the origin:

“Connection”:“close”,“Date”:“Fri, 26 Apr 2019 15:04:50 GMT”,“Content-Type”:“text/css”,“Cache-Control”:“no-cache”,“Access-Control-Allow-Origin”:"*",“Expires”:“Fri, 26 Apr 2019 15:04:49 GMT”,“X-Turbo-Charged-By”:“LiteSpeed”,“Vary”:“Accept-Encoding”,“Content-Encoding”:“gzip”,“Content-Length”:“997”,“Server”:“imunify360-webshield/1.6”

Firefox here as well. Enabling cache and hitting enter in the url bar - rather than refresh or hard reload (shift-refresh) - I get a cache miss with CF.

image

I’m glad @cscharff chimed in because this does look a little funny. However I chose Firefox because it’s the least cache-insane browser, I know it will get fresh content when I want it. When I get more opportunity I’ll check in a WebKit browser.

https://www.imunify360.com/ is a WAF/Virus Scanner/Security suite

Just wanted to be thorough… Chrome has similar results.

I think it’s just a symptom of how different servers react to browsers asking for a non-cached file. Don’t quote me yet though, I think we’ll need an official answer on this one.

I have never heard of it before, but checking in the cpanel associated with the domain there is a plugin called imunify360 but it seems like it is scanning for malware. There are no settings to touch or configure. Can you share the request that goes out with that response? Because the end “Expires” and “Cache-control” looks like the Cloudflare result that I cannot understand where comes from.

I believe these were the headers sent with the request (not a tool I use everyday so… chance I could be wrong for sure):
“CF-RAY”:“4cd917502d698c5c-SFO-DOG”,“Connection”:“Keep-Alive”,“accept”:"/",“CF-IPCountry”:“US”,“Accept-Encoding”:“gzip”,“CF-Connecting-IP”:“2606:4700:2001:10b:0:0:0:4”,“Host”:“content.zeal.global”,“CDN-Loop”:“Cloudflare”,“X-Forwarded-Proto”:“https”,“X-Forwarded-For”:“2606:4700:2001:10b:0:0:0:4”,“CF-Visitor”:"{“scheme”:“https”}",“user-agent”:“curl/7.62.0”

Apparently imunify360 has the ability to whitelist Cloudflare IPs (not that i know how)… might try that?

I cannot reproduce this in any way.

I have just written my host, lets see what they reply. Thank you for checking what you receive internally.

They replied:

We do and have had this setting enabled since installing Imunify360 on the server hosting your site. We did this a couple weeks ago. We can try turning the setting off to see if that helps; I know on a prior version we are using on a different server the setting doesn’t work with Cloudflare at all in some cases but we seemed to have no reports of issues on the newer version until now so it may still be buggy I believe.

My host has disabled the WebShield feature of Imunify360, and now it responds with the correct headers.

I have however noticed an interesting issue. Now I receive the correct headers:

Date: Fri, 26 Apr 2019 18:28:03 GMT
Content-Type: text/css
Content-Length: 3183
Connection: keep-alive
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=2592000, s-maxage=14400
Cf-Bgj: minify
Cf-Polished: origSize=3195
Expires: Sun, 26 May 2019 18:24:30 GMT
Vary: Accept-Encoding
X-Turbo-Charged-By: LiteSpeed
CF-Cache-Status: HIT
Accept-Ranges: bytes
Expect-CT: max-age=604800, report-uri="https://report-uri.Cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: Cloudflare
CF-RAY: 4cda9d40bbfe76d0-LAX

But the response time did not change. When I had CF-Cache-Status: MISS I have response times between 800-900 ms, now they are the same.

When I added Edge Cache TTL before I got response times close to 100ms. So there seems to be no effect of the cache: hit?

Never mind on the speed, I had debugging setup because of the earlier issues on my personal network. Just disabled that and speed is up!

This topic was automatically closed after 30 days. New replies are no longer allowed.