I had a complaint from a customer about one of their website pages not updating. So I checked the resources, and it turns out Cloudflare was caching the page. All our websites are strictly set to “respect existing headers” and pages should never cache (which should and would be default behavior for all html pages of course).
Attached are two screenshots:
The page was cached in Cloudflare with expire set 10 years ahead for an unknown reason.
After I cleared cache manually, it works again, and you can see the correct cache headers from origin.
This is the second time this happens in a week, after second complaint from two different customers. Why is Cloudflare randomly caching pages?
Needless to say, and as you can also see in the link, this should and will not lead to pages getting cached.
I have had this setup on Cloudflare for dozens of websites for at least 10 years. Nothing changed on the origin or in Cloudflare settings, yet there is now some random glitch on Cloudflare where pages are randomly leaked into cache.
Yes, it matters. The more information you provide, the better the community can assist you.
Cache everything will cache your HTML too.
10 years? How? The Cache everything rule (CF Template) is pretty new.
And why you’re using cache everything if you don’t want everything cached? Cloudflare will cache files like images, css, js, etc by default without the need to create a cache rule.
It does not matter to the issue. At least it should not.
No, not if you have “respect existing cache headers”, and we always send no-cache for normal html. This you can clearly witness in the link I provided.
I don’t know how long you have been using Cloudflare, but there was always “page rules” > “Cache Everything”, which does the same thing. It basically tells Cloudflare to cache files that otherwise might not get cached by default, although origin headers will be respected.
Because I wanted things like mp3 and mp4 files also cached.
As you see when you access the page in the link, everything is working correctly. Cache everything, yet the respecting headers (yes, I control caching from the origin, which surely makes sense).
If the issue disappears after disabling “Cache Everything”, it would ultimately mean there is a random bug on Cloudflare that at some point it decides to ignore origin cache headers. This is not an explanation or a solution.
Serving videos without Stream? What does it mean? Where can I see anything in that page that mentions anything about this?
This is normal small mp4/mp3 files. Strange that Cloudflare allows it then, for more than 15 years. If you are trying to say “it’s not allowed to cache mp4 files on Cloudflare”, then why do they allow it then? Just disable it technically.
Apart from the above, this has absolutely nothing to do with the issue of course.
First of all, a html page is not a file type, we can’t create a rule based on extension. Even if I made a rule for example on all request that do NOT end with an extension, what’s the point? I already have cache headers forwarded by NGINX, which are and should be respected by Cloudflare.
I am posting this, because it might be helpful for the cloudflare team and others. Your suggestion is to ignore the issue and just make an overlapping rule, although it should be working as is. It’s doesn’t explain the issue, which looks like a recent Cloudflare bug and therefore should be reported.
While you could embed a video from another provider, we limited your ability to use our services to deliver video bits from our network to your visitors. This is because every second of a typical video requires as much bandwidth as loading a full web page. Over time we recognized that some of our customers wanted to stream video using our network. To accommodate them, we developed our Stream product. Stream delivers great performance at an affordable rate charged based on how much load you place on our network.
Unfortunately, while most people respect these limitations and understand they exist to ensure high quality of service for all Cloudflare customers, some users attempt to misconfigure our service to stream video in violation of our Terms of Service *
I am aware that Cloudflare doesn’t want to have massive video files streamed through their network. However, I don’t see what that has to do with Cloudflare Cache, as videos will still goes through their network. I imagine they have closed a few accounts because of inappropriate usage, but I can’t see any wording that you are not allowed to use mp4 files with Cloudflare. Even searching on Google and other posts here, it’s says it’s fine. At best, some grey zone warning. There are millions of Cloudflare websites that host a few mp4 files on their website of course.
(http.request.uri.path.extension in {“filetype1” “filetype2”}) for the extra file types you want cached instead of using cache everything. And if you’re using Page Rules, it’s best to use Cache Rules instead to have much better control.
What are discussing here? I am reporting what seems to be some kinda bug on Cloudflare. Your suggestions don’t seem to explain or solve this.
Why make things difficult though? Why would this solve the issue if the cause of the issue is still unknown? I have setup the server properly with cache headers for the files I want cached, and Cloudflare respects this, as in the Cache configuration “respect cache headers”. The server is basically setup exactly as it should work also without Cloudflare.
This does not explain the bug.
I do use Cache Rules. I was only pointing out that this rule was also formerly available through Page Rules.
Don’t see any difficulty with it. It’s even easier and fast to just use Cloudflare to configure and maintain cache rules, headers, among other things than configuring and maintaining it at origin side.
If you just want to insist with this, just contact Cloudflare customer service and they’ll simply tell you the verdict.
However, FWIW, you might misunderstand no-cache directive. Maybe no-store is more appropriate?
Is it possible that you hit the node that stored stale cache in the past?
(Noticing the CF-Ray, the requests hit nodes in different datacenters.)
Also, Cloudflare is a fairly modern proxy implementation, you might also want to refrain from sending Expires header as it may interfere with Cache-Control in some way. (When there are some bugs.)
I understand that. It’s not critical. Just seemed worth posting, since it happened twice in a couple of weeks, and I have had this exact setup for, yes I guess almost 10 years.
I’m not actually sending any cache-control header. It’s automatically added by Nginx Expires header when Expires is set to not cache. Besides, how can cache-control: no-cache be misinterpreted? The browser understands it. Either you would read the expires header or the cache-control header, both are set correctly to not cache.
Anything is possible, as the page is/was clearly cached on Cloudflare CDN for some peculiar reason. However, there doesn’t seem to have been any legitimate reason for Cloudflare to cache the response. It seems like an incorrect decision. Personally, I was wondering if it could have been caused by the request happening while the server was rebooting, perhaps combined with option “smart tiered cache”. It’s not logical, but there must have been a reason.
Then how do you control cache from headers if you can’t send Expires headers? This is how it’s done in Nginx. Expires 0 or -1 is no cache, while any positive value means cache. It will assign expires and cache-control headers based on the provided value.
I don’t want to use the default Cloudflare " Browser Cache TTL: 4 hours", because I want to control caching from server, as it should be done when setting up a server without using Cloudflare. Therefore, it’s set to “Respect existing headers”, and then you would expect Cloudflare to respect the headers unconditionally.
Anyway, this is a minor issue at the moment, and I don’t expect a resolve. It seemed worth posting here.
You can increase that time. Or use cache rules to configure cache TTL by file types or even uri.
I’ve given you several alternatives to get around this problem. But okay.
I would still disable any cache headers or expires coming from the origin server and just stick with cache rules. And if I really wanted to continue using the origin server, I wouldn’t use Expire, and would opt for CDN-Cache-Control or Cache-Control. But do as you want.
I don’t know if what you’re experiencing is actually a bug or is intentional, a misconfiguration or if it worked before, now CF has changed the way it works (which sometimes happens). If you really want to clear up your doubts, contact CF support.
Starting from understanding the headers, some of the questions may clear up.
If there is a Cache-Control header with the max-age or s-maxage directive in the response, the Expires header is ignored.
In theory, if directives are conflicted, the most restrictive directive should be honored. So the example below is basically meaningless because private , no-cache , max-age=0 and must-revalidate conflict with no-store .
Read together as: if no-store directive is there, Expires will be ignored.
But no-store, max-age=0 can be a viable option too, just in case.
Basically, Expires is an old way of doing things.
For Nginx, add_header directive can be used to set header of static files.
I am just aware that this is a common way to control cache in Nginx. If a lot of people use it, there shouldn’t be a bug. But maybe, worth trying other headers/approaches anyway.
Are you still here? Your post isn’t even remotely close to explaining the issue.
This default is in place for lazy server admins who “forget” to set expires headers correctly for their files. I am not one of them. I don’t need to increase that time, because files from our server already have the correct expires time. For your info, when using the default “4 hours”, that would only apply anyway if the origin did NOT include expires headers or the expires headers were lower than 4 hours. If the expires headers are longer than 4 hours (most static assets), it won’t even apply.
Why? Caching is setup properly. Why use Cloudflare to override valid cache settings from origin? There are many reasons to assign them from server, not only because it should be setup as you want it to work WITHOUT Cloudflare, but also because you have some app-specific cache headers. Besides, this has nothing to do with the reported incident.
If you don’t know, then why are you replying? No it hasn’t changed the way … It doesn’t seem like you care to even read the original post? It is working, exactly as it should, except there has been a glitch, and I was trying to find out why.
Yes I have contacted them also thank you.
Listen, I have been working with Cloudflare more than 15 years, and in that time, I have also uncovered a few small bugs. It’s not unheard of. If you have no idea about why the issue in this post was happening, then why waste time on loads of unrelated replies?
There is neither max-age or s-maxage in the cache-control header, so not sure why you are bringing this up.
What conflicting directives are there in the response headers?
Exactly what headers are you suggesting to use for a html page?
Regardless of how you twist and turn this, there is no logical explanation why Cloudflare decided to randomly cache a page response, and then it doesn’t. There is nothing in the headers that implies Cloudflare could/should cache the page … and if there was, then why just that one time?
What you are suggesting, is to remove Expires headers, because this may have randomly been causing Cloudflare to cache the page, even if the Expires header says the document is expired.
Then how do you control cache from headers if you can’t send Expires headers?
The point of my previous post is about the independent of both headers. The question seems to imply they are interpreted together.
I could not validate the header send from your origin that cause that particular cache handling. But the expires header set at the end of 2037 tell this might be related to Y2K38 problem. Maybe some value that cause Nginx to wrapped over? (In any case, only you and Cloudflare can work together to identify and reproduce the bug.)
My projects and your projects would have different requirements. But in all my projects, I set cache-control to either public or private with proper max-age and may implement the cache purging API in some situations.
And I really like when they are caching my HTML pages. They seem to respect cache-control headers.
