Urgent: Cloudflare not caching despite cache-control header

This is urgent: my site at https://busy-beavers.tigyog.app/ has unexpected traffic, and the origin servers are running red-hot because Cloudflare has decided to not cache most responses!

Here’s an example:

$ curl --head https://everyday-data-science.tigyog.app/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0HTTP/2 200
date: Fri, 25 Nov 2022 18:18:17 GMT
content-type: text/html; charset=utf-8
content-security-policy: default-src 'self' https://tigyog.app; connect-src 'self' https://tigyog.app https://*.google-analytics.com https://accounts.google.com; script-src 'self' https://tigyog.app http://www.googletagmanager.com https://www.googletagmanager.com https://accounts.google.com https://js.stripe.com/ 'unsafe-eval' 'unsafe-inline'; style-src-elem 'self' https://tigyog.app https://accounts.google.com 'unsafe-inline'; style-src 'self' https://tigyog.app https://accounts.google.com/ https://lh3.googleusercontent.com/ 'unsafe-inline'; font-src 'self' https://tigyog.app; frame-src 'self' https://tigyog.app * data:; img-src 'self' https://tigyog.app data:; object-src 'self' https://tigyog.app data:
cache-control: public, max-age=30, s-maxage=60, stale-while-revalidate=3600
vary: Accept-Encoding
x-do-app-origin: 1e494334-e6b1-11ec-b1dc-0c42a19a82a7
x-do-orig-status: 200
cf-cache-status: DYNAMIC
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=FG3k3rTeGePI%2BkWcjQpY8cVFaaGbKoM1OXLjoRkavOPmNaDzxknC7CdmT%2FuvOgDTnKr6y0cNSn88USZtLdgIirurBfihvhSb3wilVzzvCqZT9L4fEfmVMgeiDiTENGVzgj3Zt1BDbZGrxq0454%2FsvRpTfj5s84mJgB1Z7M89"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 76fc62ce4f2a406c-LHR

Note that this clearly instructs Cloudflare to cache this page for 1 minute:

cache-control: public, max-age=30, s-maxage=60, stale-while-revalidate=3600

Nevertheless, I get cf-cache-status: DYNAMIC indicating that Cloudflare is not caching this URL.

What gives? And how do I debug this?

For the pages at https://busy-beavers.tigyog.app/ under high traffic, I have added an explicit “cache rule” that marks those pages as “Eligible for cache”, after which Cloudflare seems to behave as expected. But this is not an acceptable general solution - Cloudflare needs to respect the cache-control header!

How do I tell Cloudflare to always respect the cache-control header? And why isn’t it doing this by default?

I have opened a ticket with Cloudflare Support over an hour ago, but have received no response, so I’m reaching out to the community - can you help me with this urgent issue?

HTML isn’t eligible for caching by default.

Hence this rule is required:

1 Like

Whoa, what?! Okay, is there a way I can just say “all content is eligible for cache”, and just respect the cache-control header?

Thank you for your help - this is mad!

Have you tried Page Rules?

Hey Stephan - thanks for your response. How would I use a page rule to achieve this? I had a look at this, but all I can see is a “cache level” rule that lets me choose whether to ignore query strings - I don’t see anything that lets me mark resources as “eligible for cache”.

You could, for example, use those rules to explicitly cache HTML files as well.

But if you scroll down the link I posted, there is even a section that literally says.
Cache Everything

Hey Stephan, thanks, that’s very useful - much more useful than paid Cloudflare support!

I was pretty nervous about setting “Cache Everything”, because I don’t actually want to cache everything! But it seems like it doesn’t do what the name implies - it seems like it actually just means “Cache each page according to its cache-control header, like a CDN normally should”. Is that right?

So I’ve set a page rule with the matcher *tigyog.app/*, which sets the “Cache Level” to “Cache Everything”. This appears to be working. Thank you!

I have one more problem. My app is a SaaS platform, where I plan to allow customers to have their own domains, e.g. customer1.com CNAME customer1.tigyog.app. So I think I’ll need this page rule to work for customer1.com/* and any other dynamic custom domain.

So I just want to match literally every URL. I tried setting the matcher to just *, but it complains that “Your URL should reference the domain ‘tigyog.app’ in some way.” Do you know if there is a way to bypass that?

Thanks again!

Hey, glad it helped.

You can :white_check_mark: the solution button if it worked. It will help later readers to quickly identify how to solve similar issues.

Regarding SaaS I don’t have any experience. But they do have a separate docs area about that very topic.

Okay, thanks Stephan! I’ll contact support and see if they can help regarding the SaaS issue.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.