Urgent: Cloudflare not caching despite cache-control header

Website, Application, Performance Security
#1

This is urgent: my site at https://busy-beavers.tigyog.app/ has unexpected traffic, and the origin servers are running red-hot because Cloudflare has decided to not cache most responses!

Here’s an example:

$ curl --head https://everyday-data-science.tigyog.app/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0HTTP/2 200
date: Fri, 25 Nov 2022 18:18:17 GMT
content-type: text/html; charset=utf-8
content-security-policy: default-src 'self' https://tigyog.app; connect-src 'self' https://tigyog.app https://*.google-analytics.com https://accounts.google.com; script-src 'self' https://tigyog.app http://www.googletagmanager.com https://www.googletagmanager.com https://accounts.google.com https://js.stripe.com/ 'unsafe-eval' 'unsafe-inline'; style-src-elem 'self' https://tigyog.app https://accounts.google.com 'unsafe-inline'; style-src 'self' https://tigyog.app https://accounts.google.com/ https://lh3.googleusercontent.com/ 'unsafe-inline'; font-src 'self' https://tigyog.app; frame-src 'self' https://tigyog.app * data:; img-src 'self' https://tigyog.app data:; object-src 'self' https://tigyog.app data:
cache-control: public, max-age=30, s-maxage=60, stale-while-revalidate=3600
vary: Accept-Encoding
x-do-app-origin: 1e494334-e6b1-11ec-b1dc-0c42a19a82a7
x-do-orig-status: 200
cf-cache-status: DYNAMIC
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=FG3k3rTeGePI%2BkWcjQpY8cVFaaGbKoM1OXLjoRkavOPmNaDzxknC7CdmT%2FuvOgDTnKr6y0cNSn88USZtLdgIirurBfihvhSb3wilVzzvCqZT9L4fEfmVMgeiDiTENGVzgj3Zt1BDbZGrxq0454%2FsvRpTfj5s84mJgB1Z7M89"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 76fc62ce4f2a406c-LHR

Note that this clearly instructs Cloudflare to cache this page for 1 minute:

cache-control: public, max-age=30, s-maxage=60, stale-while-revalidate=3600

Nevertheless, I get cf-cache-status: DYNAMIC indicating that Cloudflare is not caching this URL.

What gives? And how do I debug this?

For the pages at https://busy-beavers.tigyog.app/ under high traffic, I have added an explicit “cache rule” that marks those pages as “Eligible for cache”, after which Cloudflare seems to behave as expected. But this is not an acceptable general solution - Cloudflare needs to respect the cache-control header!

How do I tell Cloudflare to always respect the cache-control header? And why isn’t it doing this by default?

#2

I have opened a ticket with Cloudflare Support over an hour ago, but have received no response, so I’m reaching out to the community - can you help me with this urgent issue?

#3

HTML isn’t eligible for caching by default.

Hence this rule is required:

1 Like
#4

Whoa, what?! Okay, is there a way I can just say “all content is eligible for cache”, and just respect the cache-control header?

Thank you for your help - this is mad!

#5

Have you tried Page Rules?

#6

Hey Stephan - thanks for your response. How would I use a page rule to achieve this? I had a look at this, but all I can see is a “cache level” rule that lets me choose whether to ignore query strings - I don’t see anything that lets me mark resources as “eligible for cache”.

#7

You could, for example, use those rules to explicitly cache HTML files as well.

But if you scroll down the link I posted, there is even a section that literally says.
“Cache Everything”

#8

Hey Stephan, thanks, that’s very useful - much more useful than paid Cloudflare support!

I was pretty nervous about setting “Cache Everything”, because I don’t actually want to cache everything! But it seems like it doesn’t do what the name implies - it seems like it actually just means “Cache each page according to its cache-control header, like a CDN normally should”. Is that right?

So I’ve set a page rule with the matcher *tigyog.app/*, which sets the “Cache Level” to “Cache Everything”. This appears to be working. Thank you!

I have one more problem. My app is a SaaS platform, where I plan to allow customers to have their own domains, e.g. customer1.com CNAME customer1.tigyog.app. So I think I’ll need this page rule to work for customer1.com/* and any other dynamic custom domain.

So I just want to match literally every URL. I tried setting the matcher to just *, but it complains that “Your URL should reference the domain ‘tigyog.app’ in some way.” Do you know if there is a way to bypass that?

Thanks again!

#9

Hey, glad it helped.

You can :white_check_mark: the solution button if it worked. It will help later readers to quickly identify how to solve similar issues.

Regarding SaaS I don’t have any experience. But they do have a separate docs area about that very topic.

#10

Okay, thanks Stephan! I’ll contact support and see if they can help regarding the SaaS issue.

#11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.