Cache everything and don't cache 404 at all

Hey all,

I want to use Cloudflare cache to minimize requests to S3 bucket and cache small JSON files, so I’m planning to create page rule and set it to cache everything.

My problem is starting after first 404 request, as you probably know Cloudflare will cache this request and in the next request even if the file exist in the second try Cloudflare will return 404.
Can I do something about it ?

Thanks in advanced!!

A 404 should be cached only for five minutes

Thank you @sandro!
Do you know in any chance if there is a limit on amount of files that can be cached?

In my example I have a millions of files in json in S3 is it against Cloudflare policy?

Just want to be sure, if I understood correctly:

  1. The TTL in the link you provided also applied to cache everything.
  2. The TTL for 200 response is 2 hours.

is that correct ?

I also found out that I can add to the page rule this setting:
Edge Cache TTL = a month

How can I verify this setting applied only to 200 response and 404 response is still 5 minutes or else ?

There is a default TTL and you can partially override it with a page rule, respectively also workers. As for the amount of files, I dont think Cloudflare will cache all your files, even if they are regularly accessed, but you might want to clarify this with support.

Thank you!
Have a great weekend :slight_smile:

You can send a no-cache directive from your origin and we won’t cache the 404 error… the tradeoff is that requesting a non-existent file can be a DoS vector so by forcing the server to respond to each request instead of returning a cached response it has the potential to increase server load.

2 Likes

Thank you @cs-cf!

The requests are coming from our backend and if we don’t have the json in S3 (hit 404) we’ll have the file right after so it’ll be available in the next request (getting the json from external API).

This topic was automatically closed after 30 days. New replies are no longer allowed.