Requests from Pages back to itself through the CDN zone doesn't respect caching

I’m using Pages with a Business plan deploying a Remix app.

I have a slow 3rd party API that I’m fetching from my Pages function (it takes anywhere from 7-10 seconds to respond with results, but the data only changes daily or weekly at most). I was thinking I could use Cloudflare as poor-persons redis alternative and use cache control headers to reserve this request from my pages function.

I’ve enabled Caching with Cache Everything turned on for my domain and created a route /program.json that serves the results of this slow API call and it caches correctly if I request it directly from the browser or curl.

However, if I request this route from my Pages app within my function/worker it appears to bypass the cache and hit the Pages origin server directly.

I’ve tried setting { cf: {cacheTtl: 120, cacheEverything: true}} on the fetch to my route, but it doesn’t seem to do anything.

Everything I’ve tried seems to indicate that any fetch from my Pages app to my domain bypasses the cache and hits the origin everytime.

My next step is to just do some data caching using KV or Redis, but I was hoping this might work. Any ideas?

I am having the same issue as well.

Though through testing, I find out that, it may be dependent on if the third party API is also behind Cloudflare?

See this test app: https://test-remix-8fw.pages.dev/
Both are fetching from a dynamic API. The beer one does not sit behind Cloudflare, while the dog one does. As you can see, the beer one stays constant, while the dog one updates for each refresh. The edge TTL is set to 60 so the beer one should update each minute.

The random beer API is: https://random-data-api.com/api/beer/random_beer
The random dog API is: https://dog.ceo/api/breeds/image/random

Can you help confirm if your third party API is also behind Cloudflare? You can know that by accessing such as example.com/cdn-cgi/trace. Thanks.

Reading further, it seems to be another cause.

According to: Customize cache · Cloudflare Cache (CDN) docs

Resources that match a Cache Everything Page Rule are still not cached if the origin web server sends a Cache-Control header of max-age=0, private, no-cache, or an Expires header with an already expired date.

Probably that is why? Although this page says the behaviour can be overridden, but I am not sure if the cacheTtl is equivalent to the Edge TTL setting.

I can confirm that, cache API works to override origin cache controls, while fetch cache does not.

See latest example at: https://test-remix-8fw.pages.dev/