I have a json endpoint on a specific path, that i need to keep cached for as long as i can, cause it’s very database intensive. It generates thousands of URLs as requested by the users, and then each should be cached for 14 days.
But then, everyday at 00h(UTC), Cloudflare is cleaning the cache ONLY for this path, for no reason:
In the graph you can see that, when i was using APO, it wasn’t holding the cache for this path. I was using the same page rule, and removing all cookie headers.
With APO: requests were hitting Origin all day, and Cloudflare only from 00h to 4am.
With Page Rule: Cache is served from Cloudflare all day, and from origin from 00h till it’s full again.
I had to create a new database only for this endpoint. Precisely at 00h, the load spikes at 40 (for a 2 vCPUs managed DB).
I’m 100% sure that there’s nothing in our end purging this cache. We don’t have any code in place to clear this specific path. As i said, this path generates a few thousand URLs. There’s no way to clear one by one manually, and i’m not purging the entire site:
There are no traffic spikes, or attacks. This json is not a public path, it’s used only to generate some widgets in my theme:
Update Jul/15: In this specific case, it wasn’t a bug in cache. Our script had a date string that changed everyday. It should be static, but an old dev made it like this and didn’t told us. But at the end, this topic led to other interesting points about caching