Cloudflare CDN does not cache for the maximum Edge Cache TTL duration

[postmaster2]

Hi,

the situation is pretty basic - Edge cache for 7 days is not being enforced by the PageRules
Below are my rules

I have a custom script that load the sitemap.xml from the domain, and fetch URLs that are to be cached by the rules

Below is the code that handles the requests (via cURL)

$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HEADER => 1,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 999,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
));
$response = curl_exec($curl);

When $url is the URL to fetch.

The script handles the output and creates a CSV file with the output of CF-Cache-Status header.
Below is one of the lines

Date, Hour, URL, StatusCode, CacheStatus
2022-12-19,10:20:03,https://*****/pt/shop/****.html,200,MISS

What happens is this script executes every night (since 14 December) and every day i get about 60-65% of misses from a global lists of around 3300 links.

This makes no sense, since at least from 15-16 January all links should be cached and we should always get a HIT status

Regards

[domjh]

I might be misunderstanding your issue here but I think this might be the correct behaviour. Cloudflare is not a push CDN, setting a TTL of 7 days does not guarantee that all datacenters will store a copy for 7 days, just that any copies will not be served after 7 days.

[postmaster2]

Hi,

TTL for 7 days, should mean that after the access (from the custom script), the page (html) should be store on edge cdn for 7 days.
At least its my understanding - since the custom script fetch the URL, the contents (html) should stay on the edge for 7 days.

Putting this differently:
Imagine i clear all the CDN cache, then manage to load all the URLs, they should be placed on Edge for 7 days - which is not happening

Regards

[michael]

@postmaster2

TTL for 7 days, should mean that

Cache-control max-age defines the time for which a response should be considered fresh. It does not mean that any proxy cache must cache for the defined value.

RFC ft-ietf-httpbis-cache: HTTP Caching

Like all proxy caches, Cloudflare will evict objects according to their own algorithm. Accounts on lower tiers will be more likely to have objects evicted compared to Enterprise plans. Objects that are accessed less frequently are also more likely to be evicted.

You can use Cache Reserve if you require essentially guaranteed cache lifetime. This is a paid add-on on all plans.