The Google crawl rate have decreased and the server response time is very unsteady

Problem is that first time visitors from a particular location are getting slower page speed times with Cloudflare (mainly for Google Bots). That’s because there’s an additional step of the EDGE server requesting the URL from the origin – instead of the visitor requesting it directly.
subsequent visits will be faster. But with so many servers, that’s a LOT of visitors who are getting very low page speed times.

We want to cache the complete site on the edge so that visitor shouldn’t see it miss from any location, we have done 2 things for that:

  1. Cache Everything Page rule for complete website and Edge Cache TTL to 1 month
  2. WordPress APO plugin enabled which suppose to cache the wordpress site to increase the response time (https://developers.cloudflare.com/automatic-platform-optimization/)

Cache-control header has max-age=31536000

But still after a while or pause we see cache MISS for first time and then from third request it start with HIT. Though by above 2 things it should have cache the website in advance and should not forwarding to origin. The main problem remains with google bots which shows less crawling requests and high response time since cloudflare enabled.

Try enabling Argo Tiered Caching on your zone. By default, the CF data center that receives the request from the visitor will forward the request to the origin once the data center can’t find any cache of the requested resource. Argo Tiered Caching might be able to help you since the lower tier data center will query the upper tier data center for cache before requesting from the origin.

Thanks, I have enabled that as well. Now need to review it for few days

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.