Hello all,
From the last few days I have seen on Pagespeed that Lighthouse is not able to fetch my robots.txt, please see the screenshot. I tried troubleshooting but it is still showing the same. My robots.txt is fine as well. Here are the robots.txt details: (https)://www.(androidcrunch).com/robots.txt
User-Agent: *
Allow: /wp-content/uploads/
Disallow: /cdn-cgi/
Disallow: /wp-admin/
Disallow: /readme.html
Disallow: /refer/
I’m experiencing the same issue. If you find a solution, could you please inform me
Me too. I had the tech guy at my hosting company stumped about this. He feels “it’s a Google thing”.
After searching for this issue online, I understood that this is a Cloudflare or Google issue. That is why posted here, finally. I would appreciate it if a Cloudflare personnel confirms this.
I can see that the issue is showing for every other website and not for a particular website. Looks like this issue is specific to Google only. I got a reply here from Google:
This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.
