So it looks like Cloudflare is blocking Google Bots on my website. Using https://httpstatus.io/ I can check if Google Bots can crawl my site, but it’s returning a 403 error on all pages, but specifically: https://noisypixel.net/sitemap_index.xml
Let me know if this is something that I have accidentally set on my side or what I can do to fix this. Thanks for the help!
Here’s another screenshot of the issue showing that I’m using the User Agent Googlebot.
Take a look at your Firewall Events Log and that should show you the security setting that’s blocking those requests.
Ooooh, I see, it’s catching it as a fake Googlebot:
Do you think that this is only the case for this tool and not Googlebot proper?
A Google bot that runs on AWS? That sure doesn’t look right.
This topic was automatically closed after 30 days. New replies are no longer allowed.