403 Error With Google Bots


So it looks like Cloudflare is blocking Google Bots on my website. Using https://httpstatus.io/ I can check if Google Bots can crawl my site, but it’s returning a 403 error on all pages, but specifically: https://noisypixel.net/sitemap_index.xml

Let me know if this is something that I have accidentally set on my side or what I can do to fix this. Thanks for the help!

Here’s another screenshot of the issue showing that I’m using the User Agent Googlebot.

Take a look at your Firewall Events Log and that should show you the security setting that’s blocking those requests.

Ooooh, I see, it’s catching it as a fake Googlebot:

Do you think that this is only the case for this tool and not Googlebot proper?

A Google bot that runs on AWS? That sure doesn’t look right.

This topic was automatically closed after 30 days. New replies are no longer allowed.