403 Error With Google Bots

[azariosays]

Hi,

So it looks like Cloudflare is blocking Google Bots on my website. Using https://httpstatus.io/ I can check if Google Bots can crawl my site, but it’s returning a 403 error on all pages, but specifically: https://noisypixel.net/sitemap_index.xml

Let me know if this is something that I have accidentally set on my side or what I can do to fix this. Thanks for the help!

[azariosays]

Here’s another screenshot of the issue showing that I’m using the User Agent Googlebot.

[sdayman]

Take a look at your Firewall Events Log and that should show you the security setting that’s blocking those requests.

[azariosays]

Ooooh, I see, it’s catching it as a fake Googlebot:

Do you think that this is only the case for this tool and not Googlebot proper?

[sdayman]

A Google bot that runs on AWS? That sure doesn’t look right.