I am facing high host connectivity issues in google console When I look into the server it works fine but when I look into the firewall event. I found that google Bots are blocked because of rate-limiting. How can I allow Good bots from rate-limiting?
Hi there,
By default Cloudflare should not be rate-limiting GoogleBot - we have Google Bot globally allowed to not be challenged by rate-limtinig.
Just to confirm, you are using our Rate-limiting product and you are seeing seeing in our firewall events Google Bot being rate-limited?
There is a way to confirm if a Google Bot is genuine, rather than someone just simulating GoogleBot by setting the user-agent Googlebot and other Google crawler Verification | Google Search Central | Documentation | Google Developers
Do you have IP Address from the firewall events that you are seeing Google Bot be challenged?
1 Like
Yes, I have checked. Its google bot ip. Please find the ip and firewal event screenshot below
i.p - 66.249.73.254
https://www.awesomescreenshot.com/image/9772301
Thank you
I see GoogleBot being rate limited when accessing cdn-cgi/rum on my own domains. I have no rate limiting rules configured on any of these domains.
Hi there,
Unfortunately the screenshot is no longer available. I suspect this would be a good one to raise as a support ticket so can take a look. If you can raise a ticket and quote the ticket number here, I will follow up with you via the support ticket.
This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.