Hi , i have a little problem - google bot get 403 .Im using google testing tools and screaming SEO frog and when im trying parse i get 403 error (captcha). When i turn off CF i get 200 and all good. i know that forum already have same topics , but their solutions didnt help me. i already add allow
user-agent
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
and
Known bots
cf.client.bot.
Maybe someone can help me . thanks
Hi,
If Googlebot is being challenged or blocked by a Cloudflare tool, you should be able to find more details by visiting WAF / Firewall Events. Search for events that match Googlebot’s IP addresses or UA, and see which Cloudflare service (which Firewall or WAF rule, or whether it is Bot Fight Mode, etc) is doing the blocking. From there, you’ll be able to adjust the settings of the specific service to avoid this issue.
Thanks for your reply , but i have nothing that can block it in waf or bot fight mode
If your Cloudflare Firewall Events do not show any blocks, that means Googlebot is getting a 403 from your server. Check your origin server logs and, if needed, contact your hosting provider’s support team.
thanks