403 error after pass to cloudflare

Hi,
After install cloudflare i experience a 403 from pagespeed , search console(for example impossible to fetch robots txt) and we have lose 61% of traffic.

There’s some setting i miss to setup ? I see that we have 403 error on all pages from Google.

Domain: campoelettrico.it

Please it’s important i don’t know what’s happen.

Regards from Enrico.

May I suggest below article for more information about this error:

In your robots.txt file I do not see any entry which could allow Google to fetch your sitemap.

Either define again (duplicate) User-Agent: * and Sitemap: path or define manually User-Agent: Googlebot (multiple).

Otherwise, do you see any blocked or challenged events at Firewall tab → Firewall Events from Cloudflare dashboard for your domain name?

Are you using any custom-made Firewall rules like country blocking, or maybe Bot Fight mode is enabled, etc. at Cloudflare dashboard?

You do have some syntax error, duplicate “content” attribute?:

<meta name="robots" content="index,follow" content="max-image-preview:large" />

Should be:

<meta name="robots" content="index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1">
1 Like

Hi
Bot fighting is enable. Could be this ?

Which Bot Fight? Regular, or did you configure Super Bot Fight Mode?

Regular. Anyway from the day i set cloudflare this problem happen before no problem.

Around 6k bot attack in 30 minute for that i switch on.

Regards. Enrico.

It wouldn’t surprise a lot (just a little) if regular Bot Fight Mode blocked Pagespeed Insights because it’s supposed to let Google’s crawler through. I guess Pagespeed Insights isn’t on the same approved list as the Google crawlers.

Hi,
I think it was heavy firewall of cloudflare set to high on both DDoS and firewall.

Google it’s a bot so maybe cloudflare block all bots because after change setting to medium it start to crawler again.

Anyway cloudflare was the only change before that happen, so I don’t know why but stop Google bot and give 403

Hope to have solve in this way because before everything working good.

I update till 2-3 days to tell that it’s true or not.

Enrico.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.