Google search console error 5xx

Hello . My website is using Cloudflare dns and sometimes google search console give me error 5xx
is this issue related to cloudflare dns or not?

Do you have some Security options enabled?
What about Cloudflare Firewall?

5xx is not actually so specified, there are multiple ones. Which one do you get from the below article?:

If so, kindly check here too:

Yes firewall is enabled and I can not disable it because there are many attack from USA and other wheres

I believe you can, if you want.
May I ask you to post a screenshot of your Firewall rule(s)?

May I also ask if they are really an attacks or crawling events from Bing/Microsoft, etc.?

You can create a rule and put it on top to bypass real Google crawler using multiple criteria like:

  1. has to contain googlebot
  2. has to be from Google ASN
  3. has to access robots.txt

Or, vice versa, include it into your existing rule with action “block” like:

“Block all requests that are going to robots.txt which are not from Google ASN and do not contain Googlebot string in a user-agent” (meaning, allow only the real Google bot to access your robots.txt which contains sitemap link etc.)

  1. accessing robots.txt
  2. isn’t from Google ASN
  3. and contains Googlebot in it’s User-Agent

(http.request.uri.path contains "robots.txt" and ip.geoip.asnum ne 15169 and not http.user_agent contains "Googlebot")

Just an example or suggestion for Google crawler.

There are other ways, like using cf.client_bot.

actually I dont have any firewall rule but this is spam attacking

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/


this is robots.txt file

Looks like Cloudflare is doing what is needed to protect. Challenges the visitor/request as needed.
Click on few of them and check what do you see as a request, from where is it comming from, is it a referee.
You say “SPAM attack”, but how can you be sure?

I am not sure how this could be related to Google Search Console errors.

Good to know, but your issue is not with robots.txt obviously and as an example of what I just posted above how to block all and allow only Google to pass - if there are 5xx errors in GSC reporting.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.