Google Bot was blocked

[shopsystems]

What is the name of the domain?

rhinozeros-versand.de

What is the error number?

403

What is the error message?

no access - Google was blocked

What is the issue you’re encountering

Since 02/23/2025 Googlebot got a 403 only on rhinozeros-versand.de, although other hosts like i.e. dienstbekleidung24.com with same configuration had no problems. Can you please check this?

What steps have you taken to resolve the issue?

I switched off all WAF Rules (cloudflare managed) and Block AI Scrapers and Crawlers rule. (That was the last, i turned on. Is that the issue?)

I had to deactivate cloudflare, because no adjusting of settings helped.

[sjr]

You can find the reason a request was blocked in your security event log and then adjust any settings or rules as required…
https://dash.cloudflare.com/?to=/:account/:zone/security/events

[shopsystems]

Thanks for your reply. Why does a managed rule block AS15169 GOOGLE?
Ich have a custum rule, that allows all user agents that contain google. Why does’nt that work ((http.user_agent contains “google”))?
Anyway i set the Bot Fight Mode for Definite Bots to other Domains, too. But here Google is not blocked. Why?
How can i reset it?
I turned off Bot Fight Mode and waited 10 min. No results. Thanks for your answer.

[shopsystems]

Cloudflare Bot Management rules for all plans > Manage AI bots blocked AS15169 GOOGLE. See: Ray-ID 92035f57c9ce452b

[shopsystems]

I switched off all WAF Rules, but cloudflare ist still blocking.
Ray-ID 920386a6cc687bd0
66.249.70.8
AS15169 GOOGLE
United States

[sjr]

Did you find the reason the request was blocked in the event log as I posted above?

There have been a small number of reports of this, someone has flagged it for checking.

[shopsystems]

There is no reason given. We still get blocked, alltough all WAF rules are switched off. Here is one JASON Exmaple:

{
“action”: “block”,
“clientASNDescription”: “GOOGLE”,
“clientAsn”: “15169”,
“clientCountryName”: “US”,
“clientIP”: “66.249.70.7”,
“clientRefererHost”: “www.rhinozeros-versand.de”,
“clientRequestHTTPHost”: “www.rhinozeros-versand.de”,
“clientRequestHTTPMethodName”: “GET”,
“clientRequestHTTPProtocol”: “HTTP/1.1”,
“clientRequestPath”: “/shariff-backend-php/”,
“clientRequestQuery”: “”,
“datetime”: “2025-03-14T14:42:31Z”,
“ref”: “”,
“rayName”: “920489c379e4afb3”,
“ruleId”: “57d7e4249c9e49efbc136171446ff5ee”,
“description”: “manage verified bots”,
“rulesetId”: “48ba18287c544bd7bdbe842a294f1ae2”,
“source”: “firewallManaged”,
“userAgent”: “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.6943.141 Safari/537.36 (compatible; Google-InspectionTool/1.0)”,
“matchIndex”: 0,
“metadata”: [
{
“key”: “ruleset_version”,
“value”: “8”
},
{
“key”: “version”,
“value”: “7”
},
{
“key”: “type”,
“value”: “customer”
},
{
“key”: “js_detection”,
“value”: “MISSING”
}
],
“sampleInterval”: 1
}