Block robot identified by bot\*

It’s Google ASN.

I use “does not equal” due to action method “block” all comming to robots.txt except Googlebot user-agent and from Google ASN. That way, only the real Google bot can access/read my robots.txt. Anyone else trying to load robots.txt will see Cloudflare error 1020 Access denied.

  • because I saw some bots using Googlebot in their User-agent name, but they are not comming from real Google IP address (which is a part of the AS number), rather some “fake Googlebot”. But, that does not mean someone from Google Cloud Platform with Googlebot is being blocked too (I am guessing they would also be allowed due to the IP address being a part of Google ASN and having Googlebot in their User-agent).