FW rule blocking Microsoft ASN without blocking Bing

Hi I want to block Microsoft and Google without blocking Bing or GoogleBot.

I did the the following:

However its not working with Microsoft:

as

What am I doing wrong?
Thanks

You might want to switch the toggle button from green to gray.
Therefore, make sure the method is “block”.

However, I might not be sure how Cloudflare checks this, by user-agent or IP range :thinking:

So, maybe like writing your own Firewall Rule … for example:

(http.request.uri.path contains "sitemap" and ip.geoip.asnum ne 15169 and not http.user_agent contains "Googlebot") or (http.request.uri.path contains "sitemap" and ip.geoip.asnum ne 8075 and not http.user_agent contains "Bingbot") or (http.request.uri.path contains "robots.txt" and ip.geoip.asnum ne 15169 and not http.user_agent contains "Googlebot") or (http.request.uri.path contains "robots.txt" and ip.geoip.asnum ne 8075 and not http.user_agent contains "Bingbot")

With the method “BLOCK”.

Using above, would be you’re allowing regular and valid Googlebot and Bingbot to visit your robots.txt file and crawl index your sitemap file(s) if any existing, while blocking even yourself out of trying to access them.

Haven’t tried it yet, but just thinking :thinking:

You’d

If you want to block Msoft I’d also block AS8068 through AS8075
We block all Msoft traffic as 99.99% of it is attack attempts and Msoft ASN’s are responsible for about 80% of attacks on our sites
40.122.76.165 resolves to ns2-201.azure-dns.net not a name ending in search.msn.com so isn’t a bingbot - See Bing Webmaster Tools & Network Tools: DNS,IP,Email

2 Likes

You might want to switch the toggle button from green to gray.

That’s the mistake! Thanks, that was really useful!

1 Like

If you want to block Msoft I’d also block AS8068 through AS8075

Thanks!

40.122.76.165 resolves to ns2-201.azure-dns.net not a name ending in search.msn.com so isn’t a bingbot - See Bing Webmaster Tools & Network Tools: DNS,IP,Email

I know, that’s why I tried to block it.
I didn’t knew that the known bots button has to be gray for this rule, as fritex pointed

However I still see IPs that belongs to blocked ASN coming to the site and causing 404 errors.

LOG

IP
AS24940

FW rule

Which rule comes first? Your “block ASN’s” rule needs to be above the “Allow BingBot & Google Rule”

I find it easier to just add the ASN’s to the Security/WAF, Tools form - also gives you the advantage that you can ban the usual suspects from every website in your account in one step, particularly as over time you will probably add a lot of ASN’s to be blocked, and you save having a huge rule with lots of ASN’s listed

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.