Bing Bot Incorrectly Blocked

I noticed that the Bing index of my website is decreasing. Running the Bing Webmaster tool gives an HTTP 400-499 ERROR. The event log shows the rule message Anomaly:Header:User-Agent - Fake Bing or MSN Bot


The block is correct - thats not a verified bingbot

You can use the Bing Webmaster Tools to verify bingbots - and that IP fails the check

3 Likes

Oh dear, the tool shows that it is indeed not a Bing bot, but why does it use this IP to access when I run the Bing Webmaster Tools check?

Because its not Bing using that IP - its just someone using the data centre in Des Moines, Iowa to attack websites - trying to read your robots.txt to get info on the the folder structure of your website

Anyone can set their User Agent to fake being a bingbot or googlebot or whatever and thats why you cant use it as a reliable check - hence Cloudflare saying that its a fake bot as the header and user agent dont match

But almost immediately after I submitted the site scan request, I checked the Cloudflare logs. The blocked page and the scan URL I submitted were consistent, and Bing Webmaster Tools also returned a page containing 400-499 errors. Will this affect my Bing index? Can I temporarily disable the block to allow Bing Webmaster Tools to complete the website scan?

It happens every now and then that Microsoft do not correctly identify the IPs used by Bingbot, you will have to take that up with their support.

Until then, you can deactivate the managed rule that blocks fake Bing bots.

1 Like

Bing do state that they dont use normal bingbot IPs for the site scan checks - they are pretty stupid for deliberately using a fake bingbot to do a bingbot check…

3 Likes

I’m afraid I can’t disable this rule because I’m on the Free plan.

It’s quite foolish to use a non-Bing BOT IP for website index checks, as it can’t fully simulate the spider visit situation. I’m afraid I can’t disable this rule because I’m on the Free plan.

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.