I have initiated a scan with bingbot through bing webmaster tools, i then monitor this in Cloudflare and i see the below bot trying to crawl my site.
IP address: 13.66.205.190
ASN: AS8075 MICROSOFT-CORP-MSN-AS-BLOCK
User agent: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
The bot is being blocked by managed rules in Cloudflare WAF & bing webmaster tools is returning the below error.
These pages have 400-499 http status code. Ensuring http status 200 might help improve SERP performance and gain user traffic.
When i try to verify the bot using the bing verify tool it tells me that this is not a bingbot but i am 100% certain it is a bingbot ip as i get the same IP every time i try to crawl my site with bingbot. When i disable the Cloudflare WAF managed rulesets bing can crawl the site fine.
How do i get around this? Any help or direction would be most appreciated.