Bingbot blocked with Error 4xx

There are multiple threads about this issue, but none seems to reach a resolution.

My site - https://www.advanced-ict.info - used to rank highly in Bing, with some pages being used as featured snippets at the top of the search results. Now it doesn’t appear at all.

When I look in Bing Webmaster tools it tells me that the sitemap gives an error 403 and other pages fail with error 4xx. Google is unaffected.

There were no firewall events at all on days when the errors occurred (I have a free account, so perhaps I’m not seeing them all?). I added a firewall rule to allow the Bingbots described on the Microsoft website and now there are multiple firewall events to say the Bingbots have been allowed (and all of the IP addresses I’ve checked are verified Bingbot addresses), but Bing Webmaster tools is still saying that Bing is being blocked.

I contacted Bing support and their response is that it’s nothing to do with them.

Is there anything else that I can do?

I’ve tried changing the firewall to “essentially off” and turned off the browser integrity check and Bing is still getting error 403 when trying to read the sitemap.

Has no-one managed to resolve this problem?

If there’s nothing in the Firewall Log, do you see anything in the server logs?

If you Pause Cloudflare from the Overview tab (lower right corner), is Bing able to crawl your site?

Thanks for the reply.

Actually, since I added the rule to allow the Bingbot there are entries in the firewall log. It’s telling me that numerous IP addresses have been allowed using my rule, and all of the addresses I’ve checked have been verified Bingbots.

However Bing webmaster tools is still telling me that its request for the sitemap is rejected with error 403 and no other pages have been indexed. Curiously, though, the robots.txt checker claims to be fetching the latest version without complaining.

I’ll try your suggestion of pausing Cloudflare altogether to see what happens.

1 Like

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.