When I check BingWebmaster and use “Fetch as BingBot” then Bing is able to crawl that particular URL of my site. However, when I use SiteScan tool, then I am getting 4xx error. (SiteScan tool shows where we need on-page SEO improvements and that’s why it is very important that SiteScan tool works.)
Then I tried TechnicalSEO tool to fetch&render and I did the test as follows for Bingbot useragent:
and then found that CloudFlare blocks Bingbot useragent and results into HTTP 403 response
HTTP/2 403 date: Thu, 06 Aug 2020 01:46:22 GMT content-type: text/html; charset=UTF-8 set-cookie: __cfduid=d3f73e32abc0f9b1a8717b943883ee1b41596678382; expires=Sat, 05-Sep-20 01:46:22 GMT; path=/; domain=.playerauctions.com; HttpOnly; SameSite=Lax cache-control: max-age=15 expires: Thu, 06 Aug 2020 01:46:37 GMT x-frame-options: SAMEORIGIN cf-request-id: 04630bbbc4000002c02a114200000001 expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" server: cloudflare cf-ray: 5be51572dbc202c0-SEA content-encoding: gzip
I have checked posts on CloudFlare Forum on
- Bingbot not able to access to my site:
- Cloudflare bing bot block: and
- Bingbot blocked:
however none of them works.
So what should I do to fix this problem?