Error with Robots txt - <html><meta http-equiv=”refresh” content=”0;/.well-known/captcha/?r=%2Frobots.txt”></meta></head></html>

General
1

Hey Guys
My Robots text is showing weirdly in Bing and a website checker ( see below ) also I have lost a lot of impressions with Google recently. Bing hasn’t indexed my website for nearly a year - possibly since I started using Cloudflare.

Robots txt - is redirecting to my home page

I have contacted Rank math and my Hosting provider - who have both done their checks on everything. and advise this is a Cloudflare problem.

Any advice, on how I can sort this out, please?

Thanks
Nikki

2

Hi,

If your Bing visits are being challenged by one of Cloudflare’s products, you should be able to check by visiting the Firewall Events page.

Once you find an event that matches a Bing visit with a challenge, you should know which product (Firewall Rule, WAF (new), WAF (previous), Bot Fight Mode etc.) is challenging Bing.

You can then set a bypass Firewall Rule or a WAF exemption to prevent this from happening.

3

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.