Robots.txt Blocked by Cloudflare

Hello Team Cloudflare, Good Morning!!

From yesterday google didn’t crawl my website and the reason shown is Robots.txt file unreachable. But, my robots.txt file is valid and its working properly.

After that i akso also my hosting provider, are they blocked google bot.m? And the answer is no. So, i believe that from the. Cloudflare side.

So, please help me how i can unblock mt site from this error as soon as possible.

Thank you.

Make sure your WAF rules aren’t blocking it, see…
https://developers.cloudflare.com/waf/custom-rules/use-cases/allow-traffic-from-verified-bot

1 Like

There is no custom rule we add in WAF… ANY OTHER SUGGESTIONS

What is the site?

Please Check

I just checked, and it’s not blocked for me:

% curl https://www.thecreativityindia.online/robots.txt    
User-agent: *
Allow: /

Sitemap: https://thecreativityindia.online/sitemap_index.xml

Do you have any more information on the block, such as an error message or error screen? There should be a RayID associated with the blocked request that you should find in Firewall Events:

1 Like

I don’t create any custom Firewall Events. i did not do anything.

any other methods?? Sir

Hello sir, any response please!!