Site can't be crawled


I maintain a site and have set Cloudflare to only allow UK residents to visit the site.

Since then, according to the customer, the site can no longer be crawled and the Google Ads no longer work. I found a post in the forum and deleted the robots.txt entry.

Is there anything else I can do?

You can make a rule to allow verified bots to access the domain which should include the google bot.

1 Like

Thank you, I’ll try this. I’m pretty new to that, hope it works.

I now got more information. They got a message that Ads are dissapproved because destination is not working. HTTP error 403.

Can this be caused by the firewall?

We transfered the website, but withouth big changes. Just hosting provider got changed, no domain or anything else.

It could be caused by the firewall. You should check the firewall events to see if you can find what is blocking the conenctions.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.