Exclude search engine bots like google, bing, etc

Hi there,

I would like to exclude known search engine bots, like google, bing etc., from BLOCK KNOWN BOTS to crawl my website.

Or maybe only BAD KNOWN BOTS are blocked and this is a silly question.

Please give me some guidance as I don’t want to block search engine bots from crawling and indexing my website pages.

Thanks

It’s not a silly question, as that wasn’t initially clear to me, either. I’m sure it would have been clear if I’d read the documentation I bet exists somewhere. But, yes, Known Bots are the good ones. Blocking the bad ones without blocking good visitors is the tricky part.

This topic was automatically closed after 30 days. New replies are no longer allowed.