I am looking for help on how to register the friendly bot.
At Aissist.io, our product occasionally need to scrape the webpages of our users in order to form a knowledge base (We don’t do internet wide web scraping). For some cases, we are blocked by Cloudflare. I tried the guidance from the internet to register a friendly bot. The guidance directed me to a online form which I filled out 2-3 months ago. Afterwards, there has been no notification or response. I am puzzled on where we are and what the next steps are.
Any other guidance on how to resolve the issue we have will be greatly appreciated.
If anyone knows the right process, please provide the guidance. Thank you very much.
I am aware of the verified bot. We don’t have a huge web scraping traffic. We only try to extract information on specific web domains, with user’s permission.