Running a feed bot, which is getting blocked

Hello cloudflare community!

We are running a feed bot, crawling rss feeds and fetching and aggregating content from them. We have noticed, that apparently we are blocked from sites that run cloudflare and possibly have bot protection turned on.

Question is: How can our bot become a ‘good bot’ on cloudflare’s side? Is there an application procedure we can go through or should we just contact the company personally? Anyone had similar issue or does anyone have a tip for us?

Many thanks for any help! :slight_smile:

I run a good bot and want for it to be added to the allowlist (cf.bot_management.verified_bot). What should I do?

To be added to the Cloudflare allowlist, please submit this online application.

It might also be that the websites your are looking at block or challenge your bot without using Bot Management. In my day job I block traffic from a list of hosting providers where I see large amounts of abuse. In such a situation you would need to contact the website directly.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.