Whats to do with bots with all 301 redirect request


i get bot traffic from user agents like
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler) 605 request

CriteoBot/0.1 (+https://www.criteo.com/criteo-crawler/) 477 request

ias-ie/3.3 (former https://www.admantx.com + Integral Ad Science | Digital ad verification solutions for all screens and devices) 294 request

Mozilla/5.0 (keys-so-bot) 288 request

Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php) 282 request

Mozilla/5.0 (compatible; TTD-Content; +https://www.thetradedesk.com/general/ttd-content) 236 request

IAS Crawler (ias_crawler; Site indexing policy | Integral Ad Science) 196 request

ias-va/3.3 (former https://www.admantx.com + Integral Ad Science | Digital ad verification solutions for all screens and devices) 174 request

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36 162 132 request

FeedBurner/1.0 (http://www.FeedBurner.com) 89 request

peer39_crawler/1.0 83 request

Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) 82 request

ias-jp/3.3 (former https://www.admantx.com + Integral Ad Science | Digital ad verification solutions for all screens and devices) 73 request

Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot) 68 request

all of them use Source operating systems Unknown , Source browsers Unknown
please suggest what to do can i block all of them

You won’t be able to block by browser type. For blocking these user agents you’ll need to create a WAF rule, using the field “User Agent” and paste the user agent you’d like to block in the value. To include all user agents you’ve listed in one rule, you can use the “Or” operator.

Here is our documentation on creating a custom WAF rule