How to exclude SEO bots from Rate limiting rules (using Enterprise plan))? Security

Website, Application, Performance Security
1

We have a rate-limiting rule for our website, but we would like to exclude bots(or more specifically SEO bots) from this Rate limiting rule, how can we achieve this

2

I just answered this on your other thread which you deleted.

1 Like
3

Many Thanks sorry for this, for our case it is screaming frog SEO bot, which is being blocked and gets 429 error , this is because for the rate-limiting rule for sure

4

I see. They’re not a regular crawler. It’s an audit tool. And it looks like something you download and run, right? If that’s the case, then you can try to add the spider’s IP address as an Allow in Firewall → Tools.

5

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.