Rate limiting rule blocking googlebot even though allow cf.client.bot is active

Hello, I have a rate limiting rule set on two of my websites in Security > WAF > Rate limiting rules and also a rule to allow known bots (cf.client.bot) in Security > WAF > Firewall rules.

According to the firewall events section, the allow rule allows multiple requests from Googlebot (as expected). Still, when I go and check the firewall events for the rate limiting rule, I see that it is blocking one specific googlebot IP (66.249.64.166).

Am I missing something?

1 Like

NOTE: I have checked the firewall events on the second site and there the googlebot IP that is being blocked is different: 66.249.64.212.

Im confused.

Your rate limiting rule needs to have a parameter to exclude known bots from it separate from any WAF rule that might allow it.

1 Like

Thanks for your response. How can I do that if the expression builder only allows URI path field? It’s grayed out and it does not allow any change:

This is the resulting expression:
(http.request.uri.path contains “/” and not http.request.uri.path contains “.”)

I believe you’ll want to edit the rule and add

and not cf.client.bot

It seems that there is some problem, Im getting the following warning when trying to add that to the rule:

cannot use field cf.client.bot, not available to this plan/phase

Does it mean that I have to upgrade to a paid plan in order to use this?

Hi, can anybody help with this?

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.