We’re having an issue hitting CloudFlares rate limiting. We’re hitting the rate limit from our own server, mainly when we run a build (Nextjs). We’re also hitting it when we view 5000 products at a time in the backend.
We have set all the options to SKIP in the WAF for our servers IP and our computers IP - but it’s still hitting rate limiting.
- We have also just added the IP’s that we use to WAF > Tools to skip rate limiting (currently waiting to see how that works out)
I’d like to add Googlebot IP ranges to skip rate limiting too. But how do I do that? Cloudflare doesn’t allow certain IP ranges.
" IP address range: Cloudflare accepts the following CIDR notations: /16 and /24 for IPv4 addresses and /64, /48, and /32 for IPv6"
Googlebot IP ranges include /28, /27 etc. (full list of Google bot IPs here: https://developers.google.com/static/search/apis/ipranges/googlebot.json
Also, would we benefit from setting up the Load Balancing with Cloudflare? We’ve not done this yet.
- We’re using Cloudflare Pro if that helps.
Thanks in advance!