I’m trying to allow for the site ahrefs to properly crawl my site to track statistics. I’ve gone ahead and added their IP range correctly (I think). Under firewall → firewall rules I added all 63 of the IP addresses as “IP Source Address” “equal” “IP Address” “or”. I did that 63 times, but when their bot checks the status of my robots.txt file it returns this error still -
The HTTP server returned error 406: “Not Acceptable”. This request was likely filtered by a firewall.
This website is protected by the Cloudflare firewall
Am I entering the IP addresses in the correct place in Cloudflare?
Am I supposed to enter it under Firewall → Tools? If so how would I enter the range? I tried entering
51.222.253.0/63 (maybe it’s formatted wrong) and it said it wasn’t accepted
Also, looking at the article below would I need to add the ~60 individual IPs starting w/ 168 as well? Any help would be appreciated as I’m new to all this.