Access for Ahrefs crawler bot

[Michall]

Hi,

Does anyone know how to properly configure Cloudflare settings so that Ahrefs Bot can access the site?

I used all the tips in the guide: https://help.ahrefs.com/en/articles/2515375-how-to-whitelist-ahrefs-ips-in-cloudflare, but I still see a message: The HTTP server returned error 403: “Forbidden”. This request was likely filtered by server configuration.

I check access at https://ahrefs.com/robot

[cbrandt]

Hi,

If requests are in fact being blocked by Cloudflare, there should be a Security/WAF event logged that can help you identify the feature that is blocking, which you can then adjust or disable. Please follow the steps in this answer to a related topic:

Form submission breaks when enabling Cloudflare

Hi, Please visit your website and perform the action that would normally result in a request being blocked. Make sure you have Developer Tools open (F12). If the form submission is blocked with a 403 status code: Wait a few minutes Go to Dashboard > Security > Events. If the request was in fact blocked by Cloudflare, you should find an event related to that block action. Depending on your site traffic, you may need to filter by IP address, User Agent, URI Path, etc. to find it. Check the “Se…