Ahrefs bot is being blocked (403)

Cloudflare is showing a 403 Forbidden error specifically to Ahrefs, which is a “good” bot.
Despite trying to create custom rules within the Web Application Firewall (WAF) to exempt Ahrefs requests from being blocked, the issue persists.

Even when the firewall is set to “SKIPPING” for Ahrefs requests, the 403 error continues to be returned, indicating that the blocking is happening elsewhere or due to another rule taking precedence.
If i look at the request in the security dashboard i can see it marked as “SKIP” which is odd as it still gets a 403. I have tried inserting the IP in the IP Access Rules but it doesn’t change anything

Any insights or suggestions on resolving this issue would be greatly appreciated.

Hello,

Sorry to hear you are having issues with this bot being blocked despite the skip rule. So just to confirm if you go to security events then lookup up the bot via its user agent string ahrefs. Do you see any other items that may be blocked it via the WAF? Or are all items showing a SKIP? If the skip is showing it may not be the WAF but the origin itself blocking it.

Another way to make sure is to do a WAF rule to allow anything that contains the user agent string ahrefs. Also make sure to select skip for the additional options at the bottom. If all you are seeing is skips I would wager that block is being served somewhere else. Additionally, all blocks are logged via RAYID so if we did issue it you would be able to look up the RAYID on the request and find the event within security events.

Thank you so much for getting back to me.
I have added the ahrefs bot as a custom rule and it is served a SKIP.
However it is still being blocked which seems odd?

I have found the Ray ID, but since it says SKIP i don’t know what to do:
image

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.