Blocking AI Bots is giving 403 errors even on normal requests, whats the blocking criteria or pattern ?
What is the current SSL/TLS setting?
Flexible
What are the steps to reproduce the issue?
Under Domain > Security > Bots, I turned ON the 2 options - Block AI bots and AI Labyrinth.
From within my PHP app, I am doing a HTTP GET (file_get_contents) to a url with 1 custom HTTP header in the request.
Earlier the HTTP request was working fine. No issues.
But after turning ON the above 2 Block AI options, its giving 403 Forbidden errors.
After turning OFF, code is again working fine.
Can you share the blocking criteria / patterns which lead to blocking of requests as potential AI bots ??
Also try to check your Cloudflare Security Events to confirm exactly what’s triggering this 403 error.
Please confirm if your Super Bot Fight mode is set up to block Verified bots or if Block AI Bots is enabled. This would block known good bots like Googlebot.
Under what section, can I check the CF Security Events ?
CF should return proper error message for the 403 blocking when CF is blocking clients as potential AI bots under the specified option. Maybe display a detailed error msg under CF dashboard ?
Please do check the following answer by @cbrandt as it provides the steps needed to check the location of Cloudflare Security Events and what to do in case in it not a Security Event situation.
After turning ON the 2 options - Block AI Bots and AI Labyrinth, my HTTP request from PHP code (file_get_contents) is getting blocked with 403 Forbidden error.
How is the above request/client categorized as an AI bot ?
Use Agent is blanks, one simple custom HTTP header is being sent, nothing else to label it as an AI Bot client req.
I changed the User Agent to Chrome but still getting the 403 errors.
Please investigate this request ?
Whats the criteria for blocking such requests ?
I want to block AI bots but because of this issue, I cannot use the feature.
In your specific case the ‘403 Forbidden’ status code indicates that the client’s request was understood by the server but cannot be fulfilled due to insufficient permissions to access the requested resource.
As I have shown in the screenshots from CF security events, its being blocked by cloudflare (not by hosting provider). That block by CF is returned as 403 forbidden error.
Anyways, I have turned OFF the BLock AI bots and AI Labyrinth options, won’t be using this buggy/beta feature.
