Cf.client.bot is matching all browsers

I have custom firewall rules set up on staging and production domains which presents a JavaScript challenge. On staging this is simply (cf.client.bot) on production it’s the same but only matches specific paths. This has been working successfully for at least six months. Last Tuesday or Wednesday (2022-04-12 ish) I began seeing intermittent 503 errors when browsing both sites, with the following error message displayed:

This web property is not accessible via this address.
DDoS protection by Cloudflare
Ray ID: xxx

This is happening on multiple different browsers. If I switch off the firewall rules on either site the problem stops. Has there been a change to the list of user agents that cf.client.bot matches?

A firewall block would have a different error code. A 503 error wouldn’t be related to your bot rules.

Is it possible for the Javascript challenge to produce a 503? Something like what is happening in this post:

I’m not 100% sure that’s what I’m seeing but it is similar. Is that issue still active?

I’ve had an image GET request which appeared as a 503 error in the browser and produced a matching event in Cloudflare’s firewall dashboard. That is currently not happening since I switched off the firewall rule and I’ve already eliminated code changes as an issue. Is there any info on what cf.client.bot uses to determine if the client is a bot? Only the failed image request had this UA string (from Cloudflare’s firewall event):

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36

https://blog.cloudflare.com/friendly-bots/ gives some information into how ‘good’ bots are determined but I’m not aware of an exact list anywhere the list is available at https://radar.cloudflare.com/verified-bots.

Do you have a screenshot of the firewall event in the activity log?

This is the firewall event log:

This is the page that appeared for the request:

Is the request yourself or one by an actual third party?

If you know the third party that generated the request, check https://radar.cloudflare.com/verified-bots to see if they’re on that list.

The request was made by one of my colleagues using the Chrome browser on Windows. Why would that browser match as a bot? The bot list is useful but I’d be interested in knowing how Cloudflare checks the client is a bot.

That IP address is a Cloudflare Worker - Workers are known bots.

The IP address 2a06:98c0:3600::103, mentioned in the article as appearing in the CF-Connecting-IP header, is a special IP address that is used for all cross-zone requests that come from Workers.

If you’d like to make sure that only your zone’s Workers can access your resources, look at the CF-Worker header or cf.worker.upstream_zone firewall field. https://developers.cloudflare.com/workers/runtime-apis/headers/

2 Likes

That’s not the same RayID. The log entry ends with ae76f, but the screenshot shows e376af

Firewall logs let you filter on the Ray ID from the screenshot.

2 Likes

Ah my bad. The correct matching request is 3 seconds later but otherwise identical.

I think you’ve found the answer to our problems. We have a complicated process where our parent company uses Cloudflare to proxy content to our site (also using Cloudflare). They made some changes last week around rate limiting which didn’t appear to be the cause of the issue we’re seeing but it looks like there have been some other changes introduced. Thanks for your help.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.