Cloudflare and Selenium Chromium

Hello. I am having an issue. I am using Selenium Automation with Chrome to run a process on my own site but Cloudflare is blocking access once Selenium tries to log in (it says “checking your browser”…). I added the IP from which Selenium is accessing the site and the exact URL also and selected “allow” but Cloudflare keeps running its check because it sees that the browser is being run with automation. Any ideas how I can manage to stop Cloudflare from blocking my automation program?

1 Like

Have you checked the Firewall Events log (Firewall → Overview) to see why your requests are being challenged?

1 Like

Thank you for the reply. It doesn’t actually block, it says “checking your browser…” and just hangs there indefinitely but that kills the Selenium process. It should just allow Selenium without running any checks.

1 Like

That’s JS (JavaScript) challenge.

2 Likes

Right, how can I prevent Cloudflare from doing this? If I have added the IP and even the exact page/URL and set its priority to 1 and indicated to “allow” it; I don’t see what else I can do…

1 Like

By checking your Firewall log to see which rule is triggering it, then making an exception to that rule.

I checked under the Firewall log, under Services, it says: Bot fight mode which I assume it is not a specific rule?

Ohhhh…that pesky Bot Fight Mode (really). It’s not very tolerant or configurable, so I’m afraid you’d have to disable it. It seems quite sure that your Selenium Automation is a bot, which it pretty much is. Unless Selenium can successfully respond to the JS Challenge (I honestly hope it can’t, otherwise that’s not much of a Bot Fight Mode), you’ve got to disable BFM.

As a second thought, where are you running this automation? If it’s from your desktop computer, you can try adding your domain into your computer’s local Hosts file with the origin IP address so you bypass Cloudflare.

Yes, I found the disable button and it should do trick (disable for a second, let the bot log in and enable again). I am using an Amazon server to run this but I suppose I can update the hosts file on the server too? Have you done this in the past? Is the idea that if I add the domain of the site to the computer’s local hosts file then the browser will somehow not be detectable as a bot? I will go through that article. THANKS.

1 Like

If you use a local hosts file, either in your AWS instance, or your desktop computer, it will connect straight to your site’s origin and not be affected by any Cloudflare settings.

1 Like

Selenium can definitely go through the JS challenge, however, the public release is likely fingerprinted as the framework leaves a lot of traces (on purpose).

1 Like

Do you know how to get around this?

Using the local hosts file to access the server directly assumes you are allowing anyone to connect to your server directly - it is always a good idea to lockdown to only the CF IP addresses for enhanced security.

If you disable Bot Mode and add your IP address to the list in Tools it will bypass the firewall and other WAF tests (see the order and priority here: Order and priority · Cloudflare Firewall Rules docs )

Check Configuring IP Access Rules – Cloudflare Help Center :

" * Allowlist : Excludes visitors from all security checks (Browser Integrity Check, I’m Under Attack Mode, the WAF, etc). This is useful if a trusted visitor is blocked by Cloudflare’s default security features. Allowlist takes precedence over block ."

Also " Requests containing certain attack patterns in the User-Agent field are checked before being processed by the general firewall pipeline. Therefore, such requests are blocked before any allowlist logic takes place. Firewall events downloaded from the API show rule_id as security_level and action as drop when this behavior occurs."

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.