A cron job on my host IP blocked from Cloudflare

Please Help!

I am running a cron job every 1 hour on the website without issue. After 1st May the cron job on the host output a log with the below content.

The helpdesk answer from my hosting provider investigated and suggests the below

Checking the content of the files we can see Please wait while your request is being verified… which is from the Cloudflare verification page, so to resolve the issue you’ll likely want to ensure Cloudflare isn’t adding a Captcha to requests from 7x.xx.0.xx (the IP from my host).

How can do this ?

Thanks in advance

What endpoint are you hitting? If it is your domain, then you can make an allow firewall rule to allow your IP address.

1 Like

No, it’s not. I’ve faced this problem on my sites and you can see some background issue in this previous reply to a similar problem. What’s blocking your cronjob is a firewall product called Webshield/Immunify360, which some hosting providers use to protect their infrastructure.

The product inserts an interstitial page (sort of a JS challenge) that causes a 302 redirect to a non-existing page, the wsidchk form you can see in your screenshot.

image

So when a bot tries to hit the origin and this firewall acts upon it, the redirect is returned instead of the expected 200 for the URL. This redirect will be cached by Cloudflare if you have a Page Rule with Cache Everything, or a Cache Rule (beta) that sets the caching for 302 to the same period of time as that for a successful (200) request, or even if your origin sends a Cache Control header with instructions to cache the 302 response.

What I’ve since done that eventually lead to eliminating this issue almost completely was to edit my Cache Rule (which I use to replace Cache-Everything Page Rule) to set the time for 302 responses to a very low number (10 seconds if I’m not wrong).

Besides having a Cache Rule with an explicit instruction for 302 redirects, you can also create a WAF Custom Rule for incoming requests that match…

URI Query contains "wsidchk"

Then Managed Challenge.

This should make the 302 redirect followed by the infringing bot to be blocked by Cloudflare. However, at this stage of events the redirect may already be cached by Cloudflare, so it’s important to set a low duration for the cache for 302 redirects.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.