A 500 Error on the CloudFlare side for all external services/tools/bots

[solosik]

Hello,

I’m experiencing a kind of 500 Error on the Cloudflare side (not my server) for almost all possible bots/services/tools.

The website is - https://www.difference101.com/

Users can still access the website, and the curl command is working fine.

But W3C Validator, Pinterest, Redirect Checker, Redbot checker all have issues with the same error (500 on the Cloudflare side)

To test, please visit any of the following services:

• REDbot: <https://www.difference101.com>
• https://validator.w3.org/check?uri=https%3A%2F%2Fwww.difference101.com
• https://www.redirect-checker.org/

Expected Result: 200

Current Result: 500 Internal Server Error on Cloudflare server (see screenshots)

Only bots/tools/services are blocked. Regular users can still use the website.

If you disable Cloudflare (bypassing Cloudflare), everything works fine.

I already tried to debug with my hosting provider, but after several tries, we can see that the problem comes from Cloudflare itself, but not from our server.

For test purposes, we even installed a fresh version of the website with default configs.

• I don’t have any Firewall rules, only default settings.
• Cloudflare bot protection is turned off, but even in this case, it should show a kind of 403 restricted error, but not a 500.
• DNS settings and SSL settings seem to be OK. However, the error persists.

Please help us, since many services that we’re using are blocked now: Pinterest, Publer, etc. They can’t access the website, read our RSS, etc.

I will be very appreciative of any help.

I really appreciate any help you can provide.

PS. I submitted a support ticket, according to the advice on their 500 error troubleshooting guide, because the issue comes from Cloudflare, and not from our server, but it seems that the Cloudflare team is quite busy at the moment…

Kind Regards,
Alex

Screenshots:

[erictung]

So I found this from the response body after running REDbot: <https://www.difference101.com>

Check your Workers script and see is there any issues or bugs within that script.

[solosik]

Thank you very much, Eric,

Indeed, I’m using Siteground hosting, so they manage our DNS configuration with Cloudflare. It seems, that they recently added their own worker “sg_worker”.

Unfortunately, this worker causes a lot of issues. See these recent topics:

• https://community.cloudflare.com/t/boxtal-plugin-not-work-correctly-with-cloudflare/225858

• https://community.cloudflare.com/t/siteground-with-cloudflare-limit-cloudflare-workers/229580/2

• https://wordpress.org/support/topic/cloudflare-worker-error-sg_worker/

I already asked Siteground to fix the issue.

We will see how they will do it.

At the moment, the best temporary solution will be to disabled the worker. However, I’m not sure what kind of other issues it may cause.

Kind Regards,
Alex

[solosik]

I will let you know their response here as soon as they answer. (it seems that it is fixed now, see UPDATE 4)

UPDATE 1 - Possible Temporary SOLUTION:

In my case (on my website - Difference101.com), the issue was related to the Cloudflare Full Page Caching functionality inside their SG Optimizer plugin.

When you enable this feature, Siteground creates a worker inside your Cloudflare. And as we know, this worker causes many issues. I asked Siteground to keep my ticket open till they fix it, and at the moment, it remains open.

The solution here might be to disable the Cloudflare Full Page Caching feature inside the Cloudflare tab and then remove the worker from your Cloudflare account.

UPDATE 2 - Further Actions:

It seems that the Siteground team is not going to fix this issue and keep ignoring it even after several requests from their customers.

In this case, we may help them pay more attention to this problem and take some small but essential actions:

1. Create a support ticket on their website and add the link to this thread

2. Create a support request on the SG Optimizer plugin page (Wordpress.org)

UPDATE 3 - List of threads with similar issues with Siteground:

1. Boxtal plugin not work correctly with cloudflare

2. Siteground with cloudflare => limit cloudflare workers - #2 by erictung

3. Cloudflare worker error (sg_worker) | WordPress.org

4. Worker threw exception

5. Cloudflare CDN for Siteground Preventing Pinterest from Pulling Meta-Data

UPDATE 4 - Fixed - Thank you all for participation!

Several days ago, my ticket was closed by Siteground support with the mention that this issue is fixed now. (please note that I didn’t test it, so please ping my post if there’re any issues again)