Occasional HTTP 520 Errors

[research]

We are experiencing occasional HTTP “520” errors when loading resources from our web-based application. If I refresh the app 50 times it might happen once.

I have reviewed Cloudflare docs and many community topics to find a solution. But so far, I haven’t been able to fix things. I spoke with our hosting company and they also weren’t able to identify any issues on their end. No errors show up in any Apache logs. I do see the successful entries in the Apache log showing HTTP 200 status. But when a file fails to load, there is no record of it in the Apache access log or error log.

Our web application consists of typical web files, like HTML, CSS, and JS. But it also loads (via JavaScript) a couple dozen JSON files that contain data. The overwhelming majority of the time, all files load fine. But occasionally one of these JSON fails to load and DevTools shows the HTTP 520 error. Whenever these data files fail to load, our application breaks (since it needs the data), so it is easy to see when something has gone wrong (as opposed to say a CSS file where you might not notice it failed to load). So, I can’t say for sure that the only files to ever fail are the JSON files. They just happen to be the most easily noticed.

Our server is a typical LAMP setup and there is not server-side PHP involved. It’s just Apache accessing files.

Does anyone have any suggestions or things we could change that might help? I’m running out of ideas.

Thanks!

[M4rt1n]

I exactly had the same issue 5 or 6 days ago. But the good thing was I previously changed something on my server configuration. I exactly had the same error at JSON files (with Shopware 6)

Let me tell the storry from the beginning.

I do have a few servers, some proxied by Cloudflare, some not. But one of the Servers I lately upgraded from Debian 9 to Debian 10. On this server I used Plesk as a hosting panel.
After doing some adjustments at serverside and adding some missing programmes I also noticed this 520 errors. But not the normal ones, I encountered:

“Error 520: Web server is returning an unknown error”

Now I still knew what I have installed and configurated ony my server in the last hours.
I started to uninstall ALL the programmes I have installed in the last hours which I could find our from the ~/.bash_history. Then I restarted the server and still encountered the issue of all few minutes getting this error.

Now I realised I had changed some webserver setting on Apache2 and here we go: this was the issue!

Sidenote: Plesk does run Nginx infront of Apache2 (I used this configuration on this Plesk Server) and this actually is Plesk stanadard.

But the issue stared since I activated the Apache2 Modules:

• Cloudflare
• HTTP2

What I did what completely solved the problem for me:

1. Uninstalled and removed cloudflare.so (mod_cloudflare) from /usr/lib/apache2/modules/
2. deactivated http2 in Plesk Panel:
   
   

After this (at least on Debian) run:

service apache2 reload && service apache2 restart

And please test again.
Hope this will help you as it helped me. I have never even before encountered issues having HTTP2 Enabled for Nginx OR Apache2 but now it seems to be a problem - cant say why?

Notice: as this problem seems to be related to the extensin cloudflare.so or http2.so this tutorial can also apply to other distros and non-Plesk Servers.

Also: I do not use mod_cloudflare (as it is deprecated) anymore but I use nginx with real_ip_header to get the real IPs of a request behind Cloudflare.

.

Tl;Dr: cant say if it was the Apache 2 module mod_cloudflare or http2 but one of these modules seems not to be compatible with Cloudflare anymore. As support for mod_cloudflare was discontinued since Debian 9 & Ubuntu 18.04 LTS the chances are that this module is not longer compatible with Cloudflare in combination with newer systems then Debian 9 & Ubuntu 18.04 LTS (INFO).

[research]

Thank you so much for sharing your experience.

We don’t use Plesk, and we don’t use any mod_cloudflare extension. Just Apache on CentOS, managed through DirectAdmin. I could look into disabling HTTP2. I did find it interesting that you also stated you use a plugin for the real IPs. I’ve seen other posts indicate that alone supposedly fixed things for them.

[M4rt1n]

I dont know how exactly the problem have to be solved. But I’m willing to share all the infos I can share.

To setup Nginx with real_ip_header I used this script as a cron:

#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
CFTEMP=/root/cloudflare-ips.txt
curl -sS https://www.cloudflare.com/ips-v4 >$CFTEMP
curl -sS https://www.cloudflare.com/ips-v6 >>$CFTEMP
sed -i -e 's/^/set_real_ip_from /' $CFTEMP
sed -i '1ireal_ip_header CF-Connecting-IP' $CFTEMP
sed -i '/[^;] *$/s/$/;/' $CFTEMP
mv $CFTEMP /etc/nginx/conf.d/cloudflare.conf
#check if everything is OK
nginx -t && printf "Valid\n" || printf "Error\n" | grep 'Valid' &> /dev/null
if [ $? == 0 ]; then
        echo "restaring nginx"
        service nginx restart
        echo "done"
else
        echo "something is wrong"
        mv /etc/nginx/conf.d/cloudflare.conf /etc/nginx/conf.d/cloudflare.conf-error
        echo "check cloudflare.conf-error file"
fi

with this config:

0 0 * * * ~/scripts/remote_ip_cf.sh (this runs daily)

to run the script and pass all CF-IPs to Nginx. Also I now have disabled the following modules:

• cloudflare (mod_cloudflare.so)
• http2 (mod_http2.so)

Since then it runs without any problem. I personally think this was related to mod_cloudflare beeing incompatible with Debian 10 and the fact that I now obtain the “real_ip” twice. Once in Nginx and once in Apache2.

If you find the solution to this please post it here I for myself am very curious about this issue.