Our website has been very slow and even timing out (error 520 and 524) even though server usage is very low
What steps have you taken to resolve the issue?
I’ve checked the server with htop and iostat but CPU, memory, and util % are all very very low (from 0% - 9%) so the server isn’t really overloaded.
I’ve also checked the database using SHOW FULL PROCESSLIST and there aren’t any long-running queries.
I’ve tried restarting the apache2 webserver and even rebooting the Lightsail instance, but it’s still very very slow.
I’m starting to suspect it may have to do with my Cloudflare configuration.
An HTTP Error 524 indicates a timeout after establishing a TCP connection and waiting for a response from your origin.
If your origin server does not return any data after 100 seconds, we will close the connection and serve a 524 error. This can happen if your server is simply taking too long because it has too much work to do - e.g. a large data query, or because the server is struggling for resources and cannot return any data in time. We recommend that you implement status polling of large HTTP processes to avoid hitting this error.
Here are the options we’d suggest to work around this:
Run the scripts via a grey-clouded subdomain.
Change the script so that it kicks off a long-running background process, and quickly returns a status which the browser can poll until the background process has completed, at which point the full response can be returned. This is the way most people do this type of activity these days - as keeping HTTP connections open for a long time is unreliable and can also be very taxing.
