I’m not really sure what caused this but here are the symptoms and events leading up to it.
I was doing a little bit of updating to my website today and turned on dev mode temporarily to test a JS script, perhaps maybe a minute or two. (timestamp 1:12pm)
About an hour after a colleague of mine attempted to upload an image (~1.5mb) to the server via wordpress admin. (timestamp 2:25pm) It failed once and she tried to resize it down to ~350kb and reupload (timestamp 2:39pm) by which point the server was already unresponsive.
Looking at my access logs I can see that there is a huge amount of traffic through Cloudflare servers starting around 2:28pm which if I had to guess appeared to be a crawl. It’s a little hard to tell what is real traffic and what might be a crawl. There were about 3000 requests processed over the following half hour.
The big problem is that this absolutely destroyed the limit on my server’s disk throughput. While this was happening no other requests could be processed and I couldn’t even ssh into it.
At best all I could do is hope that whatever is hogging the resources would stop or I’d to go in and shut the whole VM down.
I’m wondering if this combination of events triggered the CDN to want to re-cache everything. I didn’t see much in the Cloudflare logs, in fact, the bandwidth during that time was quite low. There were a lot of requests in the previous hour, but that could have been me when I turned on Dev mode. Is there anything else this could have been other than a crawl by Cloudflare?
Suppose it was a crawl, is there a way to limit how much it can process in one go? It seems absurd for it to cripple the site it’s trying to crawl by maxing out the server’s resource limits.