Speed issues AMS datacenter

A few hours ago (round noon) our site started showing performance issues. After a bit of debugging is seemed to be coming from cloudflare. When changing my hosts file to access our servers directly instead of over cloudflare the speed was great, changing it back the download times were really bad again. It seems to be specific to the download time, the initial connection time is fast but the download time is horrific during these problems for some reason (see image).
Screenshot 2023-11-24 152909

This is the 3rd time this has happened, it looks like it is coming from the AMS datacenter. When I use nordvpn to germany or united kingdom (and I see the datacenter change in the dr. flare extension) the speed issues also seem to disappear.

Are there more sites with this issue, and more importantly is there a way I can prevent / deal with this issue? (Other than not proxying our site through cloudflare ofc)