Very slow or failed load times from random Geolocations

I am seeing failed tests in Observatory but not the same each day, sometimes they are very fast sometimes they fail.
My server is snappy and responsive, wondering why sometimes the page load speed is arbitrarilly slow (10sec+) when much of the time is >1second.
The server is never under much load but it seems like there’s some issue somewhere I cannot narrow it down.

Can you try Community Tip - Fixing "The Speed test could not run"?