Google PageSpeed Fails when Cache enabled

[mranalogy]

With a website that routed through CloudFlare with standard Caching enabled:

Are there any known issues with testing that with Google PageSpeed or Pingdom’s speed test?

FYI, Cross posted this to Serverfault

SYMPTOM
When I test the speed of our site with Google PageSpeed & Pingdom’s speed test, they return horribly slow (or no) results when Cache is enabled.

=========================
My Testing

1. I’m measuring our site with Google PageSpeed insights and Pingdoms’s speed test. Our site is routed through CloudFlare for SSL and Caching.

2. *Disabled ** Cache. Waited 5 minutes, tested per the above. Page seems good (fully loading in 5-8s
   3. Google PageSpeed reports score of 81-93% (4 tests)
   3. Pingdom reports load times 5.27-7.2s

3. Google Pagespeed fails to load it: Lighthouse returned error: NOT_HTML (tested 4x over about 10min). Failed all 4 times.

4. Enabled Cache again. Repeated tests. Google PageSpeed & Pingdom both report horrible results

5. Pingdom reports: 17s 23.4s load time

6. Google Pagespeed fails: FAILED_DOCUMENT_REQUEST & Lighthouse returned error: NOT_HTML

7. Geo peformance looks terrible
   

   

8. However, when I load the page myself in Chrome (F12, Control+R refresh), it loads in fine (Chrome reports fully loaded: 2.4s, 6.5s 2.4s

9. Reenabled Caching. Waited 5m

10. Pingdom shows much better load times: 4.3s - 6.28s

11. Pagespeed works, showing 82%, 92%,
    10 Geo peformance looks good
    

    

*I’m testing Caching by disabling it via the Caching.Development Mode option.