I am using CF with Pro plan and Bot fight mode enabled.
When Bot fight mode enabled, Pagespeed does not work. It returns “Oops! Something went wrong. PageSpeed Insights encountered a problem collecting the lab data.”
When Bot fight mode is disabled Pagespeed works fine.
I checked the firewall and found ASN=“AS15169 GOOGLE” and IP= 1. 34.134.192.230 or 74.125.210.124 or 1. 74.125.210.100 or 1. 74.125.210.127 or 1. 66.249.93.207 or 74.125.210.122
If I allow IP range with CIDR=16 probably too many IP address.
Please add Pagespeed to your allow list of good bots (same but less important: mozilla observatory and securityheaders.com).
I’m also hoping to find out if the “Chrome-Lighthouse” user agent will be added to the list of good bots. We run a report that uses the PageSpeed API on a nightly basis to get pagespeed performance and we’re currently blocked from doing so.
Hi support team,
I am facing problem regarding my Pagespeed Insights Score. When I pause the Cloudflare, my site shows a good PSI score i.e. 96+ for mobile devices. But, as soon as, I resume Cloudflare to the ON state the PSI score reduces to 40 to 50. Even worse, for some days, it’s unable to calculate any PSI score for my site.
This problem remains when I enable the Cloudflare. And if pause the Cloudflare, the PSI score increases back to 96+ (even 100 sometimes) immediately. Currently, when I am writing this support mail, I am keeping Cloudflare ON/Enable.
Kindly help me resolving the issue. Thank You
In our Cloudflare account, our website is under attack and our hosting company says we will need to block the fasthttp requests. If you are not sure how, please contact Cloudflare to assist with this and you can show them the following log:
########
81.25.234.196 - - [14/Mar/2022:16:23:45 -0700] “GET / HTTP/1.1” 301 233 “-” “fasthttp”
Support sends generic automatic responses. How do we block fasthttp? Will BOT Fight Mode do it on the free plan? Do we need to upgrade to pro? Do we need to find another Host or CDN that can make it happen?
Is that a User-agent? 
If yes, you could block it by adding it to a Firewall Rule, otherwise to User-agent blocking section → both do the work, just add it and make sure the action is set to “block”.
We are seeing the same issue, Super Bot Fight Mode is blocking https://pagespeed.web.dev/ from running it’s pagespeed tests. We can see these requests blocked in the Firewall logs. Can we have someone from the Tech team acknowledge this issue?
1 Like
I made a list of use agents (not full name) that seem legitimate and blocked by super bot fight mode:
- IAS Crawler / IAS Wombles / admantx
- moatbot
- AmazonAdBot (ASN=AS14618 - AMAZON-AES)
- AWS Network Health
- AdsBot-Google
- PubMatic Crawler Bot
- GumGum-Bot
- CriteoBot
- thetradedesk
- Uptime
- Quora-Bot
- SemrushBot / SiteAuditBot
- Yahoo! Slurp / Yahoo Ad monitoring
- proximic?
- SemanticScholarBot ?
- DotBot (moz)
- Googlebot
- Clickagy Intelligence Bot
- AwarioSmartBot
- AppleNewsBot
- TelegramBot
- bingbot
- DuckDuckBot / DuckDuckGo-Favicons-Bot
- SemanticScholarBot
- YandexBot / YandexMedia / YandexPagechecker / YandexAccessibilityBot
The IAS / moat / ad networks bots blocking will reduce my ad income 
I have no way to determine if these bots are real or fake bots. Isn’t that bot fight mode is for?
2 Likes
I am having the same problem. Can this be fixed by making a firewall rule?
PageSpeed uses this IP: 66.249.0.0/16
Allow it in Tool.
1 Like
Thanks so much!
66.249.0.0/16 is a range used by Newfold Digital, Inc. under the hostname “rtr.spry.com”.
NOT PageSpeed Insights.