Bot Fight Mode preventing HSTS Preload

The HSTS Preload site allows you to submit your domains to Chromium’s HSTS Preload list, so browsers will refuse to load your site over HTTP.

When Cloudflare Bot Fight Mode is turned on for a domain, the HSTS preload site can’t verify that the domain can be preloaded, claiming that there’s “No redirect from HTTP” (e.g. from http://example.com to https://example.com). What’s actually happening though is that Bot Fight Mode is intercepting the request and swapping the redirect for a JS challenge.

For a quick fix, you can turn off Bot Fight Mode, then add your domain to the preload list, then re-enable it, but I’m not sure if that puts the domain at risk of being removed from the preload list later. It seems like entries to the list are periodically checked to ensure they’re still valid.

Could the HSTS Preload checker be marked as a “good bot” and allowed through Bot Fight Mode? If it is the case that items on the preload list are periodically verified, there might be more bots to approve than just the one behind the public-facing website.

I’m not sure if it would actually be removed from the preload list in that case. It would make sense to remove it from the HSTS preload list eventually if the tag can no longer be found, but the removal page doesn’t say anything about that.

All of the removal options tell you to keep the HSTS header but remove the preload option. In your case, it would find no HSTS header at all.

It also seems a little bit vague about whether it’ll even check for changes in the HSTS header if you don’t actually submit a removal request.

Yes, I’m not 100% sure how removal works. My concern is prompted by the message “Status: {domain} is currently preloaded, but no longer meets the requirements. It may be at risk of removal.” which appears if you check a Bot Fight Mode-enabled domain which is already on the preload list.

Okay, well I would take that as a ‘yes’ that it will check periodically, but it’s still not definite from the way it’s worded there.

I’ve just recently disabled Super Bot Fight Mode because it was causing other problems that can’t be fixed at the moment (no whitelist) and I don’t really want to enable it again to test this.

My suggestion is that you should consider whether Bot Fight Mode is worth the trouble (Super or not) and perhaps just leave it off until needed, or until there are more options to prevent issues like this.

Yes, I’m thinking I might need to turn it off again for now to be on the safe side, as I’m really not sure how the preload list works in terms of unrequested removals. I’m not sure how best to report an issue like this – I’ll likely open a support ticket when I’ve got some time.

My sites don’t exactly get masses of unwanted bot traffic, but I can see from the logs that it is intercepting a fair amount of stuff per day (scans for .env files, WordPress vulnerabilities, etc). Nothing monumental though so the impact of turning it off would be negligible.

I’m having a similar issue where Cloudflare Pages’ build servers are blocked by Bot Fight Mode too, so would report that at the same time.

I get those scans too, but not many, and they’re still caught by Cloudflare WAF. I’m not sure what type of account you have, or if this part is available on free accounts, but this is what I see in the firewall events on Pro plan for those /.env scans:

Service: WAF
Rule ID: 100016
Rule message: Version Control - Information Disclosure
Rule group: Cloudflare Specials

Other vulnerabilities are getting blocked with their own ID etc. And this is without [Super] Bot Fight Mode.

Same issue here with HSTS preload. Disabling Bot Fight Mode fixed. Perhaps adding a tooltip warning near Bot Fight Mode in the UI would be appropriate.

This is a problem, we don’t know the IP address of the robot.

Therefore, it cannot be whitelisted. After testing, I found that there are at least multiple IP addresses.

I had this before (although because I blocked Google’s AS number)

Are two of the rules you can add to the IP access rules (this may help or it may not but there may be more IP’s)