Newer version of block AI Scrapers and Crawlers

[sarang.sss29]

What is the name of the domain?

https://dash.cloudflare.com/

What is the error message?

We have a newer version of this setting available. Make sure to enable it to help you block AI Scrapers and Crawlers.

What is the issue you’re encountering

Option does nothing

What steps have you taken to resolve the issue?

I went to the Security > bots option and I am seeing this notice:
We have a newer version of this setting available. Make sure to enable it to help you block AI Scrapers and Crawlers.

When enabled, it removed the WAF rule. Prior to this, it used to create a WAF rule. Now it does not.
Tried deleting the WAF rule and enabled this option. But then it does not create a WAF rule.
Is this the new version?

What is the current SSL/TLS setting?

Full

Screenshot of the error

[fritex]

@sarang.sss29

WAF rule

From the blog post:

Any customer with an existing WAF rule set to challenge visitors with a bot score below 30 (our recommendation) automatically blocked all of this AI bot traffic with no new action on their part. The same will be true for future AI bots that use similar techniques to hide their activity.

However:

@sarang.sss29

When enabled, it removed the WAF rule. Prior to this, it used to create a WAF rule. Now it does not.
Tried deleting the WAF rule and enabled this option. But then it does not create a WAF rule.
Is this the new version?

May I ask if you’re on a Free or Paid plan?
Are you already using the maximum allowed WAF rules for your plan type?

I have had WAF rule as follows from the post below.
Also have had the warning message as you.
Once enabled, the WAF rule is disabled → hopefully a Managed Rule in the background is being enabled and active further
Might have to check Firewall Events in coming days and keep an eye on it.

EDIT: Could be a ruleset and might depend per plan type if it’s available to search up for it

Wonder if it’s also realted to the “old” and “new” WAF as it was in past

Helpful post:

Cant enable "Block AI Scrapers and Crawlers"

Make sure you have not used all of the (5 for a free plan) WAF custom rules. You need a spare one for the Block AI scrapers rule. If you have, try to combine 2 rules into one to make room. This is the rule the switch creates… [Screenshot 2024-07-04 at 21.05.21]

A bit more info about this feature:

Block AI Scrapers and Crawlers

Hi there caitrionamctiernan At this situation I would advise to leave Cloudflare’s “Block AI Scrapers and Crawlers” running and if it isn’t enough you can specify within a WAF rule a block action on those specific bots and of course to take into consideration the list here: https://radar.cloudflare.com/traffic/verified-bots If opting to add the WAF rule be attentive to have “Verified Bot Category” Hope this clarifies. Cheers!

[sarang.sss29]

Thanks for the reply. I am on the free plan. And had 4 rules in the WAF. When I enable the new Block AI option, it does not create a new WAF rule.
The previously created rule in WAF with said “older version” would disable this WAF rule when I switch the toggle to ON.
Looking at the security events, it does seem to block AI crawlers. But I am not sure.

[sdayman]

This looks like it saves you one WAF Custom Rule, so that’s nice. As far as if it works or not, I’ll start by assuming it does, just like the other bot fight settings. Those are just simplified WAF Rules.

For reference, my “old” rule that was in WAF Rules did catch this today:

Another domain caught this with the old rule:

Different domain: