Google "Indexing request rejected" with WAF rule

[kurt2]

What is the name of the domain?

boudoirphotography.mu

What is the error number?

No error

What is the error message?

Google “Indexing request rejected”

What is the issue you’re encountering

Google Not indexing page

What steps have you taken to resolve the issue?

Expression Preview

(starts_with(http.request.full_uri, “https://boudoirphotography.mu/booking”) and not cf.verified_bot_category in {“Search Engine Crawler”})
Then take action…

Choose action
JS Challenge

Was the site working with SSL prior to adding it to Cloudflare?

Yes

What is the current SSL/TLS setting?

Full

What are the steps to reproduce the issue?

I have tried several combinations : and / or, starts with/ equals to etc… nothing wilk work

Screenshot of the error

[sandro]

Check https://dash.cloudflare.com/?to=/:account/boudoirphotography.mu/security/events for what blocked the request and make the necessary adjustments.

It might be that your request is not sent from a proper Google crawler but from a different system not classified as crawler.

[kurt2]

Many thanks for pointing me in the right direction.

I found out that in “Events” :
Mar 3, 2025 10:09:41 AM - JS Challenge - Tor - 192.42.116.210
Mar 3, 2025 10:09:30 AM - JS Challenge - United States - 66.249.68.38

The first one seems to be the spam bots. The second one seems to be the legitimate Google indexing request.

So I adjusted the rule :

(starts_with(http.request.full_uri, "https://boudoirphotography.mu/booking") and ip.src.country eq "T1")

Seems to be OK this way…

[kurt2]

Hello,

An update to my issue. After a few hours the spams through my booking page started again.
There were only a few but same ones as before (I can see through the mail received i.e same subject lines).

I found the option : Security > Bots > “Bot Fight Mode” so I enabled it.

This morning the spams started again. Any idea ?

[sandro]

Does Google work now?

For another issue, please open a separate thread.