Legit Yandex Search Bot Blocked by Firewall

Hi All.

I just checked the Firewall Logs and see that
a huge Number of Blocks all related to Yandex appeared in the last 24 Hours

See Photo:


I checked the Bot Myself following Yandex recommended steps as described here

and all firewall blocks by Cloudflare in the last 24 Hours were wrong as this
accesses belonged to the legit Yandex Bot.

See here DNS Reverse Lookup

IP: 5.45.207.124
Hostname: 5-45-207-124.spider.yandex.com.

Despite that the Yandex Bot access the Webpage from a Legit IP Adress
the User Agent is also listed as Legit on their Website itself !

Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
The main indexing robot. Yes

Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)
Indexes images to display them in Yandex.Images. Yes

So this is clearly something bad on the Cloudflare side as it blocks legit yandex bot.
Would be good if this can be fixed by Cloudflare.

Obviously, crawler.

From your screenshots posted above, may I ask do you use Managed Firewall Rules (WAF)?

May I ask have you tried creating a Firewall Rule with the action allow for the User-agent that contains Yandex at setting it to 1st from above on the Firewall Rules list?

Example:

Or, creating a Firewall Rule where the cf.client.bot field is set to true if so?

Furthermore, Yandex is being listed (with some others too) on a good bots list as far as here:

1 Like

Hi fritexvz

Thank you very much for your Helpfull Tips.

I will wait some days hoping that Cloudflare will fix this problem on their own.

I just checked the Firewall Logs and see now only less than < 10 blocks for today
for the Yandex Crawler.

Hope this Blocks soon will disapear all.

If not i may come back to your Helpful Step by Step Instruction
to solve this Issue.

Marked your Reply as Solution even i think this need to be solved by Cloudflare itself
as this should be not this way.

1 Like

Looking at WAF 100203, that’s for blocking fake Yandex bots, so something is out of whack.

You could just toggle it to Simulate for now just to see what happens in the firewall log.

2 Likes

Hi sdayman.

Thanks a lot for your helpfull Reply.

Toggeling this WAF temporary to Simulate is probably the easier solution.
Marked it too as solution.

Just checked the Firewall Logs and dont see for the last 12 Hours anymore
the blocking of the yandex bot.

Could be that the problem maybe got allready fixed by cloudflare itself.

Thanks a lot to anyone for the Help.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.