Fake Google Bot

Thank you, but unfortunately none of this options work.

The first one doesn’t work, because I already used blocking by ASN.

Cloudflare shows about 700k requests from Googlebot in Analytics > Security, so I guess cf.client.bot is checked by useragent not by bot IP, hostname or ASN.

If you already blocked the network and still get requests from that network, these requests must come directly to your server and you should have a look at your server’s firewall instead.

The details are not public, but I am pretty sure that is somewhat IP block based and not user agent based.

1 Like

Logs on my server end with “X-Middleton/1”, so I think requests come from Cloudflare, because it is added by Apache CF Module.

Which Apache CF module? There is one module and it just rewrites IP addresses and doesnt add and such data. Also, that string does not seem very Cloudflare related, furthermore if requests pass through that module it, it will always be added.

The way you described it, the most likely explanation are direct requests. Whats the domain and would you feel comfortable sharing the server IP address?

Which Apache CF module? There is one module and it just rewrites IP addresses.
https://support.cloudflare.com/hc/en-us/articles/200170786-Restoring-original-visitor-IPs-Logging-visitor-IP-addresses-with-mod-cloudflare-

Requests aren’t direct because I’m now blocking about 700k of them by IP ranges:

That is the module I was referring to and that module does not add any such data.

Anyhow, if you can rule out direct requests and requests still hit your server you got the wrong ASN and should check that out.

In any case, the second expression should still block them, of course assuming they come with the user agent in question.

Which Apache CF module? There is one module and it just rewrites IP addresses and doesnt add and such data. Also, that string does not seem very Cloudflare related, furthermore if requests pass through that module it, it will always be added.

" X-Middleton/1" - my mistake, it is added by Ezoic CDN, but it means that requests arent direct.

Where do you take that from? That value appears to be appended regardless of where the request came from.

Again, if you can rule out direct requests you got the configuration wrong. But I am still pretty sure it is direct requests.

Once again, whats the domain and server IP?

In any case, the second expression should still block them, of course assuming they come with the user agent in question.

So just one rule? Can you post a screenshot of that rule? Also, try blocking instead.

For the fourth time, what is the domain?

I’m sorry, but I can’t share domain and IP.

In that case it is impossible for the community to say anything.

If you applied the expressions as I described the requests should be blocked. If they are not, either the configuration is wrong, or the requests do not match the configuration (different values), or the requests are direct.

I am afraid this is all that can be said at this point.

Hi @katarzynastarzewska, I’m an engineer on the Firewall/WAF team.

I noticed you purchased a paid plan, so you’ll have access to the WAF. There’s a rule that will block the fake Google bots that you’re seeing - ID 100201. Can you ensure that it is set to “Block/Drop”?

Let me know if that solved your problem! :smiley:

Why should either of the two mentioned rules not block it? Specifically the one referencing the bot flag?

On first glance the rules you’ve written at Fake Google Bot and their description appear to be correct.

That is precisely what I assumed. Hence I believe the issue is not with Cloudflare but rather one of the aforementioned reasons.

I’d recommend using the rule I provided. If the issue persists, @katarzynastarzewska, feel free to open a support ticket and we’ll take a look to ensure the issue solved.

If you use Ezoic, they have their own bots used for website speed tracking etc. You can either block them but the speed app will not work or make a rule to bypass them. However, what we get in user agent is “X-SiteSpeedApp-1” at the end.

Again, if you use Ezoic, turn it off and check if you see those entries. If not, then it’s clear. And for 99.9% this bot is relevant to Ezoic and does not have to be blocked.

We have also around 150k entries like this each day. Right now, we will keep it this way and if we would like to see other entries, we just exclude them from the overview.

Or, you can just switch off the WAF rule for fake bots. But then you open the door for others.