Please add a firewall rule for Google Shopping

Hi,

Since 2020-02-26, Google Shopping has changed the User-Agent header used to retrieve our XML products streams from google-xrawler to Googlebot .

Now all requests are rejected by Cloudflare Firewall by rule ID 100201 (Anomaly:Header:User-Agent - Fake Google Bot).

Can you create a rule whitelisting Google Shopping IP range please?

Thank you.

Hi @jleroy, in addition to raising a support ticket, can you post here a screenshot of those firewall events including the User-Agent field and IP of the blocked request? Feel free to remove your zone’s name and rayId from it.

3 Likes

Hi @sandro, @mdemoura: I’ve already opened a support ticket and the agent asked me to open a feature request here…

Here’s a screenshot the the last blocked event:

If that is a legitimate Google bot, it doesn’t comply with Google’s own validation process documented at: https://support.google.com/webmasters/answer/1061943.

Specifically, it fails this check:

$ host 74.125.72.161
Host 161.72.125.74.in-addr.arpa. not found: 3(NXDOMAIN)

Is there a dashboard where you can see the bot fetching errors?

1 Like

Yes, my customer sees the error in its Google Merchant Center interface (in french, sorry):

Also the request come from an “internal” Google range (not GCP’s ones):

  • 74.125.72.160
  • 74.125.72.161
  • 74.125.72.162
  • 74.125.72.163
  • 74.125.191.160
  • 74.125.191.161
  • 74.125.191.162
  • 74.125.191.163
  • 74.125.191.164
1 Like

Thanks for the detailed information, I’ve raised a ticket in my team to address this. Could you send me your support ticket ID via direct message?

1 Like

@mdemoura I can’t see the direct message button on your profile, are your DM open?
Or maybe my account is too new to have access to this functionality ?

Anyway, here’s the ticket number: 1839209.

My DMs are open, not sure what issue could be.
We’re looking into the issue, thanks for the report.

This was addressed yesterday, please let me know if the issue persists. Thanks!

2 Likes

Hi @mdemoura,

I really appreciate your feedback!

Could you just confirm if Google bots have actually gone beyond verification standards?

Thank you in advance.

@dmz

$ host 74.125.191.161
Host 161.191.125.74.in-addr.arpa. not found: 3(NXDOMAIN)

Yes, I know, what I’d like to clarify is whether the Cloudflare Team contacted Google and they confirmed this undocumented behavior as legitimate.

Anyway, thanks for the reply. :slight_smile:

When will get a method? I met the same situation, but no firewall record.

Everything is working fine, thank you!

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.