We added the below firewall rule and our google shopping feed was working fine for few days. However since last week, our shopping feed is completely offline. We are getting an error - ‘Authentication failed’.
Does anyone know the solution to this?
@leighton this can happen sometimes when the Known Bots list gets updated. You can exclude an IP directly if you run into issues, or wait until the new IP is added.
You can also make an exclusion based on Marketing feed User-agent if you want to unblock this function w/o the need to continue updating IPs if they change on Google’s side.
Had the same issue here. The weird thing is, the IP that Google crawls from for fetching the shopping feed, doesn’t validate as being a Googlebot IP (even though the user agent is Googlebot).
We had to add an extra condition to our firewall rule to say if URI matches /shoppingfeed.xml. After doing this, and refreshing our feed in the Merchant Centre, it corrected itself right away and Google could get access.
Fortunately, it seems that when they crawl the individual product pages and product images, they do use a valid Google IP. It’s only the crawler that fetches the feed which doesn’t validate according to Google’s own standards.
We could have whitelisted the IP address in question, but I suspect there is more than one.