I just purchased a domain from Cloudflare and started creating a WordPress website just to see that google search console doesn’t fetch my sitemap. I’ve read and tried every solution offered in the other posts regarding this matter but it’s still not working. When I pause Cloudflare, everything works. So what should I do?
You should see the challenged/blocked firewall events in the firewall events if you navigate to the Cloudflare dashboard → Security → Overview and lookup for Firewall events for the past 24hours or so. Once you find them, click on a particular one to find more details about it (user-agent, IP, HTTP version …). If yes, could you share some details which service was triggered that blocked you? Could be Browser Integrity Check or some other 
- you should see your origin host/server IP out there and user-agent, etc.
It knows to happen due to the WordPress using HTTP/1.0 and empty user-agent, therefore while executing WP-Cron or some other related JSON/REST API request via plugin which triggers the WAF rules (as it should normally).
Hello and thanks for your reply!
There are 8 events over there, all from yesterday:
The only Browser integrity check is this:
What should I do next?
What happens if you temporary disable Browser Integrity Check feature?
Googlebot is on the “good” and “known” list:
Google is also on a known list of “good bots”, despite there are again, the “bad” requests from the Google AS number from which Googlebot has got official IPs published too:
Nevertheless, you could try to create a Firewall Rule to allow/bypass it, if that somehow happens.
What I use to allow only Googlebot to access the sitemap.xml and robots.txt, sharing my below post which contains an Firewall Expression and a screenshot of allowing only Googlebot + Bingbot to robots and sitemap using a Firewall Rule:
Helpful article to find out more about this feature:
Nevertheless, regarding 403 error:
Possible solutions:
- Creating a Firewall Rule with cf.client.bot “true” and action “allow”, make sure it is the 1st Firewall Rule from above.
- Creating a Firewall Rule with user-agent contains bing and action “allow”, make sure it is the 1st Firewall Rule from above.
- Risky one but … add Google AS number (AS15169) to “allow” in IP Access Rules.
I disabled Browser Integrity Check but nothing happened.
I created 3 rules as you told me:
- (cf.client.bot) Allow
- (http.user_agent contains “bing action”) Allow
- (ip.geoip.asnum eq 15169) Allow
Still, nothing happened but for the first rule, I got 4 new events, every time I tried to add the sitemap to the google search console. It looks like this but still cannot fetch the sitemap:
Hello, I have requested more troubleshooting information in the ticket you currently have open.
