I think Cloudflare is blocking my sitemap.xml file

  • I have 2 servers - Development and Production.
  • Development DNS isn’t setup in Cloudflare.
  • Production DNS is configured entirely in Cloudflare.
  • Development Sitemap.xml file loads without issue, and is clearly visible and formatted correctly.
  • Production Sitemap.xml file will not loaded at all, the page just freezes up.

I’ve tried debugging both Development and Production versions of the Sitemap.xml file, and there isn’t an issue with the formatting at all, however, the Production version of the Sitemap will not load and I’m wondering whether this is being blocked by Cloudflare (assumption based on the fact that the only difference between the 2 versions are that Production is running in Cloudflare, and Development isn’t).

How can I debug this?

Many thanks

Why don’t you temporarily pause Cloudflare to test that theory?

1 Like

Thanks for that excellent suggestion, I tried it and I can confirm that Cloudflare is indeed blocking the Sitemap.xml file from loading.

How do I prevent this from occurring?

Thanks

Have you examined your WAF activity log when the request is blocked?

There are 2 security events as below.

Are these the firewall events you mean?

What steps can I take with these?

Generally, yes. That is what I was referring to. You can expand the entries to view more detail. You would want to check for any entries that show your sitemap.xml was blocked and review the reason. You can then make adjustments to avoid the block.

There are only 2 events, and none of them are to do with the sitemap.xml file:

Looking at output from testing with curl it becomes apparent that this is not the WAF, but a 502 error on your origin.

I don’t why your server is having trouble with that file when it is requested via the Cloudflare proxy. While it loaded for me during a couple of attempts via web browser, the page acted like it was still loading even after the sitemap had loaded. My final attempt ended in the fancy Error 502 page served by Cloudflare when it is sent a 502 response from the origin.

If you have access to your server logs, you might want to examine them as well to see if it is logging any errors when the sitemap.xml file is requested via Cloudflare proxy.

1 Like

Ok thanks, I’ll look at the server logs and report back.

1 Like

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.