Super Bot Fight Mode and sitemap.xml

Hi

I am on the 20$ plan and when I try to access https://www.edityouraudio.com/sitemap.xml I get a captcha page

When I try to get this page, I also get a 403

Is there a way to prevent that ?
Will Google manage to scrape this sitemap anyway ?

Thanks

i just removed the " Definitely automated" “challenge” to “allow” and I can access it again

How can I be considered “definitely automated”

It’s quite possible that the simple act of browsing that specific file makes you seem like a bot because (most) humans would never look at that file.

Thanks but still I had a 403 on the Google Search Console, which seems really bad

That’s definitely a bot.

1 Like

MM the bot that is supposed to make your website discoverable to the world

Google search console should thus be a white listed bot in cloudflare.

Is it possible to mark it as such ?

Super Bot Fight mode isn’t very forgiving. Many people who try to use end up turning it back off because there isn’t (yet) a way to add overrides in Firewall Rules.

Weel it seems that it is not useable then

1 Like

It isn’t, but no one at Cloudflare cares. Upgrade to Enterprise for $4,000+ a month then they care.

1 Like

Thanks for the info

I have a basic paid plan and have enabled SBFM and currently have no known problems. I can bring up my sitemap in Chrome browser without any issues or captchas.

I hope these teething troubles can be ironed out

2 Likes

Search crawlers like Googlebot are considered “Verified Bot”, and should not be blocked by Super Bot Fight Mode. Are you sure you don’t have other Firewall Rules that may be blocking or rate-limiting Google? If not then you may want to open a support ticket so Cloudflare support can investigate what’s blocking Google.

The Bot Report includes three traffic types:

  1. Likely automated traffic may have come from bad bots. We use heuristics, machine learning, and other techniques to spot these requests. In most cases, this traffic will hurt your site without providing anything useful in return.
  2. Likely human traffic is legitimate and important. Ideally, the vast majority of your traffic matches this type.
  3. Verified bot traffic comes from good bots on the Internet. We have verified search crawlers like Google and payment notification services like PayPal. Most users choose to allow this traffic.
1 Like

I have a basic paid plan and have enabled SBFM and currently have no known problems.

Have you checked the Firewall Events log to make sure?

Just did but i only have 72 hours left so can’t see

Maybe I’ll try again later then

Thanks for your help