70% of all pages unindexed within a day or two after implementing Cloudflare

I was facing significant bot traffic (about 150 fake visits a day), so I decided to add Cloudflare to combat the bots and fake traffic. However, three days after implementation, more than 70% of my pages were unindexed by Google. I’m unsure if this is due to Cloudflare, or if the timing was just coincidental.

I considered the possibility of a new Google core update causing the unindexing, but my site is pure and clean, adhering strictly to Google’s policies, and I’m still searching for a solution.

Regarding Cloudflare, I hadn’t thought it might be an issue (and I’m still not sure it is) because I’m aware that Cloudflare has a allowlist for Googlebot, Bing bot, etc.

I’m uncertain if Google is indexing with a proxy IP or something that Cloudflare doesn’t recognize. I don’t want to misleadingly point to Cloudflare as the problem since it might be a coincidental timing issue, but I’m trying to find a solution regardless of the cause.

Google Search Console showed many Server error (5xx) messages and numerous “crawled but not indexed” errors. I requested reindexing, and some pages were reindexed (about 80–120 pages), but others failed. I resubmitted them, and now the status is ‘Pending’.

I’m wondering if I should stop ‘Bot Fight Mode’ in Cloudflare. Is it possible that this issue is caused by Cloudflare, and what should I do to resolve it, or might this not be an issue at all.

I’m wondering if I should stop ‘Bot Fight Mode’ in Cloudflare. Is it possible that this issue is caused by Cloudflare, and what should I do to resolve it, or might this not be an issue at all

Yes, Bot Fight Mode would prevent Google from crawling your site.

To make exceptions for Google, you would require the paid Super Bot Fight Mode which is available on the pro plan.

1 Like