A few days ago, my website completely dropped off Google’s search index. Search Console showed that Googlebot was blocked; the server was returning 403 errors.
I checked my robots.txt, disabled all plugins from my Wordpress site including the WAF (Wordfence), even restored my site from backup to an earlier date. But the problem persisted. Eventually, at the suggestion of my hosting provider, I disabled Cloudflare’s DNS proxy, and lo, the site is working again. Google can once again see my site.
So my question is: what happened? Why proxy suddenly began causing problems for my site and how to fix it?
May I just ask and share here, to double-check the Wordfence options if they’re correctly set due to the compatibility with Cloudflare proxy mode.
Make sure to correctly configure Wordfence to work with Cloudflare proxy to return the correct visitor IP address under the Global Options - > CF-Connecting-IP from the link below:
My best guess might be due to the Google IP / Bot being blocked as to some of your security settings configured at Cloudflare dashboard for your domain name could cause it.
Otherwise, if the robots.txt file isn’t blocking something, despite it’s looking good to me at least
