Enabling Cloudflare returns 301 code for crawlers (php headers reported)

Website, Application, Performance
1

Hello,

Accidently noticed Google started to slowly deindex my community blog and after some digging I was getting an error for “redirect error” while checking with browser and some other tools it was showing correct, I went to GSC Support Forum and someone was kind enough to point out it was a server error as it was being signed by “x-powered-by: PHP/8.1.24”
After checking a few other websites (as this website is hosted on a reseller account) that have Cloudflare integration I noticed this domain is the only one affected so I paused cloudflare and it worked again.

I have checked all settings I could think of and did not find any reason for the website to return a 301 with server headers, but apparently once it’s paused it’s getting indexed again!

I do want to mention that from an end-user perspective there was no difference, only crawlers had this problem as they would see the 301 redirect to homepage (not 200 to article/page).

Thank you!

2

What is the domain name/home page? Is it currently proxied by Cloudflare or is it still paused?

3

hi, domain is bulldog-francez.ro and it’s paused.

4

Can you unpause it for a while?

5

sure, it’s enabled now.

6

I’m getting “too many redirects” in Safari but not in Chrome or curl. Check your SSL/TLS settings are set to Full (strict), but I don’t think it’s that. Need to look at bit more. Do you have any rules set based on User-agent in your Cloudflare settings?

curl -I https://bulldog-francez.ro
HTTP/2 200
x-powered-by: PHP/8.2.11
last-modified: Mon, 20 Nov 2023 16:33:45 GMT
cache-control: max-age=0
expires: Mon, 20 Nov 2023 17:28:34 GMT
vary: User-Agent,Accept-Encoding
content-type: text/html; charset=UTF-8
date: Mon, 20 Nov 2023 17:28:34 GMT
server: Apache
1 Like
7

none that I can remember, this is a very old account, was set up like 2 years ago and forgot about it… I re-checked about ssl and set it to (strict) + these are the only page rules I could find

Screenshot 2023-11-20 at 19.33.39
Screenshot 2023-11-20 at 19.33.391043×297 25.2 KB

  • checked ssl, waf.
    I have “bot fight mode” enabled, but checked that already and it has no impact.
    I have also checked everything I could find under “rules” and nothing besides the page rules above.
8

Unrelated to your initial question, but important nonetheless, if you want that third page rule to have any effect, you need to move it up.

1 Like
10

I have done multiple tests, talked with the hosting provider to disable the securities they have (if any) and got the test to work, show a 200 code via allowlisting in cloudflare > security > waf > tools , but I cannot do allowlist for whole google network + it does not show in events! it’s like there is another cloudflare in front of my cloudflare panel.