GSC fails to access several URLs. Cloudflare responds with 403

What is the name of the domain?

What is the error number?

403

What is the error message?

403 access forbidden

What is the issue you’re encountering

GSC reports blocked due to 4xx issue

What steps have you taken to resolve the issue?

It works with a WAF bypass rule.

Was the site working with SSL prior to adding it to Cloudflare?

Yes

What is the current SSL/TLS setting?

Full

What are the steps to reproduce the issue?

Somehow Google Search Console has identified these URLs in their index, but is unable to crawl them because Cloudflare responds with a 403 error.

First one:
http://www.linuxsecurity.com/advisories/other_advisory-2137.html/",/"source/":/"af854a3a-2127-422b-91ae-364da2661108/",/"tags/":[/"Third

Second one:
https://linuxsecurity.com/component/socialads/track/redirect/5?calt='LinuxSecurity Advertiser'ype=1&widget=459

GSC keeps failing to validate some of our URLs with junk appended to the end. These are redirecting correctly when I try, but somehow GSC keeps rejecting them as 403. In many cases, I’m not even seeing these entries in our apache logs. Why do they keep failing? How do I troubleshoot this?

May I ask if you’re using Free or Paid plan type?

I’d suggest you to double-check the Security → Events at Cloudflare dashboard under your Cloudflare account for your zone, or via direct link https://dash.cloudflare.com/?to=/:account/:zone/security/events.

You should be able to see the challenged or blocked event under the Security tab → Events at Cloudflare dashboard for your zone and know exactly which security option was triggered. You can use Filter and URI path contains component, otherwise use RayID equals the Cloudflare Ray ID from the bottom of the page (as on the shared screenshot is visible).

Once you find them, click on a particular one to find more details about it (user-agent, IP, HTTP version …). If yes, could you share some details which service was triggered that blocked you?

I found a WAF rule that was blocking quote characters. It was quite a process of tracing the access to identify the cause, though. It’s apparently only possible to view a single day-in-time in the logs, even with the Pro plan.

2 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.