After a DDoS Attack, Cloudflare blocks some requests with 403

An HTTP Flood DDoS Attack of over 1M requests came in today, we received an email letting us know that it was being handled by an automated mitigation rule.

Soon after, our clients reported that images hosted by us and API calls made to us were being denied with error 403, even though the mitigation rule blocks “HTTP requests with unusual HTTP headers or URI path”. The requests made to the domain do not have unusual headers, and no change in path. Everything was working fine before the attack, this is new.

Now, after the attack is over, the images and API calls fail and we need them back up and running like before. We cannot manually disable the mitigation rule since it is a read-only rule, and we have no other active rules of our own that could be blocking requests from other domains.

Is the 403 a block or a JS challenge page?

Check if Under Attack Mode has been enabled here and switch to “High” security instead if it is…
https://dash.cloudflare.com/?to=/:account/:zone/security/settings

Make a request to the API/images and check the reason for any challenge/block in your security events log here…
https://dash.cloudflare.com/?to=/:account/:zone/security/events

1 Like

It is a 403 block. Under Attack Mode has been off and set to High for 10 months, this is a new case. The requests do not show in Events, even though the subdomain is under Cloudflare proxy. The request only passes without a 403 when referrer is removed from the request.

For example, a request similar to this would be blocked with 403:

curl "https://responsedomain.com/image.jpg"; ^
-H “accept: image/avif,image/webp,image/apng,image/svg+xml,image/, /*;q=0.8” ^
-H “accept-language: en-US,en;q=0.9” ^
-H “priority: i” ^
-H "referer: https://requestdomain.com/"; ^
-H ^“sec-ch-ua: ^^“Chromium^^”;v=^^“124^^”, ^^“Microsoft Edge^^”;v=^^“124^^”, ^^“Not-A.Brand^^”;v=^^“99^^”^” ^
-H “sec-ch-ua-mobile: ?0” ^
-H ^“sec-ch-ua-platform: ^^“Windows^^”^” ^
-H “sec-fetch-dest: image” ^
-H “sec-fetch-mode: no-cors” ^
-H “sec-fetch-site: cross-site” ^
-H “user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0”

But a request similar to this, without the referer, would be fine:

curl "https://responsedomain.com/image.jpg"; ^
-H “accept: image/avif,image/webp,image/apng,image/svg+xml,image/, /*;q=0.8” ^
-H “accept-language: en-US,en;q=0.9” ^
-H “priority: i” ^
-H ^“sec-ch-ua: ^^“Chromium^^”;v=^^“124^^”, ^^“Microsoft Edge^^”;v=^^“124^^”, ^^“Not-A.Brand^^”;v=^^“99^^”^” ^
-H “sec-ch-ua-mobile: ?0” ^
-H ^“sec-ch-ua-platform: ^^“Windows^^”^” ^
-H “sec-fetch-dest: image” ^
-H “sec-fetch-mode: no-cors” ^
-H “sec-fetch-site: cross-site” ^
-H “user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0”

We have deployed the attached Modify Request Header rule which simply removes the “referer” header. After deploying this, the requests succeed the same as before the attack. We would like this to be a temporary solution, we would prefer to be able to see the request referer for future requests.

Another note, when adding any header with referer in it, it fails.

Can you give a real request that gets blocked?

1 Like

Sure, I’ll disable the rule for a bit. Below is a request that that is blocked with a 403, with the “referer” enabled.

This is with the rule enabled, the “referer” is removed by Cloudflare (but still shown on client-side, rules cannot change that) before sending to Origin. A 301 is expected, this is being sent by our origin server.

Seems to only be happening in browser, trying to use cURL even with referer it loads ok, so can only see the error code in the console and there’s no body text that would indicate this is a Cloudflare challege or block page (and those links otherwise load ok).

Do you have any WAF rules?

Will need to poke some more, or someone else might see what’s happening before I get back.

1 Like

Yes we have WAF rules but we have had them all disabled.

Running cURL with the referer, I get a 403.

Without the referer, everything is fine.

If I change the referer to another domain, I get the the “Wait a moment” page from Cloudflare.

I’m now getting it (had to type by hand from your screenshot so not sure if what I did first time was correct).

Given the HTML that follows the header in my case, I don’t think this is due to Cloudflare. Cloudflare produces a “pretty” HTML block page with images and the like. Perhaps take a closer look at your origin server.

In your case, the “just a moment” indicates a Cloudflare challenge (followed by HTML and JS for the challenge page), you won’t be able to pass that in cURL as it’s checking for a human.

Need to be careful not to confuse all these variables. The challenge wouldn’t happen for a sub-request in a browser as the user would pass it.

Try setting the DNS record to “DNS only” in Cloudflare to bypass it and make the request direct to your server.

curl -i https://cms.dangoweb.com/clients/bluestone-hosting/storage/uploads//2024/04/15/abstract-science-fiction-futuristic-background-with-blue-neon-lights_uid_661d233f83a78.jpg -H "referer: https://bluestone-hosting.xyz/"
HTTP/2 403
date: Mon, 22 Apr 2024 18:47:59 GMT
content-type: text/html; charset=iso-8859-1
cf-cache-status: BYPASS
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=Wfqxf%2FD%2BTdvCNo%2BSr9n20Lfq1eU7OqWhMtlCgSFf0hcuagq1OJyXdwYeFDi0ry89FZGSlBlNVftAYY3MmWZeHdQ6l9zjrgmelnykTQl%2BMkAsVBq0LhRpa0HGFqVQZKHIiZ4QFStGZAxDmBKYBK8m"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
expect-ct: max-age=86400, enforce
referrer-policy: same-origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
server: cloudflare
cf-ray: 8787c914fb2c773e-LHR
alt-svc: h3=":443"; ma=86400

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access this resource.</p>
</body></html>
1 Like

Yes the 403 page is from our origin server, but there is no reason for it to show that. Why would removing the referer using Cloudflare be fixing the issue?

This all was not happening before the attack, and we need Cloudflare proxy on the subdomain for DDoS Protection and SSL.

Before the attack we had proxy on, and everything was working completely fine. We did not modify our configuration during or after the attack.

We contacted our origin server provider, they directed us to Cloudflare and told us that the origin server was not the reason. Their last message to us:

They’re probably blocking any request that has a referrer header that isn’t your domain. I cannot allowlist your client domains, this is not on our end. Please check with Cloudflare to see the next steps.

That doesn’t make sense if the 403 is coming from your server. If Cloudflare was blocking the request, it wouldn’t get to your server at all, the 403 would be returned from Cloudflare.

No idea, you’d have to find out why your origin is doing that in the logs there.

Just turn the proxy off for a short time while you debug it.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.