How can I block 2a06:98c0:3600::103 at WAF level

[stan5]

One of my websites is being (sort of) DDoS-ed by thousands of request per minute coming from “2a06:98c0:3600::103” and a User-Agent equal to that of Google’s bot. I can’t block it based on User-Agent due to that.

Also the website is Proxied behind Cloudflare. So I can’t use block it through iptables or routing to the blackhole.

I tried blocking it by WAF rules matching the IP. e.g. (ip.src eq 2a06:98c0:3600::103) or (http.x_forwarded_for eq "2a06:98c0:3600::103")

Unfortunately, that’s not working. My server is still receiving all the abusive traffic. Any idea how I could block all traffic from that IP?

[epic.network]

That IP is in the Cloudflare proxy range 2a06:98c0::/29. Are you restoring visitor IPs?

Restoring original visitor IPs · Cloudflare Support docs

[stan5]

Yes, I’m at Nginx level (using set_real_ip_from). E.g. set_real_ip_from 2a06:98c0::/29;

All requests come with this User Agent: (Linux; Android 6.0.1; Nexus 5X Build/MMB29P). If I filter the logs by that UserAgent, I can see traffic from Google’s IPs.

[ncano]

@stan5

2a06:98c0:3600::103

This IP is used by outbound fetch requests from Workers. You can block these requests by leveraging cf.worker.upstream_zone dynamic field in WAF.

[stan5]

Thanks @ncano!

I’ve tried the following rule (ip.src eq 2a06:98c0:3600::103) or (http.x_forwarded_for eq "2a06:98c0:3600::103") or (cf.worker.upstream_zone ne "") - i.e. catch all traffic that’s coming from a worker.upstream_zone; however, that has no effect. The aggressive crawler (or whatsoever) is still hitting hard my website.

Is there a more specific rule that I can use to block all traffic from workers (given that I’m not using them)?

[stan5]

Hey @ncano, could you please be more specific how I can block that ip by using cf.worker.upstream_zone. As I shared earlier, I tried with cf.worker.upstream_zone ne "" (with the idea to block ALL worker zones); however, that doesn’t have any effect.

[ncano]

@stan5

I’ve tried the following rule (ip.src eq 2a06:98c0:3600::103) or (http.x_forwarded_for eq "2a06:98c0:3600::103") or (cf.worker.upstream_zone ne "") - i.e. catch all traffic that’s coming from a worker.upstream_zone; however, that has no effect.

I would suggest ditching ip.src and all the other conditions and block based on a filter similar to this instead:
(cf.worker.upstream_zone != "" or cf.worker.upstream_zone != "[MYZONENAME.COM]")

[kb13]

Got a site with the same exact symptoms: same IP address, same Android Nexus user agent, same claim that it is GoogleBot. And it is very aggressive. It sometimes accounts for 20% of the traffic on the site.

The last rule by @ncano didn’t work.

What is the value that should go instead of [MYZONENAME.COM]
This site does not have any custom CF workers.

[ncano]

Could you send us the screenshot of the rule that doesn’t work please?
The value should be the domain associated with your zone, i.e. example.com.

[kb13]

Thanks for clarifying what the zone is.
Why not just call it the domain name, so there is no confusion to begin with.

Here is a screenshot of the rule.
I didn’t enable it yet. Will try it out when traffic allows.

[ncano]

@kb13

Why not just call it the domain name, so there is no confusion to begin with.

Great question. In a nutshell, zone can be a subdomain or assume an ownership of multiple domains, so in some situations the word “domain” doesn’t describe this concept well enough.

@kb13

I didn’t enable it yet. Will try it out when traffic allows.

Let us know if you run into something unexpected.

[kb13]

I tested the change, which I provided a screenshot of (with the correct domain name of course), and it does not work.

It blocks all traffic to the site, and not just that pesky IPv6 2a06…103.

[kb13]

Is there any definitive solution for this annoying crawler?

I’ve tried the proposed solution, but it blocks all traffic to the site.
So it is not a solution at all.

Cloudflare should not allow such a gaping hole in their service.

[DarkDeviL]

@stan5

Is there a more specific rule that I can use to block all traffic from workers (given that I’m not using them)?

If you do NOT require any Cloudflare Workers to send HTTP requests to your site, use the expression:

cf.worker.upstream_zone ne ""

If you need Cloudflare Workers, from your own domain / zone, e.g. “example.com”, to be able to access your site, use the expression:

cf.worker.upstream_zone ne "" and cf.worker.upstream_zone ne "example.com"

If you need Cloudflare Workers, from all three different zones, e.g. “example.com”, “example.net” and “example.org”, to be able to access your site, use the expression:

cf.worker.upstream_zone ne "" and not cf.worker.upstream_zone in {"example.com" "example.net" "example.org"}

Using one of these expressions as your WAF rule, with the action BLOCK should be working to block undesired access from Cloudflare Workers (or, at least, those that are not your own).

@kb13

It blocks all traffic to the site,

Seems to me like @ncano accidentally shared the expression with:

== "" or

But actually meant to say:

!= "" and

[kb13]

Thanks for the extra info.

Since we don’t have any Cloudflare workers, I tried the simplest rule, but it didn’t do anything. The bot is still getting through …

Here is a log entry, and there are thousands more like it to various URLs.

2a06:98c0:3600::103 - - [16/Jun/2024:14:41:26 -0400] “GET /foo/bar HTTP/2.0” 418 116 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

Note that the 418 return code is from custom code we added to the application, to detect that IP address, and return a “I am a teapot” error. That way, we can parse the logs for that specific pesky crawler.

[kb13]

I ended up with a workaround that is somewhat better than having the application deal with that IP address.

The solution is to block the IP address in Nginx

location / {
    # Deny CloudFlare IPv6 worker used by bots
    deny 2a06:98c0:3600::103;

    proxy_pass                         http://127.0.0.1:8080;
 ...
}

This has the advantage of not having to invoke dynamic code (PHP) at all.

This is still a workaround though, and the proper solution is to have Cloudflare not allowing requests from these bots to come in to begin with.

[huudienkhue.le]

@stan5

One of my websites is being (sort of) DDoS-ed by thousands of request per minute coming from “2a06:98c0:3600::103” and a User-Agent equal to that of Google’s bot. I can’t block it based on User-Agent due to that.

Hi @stan5. Could you please tell me what method you used to find that IP address? Thanks.

[kb13]

In my case, it was analyzing the access log from your web server (Nginx in my case), when load is high.

There are tools that help with that, including Awstats (web based output), or goaccess (command line).

[kb13]

Even with my nginx blocking that annoying IPv6 address, there are still spikes when it hits the site hard.

By hard hitting, I mean 13,000 requests per hour, in quick succession, amounting for ~ 16% of the traffic.

Any solution?

Hey Cloudflare! you are enabling bad actors. Find a solution …

[DarkDeviL]

→ https://dash.cloudflare.com/?to=/:account/:zone/security/waf/custom-rules

Click “[Deploy]”.

→ Test it using Cloudflare Workers Playground

Add the code:

		if (url.pathname === "/blah") {
			return fetch("https://example.com");
		}

Enter"/blah" in the long (path) field, and push “[Send]”, and you’re seeing this result:

→ https://dash.cloudflare.com/?to=/:account/:zone/dns/security/events

Considering the above test, which was done through Cloudflare Workers Playground, it seems to me like the rule(s) are working perfectly fine, in order to block Workers.

Therefore, I’m getting further, towards another question:

@kb13

Since we don’t have any Cloudflare workers, I tried the simplest rule, but it didn’t do anything. The bot is still getting through …

Are you able to share all of your rules, including the order of those?