Block Unknown Browser or Operating System in Firewall (WAF)? (Possibly related to AdSense click bomb)

Hi,

I have detected unusually high traffic from Unknown Browsers and Operating Systems through CloudFlare Analytics. Today for example, out of a total of 242K visits, 163K were from Unknown Browsers and Operating Systems.

After checking through my server logs, I have deemed this type of traffic suspicious and would like to block it with the CloudFlare Firewall (WAF). Unfortunately, I did not find a way to do so. I know you can block by User Agent, but that’s impossible since the User Agents are highly varied.

For the moment, I was able to intercept the traffic and issue a JS Challenge using the Referer. All of the traffic is coming with an empty Referer. But this also means I am issuing a JS Challenge to normal users, and I would like to prevent that as much as possible. Fortunately, most of my traffic comes with a referer, so this is only impacting a small percentage of users. The JS Challenge is currently blocking 96% of the bogus traffic, so it is working pretty well indeed.

This is the expression I am using for the Firewall rule:

(http.referer eq “” and not cf.client.bot)


For anyone interested or who might be experiencing the same issue, the main reason I want to block this traffic is because the anomaly coincides with an AdSense Confirmed Click penalty I’ve had since February 28 2021. After looking at my Analytics logs, I realized I was getting click bombed by seemingly regular traffic since November 2020. CTR has been over 130% for some URLs, with clicks exceeding the number of page views. Loss in revenue is over 80%, so for the past week, I’ve been digging pretty deep to try and find the source of the issue.

One important detail I did find while looking at my Analytics AdSense report: the invalid clicks are only affecting Android and not iOS, Windows or OSX. So the bots are probably exploiting a bug in Android to generate the clicks.

I believe the traffic anomaly might be related to the AdSense click bomb because most of it is not accessing a URL on my website. Only a small portion is accessing category pages, and the rest is accessing nothing. And if a bot was to click on AdSense URLs, it wouldn’t need to access a URL on my website. It would simply need to access the ad’s link and show that it is on my server. I know this might be far fetched, but I think it’s not impossible.

Thanks.

Edit: I have concluded that this was not the source of the invalid clicks. I have not yet found what is causing the invalid clicks issue.

Did you found an IP address, user-agnet or something else in your server log files if they have had “crawl” your Website?, by which you could create a Firewall Rule and block them by that criteria?

https://www.cloudflare.com/learning/bots/what-is-click-fraud/

Have you had enabled “Bot Fight Management” at Cloudflare Dashboard or if on paid plan, as far as I remember there is some Bot Management at Enterprise plan (I believe this is not your option)?

Thanks for sharing!

Any information deos AdSense filter out bot traffic/clicks and disregard that kind of bots? Have you tried contacting Google AdSense support too regarding this issue?

Hi @fritexvz,

I found the same patterns in my server logs as the ones I found on CloudFlare (Uknown Browsers and Operating Systems, same IPs, repeat URLs, etc.) But since the User Agents vary so much, I could not create a single rule to block them all. If I could block Unknown Browsers and Operating Systems, then I could block them all easily.

Yes I did, but it didn’t do any good. Actually, it didn’t detect anything out of the ordinary and did not generate any entries in the Firewall.

AdSense does usually filter this kind of traffic, but I suspect this looks so organic that it’s difficult for their systems to detect it. Please keep in mind that up to this point, the suspicious traffic hasn’t yet been confirmed as the source of the invalid clicks. I need to look at my Analytics / AdSense reports tomorrow to see if the CTR has gone down.

I have contacted 2 AdSense support agents who have simply replied with generic “Dos and don’ts” emails and suggested I fix my ad implementation. It is absolutely impossible that an ad implementation generates 130% CTR. I have also filled out 2 invalid click report forms with detailed information and server logs, but didn’t receive a response. Google has become so cluttered with AI that it’s often impossible to get in touch with a human. When your strength becomes your Achilles’ heel…

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.