Web Traffic Analytics and Firewall sections - various improvement suggestions

I’ve been using Cloudflare for a couple of years on a website which averages around 750k pageviews per month.

Recently we’ve been facing probably the most common problem there is - spam and bot traffic. It has been rather frustrating attempting to use a combination of Cloudflare’s Analytics and Firewall sections to diagnose and curb this traffic and I’d like to explain why.

Firstly, the Analytics section. On the surface it looks OK - you can see traffic by country, browser, operating system, etc. which can actually be useful in diagnosing where a problem is coming from. For example, recently I was able to see a huge spike in requests from Russia (not a usual market for us) and many requests with an operating system and/or browser of “Unknown”.

However that’s where the usefulness ends. In the Analytics section you can’t actually do anything with this data; it’s just nice to look at. This section should be tightly integrated with the Firewall section with a button which essentially says “Create firewall rule from this filter set”. That would actually be useful, but it doesn’t exist.

In my case, I know the traffic is coming from one or two specific IP addresses. But I didn’t get that information from Cloudflare’s Analytics section - I had to get it the old-fashioned way, trawling through Apache logs on my origin server. If the Analytics section allowed me to group and filter by IP address, I could have immediately seen the offending IPs sending thousands of requests - way more than any other “legit” IP address - and instantly firewalled them.

Moreover, if you do head over to the Firewall section and try to create a firewall rule to block such spam traffic, you can’t. Yes, the firewall rules let you filter by country (“Russia”), but there’s no option to filter by operating system or browser type. I don’t want to block an entire country - just requests from that country which also have an unknown OS and browser. So the rules here don’t correspond to what’s displayed in the Analytics section which is super frustrating.

On that note, it would be immensely useful if the Firewall section added a new firewall action of “Debug” or “Log” so requests matching a rule could be logged but not blocked or challenged. This would let you fine-tune a firewall rule before activating it to prevent false positives and disruption to legit visitors.

My point is I should be able to use the Analytics section to identify malicious patterns and immediately block them. Instead I can see malicious patterns but there’s no way to block them, even if I manually head to the Firewall section, since the data/filters used on each section are different.

In summary:

  • Analytics should allow displaying/grouping/counting/filtering the source IP address of traffic
  • Analytics should have a “Create firewall rule from this filter set” button
  • Firewall rule filters should match the filters shown in the Analytics section
  • Firewall needs a “debug/log” option so matching requests can be logged for inspection but not actually blocked or challenged (to prevent false positives)