How to block traffic from Amazon AWS servers?

Hello,
Since a few days I am getting suspicious traffic from Amazon AWS servers I need to block.
I have Statcounter installed and it shows only the IP address and the hostname, all other fields are “Unknown” for these instances.

I have tried to match the timestamps and the IP addresses from Statcounter with the WAF logs in my Cloudflare account. However I can not find any of the Amazon AWS IP addresses in Cloudflare??

Does this mean they are hitting the server directly, so bypassing Cloudflare altogether??
Or is there something I am missing here?

Is the only option to block them to add firewall rules in the server firewall?

If the requests made it to your origin through Cloudflare, then they won’t show in the security log as they were permitted, not challenged or blocked.

Check…

  • the DNS record in Cloudflare set to “proxied” for the hostname the traffic is hitting
  • you have blocked all external HTTP traffic, apart from Cloudflare IP addresses, at your firewall

Doing those 2 things ensure traffic must come from Cloudflare and not direct to your origin.

We challenge AS16509 on Cloudflare for those sort of Amazon-hosted bots. If you use any external services that need to access your site, you may need to allowlist though those if they run on AWS.
https://radar.cloudflare.com/as16509

@sjr , I checked the “skipped” log in Cloudflare and these IP’s are also NOT in the not challenged log.

According to Statcounter support this “traffic” from Amazon AWS are bots and are NOT from a web browser, which is the reason statcounter can not display the actual url’s on my website that were hit. So I can not see which hostname they are hitting in the Statcounter log.

How do I block all EXTERNAL http traffic, apart from Cloudflare IP’s in the firewall?

Hello, please ignore the part about how to block ALL external traffic except cloudflare IP’s :slight_smile:

1 Like

Have you confirmed if the traffic is actually coming through Cloudflare (is the hostname proxied)? (You can give the web site if you want it checked).

If yes, have you set a WAF rule to actually challenge them?

@sjr , The hostname is proxied, checked in Cloudflare DNS settings.
All other traffic is hitting a url with the proxied hostname and urls belonging to it.

In the server raw log files I see many other bots and normal user traffic, most are coming from the usual clouflare IPS’, a very small percentage is hitting the website directly and those are usually the hacking attempts/port-scanners/wp-attacks we see every day.

But the amazon bot IP’s can not be found in ANY of the Cloudflare WAF logs, not in blocked, not in skipped…I can not even find them in the RAW server log files???
This means it’s not even http/s traffic then???

Are you restoring visitor IPs?
If not, and you see AWS IPs in your server logs (or any non-Cloudflare IP addresses that aren’t your own), then the requests are direct to your origin and not via Cloudflare.

I’m still not sure if you have any WAF rules set to block or challenge these requests. If you don’t have any rules for that traffic, then there won’t be any record in the security log as the traffic will be passed through (if it is going via Cloudflare and not direct to your origin).

1 Like

@sjr , I have many WAF rules based on AS number, IP ranges, Country, User Agent…etc…etc…these are set to Block.

And I have ONE allow rule containing two IP’s (for Admins) and the Known Bots, and these are set to Skip…should I use a different option like Challenged?

All other traffic is of course allowed, but set at high challenged.

I am not restoring visitor IP’s. Therefore I am seeing 99% cloudflare IP’s in the cloudflare log and 1% I see IP’s that are from different sources like mostly IP’s that are listed as malicious at abuseipdb.com
So I was assuming always those non cloudflare IP’s were hitting the server directly?

But Statcounter somehow has a way of displaying the actual visitor IP address instead of the cloudflare IP’s for all websites that are using Cloudflare as their CDN. And it’s in Statcounter I can ONLY see the actual IP of the amazon bots…nowhere else, not in Cloudflare and not even in the server log files.

I would rather not venture into installing packages for restoring visitor IP’s at the server level…for now.

If so, then they are probably looking at the CF-Connecting-IP header. Assuming they are (and I don’t know as I don’t use Statcounter) then the traffic is coming via Cloudflare. Therefore it’s not being blocked by your WAF rules on Cloudflare and your server logs will be hiding the Amazon bots inside Cloudflare IPs. It will only align and make sense if you restore visitor IPs so your server logs show the true visitor and not Cloudflare.

If you use Apache or Nginx, it will probably work out of the box, or you just need to enable a module (and modify the configuration files).

@sjr , I use Nginx and can implement the restore visitor ip packages.
Will this display ONLY the actual visitor IP after I implementing the restore visitor IP function or will it display both the cloudflare and actual visitor IP for any given visitor?

I use Apache and only bother with the real visitor IP (we set up to only receive traffic via Cloudflare) but I think I’ve seen for at least Nginx you can log both. There is a number of posts on here from people doing it for Nginx you can search for.

1 Like

@sjr , THANK YOU Sir for your help. This is MUCH appreciated!

2 Likes

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.