Nonstandard attack

Hello. The attacker registered the domain. Next, he added his domain to Cloudflare protection. Added an ip from which he will attack my site in the white list in his account. Requests come from CF IP addresses to the real IP address of my server. Not through my account on CF system. I tried to change all A-records, and all traffic, except for the malicious one, stopped.

Below is a piece of the log, when all A records in my account are not directed to my server, the intruder’s requests from CF addresses remained. What to do about it?

51.161.43.237 - 172.70.110.207 [10/Oct/2021:15:12:25 +0300] [1633867945.677] [0.030] “POST /a/add_news_subscribe/ HTTP/1.1” 200 66 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

51.161.43.237 - 172.70.110.207 [10/Oct/2021:15:12:26 +0300] [1633867946.157] [0.032] “POST /a/add_news_subscribe/ HTTP/1.1” 200 66 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

51.161.43.237 - 173.245.52.222 [10/Oct/2021:15:12:27 +0300] [1633867947.739] [0.037] “POST /a/add_news_subscribe/ HTTP/1.1” 200 66 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

Maybe in the CF I can specify some kind of secret key in the header. So that requests can only come through my settings? And specify it in the nginx config.

Could you send the HTTP headers of an incoming request? It’ll let us identify where those requests are coming from.

If attacker knows your real IP address, you’re already outside of CF proxied protection. You need to safe guard that real server IP and if it’s leaked, need to plug all the leaks and then change that real IP address for a new one.

You can also use Cloudflare Authenticated Origin Pull certificates can either use

  1. default Cloudflare provided TLS client certificates which won’t help if attack is using CF account themselves so already using CF default CA cert for verification or
  2. use your own custom created TLS client certificates which are signed by your own custom CA SSL certificate/intermediate rather than use CF’s default CA

See for zone level custom TLS client CF Authenticated Origin Pull https://developers.cloudflare.com/ssl/origin-configuration/authenticated-origin-pull#zone-level--customer-certificates

Or for per hostname custom TLS client CF Authenticated Origin Pull https://developers.cloudflare.com/ssl/origin-configuration/authenticated-origin-pull#per-hostname--customer-certificates.

I do this with custom created CA cert and Intermediate which signs custom TLS client certificate and using CF API upload the custom TLS client certificate to Cloudflare so that Cloudflare requests to your origin present the custom TLS client certificate to your origin and your origin server then verifies the custom TLS client certificate using the custom created CA certificate which only you and no other CF customer would have access to.

The example I scripted using CF cfssl tool at GitHub - centminmod/cfssl-ca-ssl and example at browser level GitHub - centminmod/cfssl-ca-ssl without Cloudflare - just the example adding client TLS cert to browser would be equivalent of uploading via CF API your custom client TLS cert and CF edge server requests would be equivalent to browser requests to your origin :slight_smile:

The biggest issue I see here is that it looks like your server is responding to requests to some other hostname. The server’s config for your domain should only be listening for your specific hostname(s).

1 Like

Also make sure you restored real visitors IP on your server too https://support.cloudflare.com/hc/en-us/articles/200170786-Restoring-original-visitor-IPs

It looks like it, as the IP address in the log is OVH.

I think what OP said is they re-pointed their “A” records elsewhere, yet they were still getting traffic, which must have been coming through that other account.

1 Like

Yes thank you. I meant that when we re-pointed “A” records elsewhere, malicious traffic continues from the CF IP addresses, but the traffic of real visitors is gone. And the log shows that already restored real visitors IP. The attacker attacks through CF, because in my nginx config allow traffic only from CF IP addresses. We have already changed the IP address of the server, and the attacker found it out anyway. Maybe it’s a hole in the defense that allows you to attack? Because I do not see requests directly, only from CF addresses. And, in fact, why CF does it pass bad traffic?

51.161.43.237 - real visitor IP
172.70.110.207 - CF IP

51.161.43.237 - 172.70.110.207 [10/Oct/2021:15:12:25 +0300] [1633867945.677] [0.030] “POST /a/add_news_subscribe/ HTTP/1.1” 200 66 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

Do you have server_name <your domain>; in your NGINX config? Or are you doing catch-all?

1 Like

filled in like this:
server_name firstdomain.com seconddomain.com *.firstdomain.com *.seconddomain.com;

Could you send the HTTP headers from one of the attacker’s requests?

Before you enable Authenticated Origin Pull as detailed by @eva2000, you should capture some logs with the hostname. ( include $http_host in your log_format directive)

You can then file an abuse complaint at https://abuse.cloudflare.com/. It might be accidental (somebody entered a typo in their own DNS config), but I would expect this to stop very quickly if they are using CF proxy to attack your origin.

2 Likes

Let me see if I understand. Somebody found your backend IP and is using another domain under CF that points to your backend?
That’s an interesting attack vector, I think that you can take the following actions.

  1. Deny requests coming from an invalid host header
  2. Use Authenticated origin pulls https://developers.cloudflare.com/ssl/origin-configuration/authenticated-origin-pull

The two approaches will drain some resources from your backend, however, rejecting HTTP connections is typically not expensive and can be withstood as soon as you have some computing power.

I’m not 100% sure but I think that tunneling the web app through argo could help to fix the issue.

I believe that @eva2000 owns that website so maybe he has further input on the case.

Finally, report the domains that are doing this to abuse cloudflare dot com.

Report his domain using this link (https://abuse.cloudflare.com/) this way his site can be taken down BEFORE he can execute the attack! However, while the report is being processed you can raise your security level to “High” (if you are currently NOT under attack)! However, if you are currently under attack, activate IUAM (I’m Under Attack Mode) and follow these tips (using this link, https://support.cloudflare.com/hc/en-us/articles/200170196-Responding-to-DDoS-attacks). If you are not under attack, you might want to follow these tips using the following link (https://support.cloudflare.com/hc/en-us/articles/200170166-Best-Practices-DDoS-preventative-measures)

1 Like

Good idea :smiley:

Would need custom TLS client certs with CF Authenticated origin pulls as standard default won’t work if attack is behind CF too Nonstandard attack - #4 by eva2000

2 Likes

In this case, that will have no impact. The traffic is not coming through the OPs Cloudflare account, but through somebody else’s account.

1 Like

If I may add here, this reminds me on a situation where the 2nd IP (Cloudflare) could be the thing about having 1.1.1.1/1.0.0.1 configured as a “DNS resolve” (nameserver or custom DNS servers) to make POST requests from the attacker’s OVH server obviously as an User-agent known as Googlebot (bad one in this situation).

I see the Cloudflare IPs (as a host, not my actual IP) when making each of my requests.

Why not putting some 1+2=? (image) as a captcha (or actually one) on the form (if it is a form) or an JSON request, limited by the count per minute (just in case)?

  • try for the start blocking the OVH IP, at least … or some combination User-agent Gooblebot + not Google AS number, etc. just quick suggestions …

As an example, here is an online tool - my actual IP + Cloudflare host IP (the same what you see):

1 Like