The problem is that the rate limiting rule is not blocking the traffic and it is hammering the website to the point the website is inaccessible for the period when thousands upon thousands of requests are sent.
I have two tickets open with cloudflare but they are not responding, nor have they even provided an initial reply other than the canned response informing me the ticket was submitted and the ticket number.
This is a serious problem that is crippling my small business because I need to limit these callback hits from smtp2go so they don’t take my website offline and kill performance when they hammer the site.
I need them to transmit though because they are updating the results of the email campaign and show who opened email, whose clicked etc…
What must I do to limit how frequently the following url can be accessed so the rate limit rule actually works?
Are all the requests actually coming from the same IP address? Your rule can only limit to 5 requests per IP.
Note that the hostname field is for the hostname being connected to, not where the connection is coming from.
Rate limiting features very few in free accounts. Without the actual domain name it’s hard to test. You may find using a Worker to do rate limiting works better for your use case…
If you absolutely need the domain, I will have to backup the site and restore it to a different domain and url that I can provide to you because security is not an issue if we use a different domain/url with the same website for testing and debugging purposes.
Is it possible to somehow use that JSON you mentioned earlier in a rule like what is shown below?
What he wanted to do was test to see if he could trigger the rule. Since that makes things simpler and allows to test a number of possible scenarios.
Why not start with the simplest possible logical expression for your blocking rule. If it’s a single IP address and a single URI why are there multiple OR conditions?
For example if the hostname this request is going to isn’t smtp2go.supersecretdomain.com then the last condition in your first screenshot would never be met. The other 3 conditions all appear to be partial duplicates of one another.
Create a simple rule (e.g contains smtp2go) and then test the rule.
for i in `seq 1 20`; do curl -I http://whatever.com/smtp2go; done
I can take a backup of the site and restore it to a different domain and url that I can provide to you because security is not an issue if we use a different domain/url with the same website for testing and debugging purposes.
Then i can copy the rules, and we can manually push traffic to that url all day from the same IP address and watch the rule never trigger.
The pro plan does not allow me to create a rule that limits by IP.
I cannot enter (ip.source eq “45.79.140.145”) as the expression of the rate limiting rule.
If I make a rate limit rule using the domain only and 1 hit per 10 second period, the limiting rule blocks everything as expected EXCEPT THE THING I NEED TO BLOCK.
I cannot find the traffic that I need to rate limit, in the actual log/analysis info of the rate limiting rule. It looks like that traffic continues to spam the server from 45.79.140.145 by means of user agent SMTP2GO_webhook_sender/1.0 and it is hitting the server so hard there are nonstop http 500 errors.
What on God’s green earth is the problem here? I am banging my head against a wall.
It’s not necessary. That’s not what rate limiting by IP address means. It means any single Ip address making a request that exceeds the threshold. @sjr asked if the requests were all coming from the same IP address because a. he didn’t have a real URI to work with to test to see if your rule was configured correctly and b. there are websites that receive large numbers of requests per second because they are popular and tens of thousands of visitors could be accessing the website at the same time. In that scenario rate limiting wouldn’t apply because an individual request by a visitor wouldn’t exceed the threshold even though cumulatively the traffic would be very high.
We don’t know what we don’t know.
Not necessary. You have the command above to test the rule yourself and a suggestion on how to create a simple rule vs. the one currently in place.
If the domain name you’re using is smtp2go.supersecretdomain.com I swear to god I am going to rage quit the Internet for the day. If that’s the case then instead of a rule blocking smtp2go use smtp2goWH. Again, we are trying to use the simplest possible expression at this point.
If instead of blocking requests that contain www.redacteddomain.com/wp/wp-content/plugins/nuevomailer-wp/admin/_smtp2GoWH.php?a5b7bda4bbb7a2b0b6b9929dabb793afb8b4bb9f74b6a4a8 it blocked www.redacteddomain.com/wp/wp-content/plugins/nuevomailer-wp/admin/_smtp2GoWH.php?a5b7bda4bbb7a2b0b6b9929dabb793afb8b4bb9f74b6a4a (note the last letter was not considered in the second query) would that be too broad?
What is the smallest possible match of the URL that is acceptable to block on? Use that for your simple test.
If I make a rate limit rule using the domain only and 1 hit per 10 second period, the limiting rule blocks everything as expected EXCEPT THE TRAFFIC I NEED TO RATE LIMIT.
I cannot find the traffic that I need to rate limit, in the actual log/analysis info of the rate limiting rule. It looks like that traffic continues to spam the server from 45.79.140.145 by means of user agent SMTP2GO_webhook_sender/1.0 and it is hitting the server so hard there are nonstop http 500 errors
What on God’s green earth is the problem here? I am banging my head against a wall.
If I make a rate limit rule using the domain only and 1 hit per 10 second period, the limiting rule blocks everything as expected EXCEPT THE TRAFFIC I NEED TO RATE LIMIT.
I cannot find the traffic that I need to rate limit, in the actual log/analysis info of the rate limiting rule. It looks like that traffic continues to spam the server from 45.79.140.145 by means of user agent SMTP2GO_webhook_sender/1.0 and it is hitting the server so hard there are nonstop http 500 errors
What on God’s green earth is the problem here? I am banging my head against a wall. Never seen anything like this. The IP is not in any allow list. There is a skip rule but it excludes rate limiting rules, so the expectation is this SMTP2GO_webhook_sender/1.0 from IP 45.79.140.145 is going to be rate limited and not be able to hit the server and spam generate http 500 errors…
If I don’t have the skip rule, the traffic never hits the server due to other security rules in place, so I have a skip to bypass those, but not the rate limit rule as shown below.
Yet, all traffic from from 45.79.140.145 by means of user agent SMTP2GO_webhook_sender/1.0 continues to spam the server generating http 500 errors, totally ignoring the rate limiting rule.
Why would that be your rule when I suggest creating a rule that contained smtp2go?
Starts with <> contains. https://website-i-need-to-prevent-from-being-pummeled.com <> smtp2go
…
You’ve shown a screenshot of a 500 error from the origin. That doesn’t mean the rule isn’t working, that means your origin returned a 500 error code. Seriously at this point I have to think you’re just messing with me for the fun of it.
run this command from the shell… what happens?
`for i in `seq 1 20`; do curl -I https://www.redacteddomain.com/wp/wp-content/plugins/nuevomailer-wp/admin/_smtp2GoWH.php?a5b7bda4bbb7a2b0b6b9929dabb793afb8b4bb9f74b6a4a8; done
I created the cron job you suggested to run every minute, but it doesn’t appear in cloudflare anywhere because it never leaves the server, to go out to the outside to come in via cloudflare.
I suggest you create a rate limiting rule that was simply contains smtp2go you created a rule that is (starts_with(http.request.full_uri, “https://website-i-need-to-prevent-from-being-pummeled.com”))
Contains is not the same as starts with. Nor is smtpgo the same as the full URI. My suggestion is to create a very simple rule for test purposes that meets the general criteria. I find it very unlikely that smtpgo is showing up in a lot of other requests on the site.
So a single rule using URI contains as the criteria.
The command I asked you to run should be run on your local machine from a terminal window so you can see the output. After nn requests the http response should change from 200 to a 421 (I believe).
If you run this command from the command line as written you will see a series of 404 errors returned for the requests and after 10 you will see the request change to 421 which are rate limited requests.
for i in `seq 1 20`; do curl -I https://ek4m.com/smtp2go; done
