Moderator question: Is there a way/place to create more question-like topics? The template looks more like it is about bug reports/tickets only, while I have more a question, while understanding the reason for the issue we see perfectly.
To the actual issue/question: Cloudflare can do (and by default does) URL normalization before applying its security rules, which makes sense, so they apply when a request would e.g. reach a path which shall be blocked, while the URL originally contains some encoded or relative back and forth path elements. However, there are cases where one wants to block both, when a URL contains a certain path element before normalization, as well as when it is contained after normalization. We regularly see such reaching our origin server:
AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/bin/sh)
AH10244: invalid URI path (/cgi-bin/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/bin/sh)
while we have this block rule in place
(http.request.uri.path contains "/cgi-bin/")
Reason is that the path is normalized before the rule applies, and as of the “parent directory” elements, the attacker tries to call /bin/sh via CGI, so that the /cgi-bin/ path element is not contained in the final path anymore. For the rule to apply, it would be applied before normalization, while we would want it to apply after normalization as well, to catch requests to e.g. /cgi-bin/sx.php.
Not a big issue, our webserver does not support CGI and blocks such paths as invalid in the first place. So it is more to reduce requests to and error logs at our server, and as it could be generally an interesting feature to match rules before and after normalization, if it does not exist yet.
Nice, I didn’t know that one. Probably an idea to add it to the dropdown menu of the expression builder?
I just updated the expression to
(http.request.uri.path contains "/cgi-bin/") or (raw.http.request.uri.path contains "/cgi-bin/")
Sadly I cannot really test it with curl and browser, resp. cannot find out how. Either I get a 400 Bad Request from Cloudflare (but no security analytics log entry), or I get a 404 from the origin for /bin/sh path, when using the two URI paths from our logs above as is with curl or in browser. Probably those clients do a normalization already before sending the request. I’ll check logs at Cloudflare and server later whether/where I see them, which happened several times a day before.
Hmm, more such log entries appeared on our origin server instead of Cloudflare security logs, after I adjusted the rule:
May 11 23:19:51 dietpi.com apache2[279996]: [core:error] [pid 279996:tid 282895] [client XXXX] AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/.%2e/bin/sh)
May 11 23:19:55 dietpi.com apache2[279996]: [core:error] [pid 279996:tid 283917] [client XXXX] AH10244: invalid URI path (/cgi-bin/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/%%32%65%%32%65/bin/sh)
It seems to be the same trying it again every now and then, just from a different host/IP .
But what I did not recognize at first based on the client IPs, is that the related requests are actually not from Cloudflare IPs (we use the Cloudflare header only for certain web applications, not by the webserver itself). So these must have connected via raw IP. Probably Cloudflare rejects such requests before they even reach the rules, like I got the 400 response when trying it myself. I further tried to creates error logs with /cgi-bin/../ in the path, but failed to do so. Not sure where these two path elements are collapsed, but when adding .htaccess or .htsomething (the first is blocked by another rule at Cloudflare, the other at our webserver), in both logs, at Cloudflare as well as on our webserver, /cgi-bin/../ is not shown at all.
So at the moment I fail to prove that raw.http.request.uri.path works as expected, but I cannot prove that it does not work either. The related logs at our origin did not pass Cloudflare anyway. In any case raw.http.request.uri.path answers my question, so I assume this solved until I do actually see a /cgi-bin/ request reaching our server from a Cloudflare IP.
Ok, I did some testing, and raw isn’t going to pick up the collapsed directory traversal.
EDIT: That would explain the following caveat the docs make:
This raw field may include some basic normalization done by Cloudflare’s HTTP server.
It /will/ pick up double slashes in raw mode, where it would only see a single slash without raw when normalizing is enabled.
In short, it looks like Cloudflare has chosen to correct these paths so they reflect the actual request, which can be properly blocked. It will ignore garbage parts of the path that are never going to reach the server.
Going back to the original issue, why would you want to block /cgi-bin/ at the start of a directory traversal URL if someone can just replace /cgi-bin/ with anything else and achieve the same results?
Does the request really not reach the server like this? I mean, as said, I do not see such a request logged at our server from a Cloudflare IP, and Cloudflare edge servers seem to see them as invalid as our webserver does, hence block them with 400 before they even reach the WAF.
I actually don’t know how classic CGI works, since I used PHP-FPM as PHP handler since administrating webservers. Would such a request really be treated the same when replacing /cgi-bin/ with something else, if there is an actual CGI handler present? Since bad bots do such requests, I can only imagine that there are old/buggy webserver or CGI implementations/versions which do give shell access (or theoretically access to any other file) with such a request. Modern webservers surely/obviously prevent it.
My aim was to reduce error logs on our server by catching some more bad requests at Cloudflare. But as said, those were bypassing Cloudflare, and would probably not be able to pass Cloudflare anyway (at a level even before the WAF).
.