Cloudflare detecting our MitM

The company I work for monitors web activity of registered sex offenders, this is achieved by installing a client on the device which intercepts browser traffic using a proxy (MitM), however, Cloudflare detects our MitM & most web sites using Cloudflare do not load correctly. Is it possible to add a global exception for our client or some other method of allowing Cloudflare to work with our client?

Do you have an example of the error that you see from Cloudflare? It sounds like something specific to your software.

image

I don’t encounter any issues when using mitmproxy and accessing Cloudflare websites.

1 Like

Hi,

We see a number of different issues depending on the web site. Below are a couple of examples

Often the an issue is only seen browsing a site & not often on the home/landing page.

Chrome reports errors such as error 1020 or

  • “ERR_CACHE_READ_FAILURE 200 (OK)”

-“Uncaught (in promise) TypeError: Failed to fetch at Xt.window.fetch.window.fetch (index.js:28:1)”

On facebook buisness we are seeing: ‘This Page Isn’t Available Right Now’ message and the Chrome console spews out errors messages like “Response contained invalid JSON. Reason: Unexpected token in JSON at position 0”.

I undestand what you are saying about something specific but we monitor 10,000’s of websites everyday & we only get these types of errors on sites that use Cloudflare, also, none of the errors we are seeing are generated by our software but by the browser.

Our developers believe it is due to the fact that we are using an MitM & Cloudflare is recognising this & this is somehow affecting the communication.

Regards

Paul Braithwaite

t: +44 161 696 3446

w: esafeglobal.com

linkedin iconYoutube icon

Disclaimer:

This email and its attachments are intended for the above named only and may be legally privileged or confidential. If they come to you in error you must take no action based on them, nor must you disclose, copy, distribute or use them. If you have received this email in error, please reply to this message with ERROR in the subject line and delete the content from your system. Thank you and please note that we monitor our email system and may record your emails.

If you’re doing MITM then I guess you’re inherently proxying the traffic - it sounds more like your traffic is triggering either Managed Rules (part of Cloudflare’s WAF solution which are rulesets maintained by Cloudflare) or the websites have setup their own rules (blocking known bots, proxies, GeoIP, etc).

In any case, it’s not that the traffic is being MITM’d (as you can test with something like mitmproxy) but rather that your particular proxy is upsetting Cloudflare’s security offerings.

Unfortunately, in any case, Cloudflare won’t override a customers security rules. As you can see in https://support.cloudflare.com/hc/en-us/articles/360029779472-Troubleshooting-Cloudflare-1XXX-errors#error1020 it indicates that 1020 means the block is a rule that a customer has defined.

Whilst I agree that Cloudflare’s firewall is blocking you, it’s not something that they’re doing against you in particular but rather their customers (such as Selfridges) have opted to block your traffic for one reason or another.

You’d need to reach out to the websites that you’re being blocked by and provide them with the RayID/timestamps/etc to get any clarity into what specifically is blocking you since that information can only be provided to the website owner (otherwise malicious actors could find ways to get around rules if they were told what blocked them).

Hi,

Many thanks for your response.

Your reply was along the lines that we thought & it would be up to the individual web site to allow.

Once again thanks for your help.

Regards

Paul Braithwaite

t: +44 161 696 3446

w: esafeglobal.com

linkedin iconYoutube icon

Disclaimer:

This email and its attachments are intended for the above named only and may be legally privileged or confidential. If they come to you in error you must take no action based on them, nor must you disclose, copy, distribute or use them. If you have received this email in error, please reply to this message with ERROR in the subject line and delete the content from your system. Thank you and please note that we monitor our email system and may record your emails.

There have been similar cases reported in the forum in the past regarding SaSe providers. There isn’t an easy fix that Cloudflare can do to satisfy all parts of the equation. Most of the time, global exceptions aren’t considered because while customer A is OK with an exception, customers B, C, and D are not. The main issue is that malicious actors can easily exploit any exception made (unless it’s very granular, which takes a lot of time).

I suspect that your proxy is triggering SBFM or Bot Management since the TLS fingerprint present in the connection doesn’t match the rest of the headers.

All being said, Cloudflare is looking for a solution to the problem; however, you will likely wait for a while until progress is made (months if not years).
If you have contact with the owners/administrators of the website, it will be best to ask them to allow list your JA3 fingerprint.

Hi,

Many thanks for the detailed response. I think we are likely just to “live” with the situation for the time being, as contacting each web hosting company & getting them to agree with modifying their rules is very unlikely.

Thanks again.

Regards

**Paul {redacted}

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.