The company I work for monitors web activity of registered sex offenders, this is achieved by installing a client on the device which intercepts browser traffic using a proxy (MitM), however, Cloudflare detects our MitM & most web sites using Cloudflare do not load correctly. Is it possible to add a global exception for our client or some other method of allowing Cloudflare to work with our client?
Do you have an example of the error that you see from Cloudflare? It sounds like something specific to your software.
I donât encounter any issues when using mitmproxy and accessing Cloudflare websites.
1 Like
Hi,
We see a number of different issues depending on the web site. Below are a couple of examples
Often the an issue is only seen browsing a site & not often on the home/landing page.
Chrome reports errors such as error 1020 or
- âERR_CACHE_READ_FAILURE 200 (OK)â
-âUncaught (in promise) TypeError: Failed to fetch at Xt.window.fetch.window.fetch (index.js:28:1)â
On facebook buisness we are seeing: âThis Page Isnât Available Right Nowâ message and the Chrome console spews out errors messages like âResponse contained invalid JSON. Reason: Unexpected token in JSON at position 0â.
I undestand what you are saying about something specific but we monitor 10,000âs of websites everyday & we only get these types of errors on sites that use Cloudflare, also, none of the errors we are seeing are generated by our software but by the browser.
Our developers believe it is due to the fact that we are using an MitM & Cloudflare is recognising this & this is somehow affecting the communication.
Regards
Paul Braithwaite
t: +44 161 696 3446
Disclaimer:
This email and its attachments are intended for the above named only and may be legally privileged or confidential. If they come to you in error you must take no action based on them, nor must you disclose, copy, distribute or use them. If you have received this email in error, please reply to this message with ERROR in the subject line and delete the content from your system. Thank you and please note that we monitor our email system and may record your emails.
If youâre doing MITM then I guess youâre inherently proxying the traffic - it sounds more like your traffic is triggering either Managed Rules (part of Cloudflareâs WAF solution which are rulesets maintained by Cloudflare) or the websites have setup their own rules (blocking known bots, proxies, GeoIP, etc).
In any case, itâs not that the traffic is being MITMâd (as you can test with something like mitmproxy) but rather that your particular proxy is upsetting Cloudflareâs security offerings.
Unfortunately, in any case, Cloudflare wonât override a customers security rules. As you can see in https://support.cloudflare.com/hc/en-us/articles/360029779472-Troubleshooting-Cloudflare-1XXX-errors#error1020 it indicates that 1020 means the block is a rule that a customer has defined.
Whilst I agree that Cloudflareâs firewall is blocking you, itâs not something that theyâre doing against you in particular but rather their customers (such as Selfridges) have opted to block your traffic for one reason or another.
Youâd need to reach out to the websites that youâre being blocked by and provide them with the RayID/timestamps/etc to get any clarity into what specifically is blocking you since that information can only be provided to the website owner (otherwise malicious actors could find ways to get around rules if they were told what blocked them).
Hi,
Many thanks for your response.
Your reply was along the lines that we thought & it would be up to the individual web site to allow.
Once again thanks for your help.
Regards
Paul Braithwaite
t: +44 161 696 3446
Disclaimer:
This email and its attachments are intended for the above named only and may be legally privileged or confidential. If they come to you in error you must take no action based on them, nor must you disclose, copy, distribute or use them. If you have received this email in error, please reply to this message with ERROR in the subject line and delete the content from your system. Thank you and please note that we monitor our email system and may record your emails.
There have been similar cases reported in the forum in the past regarding SaSe providers. There isnât an easy fix that Cloudflare can do to satisfy all parts of the equation. Most of the time, global exceptions arenât considered because while customer A is OK with an exception, customers B, C, and D are not. The main issue is that malicious actors can easily exploit any exception made (unless itâs very granular, which takes a lot of time).
I suspect that your proxy is triggering SBFM or Bot Management since the TLS fingerprint present in the connection doesnât match the rest of the headers.
All being said, Cloudflare is looking for a solution to the problem; however, you will likely wait for a while until progress is made (months if not years).
If you have contact with the owners/administrators of the website, it will be best to ask them to allow list your JA3 fingerprint.
Hi,
Many thanks for the detailed response. I think we are likely just to âliveâ with the situation for the time being, as contacting each web hosting company & getting them to agree with modifying their rules is very unlikely.
Thanks again.
Regards
**Paul {redacted}
1 Like
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.