Iranian government is censoring sites with Universal SSL (CloudFlare Inc ECC CA-2)

Hey guys,
Iranian government is censoring new web sites added to Cloudflare and it only applies to HTTPS site with new Cloudflare CA: CloudFlare Inc ECC CA-2.
It’s so confusing, some new websites are accessible (like http://hamyarwp.com/) but others not! (like https://rexupload.com/). Any new website I add to my panel or colleagues, have the same issue with Universal SSL.

My Cloudflare configuration:

SSL: Full
Always Use HTTPS: on
HTTP Strict Transport Security (HSTS): on and valid for 6 months
Minimum TLS Version: TLS 1.2
Opportunistic Encryption: on
TLS 1.3: Enabled + 0-RTT

In popular Iranian ISPs, like IranCell-AS and Asiatech Data Transfer after Client hello we get timeout error:

(304) (OUT), TLS handshake, Client hello (1):

cURL failed:
AS44244 Iran Cell Service and Communication Company: http://termbin.com/4i39
AS47796 Pardaz Gostar Ertebatat Berelian Limited Liability Company: https://termbin.com/bwwt
AS43754 Asiatech Data Transfer Inc PLC: https://termbin.com/9gqa

cURL succeeded:
AS50810 Mobin Net Communication Company (Private Joint Stock): https://termbin.com/270o
AS24631 Tose’h Fanavari Ertebabat Pasargad Arian Co. PJS: https://termbin.com/c35o
AS57230 Aria Web Development LLC: https://termbin.com/lxe6
AS61173 Green Web Samaneh Novin Co Ltd: https://termbin.com/3pcq

As you can see some users in AS43754 Asiatech can access the website and others can’t!
Check this RIPE Atlas test result:
Minimum TLS 1.3: https://atlas.ripe.net/measurements/22693776/#!probes
Minimum TLS 1.0: https://atlas.ripe.net/measurements/22703933/#!probes

Beside these I found out another website, http://hamyarwp.com/ is mostly accessible in Iran.
I don’t know how the owner configured Cloudflare, but there is no differences in TLS handshake process, just their IP range is different and I tried to load https://rexupload.com/ through their Cloudflare Anycast range and again I receive timeout!

curl -4svI --resolve rexupload.com:443:104.28.0.138 https://rexupload.com/

cURL --resolve failed:
AS44244 Iran Cell Service and Communication Company: https://termbin.com/2udn

Both X.509 certificates and root CA certificates are the same just letter f in Cloudflare, for rexupload.com is in capital F and for hamyarwp.com is lowercase f! :cowboy_hat_face:

Disabling TLS 1.2, only TLS 1.3 + 0-RTT made no difference and OCSP stapling is working fine:

openssl s_client -connect rexupload.com:443 -tlsextdebug  -status | grep -i -A 20 OCSP

OCSP response:
======================================
OCSP Response Data:
OCSP Response Status: successful (0x0)
Response Type: Basic OCSP Response
Version: 1 (0x0)
Responder Id: 3E742D1FCF4575047E3FC0A2873E4C43835113C6
Produced At: Sep 1 09:29:59 2019 GMT
Responses:
Certificate ID:
Hash Algorithm: sha1
Issuer Name Hash: 2B0413693DF1D33D7E89CBA055CF204F9C158C9D
Issuer Key Hash: 3E742D1FCF4575047E3FC0A2873E4C43835113C6
Serial Number: 01621D1A527AF3F5AE2B4B25D4858146
Cert Status: good
This Update: Sep 1 09:29:59 2019 GMT
Next Update: Sep 8 08:44:59 2019 GMT
Signature Algorithm: ecdsa-with-SHA256
30:45:02:20:57:bf:32:a0:be:48:53:57:7e:c1:1f:e5:0b:1e:
df:f5:6e:20:60:49:a3:8c:65:e7:62:43:b5:89:f6:f9:77:0b:
02:21:00:8f:62:0c:0f:e1:59:da:8c:bf:89:4e:88:73:87:66:
58:10:6c:07:45:f2:98:44:01:49:24:51:c6:7c:cb:94:ca

DNSSEC is configured:

https://dnssec-analyzer.verisignlabs.com/rexupload.com :

In SSLlabs test, the only difference is hamyarwp.com Doesn’t have DNS Certification Authority Authorization (CAA) Policy

hamyarwp.com: https://www.ssllabs.com/ssltest/analyze.html?d=hamyarwp.com&s=104.28.0.138
rexupload.com: https://www.ssllabs.com/ssltest/analyze.html?d=rexupload.com&s=104.24.116.187

One of our Iranian users mentioned he can open rexupload.com via Windows 10 - 64 bit Google Chrome but can not open it via Ubuntu Linux connected to the same modem (so same public IP and ISP).
AS42337 Respina Networks & Beyond PJSC: https://termbin.com/ed4x
In this case, there may be a request that some operating systems send and it triggers governmental DPI (Deep Packet Inspection) firewall and it blocks the request but other OSs don’t send the request and they can open the website properly!

We couldn’t test ESNI because when we start cloudflared and activate ESNI on Firefox, we can not even open cloudflare.com! and this only happens to users on the same ISPs who could not open rexupload.com!
There may be some similarities between ESNI - DNS over HTTPS and Universal SSL requests!

I will order Dedicated SSL and let you know if that fixed the problem, but if you are from Cloudflare support team, please check hamyarwp.com and let us know what is the difference so we can implement it on other websites.

cURL for https://hamyarwp.com/ succeeded:
AS44244 Iran Cell Service and Communication Company: https://termbin.com/5sjr
AS50810 Mobin Net Communication Company (Private Joint Stock): https://termbin.com/su2l

Thanks for reading this far, any idea and comment may be helpful.
Thanks in advance.

P.S. 1: SSL test for hamyarwp.com:443 -> https://atlas.ripe.net/measurements/22698882/#!general
P.S. 2: All old websites with COMODO ECC Domain Validation Secure Server CA 2 like sni81127.cloudflaressl.com are working fine, this issue is only related to new Universal SSLs.
P.S. 3: I can provide web proxy, VPN or remote Desktop from Iran to test thoroughly.

After reading all topics about Iran, I should add some updates:
First of all, @sdayman and @sandro , I appreciate your help, thanks for your replies on topics about Iranian users issues.
Cloudflare was so popular in Iran, but governmental DPI (Deep Packet Inspection) program had huge impact on Cloudflare traffic from Iran and made everything for Iranian web site owners confusing too. I have to talk about some myths:
a) Cloudflare IPs are not blocked in Iran, all Cloudflare Anycast IPs are accessible via ping and telnet.
b) Iranian government doesn’t develop censorship software itself, mostly they get help from Russia and China, so anything is happening in Iran, might be happening right now in Russia, China, Kazakhstan and other counties …
c) I’m not asking you or Cloudflare to fight against censorship, we are looking for some features those match our needs and help us in fixing our issues, we are not anti-censorship protesters, we are business owners trying to fix some technical issues! :cowboy_hat_face:
For example Cloudflare helped every one by adding Let’s encrypt support to Universal SSL, we are looking for the same feature to fix another issue.
d) If Iranian DPI is blocking all requests to websites with Universal SSL, why https://hamyarwp.com/ is accessible? there is not even a single difference between this and https://rexupload.com/ in SSL or any other configuration! It’s just a blog about Wordpress templates, plugins and etc, I’m 100% sure government didn’t whitelist only this website in their DPI! :smiley:

Long story short, here is the situation:
A) If we disable HTTP/2, all websites behind Cloudflare are accessible, as mentioned here, via cURL, Wget or even Firefox with disabled HTTP/2, we can open https://rexupload.com/
cURL: AS44244 Iran Cell Service and Communication Company: https://termbin.com/et7b
B) As mentioned here, If we first open a subdomain, we get the TLS handshake established and then we can open the naked domain domain too, but until Cloudflare server closes the TLS connection.
We first open https://www.rexupload.com/ or https://sub.rexupload.com/, then we can open https://rexupload.com.
C) There is no issue with websites via Let’s encrypt certificate like https://amnk.ir in Iran.
D) All websites with old Universal TLS like https://blogfa.com are accessible without any issues.

So what is your idea? Is there any relation between HTTP/2, new Universal SSLs and establishing TLS handshake via sub domain first? :cold_face:

I’m out of idea right now, please share your comments, I will provide any information or test results needed.
I would prefer some easier tests than running tcpdump on the machine and decrypt every single packet :sweat_smile: but if that’s the only way, I’m OK with it.

Let’s put an end to topics here about Iran and fix it once for ever! :cowboy_hat_face:

1 Like

Two updates:

  1. I found out the censorship only applies to domains with .com and .net TLDs!
    Seems government DPI is not interested in censoring other TLDs and I can conclude they are using SNI header to block websites.
    One of our websites with .cloud TLD and CloudFlare Inc ECC CA-2 is working like a charm!
  2. DNS-over-HTTPS will be rolled out by default in Firefox for U.S. users starting at the end of September 2019. Firefox will default to using Cloudflare's 1.1.1.1 at first, but that may change if other resolvers adopt a comparably strong privacy policy.

Nick Sullivan on Twitter

What’s next in making Encrypted DNS-over-HTTPS the Default - Future Releases

That will fix all Iranians problems with new CA and it will force Iranian government to stop blocking 1.1.1.1 DNS over HTTPS traffic.
I hope Google Chrome activates DNS over HTTPS by default for all users soon.

1 Like

regarding Chrome DoH’s future, see How to enable DNS over HTTPS in Chrome

1 Like

Really great work. I hope it’s true with opening 1.1.1.1. I was pulling my hair out for 4-5 days trying to figure out what the issue was.

I knew about this issue last year, but there wasn’t a solution then or content, so I just kept sites in Iran off SSL. However, I was forced to in order to get some features working.

Now, I purchased an SSL instead of free Let’s Encrypt or Cloudflare & got off all CDN’s since they would break the site, but the site is now painfully slow.

Wish there was a way to check those working site’s Cloudflare config too.

Anyways, I am following the thread and your work/progress. The most documented & thorough coverage of the issue so far. Wanted to let you know it’s much appreciated

1 Like