DNS over HTTPS not working Nov-26-2019

user578 · November 27, 2019, 3:13am

Today Nov-26–2019 all my DNs over HTTPs devices which include linux , Mac OS and Android stopped working over DOH.The Cloudflare settings are as follows.

https://1.1.1.1/help#eyJpc0NmIjoiTm8iLCJpc0RvdCI6Ik5vIiwiaXNEb2giOiJObyIsInJlc29sdmVySXAtMS4xLjEuMSI6IlllcyIsInJlc29sdmVySXAtMS4wLjAuMSI6IlllcyIsInJlc29sdmVySXAtMjYwNjo0NzAwOjQ3MDA6OjExMTEiOiJObyIsInJlc29sdmVySXAtMjYwNjo0NzAwOjQ3MDA6OjEwMDEiOiJObyIsImRhdGFjZW50ZXJMb2NhdGlvbiI6IllZWiIsImlzV2FycCI6Ik5vIiwiaXNwTmFtZSI6IkNsb3VkZmxhcmUiLCJpc3BBc24iOiIxMzMzNSJ9

The traceroute shows access to the 1.1.1.1 network 3 3.52.251.198.in-addr.arpa (198.251.52.3) 20.584 ms 81.945 ms 56.215 ms

4 198.251.49.89 (198.251.49.89) 21.406 ms 21.144 ms 22.069 ms 5 198.251.51.56 (198.251.51.56) 22.771 ms 22.709 ms 21.188 ms 6 198.251.50.16 (198.251.50.16) 22.807 ms 78.695 ms 96.269 ms 7 cloudflare.ip4.torontointernetxchange.net (206.108.34.208) 106.362 ms 93.731 ms 95.215 ms 8 one.one.one.one (1.1.1.1) 164.851 ms * 88.710 ms

My Firefox setting has DNS over hTTPS enabled no extensions and configured for Cloudflare and my about:config settings network.trr.bootstrapAddress;1.1.1.1 network.trr.mode;3 network.trr.resolvers;[{ “name”: “Cloudflare”, “url”: “https://mozilla.cloudflare-dns.com/dns-query” }]

Network mode 3 means DOH should only resolve using DOH but I can still get to other sites instead of a message say “hmm where having trouble finding that site”

Does anyone know why this is occurring?

sandro · November 27, 2019, 3:18am

Are you running Windows 10?
If so, what is the output of the following PowerShell command?

(Invoke-WebRequest -Uri 'https://1.1.1.1/dns-query?ct=application/dns-json&name=cloudflare.com').RawContent

user578 · November 27, 2019, 3:29am

I am running on a Mac and Android and Linux

sandro · November 27, 2019, 3:29am

In that case run

curl -v 'https://1.1.1.1/dns-query?ct=application/dns-json&name=cloudflare.com'

user578 · November 27, 2019, 3:35am

The result

Trying 1.1.1.1…
TCP_NODELAY set
Connected to 1.1.1.1 (1.1.1.1) port 443 (#0)
ALPN, offering h2
ALPN, offering http/1.1
Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
successfully set certificate verify locations:
CAfile: /etc/ssl/cert.pem
CApath: none
TLSv1.2 (OUT), TLS handshake, Client hello (1):
TLSv1.2 (IN), TLS handshake, Server hello (2):
TLSv1.2 (IN), TLS handshake, Certificate (11):
TLSv1.2 (IN), TLS handshake, Server key exchange (12):
TLSv1.2 (IN), TLS handshake, Server finished (14):
TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
TLSv1.2 (OUT), TLS change cipher, Client hello (1):
TLSv1.2 (OUT), TLS handshake, Finished (20):
TLSv1.2 (IN), TLS change cipher, Client hello (1):
TLSv1.2 (IN), TLS handshake, Finished (20):
SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
ALPN, server accepted to use h2
Server certificate:
subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
start date: Jan 28 00:00:00 2019 GMT
expire date: Feb 1 12:00:00 2021 GMT
subjectAltName: host “1.1.1.1” matched cert’s IP address!
issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
SSL certificate verify ok.
Using HTTP2, server supports multi-use
Connection state changed (HTTP/2 confirmed)
Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
Using Stream ID: 1 (easy handle 0x7fe24b008a00)

GET /dns-query?ct=application/dns-json&name=cloudflare.com HTTP/2
Host: 1.1.1.1
User-Agent: curl/7.54.0
Accept: /

Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 400
< date: Wed, 27 Nov 2019 03:34:27 GMT
< content-length: 0
< access-control-allow-origin: *
< expect-ct: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct”
< server: cloudflare
< cf-ray: 53c10be5f882abac-YYZ
<
Connection #0 to host 1.1.1.1 left intact

sandro · November 27, 2019, 3:40am

Interesting, you seem to get a 400 error.
That is most likely an issue restricted to the YYZ datacentre.

What if you try 1.0.0.1?

curl -v 'https://1.0.0.1/dns-query?ct=application/dns-json&name=cloudflare.com'

@dane @irtefa

user578 · November 27, 2019, 3:43am

What do you mean 400 error? The connectivity script shows access to both 1.1.1.1 and 1.0.0.1 also my traceroute is successful to 1.1.1.1

sandro · November 27, 2019, 3:43am

Post the output of the second command.

sandro · November 27, 2019, 3:44am

And while we are at it, lets try these too.

curl -v --http1.1 'https://1.1.1.1/dns-query?ct=application/dns-json&name=cloudflare.com'
curl -v --http1.1 'https://1.0.0.1/dns-query?ct=application/dns-json&name=cloudflare.com'

user578 · November 27, 2019, 7:10am

result 1
curl -v --http1.1 ‘https://1.1.1.1/dns-query?ct=application/dns-json&name=cloudflare.com’

Trying 1.1.1.1…
TCP_NODELAY set
Connected to 1.1.1.1 (1.1.1.1) port 443 (#0)
ALPN, offering http/1.1
Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
successfully set certificate verify locations:
CAfile: /etc/ssl/cert.pem
CApath: none
TLSv1.2 (OUT), TLS handshake, Client hello (1):
TLSv1.2 (IN), TLS handshake, Server hello (2):
TLSv1.2 (IN), TLS handshake, Certificate (11):
TLSv1.2 (IN), TLS handshake, Server key exchange (12):
TLSv1.2 (IN), TLS handshake, Server finished (14):
TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
TLSv1.2 (OUT), TLS change cipher, Client hello (1):
TLSv1.2 (OUT), TLS handshake, Finished (20):
TLSv1.2 (IN), TLS change cipher, Client hello (1):
TLSv1.2 (IN), TLS handshake, Finished (20):
SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
ALPN, server accepted to use http/1.1
Server certificate:
subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
start date: Jan 28 00:00:00 2019 GMT
expire date: Feb 1 12:00:00 2021 GMT
subjectAltName: host “1.1.1.1” matched cert’s IP address!
issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
SSL certificate verify ok.

GET /dns-query?ct=application/dns-json&name=cloudflare.com HTTP/1.1
Host: 1.1.1.1
User-Agent: curl/7.54.0
Accept: /

< HTTP/1.1 400 Bad Request
< Date: Wed, 27 Nov 2019 07:05:40 GMT
< Content-Length: 0
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< Expect-CT: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct”
< Server: cloudflare
< CF-RAY: 53c241498ec6caa4-YYZ
<

Connection #0 to host 1.1.1.1 left intact

result 2
curl -v --http1.1 ‘https://1.0.0.1/dns-query?ct=application/dns-json&name=cloudflare.com’

Trying 1.0.0.1…
TCP_NODELAY set
Connected to 1.0.0.1 (1.0.0.1) port 443 (#0)
ALPN, offering http/1.1
Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
successfully set certificate verify locations:
CAfile: /etc/ssl/cert.pem
CApath: none
TLSv1.2 (OUT), TLS handshake, Client hello (1):
TLSv1.2 (IN), TLS handshake, Server hello (2):
TLSv1.2 (IN), TLS handshake, Certificate (11):
TLSv1.2 (IN), TLS handshake, Server key exchange (12):
TLSv1.2 (IN), TLS handshake, Server finished (14):
TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
TLSv1.2 (OUT), TLS change cipher, Client hello (1):
TLSv1.2 (OUT), TLS handshake, Finished (20):
TLSv1.2 (IN), TLS change cipher, Client hello (1):
TLSv1.2 (IN), TLS handshake, Finished (20):
SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
ALPN, server accepted to use http/1.1
Server certificate:
subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
start date: Jan 28 00:00:00 2019 GMT
expire date: Feb 1 12:00:00 2021 GMT
subjectAltName: host “1.0.0.1” matched cert’s IP address!
issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
SSL certificate verify ok.

GET /dns-query?ct=application/dns-json&name=cloudflare.com HTTP/1.1
Host: 1.0.0.1
User-Agent: curl/7.54.0
Accept: /

< HTTP/1.1 400 Bad Request
< Date: Wed, 27 Nov 2019 07:10:06 GMT
< Content-Length: 0
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< Expect-CT: max-age=604800, report-uri=“https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct”
< Server: cloudflare
< CF-RAY: 53c247c7cf4ecab8-YYZ
<

Connection #0 to host 1.0.0.1 left intact
MacBook-Pro:Documents mac$

sandro · November 27, 2019, 9:35am

You skipped the second command, but as the others all show the same response I assume that wont make a difference.

As I said in my earlier response this must be an issue local to YYZ and it would be best if you contacted support about it.

user578 · November 27, 2019, 2:23pm

You mean my ISP vmedia? I connected via a VPN the Data center was NTT and the issue continues. I also looked at Firefox “about:networking#dns” and it shows a the following TRR values

Hostname Family TRR Addresses Expires (Seconds)
r4---sn-oguesnz6.googlevideo.com ipv4 true 74.125.171.250
1609
etc…

The TRR value for all sites is true which means Firefox is using DOH.

What is your reasoning? How would you explain the fault?

sandro · November 27, 2019, 2:25pm

Did I say that? Please reread my original response from eleven hours ago. Only support can help you in this case.

user578 · November 27, 2019, 2:34pm

You did not answer my question.

What is your reasoning? How would you explain the fault?

I only deal with my ISP Vmedia on which I am currently running public IP address 198.251.56.153. If there is a data center problem I should see the fault on Cloudflares `status page httsp://www.cloudflarestatus.com

sandro · November 27, 2019, 2:37pm

And you didnt read my answer, otherwise your question would have been answered.

Apparently a local issue of YYZ, as I have said three times now.

You need to contact support, as I have also said three times now

Ideally, but not necessarily.

mjsuhonos · November 27, 2019, 7:01pm

I am experiencing the same issue; DNS over HTTPS suddenly not working in the past 24h, where it was working previously. I’m also using the YYZ Cloudflare Data Center.

https://1.1.1.1/help#eyJpc0NmIjoiTm8iLCJpc0RvdCI6Ik5vIiwiaXNEb2giOiJObyIsInJlc29sdmVySXAtMS4xLjEuMSI6IlllcyIsInJlc29sdmVySXAtMS4wLjAuMSI6IlllcyIsInJlc29sdmVySXAtMjYwNjo0NzAwOjQ3MDA6OjExMTEiOiJObyIsInJlc29sdmVySXAtMjYwNjo0NzAwOjQ3MDA6OjEwMDEiOiJObyIsImRhdGFjZW50ZXJMb2NhdGlvbiI6IllZWiIsImlzV2FycCI6Ik5vIiwiaXNwTmFtZSI6IkNsb3VkZmxhcmUiLCJpc3BBc24iOiIxMzMzNSJ9

I’ve tried submitting a support ticket but am getting an HTTP 500 error:

API Request Failed: POST /api/v4/accounts/790850bba7535133396c6f4d98a2a560/support/ticket/create (500)

Is there some other way to follow up on this issue? Thanks in advance.

sandro · November 27, 2019, 7:02pm

You also seem to go via YYZ, so it will be the same issue. You can also drop them an email to [email protected].

@dane @irtefa @cloonan

mjsuhonos · November 27, 2019, 7:08pm

You also seem to go via YYZ, so it will be the same issue.

I understand that, but without the ability to submit a support ticket, I don’t know how to escalate / notify Cloudflare of the issue. Can anyone else confirm problem submitting support tickets?

Edit: Ah, thanks, didn’t know about the email contact. Thanks!

user578 · November 27, 2019, 9:31pm

Where did you initially make the support request to Cloudflare? I do not see any change in cloudflare status.

The IP address my workstations are using for DOH resolution is 108.162.240.42

sandro · November 27, 2019, 9:34pm

You are referring to the error @mjsuhonos posted?

Topic		Replies	Views
Dns over https not working 1.1.1.1	9	5168	June 18, 2021
DNS over HTTPS for Family not working DNS & Network	4	2722	May 16, 2021
DNS over https using https://104.16.249.249/dns-query DNS & Network dns , dash-ssl-tls	18	15860	April 11, 2019
DOH firefox setting network.trr.mode=3 DNS & Network dns , dash-getting-started	7	8444	January 7, 2019
Connection information incorrect for "https://cloudflare-dns.com/help/" 1.1.1.1 dns , dns-resolver	5	3221	June 18, 2021

DNS over HTTPS not working Nov-26-2019

Related topics