DNS over TLS cant verify certificate

Hi there,

recently i ran into problems with 1.1.1.1 and DNS over TLS. Unbound throws this error:

[659:0] error: ssl handshake failed crypto error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
[659:0] notice: ssl handshake failed 1.1.1.1 port 853

Quad9 works like a charm. Is there a way where i can verify if its a problem with my certs (or my local systems) and not with 1.1.1.1?

Could it be that your ISP hijacked that address?

Whats the output of these commands?

dig +short CHAOS TXT id.server @1.1.1.1

openssl s_client -connect 1.1.1.1:853 -servername 1.1.1.1

openssl s_client -connect 1.0.0.1:853 -servername 1.0.0.1

curl -vH 'accept: application/dns-json' 'https://1.1.1.1/dns-query?name=cloudflare.com'

curl -kvH 'accept: application/dns-json' 'https://1.1.1.1/dns-query?name=cloudflare.com'

dig +short CHAOS TXT id.server @1.1.1.1
OUTPUT: FRA

curl -vH ‘accept: application/dns-json’ ‘https://1.1.1.1/dns-query?name=cloudflare.com

*   Trying 1.1.1.1:443...
* TCP_NODELAY set
* Connected to 1.1.1.1 (1.1.1.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
*  start date: Jan 28 00:00:00 2019 GMT
*  expire date: Feb  1 12:00:00 2021 GMT
*  subjectAltName: host "1.1.1.1" matched cert's IP address!
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x438206175e0)
> GET /dns-query?name=cloudflare.com HTTP/2
> Host: 1.1.1.1
> user-agent: curl/7.67.0
> accept: application/dns-json
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200
< date: Tue, 31 Dec 2019 16:02:00 GMT
< content-type: application/dns-json
< content-length: 287
< access-control-allow-origin: *
< cache-control: max-age=148
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 54dd79af88c797d2-FRA
<
* Connection #0 to host 1.1.1.1 left intact
{"Status": 0,"TC": false,"RD": true, "RA": true, "AD": true,"CD": false,"Question":[{"name": "cloudflare.com.", "type": 1}],"Answer":[{"name": "cloudflare.com.", "type": 1, "TTL": 148, "data": "104.17.175.85"},{"name": "cloudflare.com.", "type": 1, "TTL": 148, "data": "104.17.176.85"}]}

curl -kvH ‘accept: application/dns-json’ ‘https://1.1.1.1/dns-query?name=cloudflare.com

*   Trying 1.1.1.1:443...
* TCP_NODELAY set
* Connected to 1.1.1.1 (1.1.1.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
*  start date: Jan 28 00:00:00 2019 GMT
*  expire date: Feb  1 12:00:00 2021 GMT
*  subjectAltName: host "1.1.1.1" matched cert's IP address!
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x438206175e0)
> GET /dns-query?name=cloudflare.com HTTP/2
> Host: 1.1.1.1
> user-agent: curl/7.67.0
> accept: application/dns-json
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200
< date: Tue, 31 Dec 2019 16:02:00 GMT
< content-type: application/dns-json
< content-length: 287
< access-control-allow-origin: *
< cache-control: max-age=148
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 54dd79af88c797d2-FRA
<
* Connection #0 to host 1.1.1.1 left intact
{"Status": 0,"TC": false,"RD": true, "RA": true, "AD": true,"CD": false,"Question":[{"name": "cloudflare.com.", "type": 1}],"Answer":[{"name": "cloudflare.com.", "type": 1, "TTL": 148, "data": "104.17.175.85"},{"name": "cloudflare.com.", "type": 1, "TTL": 148, "data": "104.17.176.85"}]}[root@Server04 unbound]# ^C
[root@Server04 unbound]#
[root@Server04 unbound]#
[root@Server04 unbound]#
[root@Server04 unbound]# curl -kvH 'accept: application/dns-json' 'https://1.1.1.1/dns-query?name=cloudflare.com'
*   Trying 1.1.1.1:443...
* TCP_NODELAY set
* Connected to 1.1.1.1 (1.1.1.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
*  start date: Jan 28 00:00:00 2019 GMT
*  expire date: Feb  1 12:00:00 2021 GMT
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x4006489fac0)
> GET /dns-query?name=cloudflare.com HTTP/2
> Host: 1.1.1.1
> user-agent: curl/7.67.0
> accept: application/dns-json
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200
< date: Tue, 31 Dec 2019 16:03:04 GMT
< content-type: application/dns-json
< content-length: 285
< access-control-allow-origin: *
< cache-control: max-age=84
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 54dd7b40090f97cc-FRA
<
* Connection #0 to host 1.1.1.1 left intact
{"Status": 0,"TC": false,"RD": true, "RA": true, "AD": true,"CD": false,"Question":[{"name": "cloudflare.com.", "type": 1}],"Answer":[{"name": "cloudflare.com.", "type": 1, "TTL": 84, "data": "104.17.175.85"},{"name": "cloudflare.com.", "type": 1, "TTL": 84, "data": "104.17.176.85"}]}

The openssl commands doesnt seem to work for me. I use arch linux

And the other two commands?

doesnt work for me :confused:

“Doesnt seem to work” is not a very exact description. What does not work? Maybe you just need to install OpenSSL.

nvm. seems like it does not like the verbose option

CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert ECC Secure Server CA
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = cloudflare-dns.com
verify return:1
---
Certificate chain
 0 s:C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = cloudflare-dns.com
   i:C = US, O = DigiCert Inc, CN = DigiCert ECC Secure Server CA
 1 s:C = US, O = DigiCert Inc, CN = DigiCert ECC Secure Server CA
   i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFxjCCBUygAwIBAgIQAczjGN6fVn+rKySQH62nHTAKBggqhkjOPQQDAjBMMQsw
CQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSYwJAYDVQQDEx1EaWdp
Q2VydCBFQ0MgU2VjdXJlIFNlcnZlciBDQTAeFw0xOTAxMjgwMDAwMDBaFw0yMTAy
MDExMjAwMDBaMHIxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYw
FAYDVQQHEw1TYW4gRnJhbmNpc2NvMRkwFwYDVQQKExBDbG91ZGZsYXJlLCBJbmMu
MRswGQYDVQQDExJjbG91ZGZsYXJlLWRucy5jb20wWTATBgcqhkjOPQIBBggqhkjO
PQMBBwNCAATFIHCMIEJQKB59REF8MHkpHGNeHUSbxfdxOive0qKksWw9ash3uMuP
LlBT/fQYJn9hN+3/wr7pC125fuHfHOJ0o4ID6DCCA+QwHwYDVR0jBBgwFoAUo53m
H/naOU/AbuiRy5Wl2jHiCp8wHQYDVR0OBBYEFHCV3FyjjmYH28uBEMar58OoRX+g
MIGsBgNVHREEgaQwgaGCEmNsb3VkZmxhcmUtZG5zLmNvbYIUKi5jbG91ZGZsYXJl
LWRucy5jb22CD29uZS5vbmUub25lLm9uZYcEAQEBAYcEAQAAAYcEop+ENYcQJgZH
AEcAAAAAAAAAAAAREYcQJgZHAEcAAAAAAAAAAAAQAYcQJgZHAEcAAAAAAAAAAAAA
ZIcQJgZHAEcAAAAAAAAAAABkAIcEop8kAYcEop8uATAOBgNVHQ8BAf8EBAMCB4Aw
HQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGkGA1UdHwRiMGAwLqAsoCqG
KGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9zc2NhLWVjYy1nMS5jcmwwLqAsoCqG
KGh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9zc2NhLWVjYy1nMS5jcmwwTAYDVR0g
BEUwQzA3BglghkgBhv1sAQEwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGln
aWNlcnQuY29tL0NQUzAIBgZngQwBAgIwewYIKwYBBQUHAQEEbzBtMCQGCCsGAQUF
BzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wRQYIKwYBBQUHMAKGOWh0dHA6
Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEVDQ1NlY3VyZVNlcnZlckNB
LmNydDAMBgNVHRMBAf8EAjAAMIIBfgYKKwYBBAHWeQIEAgSCAW4EggFqAWgAdgCk
uQmQtBhYFIe7E6LMZ3AKPDWYBPkb37jjd80OyA3cEAAAAWiVHhSLAAAEAwBHMEUC
IQDlnoPeMXtFkRsy3Vs0eovk3ILKt01x6bgUdMlmQTFIvAIgcAn0lFSjiGzHm2eO
jDZJzMiP5Uaj0Jwub9GO8RkxkkoAdQCHdb/nWXz4jEOZX73zbv9WjUdWNv9KtWDB
tOr/XqCDDwAAAWiVHhVsAAAEAwBGMEQCIFC0n0JModeol8b/Qicxd5Blf/o7xOs/
Bk0j9hdc5N7jAiAQocYnHL9iMqTtFkh0vmSsII5NbiakM/2yDEXnwkPRvAB3ALvZ
37wfinG1k5Qjl6qSe0c4V5UKq1LoGpCWZDaOHtGFAAABaJUeFJEAAAQDAEgwRgIh
AL3OPTBzOZpS5rS/uLzqMOiACCFQyY+mTJ+L0I9TcB3RAiEA4+SiPz0/5kFxvrk7
AKYKdvelgV1hiiPbM2YHY+/0BIkwCgYIKoZIzj0EAwIDaAAwZQIwez76hX2HTMur
/I3XRuwfdmVoa8J6ZVEVq+AZsE7DyQh7AV4WNLU+092BrPbnyVUFAjEAzUf5jdz1
pyc74lgOunC7LBE6cPtWbzfGpJiYyT/T+c5eIAwRYziKT0DKbaql7tiZ
-----END CERTIFICATE-----
subject=C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = cloudflare-dns.com

issuer=C = US, O = DigiCert Inc, CN = DigiCert ECC Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2902 bytes and written 402 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-AES256-GCM-SHA384
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES256-GCM-SHA384
    Session-ID: 7950516A90C0744DF9485235C815D4B67D83EB31BE79B084267E285EE48E63F4
    Session-ID-ctx:
    Master-Key: 1BF655770ADA41769980515FD37F299159DE852E2BF81F08F94736201281B8AAAE41551A3E43D9569FB843FC0747D993
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 15 17 4c c9 d6 b1 91 47-dd 5d 49 0b b3 ca f7 c1   ..L....G.]I.....
    0010 - eb 12 1e d0 be 7c 39 45-6d 5d 74 df bc 22 17 9d   .....|9Em]t.."..
    0020 - 2a 7a a3 c1 ff 88 c8 7a-96 f1 83 01 f6 8a 52 b3   *z.....z......R.
    0030 - e4 9c f2 cf ce 36 4c 8d-62 9b 6e f6 23 2a fe 2f   .....6L.b.n.#*./
    0040 - 9a 90 15 d1 4b 8b 9c 31-a3 8e bc 7b 97 5c 8f b9   ....K..1...{.\..
    0050 - bc 30 36 6d 3e 65 a8 49-0e 87 fc fb 60 1a d0 7c   .06m>e.I....`..|
    0060 - 86 5e 9a de 6e 69 25 f3-b4 ea c3 5e da 8f e4 5b   .^..ni%....^...[
    0070 - 5f 8a 7f 48 74 de 99 3f-52 38 ad d7 6a f5 f3 0c   _..Ht..?R8..j...
    0080 - f1 09 b7 9f dd ff d7 8e-e1 ea c6 ee bf ef 47 8e   ..............G.
    0090 - 14 30 38 ba 8f b2 75 0d-f8 c2 58 b6 52 57 55 10   .08...u...X.RWU.
    00a0 - 85 ad ba 20 24 12 d9 58-7c 21 ce f0 cd 4f c9 0c   ... $..X|!...O..
    00b0 - ad 33 b8 9f e5 21 11 e6-2f fd 75 17 a8 49 a1 26   .3...!../.u..I.&

    Start Time: 1577808573
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
closed

i see that cloudflare uses tls1.2 and not 1.3. Could that be the problem?

Oops, that was actually my bad. I added a non-existent option.

That all actually looks okay, so I’d rather rule out an issue with your ISP. Maybe the software you are using does not have a certificate store configured and hence cant verify the certificates. You might want to clarify this over at Unbound-users Info Page

ok thank you :slight_smile:

Hang on, the CN doesnt match the address. What address are you using to connect? The IP address or cloudflare-dns.com?

Though the IP address is listed as SAN, so it actually should still work.