guyp2k
December 12, 2019, 3:11pm
1
After I day of troubleshooting I can’t seem to address the issue below:
Dec 12 09:03:52 mintvm cloudflared[5706]: time=“2019-12-12T09:03:52-06:00” level=error msg=“failed to connect to an HTTPS backend "https://1.1.1.1/dns-query\ ”" error=“failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: x509: cannot validate certificate for 1.1.1.1 because it doesn’t contain any IP SANs”
any suggestions or direction?
Thanks
sandro
December 12, 2019, 3:13pm
2
Are you sure you are actually reaching Cloudflare and they address hasnt been hijacked by your ISP? The latter would seem most likely in this case.
What does a traceroute to 1.1.1.1 say?
Do you also have issues with 1.0.0.1?
2 Likes
guyp2k
December 12, 2019, 3:48pm
3
Yes, it’s both 1.1.1.1 and 1.0.0.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max
1 192.168.1.1 0.951ms 0.575ms 0.512ms
2 x.x.x.x 13.294ms 13.973ms 10.064ms
3 x.x.x.x 13.542ms 9.263ms 16.162ms
4 x.x.x.x 17.711ms 13.876ms 14.671ms
5 x.x.x.x 16.519ms 15.260ms 14.436ms
6 1.1.1.1 16.219ms 14.473ms 15.167ms
guyp@mintvm:~$ traceroute 1.0.0.1
traceroute to 1.0.0.1 (1.0.0.1), 64 hops max
1 192.168.1.1 0.864ms 0.638ms 0.615ms
2 x.x.x.x 11.873ms 9.650ms 11.503ms
3 x.x.x.x 11.594ms 10.502ms 12.996ms
4 x.x.x.x 18.051ms 15.343ms 15.041ms
5 x.x.x.x 17.405ms 17.677ms 18.375ms
6 1.0.0.1 16.878ms 14.431ms 15.688ms
Thanks
sandro
December 12, 2019, 3:52pm
4
Censoring debug output is not really helpful. Post the actual traceroutes. But it would seem very much as if your ISP hijacked the addresses. You should probably contact your ISP.
guyp2k
December 12, 2019, 4:00pm
5
My ISP is comcast and if I use 1.1.1.1 as my normal DNS no issues. Below is my config and dig command if that is helpful:
; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @127.0.0.1 -p 5053 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 37665
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 611e4ff4503dc91e (echoed)
;; QUESTION SECTION:
;google.com . IN A
;; Query time: 241 msec
;; SERVER: 127.0.0.1#5053(127.0.0.1)
;; WHEN: Thu Dec 12 09:56:13 CST 2019
;; MSG SIZE rcvd: 51
CLOUDFLARED_OPTS=–port 5053 --upstream https://1.1.1.1/dns-query --upstream https://1.0.0.1/dns-query
Thanks
sandro
December 12, 2019, 4:03pm
6
You need to post the traceroute.
sandro
December 12, 2019, 4:09pm
8
That does seem as if your ISP hijacked these addresses. All of that is still within their networks.
What do these commands return?
dig +short CHAOS TXT id.server @1.1.1.1
dig @1.1.1.1 cloudflare.com
openssl s_client -connect 1.1.1.1:443
sandro
December 12, 2019, 4:16pm
10
Well, yes you do seem to connect to Cloudflare, however the certificate does return the right configuration as well.
-----BEGIN CERTIFICATE-----
MIIFxjCCBUygAwIBAgIQAczjGN6fVn+rKySQH62nHTAKBggqhkjOPQQDAjBMMQsw
CQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSYwJAYDVQQDEx1EaWdp
Q2VydCBFQ0MgU2VjdXJlIFNlcnZlciBDQTAeFw0xOTAxMjgwMDAwMDBaFw0yMTAy
MDExMjAwMDBaMHIxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYw
FAYDVQQHEw1TYW4gRnJhbmNpc2NvMRkwFwYDVQQKExBDbG91ZGZsYXJlLCBJbmMu
MRswGQYDVQQDExJjbG91ZGZsYXJlLWRucy5jb20wWTATBgcqhkjOPQIBBggqhkjO
PQMBBwNCAATFIHCMIEJQKB59REF8MHkpHGNeHUSbxfdxOive0qKksWw9ash3uMuP
LlBT/fQYJn9hN+3/wr7pC125fuHfHOJ0o4ID6DCCA+QwHwYDVR0jBBgwFoAUo53m
H/naOU/AbuiRy5Wl2jHiCp8wHQYDVR0OBBYEFHCV3FyjjmYH28uBEMar58OoRX+g
MIGsBgNVHREEgaQwgaGCEmNsb3VkZmxhcmUtZG5zLmNvbYIUKi5jbG91ZGZsYXJl
LWRucy5jb22CD29uZS5vbmUub25lLm9uZYcEAQEBAYcEAQAAAYcEop+ENYcQJgZH
AEcAAAAAAAAAAAAREYcQJgZHAEcAAAAAAAAAAAAQAYcQJgZHAEcAAAAAAAAAAAAA
ZIcQJgZHAEcAAAAAAAAAAABkAIcEop8kAYcEop8uATAOBgNVHQ8BAf8EBAMCB4Aw
HQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGkGA1UdHwRiMGAwLqAsoCqG
KGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9zc2NhLWVjYy1nMS5jcmwwLqAsoCqG
KGh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9zc2NhLWVjYy1nMS5jcmwwTAYDVR0g
BEUwQzA3BglghkgBhv1sAQEwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGln
aWNlcnQuY29tL0NQUzAIBgZngQwBAgIwewYIKwYBBQUHAQEEbzBtMCQGCCsGAQUF
BzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wRQYIKwYBBQUHMAKGOWh0dHA6
Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEVDQ1NlY3VyZVNlcnZlckNB
LmNydDAMBgNVHRMBAf8EAjAAMIIBfgYKKwYBBAHWeQIEAgSCAW4EggFqAWgAdgCk
uQmQtBhYFIe7E6LMZ3AKPDWYBPkb37jjd80OyA3cEAAAAWiVHhSLAAAEAwBHMEUC
IQDlnoPeMXtFkRsy3Vs0eovk3ILKt01x6bgUdMlmQTFIvAIgcAn0lFSjiGzHm2eO
jDZJzMiP5Uaj0Jwub9GO8RkxkkoAdQCHdb/nWXz4jEOZX73zbv9WjUdWNv9KtWDB
tOr/XqCDDwAAAWiVHhVsAAAEAwBGMEQCIFC0n0JModeol8b/Qicxd5Blf/o7xOs/
Bk0j9hdc5N7jAiAQocYnHL9iMqTtFkh0vmSsII5NbiakM/2yDEXnwkPRvAB3ALvZ
37wfinG1k5Qjl6qSe0c4V5UKq1LoGpCWZDaOHtGFAAABaJUeFJEAAAQDAEgwRgIh
AL3OPTBzOZpS5rS/uLzqMOiACCFQyY+mTJ+L0I9TcB3RAiEA4+SiPz0/5kFxvrk7
AKYKdvelgV1hiiPbM2YHY+/0BIkwCgYIKoZIzj0EAwIDaAAwZQIwez76hX2HTMur
/I3XRuwfdmVoa8J6ZVEVq+AZsE7DyQh7AV4WNLU+092BrPbnyVUFAjEAzUf5jdz1
pyc74lgOunC7LBE6cPtWbzfGpJiYyT/T+c5eIAwRYziKT0DKbaql7tiZ
-----END CERTIFICATE-----
guyp2k
December 12, 2019, 4:17pm
11
Could this be a FW issue going over port 5053??
sandro
December 12, 2019, 4:20pm
12
How is a firewall and that port related here? The original error message only referred to the URL in question. If you do some rewriting or have some MITM setup, that could of course change things, but that would be down to your setup.
guyp2k
December 12, 2019, 4:23pm
13
sandro:
MITM setup
No proxy, just a VM running Linux Mint 19.2…
sandro
December 12, 2019, 4:25pm
14
Then what about the firewall and port 5053?
The OpenSSL output is in stark contrast to what the original error said. It connects properly and does return the right certificate. Thats cloudflared in the first message, right?
guyp2k
December 12, 2019, 4:26pm
15
Yes, it is cloudflared in the first message.
Thanks
guyp2k
December 12, 2019, 4:42pm
17
Yup I think it was a FW issue and seems to be working now, doh!
root@mintvm:/etc# dig @127.0.0.1 -p 5053 google.com
; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @127.0.0.1 -p 5053 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12051
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1452
; PAD: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 (“…”)
;; QUESTION SECTION:
;google.com . IN A
;; ANSWER SECTION:
google.com . 289 IN A 64.233.177.101
google.com . 289 IN A 64.233.177.102
google.com . 289 IN A 64.233.177.113
google.com . 289 IN A 64.233.177.138
google.com . 289 IN A 64.233.177.139
google.com . 289 IN A 64.233.177.100
;; Query time: 16 msec
;; SERVER: 127.0.0.1#5053(127.0.0.1)
;; WHEN: Thu Dec 12 10:42:07 CST 2019
;; MSG SIZE rcvd: 316
sandro
December 12, 2019, 4:46pm
18
That shouldnt be related to port 5053, however. If you had blocked cloudflared from accessing the Internet, it would certainly explain why it couldnt validate the certificate, because it couldnt connect in the first place, however if that was the case, the error message would be quite misleading. That wouldnt have been an invalid certificate but rather a timed out or reset connection.
guyp2k
December 12, 2019, 4:49pm
19
Understood, to test I created a pass rule for the host until I can determine specifically what was/is occurring and setup the ACLs accordingly.
Thanks
system
Closed
January 12, 2020, 9:12pm
20
This topic was automatically closed after 31 days. New replies are no longer allowed.