[SOLVED] Unbound configuration not connecting to [email protected] for DNS over TLS


#1

I have a server running a Unbound (unbound.org) DNS forwarder.
It works when connected to quad9’s DNS-over-TLS server [email protected], and fails when connected to [email protected]

It seems to fail at the initial TCP setup… Any ideas?
unbound 13571 13572 unbound 4u IPv4 1101199 0t0 TCP *:domain (LISTEN)
unbound 13571 13572 unbound 5u IPv4 1101200 0t0 TCP 127.0.0.1:ub-dns-control (LISTEN)
unbound 13571 13572 unbound 17u IPv4 1101486 0t0 TCP 23.239.0.79:49482->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 18u IPv4 1101487 0t0 TCP 23.239.0.79:49484->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 19u IPv4 1104082 0t0 TCP 23.239.0.79:49468->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 20u IPv4 1104083 0t0 TCP 23.239.0.79:49470->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 21u IPv4 1104152 0t0 TCP 23.239.0.79:49472->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 22u IPv4 1104153 0t0 TCP 23.239.0.79:49474->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 23u IPv4 1104081 0t0 TCP 23.239.0.79:49466->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 24u IPv4 1104154 0t0 TCP 23.239.0.79:49476->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 25u IPv4 1101476 0t0 TCP 23.239.0.79:49478->1.1.1.1:853 (SYN_SENT)
unbound 13571 13572 unbound 26u IPv4 1101485 0t0 TCP 23.239.0.79:49480->1.1.1.1:853 (SYN_SENT)


#2

s/unbound.org/unbound.net/


#4

I’m probably missing something silly, but here’s the failing TCP handshake to [email protected]:

23:42:38.504684 IP (tos 0x0, ttl 64, id 17979, offset 0, flags [DF], proto TCP (6), length 60)
    23.239.0.79.49416 > 1.1.1.1.853: Flags [S], cksum 0x1a6e (incorrect -> 0x5685), seq 1350584707, win 29200, options [mss 1460,sackOK,TS val 1292745148 ecr 0,nop,wscale 7], length 0
	0x0000:  0000 0c9f f00e f23c 91db 1f7f 0800 4500  .......<......E.
	0x0010:  003c 463b 4000 4006 da41 17ef 004f 0101  .<F;@[email protected]
	0x0020:  0101 c108 0355 5080 4983 0000 0000 a002  .....UP.I.......
	0x0030:  7210 1a6e 0000 0204 05b4 0402 080a 4d0d  r..n..........M.
	0x0040:  b9bc 0000 0000 0103 0307                 ..........
23:42:38.505505 IP (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    1.1.1.1.853 > 23.239.0.79.49416: Flags [S.], cksum 0x280f (correct), seq 1449133533, ack 1350584708, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 10], length 0
	0x0000:  f23c 91db 1f7f 8478 ac0d 79c1 0800 4500  .<.....x..y...E.
	0x0010:  0034 0000 4000 3d06 2385 0101 0101 17ef  [email protected]=.#.......
	0x0020:  004f 0355 c108 5660 05dd 5080 4984 8012  .O.U..V`..P.I...
	0x0030:  7210 280f 0000 0204 05b4 0101 0402 0103  r.(.............
	0x0040:  030a                                     ..
23:42:38.505544 IP (tos 0xc0, ttl 64, id 8531, offset 0, flags [none], proto ICMP (1), length 80)
    23.239.0.79 > 1.1.1.1: ICMP 23.239.0.79 tcp port 49416 unreachable, length 60
	IP (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    1.1.1.1.853 > 23.239.0.79.49416: Flags [S.], cksum 0x280f (correct), seq 1449133533, ack 1350584708, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 10], length 0
	0x0000:  0000 0c9f f00e f23c 91db 1f7f 0800 45c0  .......<......E.
	0x0010:  0050 2153 0000 4001 3e5b 17ef 004f 0101  [email protected]>[...O..
	0x0020:  0101 0303 1763 0000 0000 4500 0034 0000  .....c....E..4..
	0x0030:  4000 3d06 2385 0101 0101 17ef 004f 0355  @.=.#........O.U
	0x0040:  c108 5660 05dd 5080 4984 8012 7210 280f  ..V`..P.I...r.(.
	0x0050:  0000 0204 05b4 0101 0402 0103 030a       ..............

And for comparison, here’s the working handshake to [email protected]:

22:53:54.158593 IP (tos 0x0, ttl 64, id 19426, offset 0, flags [DF], proto TCP (6), length 60)
    23.239.0.79.45068 > 9.9.9.9.853: Flags [S], cksum 0x2a7e (incorrect -> 0x5655), seq 1276110626, win 29200, options [mss 1460,sackOK,TS val 508973153 ecr 0,nop,wscale 7], length 0
	0x0000:  4500 003c 4be2 4000 4006 c48a 17ef 004f  E..<[email protected]@......O
	0x0010:  0909 0909 b00c 0355 4c0f e722 0000 0000  .......UL.."....
	0x0020:  a002 7210 2a7e 0000 0204 05b4 0402 080a  ..r.*~..........
	0x0030:  1e56 5061 0000 0000 0103 0307            .VPa........
22:53:54.168133 IP (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    9.9.9.9.853 > 23.239.0.79.45068: Flags [S.], cksum 0x3ae9 (correct), seq 3619463078, ack 1276110627, win 28960, options [mss 1460,sackOK,TS val 4081693084 ecr 508973153,nop,wscale 8], length 0
	0x0000:  4500 003c 0000 4000 3d06 136d 0909 0909  E..<[email protected]=..m....
	0x0010:  17ef 004f 0355 b00c d7bc 9fa6 4c0f e723  ...O.U......L..#
	0x0020:  a012 7120 3ae9 0000 0204 05b4 0402 080a  ..q.:...........
	0x0030:  f349 b19c 1e56 5061 0103 0308            .I...VPa....
22:53:54.168169 IP (tos 0x0, ttl 64, id 19427, offset 0, flags [DF], proto TCP (6), length 52)
    23.239.0.79.45068 > 9.9.9.9.853: Flags [.], cksum 0x2a76 (incorrect -> 0xd9e7), seq 1, ack 1, win 229, options [nop,nop,TS val 508973163 ecr 4081693084], length 0
	0x0000:  4500 0034 4be3 4000 4006 c491 17ef 004f  [email protected]@......O
	0x0010:  0909 0909 b00c 0355 4c0f e723 d7bc 9fa7  .......UL..#....
	0x0020:  8010 00e5 2a76 0000 0101 080a 1e56 506b  ....*v.......VPk
	0x0030:  f349 b19c                                .I..

#5

Quite a few AS’s not working for 1.1.1.1 Check out status from 180 global cities :- https://rzrds.share.thousandeyes.com


#6

And yet regular DNS via UDP on [email protected] seems to work fine.


#7

I’m very near L.A., and my Spectrum connection doesn’t let me reach 1.1.1.1 under any circumstances. The Thousandeyes map says LA is working. I have a ticket open with Spectrum to fix this.


#8

@sdayman - we leverage RouteViews.org route monitors - specifically - cenic.net in the LA area. I suspect that many transit / ISP’s have excluded this subnet from their RIB’s.


#9

Can you telnet to 1.1.1.1 and/or 1.0.0.1 on 853?


#10

From 23.239.0.79:
# openssl s_client -connect 1.0.0.1:853 <-- This works.
(so 1.0.0.1 is fine, only 1.1.1.1 is unhappy)
# openssl s_client -connect 1.1.1.1:853 <-- This hangs.

# nmap -sS 1.1.1.1 <-- TCP SYN scan shows 53/tcp, 443/tcp, 853/tcp open
# nmap -sT 1.1.1.1 <-- TCP connect scan hangs

Thank you cscharff for looking into it on a busy monday :slight_smile:


#11

If you point unbound to 1.0.0.1 does it work?

I’m not sure why you can’t hit 1.1.1.1 on 853 (network issue most likely) but just to confirm that you are hitting Cloudflare’s DNS at 1.1.11 can you run these commands:

dig cnn.com @1.1.1.1 ANY
dig cnn.com @1.0.0.1 ANY

You should see “status: NOTIMP,” in the responses from both. If you do… then yeah… network something, something. If 1.1.1.1 returns a different answer then something else is actually listening /capturing 1.1.1.1 DNS queries (in that case can you do a traceroute to 1.1.1.1)?


#12

Yes, if I point unbound to 1.0.0.1 it works fine.

Because any connection from my host to any TCP port on 1.1.1.1 hangs (malformed SYN_ACK above?), the dig command hangs. (didn’t we deprecate ANY? :slight_smile: )
# dig cnn.com @1.1.1.1 ANY <- hangs
# dig cnn.com @1.0.0.1 ANY <-- works fine

# traceroute -n 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
 1  23.92.24.3  0.874 ms  0.907 ms 23.92.24.2  0.590 ms
 2  173.230.159.10  1.879 ms  1.876 ms 173.230.159.0  0.791 ms
 3  206.223.116.237  0.989 ms  0.982 ms 173.230.159.9  0.744 ms
 4  206.223.116.237  0.734 ms * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
#```

That last hop is equinix-sanjose.as13335.net. in California.
23.239.0.79 is in Fremont, CA.

#13

Yeah… looks like a network issue for you unfortunately. But since 1.0.0.1 works you can still use us. :slight_smile:


#14

Thanks, will do.


#15

Hi.

If you are able to connect to 1.0.0.1, but not to 1.1.1.1 it could be “bug” of some Cisco routers. As I understand, Cisco still uses address 1.1.1.1 as something like configuration address in some models. It has historical reason, because 1.0.0.0/8 had been assigned to APNIC just 8 years ago. Check “cisco 1.1.1.1” on Google for more details.


#16

My bad. It was an errant rule on the local IPTABLES filter that was blocking 1.1.1.1 packets. Sorry, folks.


#17

I have a similar issue, but it is not firewall issue.

1.0.0.1 responds with THROWAWAY

this doesn’t happen for all hosts

here is my log from unbound:

# Cloudflare 1.0.0.1
[1523215952] unbound[69168:0] info: iterator operate: query google.com. ANY IN
[1523215952] unbound[69168:0] info: response for google.com. ANY IN
[1523215952] unbound[69168:0] info: reply from <.> 1.0.0.1#853
[1523215952] unbound[69168:0] info: query response was THROWAWAY
[1523215952] unbound[69168:0] info: processQueryTargets: google.com. ANY IN
[1523215952] unbound[69168:0] info: sending query: google.com. ANY IN
[1523215952] unbound[69168:0] debug: sending to target: <.> 1.0.0.1#853

# Quad 9
[1523216876] unbound[69168:0] info: iterator operate: query google.com. ANY IN
[1523216876] unbound[69168:0] info: response for google.com. ANY IN
[1523216876] unbound[69168:0] info: reply from <.> 9.9.9.9#853
[1523216876] unbound[69168:0] info: query response was ANSWER
[1523216876] unbound[69168:0] info: finishing processing for google.com. ANY IN
[1523216876] unbound[69168:0] debug: validator[module 0] operate: extstate:module_wait_module event:module_event_moddone
[1523216876] unbound[69168:0] info: validator operate: query google.com. ANY IN
[1523216876] unbound[69168:0] debug: validator[module 0] operate: extstate:module_state_initial event:module_event_pass
[1523216876] unbound[69168:0] info: validator operate: query google.com. DS IN
[1523216876] unbound[69168:0] debug: iterator[module 1] operate: extstate:module_state_initial event:module_event_pass
[1523216876] unbound[69168:0] info: resolving google.com. DS IN
[1523216876] unbound[69168:0] info: processQueryTargets: google.com. DS IN
[1523216876] unbound[69168:0] info: sending query: google.com. DS IN
[1523216876] unbound[69168:0] debug: sending to target: <.> 9.9.9.9#853

#18

I think “ANY” has been deprecated at cloudflare, for security
reasons.


#19

Can confirm.


#20

Thanks! That explains it, I was checking with

host -a google.com

which is the same as

host -v -t ANY google.com

My next question is off topic, but I’m getting about 150ms for the following request, when it is non-cached:

host -v -t A google.com

Is it OK, what do you think?

Thanks!


#21

Sorry for the late reply. It seems strange a 150ms response time for google.com! Can you tell me where you are and run a ping and a traceroute towards 1.1.1.1 and 1.0.0.1? Do the same host command towards 1.0.0.1 as well.