1.1.1.1 high latency Boulder/Denver/Colorado

1.1.1.1 latency vs Google DNS has always been poor on CenturyLink fiber for me in Boulder, CO

Traceroute always showed 1.1.1.1 going to St Louis, MO. Google DNS to California.
1.1.1.1 30ms
8.8.8.8 4ms

I recently noticed a change, 1.1.1.1 got better, 15ms, but goes to Saltlake City. Still google going to California much faster…

Cloudflare DNS

 traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max, 40 byte packets
 1  ring (192.168.42.1)  0.464 ms  0.429 ms  0.407 ms
 2  hlrn-dsl-gw03.hlrn.qwest.net (207.225.112.3)  4.097 ms  2.962 ms  3.402 ms
 3  63-225-124-17.hlrn.qwest.net (63.225.124.17)  8.982 ms  3.800 ms  3.854 ms
 4  * * *
5  ae-6-3.bar1.SaltLakeCity1.Level3.net (4.69.220.109)  16.116 ms  16.123 ms  16.185 ms
6  4.35.175.166 (4.35.175.166)  17.300 ms  16.313 ms  17.532 ms
7  one.one.one.one (1.1.1.1)  16.744 ms  16.182 ms  16.148 ms

Google DNS

# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 40 byte packets
 1  ring (192.168.42.1)  0.747 ms  0.514 ms  0.459 ms
 2  hlrn-dsl-gw03.hlrn.qwest.net (207.225.112.3)  3.403 ms  3.823 ms  4.106 ms
 3  63-225-124-17.hlrn.qwest.net (63.225.124.17)  10.293 ms  3.277 ms  3.850 ms
 4  dvr3-edge-01.inet.qwest.net (205.171.210.9)  6.607 ms  4.305 ms  4.151 ms
 5  72.14.213.218 (72.14.213.218)  4.603 ms  3.872 ms
    72.14.219.162 (72.14.219.162)  3.802 ms
 6  108.170.254.81 (108.170.254.81)  4.363 ms *
    10.252.213.254 (10.252.213.254)  5.061 ms
 7  216.239.49.41 (216.239.49.41)  5.000 ms  4.097 ms
    74.125.251.193 (74.125.251.193)  4.193 ms
 8  dns.google (8.8.8.8)  4.050 ms  3.891 ms  5.044 ms

Latency from Boulder to California at the closest point can’t be lower than 16ms (roundtrip). Google needs to be serving at a closer location. At 4ms has to be Boulder or Denver.

What colo do you see when visiting this URL?
https://1.1.1.1/cdn-cgi/trace

fl=143f26
h=1.1.1.1
ip=97.122.81.188
ts=1605223063.582
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36
colo=SLC
http=http/2
loc=US
tls=TLSv1.3
sni=off
warp=off
gateway=off

let’s measure on the terminal, Boulder to Salt Lake City should be lower than 15ms:

ping -c 10 1.1.1.1
ping -c 10 8.8.8.8
ping -c 10 9.9.9.9

# ping -c 10 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=58 time=14.767 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=58 time=14.954 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=58 time=15.279 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=58 time=15.265 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=58 time=14.910 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=58 time=15.298 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=58 time=15.426 ms
64 bytes from 1.1.1.1: icmp_seq=7 ttl=58 time=14.948 ms
64 bytes from 1.1.1.1: icmp_seq=8 ttl=58 time=15.372 ms
64 bytes from 1.1.1.1: icmp_seq=9 ttl=58 time=14.850 ms

--- 1.1.1.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 14.767/15.107/15.426/0.231 ms

# ping -c 10 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=117 time=5.141 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=3.405 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=3.236 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=117 time=3.675 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=117 time=3.110 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=117 time=3.228 ms
64 bytes from 8.8.8.8: icmp_seq=6 ttl=117 time=3.354 ms
64 bytes from 8.8.8.8: icmp_seq=7 ttl=117 time=3.484 ms
64 bytes from 8.8.8.8: icmp_seq=8 ttl=117 time=3.150 ms
64 bytes from 8.8.8.8: icmp_seq=9 ttl=117 time=3.609 ms

--- 8.8.8.8 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 3.110/3.539/5.141/0.563 ms

# ping -c 10 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: icmp_seq=0 ttl=58 time=30.678 ms
64 bytes from 9.9.9.9: icmp_seq=1 ttl=58 time=30.429 ms
64 bytes from 9.9.9.9: icmp_seq=2 ttl=58 time=30.478 ms
64 bytes from 9.9.9.9: icmp_seq=3 ttl=58 time=30.570 ms
64 bytes from 9.9.9.9: icmp_seq=4 ttl=58 time=30.202 ms
64 bytes from 9.9.9.9: icmp_seq=5 ttl=58 time=30.218 ms
64 bytes from 9.9.9.9: icmp_seq=6 ttl=58 time=30.335 ms
64 bytes from 9.9.9.9: icmp_seq=7 ttl=58 time=30.284 ms
64 bytes from 9.9.9.9: icmp_seq=8 ttl=58 time=30.357 ms
64 bytes from 9.9.9.9: icmp_seq=9 ttl=58 time=30.440 ms

--- 9.9.9.9 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 30.202/30.399/30.678/0.144 ms

BTW that screenshot i posted is a Grafana/InfluxDB graph of Telegraph Pinger input from my pfSense router.

[[inputs.ping]]
    interval = "60s"
    urls = ["1.1.1.1","8.8.8.8","9.9.9.9"]
    deadline = 0

Not sure if its just CenturyLink Fiber:

Closest thing I could find using the Speedtest UI to Saltlake:

Ping time <> DNS lookup time

2 Likes

lol ok. yea, i get that, but having a round trip time 3-4x is not going to help, I wouldn’t think CloudFlare’s DNS servers are 5x faster than Google’s…

# ./dnstest.sh
                  test1   test2   test3   test4   test5   test6   test7   test8   test9   test10  Average
pfsense           1 ms    37 ms   1 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms      4.60
pihole-vm         2 ms    2 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms      1.20
pihole-rpi4       7 ms    2 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms    1 ms      1.70
cloudflare1       28 ms   29 ms   28 ms   29 ms   29 ms   28 ms   28 ms   28 ms   29 ms   30 ms     28.60
cloudflare2       29 ms   29 ms   29 ms   29 ms   29 ms   29 ms   28 ms   28 ms   29 ms   29 ms     28.80
google1           4 ms    5 ms    15 ms   14 ms   15 ms   15 ms   3 ms    14 ms   4 ms    14 ms     10.30
google2           4 ms    4 ms    14 ms   4 ms    25 ms   4 ms    4 ms    14 ms   3 ms    14 ms     9.00
quad9             33 ms   32 ms   31 ms   32 ms   32 ms   166 ms  32 ms   53 ms   32 ms   35 ms     47.80

Hi, you should be able to reach our Denver PoP now.

1 Like

Yep! Look at that drop! Thank you!

Traceroute also looks good and short now!

# traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max, 40 byte packets
 1  ring (192.168.42.1)  0.713 ms  0.643 ms  0.504 ms
 2  hlrn-dsl-gw03.hlrn.qwest.net (207.225.112.3)  3.920 ms  3.643 ms  3.740 ms
 3  * 63-225-124-17.hlrn.qwest.net (63.225.124.17)  4.456 ms  3.754 ms
 4  4.68.38.145 (4.68.38.145)  4.207 ms  4.522 ms  4.669 ms
 5  4.53.3.98 (4.53.3.98)  5.051 ms  5.560 ms  5.789 ms
 6  one.one.one.one (1.1.1.1)  5.261 ms  4.490 ms  4.273 ms