Degraded latency performance in DFW

cdn
bug

#1

In the Dallas/Ft. Worth (USA) area on AT&T Internet, and I would always get pings to 1.1.1.1 in ~10ms, now it’s consistently ~40ms as of a couple of weeks ago. I’ve noticed in traceroutes and the help page that I seem to get rerouted to PHX cloud centers now, used to be DFW.

Debug info: https://1.1.1.1/help#eyJpc0NmIjoiWWVzIiwiaXNEb3QiOiJObyIsImlzRG9oIjoiTm8iLCJyZXNvbHZlcklwLTEuMS4xLjEiOiJZZXMiLCJyZXNvbHZlcklwLTEuMC4wLjEiOiJZZXMiLCJyZXNvbHZlcklwLTI2MDY6NDcwMDo0NzAwOjoxMTExIjoiTm8iLCJyZXNvbHZlcklwLTI2MDY6NDcwMDo0NzAwOjoxMDAxIjoiTm8iLCJkYXRhY2VudGVyTG9jYXRpb24iOiJQSFgiLCJpc3BOYW1lIjoiQ2xvdWRmbGFyZSIsImlzcEFzbiI6IjEzMzM1In0=

Traceroutes:

traceroute to 1.1.1.1 (1.1.1.1), 64 hops max, 72 byte packets
 1  192.168.1.1 (192.168.1.1)  1.851 ms  0.885 ms  0.878 ms
 2  adsl-108-223-131-254.dsl.rcsntx.sbcglobal.net (108.223.131.254)  13.878 ms  9.154 ms  8.408 ms
 3  12.83.34.249 (12.83.34.249)  11.037 ms  10.661 ms  11.839 ms
 4  12.123.18.233 (12.123.18.233)  17.460 ms  14.883 ms  16.062 ms
 5  192.205.36.206 (192.205.36.206)  11.172 ms  10.692 ms  10.038 ms
 6  if-ae-31-2.tcore1.un0-phoenix.as6453.net (66.110.56.206)  33.535 ms  33.835 ms  33.629 ms
 7  66.110.80.19 (66.110.80.19)  41.921 ms  41.730 ms  41.627 ms
 8  one.one.one.one (1.1.1.1)  41.189 ms  40.256 ms  40.975 ms

traceroute to 1.0.0.1 (1.0.0.1), 64 hops max, 72 byte packets
 1  192.168.1.1 (192.168.1.1)  1.991 ms  0.897 ms  0.827 ms
 2  adsl-108-223-131-254.dsl.rcsntx.sbcglobal.net (108.223.131.254)  8.906 ms  9.006 ms  8.895 ms
 3  12.83.34.249 (12.83.34.249)  9.977 ms  10.736 ms  11.526 ms
 4  12.123.18.233 (12.123.18.233)  17.152 ms  13.368 ms  16.467 ms
 5  192.205.36.206 (192.205.36.206)  11.008 ms  11.375 ms  11.244 ms
 6  if-ae-31-2.tcore1.un0-phoenix.as6453.net (66.110.56.206)  34.853 ms  33.891 ms  33.927 ms
 7  66.110.80.19 (66.110.80.19)  41.931 ms  41.955 ms  42.424 ms
 8  one.one.one.one (1.0.0.1)  41.227 ms  41.842 ms  40.881 ms

Cloudflare sites seem to be routing to DFW properly, so this seems to just be affecting 1.1.1.1/1.0.0.1:

fl=15f115
ts=1545682000.575
uag=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
colo=DFW
loc=US
tls=TLSv1.3

Edit: Now it looks like it’s affecting some Cloudflare sites too:

fl=66f5
ts=1545945918.025
uag=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
colo=PHX
loc=US
tls=TLSv1.3

High pings 40ms+ to 1.1.1.1 in Melbourne AUS/Internode
#2

What does this say?

dig +short CHAOS TXT id.server @1.1.1.1

#3

Says “PHX” now for both 1.1.1.1 and 1.0.0.1. Used to be “DFW” before.


#4

I’m guessing there’s some network latency between you and DFW, so PHX is faster for now.


#5

After escalating through a support ticket, Cloudflare contacted an upstream provider and the routing has been fixed.