Latency to reach cloudflace CDN edge servers used to be ~10ms coming from AS3320, now it’s ~100ms
What feature, service or problem is this related to?
DNS records
What are the steps to reproduce the issue?
When I ping the edge servers in cloudflare’s CDN serving my site, e.g., 188.114.97.3 I used to get around 10ms latency coming from a source IP within AS3320. Now I get ~100ms:
[nix-shell:~]$ ping 188.114.97.3
PING 188.114.97.3 (188.114.97.3): 56 data bytes
64 bytes from 188.114.97.3: icmp_seq=0 ttl=159 time=103,654 ms
64 bytes from 188.114.97.3: icmp_seq=1 ttl=160 time=103,175 ms
64 bytes from 188.114.97.3: icmp_seq=3 ttl=162 time=103,344 ms
When I ping the same ip from another german AS AS680 or AS12306 I get ~7ms or ~2ms. And this was formerly also the latency I saw in AS3320. Website access is consequently rather slow currently and way slower than contacting the origin server directly from AS3320.
Looking at traceroute, I see, that coming from AS3320 I’m talking to an edge server in the US, while coming from AS680 or AS12306 I reach an edge server in Frankfurt in germany, as expected.
I know that this can be a problem within AS3320, but shouldn’t the IP be resolved using anycast and BGP? Seams like suddenly the edge server in the US is prefered over those in germany when coming from an IP within AS 3320. This is really suspicious and as stated above way slower than contacting the origin server. Any suggestions?
That is unfortunately a well-known problem, which has been brought up multiple times already, such as e.g. here:
I wouldn’t call it suddenly though.
As you’ve seen latency is better from other ISP’s, that apparently seem to want to participate (more) in creating a better Internet experience for their own customers, cancel your subscriptions with AS3320 Deutsche Telekom and move to these other ISP’s that are doing better.
You can target a specific country (e.g. Germany) with GeoDNS, but you can’t (always) do that in the same way with Anycast and BGP.
Anycast will take you to the closest location based on the network path.
Closest network path isn’t definitive the same as closest physical / geographical distance.
Thank you so much for your quick reply! Unfortunately, that makes total sense… really sad! Now I remember that I heard about similar issues about other big ISP ASes that the Telekom was unwilling to peer with. Honestly I thought we live in 2024 and such issues would not exist anymore, esp. for big relevant AS that are surely used by a lot of Deutsche Telekom customers in AS3320. Not talking about the weirdness that now traffic that could use a local edge server in Germany is routed to the US and back. Anyway… thanks for your reply and really sad… I’ll fix it using DNS split horizon locally I think.
The Internet is unfortunately very weird, especially in this area, - I myself was also in the past under the impression that the ISP’s of course would do what’s best for the connectivity, and their own customers, but unfortunately, I quickly learned that it didn’t turn out that way.
This will (unfortunately) only work around the problem for yourself, and not for e.g. users / customers though.