Can someone confirm this? I have multiple sites on CF and every time the connection is really really slow. However friends from other ISPs and countries do not experience this slowness.
Windows Trace:
C:\Users\alexp>tracert aebian.org
Tracing route to aebian.org [2a06:98c1:3120::3]
over a maximum of 30 hops:
1 2 ms 1 ms 6 ms p200300C1C741Ff8D96988ffffe67420C.dip0.t-ipconnect.de [2003:c1:c741:ff8d:9698:8fff:fe67:420c]
2 8 ms 8 ms 12 ms 2003:0:8800:2800::1
3 113 ms 119 ms 113 ms 2003:3c0:1600:8000::1
4 172 ms * 166 ms 2003:3c0:1600:800a::2
5 187 ms 166 ms * ae-0.r24.asbnva02.us.bb.gin.ntt.net [2001:418:0:2000::269]
6 175 ms 164 ms 161 ms ae-0.a08.asbnva02.us.bb.gin.ntt.net [2001:418:0:2000::2b9]
7 162 ms 164 ms 179 ms 2001:418:0:5000::a9b
8 165 ms 174 ms 166 ms 2400:cb00:352:3::
9 168 ms 164 ms * 2a06:98c1:3120::3
10 * 168 ms 170 ms 2a06:98c1:3120::3
Trace complete.
Mac Trace:
# aebian at Destiny.nethavn in ~ [22:23:49]
→ traceroute6 aebian.org
traceroute6: Warning: aebian.org has multiple addresses; using 2606:4700:3031::6815:4bcc
traceroute6 to aebian.org (2606:4700:3031::6815:4bcc) from 2003:c1:c741:ff8d:28ad:2673:f2a9:4e0c, 64 hops max, 28 byte packets
1 p200300c1c741ff8d96988ffffe67420c.dip0.t-ipconnect.de 2.651 ms 2.372 ms 1.925 ms
2 2003:0:8800:2800::1 8.247 ms 7.931 ms 10.829 ms
3 2003:3c0:1600:8000::1 136.118 ms * 200.704 ms
4 2003:3c0:1600:800a::2 177.453 ms 168.408 ms 170.063 ms
5 *
ae-1.r25.asbnva02.us.bb.gin.ntt.net 164.264 ms
ae-0.r24.asbnva02.us.bb.gin.ntt.net 205.288 ms
6 ae-0.a08.asbnva02.us.bb.gin.ntt.net 169.704 ms 239.376 ms 226.611 ms
7 *
2001:418:0:5000::a9b 194.863 ms
2001:418:0:5000::f13 165.000 ms
8 2400:cb00:601:3:: 188.625 ms
2400:cb00:16:3:: 178.267 ms
2400:cb00:350:3:: 203.598 ms
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
31 * * *
32 * * *
33 * * *
34 * * *
35 * * *
36 * * *
37 * * *
38 * * *
39 * * *
40 * * *
The connection from which this tracert were made, has 100Mbits and according to e.g fast.com this speed is achieved right before and after the tracert.
There was a post from 2021 that said that changes were made and a few people confirmed it working. However I cannot confirm this now in 2024.
If other infos or logs are needed feel free to reach out.
All ISP’s that are a direct subsidiary of AS3320 Deutsche Telekom will be seeing these issues.
In addition, networks that are a customer of AS3320 Deutsche Telekom, but do not have another, more direct route to AS13335 Cloudflare, would similarly be seeing these issues…
If you’re paying anything to AS3320 Deutsche Telekom, I suggest you to complain to them, for providing such bad quality, and if you cannot get them to man up, as quoted above, then find another ISP that isn’t depending on, or related to AS3320 Deutsche Telekom in any way.
When/if AS3320 Deutsche Telekom does not want to ensure proper capacity on their network links, or otherwise do not want to peer (locally) with other networks, then there is absolutely nothing that these other networks (including Cloudflare) can do about it.
Got a link to the specific post you’re referring to?
So, I created a support case with the ISP (German Telekom) and it seems like they partially solved the issue.
The modem didn’t do an update because of some Security Issues they claimed, but they said they will force an update and in-fact, they router did do an update. They also claimed that because of that, a resolve of certain IPs was not working . I highly doubt their statement but what can you do…
The website connection is now much faster than it was before for some of my websites.
Others still take ages to load since I’m still routed over USA to the CF net…
C:\Users\alexp>tracert -6 aebian.org
Tracing route to aebian.org [2606:4700:3035::ac43:b5f8]
over a maximum of 30 hops:
1 1 ms 7 ms 1 ms p200300C1c7070c8c96988FfFfE67420c.dip0.t-ipconnect.de [2003:c1:c707:c8c:9698:8fff:fe67:420c]
2 50 ms 9 ms 14 ms 2003:0:8800:2800::1
3 * * * Request timed out.
4 172 ms 168 ms 160 ms 2003:3c0:1600:800a::2
5 188 ms 161 ms 162 ms ae-1.r25.asbnva02.us.bb.gin.ntt.net [2001:418:0:2000::26b]
6 162 ms 168 ms 161 ms ae-1.a08.asbnva02.us.bb.gin.ntt.net [2001:418:0:2000::2bb]
7 159 ms 162 ms 159 ms 2001:418:0:5000::a9b
8 170 ms 202 ms 175 ms 2400:cb00:354:3::
9 * 161 ms 161 ms 2606:4700:3035::ac43:b5f8
Guess I’ll have to create another support request with them. They however called me really fast, so at least that is good. Probably gonna create another tomorrow afternoon.
By looking at the traceroutes shared over there, nothing actually changed in the path from AS5483 Magyar Telekom towards AS13335 Cloudflare.
Routing is however two-way, so it could have been nice to the path in the other direction (AS13335 Cloudflare towards AS5483 Magyar Telekom), as the path there could be completely different from the other direction.
Here it sounds like they’re talking about the modem / router located on your address?
There’s nothing in that which can be the culprit to the current issue, with the routing to the United States.
I know the game very well, with those incumbents as well as other companies that are anywhere from tough to impossible, to “work together with”.
The modem / router, or any other kind of “gear” located on your address, will NOT be the changing anything in regards to the peering and/or routing policies of AS3320 Deutsche Telekom.
So yeah, anything you (or they) do there, won’t be changing anything in regards to the routing to the United States.
As the domain name you mentioned is matching your nickname “Aebian”, it somehow makes me think a personal domain?
Do all these multiple sites" you refer to, use the Free plan from Cloudflare?
Yeah, modem is local at my apartment, thats what they remote-updated.
Afaik they did nothing on their side.
Oh yeah, it is my personal blog and my Exchange Online has this as additional domain as well for mail services. Somewhat of a domain for personal / private stuff.
Yup, all of them. And they are using the Tunnel via cloudflared (CF One).
another example domain would be groupwiki.nethavn.group
The most ridiculous thing is, that their support case form is only allowing for 600 characters at most, so I can’t provide detailed information there… I hope I can add links there so I put most of the info in a Github gist hoping they know what Github is enough to click that link then.
Will create a new case tomorrow with them and see how this goes.
Trying to fix a switch / edge gateway side issue by updating a client, yeah suuuure.
Just for the Information sake. I re-opened the ISP Support request again and provided information via GitHub Gist in very detail. I also included a working traceroute from a non-Telekom connection to show them the real fix
Now I have to wait and see what the person gets back with.
I’ll keep y’all posted.
So I had a long call with the Telekom again. This time I got another agent on the phone.
She described, that the issue faced would be due to different security standards that Telekom and Cloudflare use.
The first option she gave was “Maybe you can use a VPN and check if it yields better results”.
I then told her, that this would be only a workaround and a a person would need a VPN-Provider to actually test this workaround.
She understood that this is requiring additional services not everyone wants/have.
I then directed here to the GitHub Gist link I provided in the contact form (she didn’t find that at first) and she told me that this will now be sent to the 3rdLevel Support to check for options.
Fingers crossed that nothing happens. (Oh did I say that out loud?)
I’ll keep you posted again.
Hi Guys, I wanted to update you on the issue described in the link above. Since the topic was auto-closed, here is a new topic with an update to it.
I got a phone call this morning where a guy from Telekom Support was saying that the issue has been discussed with the back-office and no changes will be made in their network.
According to them, their system thinks that the peer in the US is the faster one (albeit it isn’t) and they still blame it on CF needing to do a fix…
I have in the meantime also filled a complaint with the authorities and will likely switch to a different ISP in the near future.
The issue is obviously not solved, but what can you do if your ISP is stupid enough to resist clear evidence that the problem is on their side…
I have to see what the authorities will do about it, I explained the issue and very detailed depth and await their feedback.
What steps have you taken to resolve the issue?
Filed a complaint with the German federal authority for telecommunication (Bundesnetzagentur)
What feature, service or problem is this related to?
DNS records
What are the steps to reproduce the issue?
Be a customer of the German Telekom and try to access a site proxied trough CF and get routed via NYC instead of DUS or FRA.
I’m situated in southern Bavaria and rely on CF tunnels to provide my self hosted websites.
I’m experiencing a similar issue where my websites are quick and responsive outside my home’s WLAN, but when I access the Internet (via Telekom), the ping is 100 ms or more. Pinging via a hotspot connection shows ~20 ms.
I also appear to get re-routed via nyc.us.net.dtag.de from home, but muc.de.net.telefonica.de when using my hotspot connection.
Any guidance and/or result from this? I would also like to complain if it helps.
Well, I complained via the Bundesnetzagentur. I posted clear evidence that Telekom is the issue, but Telekom responded to the claims stating it is Cloudflare who is at fault here:
I just switched from CF Free plan to Pro (wanted to try it for a year anyhow), and now my Website is reachable with a <10ms ping… Maybe Telekom is not at fault after all?
I would guess that your IP address of the website endpoint changed and this gets routed differently from the Telekom network.
Both the ISP of my friends house and the company I work for route all my free websites just fine, only Telekom causes troubles. After all a tracert clearly shows Telekom at fault here…
My traceroute used to result in (under CF Free plan):
1 192.168.1.1 (192.168.1.1) 1.788 ms 1.216 ms 1.001 ms
2 p3e9bf208.dip0.t-ipconnect.de (62.155.242.8) 9.782 ms 8.086 ms 8.470 ms
3 nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202) 103.429 ms 103.617 ms 104.034 ms
4 nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202) 104.725 ms 102.995 ms 150.559 ms
5 80.156.160.213 (80.156.160.213) 96.596 ms 94.750 ms
if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14) 101.308 ms
6 if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14) 100.893 ms
66.198.70.2 (66.198.70.2) 105.143 ms
if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14) 101.325 ms
7 66.198.70.2 (66.198.70.2) 104.038 ms *
162.158.61.101 (162.158.61.101) 108.559 ms
8 188.114.96.3 (188.114.96.3) 103.994 ms 103.904 ms 104.077 ms
and is now (under CF Pro plan):
1 192.168.1.1 (192.168.1.1) 18.692 ms 1.210 ms 2.729 ms
2 p3e9bf208.dip0.t-ipconnect.de (62.155.242.8) 10.450 ms 7.656 ms 7.308 ms
3 m-ef2-i.m.de.net.dtag.de (62.153.181.86) 10.090 ms 9.845 ms 9.286 ms
4 80.150.168.185 (80.150.168.185) 15.129 ms 15.335 ms 16.421 ms
5 cloudflare-gw.cr0-muc1.ip4.gtt.net (141.136.100.98) 10.287 ms 9.058 ms 9.179 ms
6 104.26.14.172 (104.26.14.172) 8.813 ms 9.420 ms 9.054 ms
I used to get (and still do for some of my websites under the CF Free plan):
1 192.168.1.1 (192.168.1.1) 1.788 ms 1.216 ms 1.001 ms
2 p3e9bf208.dip0.t-ipconnect.de (62.155.242.8) 9.782 ms 8.086 ms 8.470 ms
3 nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202) 103.429 ms 103.617 ms 104.034 ms
4 nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202) 104.725 ms 102.995 ms 150.559 ms
5 80.156.160.213 (80.156.160.213) 96.596 ms 94.750 ms
if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14) 101.308 ms
6 if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14) 100.893 ms
66.198.70.2 (66.198.70.2) 105.143 ms
if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14) 101.325 ms
7 66.198.70.2 (66.198.70.2) 104.038 ms *
162.158.61.101 (162.158.61.101) 108.559 ms
8 188.114.96.3 (188.114.96.3) 103.994 ms 103.904 ms 104.077 ms
and now (for my website under the CF Pro plan) I get:
1 192.168.1.1 (192.168.1.1) 18.692 ms 1.210 ms 2.729 ms
2 p3e9bf208.dip0.t-ipconnect.de (62.155.242.8) 10.450 ms 7.656 ms 7.308 ms
3 m-ef2-i.m.de.net.dtag.de (62.153.181.86) 10.090 ms 9.845 ms 9.286 ms
4 80.150.168.185 (80.150.168.185) 15.129 ms 15.335 ms 16.421 ms
5 cloudflare-gw.cr0-muc1.ip4.gtt.net (141.136.100.98) 10.287 ms 9.058 ms 9.179 ms
6 104.26.14.172 (104.26.14.172) 8.813 ms 9.420 ms 9.054 ms
I don’t get why there is a reroute to NYC for some sites and not for the other. If it was 240 USD to get this problem fixed, then that’s more than a subpar solution in my opinion.
Yes but the problem only seems to arise on Telekom. My friends on O2 or Vodafone do not report any of those routing problems to my websites on a free plan.
Same goes for the connection in my company. If it would be a CF issue then all sites and ISPs would be affected, but they aren’t.
I mean the routing happens from Telekom side, the two lines after your router are directly from Telekom:
p3e9bf208.dip0.t-ipconnect.de (62.155.242.8) 9.782 ms 8.086 ms 8.470 ms
3 nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202) 103.429 ms 103.617 ms 104.034 ms
Which makes me wonder how CF would cause this? If Telekom routes badly, nothing CF can do…
