Bad Routing from DTAG in Germany?

Thanks for your Report. I will probably create a Support Ticket this evening because nobody from Cloudflare responded till now

Ticktet’s probably a good idea, thanks!
Just to add though: I‘ve also created a ticket through 1.1.1.1 (their DNS solution) so maybe we can get some attention from multiple sources.

1 Like

I created a Ticket with the Cloudflare Support.
Ticketnumber is 2102224 if anyone from Cloudflare should see this and want to look

3 Likes

This kinda annoying this thread got 20 reply and no one replied from cf.

Yes, let’s see when I get a response to the ticket. They are currently warning of longer waiting times than usual.

@cloonan You are in the CCs for the Ticket. Could you maybe look if there are any new Infos because i didnt recieved any Form of Answer to the Ticket yet. Would be really nice if someone from Cloudflare would say something. At least that they look at it or something like that

Update: I got an update on my ticket. The employee did another traceroute from Amsterdam and Frankfurt and both times he landed in Paris. Cloudflare will check this now.

Hi again,

On checking with our network team, Amsterdam should be back and serving requests:

Router: gin-f2c-thar1
Site: DE, Frankfurt, F2C
Command: traceroute inet4 1.1.1.1 as-number-lookup

traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 52 byte packets
 1  if-ae-2-2.tcore1.fnm-frankfurt.as6453.net (195.219.156.129)  6.651 ms  6.570 ms  6.651 ms
     MPLS Label=39 CoS=0 TTL=1 S=1
 2  if-ae-6-2.tcore1.av2-amsterdam.as6453.net (195.219.194.149)  6.675 ms  7.025 ms *
     MPLS Label=29 CoS=0 TTL=1 S=1
 3  one.one.one.one (1.1.1.1) [AS  13335]  6.726 ms  6.388 ms  7.386 ms

Still working on Frankfurt, that may take a little longer.

1 Like

Awesome, thank you very much for the update and staying on this!

Im still in contact with the Support. It worked for maybe 1 hour the day i reported back and then got routed over Paris again. So the Support is still on it

1 Like

Thanks for the update, I haven’t even heard anything back yet from 1.1.1.1 support.

thank you very much for staying on this issue and getting in contact with cloudflare

I got this reply 2 days ago.

Thanks for your patience here.

The Cloudflare Network engineering team is conducting some tests in the FRA data center so traffic is expected to go to other locations for the time being.

I reopend the Ticket now because the Problem is still there and i dont believe that a Test is the Reason for this.

1 Like

Update: I have now received a response from a Cloudflare Escalation Engineer. Cloudflare is aware of the problem and knows that it has existed for more than 2 months, yet Cloudflare does not have a timetable for when the problem will be fixed.

Hello,

First of all, I’d like to apologize for the routing issue you’re seeing. We have been running tests since early January 2021 to investigate a critical issue within our Frankfurt PoP. As part of this test, we shut off our peering session with Tata in Frankfurt. In the absence of Frankfurt, you can expect traffic to route to our datacenters in Paris or Amsterdam instead. I’m afraid that there’s no ETA right now on when this test may conclude. You are correct that this issue is ongoing and has been ongoing for >2 months.

Sorry for the inconvenience. Please let us know if you have any further questions or concerns.

I’m afraid there is nothing we can do but wait until Cloudflare finds the problem in Frankfurt and re-enables the peering session.

That’s a bit disappointing but thanks for the update.
Could you maybe mention that there is also an issue with the TATA services in Paris, as we have discussed above?
Being routed through Paris shouldn’t be a problem by itself but it seems to have issues as well.

I can confirm this issue as well. It started exactly the second week of January, on 11.2.2021.
I am connected via Unitymedia/Vodafone. I have a smokeping running and there is an increase in latency since week 2 of this year where it jumped from little less than 10ms to an average of almost 20ms.

I am literally in Frankfurt but after TATA it goes to Clouldflare Paris.
Here is my traceroute.

[email protected] ~> traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max, 52 byte packets
 1  goldengate.bunker.marco.cx (10.115.19.1)  4.479 ms  1.540 ms  2.082 ms
 2  * * *
 3  1211f-mx960-01-ae10-1215.dtm.unity-media.net (81.210.132.34)  66.037 ms  134.185 ms  11.176 ms
 4  de-fra01b-rc1-ae-45-0.aorta.net (84.116.196.198)  16.663 ms  14.941 ms  9.847 ms
 5  ro-cj01a-rd3-ae-1-60.aorta.net (84.116.133.25)  151.006 ms  29.347 ms  148.171 ms
 6  213.46.179.138.aorta.net (213.46.179.138)  15.617 ms  12.382 ms  11.253 ms
 7  if-ae-54-2.tcore1.fr0-frankfurt.as6453.net (195.219.219.72)  154.987 ms  143.808 ms  178.002 ms
 8  if-ae-55-2.tcore2.pvu-paris.as6453.net (80.231.245.6)  26.732 ms  34.646 ms  146.722 ms
 9  if-ae-49-2.tcore1.pvu-paris.as6453.net (80.231.153.20)  25.982 ms  37.998 ms  29.404 ms
10  if-ae-11-2.tcore1.pye-paris.as6453.net (80.231.153.50)  30.413 ms  52.480 ms  32.613 ms
11  80.231.154.14 (80.231.154.14)  55.084 ms  29.809 ms  33.163 ms
12  one.one.one.one (1.1.1.1)  33.687 ms  27.958 ms  25.755 ms

I dont really see a Problem other than the increased latency and slower Downstream due to slow Transfer Speeds inside Tatas Network. What Problem do you exactly have again?

This fits to the statement of the Cloudflare employee that they have switched off their peering with Tata in Frankfurt at the beginning of the year to fix a problem which they have not managed so far. From the network of O2 you can reach Cloudflare’s Frankfurt POP normally because it runs over the DECIX and not over Tata. But the DTAG and Vodafone are routing over TATA which ends in Paris, because the disabled Frankfurt Tata Peering

At least Vodafone landline internet connections route via TATA. If I connect via my mobile phone (VF) and do a trace, then it goes straight from VF to DECIX and CF.

I guess it is because your vodafone mobile contract is made with “vodafone deutschland” which has a different peering than Vodafone west (ex unitymedia) that runs your HFC connection when living in baden-württemberg, Nordrhein-Westfalren or Hessen.
Vodafone West is using liberty global infrastructure (Aorta.net) which uses TATA as transit to Cloudflare.

The latency when it jumps to TATA was just really bad, outside of what you’d expect even when taking the longer route (Sometimes seconds for me) but it’s actually not the case anymore so I’ll withdraw that part. :smile: