Wrong routing from Wind Infostrada (AS1267)

Hi all.
I have a Wind Infostrada (AS1267) connection, I’m in Napoli (Italy) and all the traffic is correctly routed to Rome on NAMEX (NAP).
I generally have a 6ms of latency to all Cloudflare DNS and websites.
Starting from today, 1.1.1.1 seems to be routed to Milan and Cloudflare sites to Amsterdam.
Now I have more then 20ms of latency to 1.1.1.1 and 35ms of latency to Cloudflare websites.

See traceroute to 1.1.1.1

[email protected]:~# traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
1 192.168.1.1 (192.168.1.1) 0.738 ms 0.407 ms 0.462 ms
2 151.6.154.21 (151.6.154.21) 2.446 ms 2.293 ms 2.140 ms
3 151.6.240.10 (151.6.240.10) 1.988 ms 1.812 ms 1.661 ms
4 151.6.104.146 (151.6.104.146) 1.526 ms 2.202 ms 2.069 ms
5 151.6.1.208 (151.6.1.208) 27.536 ms 151.6.4.205 (151.6.4.205) 5.930 ms 151.6.1.208 (151.6.1.208) 27.272 ms
6 151.6.1.15 (151.6.1.15) 14.234 ms 151.6.0.8 (151.6.0.8) 15.205 ms 151.6.1.13 (151.6.1.13) 14.585 ms
7 151.6.1.180 (151.6.1.180) 15.314 ms 151.6.6.53 (151.6.6.53) 15.430 ms 151.6.1.176 (151.6.1.176) 14.595 ms
8 * * *
9 one.one.one.one (1.1.1.1) 19.837 ms 19.674 ms 19.543 ms

Traceroute to 104.31.69.91 (Cloudflare IP of a Cloudflare website):
[email protected]:~# traceroute 104.31.69.91
traceroute to 104.31.69.91 (104.31.69.91), 30 hops max, 60 byte packets
1 192.168.1.1 (192.168.1.1) 0.765 ms 0.472 ms 0.506 ms
2 151.6.154.21 (151.6.154.21) 1.843 ms 1.697 ms 1.534 ms
3 151.6.240.10 (151.6.240.10) 2.186 ms 2.040 ms 1.906 ms
4 151.6.104.146 (151.6.104.146) 2.020 ms 1.665 ms 1.759 ms
5 151.6.4.172 (151.6.4.172) 5.525 ms 5.397 ms 151.6.4.208 (151.6.4.208) 6.494 ms
6 151.6.3.60 (151.6.3.60) 5.683 ms 151.6.3.44 (151.6.3.44) 6.130 ms 151.6.3.201 (151.6.3.201) 5.461 ms
7 ams-ix.as13335.net (80.249.211.140) 45.515 ms 49.193 ms 48.914 ms
8 104.31.69.91 (104.31.69.91) 36.293 ms 36.135 ms 36.003 ms
[email protected]:~#

Any chance to route all again to Rome (NAMEX)?
Thank you.

Hi Fabio!

I presume the choice is done by Wind as their Milan peering capacity is higher than Rome’s (way higher, 13 times as much). Milan is currently a bit over capacity (not even all Enterprise websites go there) and is being managed by Cloudflare by moving websites to Rome and other European locations.

Since Wind doesn’t want to load Rome too much (they only have 10Gbps) they will go to Frankfurt and Amsterdam.

What is the traceroute to cloudflare.com?

Source: https://peeringdb.com/net/2174

Hi Marco,
cloudflare.com” is routed to Rome (NAMEX) and “www.cloudflare.com” is routed to Milan.
See:

[email protected]:~# traceroute cloudflare.com
traceroute to cloudflare.com (104.17.175.85), 30 hops max, 60 byte packets
1 192.168.1.1 (192.168.1.1) 0.765 ms 0.384 ms 0.479 ms
2 151.6.154.21 (151.6.154.21) 2.054 ms 1.919 ms 1.781 ms
3 151.6.240.8 (151.6.240.8) 1.622 ms 2.358 ms 2.219 ms
4 151.6.104.144 (151.6.104.144) 2.085 ms 1.944 ms 1.823 ms
5 151.6.1.208 (151.6.1.208) 6.182 ms 151.6.4.172 (151.6.4.172) 6.038 ms 151.6.4.205 (151.6.4.205) 5.895 ms
6 151.6.1.165 (151.6.1.165) 6.250 ms 151.6.1.109 (151.6.1.109) 6.081 ms 5.798 ms
7 cloudflare-nap.namex.it (193.201.28.33) 6.717 ms 7.582 ms 7.599 ms
8 104.17.175.85 (104.17.175.85) 6.305 ms 6.196 ms 6.018 ms

[email protected]:~# traceroute www.cloudflare.com
traceroute to www.cloudflare.com (104.17.209.9), 30 hops max, 60 byte packets
1 192.168.1.1 (192.168.1.1) 1.015 ms 0.790 ms 0.804 ms
2 151.6.154.21 (151.6.154.21) 2.532 ms 2.443 ms 2.337 ms
3 151.6.240.10 (151.6.240.10) 2.238 ms 2.123 ms 2.013 ms
4 151.6.104.142 (151.6.104.142) 2.757 ms 2.661 ms 3.658 ms
5 151.6.4.158 (151.6.4.158) 7.001 ms 151.6.3.66 (151.6.3.66) 7.690 ms 151.6.5.238 (151.6.5.238) 12.733 ms
6 151.6.2.28 (151.6.2.28) 15.447 ms 151.6.3.194 (151.6.3.194) 51.275 ms 151.6.3.190 (151.6.3.190) 51.083 ms
7 151.6.6.141 (151.6.6.141) 16.364 ms 151.6.0.200 (151.6.0.200) 17.886 ms 151.6.6.141 (151.6.6.141) 18.710 ms
8 * * *
9 104.17.209.9 (104.17.209.9) 19.159 ms 18.867 ms 18.472 ms

Well, how to know if it’s a Wind (the ISP) or a Cloudlfare choose?
Are we sure is a Wind choose?

Thank you.

We can’t know if it was Cloudflare or Wind that chose this, but the fact that Cloudflare’s own website is in both locations means that most likely Wind is the one choosing here. Their website is announced everywhere for sure.

Hi Marco.
I’m in contact with an IP NETWORK ENGINEERING of Wind Infostrada to check and understand why.
Anyway, Wind Infostrada told me they have 4 x 100 Gb/sec to NAMEX (and not only 10 Gb/sec).
I think Cloudflare should check routing.

Oh, it seems like their Peering DB entry hasn’t been updated in 4+ years, that can be an issue given how much other networks use it.

Also, NaMeX themselves say they have a selective peering policy. They should contact Cloudflare’s NOC and talk to them, I can’t do anything myself. Also, try opening a ticket with Cloudflare support, but this may not be the best time to do that given the strain that the current world situation is putting on the internet in general.

I have had the confirm from Wind that Cloudlfare does not announce the route 1.1.1.0/24 (and also other) on NAMEX but announce it at MIX.
This is the reason why I reach Cloudlfare IPs via Milan (MIX) and not more via Rome (NAMEX).
I cannot understand why on “Cloudflare System Status” website the FCO (Rome) location is marked in green as “Operational”.

Is there some tech support of Cloudflare here?
Thank you.

I have on good authority that Cloudflare does announce that route there as other ISPs go to Rome and Cloudflare does announce that.

Check out this looking glass (https://lg.connesi.it), do a ping or traceroute to 1.1.1.1 from the NaMeX route server and you will see it remain in Rome. It’s Wind’s peering policy limiting it to MIX, not Cloudflare’s.

Also, as I predicted before, Wind doesn’t have 400Gbps at NaMeX. It would be insane given the total traffic load there (~150Gbps maximum). They have 30Gbps (https://my.namex.it/customer/detail/55).

It’s Operational because it works and is active. There was a couple of hours with problems on the 20th, but they have since been solved. https://www.cloudflarestatus.com/incidents/8r2gm3mj3mzy

There is, but it will be useless to contact since it’s not their fault…

To contact Cloudflare Customer Support, login & go to https://dash.cloudflare.com/?account=support and select get more help. Please give Support the complete details and link to your Community post and share the ticket number here. If you receive an automatic response that does not help you, please reply and indicate you need more help.

Now 1.1.1.1 and 1.0.0.1 are corrected routed to NAMEX (Rome).
But some Cloudflare website (IP 104.31.68.91 and 104.31.69.91) are still routed to AMS-IX (Amsterdam)…

It has never been guaranteed the all websites, especially on the lower plan levels, will be announced at all POPs and, just like the 1.1.1.0/24 route, an ISP may decide to route some to other locations.

You sure (https://lg.connesi.it) is reliable?
If from Milan MIX Router I do a tracertoute of 1.1.1.1 it goes in Rome and not in Milan… strange. I expect 1.1.1.1 is also on MIX…

Yeah, it has now changed, BGP it’s weird… 1.1.1.1 is also in Milan, that I can confirm myself since my own internet connection goes there.

I know 1.1.1.1 is also in Milan, I was joking about the Cludflare routes change very frequently and sometimes its “policy” to reach the most nearest location is not matched.
As reported before, I still access on some Cloudflare websites via Amsterdam and I’m from Napoli and I don’t think taht Amsterdam is the nearest, there are in the middle Rome, Milan, Switzerland, France, Germany…
So also trust me, before 1.1.1.1 was routed to Milan, is about after 12:00 (CEST) that is correctly routed to Rome (my real nearest).

Well, nearest location is always relative. Nearest in the network may be not physically nearest. The choice depends on sooo many factors there…

Most likely for Wind it’s the best route at the moment. Let’s imagine something like this: the website is announced in LHR, AMS, FRA, FCO. At MXP it’s not announced since Cloudflare has less bandwidth at the moment and the load is higher.

Now, Wind has lower bandwidth at FCO than other locations, so they bypass it, MXP is second best choice but it’s not announced. They only peer in LHR, AMS, FRA, all other locations go through a third party which charges them or Cloudflare isn’t present, so they put higher priority for their own peering arrangements. Once the packet is in MXP and needs to be routed on the European links they may choose to do a round-robin approach, some IPs on one, some on the other, or decide to allocate them based on total number of routes, or traffic, or whatever.

This applies to both IPv4 and IPv6 and they can have different choices independently of each other.

To this you need to be add possible outages in equipment and connections which may modify routes often.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.