Route to US when client and server is in EU

What is the name of the domain?

zendorka.com

What is the issue you’re encountering

I’m in Hungary and reaching my server via CF took long so I tracerouted it lead to Ashbourn, Virginia. Now, I’m in Hungary, Budapest. CF has servers in Budapest so why route it to the USA? BTW my server is at Falkenstein.

What steps have you taken to resolve the issue?

Made sure I’m not on a VPN.

What are the steps to reproduce the issue?

traceroute -6 zendorka.com:heavy_check_mark:  3s 
traceroute to zendorka-com (2606:4700:3034::6815:1ca1), 30 hops max, 80 byte packets
1 xxxxx-dsl-pool-telekom-hu (xxxxx) 1.923 ms 1.874 ms 1.856 ms
2 * * *
3 * * *
4 * 2003:0:f00::7a5 (2003:0:f00::7a5) 35.556 ms *
5 * * *
6 2003:3c0:1600:8000::1 (2003:3c0:1600:8000::1) 125.593 ms * *
7 ash-b2-link.ip.twelve99-net (2001:2035:0:2829::1) 120.023 ms 117.958 ms 117.955 ms
8 cloudflare-ic328259-ash-b2.ip.twelve99-cust-net (2001:2000:3080:7bc::2) 182.052 ms 196.035 ms 185.944 ms
9 2400:cb00:354:3:: (2400:cb00:354:3::slight_smile: 194.093 ms 180.189 ms 2400:cb00:626:3:: (2400:cb00:626:3::slight_smile: 177.837 ms
10 2400:cb00:352:1024::ac46:2944 (2400:cb00:352:1024::ac46:2944) 179.775 ms 2400:cb00:352:1024::ac46:295e (2400:cb00:352:1024::ac46:295e) 183.767 ms 2400:cb00:354:1024::ac46:856f (2400:cb00:354:1024::ac46:856f) 185.679 ms

That is unfortunately a well-known problem, which has been brought up multiple times already, such as e.g. here:

2 Likes

Thank you.
This is hilarious. :slight_smile: Ping from Eastern-Hungary to Falkenstein 28ms, ping to Budapest 10ms (that’s what I hoped for) but for static content now I get 250ms connection time, for dynamic 500.
What is even more hilarious that routing so much EU data though US can go on for years in times when so many eyes are on data protection.
What era to live in. :slight_smile:

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

What is the name of the domain?

zendorka.com invictus.com

What is the issue you’re encountering

for some sites at the same location nonoptimal route from client to CF POP

What steps have you taken to resolve the issue?

used nslookup and dig and ping and also https://dnschecker.org

What feature, service or problem is this related to?

I don’t know

What are the steps to reproduce the issue?

I reported this few days ago and though I just have to live with it: Route to US when client and server is in EU
However I noticed my other server, on the same pc, routes directly over the closest regional POP as expected and not over the US.

So it can not be because of a global “CF does not peer with DT issue”.

Then I check the DNS records all over the world of a third, older site using CF. Medium traffic mostly from US and EU regions.
I see that most of the world is using the same IP, but some EU countries have their unique IP, and US is covered with two distinct IPs.

For me it looks like the regional POP is utilized when traffic is over some threshold, otherwise DNS points to a default or central CF server.

Hi,

Thank you for reaching out to us. I can see that zendorka.com yesterday was struggling with 521 errors. A 521 error happens when Cloudflare is unable to make a TCP connection to your origin server, typically because the connection was refused, which often can be caused by security or firewall software.

I would make sure that your hosting provider confirms that the Cloudflare IP ranges listed in the URL above are fully allowed from any security software, firewall, etc., to ensure there is no rate limiting or blocking of our edge server requests to your infrastructure. This should ensure that Cloudflare can consistently make a connection to your origin server to retrieve content and serve it to your visitors.

We publish a full list of our IP ranges, so that you can allowlist them accordingly.

Because Cloudflare operates as a reverse proxy, the IP address your server will see is one of a limited number of Cloudflare IPs. In that sense, many actual visitors may all come from the same IP address, which can cause firewalls or security software that is not appropriately allowing the Cloudflare IP ranges to block this traffic - it may see it as excessive or malicious.

For more information, you can review the following link: Troubleshooting Cloudflare 5XX errors | Cloudflare Support docs

Also, zendorka.com has cache-status set as DYNAMIC.

If you’re seeing the TTFB is unusually high on dynamic resources (CF-Cache-Status is not HIT , see Cloudflare cache responses):

  1. Follow the customize the cache guide to make sure that any resource that can be cached appropriately is cached. You can review the Cache Analytics on the Cloudflare Dashboard to check for cache performance and potential improvements.
  2. Make sure your origin web server is performing well: you might need to check with your website administrator or hosting provider.

You can also make use of the Cloudflare Observatory that can help you understand why your website is slow: https://blog.cloudflare.com/cloudflare-observatory-generally-available/ We recommend you to test the performance of your website using the Speed Test under Speed → Observatory in the Cloudflare Dashboard. You can then find more details here on how to understand the results.

I hope this helps. Please let us know if you have other questions.

Thanks for taking the time to answer Lili.

The other day I’ve been rescaling the origin server that caused the 521 errors you noticed - most of the time the server is up and running fine.
Apart from that connectivity is just fine, no firewalls etc. in the way.

I’ve checked the cache, it’s working, dynamic pages are not cached, static content is cached.
The slowdown comes from the fact they are cached at Ashbourn USA, at not Budapest HU (closest CF point of presence for all of my target audience, all of Hungary)
Since origin server is at Falkenstein DE, packets have a suboptimal route, from HU to USA to DE and back, instead of HU to HU-BP to DE and back.

And I don’t get why? Caching at the right edge server would be the core task for the CF cache.

And that is the DTAG issue, because DTAG does not want to peer locally with Cloudflare.

From my view, it has been shared with you before.

That is exactly what is being done.

DTAG decides to take you to the United States.

The right edge server for the caching is therefore the one in United States, as you reaching the PoP in the United States, because your ISP routes you to that location.

Caching somewhere else would create even more slowdowns than you already see, as traffic would be bouncing back and forth across continents multiple times, if the caching should happen somewhere else.

No Cloudflare ↔ DTAG peering exist in any locations worldwide.

You will need to complain to DTAG, if you want change DTAG’s bad routing.

2 Likes

@DarkDeviL Thanks for looking into this once again.
Suppose you are right and the global peering issue is behind this. Then why aug dot hu is not affected despite it’s on the same server and network than zendorka dot com?

The routing to United States is DTAG, according to the previous part of the discussion.

DTAG takes traffic for both of the mentioned domain names to the United States.

If you are having problems that exist on one of the domain names, but not the other, then the problem isn’t to be found within Cloudflare, but outside of Cloudflare.

Did you test it? My test initiated from multiple Hungarian residence IP addresses show that requests to my newer domain are routed over an US CF POP, and the older domain over the Budapest (capital of Hungary) CF POP.
I also run tests worldwide, and a handful of countries seem to route requests to my older domain over a CF POP in the country the request originates from, while the rest over the US.

Sounds valid, that’s why I’m trying hard to erode any differences between the two sites, and by now there’s not much left:

  • both sites are on the same cloud server, connected to the same network
  • both sites served by the same software
  • same cf setting

See, the only black box part that remained is CF. All the rest is identical.
And just one more thing… an experience that makes me think there this could be an edge case CF code can not handle internally and falls back to the safe deafult which is to route it over Ashbourn: when I played around with IPv4/IPv6 A/AAAA records for my 3 sites under the same IP, at one point when I changed settings for site 2, the route of site 1 changed from the expected fast route to Asbourn!

Hmm, it just occurred to me:
DNS for both domains is fully at CF and are proxied.
So instead of the real server IPs (which are two IPv6 addresses from the same /64 block) Cloudflare’s algorithm selects a POP and reports that IP instead.
When I ping the two domains the ping times clearly show the difference that for one domain the POP is faaar away from me, while the other is close by.
See? Why did that algorithm gives a sane response to one IPv6 address, while falls back to some defaults for another almost IP?

Made one more experiment. Swapped the AAAA records of the two domains to see if something I could not think of at the server side might affect the CF POP selection decision, and guess what, it is not. aug dot hu still routed close and fast while zendorka routed around the world and slow.

No matter what I do, CF handles one domain nicely, another badly.
Same location, same server environment, same CF site settings.

Yes.

Which one is it that you refer to as the newer domain, and which one is the older domain?

If I understood you well, aug.hu is the older one?

And out of curiosity, what is the kind of time frame, e.g. how do we calculate old versus new?

Can you share the actual Cloudflare IPv4 and IPv6 addresses you see from your end, as well as results you see from your end (e.g. local / EU POP vs NA POP) for each of them?

Both of them was being routed to the US when I tested it the other day.

Yes aug is the older / fast / using Budapest CF POP.
The ping results always correlate with page load times, so I only share those.
It’s like this, all the time for me since I started measuring it about half year ago.

PING aug.hu (2a06:98c1:3121::a) 56 data bytes
64 bytes from 2a06:98c1:3121::a: icmp_seq=1 ttl=56 time=17.4 ms
64 bytes from 2a06:98c1:3121::a: icmp_seq=2 ttl=56 time=14.9 ms
64 bytes from 2a06:98c1:3121::a: icmp_seq=3 ttl=55 time=16.8 ms
--- aug.hu ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 14.924/16.369/17.385/1.049 ms

PING aug.hu (188.114.97.10) 56(84) bytes of data.
64 bytes from 188.114.97.10: icmp_seq=1 ttl=55 time=17.5 ms
64 bytes from 188.114.97.10: icmp_seq=2 ttl=55 time=17.3 ms
64 bytes from 188.114.97.10: icmp_seq=3 ttl=55 time=18.8 ms
--- aug.hu ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 17.319/17.870/18.778/0.646 ms

PING zendorka.com (2606:4700:3035::ac43:aaed) 56 data bytes
64 bytes from 2606:4700:3035::ac43:aaed: icmp_seq=1 ttl=54 time=118 ms
64 bytes from 2606:4700:3035::ac43:aaed: icmp_seq=2 ttl=54 time=118 ms
64 bytes from 2606:4700:3035::ac43:aaed: icmp_seq=3 ttl=54 time=117 ms
--- zendorka.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 116.870/117.462/117.767/0.418 ms

PING zendorka.com (104.21.28.161) 56(84) bytes of data.
64 bytes from 104.21.28.161: icmp_seq=1 ttl=53 time=118 ms
64 bytes from 104.21.28.161: icmp_seq=2 ttl=53 time=117 ms
64 bytes from 104.21.28.161: icmp_seq=3 ttl=53 time=119 ms
--- zendorka.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 116.987/117.910/119.207/0.943 ms

AS3320 Deutsche Telekom (Germany) takes the traffic for all of the mentioned IP addresses to the United States.

AS5483 Magyar Telekom (Hungary) takes the traffic for those two IP addresses towards something <= 20 ms.

What is interesting is, that the traffic from AS5483 Magyar Telekom seems to pass over unallocated IP addresses, according to traceroutes.

That would sound to be a misconfiguration from AS5483 Magyar Telekom’s end, or between them and one of their peering partners, as the unallocated IP addresses comes in the first step after AS5483 Magyar Telekom themself.

My traceroutes are not in line what you state:
(Attaching image as links are disallowed)

What about the IPv6 ones, such as e.g. “2a06:98c1:3121::a”?