[Feature request] Geolocation/Client Subnet support for Cloudfront domains

_az · 2018-08-27 11:59:20 UTC

I’ve noticed that since launch Cloudflare will often return records for distant Cloudfront edges, even when querying a local 1.1.1.1 edge.

This is a painful interaction for a CDN product that is supposed to reduce global latency.

I believe that it is a Cloudflare issue because other recursive resolvers, and the identity.cloudfront.net resolver do not seem to ever suffer from this degradation:

2018/08/27 21:43:52 [resolver=1.1.1.1:53,msgId=61668,host=13.33.28.138,rtt=220.251322ms]
2018/08/27 21:43:53 [resolver=8.8.8.8:53,msgId=8640,host=54.230.245.245,rtt=55.775511ms]
2018/08/27 21:43:53 [resolver=identity.cloudfront.net:53,msgId=33957,host=54.230.245.213,rtt=47.597676ms]

(RTT measured by TCP open to one of the A RRs, this example was from “MEL” Cloudflare resolver).

I’m currently running a longer-term test to figure out how often this happens, but it’s often enough to be annoying. It seems to affect all Cloudfront domains (e.g. all of the Atlassian sites).

Cloudfront uses DNS rather than Anycast to select the closest edge. It’s not clear to me exactly how this is interacting with Cloudflare, but they definitely support EDNS and this shouldn’t be happening.

matteo · 2018-08-27 12:15:13 UTC

The issue there may be that Cloudflare’s 1.1.1.1 doesn’t support EDNS. With some vendors they have worked to provide a better experience, maybe Cloudfront is not there or is coming.

_az · 2018-08-27 12:17:01 UTC

Oh, yikes. I swear I had read that Cloudflare was sending an anonymized client subnet. Must have been another resolver.

Very convenient then that if there is no EDNS, that most of the time it actually does resolve to a local edge.

matteo · 2018-08-27 12:18:20 UTC

I would assume that they use the edge’s IP to locate themselves to the EDNS DNS instead of yours for those with whom they worked.

_az · 2018-08-27 12:19:14 UTC

Yes, that’s what I thought was happening initially! But going by the literal interpretation of the page you linked, no Client Subnet data is sent at all.

matteo · 2018-08-27 12:21:46 UTC

I think they are not sending any EDNS IP. Because all Cloudflare’s IPs are geolocated to the US, but they have probably given a list of IP <-> PoP relations to Akamai, Google and the likes. That is not something that is public, but I would assume neither Google nor Akamai would launch a DDoS towards Cloudflare so they feel safe doing that.

_az · 2018-08-27 12:36:21 UTC

Looks like you’re spot on.

From the 8.8.8.8 resolver (74.125.41.9 locally) I can see that an OPT record with EDNS Client Subnet is sent.

From 1.1.1.1 (162.158.1.47 locally) an OPT record is sent but without any extensions.

I guess this is really a feature request, then.

This also seems to affect CNAME flattening in authoritative DNS product as well, which is a bummer. Changing to another DNS host might be on the horizon for me, can’t afford the $200/mo/domain it would take to use be able to use Cloudflare’s CDN in Australia .

matteo · 2018-08-27 16:52:08 UTC

What kind of back end are you running?

mvavrusa · 2018-08-29 03:56:45 UTC

I’ll try to reach out. Even without ECS, they should be able to geolocate you approximately using the Cloudflare’s PoP location. If they absolutely do need ECS, we’ll try to work something out.

seamus · 2018-09-26 06:25:02 UTC

FWIW, this appears to happen with Office365 as well.

From an Australian ISP:

Typically i would expect my result to be along the lines of:

The difference is ~ 200ms of latency, which really shows in desktop applications (like outlook).

Its worth nothing this appears to be random. At times I do get AUS results, while other times i get USA/Canada. I get that ECS isn’t exposed for privacy reasons, but perhaps a “bypass” capability in cloudflared (which is what im using for DNS over HTTPS) may be ideal until geo-location data can be updated accordingly?

irtefa · 2018-09-26 19:21:40 UTC

Hi Seamus,

I know it is incredibly frustrating to see ~200ms latency. We are working on a plan to remediate this. We will share an update with the community very soon on how we are going to solve this.

Thanks for your patience!

️:v:️

Irtefa

german.garciadoego · 2018-10-15 18:43:41 UTC

Almost a year has passed and the problems are still there … more than 6 months ago they indicated that it was under consideration to enable EDNS … We continue in the sweet wait using GOOGLE DNS, with which if things work correctly …
Are you going to take a lot more time to consider it ???

Greetings.

matteo · 2018-10-15 19:44:04 UTC

A month, not a year…

german.garciadoego · 2018-10-16 10:57:14 UTC

I said “ALMOST” a year … and it was not a month … If you verify the link that I put, you will see that these problems come from the beginning of the year, when cscharff had already commented that they were “considering” …

Basically this is what happens …
https://www.sajalkayan.com/post/cloudflare-1dot1dot1dot1.html

I hope they reconcideren and enable EDNS support to optimize the cdn connections …

Greetings.

matteo · 2018-10-16 11:08:41 UTC

Didn’t look at the link, but it’s still less than 6 months. Plus I doubt they will ever enable EDNS, if you want it/need it use something else.

The issue that happens is known to everyone, but that is what happens with a bit more privacy for the user.

They have optimized some CDNs, Google’s, Akamai’s and Amazon’s are fine here.

german.garciadoego · 2018-10-16 11:17:32 UTC

With respect to Google, they had mentioned that it was solved … however, most of the time certain connections, in my case, are directed to Peru … rarely within Argentina (EZE).
So I do not see that it is very solved that we say …

german.garciadoego · 2018-11-15 14:51:19 UTC

Hello good day!!! Was there any resolution to these drawbacks ???
Can you give an update to this topic please ???

Thanks and regards!!!

genuine3000 · 2018-11-20 00:11:42 UTC

Is there someone that can response if there will be esdn support because this has a serious issue in decreasing performance so is this in the roadmap

german.garciadoego · 2018-11-20 16:46:35 UTC

It seems abandoned this forum … No support here so I can give an answer ???

Thank you very much already!!!

matteo · 2018-11-21 00:02:19 UTC

Gonna reply to both @german.garciadoego and @genuine3000:

I don’t think this will ever be in the roadmap. Of course I am not on the team, but the premise of 1.1.1.1 is privacy and EDNS Client Subnet is the opposite of that. If that’s what you need fine, otherwise there are fine alternatives depending on your location.