DNS over TLS > Wrong look ups for co.uk domains


#1

So im having some confusion. I have unbound up and happy running a rDNS using TLS. This works great with all the europe based DNS over TLS open servers. It does NOT work when looking up co.uk based domains when i use 1.1.1.1. I get incorrect results.

theregister.co.uk resolves to 195.62.28.41 rather then 104.18.225.129 which has a nameserver by cloudflare. So you would think 1.1.1.1 would provide the correct lookup as its supposed to be DNSSEC and delivered via TLS and the nameserver is Cloudflare.

This may be intermittent, but very often.

This is not the only site. Ive seen it also effect other *.co.uk sites.

All other sites I have tried in every country work. Again, I have no issue if I use any DNS over TLS service in Europe.

I like Cloudflare’s speed, I need this to work please :slight_smile: I don’t see how I could have done something wrong ? As I change the resolver and it works. Change back to 1.1.1.1 / 1.0.0.1 and it does not.


#2

This also effects https://news.bbc.co.uk/ which is resolving to the same 195.62.28.41

Ive tried LOTS of other sites worldwide and it seems only the *.co.uk sites do this. Also it seems ALL *.co.uk sites resolve to this same incorrect IP.


#3

You are not alone on this. I was racking my brain over the weekend before I came to the conclusion that there was just something odd going on with CloudFlare’s resolution of theregister (and other UK domains).

As you mentioned, something like ‘Resolve-DnsName theregister.co.uk -Server 1.1.1.1’ resolves 195.62.28.41, but ‘Resolve-DnsName theregister.co.uk -Server 8.8.8.8’ or 9.9.9.9 provides 5 different IPs from Cloudflare.

In the meantime, I swapped over to Google DNS so that I could have accurate resolution, but do hope that this gets resolved as I did prefer the speed of Cloudflare’s DNS.


#4

Interesting… Well at least im not alone :slight_smile:

So. How do we get Cloud Flare’s attention to look at this ?

Is this issue true for normal DNS thru 1.1.1.1 or only DNS-TLS ?


#5

Woudl either of you mind running the dig commands referenced here to help us track down which colo you might communicating with?

Have problems with 1.1.1.1? *Read Me First - Cloudflare Community


#7

Sorry should have pasted that for the start of the topic. I dont have them now.

And this morning, everything is fine. Resolution is working correctly for all of the *.co.uk domain it looks like

Im new to using Cloudflare, new to using DNSSEC and new to using DNS over TLS. It seems disturbing to me that with all that verification and DNS security that a entire major TLD was being resolved incorrectly. In fact, thats hard to fathom how that occured technically with DNSSEC.

All the traffic for all of Cloudflare resolved TLD lookups for *.co.uk was ending up at this guys site. He is gonna have some interesting web stats… www.1extremevaleting.co.uk Gotta wonder why his IP ended up the resolution.

So issue resolved for me so far. I will keep on eye on it tho.


#8

The issue is resolved for me as well and I as such cannot illustrate the error. Should it return, I will post the details requested in the “Have problems with 1.1.1.1?” post.


#9

Im a bit disturbed by this issue. DNSSEC would seem to implicitly make this issue impossible. A entire TLD worth of domains was pointing at one IP ? This would seem to point to a MUCH more serious error in the Cloudflare system and somehow it was resolving a entire TLD set of hosts to the incorrect IPs. This could not have happened with a correctly configured DNSSEC ?

So exactly what was seriously misconfigured on Cloudflare’s side of this ? It was obviously not doing DNSSEC for these hosts. So, does that mean Cloudflare is not doing correct DNSSEC for any resolution ?

This would seem to be a very serious issue that reguards very core issues like a lack of auditing whats going on.

Are there official Cloudflare representives on this forum ? I would like to know exactly what was going on.

I was doing DNS over TLS to 1.1.1.1. The TLS verified the cert was Cloudflare. Yet 1.1.1.1 and 1.0.0.1 were resolving a entire TLD incorrectly and stunningly pointing to one IP.

So… Cloudflare… What happened ?


#10

bbc.co.uk and theregister.co.uk don’t use DNSSEC.

(nic.uk, the domain for the uk nameservers, also doesn’t use DNSSEC.)

1.1.1.1 is a validating resolver, but DNSSEC can’t protect zones that choose not to use it.

(Also uk uses a 1024-bit ZSK, but that is definitely not relevant.)

This sounds potentially serious, and I’m curious to know what happened, but the DNS – and particularly insecure zones – are somewhat vulnerable to certain attacks, like BGP hijacking or cache poisoning, even if a resolver is doing everything right.

The BBC and The Register both have some support for HTTPS, partially mitigating DNS or IP layer attacks.


#11

Well im new to DNSSEC. Excuse my ignornace. So ONLY a domain that is DNSSEC would have been fully vetted ? Its a interesting oversight that got exposed here then. You cant trust Cloudflare on hosts that dont do DNSSEC then ? So there is no checking or validating anything on TLDs if a host is not DNSSEC ?

Yes DNS is a mess huh ? This shows it…

I wish I had realized the scope of the issue when it was still a issue. I would have liked to have found a DNSSEC .co.uk and see what happened with a DNSSEC resolution.

Sorry if im a idiot on these things. Im new to DNSSEC and DNS over TLS.


#12

So, it seems like somebody fatfingered the zone for accountantwhitby .co.uk to co.uk in Cloudflare DNS.

I’ve been troubleshooting a mailflow issue affecting .co.uk recipients from our environent. One of our mail filters was attempting to send all mail to *.co.uk domains through mail3.eqx.gridhost .co.uk. This filter was using 1.1.1.1 DNS server (non-TLS).

While researching, I found this thread. I looked up the IP that everyone cited, 195.62.28.41, on ipaddress .com to see what some hostnames were that are pointing to this IP.

The first one in the list is www. accountantwhitby. co.uk. The MX record for this domain is mail3.egx.gridhost .co.uk, and the A record of course is 195.62.28.41.

Has there been any acknowledgement from Cloudflare about this snafu?


#13

Cloudflare 1.1.1.1 DNS, like Google Public DNS, OpenDNS, or Quad9 DNS, doesn’t manually enter the domain data—they forward queries to authoritative name servers, so nobody fat-fingered this.

It’s unclear how this error may have happened, it could have been some issue with the QNAME minimization implementation in the Knot resolver used by Cloudflare and/or some oddity in the nic.uk authoritative name servers for co.uk and .uk domains.

There is a wildcard MX record *.accountantwhitby.co.uk, which is served by the same name servers as co.uk. – it may be that due to a bug this wildcard was applied at a higher level. If there was a vulnerability in the Knot resolver or whatever authoritative name server software is used by nic.uk, they are unlikely to announce details until a patch has been widely distributed (if ever).


#14

The delegation is – at least now:

accountantwhitby.co.uk. 172800  IN      NS      ns3.yourcms.info.
accountantwhitby.co.uk. 172800  IN      NS      ns4.yourcms.info.

#15

The issue has not returned as far as I have experenced.

The issue had existed at least for a week. Maybe longer. It got cleared hours after I posted about it. It would seem someone who reads this saw the post and fixed the issue. I suppose it coulda have been a coincidence tho too.

If it could happen to this TLD, it could happen to any of them. That would seem to be a pretty big issue in my mind. You would think Cloud Flare would have some automated testing that runs all the time that checks and verifys known resolutions and alerts if it finds a issue. Thats a pretty easy script to write. A good test would probe all sorts of paths and various ways things could go wrong. Seeing as tho its company reputation is based on its accuracy of resolutions it would seem unwise to just trust ?

It was funny tho. A El Reg story about DNS encryption got me to install Unbound at home and the first site i want back to after I got it working was El Reg and it did not resolve. https://www.theregister.co.uk/2018/08/20/dns_interception/


#16

You’re right about the NS records for accountantwhitby.co.uk. I was misled by my shell script that prints the NS records for a domain name—it was confused by the fact that an NS or SOA query for accountantwhitby.co.uk to the yourcms.info name servers returned a NODATA response, so it tried again at the next level up.

I won’t respond any further here, as this may be related to (https://access.redhat.com/security/cve/cve-2018-12020).


#17

Your script had a issue related to this embargoed vulnerability or the issue we are discussing ?

Thats a fairly serious CVSS and issue you posted. What leads you to believe its related ?

Thats a fairly big leap from that vulnerability to a compromise of a entire set of TLD resolutions. Im also not sure why someone would want to redirect a entire TLD for gain ?

This is getting more interesting tho :slight_smile:


#18

Just so others have a good understanding of this CVE. It does not look embargoed.

https://nvd.nist.gov/vuln/detail/CVE-2018-12020