1.1.1.1 not resolving route53 domain (AWS QNAME minimization problem)


#1

Domain: sheerenloo.aysist7.nl

##nslookup sheerenloo[.]aysist7[.]nl
Server: 1dot1dot1dot1[.]cloudflare-dns[.]com
Address: 1[.]1[.]1[.]1

*** 1dot1dot1dot1[.]cloudflare-dns[.]com can’t find
sheerenloo[.]aysist7[.]nl: Non-existent domain

##nslookup sheerenloo[.]aysist7[.]nl 1[.]1[.]1[.]1
Server: 1dot1dot1dot1[.]cloudflare-dns[.]com
Address: 1[.]1[.]1[.]1

*** 1dot1dot1dot1[.]cloudflare-dns[.]com can’t find
sheerenloo[.]aysist7[.]nl: Non-existent domain

##nslookup sheerenloo[.]aysist7[.]nl 1[.]0[.]0[.]1
Server: 1dot1dot1dot1[.]cloudflare-dns[.]com
Address: 1[.]0[.]0[.]1

*** 1dot1dot1dot1[.]cloudflare-dns[.]com can’t find
sheerenloo[.]aysist7[.]nl: Non-existent domain

##nslookup sheerenloo[.]aysist7[.]nl 8[.]8[.]8[.]8
Server: google-public-dns-a[.]google[.]com
Address: 8[.]8[.]8[.]8

Non-authoritative answer:
Name: rest-production[.]aysist[.]ayton[.]unitt-route53[.]com
Addresses: 54[.]93[.]159[.]185
18[.]184[.]91[.]137
Aliases: sheerenloo[.]aysist7[.]nl

##nslookup -class=chaos -type=txt id[.]server 1[.]1[.]1[.]1
Server: 1dot1dot1dot1[.]cloudflare-dns[.]com
Address: 1[.]1[.]1[.]1

Non-authoritative answer:
id[.]server text =

    "ams01"

##nslookup -class=chaos -type=txt id[.]server 1[.]0[.]0[.]1
Server: 1dot1dot1dot1[.]cloudflare-dns[.]com
Address: 1[.]0[.]0[.]1

Non-authoritative answer:
id[.]server text =

    "ams01"

##nslookup -type=txt whoami[.]cloudflare[.]com ns3[.]cloudflare[.]com
Server: ns3[.]cloudflare[.]com
Address: 162[.]159[.]0[.]33

whoami[.]cloudflare[.]com text =

    "85[.]147[.]43[.]xxx"

#2

The domain’s DNS records are wrong. Resolvers that use QNAME minimisation, like 1.1.1.1, or that receive queries for ayton.unitt-route53.com will behave that way.

The unitt-route53.com zone contains a delegation for ayton.unitt-route53.com and a delegation for aysist.ayton.unitt-route53.com.

$ dig +norecurse @ns-758.awsdns-30.net ayton.unitt-route53.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +norecurse @ns-758.awsdns-30.net ayton.unitt-route53.com
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26487
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ayton.unitt-route53.com.       IN      A

;; AUTHORITY SECTION:
ayton.unitt-route53.com. 300    IN      NS      ns-1199.awsdns-21.org.
ayton.unitt-route53.com. 300    IN      NS      ns-1557.awsdns-02.co.uk.
ayton.unitt-route53.com. 300    IN      NS      ns-363.awsdns-45.com.
ayton.unitt-route53.com. 300    IN      NS      ns-962.awsdns-56.net.

;; Query time: 0 msec
;; SERVER: 205.251.194.246#53(205.251.194.246)
;; WHEN: Wed Jun 06 22:16:30 UTC 2018
;; MSG SIZE  rcvd: 189

$ dig +norecurse @ns-1066.awsdns-05.org aysist.ayton.unitt-route53.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +norecurse @ns-1066.awsdns-05.org aysist.ayton.unitt-route53.com
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53142
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;aysist.ayton.unitt-route53.com.        IN      A

;; AUTHORITY SECTION:
aysist.ayton.unitt-route53.com. 60 IN   NS      ns-1032.awsdns-01.org.
aysist.ayton.unitt-route53.com. 60 IN   NS      ns-1709.awsdns-21.co.uk.
aysist.ayton.unitt-route53.com. 60 IN   NS      ns-440.awsdns-55.com.
aysist.ayton.unitt-route53.com. 60 IN   NS      ns-787.awsdns-34.net.

;; Query time: 18 msec
;; SERVER: 205.251.196.42#53(205.251.196.42)
;; WHEN: Wed Jun 06 22:16:52 UTC 2018
;; MSG SIZE  rcvd: 196

Route 53 shouldn’t let you do that: Logically, only one of the two can exist, but it allows both of them to sort of exist.

Now, the ayton.unitt-route53.com zone doesn’t contain a delegation to aysist.ayton.unitt-route53.com:

 $ dig +norecurse @ns-363.awsdns-45.com aysist.ayton.unitt-route53.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +norecurse @ns-363.awsdns-45.com aysist.ayton.unitt-route53.com
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 13716
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;aysist.ayton.unitt-route53.com.        IN      A

;; AUTHORITY SECTION:
ayton.unitt-route53.com. 900    IN      SOA     ns-1199.awsdns-21.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

;; Query time: 0 msec
;; SERVER: 205.251.193.107#53(205.251.193.107)
;; WHEN: Wed Jun 06 22:17:30 UTC 2018
;; MSG SIZE  rcvd: 141

Therefore resolvers may conclude that aysist.ayton.unitt-route53.com does not exist.

You need to add the NS records for aysist.ayton.unitt-route53.com to the ayton.unitt-route53.com zone and then remove them from the unitt-route53.com zone.


#3

Thanks @mnordhoff, I was walking a colleague through the same process and explaining some of the finer points of DNS debugging earlier before I had to run home. @user2441 are you CNAMEd to a 3rd party service or do you control both the original record you were looking up and the CNAME target?

I can put in a request to try and bypass query minimization for unitt-route53.co but in an ideal world they would fix their DNS.


#4

I don’t control either of these domains, I’m just an end user experiencing problems. Solved it by adding some rows to the hosts-file, but this 1.1.1.1-behaviour is quite unique and frustrating.


#5

Yeah… security often has unexpected side effects. We also reject sites with invalid DNSSEC… DNSSEC is a way to sign your records to validate the records returned are correct. Seems like a good thing right? But > 30% of DNSSEC records out there are invalid. So what to do?

Sorry for the frustration though we definitely get it.


#6

Are you a customer? Or a customer of a customer? If you can find a way to contact the operators of unitt-route53.com, it would be great if their broken DNS configuration could get fixed.

Only resolvers like 1.1.1.1 that use QNAME minimisation will always get NXDOMAIN, but most resolvers can be manipulated into getting it if you send them some queries for ayton.unitt-route53.com.

And it’s wrong either way.


#7

Qname minimization is more about performance (maybe privacy), and I understand your point of view. However, things would not be insecure per-se, if you would send the whole qname as-is. Communication should always be “conservative in what you send, liberal in what you accept from others”. Maybe you guys could have some discussion with Amazon about the behaviour they tolerate from their customers (see https://news.ycombinator.com/item?id=15893103 ). Or maybe a fall-back to default behaviour for multi-level qnames that fail the initial lookup.


#8

I just needed the website to fill in some forms, and discovered 1.1.1.1 was the culprit in not resolving the domain. Didn’t reach out to the company, as it will take me too much time to find the one responsible with the right skills and know-how to solve this (IT-ers/sysadmins do really like their privacy I guess).


#9

Qname minimization is actually a great example of being conservative in what you send as it sends less data, only the exact data that a particular upstream server needs to answer the question and nothing more.

Amazon is well known for not caring about how their Route53 performs, and not following standards consistently and has made it clear to the DNS operators community that they do not make resolving protocol/implementation problems a priority and expect the world to work around their brokenness. This is their choice, obviously, but I applaud companies that don’t apply custom workarounds for brokenness.

There is no fallback possible as there is no error, Amazon is clearly saying “This does not exist”, which Cloudflare takes literally and correctly. It would not be correct behaviour to ignore an authoritative “no” and keep rephrasing the question in a different way until you get a different answer.

This particular case may be something the domain owner can solve by adding the missing records, or it could be inherent brokenness in the Route53 platform. In fact, it is just as likely that the administrator misconfigured this and failed to test with real world resolvers or proper diagnostic tools. It happens.

(And to be clear, I’m not affiliated with either Amazon or Cloudflare, just another operator of various services including DNS servers).


#10

The issue is 100% that Amazon’s customer entered invalid and inconsistent records.They’re the only ones who can fix their mistake, and they should. Everything is working as designed.

Route 53 could have been designed differently – like rejecting attempts to create occluded records, or serving the least specific delegation and consistently breaking the zone, or warning users about problematic configurations – but they’re apparently resistant to making potentially backwards incompatible changes now.