Cloudflare DoT and DNSSEC

Zero Trust 1.1.1.1
1

I am fairly new to all of this but I have managed to set up the unbound DNS server with DNSSEC which is working just fine. I have also configured it to use DoT and that appears to be working just fine provided I turn OFF DNSSEC.

With everything enabled I see that DoT says “NO”

https://1.1.1.1/help#eyJpc0NmIjoiTm8iLCJpc0RvdCI6Ik5vIiwiaXNEb2giOiJObyIsInJlc29sdmVySXAtMS4xLjEuMSI6IlllcyIsInJlc29sdmVySXAtMS4wLjAuMSI6IlllcyIsInJlc29sdmVySXAtMjYwNjo0NzAwOjQ3MDA6OjExMTEiOiJZZXMiLCJyZXNvbHZlcklwLTI2MDY6NDcwMDo0NzAwOjoxMDAxIjoiWWVzIiwiZGF0YWNlbnRlckxvY2F0aW9uIjoiTEhSIiwiaXNwTmFtZSI6IkNsb3VkZmxhcmUiLCJpc3BBc24iOiIxMzMzNSJ9

I see this in the unbound log files:

30/09/2019 16:35:47 … info: validation failure “a9098a66-6039-4ba2-be0f-db578f83a39d is-cf cloudflareresolve com A IN”
30/09/2019 16:35:47 …info: validation failure “a9098a66-6039-4ba2-be0f-db578f83a39d is-cf cloudflareresolve.com AAAA IN”
30/09/2019 16:35:49 … info: validation failure “a9098a66-6039-4ba2-be0f-db578f83a39d is-dot cloudflareresolve com A IN”
30/09/2019 16:35:49 … info: validation failure “a9098a66-6039-4ba2-be0f-db578f83a39d is-dot cloudflareresolve com AAAA IN”

if I turn OFF unbounds ‘validator’ I see that Dot is working:

https://1.1.1.1/help#eyJpc0NmIjoiWWVzIiwiaXNEb3QiOiJZZXMiLCJpc0RvaCI6Ik5vIiwicmVzb2x2ZXJJcC0xLjEuMS4xIjoiWWVzIiwicmVzb2x2ZXJJcC0xLjAuMC4xIjoiWWVzIiwicmVzb2x2ZXJJcC0yNjA2OjQ3MDA6NDcwMDo6MTExMSI6IlllcyIsInJlc29sdmVySXAtMjYwNjo0NzAwOjQ3MDA6OjEwMDEiOiJZZXMiLCJkYXRhY2VudGVyTG9jYXRpb24iOiJMSFIiLCJpc3BOYW1lIjoiQ2xvdWRmbGFyZSIsImlzcEFzbiI6IjEzMzM1In0=

I am assuming that the URL’s I copied from the 1 1 1 1 help page show the information as I see it when I go there, in both configurations. (DNSSEC Off & ON)

Is DNSSEC working correctly for Cloudflare subdomains? How can I check and how can I resolve the issue that the help page says DoT is NOT a '“Yes” when DNSSEC is on.

I hope I have included all the relevant information. (Apparently I can only put two links in a page so some line I have edited to get around the issue by removing full stops.)

Thanks for any help/suggestions.

1 Like
2

Though in this case it seem you are not using Cloudflare at all.

By “unbound” are you referring to Unbound (DNS server) - Wikipedia?

3

Yes NLnet Labs - Unbound - About

In the configuration I have all queries forwarded to Cloudflare I would post the lines but putting them in this post does not work and I am not able to upload a text file with them in.

I also have another application that show what unbound.exe is connecting to and it is only the cloudflare servers.

The validation failures I posted e.g:
validation failure 1d0567f5-fd89-4bca-90bb-8b3d60c6449b.is-cf.cloudflareresolve.com.

have a cloudflare as the domain

Using this tool you can see that the subdomains return no information and thus unbound gives a validation failure:

http://dnsviz.net/d/1d0567f5-fd89-4bca-90bb-8b3d60c6449b.is-cf.cloudflareresolve.com/dnssec/

It is only cloudflareresolve.com that validates with DNSSEC.

Are you suggesting that unbound should ignore the subdomain part of the DNS name so long as something at the end of the subdomains validates correctly?

If so I will go back and see what they have to say.

1 Like
4

I am afraid it is not quite clear to me what the actual issue is. Is it a failed validation of one hostname or a general DNS issue?

As I mentioned before, from your first link it would seem as if you are not using Cloudflare at all. The page clearly says you are not connected to it.

It might be best to ask at Unbound-users Info Page

5

I already have:

====
It seems that cloudflare is using special subdomains (answered only from their 1.1.1.1 resolvers) for the online checks and they are not handling DNSSEC properly in regards to answers for these special subdomains.
Unbound is complaining about not being able to build a trust chain because it can’t get the DS information.

I suspect that if you turn off DNSSEC validation on your unbound
module-config: “iterator”
the online check should work. Although I would advise to turn it on again after you do the online check.

And just to be clear, this does not mean that 1.1.1.1 cannot do DNSSEC.
I am only talking about the subdomains used for this specific online check.

=====

Hence the post here.

So right now between a rock and a hard place.

6

Fair enough, I’d probably drop support an email or, maybe even better, @irtefa could comment on it.

Pinging @mnordhoff too.

1 Like
7

I have been doing some more reading and have updated my original post to unbound-users and I will update again when I have had a reply.

8

@sandro @mnordhoff @irtefa

OK so I dropped another post to unbound-users and below is the reply I got:
(I would have uploaded a .txt file but they are not allowed sigh…)
I have removed names where appropriate to protect privacy.
So still looking for answers from Cloudflare

==============================
Hi Ray,

I think that in all this you forgot that:

The zone is-cf.cloudflareresolve.com. does not exist on the internet.
Only the 1.1.1.1 resolvers (and the resolvers that forward to those resolvers like your unbound) will know of that zone and give a different answer. This was specifically crafted for the online test you showed in your previous emails. Everything else will just get the NSEC answer that you posted below.

As stated in previous emails the answer you get from the 1.1.1.1 resolvers is not signed properly. This is also what unbound is logging.
Unbound gets an unsigned record (CNAME) in a supposedly signed zone and this fails validation. That is why you see the SERVFAIL answer.

Also see some answers inline where I try to clarify your comments.

– xxx

On 03/10/2019 17:13, XXXX wrote:

Hi xxx,

I have been reading this page:

https://support.cloudflare.com/hc/en-us/articles/360021111972-Troubleshooting-DNSSEC

Taking this error from the log file using Unbound V1.9.4:

03/10/2019 15:29:45 C:\Program Files\Unbound\unbound.exe[11788:0]
info: validation failure
<5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com. A
IN>: DS got unsigned CNAME answer from 2606:4700:4700::1001 and
1.0.0.1 for DS is-cf.cloudflareresolve.com. while building chain of
trust

================================

There is no ‘ad’ flag present in the response so DNSSEC has failed using unbound and it returns no data hence the error message in the log file.

C:>dig
5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com.
+dnssec

; <<>> DiG 9.14.6 <<>>
5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com.
+dnssec ;; global options: +cmd ;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 37604 ;; flags:
qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096 ;; QUESTION SECTION:
;5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com. IN
A

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 03 15:50:46 GMT Summer Time 2019 ;; MSG SIZE rcvd:
93

================================

Using my ISP’s router I get this returned with an NSEC record to say
the name queried does not exist (have I got that correct?)

Yes, your ISP router is probably using your ISP’s resolvers (or other resolvers but not the 1.1.1.1 resolvers for sure) which do not know about the is-cf.cloudflareresolve.com. zone.

C:>dig @192.168.1.254
5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com.
+dnssec

; <<>> DiG 9.14.6 <<>> @192.168.1.254
5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com.
+dnssec ; (1 server found) ;; global options: +cmd ;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10727 ;; flags: qr
rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; MBZ: 0x0384, udp: 4096 ;; QUESTION
SECTION:
;5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com. IN
A

;; AUTHORITY SECTION:
cloudflareresolve.com. 0 IN SOA cloudflareresolve.com. dns.cloudflare.com. 2018100101 21600 3600 604800 0
5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com. 0 IN NSEC \000.5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com. HINFO TXT AAAA LOC SRV CERT SSHFP RRSIG NSEC TLSA HIP OPENPGPKEY SPF
cloudflareresolve.com. 0 IN RRSIG SOA 13 2 3600 20191011133931 20191003103931 64088 cloudflareresolve.com. ipLptq2Quw5wHXI5i09sydRWMJXiPVEP7F0jTuFsxvIYrgo/KW1mLbMV 30Iqokf79cF+Ruwhsg6D+cv+6klZ5A==
5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudflareresolve.com. 0 IN
RRSIG NSEC 13 4 3600 20191011144045 20191003114045 64088
cloudflareresolve.com.
VChj42IfF+549QC1k/q8NJglytBVYZOXovTENcIqRQ5YWkvdrJrC+07a
zvX7lcTeI092wIfYIHl6pImhKDABzA==

;; Query time: 152 msec
;; SERVER: 192.168.1.254#53(192.168.1.254) ;; WHEN: Thu Oct 03
15:52:24 GMT Summer Time 2019 ;; MSG SIZE rcvd: 473

================================

The NSEC I think says remove the first subdomain (if doing a DNSSEC chain of trust walk) and try again. Repeat as many times as required.

In short the NSEC proves that the domain you asked for does not exist.

So, if I knock off the first part of the subdomain:
5a196f4c-afe8-442e-b258-ee7373f136a5 I get the same results

I am assuming you are referring to unbound and the SERVFAIL answer. Yes, the reply has the same DNSSEC problem (unsigned CNAME record).

If I then knock off the: is-cf I see this result:

C:>dig cloudflareresolve.com. +dnssec

; <<>> DiG 9.14.6 <<>> cloudflareresolve.com. +dnssec ;; global
options: +cmd ;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36595 ;; flags: qr
rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096 ;; QUESTION SECTION:
;cloudflareresolve.com. IN A

;; AUTHORITY SECTION:
cloudflareresolve.com. 60 IN SOA cloudflareresolve.com. dns.cloudflare.com. 2018100101 21600 3600 604800 0
cloudflareresolve.com. 60 IN RRSIG SOA 13 2 3600 20191011133931 20191003103931 64088 cloudflareresolve.com. ipLptq2Quw5wHXI5i09sydRWMJXiPVEP7F0jTuFsxvIYrgo/KW1mLbMV 30Iqokf79cF+Ruwhsg6D+cv+6klZ5A==
cloudflareresolve.com. 60 IN NSEC \000.cloudflareresolve.com. NS SOA HINFO MX TXT AAAA LOC SRV CERT SSHFP RRSIG NSEC DNSKEY TLSA HIP OPENPGPKEY SPF
cloudflareresolve.com. 60 IN RRSIG NSEC 13 2 3600 20191011143217 20191003113217 64088 cloudflareresolve.com. +xoOFZv/dFhYHHrkzdZtCY2RXv3VKLXwrHkQcp2pQFuQQc49lgj7QIy4 WoD6OqvJy0oyRclznazLBpYu5kXNjQ==

;; Query time: 211 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 03 15:57:09 GMT Summer Time 2019 ;; MSG SIZE rcvd:
387

Which does have the ‘ad’ flag set and unbound returns the data. The same is true for the ISP’s router.

The cloudflareresolve.com domain is properly answered by the cloudlflare nameservers. No special answering from the 1.1.1.1 resolvers required for this one.

So in this instance the domain name validates and therefore all is OK.

Using:

http://dnsviz.net/d/5a196f4c-afe8-442e-b258-ee7373f136a5.is-cf.cloudfl
areresolve.com/dnssec/

DNSSEC is validated and all looks just fine.

The same is true for:

DNSSEC Analyzer - 5a196f4c-afe8-442e-b258-ee737
3f136a5.is-cf.cloudflareresolve.com

These tools don’t use the 1.1.1.1 resolvers for these domains so they just get the NSEC answers above which are properly signed.

These words I think explain what is happening:

“Dig also retrieves the public key used to verify the DNS record. A domain’s DNS records are all signed with the same public key. Therefore, query for the root domain’s public key, not the subdomain’s public key:”

They are on this page (mentioned at the start):
https://support.cloudflare.com/hc/en-us/articles/360021111972-Troubles
hooting-DNSSEC

So unless my reading and understanding of what should be happening is totally incorrect Unbound should be ignoring subdomains and tracking back to the actual domain name and if that validates then it should return the appropriate data.

The DNSSEC chain of trust is from the trust anchor (usually the root) down to all the records required until you get your final answer. If anything cannot validate in that path, then validation fails.

If I have this wrong then dnsviz and verisign should also fail the DNSSEC checks but they don’t appear to do so.
DNSviz and verisign’s tool do not fail the DNSSEC validation because they get the properly signed NSEC answers above.

Please let me know if I still have this all wrong and apologies if I have.

Regards

Ray

==============================

9

Sorry, cant comment on it. Way too little experience with DNSSEC :slight_smile:

Did support say anything?

@cloonan @cs-cf

It is public anyhow in their mailing list :slight_smile:

10

Not heard anything from anyone else at this time. Hopefully someone with a good understanding of what is supposed to happen will comment soon. Your pings have not elicited a response so far.

11

Well, did you open a support ticket at all?

12

Well I may be stupid but if I could find a link/e-mail address, anything that allowed me to do that I would, but the Cloudflare site just seems to go round in circles when clicking on the support link at the top of the page.

Perhaps you can point me to the correct location, that would be a great help.

13

The general way to open a ticket is at https://support.cloudflare.com/requests/new, however that requires one to select a domain, which in this case might not be possible, so simply drop an email to their support@ address. That should get you an autoresponse with a ticket number, post that number here so @cloonan can track it.

Should the ticket be automatically closed, simply reply to it to re-open it.

1 Like
14

Thanks for the pointer ticket now opened (#1763476)

As you pointed out I do not have a domain registered (because I don’t have one) so getting the support information was not possible.

We will see what happens.

15

@sandro

I just read your post on the other topic (which you since retracted), that said I have just taken a look at the support case I opened and there has been no update from CF so far.

I think your suggestion was/is a slightly different case though.

How do I get CF to respond?

16

@sandro @cloonan @cs-cf

As requested I opened a support ticket, its been 10 days now and no response whatsoever! How do I move things along?

17

Can you check the ticket status? Could it be it was automatically closed?

18

It’s open and assigned to an engineer, will keep an eye out on progress.

19

@sandro @cloonan

Thanks

20

I believe the issue here is one identified by another forum poster previously, though you did a nice job of laying out the underlying debugging. It’s a problem with the tool, but a bit of a corner case. I believe the engineering team is aware of the issue but imagine it’s a bit of a low priority to resolve (though I’ll ping someone to ask that perhaps we at least document it as a known issue online).

1 Like