MEL1.1.1.1 POP not resolving domains, DNSSEC failures on root `com`,`tv` etc zones

Zero Trust 1.1.1.1
1 
$ dig +short CHAOS TXT id.server @1.1.1.1
"MEL"

Cloudflare failures:

$ dig @1.1.1.1 google.com

; <<>> DiG 9.10.6 <<>> @1.1.1.1 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 27800
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; OPT=15: 00 0a 66 61 69 6c 65 64 20 74 6f 20 76 65 72 69 66 79 20 44 53 20 63 6f 6d 2e ("..failed to verify DS com.")
;; QUESTION SECTION:
;google.com.                    IN      A

;; Query time: 318 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Wed Oct 04 19:09:46 AEDT 2023
;; MSG SIZE  rcvd: 69


$ dig @1.0.0.1 twitch.tv

; <<>> DiG 9.10.6 <<>> @1.0.0.1 twitch.tv
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17293
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; OPT=15: 00 03 ("..")
; OPT=15: 00 0a 66 61 69 6c 65 64 20 74 6f 20 76 65 72 69 66 79 20 44 53 20 74 76 2e ("..failed to verify DS tv.")
;; QUESTION SECTION:
;twitch.tv.                     IN      A

;; ANSWER SECTION:
twitch.tv.              30      IN      A       151.101.194.167
twitch.tv.              30      IN      A       151.101.2.167
twitch.tv.              30      IN      A       151.101.66.167
twitch.tv.              30      IN      A       151.101.130.167

;; Query time: 15 msec
;; SERVER: 1.0.0.1#53(1.0.0.1)
;; WHEN: Wed Oct 04 19:11:23 AEDT 2023
;; MSG SIZE  rcvd: 137

Google is OK:

$ dig @8.8.8.8 google.com

; <<>> DiG 9.10.6 <<>> @8.8.8.8 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33813
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             22      IN      A       142.250.70.174

;; Query time: 13 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Oct 04 19:13:07 AEDT 2023
;; MSG SIZE  rcvd: 55
4

Seeing these errors as well since 1 hour, some roots like at present a lot of intermittent DNSSEC failures, but it rarely also reproduces on com:

; <<>> DiG 9.18.18 <<>> google.com @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 33599
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; EDE: 10 (RRSIGs Missing): (failed to verify DS com.)
;; QUESTION SECTION:
;google.com.			IN	A

;; Query time: 6 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Wed Oct 04 10:33:10 CEST 2023
;; MSG SIZE  rcvd: 69
5

Hi,

This issue is being worked on and reported on our statuspage here:

1 Like
6

The status is quite vague, so is it related that some instances of 1.1.1.1 are serving signatures that are expired for hours? Example:

dig @1.1.1.1 +dnssec NS . +nsid

; <<>> DiG 9.18.16 <<>> +dnssec +nocr +noclass @1.1.1.1 +dnssec NS . +nsid
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16003
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 14, AUTHORITY: 0, ADDITIONAL: 27

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
; NSID: 33 31 6d 35 39 (“31m59”)
;; QUESTION SECTION:
;. IN NS

;; ANSWER SECTION:
. 511814 NS a.root-servers.net.
. 511814 NS b.root-servers.net.
. 511814 NS c.root-servers.net.
. 511814 NS d.root-servers.net.
. 511814 NS e.root-servers.net.
. 511814 NS f.root-servers.net.
. 511814 NS g.root-servers.net.
. 511814 NS h.root-servers.net.
. 511814 NS i.root-servers.net.
. 511814 NS j.root-servers.net.
. 511814 NS k.root-servers.net.
. 511814 NS l.root-servers.net.
. 511814 NS m.root-servers.net.
. 511814 RRSIG NS 8 0 518400 20231004050000 20230921040000 11019 . [omitted]

;; ADDITIONAL SECTION:
a.root-servers.net. 511814 A 198.41.0.4
a.root-servers.net. 511814 AAAA 2001:503:ba3e::2:30
b.root-servers.net. 511814 A 199.9.14.201
b.root-servers.net. 511814 AAAA 2001:500:200::b
c.root-servers.net. 511814 A 192.33.4.12
c.root-servers.net. 511814 AAAA 2001:500:2::c
d.root-servers.net. 511814 A 199.7.91.13
d.root-servers.net. 511814 AAAA 2001:500:2d::d
e.root-servers.net. 511814 A 192.203.230.10
e.root-servers.net. 511814 AAAA 2001:500:a8::e
f.root-servers.net. 511814 A 192.5.5.241
f.root-servers.net. 511814 AAAA 2001:500:2f::f
g.root-servers.net. 511814 A 192.112.36.4
g.root-servers.net. 511814 AAAA 2001:500:12::d0d
h.root-servers.net. 511814 A 198.97.190.53
h.root-servers.net. 511814 AAAA 2001:500:1::53
i.root-servers.net. 511814 A 192.36.148.17
i.root-servers.net. 511814 AAAA 2001:7fe::53
j.root-servers.net. 511814 A 192.58.128.30
j.root-servers.net. 511814 AAAA 2001:503:c27::2:30
k.root-servers.net. 511814 A 193.0.14.129
k.root-servers.net. 511814 AAAA 2001:7fd::1
l.root-servers.net. 511814 A 199.7.83.42
l.root-servers.net. 511814 AAAA 2001:500:9f::42
m.root-servers.net. 511814 A 202.12.27.33
m.root-servers.net. 511814 AAAA 2001:dc3::35

;; Query time: 5 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Wed Oct 04 12:55:15 CEST 2023
;; MSG SIZE rcvd: 1106

7

Which timeframe was this happening and is it still occurring? If it was only during the timeframe on the incident page then it was likely related.

8

I had posted it with timestamps. Now I can’t get it return outdated signatures anymore (tried just repeating this single query for now). So hopefully it’s OK.

9

Hi @vcunat and others who’s interested, we have a blog post disclosed more details: 1.1.1.1 lookup failures on October 4th, 2023

3 Likes
10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.