Troubleshooting DNS Resolution errors

TheWaterOnFire · September 4, 2020, 4:31pm

I’ve been using 1.1.1.1 for my home network’s upstream DNS service, but recently queries for various Zoom subdomains (the ones my family is using for e-Learning!) are returning intermittent SERVFAILs:

❯ drill -V3 op97-org.zoom.us @1.1.1.1
;; ->>HEADER<<- opcode: QUERY, rcode: SERVFAIL, id: 33951
;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; op97-org.zoom.us.    IN      A

;; ANSWER SECTION:

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 6 msec
;; SERVER: 1.1.1.1
;; WHEN: Fri Sep  4 11:05:22 2020
;; MSG SIZE  rcvd: 34

via https:
❯ curl “https://cloudflare-dns.com/dns-query?ct=application/dns-json&name=op97-org.zoom.us&type=A”
{“Status”:2,“TC”:false,“RD”:true,“RA”:true,“AD”:false,“CD”:false,“Question”:[{“name”:“op97-org.zoom.us”,“type”:1}]}

These domains resolve fine on 8.8.8.8 and 9.9.9.9.

How can I help troubleshoot and/or inform the right folks of the issue?

Thanks!

Connection information: Link

heavypackets · September 4, 2020, 7:30pm

I am also getting SERVFAIL on many *.zoom.us domains. Adding manual entries for now.

I have tried using https://1.1.1.1/purge-cache/ to purge the CNAME entries manually but that hasn’t worked. That’s the only tool I know we can use, but it generally works well. In this case, the entry isn’t stale, it’s broken.

anb · September 8, 2020, 5:48pm

Hi @TheWaterOnFire & @heavypackets,

Sorry for the inconvenience! We’ve made some changes, it should solve your issue. Please let us know if the resolution still fail. Thanks.

TheWaterOnFire · September 9, 2020, 3:57pm

Looks better for me! Thank you. What was the change?

phealy · September 16, 2020, 1:39pm

I’m still seeing this issue against the ORD datacenter - intermittent SERVFAIL when looking up *.zoom.us domains. I’m using CoreDNS as a forwarder for DNS-over-TLS.

Here’s a link to 1.1.1.1/help for me.

Here’s my corefile:

.:5353 {
        errors
        log
        forward . tls://1.1.1.1 tls://1.0.0.1 {
                tls_servername cloudflare-dns.com
                health_check 5s
        }
}

A failed request (note that I’m using OP’s domain as I don’t want to post the one I was trying publically and it has the same issue for me):

root@pihole-6c68ffb9c6-c9kxs:/# dig -p 5353 @127.0.0.1 op97-org.zoom.us.

; <<>> DiG 9.10.3-P4-Debian <<>> -p 5353 @127.0.0.1 op97-org.zoom.us.
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 18878
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; OPT=12: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ("...................................................................................................................................................................................................................................................................................................................................................................................................................................")
;; QUESTION SECTION:
;op97-org.zoom.us.              IN      A

;; Query time: 21 msec
;; SERVER: 127.0.0.1#5353(127.0.0.1)
;; WHEN: Wed Sep 16 08:35:16 CDT 2020
;; MSG SIZE  rcvd: 468

Same query, successful a few retries later:

root@pihole-6c68ffb9c6-c9kxs:/# dig -p 5353 @127.0.0.1 op97-org.zoom.us.

; <<>> DiG 9.10.3-P4-Debian <<>> -p 5353 @127.0.0.1 op97-org.zoom.us.
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41681
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; OPT=12: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 (".............................................................................................................................................................................................................................................................................................................................................................................................")
;; QUESTION SECTION:
;op97-org.zoom.us.              IN      A

;; ANSWER SECTION:
op97-org.zoom.us.       300     IN      CNAME   us02web.zoom.us.
us02web.zoom.us.        60      IN      A       3.235.73.121

;; Query time: 80 msec
;; SERVER: 127.0.0.1#5353(127.0.0.1)
;; WHEN: Wed Sep 16 08:36:11 CDT 2020
;; MSG SIZE  rcvd: 506

Any help would be appreciated!

phealy · September 16, 2020, 1:40pm

Oh, and here’s the CoreDNS logs for the failure and the success:

[INFO] 127.0.0.1:55580 - 45088 "A IN op97-org.zoom.us. udp 45 false 4096" SERVFAIL qr,rd,ra 474 2.014952016s
[INFO] 127.0.0.1:50008 - 41681 "A IN op97-org.zoom.us. udp 45 false 4096" NOERROR qr,rd,ra 506 0.07151834s

anb · September 16, 2020, 3:03pm

@phealy could you please try again? Should be fixed.

@TheWaterOnFire there was an attack triggering our security policy.

phealy · September 21, 2020, 3:40pm

It’s been reliable since then, so thank you!