Hi all,
We have been experiencing intermittent network issues where occasionally requests to https://sentry.io will fail due to curl error code [6] Couldn’t resolve host name (Could not resolve host: sentry.io).
Our servers are using Cloudflare’s DNS; 1.1.1.1 and 1.0.0.1.
I have been able to run a packet capture and replicate this issue happening, and looking at the DNS requests and responses I believe I have found the root cause:
In the first set of queries, we get answers for both A queries from 1.1.1.1 and 1.0.0.1. The subsequent HTTP request made by curl went through successfully.
In the second set of requests, the first A response we receive back from 1.1.1.1 has an empty answer section.
I believe that this empty answer is what causes curl to produce error code 6 “couldn’t resolve host name”.
It doesn’t appear to wait for the response from 1.0.0.1 (the 2 DNS servers in our /etc/resolv.conf are 1.1.1.1 and 1.0.0.1).
These are just 2 specific examples where I was able to replicate while running tcpdump, but this would happen on a fairly regular basis - every 10 requests or so.
Why would Cloudflare’s DNS sometimes return an empty answer section for the DNS query? Why did it happen only a couple of seconds apart? Why is it intermittent?
Are we running into rate limiting issues?
Any help is greatly appreciated