Argo Tunnel: Error looking up Cloudflare edge IPs

Performance Cloudflare Tunnel
#1

Hello,

I have been running Argo Tunnel on my server for a long time now and never had any problems.

Yesterday it was all working fine but today I am getting an error when I try and start the tunnel…

 ERR Error looking up Cloudflare edge IPs: the DNS query failed error="lookup _[]-origintunneld._tcp.argotunnel.com: no such host"
2021-08-28T11:54:13Z ERR Please try the following things to diagnose this issue:
2021-08-28T11:54:13Z ERR   1. ensure that argotunnel.com is returning "origintunneld" service records.
2021-08-28T11:54:13Z ERR      Run your system's equivalent of: dig srv _origintunneld._tcp.argotunnel.com
2021-08-28T11:54:13Z ERR   2. ensure that your DNS resolver is not returning compressed SRV records.
2021-08-28T11:54:13Z ERR      See GitHub issue https://github.com/golang/go/issues/27546
2021-08-28T11:54:13Z ERR      For example, you could use Cloudflare's 1.1.1.1 as your resolver:
2021-08-28T11:54:13Z ERR      https://developers.cloudflare.com/1.1.1.1/setting-up-1.1.1.1/

image
image1291×163 9.54 KB

Nothing seems to have changed with the server and running the suggested dig command brings back a valid result (as far as I can tell).

; <<>> DiG 9.16.9 <<>> srv _origintunneld._tcp.argotunnel.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44041
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_origintunneld._tcp.argotunnel.com. IN SRV

;; ANSWER SECTION:
_origintunneld._tcp.argotunnel.com. 30 IN SRV   2 1 7844 region2.argotunnel.com.
_origintunneld._tcp.argotunnel.com. 30 IN SRV   1 1 7844 region1.argotunnel.com.

;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sat Aug 28 12:54:49 BST 2021
;; MSG SIZE  rcvd: 147

Has anybody else had a similar problem? Any suggestions?

Thanks,
Chris

#2

Hello @chris.child ,

This is the first report of such problem, so there’s a good chance that it is somehow related to your system/environment/network.

But just to confirm, which cloudflared version are you running with?

#3

Hi,

I also started getting the same error today. It started immediately after the automatic update of cloudflared to the latest version (2021.8.6 built 2021-08-27-2248 UTC).

To confirm this I ran the previous version of cloudflared (2021.8.3 built 2021-08-25-0722 UTC), it works fine.

Strangely, the exact same problematic version running on another computer/network is running without any issue. Although, it is for sure the latest update is causing the issue, it is still unclear to me what difference it makes between the two of my system. Both are Linux, the first one is Redhat, and the latter is Ubuntu.

#4

Thanks for the replies.

When running

cloudflared update

I get this back, note the missing version number.

2021-08-28T15:13:16Z INF cloudflared is up to date version=

Is there a better way to check the version? Where can I get a previous version of the binary to test?

Thanks again,

Chris

#5

That means it is up to date.
You can run cloudflared --version to get the version; it is also printed in the logs when you start to run the tunnel.

#6

Can you share the logs with the error please?

#7

Perfect, I will try an older version from GitHub and report back.

#8

I have just tried a version from 19 days ago https://github.com/cloudflare/cloudflared/releases/tag/2021.8.2

This seemed to work, but then it updated to the latest version. Is there a way to disable the auto update?

Thanks,

Chris

#9

Hello,

I have done some testing:

2021.8.6 - Broken and reports the above error.
2021.8.5 - Broken and reports the above error.
2021.8.3 - Working as expected

Happy to do some more testing if I can be helpful.

For those interested, I used the flag

--no-autoupdate

to disable the auto update.

Thanks,

Chris

#10

@nuno.diegues I had same errror on CentOS 7. I managed to fix the issue with a server restart though whether it was related to cloudflared updated version not sure?

Aug 28 11:31:54 hostname cloudflared[32580]: 2021-08-28T11:31:54Z INF Initial protocol http2
Aug 28 11:31:54 hostname cloudflared[32580]: 2021-08-28T11:31:54Z INF Starting metrics server on 127.0.0.1:5432/metrics
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR Error looking up Cloudflare edge IPs: the DNS query failed error="lookup _[]-origintunneld._tcp.argotunnel.com: no such host"
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR Please try the following things to diagnose this issue:
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR   1. ensure that argotunnel.com is returning "origintunneld" service records.
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR      Run your system's equivalent of: dig srv _origintunneld._tcp.argotunnel.com
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR   2. ensure that your DNS resolver is not returning compressed SRV records.
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR      See GitHub issue https://github.com/golang/go/issues/27546
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR      For example, you could use Cloudflare's 1.1.1.1 as your resolver:
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR      https://developers.cloudflare.com/1.1.1.1/setting-up-1.1.1.1/
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z INF Tunnel server stopped
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z ERR Initiating shutdown error="Could not lookup srv records on _[]-origintunneld._tcp.argotunnel.com: lookup _[]-origintunneld._tcp.argotunnel.com: no such host"
Aug 28 11:32:07 hostname systemd[1]: cloudflared.service: main process exited, code=exited, status=1/FAILURE
Aug 28 11:32:07 hostname cloudflared[32580]: 2021-08-28T11:32:07Z INF Metrics server stopped
Aug 28 11:32:07 hostname cloudflared[32580]: Could not lookup srv records on _[]-origintunneld._tcp.argotunnel.com: lookup _[]-origintunneld._tcp.argotunnel.com: no such host
Aug 28 11:32:07 hostname systemd[1]: Failed to start Argo Tunnel.
-- Subject: Unit cloudflared.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit cloudflared.service has failed.
-- 
-- The result is failed.
Aug 28 11:32:07 hostname systemd[1]: Unit cloudflared.service entered failed state.
Aug 28 11:32:07 hostname systemd[1]: cloudflared.service failed.

dig

dig srv _origintunneld._tcp.argotunnel.com

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.4 <<>> srv _origintunneld._tcp.argotunnel.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 425
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;_origintunneld._tcp.argotunnel.com. IN SRV

;; ANSWER SECTION:
_origintunneld._tcp.argotunnel.com. 300 IN SRV  1 1 7844 region1.argotunnel.com.
_origintunneld._tcp.argotunnel.com. 300 IN SRV  2 1 7844 region2.argotunnel.com.

;; Query time: 52 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Aug 28 11:39:57 UTC 2021
;; MSG SIZE  rcvd: 147

Looks like cloudflared was updated recently maybe related?

cloudflared version
cloudflared version 2021.8.6 (built 2021-08-27-2248 UTC)
#11

We’ll roll out a 2021.8.7 that should make this go away.

As to why it happens only some times (apparently rarely, and never happened in thousands of executions / runs on our end) is something that we’ll have to figure out.

2 Likes
#12

Here is a log if is simply run the quick tunnel.

I see others have also posted the full log, the error is the same as mine.

Thanks,
Subhash

cloudflared tunnell
2021-08-28T14:16:05Z INF Thank you for trying Cloudflare Tunnel. Doing so, without a Cloudflare account, is a quick way to experiment and try it out. However, be aware that these account-less Tunnels have no uptime guarantee. If you intend to use Tunnels in production you should use a pre-created named tunnel by following: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps
2021-08-28T14:16:05Z INF Requesting new quick Tunnel on trycloudflare.com...
2021-08-28T14:16:06Z INF +--------------------------------------------------------------------------------------------+
2021-08-28T14:16:06Z INF |  Your quick Tunnel has been created! Visit it at (it may take some time to be reachable):  |
2021-08-28T14:16:06Z INF |  https://recordings-brazilian-folding-uncertainty.trycloudflare.com                        |
2021-08-28T14:16:06Z INF +--------------------------------------------------------------------------------------------+
2021-08-28T14:16:06Z INF Cannot determine default configuration path. No file [config.yml config.yaml] in [~/.cloudflared ~/.cloudflare-warp ~/cloudflare-warp /etc/cloudflared /usr/local/etc/cloudflared]
2021-08-28T14:16:06Z INF Version 2021.8.6
2021-08-28T14:16:06Z INF GOOS: linux, GOVersion: devel +a84af465cb Mon Aug 9 10:31:00 2021 -0700, GoArch: amd64
2021-08-28T14:16:06Z INF cloudflared will not automatically update when run from the shell. To enable auto-updates, run cloudflared as a service: https://developers.cloudflare.com/argo-tunnel/reference/service/
2021-08-28T14:16:06Z INF Generated Connector ID: a77d730f-dc17-44b9-822f-8ac2fd20c03d
2021-08-28T14:16:06Z INF Initial protocol http2
2021-08-28T14:16:06Z INF Starting metrics server on 127.0.0.1:44903/metrics
2021-08-28T14:16:21Z ERR Error looking up Cloudflare edge IPs: the DNS query failed error="lookup _[]-origintunneld._tcp.argotunnel.com: no such host"
2021-08-28T14:16:21Z ERR Please try the following things to diagnose this issue:
2021-08-28T14:16:21Z ERR   1. ensure that argotunnel.com is returning "origintunneld" service records.
2021-08-28T14:16:21Z ERR      Run your system's equivalent of: dig srv _origintunneld._tcp.argotunnel.com
2021-08-28T14:16:21Z ERR   2. ensure that your DNS resolver is not returning compressed SRV records.
2021-08-28T14:16:21Z ERR      See GitHub issue https://github.com/golang/go/issues/27546
2021-08-28T14:16:21Z ERR      For example, you could use Cloudflare's 1.1.1.1 as your resolver:
2021-08-28T14:16:21Z ERR      https://developers.cloudflare.com/1.1.1.1/setting-up-1.1.1.1/
2021-08-28T14:16:21Z INF Tunnel server stopped
2021-08-28T14:16:21Z ERR Initiating shutdown error="Could not lookup srv records on _[]-origintunneld._tcp.argotunnel.com: lookup _[]-origintunneld._tcp.argotunnel.com: no such host"
2021-08-28T14:16:21Z INF Metrics server stopped
Could not lookup srv records on _[]-origintunneld._tcp.argotunnel.com: lookup _[]-origintunneld._tcp.argotunnel.com: no such host
#13

Just received ver 2021.8.7, and it fixes the issue.

Thanks

1 Like
#14

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.