ERR Serve tunnel error error="failed to accept QUIC stream

Since upgrading to Cloudflared version 2022.5.0, I’m getting intermittent failures on my private routing tunnel. Below is the info from my log. Nothing has changed in my environment except the upgrade to the Cloudflared.

Is this the same issue documented here?: Failed to serve quic connection · Issue #510 · cloudflare/cloudflared · GitHub
If so, what is the status on getting the issue resolved?

May 08 07:18:51 cloudflared[965772]: 2022-05-08T07:18:51Z ERR Serve tunnel error error=“failed to accept QUIC stream: timeout: no recent network activity” connIndex=3
May 08 07:18:51 cloudflared[965772]: 2022-05-08T07:18:51Z INF Retrying connection in up to 1s seconds connIndex=3
May 08 07:18:51 cloudflared[965772]: 2022-05-08T07:18:51Z INF Unregistered tunnel connection connIndex=2
May 08 07:18:51 cloudflared[965772]: 2022-05-08T07:18:51Z ERR Failed to serve quic connection error=“failed to accept QUIC stream: timeout: no recent network activity” co>
May 08 07:18:51 cloudflared[965772]: 2022-05-08T07:18:51Z ERR Serve tunnel error error=“failed to accept QUIC stream: timeout: no recent network activity” connIndex=2
May 08 07:18:51 cloudflared[965772]: 2022-05-08T07:18:51Z INF Retrying connection in up to 1s seconds connIndex=2
May 08 07:18:52 cloudflared[965772]: 2022-05-08T07:18:52Z INF Connection bf15c495-55f6-4337-abce-fccec08f4bd6 registered connIndex=3 location=IAD
May 08 07:18:52 cloudflared[965772]: 2022-05-08T07:18:52Z INF Connection 4cc856bb-0735-4625-a0ff-394f0ee2ad31 registered connIndex=0 location=PDX
May 08 07:18:52 cloudflared[965772]: 2022-05-08T07:18:52Z INF Connection ce47c560-027a-43b9-a9d3-7d97b3607ee1 registered connIndex=1 location=IAD
May 08 07:18:53 cloudflared[965772]: 2022-05-08T07:18:53Z INF Connection f52e13d1-1841-4fe6-8c2e-9a9d9965effa registered connIndex=2 location=PDX

@ericj1 can you elaborate on how you are perceiving the “intermittent failures”?

What’s your Tunnel ID? (that’s not a secret, can be shared with no problems)

1 Like

I am too having this exact same issue from last few days. Tunnels suddenly stops working with below errors,

May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF Starting tunnel tunnelID=d63da755-a723-4396-836f-948e2c6480c1
May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF Version 2022.5.0
May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF GOOS: linux, GOVersion: go1.17.5, GoArch: amd64
May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF Settings: map[config:/etc/cloudflared/config.yml cred-file:/home/santosh/.cloudflared/d63da75>
May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF Generated Connector ID: ca834f77-bddc-4fb6-b0e4-cc34da9b5916
May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF Initial protocol quic
May 10 21:35:27 Ubuntu cloudflared[56025]: 2022-05-10T16:05:27Z INF Starting metrics server on 127.0.0.1:42935/metrics
May 10 21:35:28 Ubuntu cloudflared[56025]: 2022/05/10 21:35:28 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB)>
May 10 21:35:29 Ubuntu cloudflared[56025]: 2022-05-10T16:05:29Z INF Connection 8c84d983-f0f1-4ea6-98cf-<> registered connIndex=0 location=DEL
May 10 21:35:29 Ubuntu systemd[1]: Started cloudflared.
May 10 21:35:29 Ubuntu cloudflared[56025]: 2022-05-10T16:05:29Z INF Connection c357733b-f545-4157-bf41-026fe1bfb7e6 registered connIndex=1 location=BOM
May 10 21:35:30 Ubuntu cloudflared[56025]: 2022-05-10T16:05:30Z INF Connection da119a52-8eab-455a<> registered connIndex=2 location=DEL
May 10 21:35:31 Ubuntu cloudflared[56025]: 2022-05-10T16:05:31Z INF Connection 797710a2-f502<> registered connIndex=3 location=BOM
May 11 01:46:15 Ubuntu cloudflared[56025]: 2022-05-10T20:16:15Z INF Unregistered tunnel connection connIndex=2
May 11 01:46:15 Ubuntu cloudflared[56025]: 2022-05-10T20:16:15Z WRN Failed to serve quic connection error="failed to accept QUIC stream: timeout: no recent netwo>
May 11 01:46:15 Ubuntu cloudflared[56025]: 2022-05-10T20:16:15Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" >
May 11 01:46:15 Ubuntu cloudflared[56025]: 2022-05-10T20:16:15Z INF Retrying connection in up to 1s seconds connIndex=2
May 11 01:46:16 Ubuntu cloudflared[56025]: 2022-05-10T20:16:16Z WRN If this log occurs persistently, and cloudflared is unable to connect to Cloudflare Network w>
May 11 01:46:16 Ubuntu cloudflared[56025]: If you are using private routing to this Tunnel, then UDP (and Private DNS Resolution) will not workunless your cloudf>
May 11 01:46:16 Ubuntu cloudflared[56025]: 2022-05-10T20:16:16Z INF Switching to fallback protocol http2 connIndex=2
May 11 01:46:16 Ubuntu cloudflared[56025]: 2022-05-10T20:16:16Z INF Connection b31fda6c-65af-4573<> registered connIndex=2 location=DEL
May 11 04:23:48 Ubuntu cloudflared[56025]: 2022-05-10T22:53:48Z INF Unregistered tunnel connection connIndex=0
May 11 04:23:48 Ubuntu cloudflared[56025]: 2022-05-10T22:53:48Z WRN Failed to serve quic connection error="failed to accept QUIC stream: timeout: no recent netwo>
May 11 04:23:48 Ubuntu cloudflared[56025]: 2022-05-10T22:53:48Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" >
May 11 04:23:48 Ubuntu cloudflared[56025]: 2022-05-10T22:53:48Z INF Retrying connection in up to 1s seconds connIndex=0
May 11 04:23:50 Ubuntu cloudflared[56025]: 2022-05-10T22:53:50Z WRN If this log occurs persistently, and cloudflared is unable to connect to Cloudflare Network w>
May 11 04:23:50 Ubuntu cloudflared[56025]: If you are using private routing to this Tunnel, then UDP (and Private DNS Resolution) will not workunless your cloudf>
May 11 04:23:50 Ubuntu cloudflared[56025]: 2022-05-10T22:53:50Z INF Switching to fallback protocol http2 connIndex=0

If I restart the cloudlflared service systemctl restart cloudflared then it will work for few minutes and then again will stop working.

I narrowed down the issue to our Veeam backup process and my Linux VM. Veeam Backup & Replication works in conjunction with VMware to create and delete a snapshot of a VM during the backup process. When the snapshot is deleted, disk consolidation occurs. The disk consolidation was causing the tunnel to disconnect, even though the cloudflared tunnel was still up. Would be great if the tunnel would automatically recover, but I was able to create a workaround with a crontab job to restart the cloudflared tunnel service everyday at a set time, after the backup of the effected VM has completed.