A significant percentage of my incoming requests will fail with “Canceled by remote with error code 0”. After several hours, my tunnel will restart with “Initiating graceful shutdown due to signal terminated”
After restarting, these errors will be completely gone for several hours. The errors will gradually increase in frequency until my tunnel once again restarts with “Initiating graceful shutdown due to signal terminated”
I’m running the latest docker image (published 5 days ago).
Short excerpt of my logs below.
2023-06-24T22:30:20Z ERR Request failed error="stream 132065 canceled by remote with error code 0" connIndex=3 dest=https://redacted.com event=0 ip=xxx.xxx.xxx.xxx type=http
2023-06-24T22:31:05Z ERR error="stream 132081 canceled by remote with error code 0" cfRay=7dc868cefde989b0-SIN event=1 originService=http://nginx
2023-06-24T22:31:05Z ERR Request failed error="stream 132081 canceled by remote with error code 0" connIndex=3 dest=https://redacted.com event=0 ip=xxx.xxx.xxx.xxx type=http
2023-06-24T22:31:08Z ERR error="stream 132093 canceled by remote with error code 0" cfRay=7dc868dd0d523806-IAD event=1 originService=http://nginx
2023-06-24T22:31:08Z ERR Request failed error="stream 132093 canceled by remote with error code 0" connIndex=3 dest=https://redacted.com event=0 ip=xxx.xxx.xxx.xxx type=http
2023-06-24T22:32:54Z INF Initiating graceful shutdown due to signal terminated ...
2023-06-24T22:32:55Z INF Unregistered tunnel connection connIndex=2 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z ERR Failed to serve quic connection error="context canceled" connIndex=2 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z INF Unregistered tunnel connection connIndex=0 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z ERR Failed to serve quic connection error="context canceled" connIndex=0 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z INF Unregistered tunnel connection connIndex=1 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z ERR Failed to serve quic connection error="context canceled" connIndex=1 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:32:55Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=xxx.xxx.xxx.xxx
2023-06-24T22:33:06Z INF Starting tunnel tunnelID=2f6e1df0-9b2e-4eda-b5ce-e44a9c55eb95
2023-06-24T22:33:06Z INF Version 2023.6.1
2023-06-24T22:33:06Z INF GOOS: linux, GOVersion: go1.19.10, GoArch: amd64
2023-06-24T22:33:06Z INF Settings: map[no-autoupdate:true token:*****]
2023-06-24T22:33:06Z INF Generated Connector ID: 3f68748d-dc6d-420b-8c17-143063c39561
2023-06-24T22:33:06Z INF Initial protocol quic
2023-06-24T22:33:06Z INF ICMP proxy will use 172.18.0.6 as source for IPv4
2023-06-24T22:33:06Z INF ICMP proxy will use :: as source for IPv6
2023-06-24T22:33:06Z INF Starting metrics server on 127.0.0.1:41563/metrics
2023/06/24 22:33:06 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size for details.