Increasing periods of Tunnel Brown-out with 502 Bad gateway errors

What is the name of the domain?

sglang.model.serve.domain.com

What is the error number?

502

What is the error message?

Bad Gateway error on the client API side, or Connection closed by UNKNOWN port 65535 on Client SSH side

What is the issue you’re encountering

May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z WRN Failed to serve tunnel connection error="failed to accept QUIC stream: timeout: no recent network activity" con> May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=0 event=> May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=198.41.192.37 May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z WRN Connection terminated error="timeout: no recent network activity" connIndex=2 May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z WRN Failed to serve tunnel connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200> May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.193 May 21 16:18:22 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:22Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.193 May 21 16:18:23 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:23Z ERR Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 May 21 16:18:23 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:23Z ERR Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=0 May 21 16:18:24 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:18:24Z ERR Connection terminated error="timeout: no recent network activity" connIndex=1 May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Using [CurveID(4588) CurveID(25497) CurveP256] as curve preferences connIndex=1 event=0 ip=198.41.200.53 May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Using [CurveID(4588) CurveID(25497) CurveP256] as curve preferences connIndex=3 event=0 ip=198.41.200.33 May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Using [CurveID(4588) CurveID(25497) CurveP256] as curve preferences connIndex=2 event=0 ip=198.41.192.67 May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Using [CurveID(4588) CurveID(25497) CurveP256] as curve preferences connIndex=0 event=0 ip=198.41.192.37 May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Registered tunnel connection connIndex=2 connection=7648d65f-f59a-48f6-aea6-c2919fd8cb15 event=0 ip=198.41.192.> May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Registered tunnel connection connIndex=1 connection=a479714f-c9da-4b3c-b1cc-82ec705a99dc event=0 ip=198.41.200.> May 21 16:22:55 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:55Z INF Registered tunnel connection connIndex=0 connection=eb3794d3-5dd9-49e4-b7f5-655af9df3564 event=0 ip=198.41.192.> May 21 16:22:56 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:22:56Z INF Registered tunnel connection connIndex=3 connection=30711897-9a05-47a0-9454-5207ed15f750 event=0 ip=198.41.200.> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z WRN Failed to serve tunnel connection error="failed to accept QUIC stream: timeout: no recent network activity" con> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=2 event=> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.192.67 May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z ERR error="Incoming request ended abruptly: context canceled" connIndex=2 event=1 ingressRule=1 originService=http> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=2 dest=https://model-serve-s> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z ERR error="Incoming request ended abruptly: context canceled" connIndex=3 event=1 ingressRule=1 originService=http> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=3 dest=https://model-serve-s> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z WRN Failed to serve tunnel connection error="failed to accept QUIC stream: timeout: no recent network activity" con> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 event=> May 21 16:25:13 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:13Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.33 May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z WRN Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=2 May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z ERR error="Incoming request ended abruptly: context canceled" connIndex=0 event=1 ingressRule=1 originService=http> May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=0 dest=https://model-serve-s> May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z WRN Failed to serve tunnel connection error="failed to accept QUIC stream: timeout: no recent network activity" con> May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=0 event=> May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=198.41.192.37 May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z ERR error="Incoming request ended abruptly: context canceled" connIndex=0 event=1 ingressRule=1 originService=http> May 21 16:25:14 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:14Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=0 dest=https://model-serve-s> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR error="Incoming request ended abruptly: context canceled" connIndex=1 event=1 ingressRule=1 originService=http> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=1 dest=https://model-serve-s> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR error="Incoming request ended abruptly: context canceled" connIndex=1 event=1 ingressRule=1 originService=http> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR Failed to serve tunnel connection error="failed to accept QUIC stream: timeout: no recent network activity" con> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=1 dest=https://model-serve-s> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=1 event=> May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.53 May 21 16:25:15 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:15Z ERR Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 May 21 16:25:16 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:16Z ERR Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=0 May 21 16:25:16 guardis-ai-MS-7D53 cloudflared[2270]: 2025-05-21T21:25:16Z ERR Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=1 M

What steps have you taken to resolve the issue?

Reinstalled Cloudflared, reinstalled OS, monitored resource usage ( > 50% of CPU, GPU, RAM is always available).

What are the steps to reproduce the issue?

cloudflared version 2025.2.1 (built 2025-02-27-1123 UTC)

Hi there,

Is this only happening on the same machine/connection?
Able to reproducce on similar machine + different connection or different machine + different connection?

Thank you.

I’ve noticed the
Connection closed by UNKNOWN port 65535 on other machines with different workloads.

This machine specifically has two connections (SSH and HTTP tunnel) and both get impacted with this issue simultaneously. I will note that during these Cloudflared brown-outs, the machine still has access to internet, and HTTP and SSH services work through the regular ports both with @localhost and remotely (not using tunnels).

All of these machines use Linux Debian based. The archictectures are ARM and x86, but the x86 machine seems to be the most impacted.

These brown-out periods can last upwards of 10 minutes

This is not resolved. Just posting so that the issue remains open. Assistance and more comments on how to debug and potential causes is welcomed.

Thank you

@oshariff

do you have any suggestions? Either to resolve or to help debug the issue?

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.