Websockets disconnected in AWS Tokyo

I have noticed that websocket clients accessing Cloudflare from AWS Tokyo get cut off after about 5 minutes. This happens for multiple domains hosted by Cloudflare (api.huobi.com, api.bitfinex.com). This does not happen to identical websocket clients that access Cloudflare from other regions (e.g. Ireland, N. Virginia).

Logs from tcpdump show that Cloudflare sends a TCP reset after 1-5 minutes, despite both client and server being in sync on packets sent in each direction. Application heartbeats are also sent in both directions.

There is something special about maintaining a session with Cloudflare in Tokyo (possible in all of Asia). Is there anything I can do (e.g. headers sent) to prevent Cloudflare from resetting my client sessions?

Would like to confirm: this is a websocket, not a long-standing http connection/http stream?

Irrelevant, I doubt you misunderstood WebSockets for a long-standing HTTP connection.

Not to put CF down, but see the following HN comment:

If you’re intending to use CF websockets, be prepared for random (and potentially massive) connection drops, and be sure you’re architected to handle these disconnects gracefully. Cloudflare rolling restarts have caused hundreds of thousands of websocket connections to be disconnected in a matter of minutes for us.

Also see the comment by pencoyd there. You should make sure both your server and clients handle the 1001 status code and reconnect when they receive that…

Yeah, this is a websocket, and not a long-standing HTTP connection. Although, it is possible that the symptom also appears for vanilla HTTP. I just haven’t tried.

I understand that maintaining a long-standing websocket session on the Internet is not supposed to be super reliable, but these are failing an awful lot. Also, they fail way more frequently from Tokyo than from Europe or North America. I think something is broken with the CF websocket proxies in Tokyo. I’d just like to know if there is anything I can do to keep sessions alive for longer.

I have this exact same issue in AWS Tokyo connecting to an API via Cloudflare. I have captured packets and despite the websocket connection being in a healthy state with plenty of data flowing, after a hard to predict amount of time (1-45 minutes) I receive a TCP RST.

Is there anything that can be done to get a more stable websocket?

This topic was automatically closed after 30 days. New replies are no longer allowed.