Socket.io randomly disconnecting

I own a semi-popular typing game that gets a quite a bit of traffic, especially since it’s Multiplayer.

We almost exclusively rely on Socket-io, and since joining Cloudflare we have had several problems for at least 5-10% of our users.

Stack:
Nginx, Express, Socket-io

Problem:
When playing in a match specifically (since that’s the easiest to note) the user would randomly get disconnected and the match would not finish because of course it was disconnected. While we could introduce a build-in reconnection logic, it would still revolve around creating several edge cases that would help only short term.

When a player disconnects, Socket-io registers it as a “transport close” and even Socket.io’s logging doesn’t specify any errors. However most of the time it will just reconnect instantly and act as if nothing went wrong.

Testing:
Over the past bit, we’ve tested created different servers and hosting the same game and trying different configuration and this is what we’ve pretty much discovered.

All of these tests were conducted doing 20 matches on each.

  • Socket-io with SSL and Nginx - working
  • Socket-io with Nginx - working
  • Socket-io with Cloudflare, Nginx and SSL- doesn’t work
  • Socket-io with Cloudflare and Nginx - doesn’t work

Now over the periods of time we’ve tried ignoring this issue, but of course with a growing player base the problem has arose several times over the course of just a couple months so we are trying to find an actual solution.

The reason why we do not open the Socket-io portion outside of Cloudflare is due to DDoS. It’s a common problem in every gaming community so it’s very hard to allow that exposure to our servers at the moment.

We’ve also tried modifying the pings, making sure it isn’t a problem with Cloudflare automatically disconnecting after X amount of time. We’ve also reflected this change in Nginx to also make sure. We have also used trust proxy in Express, we’ve pretty much tried everything at this point.

Same issue here

Do you have any status codes or messages that you can send here?

are you using free plan? or paid plan?

As mentioned above, all socket.io debug states is a “transport close”. Socket.io doesn’t use status codes since it’s just a constant stream (at least from the WS aspect of it, which we primarily use).

Using Free plan, but we have tried the $20 paid plan to same result

Are you aware of the websocket limits of CF?

In particular:

Technical note
When Cloudflare releases new code to its global network, we may restart servers, which terminates WebSockets connections.

Websockets close with error 1006 when this occurs.

1 Like

So what is the actual solution around this? And why is this only for 5-10% of people on average where they can replicate this problem consistently compared to others.

Bump.

The solution is just to make sure you have reconnect logic. Users being disconnected due to Cloudflare or general network problems isn’t uncommon.

I’m not sure there’s really better advice than just make sure you have reconnect logic.

1 Like

We could implement reconnect logic, but by the time it reconnects it’s hard to carry a state in a game that’s dependent on when a player finishes first and when a player doesn’t finish first (think of it as a racing game essentially).

This is a fault on Cloudflare and want to know why this occurs and if there’s a specific configuration that prevents Cloudflare from causing these issues.

We already listed why it occurs. When CF pushes new code, it closes existing WebSockets. Short of maybe an enterprise contract where you get a dedicated 1U at DCs (which would be unlikely and expensive), no CF configuration disables this. Even Discord, which runs its chat gateway on WebSockets, doesn’t get to change this functionality and they’re certainly one of CF’s top 500 enterprise customers.

If specific players can re-create a disconnection bug constantly every time where others cannot re-create it. Then this is not why it is closing.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.