How many users Nodejs & Socket.io can support?

P

3

22

I got a website that implements the broadcast.emit. that sends message to all users of the site. It is just the simplest implementation of socket.io but I currently having a problem. When I hit about 100 concurrent users. The nodejs server starts to lag until it hangs that I can no longer access my whole website. When we checked the server. The nodejs is taking 100% of the CPU. Is it normal?

I was wondering how many users can socket.io supports? And when this thing happen is there a way to restart the nodejs server programatically?

Prather answered 15/4, 2012 at 11:53 Comment(1)

Can you give us some info on the server specs? – Quitclaim 14/9, 2012 at 20:41

A

17

At least 250k concurrent connections (and for most use cases bottleneck is memory)

Amundson answered 15/4, 2012 at 15:12 Comment(13)

Thanks Andrey. Actually no issues in the server memory. It uses less than 5% of server memory. Also i forgot to mention that the nodejs is in the same box as my web & db server – Prather 15/4, 2012 at 16:32

250k referenced above is a HelloWorld type of benchmark without socket.io. In reality c10k is close to real (depending on your CPU and application complexity). For example, for a card game I'm running, I hit 100% CPU usage at about c3k. – Character 12/9, 2012 at 12:24

1m concurrent connections - blog.caustik.com/2012/08/19/node-js-w1m-concurrent-connections – Amundson 13/9, 2012 at 0:20

@MilanBabuškov - you problem is probably has complexity of n^2 if all users send messages to all users. It's more about number of messages per second than number of concurrent TCP connections – Amundson 13/9, 2012 at 0:22

No, each game is played by 4 players. So each message is sent to only 4 users. – Character 13/9, 2012 at 5:43

"1m concurrent connections" is also without socket.io. Node.js scales great, but I'm not so sure about socket.io. – Character 13/9, 2012 at 5:44

Guys just to update. We move to Amazon EC2 instances and now can handle our current users. Our current max concurrent users is 500 so not that heavy yet. – Prather 24/10, 2012 at 4:36

The benchmark is wrong. A single instance of Socket.io can only get to around 10K concurrent users (each sending a JSON object every 6 seconds)... To go beyond that you need to have multiple socket.io instances with HAProxy or nginx in front. – Unreeve 10/7, 2014 at 9:59

@Unreeve what if they send request every 60 seconds (or idle)? – Amundson 10/7, 2014 at 11:30

@AndreySidorov I'm not sure - Maybe if idle; I will have to test. Memory is definitely not the bottleneck though - It's CPU by far. There is a significant overhead cost for having many connections open. I did some tests with engine.io and the CPU cost of having 100 connections each sending 1000 messages per second is much less demanding (around a quarter to 1/6th of CPU usage) than having 10000 connections each sending 10 messages per second (even though the throughput is the same). This may be due to the OS having to do more context switching when you have many sockets sharing CPU? – Unreeve 11/7, 2014 at 9:20

Interesting comparison. 100 x 1000 (or 10 x 10000) is quite big number by itself ( can you handle 100k http rps?), but ratio of those two is something worth investigating. There should be no context switching on os level - it's still one sequential stream of bytes coming to network interface, you can dispatch them in one thread of execution – Amundson 11/7, 2014 at 13:56

@AndreySidorov I wonder what's causing this effect then. Definitely worth investigating. – Unreeve 12/7, 2014 at 10:40

@AndreySidorov i am new to socket.io , i have a doubt what you are saying is number of connection determine by the hardware( CUP and RAM ) ? – Seventeenth 4/7, 2021 at 7:15

O

20

I have a multiplayer card game. Socket.io maxes out my CPU at around 3000 concurrent users. This is on Intel i7 CPU. Because of this I have to run multiple node/socket.io processes to handle the load.

For 100 concurrent connections you should be fine. Perhaps you are using some VPS and CPU is shared with all the other VMs? Do you run a dedicated server?

Also, check your code. You might be doing synchronously some stuff that should go async.

Oddity answered 12/9, 2012 at 12:26 Comment(4)

If you were to use the cluster module and used more cores, would that number increase? Or are you already using the cluster module? – Gaynor 31/1, 2013 at 7:5

Currently, I run 7 node processes using HAProxy as frontend. There are around 9000 concurrent users (~1300 per node) and everything works without problems. All on a single server. – Character 31/1, 2013 at 11:39

@MilanBabuškov Thanks for the real life examples, just curious since you have 7 node processes running, how are you handling the in memory database? Pub/Sub w/ redis? I'm about to launch a action RPG with websockets and was thinking if I should re-write everything to support redis or I should be fine with 1 server. Also, how many messages are being sent per minute per each user? – Lag 20/9, 2015 at 6:36

I'm sharding the data and storing all the info in plain JavaScript objects. I use HAProxy to ensure that the same player always connects to the same server. Note that it's a card game (turn based), so each player sends about 20-30 messages per minute. – Character 18/11, 2015 at 15:15