How many system resources will be held for keeping 1,000,000 websocket open? [closed]

Question

Websocket is good, but would it be able to handle 1,000,000 concurrent connections?

How many system resources will be needed for keeping 1,000,000 websocket open?

Thanks!

Grouch

Answer 1

Updated Answer

Short answer: yes, but it's expensive.

Long answer:

This question is not unique to WebSockets since WebSockets are fundamentally long-lived TCP sockets with a HTTP-like handshake and minimal framing for messages.

The real question is: could a single server handle 1,000,000 simultaneous socket connections and what server resources would this consume? The answer is complicated by several factors, but 1,000,000 simultaneous active socket connections is possible for a properly sized system (lots of CPU, RAM and fast networking) and with a tuned server system and optimized server software.

The number of connections is not the primary problem (that's mostly just a question of kernel tuning and enough memory), it is the processing and sending/receiving data to/from each of those connections. If the incoming connections are spread out over a long period, and they are mostly idle or infrequently sending small chunks of static data then you could probably get much higher than even 1,000,000 simultaneous connections. However, even under those conditions (slow connections that are mostly idle) you will still run into problems with networks, server systems and server libraries that aren't configured and designed to handle large numbers of connections.

See Alessandro Alinone's answer about approximate resource usage for 500,000 connections.

Here are some older but still applicable resources to read on how you would configure your server and write your server software to support large numbers of connections:

• What is the theoretical maximum number of open TCP connections that a modern Linux box can have
• http://www.kegel.com/c10k.html

Rabjohn

Apparantly 12 million socket connections is possible on a single JVM. See how they did it mrotaru.wordpress.com/2013/10/10/… – Ethbinium

@JacquesKoorts thx – Albumenize

I think the total count of websocket connections alone is not a problem and kernel can handle 10M+ just fine. The problem is buffering (e.g. if you need to push lots of data to many sockets and the client doesn't flush the socket you end up having lots of RAM reserved for outgoing TCP/IP buffers) and data per socket on the server. For example, if you run Node.js on the server, the total RAM per connection to hold any objects related to a single connection. In theory one could optimize that, too, but it would be hugely expensive because you would need similar code quality to Linux kernel. – Coulombe

Answer 2

On today's systems, handling 1 million concurrent TCP connections is not an issue.

I can affirm that based on our own tests (full disclosure: I am the CTO at Lightstreamer).

We had to demonstrate several times, to some of our customers, that 1 million connections can be reached on a single box (and not necessarily a super-monster machine). But let me recap the configuration where we tested 500K concurrent connections, as this is a much more recent test performed on Amazon EC2.

We installed Lightstreamer Server (which is a WebSocket server, among other things) on a m2.4xlarge instance. This means 8 cores and 68.4 GiB memory.

We launched 11 client machines to create 500,000 concurrent connections to the Lightstreamer Server. The test was configured so that the total outbound throughput from the server was 90,000 updates/s, resulting in peaks of 450 Mbit/s outbound bandwidth.

The server never used more than 13 GiB of RAM and the CPU was stable around 60%.

With at least 30 GiB RAM you can handle 1 million concurrent sockets. The CPU needed depends on the data throughput you need.

Agathaagathe

I'm assuming this was some flavor of linux. Could you please share additional info on how the kernel was tuned? max file descriptors/tcp window sizes etc? – Cataphyll

It was vanilla Amazon Linux. Max file descriptors were increased. TCP send buffer was reduced to 1600 bytes (done by default by Lightstreamer, though it can be manually tuned). MSS was default. – Agathaagathe

Does this soft fully free or needs some fee to use it? – Sardinian

@AvtandilKavrelishvili: There are both a free edition and a paid edition. – Agathaagathe

Would it be easier on the server if instead of websockets I used regular polling? – Mixup

Usually, polling in both its flavors (periodic polling and long polling) puts more load on the server than a persistent websocket connection. – Agathaagathe

Polling should be used if you don't need anything resembling real-time message passing. If one poll event per hour is enough for your use case, it will require less resources on the server than a websocket. – Coulombe