Windows limitation on number of simultaneously opened sockets/connections per machine

Question

Let's say I have Windows 7 with one real network interface and few loopback interfaces. I have IOCP enabled server that accepts connections from clients. I'm trying to simulate as much as possible real client connections to the server.

My client code simply establishes X amount of socket connections (note that client binds to a given interface):

        const Int32 remotePort = 12345;
        const Int32 MaxSockets = 60000;

        Socket[] s = new Socket[MaxSockets];
        IPEndPoint bindEndpoint = new IPEndPoint(IPAddress.Parse(args[0]), 0);
        for (Int32 i = 0; i < MaxSockets; i++)
        {
            s[i] = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
            s[i].SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
            s[i].Bind(bindEndpoint);
            s[i].Connect(args[1], remotePort);

            IPEndPoint socketInfo = (IPEndPoint)s[i].LocalEndPoint;
            Console.WriteLine(String.Format("Connected socket {0} {1} : {2}", i, socketInfo.Address, socketInfo.Port));
        }

On a loopback interface I have several IPs that I use for binding. In addition, I also use real interface to bind on. I ran into a problem when amount of opened sockets is around 64K per machine:

Unhandled Exception: System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full

I've tried several helpless things like: - setting MaxUserPort to max value and some other recommended TCPIP settings in the registry. - trying to run two servers on different interfaces (real interfaces and loopback) and using several clients.

Is it a known limitation in Windows or its possible to overcome it somehow?

Thanks for the help!

Direct

Check this answer – Oliviaolivie

Thanks, I've read that thread but it does not really answer the question. I've tried what CharlesO did, but it didn't really work. – Direct

"note that client binds to a given interface". Why? – Impunity

In general because of TCP/IP limitation in number of ports per one IP address. So, I wanted, e.g. 64K open ports on interface1, and 64K open ports on interface2, and so on. But the total/overall amount of ports that I can bind on all interfaces in the system is still around 64K. E.g. here is the post: https://mcmap.net/q/1019450/-what-is-the-upper-limit-on-the-number-of-open-sockets-i-can-have-in-windows-server-2003 – Direct

Answer 1

I have found on some Microsoft page that:

... HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort registry subkey is defined as the maximum port up to which ports may be allocated for wildcard binds. The value of the MaxUserPort registry entry defines the dynamic port range...

So, if I force the endpoint to use a certain port, e.g.

IPEndPoint bindEndpoint = new IPEndPoint(IPAddress.Parse(args[0]), 54321);

Then I can open more than 64K simultaneous sockets in the system.

Direct

Answer 2

In your code example, you are calling Bind(bindEndpoint), but you do not show how bindEndpoint is defined. Check that :

• Your system actually has multiple IP addresses (loopback does not count)
• You are actually setting the IP Address of the endpoint to an IP address (not loopback)
• The binds are being spread across multiple IP addresses

The loopback address does not count because many systems treat it specially for routing and binding purposes. So binding to ports in loopback may be sucking up the ports across all addresses the same as if you were binding to INADDR_ANY (0.0.0.0).

Lorsung

I have placed the complete code now. As I explained, even if I bind on different interfaces (real and loopback, or only loopback): server1 and client1 on interface1, server2 and client2 on interface2, etc. still the maximum number of opened sockets per system is around 64K. – Direct

Answer 3

Both TCP and UDP use an unsigned 16-bit integer to designate port number. I don't imagine any implementation in any operating system is going to be able to open more than 65535 sockets per bound address at best. Additionally, I wouldn't be surprised if Windows doesn't implement fully isolated state tables for each adapter or each bound address but instead relies on a global state table. If that is the case, it would be a Windows network architecture limit instead of a soft, configurable limit.

Kenaz

Answer 4

I've developed a load testing tool.

Running on Windows 10/16G RAM, it could created 60,000 connections with server successfully.

But when try to create more connections, the tool will report "socket WinError 10055 No Buffer Space Available" soon.

Accord to this article, I think the limitation is the overall socket buffer size of whole OS, not the number of opened file.

Names