Question about listening and backlog for sockets

Question

I am writing an application in C# that needs to handle incoming connections and I've never done server side programming before. This leads me to these following questions:

• Pros and cons of high backlog / low backlog? Why shouldn't we set the backlog to a huge number?
• If I call Socket.Listen(10), after 10 Accept()s do I have to call Listen() again? Or do I have to call Listen() after every Accept()?
• If I set my backlog to 0 and hypothetically two people want to connect to my server at the same time, what would happen? (I am calling Socket.Select in a loop and checking readability of the listening socket, after I handle the first connection would the second connection be successful upon the next iteration if I called Listen() again?)

Thanks in advance.

Recognize

Answer 1

The listen backlog is, as Pieter said, a queue which is used by the operating system to store connections that have been accepted by the TCP stack but not, yet, by your program. Conceptually, when a client connects it's placed in this queue until your Accept() code removes it and hands it to your program.

As such, the listen backlog is a tuning parameter that can be used to help your server handle peaks in concurrent connection attempts. Note that this is concerned with peaks in concurrent connection attempts and in no way related to the maximum number of concurrent connections that your server can maintain. For example, if you have a server which receives 10 new connections per second then it's unlikely that tuning the listen backlog will have any affect even if these connections are long lived and your server is supporting 10,000 concurrent connections (assuming your server isn't maxing out the CPU serving the existing connections!). However, if a server occasionally experiences short periods when it is accepting 1000 new connections per second then you can probably prevent some connections from being rejected by tuning the listen backlog to provide a larger queue and therefore give your server more time to call Accept() for each connection.

As for pros and cons, well the pros are that you can handle peaks in concurrent connection attempts better and the corresponding con is that the operating system needs to allocate more space for the listen backlog queue because it is larger. So it's a performance vs resources trade off.

Personally I make the listen backlog something that can be externally tuned via a config file.

How and when you call listen and accept depends upon the style of sockets code that you're using. With synchronous code you'd call Listen() once with a value, say 10, for your listen backlog and then loop calling Accept(). The call to listen sets up the end point that your clients can connect to and conceptually creates the listen backlog queue of the size specified. Calling Accept() removes a pending connection from the listen backlog queue, sets up a socket for application use and passes it to your code as a newly established connection. If the time taken by your code to call Accept(), handle the new connection, and loop round to call Accept() again is longer than the gap between concurrent connection attempts then you'll start to accumulate entries in the listen backlog queue.

With asynchronous sockets it can be a little different, if you're using async accepts you will listen once, as before and then post several (again configurable) async accepts. As each one of these completes you handle the new connection and post a new async accept. In this way you have a listen backlog queue and a pending accept 'queue' and so you can accept connections faster (what's more the async accepts are handled on thread pool threads so you don't have a single tight accept loop). This is, usually, more scalable and gives you two points to tune to handle more concurrent connection attempts.

Dunseath

hi, im trying to write tests... and the code is not... making it clear what is the problem... im looping over 10 000 client request... each connection make its own tcp connection. I have set the backlog to 20 000. the listener is accepting connection asynchronous and putting them into their own thread. I have told each accepted connection on the listener to do 5 secs of work. almost instantly I am getting refused by the client... I KNOW THIS IS NOT REAL WORLD TESTS... i'm just trying to understand all the limitations... could you assist in helping me understand.. – Fanfare

1) You'd be better off writing a question rather than adding a vague comment to an existing answer. 2) Scaling to 10,000 using a "thread per connection" model is not ideal. 3) this kind of test could run into various system resource limits depending on how it's structured (TIME_WAIT) being one of them. 4) A backlog of 20,000 just says "allow 20,000 connections attempts to be pending" it doesn't affect how many active connections are possible, 20,000 is a very big number for this and may be bigger than the OS allows... 5) There are lots MORE things you could be doing wrong, I'd need to see code – Dunseath

@LenHolgate Are data requests made by client using the send/write calls , also queued up in the listen backlog queue? While calling the send/write from client code we give the server's listening socket as the input to these calls. – Rubella

No. It's just for new connection attempts. – Dunseath

Answer 2

What the backlog does is provide a queue with clients that are trying to connect to the server, but which you haven't processed yet.

This concerns the time between when the client actually connects to the server and the time you Accept or EndAccept the client.

If accepting a client takes a long time, it is possible that the backlog becomes full and new client connections will be rejected until you had time to process clients from the queue.

Concerning your questions:

1. I don't have information on that. If the default number does not pose any problems (no rejected client connections) leave it at its default. If you see many errors when new clients want to connect, increase the number. However, this will probably be because you take too much time accepting a new client. You should solve that issue before increasing the backlog;

2. No, this is handled by the system. The normal mechanism of accepting clients takes care of this;

3. See my earlier explanation.

Discourse

Excuse me, does it mean the connections queued in backlog queue have no relation with the max connections a server can take? For example, a server with backlog 10 can still take 100 or even 1000 clients. Thanks. – Gros

Answer 3

Try this program and you will see what is backlog good for.

using System;
using System.Net;
using System.Net.Sockets;

/*
   This program creates TCP server socket. Then a large number of clients tries to connect it.
   Server counts connected clients. The number of successfully connected clients depends on the BACKLOG_SIZE parameter.
 */


namespace BacklogTest
{
    class Program
    {
        private const int BACKLOG_SIZE = 0; //<<< Change this to 10, 20 ... 100 and see what happens!!!!
        private const int PORT = 12345;
        private const int maxClients = 100;

        private static Socket serverSocket;
        private static int clientCounter = 0;

        private static void AcceptCallback(IAsyncResult ar)
        {
            // Get the socket that handles the client request
            Socket listener = (Socket) ar.AsyncState;
            listener.EndAccept(ar);
            ++clientCounter;
            Console.WriteLine("Connected clients count: " + clientCounter.ToString() + " of " + maxClients.ToString());

            // do other some work
            for (int i = 0; i < 100000; ++i)
            {
            }

            listener.BeginAccept(AcceptCallback, listener);
        }

        private static void StartServer()
        {
            // Establish the locel endpoint for the socket
            IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Any, PORT);

            // Create a TCP/IP socket
            serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

            // Bind the socket to the local endpoint and listen
            serverSocket.Bind(localEndPoint);
            serverSocket.Listen(BACKLOG_SIZE);
            serverSocket.BeginAccept(AcceptCallback, serverSocket);
        }

        static void Main(string[] args)
        {
            StartServer();

            // Clients connect to the server.
            for (int i = 0; i < 100; ++i)
            {
                IPAddress ipAddress = IPAddress.Parse("127.0.0.1");
                IPEndPoint remoteEP = new IPEndPoint(ipAddress, PORT);

                // Create a TCP/IP socket and connect to the server
                Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                client.BeginConnect(remoteEP, null, null);
            }

            Console.ReadKey();
        }
    }
}

Spermophile